ESP32-CAM | ESP32 Camera Module with Face Recognition

ES32-CAM-esp-32-camera-module.jpg

A step by step guide to setting up the Ai-Thinker ESP32-CAM with Espressif’s ESP32 development environment and the ESP-WHO libraries for face detection and recognition.

With this module you get face detection for ~ $7!  They have them on AliExpress here.

The AI-Thinker ESP32-CAM module features an ESP32-S chip, an OV2640 camera and a microSD card slot. The software needs to be compiled and uploaded using the Espressif development environment.

Follow the steps below to download and install everything you need to develop on the ESP32 including packages for the ESP-IDF and the ESP32 toolchain. If you prefer to use the Arduino IDE (it’s a lot easier to set up the web server demo) there’s another tutorial here: ESP32-CAM using Arduino IDE

ESP32 Development Environment

Download the 32bit version (-i686-) of MYSYS from here: http://www.msys2.org/
Install following the steps on the page above but uncheck the “Run MSYS2 32-bit now” checkbox at the end.
Navigate to C:\msys32\ and double-click mingw32.exe to run the application.
Copy the contents of the file here https://raw.githubusercontent.com/espressif/esp-idf/master/tools/windows/windows_install_prerequisites.sh to a new file in C:\msys32\
Copy or paste the command below and press enter:

C:/msys32/windows_install_prerequisites.sh

You are likely to see an error message similar to this:

***fatal error - cygheap base mismatch detected

Close the terminal window and again navigate to C:\msys32\ and double-click mingw32.exe to re-open the terminal.
Press the up arrow to see the C:/msys32/windows_install_prerequisites.sh command and press enter.

When the script has completed, type or paste the following commands one at a time.

cd ~
mkdir esp
cd ~/esp
git clone --recursive https://github.com/espressif/esp-idf.git

NOTE: If you run into an error: cannot fork() type or paste the command below:

c:/msys32/autorebase.bat

and run the git clone command again.

If you look in c:\msys32\home\**yourusername**\esp\ you will see all the APIs and libraries for the ESP32 have been installed.

A user profile script needs to be added so projects will build without the path to these files having to be entered every time.

Find the path to the ESP-IDF. Following this tutorial it will be C:\msys32\home\**yourusername**\esp\esp-idf\
Create a new file named export_idf_path.sh in C:/msys32/etc/profile.d/
Open it in Notepad and add the following line to the new export_idf_path.sh file replacing the IDF_PATH with your own. Check you are using forward slashes.

export IDF_PATH="C:/msys32/home/**yourusername**/esp/esp-idf"

Save and close this new file.

Close the MSYS2 terminal window and then double click mingw32.exe to  open it again. Check if IDF_PATH is set by typing:

printenv IDF_PATH

To complete set up, paste the following commands in the MSYS2 terminal window

cd ~
C:/msys32/mingw32/bin/python.exe -m pip install --user -r C:/msys32/home/**yourusername**/esp/esp-idf/requirements.txt

The ESP32 development environment is now set up.

ESP-WHO Face Detection and Recognition Libraries

To set up the face detection and recognition libraries and demos, type or paste the following commands in the MSYS2 terminal window

cd ~
git clone --recursive https://github.com/espressif/esp-who.git

If you look in c:\msys32\home\**yourusername**\esp-who\ you will see the ESP-WHO libraries and examples have been installed.

Connections

The ESP32-CAM doesn’t come with a USB connector so you need either a CP2102 or a FT232RL USB to TTL Serial Converter to connect it to your PC. When using WiFi the ESP32 can use more current than is supplied via USB through these devices resulting in the module crashing and rebooting. I found the top device worked fine (I bought it here) but the FT232RL wouldn’t work without a separate supply to the 5V pin. If you use a FT232 with 5v and 3.3v, set the voltage jumper to 3.3v so the serial connections are at 3.3v. You might be able to use the 5V from the side of the board to power the 5v pin on the camera.

USB to ESP32-CAM Wiring Diagram

 

Wire the module according to the diagram above and plug into your PC. The green connection should be connected when flashing the module and disconnected when running.

Open Device Manager and make a note of the port number of the serial convertor:

COM Port

Camera Web Server Example

In the MSYS2 terminal window type or paste the following:

cd ~/esp-who/examples/single_chip/camera_web_server/
make menuconfig

In the window that opens you need to change the following settings

Serial flasher config —> Default serial port (Change to the port number shown in Device Manager – ie COM3)
Camera Web Server —> WiFi Settings —> (Add your WiFi SSID and Password)
Camera Web Server —> Camera Pins —> Select Camera Pinout —> (Select ESP32-CAM by AI-Thinker)
Component config —> ESP32-specific —> SPI Ram config —> Type of SPI RAM chip (Select Auto-detect)

Save the configuration.

In the terminal window, type:

make flash

If you get an error that the device fails to connect. Check that the IO0 pin (green in the diagram) is connected to GRD and press the reset button under the module and try again.

When the device has completed flashing. Unplug IO0 from GND and type:

make monitor

Press the reset button on the ESP32-CAM and watch the start up sequence in the terminal.

Look for the IP address the ESP32 has been given on your network camera wifi where it outputs got ip:xxx.xxx.xxx.xxx

Connected to WiFi

Type that IP address into your browser. You should be able to see a GUI on the left where you can control elements of the camera, set face detection and face recognition. Click Get Still to take a photo. Click Start Stream to see a video stream from the camera.

ESP-WHO GUI

It’s worth checking the jumper next to the IPEX connector on the bottom of the board to make sure it’s set for the internal aerial otherwise you may see low frame-rates and poor reception.

Resources

Here’s a list of resources that I used to get this working

https://github.com/espressif/esp-who – Homepage for the face recognition library
https://docs.espressif.com/projects/esp-idf/en/latest/get-started/windows-setup-scratch.html – Setting up the ESP Dev Environment from scratch. I found the prebuilt version had problems
https://github.com/VirgiliaBeatrice/esp32-devenv-vscode/blob/master/tutorial.md – Alternative setup method with further info about debugging etc
http://wiki.aithinker.com/esp32-cam – Chinese site for the module manufacturer

41 Replies to “ESP32-CAM | ESP32 Camera Module with Face Recognition”

  1. nocode says:

    how and where can you edit the html page ? which file has the sourcecode of the html of the webserver ?

    i programmed via arduino ide .

    https://i.imgur.com/qmeV07s.png

    1. WordBot says:

      I’m still looking into this but I think it’s this: https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Camera/CameraWebServer/camera_index.h . The page html is gzipped and converted to a const char array of hex values.

      UPDATE: New tutorial on editing the HTML: https://robotzero.one/esp32-cam-custom-html/

  2. mhogan says:

    This was extremely helpful. Thanks! Is it possible to save an image to the built-in SD card? I’ve been trying to get the card to work but so far the ESP32-CAM won’t recognize it.

    1. WordBot says:

      Here’s a start – https://github.com/raphaelbs/esp32-cam-ai-thinker/tree/master/examples/sd_jpg this is for the Espressif IDF. I’ve not tried it myself.

      1. WordBot says:

        This tutorial has a section for saving to the SD card – https://robotzero.one/time-lapse-esp32-cameras/

  3. Gautham says:

    Nice tutorial , thanks for sharing ,
    Is it possible to handle face recognition , movement detection and other task like registering the name of the face recognized in cloud with this , are all these possible with the 4MB RAM ?

    1. WordBot says:

      Not sure about movement detection with the current library but face detection and sending the name of the person to the cloud would be possible with some work.

      EDIT: Check this page for tutorials for sending names via POST or Blynk: https://robotzero.one/esp32-camera-projects/

  4. Jairo says:

    I think there’s an error in export_idf_path needs an “export” so that the variable is available outside that script i.e.
    export IDF_PATH=”C:/msys32/home/username/esp/esp-idf”

    1. WordBot says:

      You’re right! Thanks for pointing that out. I’ve fixed it in the article.

  5. Emmanuel says:

    Running into issues of pySerial not being installed when I run Make Flash. Any workaround for this?

  6. David says:

    Hi,
    thanks for the details on getting this up an running.
    I am only get MJPG: 10003B 1324ms (0.8fps), AVG: 2156ms (0.5fps), 0+0+0+0=0 0 for qVGA where I see some people saying the get 30fps + , any ideas why my frame rate would be so bad? I had a look at the timings and the initial connect is 261ms, waiting TTFB is 544ms and download is 1.30s. I have no issues streaming and have good download speeds with everything else vir wifi.

    Regards
    David.

  7. varun says:

    hi,
    how to solve python dependencies error .because of this error i could not able to flash the unit.
    can anyone send the link to download this tool chain version (crosstool-ng-1.22.0-80-g6c4433a). i need in both 32 and 64 bit.

    1. WordBot says:

      Hi, Maybe something here will help: https://esp32.com/viewtopic.php?t=7017&start=20 Is the tutorial above not working at the moment?

  8. ram says:

    cant able to install toolchain set up for UBUNTU 16.04 os.Is there anyother way to install the toolchain setup?

  9. Rockman says:

    Which kit is this ESP32 cam comes up on from these three?
    ESP32 DevKitC
    ESP-WROVER-KIT
    ESP32-PICO-KIT

    1. WordBot says:

      It’s the Ai-Thinker. You should see it as an option when you make menuconfig. It’s here in the code: https://github.com/espressif/esp-who/blob/master/examples/single_chip/camera_web_server/main/include/app_camera.h

  10. Rockman says:

    Serial flasher config —> Default serial port (Change to the port number shown in Device Manager – ie COM3)

    This changes cannot able to do in UBUNTU 16.04 .How to to do this please comment

  11. sunil says:

    HI @WordBot,
    i can able to take a still with the camera but face detection and face reorganization is not happening, Do any one know for what reason it is not happening.please help me out to solve this issue.

    1. WordBot says:

      When you turn Face Detection on and press Start Stream do you get a green box around your face? It has to be the right way up. Here’s a video: https://www.youtube.com/watch?v=gZgGniTLCiU

  12. ehatzidimi says:

    is it easy to activate a pin out when it recognize a registered face?

    1. WordBot says:

      There’s some examples here: https://github.com/espressif/esp-who/tree/master/examples/single_chip I imagine you could add a line of code to set a pin high or low when a face is recognised to one of the examples.

      UPDATE: In the Arduino IDE this example sets a pin high: https://robotzero.one/esp32-face-door-entry/

  13. alxexandru says:

    very nice and cheap project !
    can we do with this face recognition to say “hello John” , if John face is on front of camera?

  14. Thomas Kjeldsen says:

    this guide……. lack information….
    my machine dont know git… ohhh you need to install it… nice to know….
    the msys32 basic respond is unknown command

    1. WordBot says:

      You need to install msys32 first and then run through the instructions using this.

  15. David says:

    hi Robotzero,

    thanks for using time to teach us :-). Do you had any idea what happen here? My issue is documented here:

    https://github.com/espressif/esp-idf/issues/3730 (make flash failed)

    esp-who: newest one, download from today
    esp-idf: 3.2.2
    msys64: newest one, download from today
    cat c:\Users\tuan\Documents\Arduino\esp-who\esp-idf\tools\toolchain_versions.mk
    SUPPORTED_TOOLCHAIN_COMMIT_DESC = crosstool-ng-1.22.0-80-g6c4433a
    SUPPORTED_TOOLCHAIN_GCC_VERSIONS = 5.2.0
    CURRENT_TOOLCHAIN_COMMIT_DESC = crosstool-ng-1.22.0-80-g6c4433a
    CURRENT_TOOLCHAIN_COMMIT_DESC_SHORT = 1.22.0-80-g6c4433a
    CURRENT_TOOLCHAIN_GCC_VERSION = 5.2.0

    # xtensa-esp32-elf-gcc –version (msys64: mingw32.exe )
    xtensa-esp32-elf-gcc.exe (crosstool-NG esp32-2019r1) 8.2.0

    so it seem the toolchain from msys64 is newer than the one inside “ESP-WHO” (https://github.com/espressif/esp-who.git)

    1. WordBot says:

      I’m not sure what you’ve done but I would delete everything and then follow this step by step: https://robotzero.one/esp32-camera-module/ I’ve not tested that it still works for a few months but it should be OK.

  16. David says:

    Thanks, I did follow your step 100% 🙂 and recheck it many times.
    I am afraid it will not help due to the new msys2’s toolchain is newer than the one from ESP-WHO git, beside some other thing is missing as well. As one wrote on git, all that continuously changes and dependency (esp32 needs msys2) make a new guide/tip 2 decade old as soon you public it.
    what i had learn from it : from now on I will save the zip and share the versions to the user so they know what expected them.
    msys2: toolchain= crosstool-NG esp32-2019r1, gcc: 8.2.0
    “ESP Who” expect: toolchian=crosstool-ng-1.22.0-80-g6c4433a ; gcc:5.2 (cat c:\Users\tuan\Documents\Arduino\esp-who\esp-idf\tools\toolchain_versions.mk)

    I hope the espressif will help to sove out.
    br

  17. WordBot says:

    I just ran through the tutorial and apart from one restart of mingw32 (think I lost internet) and having to run an extra command (now added to the tutorial)…

    C:/msys32/mingw32/bin/python.exe -m pip install –user -r C:/msys32/home/**yourusername**/esp/esp-idf/requirements.txt

    it all worked fine. You just download the 32bit version of MYSYS and paste all the commands into the MYSYS terminal. It should download and install everything for you.

  18. David says:

    hi robotzero
    Thanks for your patience, 2 thing I did wrong: i use msys2 64bit and i reused the esp-idf inside esp-who.
    git clone –recursive https://github.com/espressif/esp-idf.git
    that esp-idf has the correct toolchain, which support by d:\msys32\opt\xtensa-esp32-elf\
    nb: i compare the xtensa directory at 32bit & 64bit, they are similar. so it will be ok to use msys2 64bit as well I guess (didnt try this, the “make flash” take almost 15min)

    c:\Users\tuan\Documents\Arduino\esp\esp-idf\tools\toolchain_versions.mk
    SUPPORTED_TOOLCHAIN_COMMIT_DESC = esp32-2019r1
    SUPPORTED_TOOLCHAIN_GCC_VERSIONS = 8.2.0

    CURRENT_TOOLCHAIN_COMMIT_DESC = esp32-2019r1
    CURRENT_TOOLCHAIN_COMMIT_DESC_SHORT = esp32-2019r1
    CURRENT_TOOLCHAIN_GCC_VERSION = 8.2.0

    Nice too know how to use “make” for the code, I did tried the same code on arduino IDE (I choose the board “AI thinker esp32 CAM”), it is faster (took around 2min)
    I exclude those folders in my anti virus AVG to get speed up the arduino compile.
    c:\Users\tuan\AppData\Local\Arduino15\
    c:\Users\tuan\Documents\Arduino
    d:\program files\arduino

    ….
    got this error
    esptool.py v2.7-dev
    Flashing binaries to serial port /dev/ttyS15 (app at offset 0x10000)…
    esptool.py v2.7-dev
    Serial port /??/COM16
    Traceback (most recent call last):

    I think too much I think 🙂 , i should just use COM16 (my usb-> serial use port 16) in “make menuconfig” as you wrote in the guide. Another 15min need to wait…..

    Anyway 1 cup of black coffee is sent with UPS 🙂 …thanks you for your help & well done.
    I will let the guy at ExpressIF know, so they can close the ticket. (they use an old toolchain inside esp-who)
    br David

  19. David says:

    # make flash
    Toolchain path: /opt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
    Toolchain version: esp32-2019r1
    Compiler version: 8.2.0
    Flashing binaries to serial port com16 (app at offset 0x10000)…
    esptool.py v2.7-dev
    Serial port com16
    ….
    Compressed 3072 bytes to 63…
    Wrote 3072 bytes (63 compressed) at 0x00008000 in 0.0 seconds (effective 2527.6 kbit/s)…
    Hash of data verified.
    Leaving…
    Hard resetting via RTS pin…

    Wow, I think using “make”, it makes my esp-32 more responsive (on the website) than using arduino IDE. When I play around with arduino I think the frame on the website is more lagging, slow….
    I

    1. WordBot says:

      The IDF version probably has more recent code for the webserver example. Arduino setup is muuuuch easier! At the moment the face detection doesn’t work in the current version of ESP32 library for Arduino IDE.

  20. David says:

    arduino code:
    c:\Users\tuan\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.3-rc1\libraries\ESP32\examples\Camera\CameraWebServer\CameraWebServer.ino

    C code:
    c:\Users\tuan\Documents\Arduino\esp-who\examples\single_chip\camera_web_server\

    I will definitive look for C code in the future 🙂
    c:\Users\tuan\Documents\Arduino\esp-who\examples\single_chip\camera_web_server\ 121.043.810 05/07/2019 18.54 —-
    c:\Users\tuan\Documents\Arduino\esp-who\examples\single_chip\detection_with_command_line\ 24.221 03/07/2019 14.24 —-
    c:\Users\tuan\Documents\Arduino\esp-who\examples\single_chip\recognition_solution\ 45.851 03/07/2019 14.24 —-
    c:\Users\tuan\Documents\Arduino\esp-who\examples\single_chip\recognition_wechat\ 95.765 03/07/2019 14.24 —-
    c:\Users\tuan\Documents\Arduino\esp-who\examples\single_chip\recognition_with_command_line\ 29.174 03/07/2019 14.24 —-

    1. WordBot says:

      Arduino code is C (sort of). Most of the libraries from the IDF examples https://github.com/espressif/esp-who/tree/master/examples/single_chip will work in the Arduino IDE. Nearly all of the tutorials I’ve written recently use code taken from them because it will also compile with the Arduino IDE. You have to make changes to fit the Arduino programming style such as using void setup() and void loop(). More reading: https://www.esp32.com/viewtopic.php?t=5669

  21. Danny says:

    even though my dev. environment is FreeBSD, it went without a stich!!
    at the moment the only problem I see is that there are some horizontal lines
    when streaming, any idea what it can be?

    1. WordBot says:

      Probably the hardware itself. The ESP32-CAM isn’t that well designed and suffers from heat problems. I’ve had to replace a couple of the cameras on mine.

  22. danny says:

    I just tried it on a ttgo-camera-plus, and no horizontal lines!
    great work!
    you might want to add it to the list of supported.

  23. ed says:

    I found the face recognition to only work with the previous ESP32 core, nor with core 1.0.2

    1. WordBot says:

      Yeah the newest Arduino IDE ESP32 hardware libraries aren’t working at the moment: https://github.com/espressif/arduino-esp32/issues/2731

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

scroll to top