ESP32-CAM Face Recognition for Home Automation

Featured

Using face recognition to open a door or control other home automation devices

This tutorial will explain how to save enrolled images in the on-board flash so they survive the ESP32 powering off and use these saved recognitions to control devices connected to the ESP32. There are three steps.

  • Create a new partition scheme to enable persistent storage
  • Modify the CameraWebServer example sketch to save face data to the new partition
  • Use these saved recognitions to control devices connected to the ESP32

Before following this tutorial make sure your camera works by following this tutorial Ai-Thinker ESP32-CAM in the Arduino IDE

Persistent Storage Partition Scheme

A new partition scheme with persistent storage on the on-board flash is needed.

If you are using 1.0.5 of the Arduino ESP32 hardware libraries you can just add a partitions.csv file to the folder containing the Sketch and this will upload with the Sketch. Note you should choose the ‘Huge App’ partition scheme from the tools menu when uploading.

You can download a suitable partition from here: https://robotzero.one/wp-content/uploads/2019/04/partitions.csv

If you are using 1.0.4 of the hardware libraries you can add a new partition scheme to the IDE by downloading a scheme I created from here: https://robotzero.one/wp-content/uploads/2019/04/rzo_partitions.csv

Add this file to the directory containing the other partition schemes. This is found in one of two places, depending on how you installed the Arduino IDE.

Arduino IDE installed from the Windows Store:

C > Users > *your-user-name* > Documents > ArduinoData > packages > esp32 > hardware > esp32 > 1.0.4 > tools > partitions

Arduino IDE installed from the Arduino website:

C > Users > *your-user-name* > AppData > Local > Arduino15  > packages > esp32 > hardware > esp32 > 1.0.4 > tools > partitions

The new scheme has to be added to your ESP device in the boards manager configuration file – boards.txt. Again this is found in one of two places.

Arduino IDE installed from the Windows Store:

C > Users > *your-user-name* > Documents > ArduinoData > packages > esp32 > hardware > esp32 > 1.0.4

Arduino IDE installed from the Arduino website:

C > Users > *your-user-name* > AppData > Local > Arduino15  > packages > esp32 > hardware > esp32 > 1.0.4

Add the following three lines below the existing partitionScheme options for the esp32wrover board in this boards.txt file.

esp32wrover.menu.PartitionScheme.rzo_partition=Face Recognition (2621440 bytes with OTA)
esp32wrover.menu.PartitionScheme.rzo_partition.build.partitions=rzo_partitions
esp32wrover.menu.PartitionScheme.rzo_partition.upload.maximum_size=2621440

Close and reopen the IDE to confirm the new ‘Face Recognition’ partition scheme is available in the Tools menu.

There’s an article here that explains in much more detail how to set up a new scheme and duplicate a board definition: Partition Schemes in the Arduino IDE

Capture Face Data to Persistent Storage

The CameraWebServer example in the IDE doesn’t save enrolled faces in a way that will survive power loss. To modify it to use the new partition a few changes need to be made to the code.

In the Arduino IDE,  make a copy of your working CameraWebServer Sketch from the previous tutorial by saving it with a new file name such as CameraWebServerPermanent.

You should see three tabs in the Arduino IDE similar to the image below:

Arduino IDE Tabs

In the second tab (app_httpd.cpp) make the following changes.

After #include “fr_forward.h” (around line 24) add:

#include "fr_flash.h";

Change int8_t left_sample_face = enroll_face(&id_list, aligned_face);(&id_list, aligned_face);  (around line 178) to:

int8_t left_sample_face = enroll_face_id_to_flash(&id_list, aligned_face);

After face_id_init(&id_list, FACE_ID_SAVE_NUMBER, ENROLL_CONFIRM_TIMES); (around line 636) add:

read_face_id_from_flash(&id_list);

Flash and run this Sketch in the same way as before. Enrolled face data is now being saved to the new partition on the flash memory.

Face Recognition Trigger Event

I’ve written a Sketch that sets a pin HIGH on the board while a known face is detected. After 5 seconds pass, if there is no recognised face the pin is set LOW. The code in the function rzoCheckForFace() in the Sketch below can be changed to whatever function you require when a face is recognised. I prefer to set pins as low during setup because mentally I think of it as ‘off’ and then set them high when I want to trigger something because I think of it as ‘on’. You can use pin 12 instead or in addition to pin 2

Paste the following code into a new sketch

When you flash and run this new Sketch you should see ‘Face recognised’ in the serial monitor when a matched face is found.

Opening a Door

The Sketch above combined with a relay or Mosfet module can be used to switch an electrical device on or off. This can be used to open or unlock a door

The diagram below shows the wiring for a opening a lock. From the Sketch above, when a face is recognised, pin IO2 is set HIGH and the relay closes so current flows from the high voltage power supply to the electric door lock.

ESP32-CAM Door Lock Relay

While setting up and testing a project like this you might prefer to have the serial device connected as below

ESP32-CAM Door Lock Relay with USB

The door lock in this example could be anything you want to temporarily supply power to.

The project will work with many relay and Mosfet modules that have a 3v input such as the items below. The opto isolated Mosfet module (green connectors) needed the resistor bypassed. The smaller red one works with just the signal and ground connected.

These are available from eBay: Basic Mosfet | Opto Isolated Mosfet | Opto isolated 3v Relay

There’s lots of options for controlling a high voltage device from the ESP32 and it is important to understand the dangers to yourself and the ESP32 if you use the wrong solution.  The mechanical relay board is rated for higher voltages but I personally wouldn’t use mains electricity with a device like this.

Deleting Faces from the Memory

If you need to clear the stored faces, paste and upload the Sketch below

#include "esp_camera.h"
#include "fd_forward.h"
#include "fr_forward.h"
#include "fr_flash.h"

#define ENROLL_CONFIRM_TIMES 5
#define FACE_ID_SAVE_NUMBER 7

static face_id_list id_list = {0};

void setup() {
  Serial.begin(115200);
  face_id_init(&id_list, FACE_ID_SAVE_NUMBER, ENROLL_CONFIRM_TIMES);
  read_face_id_from_flash(&id_list);// Read current face data from on-board flash
  Serial.println("Faces Read"); 
  while ( delete_face_id_in_flash(&id_list) > -1 ){
        Serial.println("Deleting Face");
  }
        Serial.println("All Deleted");  
}

void loop() {
}

I have some more tutorials like this coming. Feel free to buy me a coffee to help development 😉 .

References

Event trigger for IDF: https://github.com/espressif/esp-who/blob/master/examples/single_chip/recognition_with_command_line/main/app_facenet.c
In-depth relay information: https://www.youtube.com/watch?v=d9evR-K6FAY

195 Replies to “ESP32-CAM Face Recognition for Home Automation”

  1. Mel says:

    You’ve used IO2 or 12 for the relay control. Aren’t these pins also used for SD card? I’m looking for any module pins that are “free”, seem to be scarce commodity; conflicts with flash LED, SD etc.; idea would be to use an external PIR to save jpeg to SD card, or means to display image remotely. Also would this module work with the OV2640 fisheye camera?

    1. WordBot says:

      Yeah.. All the pins are assigned to something. Those two seem to be useable for other things but probably not at the same time as using the SD reader. I’ve got a tutorial in progress for writing to the micro-SD or uploading. Not sure about the fisheye.. there’s a few of the little OV2640 cameras and I’m not sure the cables are the same.

  2. Vitor Faeda Dalto says:

    Hello, there is no way of sending the face recognition data/trigger via web?
    Like a FTT ?
    I would like to send a message to Home Assistant, inside it it’s easier to make any action.
    Is that possible ?
    Here is an exemple:
    Individual 1 appears in front of the camera ->
    ESP sends package to my arduino IP -> action 1 speaker says “hello individual 1”, action 2 August Smart Lock opens the door.
    It’s possible ?
    Thanks

    1. WordBot says:

      Hi, what’s FTT? Rather than making a pin go high or low on the board you want it to send an HTTP request to a device on your network? Yep.. should be possible. I have a Sonoff device arriving soon so I’ll do a tutorial for that. Should be easy enough to change it to point to an Arduino.

      EDIT: Sonoff tutorial here: https://robotzero.one/unlock-door-face-detection-sonoff/

  3. Mikk7310 says:

    Hello, great tutorial, thanks! How can I delete the stored face?

    1. WordBot says:

      Hi, thanks! I’ve added a Sketch at the end of the article that deletes the stored faces.

  4. Jose Godinho says:

    Hello
    Excellent tutorial, albeit a lot of people were ask the same, would it be possible, instead of change the GPIO state, send a MQTT info about the face recognized?

    I think that would entitle to several coffees

    Regards

    Jose Godinho

    1. WordBot says:

      Hi, How are you using MQTT in your application?

  5. kwluk717 says:

    Hi,
    Do you think is it possible to save the face information to SD card so we can recognize other people by replaced the card ? Thank you.

    1. WordBot says:

      Hi, it’s on my list of things to look into.

  6. Didace says:

    hello Sir! after following your instructions I tried to run the given sketch for triggering the relay when a recognized face is detected but on serial I get
    rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
    configsip: 0, SPIWP:0xee
    clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
    mode:DIO, clock div:2
    load:0x3fff0018,len:4
    load:0x3fff001c,len:1100
    load:0x40078000,len:9232
    load:0x40080400,len:6412
    entry 0x400806a8
    E (1719) fr_flash: Not found

    1. WordBot says:

      Hi, Did you follow the whole tutorial including setting up a new partition and adding some faces to the device?

  7. kami says:

    Hi,
    have you tested the OTA Update? I cannot get it running? I also got problems to attach an interrupt. Have you tried this?
    Thanks a lot.

    Cu kami

    1. WordBot says:

      I’ve not tested OTA but I’m not sure if there’s space on the ESP32-CAM to store the sketch, faces and have space for an OTA partition. Reading the pins is hard, or maybe not even possible if you want to use Wifi https://desire.giesecke.tk/index.php/2018/01/30/adc2-channel-cannot-be-used-when-wifi-is-in-use/ I would love to be proved wrong on this!

  8. kami says:

    Hi, thanks a lot for the answer. In the partition table there is space for ota. I will try it on the weekend.

    Can you give me some information about the broken face recognition in 1.0.2?

    Thanks a lot.

    Cu kami

    1. WordBot says:

      There’s supposed to be a new Arduino IDE board setup for ESP32 release soon. Hopefully that will fix it.

  9. En says:

    Hi,
    I use your source code, always show me this error,

    exit status 1
    ‘mtmn_config_t’ does not name a type

    Have you ever encountered this problem?

    1. WordBot says:

      Hi, Which version of the ESP32 libraries are you using? library version

  10. En says:

    thanks reply, also version1.0.1

  11. En says:

    have you install other #include ?
    maybe #include “mtmn.h” or something else ?

  12. En says:

    I JUST FIX IT!!!!
    It’s the Filename Extension,
    I change .ino to .h

  13. WordBot says:

    which file did you change .ino to .h?

  14. En says:

    the new sketch I paste from the chapter “Face Recognition Trigger Event”,
    I encounter a new problem. relayPin have no response

  15. En says:

    I didn’t see ‘Face recognised’ in the serial monitor,
    I create a new sketch as 123.h, and put it in a file with other sketch(CamerawWebserver.ino, app_http.cpp, camera_index.h),
    Is that correct?

    sorry for bothering

  16. En says:

    I figure it out, sorry for stupid question lol

  17. Ridoillah says:

    what esp32cam method use for face recognition ?

    1. WordBot says:

      MobileFace – https://github.com/espressif/esp-who scroll down for details

  18. Paulo Freitas says:

    Olá
    Como eu faço para colocar as faces conhecidas no ESP, não entendi esta parte, o restante está configurado e parece estar funcionando.

    rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
    configsip: 0, SPIWP:0xee
    clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
    mode:DIO, clock div:1
    load:0x3fff0018,len:4
    load:0x3fff001c,len:1100
    load:0x40078000,len:9232
    load:0x40080400,len:6400
    entry 0x400806a8
    E (1716) fr_flash: No ID Infomation
    Get one frame in 69 ms.
    Get one frame in 61 ms.
    Get one frame in 56 ms.
    Get one frame in 55 ms.
    Get one frame in 55 ms.

    1. WordBot says:

      Hi, Have you saved some faces by following the part Capture Face Data to Persistent Storage?

  19. Denis says:

    Ok first of thank you for a great project….I followed everything step by step and everything went without a hitch but there is a problem …. when i go to enroll faces nothing happens ( i am using ESP32 version 1.0.1 on the Arduino IDE)
    Of course i am toggling the buttons for facial recognition and detection
    on the serial monitor i do get this in the beginning……… E (5628) fr_flash: Not found

    What an i doing wrong how can i enroll faces
    Thanks again
    Denis

    1. WordBot says:

      Hi, Does the normal CameraWebServer example work for you. Can you enrol a face and it recognise it? If that works try with the edits in this tutorial (including the new partition scheme) and see if you can enrol faces.

  20. Denis says:

    No i get that back the original sketch does detect faces i tried it again and it took a while but it did register my face
    Denis

  21. Denis says:

    Ok so i got it flashed again per your instructions now the face detection works just fine … detection that is….. and it says Intruder Alert when there is no face registered.
    When i click on Enroll Face (just once) it starts this wired loop that goes exactly like this: “Note from Sample 4 jumps to Sample 7”
    ID(0) Sample 1
    ID(0) Sample 2
    ID(0) Sample 3
    ID(0) Sample 4
    ID(0) Sample 7 than it moves to ID (1) Sample 1 though 4 than jumps to 7 than on ID(2) Sample 1 through for jump to 7 up to ID 5 and back again in a loop.

    When you stop this process that does not finish by itself by switching off face detection and recognition and you start it back on again it says Intruder Alert like it has not saved any images.

    Whats causing this behavior and how do we fix it….by the way with the stock sketch it saves and recognizes the faces just fine

    thank you for your help
    Denis

  22. WordBot says:

    I’m not sure why it’s doing that. Are you using the new partition? I wonder if it can’t save the ‘face’.

    Is there anything in the serial monitor?

  23. Denis says:

    Where is the new partition ??
    I am using the downloaded one from the instructions above
    https://robotzero.one/wp-content/uploads/2019/04/rzo_partitions.csv

    …. is there a new csv??

    I am also adding the three lines into board.txt
    “`esp32wrover.menu.PartitionScheme.rzo_partition=Face Recognition (2621440 bytes with OTA)
    esp32wrover.menu.PartitionScheme.rzo_partition.build.partitions=rzo_partitions
    esp32wrover.menu.PartitionScheme.rzo_partition.upload.maximum_size=2621440 “`

    One think i have noticed is when the camera reboots it gives the following error
    E (52290 Could not Load (or Could not find): fr_flash.h
    in the serial monitor other than that it just keeps displaying the data for the captured frames on the serial monitor.

    So i think the above error is causing that issue ….what is that??

    Denis

    1. WordBot says:

      > E (52290 Could not Load (or Could not find): fr_flash.h
      Is that the exact error? fr_flash.h is a file needed for the sketch to run but it shouldn’t compile if it doesn’t exist.

      If the error message mentions fr_flash is missing it’s because the ESP32 can’t find the partition to store the face data. Did you select the new partition similar to this screen capture: https://robotzero.one/wp-content/uploads/2019/04/arduino-ide-custom-partition-scheme.jpg ?

  24. FrenchyRic says:

    I had the same error (Fr_flash.h missing), you have to select: tools -> Partition scheme -> Face Recognition (xxx Bytes with OTA)

  25. Denis says:

    Ok well the Partision Scheme with Face Recognition (xxx Bytes with OTA) seems to be fixing the problem of the fr_flash.h missing and there are no there errors. So it seems like it will work but still i can not test it successfully and can not store a face id yet. Because either the wifi connectivity gets extremely bad when uploading this code or the camera slows down i cant really tell.
    When i start the camera feed it will show one or two frames than it wont even get a frame per second so it will loose connectivity the web page with the webserver there is no way to run facial recognition and store face id’s.

    I will try to go back to the stock sketch and test the wifi again maybe it has gone crazy but with the stock code the face recognition and the cam was previously working very well.

    Anyone else having this issue ??

    Denis

    1. vimal says:

      me E (5857) fr_flash: Not found

      1. WordBot says:

        Does the project still work? I think you might see this error when you first start the ESP32 because nothing has been saved yet.

  26. Denis says:

    Here is the serial monitor out put …note the fps
    Also the wifi access point is like 10 feet away

    WiFi connected
    E (6595) fr_flash: No ID Infomation
    Starting web server on port: ’80’
    Starting stream server on port: ’81’
    Camera Ready! Use ‘http://192.168.1.17’ to connect
    MJPG: 8994B 170ms (5.9fps), AVG: 170ms (5.9fps), 0+0+0+0=0 0
    MJPG: 8869B 1341ms (0.7fps), AVG: 755ms (1.3fps), 0+0+0+0=0 0
    MJPG: 8892B 4264ms (0.2fps), AVG: 1925ms (0.5fps), 0+0+0+0=0 0
    MJPG: 7850B 3524ms (0.3fps), AVG: 2324ms (0.4fps), 0+0+0+0=0 0
    MJPG: 5589B 540ms (1.9fps), AVG: 1967ms (0.5fps), 0+0+0+0=0 0
    MJPG: 8501B 3638ms (0.3fps), AVG: 2246ms (0.4fps), 0+0+0+0=0 0
    MJPG: 10435B 2149ms (0.5fps), AVG: 2232ms (0.4fps), 0+0+0+0=0 0

  27. Denis says:

    It looks like Circuit Board Antenna Connected… not the external one ….but i do have an external antenna connected to it should i take it off and see whats up??

    1. WordBot says:

      You could try but if it’s not connected to anything it shouldn’t affect the signal. If you can it might be better to move the jumper so the external antenna is connected. I found the WiFi really affects the FPS.

  28. Rolly says:

    Hello!

    Great article, I wish you can do a sketch where the face maintenance (add and delete) and detetion are included.

    1. WordBot says:

      Hi,
      You can use this tutorial to manage the faces in the system: https://robotzero.one/esp-who-recognition-with-names You could either switch between the two sketches or take the code from the tutorial above and combine them.

  29. Denis says:

    Hello again
    I followed everything step by step, double checked and triple checked, re-flashed it a couple of times too…i solved the problem with the wifi dropping frames but still no facial recognition when i use your code and partition after i click on Enroll Face nothing happens and i can confirm that the recognition works with the stock sketch
    When i go step by step to your screen shot here https://robotzero.one/wp-content/uploads/2019/04/arduino-ide-custom-partition-scheme.jpg i see that your Board says Robot Zero One is that just for you guys developing cause we do not have that board and the other settings that your board has…..i am of course selecting the partition scheme with Face Recognition (2621440 bytes with OTA)

    What should we do next there is no error when you reset the board on the serial monitor other than this:
    E (7651) fr_flash: No ID Infomation
    Which i guess it means that there is not face id stored yet.

    Thanks again
    Denis

    1. WordBot says:

      The ‘Robot Zero One’ board is just a copy of the esp32wrover board. I did it like that in the other tutorial to make it less likely that people would break something. I don’t know why it’s not working for you. I’ll try to find time to make a video this week to show the process.

  30. SeJeong says:

    Hello.
    Thank you for your great work.
    I’d like to chage the relay module to servo motor because I want to control the door model by servo motor.
    If I just chage the module and put the the line which are “#include “, “Servo relayPin;”, could it work?

    1. WordBot says:

      Hi, I’m adding a general servo tutorial tomorrow. With a servo motor you have to tell it how far to turn. Probably in your case you would add a function to the Sketch that when run, turns the servo enough for the door to open, waits for a certain time and then turns the servo back to close the door.

      1. SeJeong says:

        Thank you for your reply.
        I changed your code and completed my code.
        However, the motor doesn’t work.
        I connected the motor to GPIO2 pin of esp 32 cam and supplied the external power supply to motor by battery 5V.
        I guessed that very small output signal is given to GPIO2 because I coudn’t hear any begining sound from motor.
        How can I solve this problem?

        Below is my partial code.

        void rzoCheckForFace() {
        currentMillis = millis();
        if (run_face_recognition()) { // face recognition function has returned true
        Serial.println(“Face recognised”);
        // digitalWrite(servoPin, HIGH);
        servo.write(170);
        Serial.println(“Motor Moved”);
        delay(30000);
        openedMillis = millis(); //time relay closed
        }
        if (currentMillis – interval > openedMillis){ // current time – face recognised time > 5 secs
        // digitalWrite(servoPin, LOW);
        servo.write(0);
        delay(300);
        }
        }

        In addition, I add initial state of motor in void setup() part.

        void setup() {
        Serial.begin(115200);
        servo.attach(servoPin);
        servo.write(0);
        ….

        1. WordBot says:

          Are you using the normal Arduino servo library? I’m not sure it will work. See this tutorial for servo code: https://robotzero.one/pan-and-tilt-control-for-an-esp32-cam/

  31. Nothing2Fancy says:

    Does anybody know how I can augment this code to trigger a dfplayer (a cheap broken out mp3 player,) by sending a hexadecimal number command to serial tx/rx? (That’s how you play certain audio files from the onboard sd-card.)

    I am trying to make a smart(ish) doorbell system that would recognize houshold members and greet them, by name, upon ringing the doorbell (or in this case activating a momentary button switch) which would first check the database for a saved face, play a file associated with said person, then ring a bell by way of either closing a relay or playing a different file (like a sample of Big Ben for instance) to a speaker on the inside of the door.)

    A visitor without a saved face would have a file played implying an unknown person is calling. I would also really like to be able to text a live snapshot with Telegram. Having Alexa chime in (pun intended, …sadly,) would also be pretty cool and in lieu of using an actual doorbell.

  32. Denis says:

    Let me know on the video or if you have time please let me send you the cam and a prepaid return envelope and you can trouble shoot it hands on ….i am desperate and i really need to get this project complete as it fits my situation perfectly

    Thank you i have tried everything and the board just wont recognize faces after flashed with your sketch

  33. Maikl says:

    Hello! Please write a sketch for the tutorial: https://robotzero.one/esp-who-recognition-with-names. I can treat you to coffee. I tried to redo the sketch to control the relay, but I have little experience and nothing worked.

    1. WordBot says:

      Hi, I have a tutorial video for access control with face recognition nearly ready (a combination of this tutorial and the one with names). If you subscribe to my Youtube channel https://www.youtube.com/channel/UCpqVEutU_Jcq34ws4tyUReQ you should get notified when it’s live. There will be a sketch to download.

  34. Maikl says:

    Oh no! I meant that the sketch was without WEB Cockets, without WEB server, if the connection is lost, the program does not work. We need a fully autonomous operation of the module. Please learn how to disable WEB Cockets.

    1. WordBot says:

      Would you be able to use one sketch to capture the faces and then upload another to run autonomously? It’s tricky to have it running in a loop waiting for new faces and then switch out of this to only be in access control mode. This tutorial works without being connected to a browser: https://robotzero.one/face-recognition-with-blynk/ but you have to save faces using another sketch.

  35. Maikl says:

    I would like to remake the sketch from this lesson so that it can read the names recorded on the flash memory from the sketch “FaceDoorEntryESP32Cam”. Can you help me with this?

    1. WordBot says:

      I’m not really sure what you want to do. Can you explain the full process please?

  36. Maikl says:

    Hello! I apologize for the poor translation. The point is this:
    1. Download the “FaceDoorEntryESP32Cam” sketch to the ESP32 board.
    2. Write the names in memory.
    3. Fix the sketch “ESP32-CAM Face Recognition for Home Automation” so that it can read the names recorded in the previous paragraph.
    4. Download the fixed sketch “ESP32-
    CAM Face Recognition for Home Automation. “

    1. WordBot says:

      Can you tell me what you want you want the final project to do? When you say ‘read’ do you mean say the names through a speaker?

  37. Maikl says:

    Hello Friend!!! I do not need a voice alert. I will try to explain in another way. If I upload the “FaceDoorEntryESP32Cam” sketch to ESP32 and write down the names to search (in the area fr 0x290000, 0xEF000). Then I will upload the “ESP32-CAM Face Recognition for Home Automation” sketch to ESP32. Will it work ???

    1. WordBot says:

      Ok I think I understand. I don’t think it will work because https://robotzero.one/esp32-face-door-entry/ has the function read_face_id_from_flash but https://robotzero.one/esp-who-recognition-with-names/ has read_face_id_from_flash_with_name So the data is probably in incompatible formats. If you don’t need names (I don’t think you do?) you can always just save them without names as in the https://robotzero.one/esp32-face-door-entry/ tutorial in the ‘Capture Face Data to Persistent Storage’ section. The one without names works better because the ESP libraries for 1.0.1 has less bugs.

  38. Maikl says:

    Hello! Yes, I have already encountered the fact that the names have incompatible formats. My question is about the Access Control with Face Recognition project. Is it possible to change the void loop () function in a sketch so that it would be possible to search for faces without connecting to a WEB server?

    1. WordBot says:

      Yep but I don’t have time to code it at the moment. Why not just use the Home Automation tutorial? It works without a web server.

  39. Rodrigo Franco says:

    Hello, first thank you, it is a great tutorial, question,
    how many faces can be enrolled?
    thank you.

    1. WordBot says:

      Hi, I thought there was a limit of around 7 (this code: #define FACE_ID_SAVE_NUMBER 7) but maybe not… https://github.com/espressif/esp-who/issues/118

  40. RODRIGO FRANCO MARTINEZ says:

    Ok, thanks a lot for the information.
    Greetings is a great project.

  41. SOLEHIN AZRAIN BIN SULAIMAN says:

    hi
    i want to ask why i get this message after delete the face
    Camera Ready! Use ‘http://192.168.43.244’ to connect
    [D][WiFiClient.cpp:509] connected(): Disconnected: RES: -1, ERR: 104
    [D][WiFiClient.cpp:509] connected(): Disconnected: RES: -1, ERR: 104
    [E][WiFiClient.cpp:392] write(): fail on fd 62, errno: 104, “Connection reset by peer”
    [E][WiFiClient.cpp:392] write(): fail on fd 63, errno: 104, “Connection reset by peer”

    this error will show when i reload the page

    1. WordBot says:

      Hi, This is the Wi-FI library. It can be quite glitchy when streaming video on the ESP32-CAM. Does it delete the face from the system Ok?

  42. SOLEHIN AZRAIN BIN SULAIMAN says:

    assertion “l->size == size” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/components/esp-face/face_recognition/fr_flash.c”, line 85, function: read_face_id_from_flash
    abort() was called at PC 0x400db8eb on core 1

    Backtrace: 0x4008d888:0x3ffb1ed0 0x4008dab5:0x3ffb1ef0 0x400db8eb:0x3ffb1f10 0x400e93f6:0x3ffb1f40 0x400d0d50:0x3ffb1f80 0x400d1f33:0x3ffb1fb0 0x4008a53d:0x3ffb1fd0

    i got this error when trying to delete face
    if i upload the example code for camera it working find without any error

  43. WordBot says:

    Are you using the code from the section above titled “Deleting Faces from the Memory” to delete the faces? Have you saved some faces successfully?

  44. munish sharma says:

    hi how to get it work without oppening the url everytime?

    1. WordBot says:

      This version will work without the browser: https://robotzero.one/access-control-with-face-recognition/

      1. Stefan Spurgin says:

        Hi, im searching for all comments on all your projects in attend to find a proper version who work without a browser.
        1.)First i found in your comment version Esp32camFacePremium.zip(its not working, a few errors and constant reeboting esp32), i was asked about this version in one comment, i dont know where.
        2.)”This version will work without the browser: https://robotzero.one/access-control-with-face-recognition/” this your comment and link directing on a first main project, esp32cam with names and websocket, version that dont working when browser close
        3.)I tryed a version for home automation, modified code, put partition, im using 1.05 hardware library. And when i run modified webserver to capture a face, and then we need to upload sketch for automation, but stucked on first webserver, because cant capture face, i dont know why, but code dont use flash memory proper to store faces, only main project esp32camFaceLook with websocket uses flash, store faces, unlocking,locking etc. But this version cant run without browser.

        Its so confusing where to find proper tested version that working without web browser. Can you help with it.

        1. WordBot says:

          Hi, The Premium version should work. I think I always tested with 1.0.4. Make sure you are using the same CameraWebServer example that comes with the hardware library you have installed.

          It’s quite tricky to be able to disconnect and keep the detection running.

          I wish I had more time to work on this all but unless someone wants to sponsor or employ me to do this I have to work on other things.

  45. fernando says:

    I can’t explain my error, my english is so bad, but i’m gonna try. When I go to http link camera server, i don’t see the “streaming” above the “Type the person’s name here” and the camera on the left side doesn’t appear.

    1. WordBot says:

      Do any of the other tutorials work? Does the CameraWebServer example work?

  46. JSosa says:

    Hello, I was trying this sketch but i allways get in the serial “Get one frame in xx ms”, any idea what is the problems. thak you very much.

    1. WordBot says:

      Hi, Is that all you see in serial?

  47. sornchai says:

    I have already used the code offline. but in serial monitor show “Camera capture failed”

    1. WordBot says:

      Does the camera work with the CameraWebServer example?

  48. Peter Tasker says:

    Hi, First many thanks for a first class project, I have it working although it does freeze occasionally.

    Have you ever thought of modifying this project to read and recognize car number plates (licence plates). I would like to do this in order to open gates when a correct number plate is read.

    Thanks again

    Peter

    1. WordBot says:

      Great! I found that the cameras aren’t 100% reliable. Sometimes they go for ages and other times there are crashes and freezes. I think it’s Wi-Fi issues.

      I would love to do object recognition but there’s no libraries or models apart from the face recognition for the ESP32. I imagine Espressif are working on this.

  49. Andreas Mina says:

    Hello I am working with esp32 and arduino connected together. I managed to sucsessfully upload everything and the system is working(the only difference I am using arduino uno , and insted of 3v on relay I use Vcc of esp32). I saved my face 3 times, the camera is recognized me but something is wrong, I do not get the right messages on serial monitor, and my electric lock is not opening.

    14:12:05.831 -> MJPG: 5443B 191ms (5.2fps), AVG: 196ms (5.1fps), 130+53+0+0=184 0
    14:12:06.102 -> MJPG: 5797B 268ms (3.7fps), AVG: 200ms (5.0fps), 131+121+0+0=253 0
    14:12:07.055 -> Match Face ID: 2
    14:12:07.395 -> MJPG: 11956B 1266ms (0.8fps), AVG: 254ms (3.9fps), 131+247+570+141=1089 DETECTED 2
    14:12:08.280 -> No Match Found
    14:12:08.618 -> MJPG: 11648B 1228ms (0.8fps), AVG: 306ms (3.3fps), 131+206+573+143=1054 DETECTED -1
    14:12:09.535 -> Match Face ID: 2
    14:12:09.840 -> MJPG: 13065B 1229ms (0.8fps), AVG: 358ms (2.8fps), 132+209+574+142=1058 DETECTED 2
    14:12:10.687 -> Match Face ID: 2
    14:12:11.057 -> MJPG: 13391B 1231ms (0.8fps), AVG: 410ms (2.4fps), 132+146+573+142=994 DETECTED 2
    14:12:11.904 -> Match Face ID: 2
    14:12:12.309 -> MJPG: 13085B 1227ms (0.8fps), AVG: 461ms (2.2fps), 131+143+572+142=989 DETECTED 2
    14:12:13.191 -> Match Face ID: 2
    14:12:13.531 -> MJPG: 13505B 1227ms (0.8fps), AVG: 513ms (1.9fps), 132+204+572+144=1053 DETECTED 2
    14:12:14.476 -> Match Face ID: 2
    14:12:14.950 -> MJPG: 13297B 1434ms (0.7fps), AVG: 575ms (1.7fps), 133+234+569+143=1081 DETECTED 2
    14:12:15.864 -> Match Face ID: 2
    14:12:16.200 -> MJPG: 13199B 1229ms (0.8fps), AVG: 627ms (1.6fps), 131+205+573+143=1052 DETECTED 2
    14:12:17.118 -> Match Face ID: 2
    14:12:17.424 -> MJPG: 13138B 1227ms (0.8fps), AVG: 679ms (1.5fps), 132+206+574+142=1055 DETECTED 2
    14:12:18.338 -> Match Face ID: 2
    14:12:18.643 -> MJPG: 13041B 1229ms (0.8fps), AVG: 731ms (1.4fps), 132+214+573+143=1064 DETECTED 2
    14:12:19.556 -> Match Face ID: 2
    14:12:19.895 -> MJPG: 13052B 1236ms (0.8fps), AVG: 783ms (1.3fps), 132+197+572+142=1044 DETECTED 2
    14:12:20.776 -> Match Face ID: 2
    14:12:21.114 -> MJPG: 12916B 1239ms (0.8fps), AVG: 835ms (1.2fps), 132+195+572+143=1043 DETECTED 2
    14:12:22.025 -> Match Face ID: 2
    14:12:22.533 -> MJPG: 12960B 1415ms (0.7fps), AVG: 896ms (1.1fps), 133+215+570+142=1062 DETECTED 2
    14:12:22.733 -> MJPG: 4800B 186ms (5.4fps), AVG: 895ms (1.1fps), 126+53+0+0=180 0
    14:12:22.904 -> MJPG: 4615B 191ms (5.2fps), AVG: 894ms (1.1fps), 130+53+0+0=184 0
    14:12:23.108 -> MJPG: 3757B 186ms (5.4fps), AVG: 891ms (1.1fps), 127+53+0+0=180 0
    14:12:23.276 -> MJPG: 3896B 186ms (5.4fps), AVG: 890ms (1.1fps), 127+53+0+0=180 0
    14:12:23.480 -> MJPG: 4086B 186ms (5.4fps), AVG: 890ms (1.1fps), 125+53+0+0=178 0
    14:12:23.685 -> MJPG: 3222B 209ms (4.8fps), AVG: 891ms (1.1fps), 125+77+0+0=203 0
    14:12:23.854 -> MJPG: 3387B 184ms (5.4fps), AVG: 887ms (1.1fps), 123+53+0+0=177 0
    14:12:24.057 -> MJPG: 3451B 186ms (5.4fps), AVG: 833ms (1.2fps), 126+53+0+0=179 0
    14:12:24.259 -> MJPG: 3801B 186ms (5.4fps), AVG: 781ms (1.3fps), 127+53+0+0=180 0
    14:12:24.429 -> MJPG: 3948B 187ms (5.3fps), AVG: 729ms (1.4fps), 128+53+0+0=182 0
    14:12:24.632 -> MJPG: 4008B 186ms (5.4fps), AVG: 676ms (1.5fps), 124+57+0+0=181 0
    14:12:24.799 -> MJPG: 3919B 183ms (5.5fps), AVG: 624ms (1.6fps), 124+54+0+0=179 0
    14:12:24.970 -> MJPG: 4088B 182ms (5.5fps), AVG: 572ms (1.7fps), 124+53+0+0=178 0
    14:12:25.176 -> MJPG: 4005B 185ms (5.4fps), AVG: 509ms (2.0fps), 123+53+0+0=177 0

    1. WordBot says:

      Hi, that looks like the output from the normal CameraWebServer example and not my code?

  50. pieter jonker says:

    Hi,
    If you’re not not faint at heart and know how to use a soldering-iron…….
    As output port you can use the LED port, connected to PIN 9 on the SOC. GPIO-33
    I took out the tiny LED an soldered a thin wire on it… connecting it to a new header pin.
    Now you have an independent output port.

    Next – not absolutely reguired – step
    Disconnect the flash LED by removing R13 (1K) and connect GPIO-33 via a new 1k resistor* to the base of the NPN transistor S8050 so ya can use the FLASH LED independant of the SD-card slot.

    What does this gives you:
    – You keep the full use of external RAM and SD card
    – flash-LED does not blink when you write to SD card
    – External port GPIO-33 for controlling your door opener,

    IF FLASH LED is also connected toGPIO-33 – if door opener is activated, flash LED provides visible signal.

    request: Can anyone pls modify code for using the SD-card as FR mem? than I have transferable ID-codes.

    regards,
    Ptr. (Jonker)

    1. WordBot says:

      Interesting. Could it be used as an input as well?

  51. Andreas Mina says:

    Hey again, I followed every step of your tutorial, I am uploading it and I am getting these. Also, my face is saved after I remove the power and power it again, the messages I am getting seemed to be from your code, I can see messages like “No Match Found” , “Face Not Aligned” , Match Face ID: …

    1. WordBot says:

      Hi, Do you still see : 12:08.618 -> MJPG: 11648B 1228ms (0.8fps), AVG: 306ms (3.3fps), 131+206+573+143=1054 DETECTED -1
      In serial?
      It’s this line in the CameraWebServer example:
      Serial.printf("MJPG: %uB %ums (%.1ffps), AVG: %ums (%.1ffps), %u+%u+%u+%u=%u %s%d\n",
      (uint32_t)(_jpg_buf_len),
      (uint32_t)frame_time, 1000.0 / (uint32_t)frame_time,
      avg_frame_time, 1000.0 / avg_frame_time,
      (uint32_t)ready_time, (uint32_t)face_time, (uint32_t)recognize_time, (uint32_t)encode_time, (uint32_t)process_time,
      (detected)?"DETECTED ":"", face_id
      );

  52. Andreas Mina says:

    No its not in my CameraWebServerPermanent this part of the code.

    1. WordBot says:

      Second tab (app_httpd.cpp) around line 439. It’s there.

  53. Andreas Mina says:

    Hello I’ve realized that something wrong was going on with my saved file for triggering since it was saved as a library (.h) and not as a sketch (.ino). I have changed the file to .ino and now when I try to upload I get this error.

    Multiple libraries were found for “WiFi.h”
    Used: C:\Users\antre\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi
    Not used: C:\Program
    exit status 1
    ‘mtmn_config_t’ does not name a type

    1. WordBot says:

      OK. You don’t need any other files for this just the one .ino file.
      Try replacing:
      mtmn_config_t init_config()
      {
      mtmn_config_t mtmn_config = {0};
      mtmn_config.min_face = 80;
      mtmn_config.pyramid = 0.7;
      mtmn_config.p_threshold.score = 0.6;
      mtmn_config.p_threshold.nms = 0.7;
      mtmn_config.r_threshold.score = 0.7;
      mtmn_config.r_threshold.nms = 0.7;
      mtmn_config.r_threshold.candidate_number = 4;
      mtmn_config.o_threshold.score = 0.7;
      mtmn_config.o_threshold.nms = 0.4;
      mtmn_config.o_threshold.candidate_number = 1;
      return mtmn_config;
      }

      with
      static inline mtmn_config_t app_mtmn_config()
      {
      mtmn_config_t mtmn_config = {0};
      mtmn_config.type = FAST;
      mtmn_config.min_face = 80;
      mtmn_config.pyramid = 0.707;
      mtmn_config.pyramid_times = 4;
      mtmn_config.p_threshold.score = 0.6;
      mtmn_config.p_threshold.nms = 0.7;
      mtmn_config.p_threshold.candidate_number = 20;
      mtmn_config.r_threshold.score = 0.7;
      mtmn_config.r_threshold.nms = 0.7;
      mtmn_config.r_threshold.candidate_number = 10;
      mtmn_config.o_threshold.score = 0.7;
      mtmn_config.o_threshold.nms = 0.7;
      mtmn_config.o_threshold.candidate_number = 1;
      return mtmn_config;
      }
      mtmn_config_t mtmn_config = app_mtmn_config();

  54. Andreas Mina says:

    Yes this part of the file is in app_httpd.cpp

  55. Andreas Mina says:

    by replacing the code I am getting the same error…the triggering file must be .ino or .h? I have it as .ino……I can’t see what the problem is

  56. Andreas Mina says:

    these is the error message,

    Arduino: 1.8.10 (Windows 10), Board: “ESP32 Wrover Module, Face Recognition (2621440 bytes with OTA), QIO, 40MHz, 115200, None”

    Face_Rec:18:8: error: ‘mtmn_config_t’ does not name a type

    static inline mtmn_config_t app_mtmn_config()

    ^

    Face_Rec:41:15: error: redefinition of ‘mtmn_config_t mtmn_config’

    mtmn_config_t mtmn_config = init_config();

    ^

    C:\Users\antre\Documents\CameraWebServerPermanent\Face_Rec.ino:36:15: note: ‘mtmn_config_t mtmn_config’ previously declared here

    mtmn_config_t mtmn_config = app_mtmn_config();

    ^

    Face_Rec:41:41: error: ‘init_config’ was not declared in this scope

    mtmn_config_t mtmn_config = init_config();

    ^

    C:\Users\antre\Documents\CameraWebServerPermanent\Face_Rec.ino: In function ‘void setup()’:

    Face_Rec:44:6: error: redefinition of ‘void setup()’

    void setup() {

    ^

    C:\Users\antre\Documents\CameraWebServerPermanent\CameraWebServerPermanent.ino:24:6: note: ‘void setup()’ previously defined here

    void setup() {

    ^

    C:\Users\antre\Documents\CameraWebServerPermanent\Face_Rec.ino: In function ‘void loop()’:

    Face_Rec:153:6: error: redefinition of ‘void loop()’

    void loop() {

    ^

    C:\Users\antre\Documents\CameraWebServerPermanent\CameraWebServerPermanent.ino:105:6: note: ‘void loop()’ previously defined here

    void loop() {

    ^

    Multiple libraries were found for “WiFi.h”
    Used: C:\Users\antre\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi
    exit status 1
    ‘mtmn_config_t’ does not name a type

    This report would have more information with
    “Show verbose output during compilation”
    option enabled in File -> Preferences.

    1. WordBot says:

      I’ve tested the code on the page and it compiles for me. Here’s another version to try: https://pastebin.com/3sgh3ytj just copy and paste into a new Sketch (File > New)

  57. Rudab says:

    Hello I managed to sucsessfully upload everything and the system was working fine for a week , suddenly two days ago i try to acess the webserver and i got the errors below and i can’t figure what dose it mean .. any help please ?
    I rebuild every thing one more time and the error still there this error will show when i reload the page .

    [D][esp32-hal-psram.c:47] psramInit(): PSRAM enabled

    [D][WiFiGeneric.cpp:336] _eventCallback(): Event: 0 – WIFI_READY
    [D][WiFiGeneric.cpp:336] _eventCallback(): Event: 2 – STA_START
    [D][WiFiGeneric.cpp:336] _eventCallback(): Event: 4 – STA_CONNECTED
    …………..[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 7 – STA_GOT_IP
    [D][WiFiGeneric.cpp:379] _eventCallback(): STA IP: 192.168.1.28, MASK: 255.255.255.0, GW: 192.168.1.1
    .
    WiFi connected
    httpd_start
    Camera Ready! Use ‘http://192.168.1.28’ to connect
    [D][WiFiClient.cpp:482] connected(): Disconnected: RES: 0, ERR: 128
    [E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 104, “Connection reset by peer”
    [E][WiFiClient.cpp:365] write(): fail on fd 60, errno: 104, “Connection reset by peer”
    [E][WiFiClient.cpp:365] write(): fail on fd 61, errno: 104, “Connection reset by peer”
    [E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 104, “Connection reset by peer”

    1. WordBot says:

      Hi, I think that’s the error when the ESP32 thinks the browser has disconnected. I think it happens when the Wi-Fi connection isn’t very good.

  58. Rudab says:

    thank you for the replay . i get a new board and i think i found the problem , every time i delete a face when i reset/restart the esp The list does not look properly arranged .
    here exactly were the code disconnected :
    i still don’t now the solution .
    static esp_err_t send_face_list(WebsocketsClient &client)
    {
    Serial.println(st_face_list.count);
    client.send(“delete_faces”); // tell browser to delete all faces
    face_id_node *head = st_face_list.head;
    char add_face[64];
    for (int i = 0; i id_name);
    Serial.println(add_face);
    client.send(add_face); // add_face the problem
    head = head->next;

    }
    }

    1. WordBot says:

      That code is from a different tutorial. Are you sure you’re not mixing two tutorials up?

  59. Rudab says:

    i feel silly now , i thought i post the reply at ” ESP-WHO Face Recognition with WebSocket Communication” tutorial .

  60. majid says:

    Great work sir,
    but, the function read face id from flash is rebooting esp32 cam again and again, help please.

    assertion “l->size == size” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/components/esp-face/face_recognition/fr_flash.c”, line 85, function: read_face_id_from_flash
    abort() was called at PC 0x400db8eb on core 1

    Backtrace: 0x4008d888:0x3ffb1ed0 0x4008dab5:0x3ffb1ef0 0x400db8eb:0x3ffb1f10 0x400e93f6:0x3ffb1f40 0x400d0d50:0x3ffb1f80 0x400d1f33:0x3ffb1fb0 0x4008a53d:0x3ffb1fd0

    Rebooting…

    1. WordBot says:

      Which version of the ESP32 Hardware library are you using? Did you make the new partition?

    2. Alya says:

      Hi sir, I’m having the same problem. Did you solve this problem? If yes, can you tell me how to solve it?

      1. WordBot says:

        Hi Could be the wrong version the ESP32 hardware library. Try 1.0.5. Let me know if it works.

  61. WATID says:

    I can’t explain my error, my english is so bad, but i’m gonna try. When I go to http link camera server, i don’t see the “streaming” above the “Type the person’s name here” and the camera on the left side doesn’t appear.

    My answer = That’s same error my project (cant show camera Left side ) ,that work is first time (one time) when add 2Face and control access(LED) and… lose Camera.. idont know why // that same solution

    WordBot
    November 24, 2019 at 5:31 pm
    Do any of the other tutorials work? Does the CameraWebServer example work?
    My answer = Yes it’s work form example code

    That’s same error my project .sorry my English is bad

    1. WordBot says:

      Do you see something in the serial monitor?

  62. WATID says:

    no massage
    last TEXT show IP address 198.162……….
    my board ESP32-CAM (Circuit board antenna) ..

  63. majid says:

    thanks for reply, this problem is solved by erasing flash(i used esp32 flash downloader) and then upload codes from start,(enrolling faces and then detection is working fine).
    Door unlock tutorial is working but after a few Unlocks esp32 reboots after showing “Guru Meditation Error” in serial monitor. how it can be solved?

    1. WordBot says:

      I made a change today with the code. Maybe that will fix it? If not do you have the ESP Exception Decoder installed?

  64. zoukeke says:

    [D][esp32-hal-psram.c:47] psramInit(): PSRAM enabled
    [E][camera.c:1049] camera_probe(): Detected camera not supported.
    [E][camera.c:1249] esp_camera_init(): Camera probe failed with error 0x20004
    Camera capture failed
    Camera capture failed

    1. WordBot says:

      Are you using the ESP32-CAM? Does the example CameraWebServer work?

  65. WATID says:

    how to esp32 access control all time ,when I add usher i need to esp auto access control ?
    some time when esp detect face …System freezes how to auto reset program

  66. Mike robinson says:

    Hi there a little help if you have time I have this uploaded have the partition installed it saves the faces during power down no problem works great but when I run the trigger event code up above in the serial monitor all i get is the message (Get one frame in 35 ms ) time fluctuates some but it never detects the face or sets the relay pin high or says anything else in the monitor any help would be amazing I’ve read it multiple times tried it over and over even ran the sketch to delete the faces and tried again still saves the face just somethin with the trigger event code thanks in advance !!!!

    1. WordBot says:

      Hi, It sounds like it’s not recognising faces. Does the CameraWebServer example work? Try using photos from a magazine or mobile and moving closer and further from the camera.

  67. Mike robinson says:

    Yea it works flawlessly with the web server and saves the images after power down also have my face enrolled multiple times so it wouldn’t have any problems the web server recognizes me very quick every single time but the trigger event has not ever recognized me , i have tried close further away lighted room then I upload the web server again and it recognizes me perfect again

    1. WordBot says:

      Are you using the ESP hardware libraries v1.0.4? There’s another script here you can try: https://pastebin.com/3sgh3ytj
      Maybe something has changed somewhere. I’ll try to test this tutorial again tomorrow.

  68. Mike robinson says:

    Ok thanks a million I will try that I think it is the latest version if I downgrade versions what would I have to change just the boards.txt file ? sorry for the multiple replies kept saying duplicate message did t realize it posted them !! Appreciate the help also wouldn’t the serial monitor at least say face not aligned or something in the serial monitor all I get is get one frame !! god bless and stay safe !!!

  69. Mike Robinson says:

    Worked perfect your awesome really appreciate the help

  70. Mike Robinson says:

    So say I had a hotspot and fixed a static up to the web server example with the persistent storage can I add the trigger event code to that web server code to where I can enroll faces on the go and also control the trigger offline without uploading two different sketches ? Just a thought that I would like to accomplish if it is possible !! Also will be sending some eth soon as Coinbase releases my transfer !! Not gonna retire but will show my appreciation for the help !!

  71. Andri says:

    Hello, I try you project error.
    Using library WiFiClientSecure at version 1.0 in folder: C:\Users\Andriasjadi\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.1\libraries\WiFiClientSecure
    exit status 1
    ‘face_id_name_list’ was not declared in this scope
    Can you help me sir…

    1. WordBot says:

      Hi, You need to update your ESP32 Hardware library to 1.0.4.

  72. Amgad Steen says:

    Dear Sir
    very good work, it works for me After modifying partition to work with al-thinker esp32-cam board but the problem it is not saving recognized faces to partition. Each time power on we have to start recognition again.
    I return back to the example from IDE “CameraWebServer” and found the same issue it is not save recognized faces to microsd-card. Each time power on we have to start recognition again.
    So where is the problem ???

    1. WordBot says:

      Hi, The faces are saved to the on-board memory not the SD card. Are you using version 1.0.4 of the ESP32 Hardware libraries?

  73. Amgad Steen says:

    yes I am using version 1.0.4 of the ESP32 and IDE rev 1.8.12 , and I follow your steps exactly but after modifying it for al-thinker esp32-cam board, and the partition appears under board in tools, But it is not saving recognized faces to partition as Each time power on we have to start recognition again.please help ??

  74. Amgad Steen says:

    I try again today and it works fine, many thanks to you.
    Is there some guide to insert the Face_Recognition sketch in main
    webcamserver so that we save faces and detect them in same sketch.

    1. WordBot says:

      Great! Check out this tutorial for an all in one solution – https://robotzero.one/access-control-with-face-recognition/

  75. Amgad Steen says:

    Thank you very much

  76. raj says:

    i want to do interfacing using esp eye using Arduino ide.do u have anything to do on it.

    1. WordBot says:

      You can use an ESP-EYE in most of the projects on the site. I’ve not tried it looks like there are four SPI solder points on the board that you may be able to use as outputs.

  77. Jay M says:

    Awesome tutorial! If all you wanted was to trigger an action if ANY face is detected, how would this change the code?? I.e. I don’t want to match faces, but I do want to know when any face is seen by the camera. Thx!!

    1. WordBot says:

      There’s some more projects here that might help: https://robotzero.one/esp32-camera-projects/ You don’t need an interface for this because you are just doing face detection and not recognition.

  78. Obaid says:

    Hi many thanks for your help but how can I turn on stream on boot of my esp 32 cam so it does not need to be activated all the time when I need to enter in my house(I have to enable stream from its web page every time).

    1. WordBot says:

      Hi, the tutorial you’ve commented on doesn’t use a network connection to run the sketch. The first part, you modify the normal CameraWebServer example to make the face saves permanent. The second part of the tutorial, the sketch runs without a web connection. The other tutorial that is on YouTube as well, there’s a version that keeps running when you disconnect here – https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip I need to work on this some more.

  79. Obaid says:

    Thanks but can you help me with this sketch I need to keep the stream always on upon boot on this one.
    https://easyelectronicsproject.com/esp32-projects/esp32cam-face-recognition-lock/

    1. WordBot says:

      You need some way of switching between the mode where you can capture faces and the mode where it just waits for a face detection. The code above uses two sketches for this but you could probably create one that does the same.. I don’t remember how without looking over it but this code does that – https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip If you want to make it simple you can drop all the network code and just use a button to save a face and use the LEDs to show when the face is successfully captured.

  80. Obaid says:

    I do not know how arduino scripts work so can you kindly tell me how can I start stream on boot on this script.
    https://easyelectronicsproject.com/esp32-projects/esp32cam-face-recognition-lock/
    and will your given script be able to do what this script does
    https://easyelectronicsproject.com/esp32-projects/esp32cam-face-recognition-lock/

    1. WordBot says:

      Sorry, I don’t really have time to fix other people’s projects at the moment but my tutorial above does the same thing but with the advantage that faces are permanently saved so they aren’t lost when the ESP32 loses power.

  81. Obaid says:

    Thank you I understand I will use your script as it does not require an internet connection to recognise faces.

  82. Hamdi says:

    Hello, any idea how to pass the face id manually, like say i want to store the face id (information) somewhere else (server) and upon restart i feed the list with my own face ids. Or at least could let me know, if I already know the face information which variables are needed ?

    1. WordBot says:

      Hi. Do you mean storing the face data? I’m not sure how to get that from the ESP32 storage and move it elsewhere.

  83. imad uddin says:

    Hello sir ……Can i add more than one output

    1. WordBot says:

      Hi, You want to use another pin to control something else? I think io12 might work but I don’t remember exactly which pins you can set high and low.

  84. imad uddin says:

    Tnx sir for repling me …
    Sir iam making door lock system but i want to add another out put …I mean 2nd out put which may be a light bulb ..I want to turn on the 2nd out put with another face

    1. WordBot says:

      Check out the other tutorial here: https://robotzero.one/access-control-with-face-recognition/

      these lines:
      sprintf(recognised_message, “DOOR OPEN FOR %s”, f->id_name);
      open_door(client);
      could be changed to do different things based on the the name of the person. Two different functions.

  85. imad uddin says:

    How can i hard reset esp32

  86. soon says:

    Thank you for your article. I have followed the instructions and have detected the side that turns the Flash on when it gets correct, but why is it correct, it can’t output the signal to control the relay. I tried adjusting delay () but still failed, forwarding the socket to the signal pin then it could not give the LOW signal that I originally set up ().
    Please explain and help me.
    Thank you!

    1. WordBot says:

      I don’t really understand what you mean. Are you trying to connect the relay to the correct pin? Try adding some print statements if you aren’t sure where it fails.

  87. sơn says:

    here is my demo.

    #include “esp_camera.h”
    #include “fd_forward.h”
    #include “fr_forward.h”
    #include “fr_flash.h”
    #define CAMERA_MODEL_AI_THINKER

    //khái báo kết nối relay và đèn
    #define trueled 4
    #define relayPin 12

    unsigned long currentMillis = 0;
    unsigned long openedMillis = 0;
    long interval = 5000; // mở khóa trong …..ms

    // khái báo chân kết nối với esp32_cam
    #define PWDN_GPIO_NUM 32
    #define RESET_GPIO_NUM -1
    #define XCLK_GPIO_NUM 0
    #define SIOD_GPIO_NUM 26
    #define SIOC_GPIO_NUM 27

    #define Y9_GPIO_NUM 35
    #define Y8_GPIO_NUM 34
    #define Y7_GPIO_NUM 39
    #define Y6_GPIO_NUM 36
    #define Y5_GPIO_NUM 21
    #define Y4_GPIO_NUM 19
    #define Y3_GPIO_NUM 18
    #define Y2_GPIO_NUM 5
    #define VSYNC_GPIO_NUM 25
    #define HREF_GPIO_NUM 23
    #define PCLK_GPIO_NUM 22

    #define ENROLL_CONFIRM_TIMES 5
    #define FACE_ID_SAVE_NUMBER 7

    // cấu hình của mô hình MTMN (nhận diện khuôn mặt kích thước ngưỡng ứng viên, kim tự tháp
    static inline mtmn_config_t app_mtmn_config()
    {
    mtmn_config_t mtmn_config = {0};
    mtmn_config.type = FAST;
    mtmn_config.min_face = 80;
    mtmn_config.pyramid = 0.707;
    mtmn_config.pyramid_times = 4;
    mtmn_config.p_threshold.score = 0.6;
    mtmn_config.p_threshold.nms = 0.7;
    mtmn_config.p_threshold.candidate_number = 20;
    mtmn_config.r_threshold.score = 0.7;
    mtmn_config.r_threshold.nms = 0.7;
    mtmn_config.r_threshold.candidate_number = 10;
    mtmn_config.o_threshold.score = 0.7;
    mtmn_config.o_threshold.nms = 0.7;
    mtmn_config.o_threshold.candidate_number = 1;
    return mtmn_config;
    }
    mtmn_config_t mtmn_config = app_mtmn_config();

    // khai báo danh sách các khuôn mặt đc ghi nhớ
    static face_id_list id_list = {0};
    dl_matrix3du_t *image_matrix = NULL;
    camera_fb_t * fb = NULL;

    // gắn ảnh mới đc căn chỉnh vào kích thước của face đc cài đặt
    dl_matrix3du_t *aligned_face = dl_matrix3du_alloc(1, FACE_WIDTH, FACE_HEIGHT, 3);

    //hàm khởi tạo
    void setup() {
    Serial.begin(115200); //tốc độ boaud

    // đặt chế độ ban đầu vào cho chân relay và led
    pinMode(relayPin, OUTPUT);
    pinMode(trueled, OUTPUT);
    pinMode(33 , OUTPUT) ; // hoạt động ở mức thấp

    digitalWrite(relayPin,LOW);

    digitalWrite(trueled, LOW);

    // khởi tạo camera và khai báo chân kết nối với esp
    camera_config_t config;
    config.ledc_channel = LEDC_CHANNEL_0;
    config.ledc_timer = LEDC_TIMER_0;
    config.pin_d0 = Y2_GPIO_NUM;
    config.pin_d1 = Y3_GPIO_NUM;
    config.pin_d2 = Y4_GPIO_NUM;
    config.pin_d3 = Y5_GPIO_NUM;
    config.pin_d4 = Y6_GPIO_NUM;
    config.pin_d5 = Y7_GPIO_NUM;
    config.pin_d6 = Y8_GPIO_NUM;
    config.pin_d7 = Y9_GPIO_NUM;
    config.pin_xclk = XCLK_GPIO_NUM;
    config.pin_pclk = PCLK_GPIO_NUM;
    config.pin_vsync = VSYNC_GPIO_NUM;
    config.pin_href = HREF_GPIO_NUM;
    config.pin_sscb_sda = SIOD_GPIO_NUM;
    config.pin_sscb_scl = SIOC_GPIO_NUM;
    config.pin_pwdn = PWDN_GPIO_NUM;
    config.pin_reset = RESET_GPIO_NUM;
    config.xclk_freq_hz = 20000000; // tần số ban camera sử dụng
    config.pixel_format = PIXFORMAT_JPEG; // khai báo chất lượng ảnh
    config.frame_size = FRAMESIZE_UXGA; // kích thước khung ảnh
    config.jpeg_quality = 10; // khung hình và chất lượng ảnh là jpeg
    config.fb_count = 2;

    //khởi tạo camera
    esp_err_t err = esp_camera_init(&config);
    if (err != ESP_OK) {
    Serial.printf(“Camera init failed with error 0x%x”, err);
    return;
    }

    // giảm kích thước khung hình để có tốc độ cao hơn
    sensor_t * s = esp_camera_sensor_get(); // lưu và cài đặt máy ảnh vào bộ nhớ không thay đổi
    s->set_framesize(s, FRAMESIZE_QVGA); // kích thước khung
    s->set_vflip(s, 1);

    face_id_init(&id_list, FACE_ID_SAVE_NUMBER, ENROLL_CONFIRM_TIMES); // khởi tạo danh sách khuôn mặt đc lưu và đăng kí
    read_face_id_from_flash(&id_list);// đọc dữ liệu khuôn mặt hiện tại từ flash trên bo mạch
    }

    // chương trình để mở khóa cửa và bật đèn led
    void rzoCheckForFace() {
    currentMillis = millis();
    if (run_face_recognition()) {
    // nó trả về giá trị true
    Serial.println(“Face recognised”);
    digitalWrite(relayPin, HIGH); //relay hoạt động
    digitalWrite(trueled, HIGH); // led mở
    openedMillis = millis(); //thời gian đóng lại của relay
    }
    if (currentMillis – interval > openedMillis) {
    // thời gian >5s thì đóng cửa
    digitalWrite(relayPin, LOW); // mở relay
    digitalWrite(trueled, LOW); // tắt Flash
    digitalWrite(33 , LOW); // bật led
    }
    }

    // hàm xác định nhận dạng khuôn mặt
    bool run_face_recognition() {
    bool faceRecognised = false;
    int64_t start_time = esp_timer_get_time();
    fb = esp_camera_fb_get();
    if (!fb) {
    Serial.println(“Camera capture failed”);
    return false;
    }

    int64_t fb_get_time = esp_timer_get_time();
    Serial.printf(“Get one frame in %u ms.\n”, (fb_get_time – start_time) / 1000); // đưa Cam vào khuôn mặt muốn nhận diện

    image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3); //ma trận khuôn mặt phát hiện so sánh kích thước của ảnh
    uint32_t res = fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item);
    if (!res) {
    Serial.println(“to rgb888 failed”);
    dl_matrix3du_free(image_matrix);
    }

    esp_camera_fb_return(fb);

    box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);

    if (net_boxes) {
    // nếu thiết lập camera ok
    if (align_face(net_boxes, image_matrix, aligned_face) == ESP_OK) {

    int matched_id = recognize_face(&id_list, aligned_face); // khai báo biến trong list khuôn mặt
    if (matched_id >= 0) {
    Serial.printf(“Match Face ID: %u\n”, matched_id);
    faceRecognised = true; // nhận diện dúng
    } else {
    Serial.println(“No Match Found”); // khong xác định được khuôn mặt
    matched_id = -1;
    }
    } else {
    Serial.println(“Face Not Aligned”); // khuôn mặt chưa đc căn chỉnh
    }

    free(net_boxes->box);
    free(net_boxes->landmark);
    free(net_boxes);
    }

    dl_matrix3du_free(image_matrix);
    return faceRecognised;
    }

    void loop() {
    rzoCheckForFace();
    delay(1000);
    }

    Trueled works normally and the relay pin when loaded code and running, does not give a signal. I use esp32_cam.. Thank you for your interest.

  88. Lyndon black says:

    Found this really good.
    However tried to rebuild it today and seems that it does not work with the new 1.0.5 ESP32 Cam Board update. HAd to role Arduino IDE back to 1.0.4 version of the board library. Then installed okay

    1. WordBot says:

      Looks like the new 1.0.5 allows uploading the partition with the sketch https://github.com/espressif/arduino-esp32/pull/3328 I’ve created a new partition scheme and added it to the tutorial. I just copied the existing Huge App scheme and replaced the SPIFFS partition with the fr one used to store the faces. If you have a moment can you see if it works for you?

  89. Saphira says:

    Heii
    I’ve got the same problem as some other users here. I get for line 14 the error: “‘mtmn_config_t’ does not name a type”
    I tried this code alone and it worked 🙂 Btw thanks a lot for this cool tutorial. But then I added other parts of code, the CameraWebServer and OTA Update example and now it doesn’t work anymore. I thought it most be some interference with the other code snippets they both worked before putting together. Sadly I don’t know enough about arduino IDE to completely understand what this “static inline” command to know why it isn’t working anymore with the other parts of the code. Could you explain what “static inline mtmn_config_t app_mtmn_config()” is for and what it does?
    Saphira

    1. WordBot says:

      Hi,
      The app_mtmn_config() is to set up the configuration settings for the face recognition. It’s like a array of configuration variables. If you get an error I think it’s because you are using a pre-1.0.4 version of the ESP32 hardware library or you have missed an included library in the top of the Sketch.

      1. Saphira says:

        I am using Version 1.0.5 of the Esp32 hardware library. I did the Update so this tutorial would be easier:) and these are the librarys:
        #include “esp_camera.h”
        #include “fd_forward.h”
        #include “fr_forward.h”
        #include “fr_flash.h”
        Is one missing?

  90. Loc Lee says:

    Thank for your project, this complete my hobby project, to return, I’ve paid you a coffee, enjoy it!

    1. WordBot says:

      I’m happy you found it useful. Thanks for the coffee!

  91. Gerard Sanchez says:

    Hello robotzero, it’s possible to use the other tutorial to save faces with their names and then use this one (or one similar) that displays the names? (For example display the name of the recognized person in a LCD)

  92. erviansyah says:

    i’ve got rebooting on cam wifi
    i using board 1.0.3rc1
    can you help me?

    1. WordBot says:

      Need to use a more recent version of the hardware library

  93. Ivan says:

    Hi Robotzero, I can’t see the code section in “Face Recognition Trigger Event” and some images, I think there’s a problem with your web page 🙁

    1. WordBot says:

      Has it re-appeared? Seems there was a problem with the cache.

      1. Ivan says:

        The images yes, but the code in that section doesn’t. After “Paste the following code into a new sketch” there’s no code ;(

        1. WordBot says:

          Hi. Code is there but maybe you have something blocking it in your browser. You can also download it from here: https://pastebin.com/3sgh3ytj

  94. Aloys Met says:

    Hello sir thank you for the great project you shared with us it was awesome I was challenged while I was creating that partition but I managed to solve that problem and I wanted to Embed GSM so that It can generate notification after locking or unlocking but I was unable to complete that so can you help me solve that by embedding GSM.
    I hope to get your answers as soon as possible sir and again thank you for the great work you done.

    1. WordBot says:

      Hi, Just add you ‘send notification’ code here:
      digitalWrite(relayPin, HIGH); //close (energise) relay
      This is the part that opens the door.

      1. Aloys Met says:

        Does it seem to look like this?

        void open_door(WebsocketsClient &client) {
        if (digitalRead(relay_pin) == LOW) {
        //Set Exact Baud rate of the GSM/GPRS Module.
        Serial.begin(9600);
        Serial.print(“\r”);
        delay(1000);
        Serial.print(“AT+CMGF=1\r”);
        delay(1000);
        Serial.print(“AT+CMGS=\”+250781147055\”\r”);
        delay(1000);
        //The text of the message to be sent.
        Serial.print(“Door is opened sir”);
        delay(1000);
        Serial.write(0x1A);
        delay(1000);
        digitalWrite(relay_pin, HIGH); //close (energise) relay so door unlocks
        Serial.println(“Door Unlocked”);
        client.send(“door_open”);
        door_opened_millis = millis(); // time relay closed and door opened
        }
        is it okay?

        1. Aloys Met says:

          and I wonder if it is possible to add these line after the message

          char recognised_message[64];
          sprintf(recognised_message, “DOOR OPEN FOR %s”, f->id_name);
          open_door(client);
          client.send(recognised_message);
          these code that will mention the one who entered through the door

          1. WordBot says:

            ah ok.. you are looking at this tutorial https://github.com/robotzero1/esp32cam-access-control/blob/master/FaceDoorEntryESP32Cam.ino Should be able to do that. f->id_name is the name of the person recognised. Not sure what type of variable the GSM library expects but you can try.

  95. Judenne Ciervo says:

    Hi, is it possible to recognize faces via sd card?

    1. WordBot says:

      Hi, Do you mean save the faces to an SD card or recognise faces saved on a card? The first one probably but I don’t have the code and it would be slow. Second one is probably easier, you just have to replace the camera code that gets the JPG with some code that reads files off the SD card.

      1. Judenne Ciervo says:

        Yes, can I have a link of the code? I really need it for my research project. thankyou

        1. WordBot says:

          I don’t have code for either of those options. You would need to code it yourself.

  96. Judenne Ciervo says:

    oh, I’m actually a beginner. So I can’t code it myself. thankyou for replying

  97. Mohir says:

    Hello,
    Thank you for this great tutorial,
    it seems things have changed
    in the WebServerCam,
    there is no #include “fr_forward.h”
    and also no variable named left_sample_face
    please advise
    thank you

    1. WordBot says:

      Hi, Yeah, you’ll need to use the versions mentioned (1.0.4 I think) for this to work. I’ve not worked with the 2.0.x library.

  98. Alya says:

    Hi, I tried to add the face id capacity to #define FACE_ID_SAVE_NUMBER 10, the code was successfully uploaded, but on the serial monitor, the esp32 cam continues to reboot showing:

    Wi-Fi connected
    Successfully configured static IP
    assertion “l->size == size” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/components/esp-face/face_recognition/fr_flash.c”, line 85, function: read_face_id_from_flash
    abort() was called at PC 0x40114563 on core 1
    Rebooting…

    Is there a way to solve the problem?

    1. WordBot says:

      Hi which version of the ESP32 hardware library are you using?

  99. Saobaan Adeyemi says:

    hello sir please i am having an error similar to this .. please can you help this is my finalyear project pleasers sir:

    0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
    configsip: 0, SPIWP:0xee
    clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
    mode:DIO, clock div:2
    load:0x3fff0018,len:4
    load:0x3fff001c,len:1216
    ho 0 tail 12 room 4
    load:0x40078000,len:10944
    load:0x40080400,len:6360
    entry 0x400806b4
    E (776) fr_flash: Not found

    1. WordBot says:

      Hi, Which ESP32 Hardware library are you using?

  100. Kamran Babaev says:

    hello, have you upgraded the FaceDoorEntryESP32CAMPremium version? I can’t find it. Help..

  101. Kamran Babaev says:

    Thank you sir for answering. I also checked the premium version but did not see the difference. I want to know why you have to activate the “access control” button when entering the yard.

  102. Kamran Babaev says:

    Thank you sir for answering. I also checked the premium version but did not see the difference. I want to know why you have to activate the “access control” button every time you enter the yard.

  103. Lam Nguyen says:

    hi, i can’t find new part partition scheme after i changed board.txt, do you know why?

    1. WordBot says:

      HI, Which version of the Arduino IDE are you using? Might just need to restart it.

      1. Lam Nguyen says:

        i am using version 1.0.6, i already restarted, but nothing happen

        1. WordBot says:

          I’m not sure if 1.0.6 works. 1.0.5 should be good to go as explained here:

          If you are using 1.0.5 of the Arduino ESP32 hardware libraries you can just add a partitions.csv file to the folder containing the Sketch and this will upload with the Sketch. Note you should choose the ‘Huge App’ partition scheme from the tools menu when uploading.

  104. Niz says:

    Hi
    This work is very great
    but I wonder if it is possible to make the esp32 cam recognize the face directly when it is turned on without entering the page?

    I hope you will respond to me as soon as possible. Thank you in advance

    1. WordBot says:

      Hi, This version works without having to connect to a browser: https://robotzero.one/esp32-face-door-entry/

      1. Niz says:

        I downloaded the code, but face recognition does not start unless you enter the web page

        1. WordBot says:

          Sure you are using this one? https://robotzero.one/esp32-face-door-entry/ There’s no web interface in this version

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

scroll to top