Using face recognition to open a door or control other home automation devices
This tutorial will explain how to save enrolled images in the on-board flash so they survive the ESP32 powering off and use these saved recognitions to control devices connected to the ESP32. There are three steps.
- Create a new partition scheme to enable persistent storage
- Modify the CameraWebServer example sketch to save face data to the new partition
- Use these saved recognitions to control devices connected to the ESP32
Before following this tutorial make sure your camera works by following this tutorial Ai-Thinker ESP32-CAM in the Arduino IDE
Persistent Storage Partition Scheme
A new partition scheme with persistent storage on the on-board flash is needed.
If you are using 1.0.5 of the Arduino ESP32 hardware libraries you can just add a partitions.csv file to the folder containing the Sketch and this will upload with the Sketch. Note you should choose the ‘Huge App’ partition scheme from the tools menu when uploading.
You can download a suitable partition from here: https://robotzero.one/wp-content/uploads/2019/04/partitions.csv
If you are using 1.0.4 of the hardware libraries you can add a new partition scheme to the IDE by downloading a scheme I created from here: https://robotzero.one/wp-content/uploads/2019/04/rzo_partitions.csv
Add this file to the directory containing the other partition schemes. This is found in one of two places, depending on how you installed the Arduino IDE.
Arduino IDE installed from the Windows Store:
C > Users > *your-user-name* > Documents > ArduinoData > packages > esp32 > hardware > esp32 > 1.0.4 > tools > partitions
Arduino IDE installed from the Arduino website:
C > Users > *your-user-name* > AppData > Local > Arduino15 > packages > esp32 > hardware > esp32 > 1.0.4 > tools > partitions
The new scheme has to be added to your ESP device in the boards manager configuration file – boards.txt. Again this is found in one of two places.
Arduino IDE installed from the Windows Store:
C > Users > *your-user-name* > Documents > ArduinoData > packages > esp32 > hardware > esp32 > 1.0.4
Arduino IDE installed from the Arduino website:
C > Users > *your-user-name* > AppData > Local > Arduino15 > packages > esp32 > hardware > esp32 > 1.0.4
Add the following three lines below the existing partitionScheme options for the esp32wrover board in this boards.txt file.
esp32wrover.menu.PartitionScheme.rzo_partition=Face Recognition (2621440 bytes with OTA) esp32wrover.menu.PartitionScheme.rzo_partition.build.partitions=rzo_partitions esp32wrover.menu.PartitionScheme.rzo_partition.upload.maximum_size=2621440
Close and reopen the IDE to confirm the new ‘Face Recognition’ partition scheme is available in the Tools menu.
There’s an article here that explains in much more detail how to set up a new scheme and duplicate a board definition: Partition Schemes in the Arduino IDE
Capture Face Data to Persistent Storage
The CameraWebServer example in the IDE doesn’t save enrolled faces in a way that will survive power loss. To modify it to use the new partition a few changes need to be made to the code.
In the Arduino IDE, make a copy of your working CameraWebServer Sketch from the previous tutorial by saving it with a new file name such as CameraWebServerPermanent.
You should see three tabs in the Arduino IDE similar to the image below:
In the second tab (app_httpd.cpp) make the following changes.
After #include “fr_forward.h” (around line 24) add:
#include "fr_flash.h";
Change int8_t left_sample_face = enroll_face(&id_list, aligned_face);(&id_list, aligned_face); (around line 178) to:
int8_t left_sample_face = enroll_face_id_to_flash(&id_list, aligned_face);
After face_id_init(&id_list, FACE_ID_SAVE_NUMBER, ENROLL_CONFIRM_TIMES); (around line 636) add:
read_face_id_from_flash(&id_list);
Flash and run this Sketch in the same way as before. Enrolled face data is now being saved to the new partition on the flash memory.
Face Recognition Trigger Event
I’ve written a Sketch that sets a pin HIGH on the board while a known face is detected. After 5 seconds pass, if there is no recognised face the pin is set LOW. The code in the function rzoCheckForFace() in the Sketch below can be changed to whatever function you require when a face is recognised. I prefer to set pins as low during setup because mentally I think of it as ‘off’ and then set them high when I want to trigger something because I think of it as ‘on’. You can use pin 12 instead or in addition to pin 2
Paste the following code into a new sketch
When you flash and run this new Sketch you should see ‘Face recognised’ in the serial monitor when a matched face is found.
Opening a Door
The Sketch above combined with a relay or Mosfet module can be used to switch an electrical device on or off. This can be used to open or unlock a door
The diagram below shows the wiring for a opening a lock. From the Sketch above, when a face is recognised, pin IO2 is set HIGH and the relay closes so current flows from the high voltage power supply to the electric door lock.
While setting up and testing a project like this you might prefer to have the serial device connected as below
The door lock in this example could be anything you want to temporarily supply power to.
The project will work with many relay and Mosfet modules that have a 3v input such as the items below. The opto isolated Mosfet module (green connectors) needed the resistor bypassed. The smaller red one works with just the signal and ground connected.
These are available from eBay: Basic Mosfet | Opto Isolated Mosfet | Opto isolated 3v Relay
There’s lots of options for controlling a high voltage device from the ESP32 and it is important to understand the dangers to yourself and the ESP32 if you use the wrong solution. The mechanical relay board is rated for higher voltages but I personally wouldn’t use mains electricity with a device like this.
Deleting Faces from the Memory
If you need to clear the stored faces, paste and upload the Sketch below
#include "esp_camera.h" #include "fd_forward.h" #include "fr_forward.h" #include "fr_flash.h" #define ENROLL_CONFIRM_TIMES 5 #define FACE_ID_SAVE_NUMBER 7 static face_id_list id_list = {0}; void setup() { Serial.begin(115200); face_id_init(&id_list, FACE_ID_SAVE_NUMBER, ENROLL_CONFIRM_TIMES); read_face_id_from_flash(&id_list);// Read current face data from on-board flash Serial.println("Faces Read"); while ( delete_face_id_in_flash(&id_list) > -1 ){ Serial.println("Deleting Face"); } Serial.println("All Deleted"); } void loop() { }
I have some more tutorials like this coming. Feel free to buy me a coffee to help development 😉 .
References
Event trigger for IDF: https://github.com/espressif/esp-who/blob/master/examples/single_chip/recognition_with_command_line/main/app_facenet.c
In-depth relay information: https://www.youtube.com/watch?v=d9evR-K6FAY
Buy Me A Coffee
If you found something useful above please say thanks by buying me a coffee here...
You’ve used IO2 or 12 for the relay control. Aren’t these pins also used for SD card? I’m looking for any module pins that are “free”, seem to be scarce commodity; conflicts with flash LED, SD etc.; idea would be to use an external PIR to save jpeg to SD card, or means to display image remotely. Also would this module work with the OV2640 fisheye camera?
Yeah.. All the pins are assigned to something. Those two seem to be useable for other things but probably not at the same time as using the SD reader. I’ve got a tutorial in progress for writing to the micro-SD or uploading. Not sure about the fisheye.. there’s a few of the little OV2640 cameras and I’m not sure the cables are the same.
Hello, there is no way of sending the face recognition data/trigger via web?
Like a FTT ?
I would like to send a message to Home Assistant, inside it it’s easier to make any action.
Is that possible ?
Here is an exemple:
Individual 1 appears in front of the camera ->
ESP sends package to my arduino IP -> action 1 speaker says “hello individual 1”, action 2 August Smart Lock opens the door.
It’s possible ?
Thanks
Hi, what’s FTT? Rather than making a pin go high or low on the board you want it to send an HTTP request to a device on your network? Yep.. should be possible. I have a Sonoff device arriving soon so I’ll do a tutorial for that. Should be easy enough to change it to point to an Arduino.
EDIT: Sonoff tutorial here: https://robotzero.one/unlock-door-face-detection-sonoff/
Hello, great tutorial, thanks! How can I delete the stored face?
Hi, thanks! I’ve added a Sketch at the end of the article that deletes the stored faces.
Hello
Excellent tutorial, albeit a lot of people were ask the same, would it be possible, instead of change the GPIO state, send a MQTT info about the face recognized?
I think that would entitle to several coffees
Regards
Jose Godinho
Hi, How are you using MQTT in your application?
Hi,
Do you think is it possible to save the face information to SD card so we can recognize other people by replaced the card ? Thank you.
Hi, it’s on my list of things to look into.
hello Sir! after following your instructions I tried to run the given sketch for triggering the relay when a recognized face is detected but on serial I get
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6412
entry 0x400806a8
E (1719) fr_flash: Not found
Hi, Did you follow the whole tutorial including setting up a new partition and adding some faces to the device?
Hi,
have you tested the OTA Update? I cannot get it running? I also got problems to attach an interrupt. Have you tried this?
Thanks a lot.
Cu kami
I’ve not tested OTA but I’m not sure if there’s space on the ESP32-CAM to store the sketch, faces and have space for an OTA partition. Reading the pins is hard, or maybe not even possible if you want to use Wifi https://desire.giesecke.tk/index.php/2018/01/30/adc2-channel-cannot-be-used-when-wifi-is-in-use/ I would love to be proved wrong on this!
Hi, thanks a lot for the answer. In the partition table there is space for ota. I will try it on the weekend.
Can you give me some information about the broken face recognition in 1.0.2?
Thanks a lot.
Cu kami
There’s supposed to be a new Arduino IDE board setup for ESP32 release soon. Hopefully that will fix it.
Hi,
I use your source code, always show me this error,
exit status 1
‘mtmn_config_t’ does not name a type
Have you ever encountered this problem?
Hi, Which version of the ESP32 libraries are you using?
thanks reply, also version1.0.1
have you install other #include ?
maybe #include “mtmn.h” or something else ?
I JUST FIX IT!!!!
It’s the Filename Extension,
I change .ino to .h
which file did you change .ino to .h?
the new sketch I paste from the chapter “Face Recognition Trigger Event”,
I encounter a new problem. relayPin have no response
I didn’t see ‘Face recognised’ in the serial monitor,
I create a new sketch as 123.h, and put it in a file with other sketch(CamerawWebserver.ino, app_http.cpp, camera_index.h),
Is that correct?
sorry for bothering
I figure it out, sorry for stupid question lol
what esp32cam method use for face recognition ?
MobileFace – https://github.com/espressif/esp-who scroll down for details
Olá
Como eu faço para colocar as faces conhecidas no ESP, não entendi esta parte, o restante está configurado e parece estar funcionando.
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8
E (1716) fr_flash: No ID Infomation
Get one frame in 69 ms.
Get one frame in 61 ms.
Get one frame in 56 ms.
Get one frame in 55 ms.
Get one frame in 55 ms.
Hi, Have you saved some faces by following the part Capture Face Data to Persistent Storage?
Ok first of thank you for a great project….I followed everything step by step and everything went without a hitch but there is a problem …. when i go to enroll faces nothing happens ( i am using ESP32 version 1.0.1 on the Arduino IDE)
Of course i am toggling the buttons for facial recognition and detection
on the serial monitor i do get this in the beginning……… E (5628) fr_flash: Not found
What an i doing wrong how can i enroll faces
Thanks again
Denis
Hi, Does the normal CameraWebServer example work for you. Can you enrol a face and it recognise it? If that works try with the edits in this tutorial (including the new partition scheme) and see if you can enrol faces.
No i get that back the original sketch does detect faces i tried it again and it took a while but it did register my face
Denis
Ok so i got it flashed again per your instructions now the face detection works just fine … detection that is….. and it says Intruder Alert when there is no face registered.
When i click on Enroll Face (just once) it starts this wired loop that goes exactly like this: “Note from Sample 4 jumps to Sample 7”
ID(0) Sample 1
ID(0) Sample 2
ID(0) Sample 3
ID(0) Sample 4
ID(0) Sample 7 than it moves to ID (1) Sample 1 though 4 than jumps to 7 than on ID(2) Sample 1 through for jump to 7 up to ID 5 and back again in a loop.
When you stop this process that does not finish by itself by switching off face detection and recognition and you start it back on again it says Intruder Alert like it has not saved any images.
Whats causing this behavior and how do we fix it….by the way with the stock sketch it saves and recognizes the faces just fine
thank you for your help
Denis
I’m not sure why it’s doing that. Are you using the new partition? I wonder if it can’t save the ‘face’.
Is there anything in the serial monitor?
Where is the new partition ??
I am using the downloaded one from the instructions above
https://robotzero.one/wp-content/uploads/2019/04/rzo_partitions.csv
…. is there a new csv??
I am also adding the three lines into board.txt
“`esp32wrover.menu.PartitionScheme.rzo_partition=Face Recognition (2621440 bytes with OTA)
esp32wrover.menu.PartitionScheme.rzo_partition.build.partitions=rzo_partitions
esp32wrover.menu.PartitionScheme.rzo_partition.upload.maximum_size=2621440 “`
One think i have noticed is when the camera reboots it gives the following error
E (52290 Could not Load (or Could not find): fr_flash.h
in the serial monitor other than that it just keeps displaying the data for the captured frames on the serial monitor.
So i think the above error is causing that issue ….what is that??
Denis
> E (52290 Could not Load (or Could not find): fr_flash.h
Is that the exact error? fr_flash.h is a file needed for the sketch to run but it shouldn’t compile if it doesn’t exist.
If the error message mentions fr_flash is missing it’s because the ESP32 can’t find the partition to store the face data. Did you select the new partition similar to this screen capture: https://robotzero.one/wp-content/uploads/2019/04/arduino-ide-custom-partition-scheme.jpg ?
I had the same error (Fr_flash.h missing), you have to select: tools -> Partition scheme -> Face Recognition (xxx Bytes with OTA)
Ok well the Partision Scheme with Face Recognition (xxx Bytes with OTA) seems to be fixing the problem of the fr_flash.h missing and there are no there errors. So it seems like it will work but still i can not test it successfully and can not store a face id yet. Because either the wifi connectivity gets extremely bad when uploading this code or the camera slows down i cant really tell.
When i start the camera feed it will show one or two frames than it wont even get a frame per second so it will loose connectivity the web page with the webserver there is no way to run facial recognition and store face id’s.
I will try to go back to the stock sketch and test the wifi again maybe it has gone crazy but with the stock code the face recognition and the cam was previously working very well.
Anyone else having this issue ??
Denis
me E (5857) fr_flash: Not found
Does the project still work? I think you might see this error when you first start the ESP32 because nothing has been saved yet.
Here is the serial monitor out put …note the fps
Also the wifi access point is like 10 feet away
WiFi connected
E (6595) fr_flash: No ID Infomation
Starting web server on port: ’80’
Starting stream server on port: ’81’
Camera Ready! Use ‘http://192.168.1.17’ to connect
MJPG: 8994B 170ms (5.9fps), AVG: 170ms (5.9fps), 0+0+0+0=0 0
MJPG: 8869B 1341ms (0.7fps), AVG: 755ms (1.3fps), 0+0+0+0=0 0
MJPG: 8892B 4264ms (0.2fps), AVG: 1925ms (0.5fps), 0+0+0+0=0 0
MJPG: 7850B 3524ms (0.3fps), AVG: 2324ms (0.4fps), 0+0+0+0=0 0
MJPG: 5589B 540ms (1.9fps), AVG: 1967ms (0.5fps), 0+0+0+0=0 0
MJPG: 8501B 3638ms (0.3fps), AVG: 2246ms (0.4fps), 0+0+0+0=0 0
MJPG: 10435B 2149ms (0.5fps), AVG: 2232ms (0.4fps), 0+0+0+0=0 0
What does the aerial connector look like on the board? https://robotzero.one/wp-content/uploads/2019/01/esp32-cam-ipex-antenna-jumper.jpg
It looks like Circuit Board Antenna Connected… not the external one ….but i do have an external antenna connected to it should i take it off and see whats up??
You could try but if it’s not connected to anything it shouldn’t affect the signal. If you can it might be better to move the jumper so the external antenna is connected. I found the WiFi really affects the FPS.
Hello!
Great article, I wish you can do a sketch where the face maintenance (add and delete) and detetion are included.
Hi,
You can use this tutorial to manage the faces in the system: https://robotzero.one/esp-who-recognition-with-names You could either switch between the two sketches or take the code from the tutorial above and combine them.
Hello again
I followed everything step by step, double checked and triple checked, re-flashed it a couple of times too…i solved the problem with the wifi dropping frames but still no facial recognition when i use your code and partition after i click on Enroll Face nothing happens and i can confirm that the recognition works with the stock sketch
When i go step by step to your screen shot here https://robotzero.one/wp-content/uploads/2019/04/arduino-ide-custom-partition-scheme.jpg i see that your Board says Robot Zero One is that just for you guys developing cause we do not have that board and the other settings that your board has…..i am of course selecting the partition scheme with Face Recognition (2621440 bytes with OTA)
What should we do next there is no error when you reset the board on the serial monitor other than this:
E (7651) fr_flash: No ID Infomation
Which i guess it means that there is not face id stored yet.
Thanks again
Denis
The ‘Robot Zero One’ board is just a copy of the esp32wrover board. I did it like that in the other tutorial to make it less likely that people would break something. I don’t know why it’s not working for you. I’ll try to find time to make a video this week to show the process.
Hello.
Thank you for your great work.
I’d like to chage the relay module to servo motor because I want to control the door model by servo motor.
If I just chage the module and put the the line which are “#include “, “Servo relayPin;”, could it work?
Hi, I’m adding a general servo tutorial tomorrow. With a servo motor you have to tell it how far to turn. Probably in your case you would add a function to the Sketch that when run, turns the servo enough for the door to open, waits for a certain time and then turns the servo back to close the door.
Thank you for your reply.
I changed your code and completed my code.
However, the motor doesn’t work.
I connected the motor to GPIO2 pin of esp 32 cam and supplied the external power supply to motor by battery 5V.
I guessed that very small output signal is given to GPIO2 because I coudn’t hear any begining sound from motor.
How can I solve this problem?
Below is my partial code.
void rzoCheckForFace() {
currentMillis = millis();
if (run_face_recognition()) { // face recognition function has returned true
Serial.println(“Face recognised”);
// digitalWrite(servoPin, HIGH);
servo.write(170);
Serial.println(“Motor Moved”);
delay(30000);
openedMillis = millis(); //time relay closed
}
if (currentMillis – interval > openedMillis){ // current time – face recognised time > 5 secs
// digitalWrite(servoPin, LOW);
servo.write(0);
delay(300);
}
}
In addition, I add initial state of motor in void setup() part.
void setup() {
Serial.begin(115200);
servo.attach(servoPin);
servo.write(0);
….
Are you using the normal Arduino servo library? I’m not sure it will work. See this tutorial for servo code: https://robotzero.one/pan-and-tilt-control-for-an-esp32-cam/
Does anybody know how I can augment this code to trigger a dfplayer (a cheap broken out mp3 player,) by sending a hexadecimal number command to serial tx/rx? (That’s how you play certain audio files from the onboard sd-card.)
I am trying to make a smart(ish) doorbell system that would recognize houshold members and greet them, by name, upon ringing the doorbell (or in this case activating a momentary button switch) which would first check the database for a saved face, play a file associated with said person, then ring a bell by way of either closing a relay or playing a different file (like a sample of Big Ben for instance) to a speaker on the inside of the door.)
A visitor without a saved face would have a file played implying an unknown person is calling. I would also really like to be able to text a live snapshot with Telegram. Having Alexa chime in (pun intended, …sadly,) would also be pretty cool and in lieu of using an actual doorbell.
Let me know on the video or if you have time please let me send you the cam and a prepaid return envelope and you can trouble shoot it hands on ….i am desperate and i really need to get this project complete as it fits my situation perfectly
Thank you i have tried everything and the board just wont recognize faces after flashed with your sketch
Hello! Please write a sketch for the tutorial: https://robotzero.one/esp-who-recognition-with-names. I can treat you to coffee. I tried to redo the sketch to control the relay, but I have little experience and nothing worked.
Hi, I have a tutorial video for access control with face recognition nearly ready (a combination of this tutorial and the one with names). If you subscribe to my Youtube channel https://www.youtube.com/channel/UCpqVEutU_Jcq34ws4tyUReQ you should get notified when it’s live. There will be a sketch to download.
Tutorial ready…
https://youtu.be/mu3-Sff0B9w
https://robotzero.one/access-control-with-face-recognition/
Oh no! I meant that the sketch was without WEB Cockets, without WEB server, if the connection is lost, the program does not work. We need a fully autonomous operation of the module. Please learn how to disable WEB Cockets.
Would you be able to use one sketch to capture the faces and then upload another to run autonomously? It’s tricky to have it running in a loop waiting for new faces and then switch out of this to only be in access control mode. This tutorial works without being connected to a browser: https://robotzero.one/face-recognition-with-blynk/ but you have to save faces using another sketch.
I would like to remake the sketch from this lesson so that it can read the names recorded on the flash memory from the sketch “FaceDoorEntryESP32Cam”. Can you help me with this?
I’m not really sure what you want to do. Can you explain the full process please?
Hello! I apologize for the poor translation. The point is this:
1. Download the “FaceDoorEntryESP32Cam” sketch to the ESP32 board.
2. Write the names in memory.
3. Fix the sketch “ESP32-CAM Face Recognition for Home Automation” so that it can read the names recorded in the previous paragraph.
4. Download the fixed sketch “ESP32-
CAM Face Recognition for Home Automation. “
Can you tell me what you want you want the final project to do? When you say ‘read’ do you mean say the names through a speaker?
Hello Friend!!! I do not need a voice alert. I will try to explain in another way. If I upload the “FaceDoorEntryESP32Cam” sketch to ESP32 and write down the names to search (in the area fr 0x290000, 0xEF000). Then I will upload the “ESP32-CAM Face Recognition for Home Automation” sketch to ESP32. Will it work ???
Ok I think I understand. I don’t think it will work because https://robotzero.one/esp32-face-door-entry/ has the function read_face_id_from_flash but https://robotzero.one/esp-who-recognition-with-names/ has read_face_id_from_flash_with_name So the data is probably in incompatible formats. If you don’t need names (I don’t think you do?) you can always just save them without names as in the https://robotzero.one/esp32-face-door-entry/ tutorial in the ‘Capture Face Data to Persistent Storage’ section. The one without names works better because the ESP libraries for 1.0.1 has less bugs.
Hello! Yes, I have already encountered the fact that the names have incompatible formats. My question is about the Access Control with Face Recognition project. Is it possible to change the void loop () function in a sketch so that it would be possible to search for faces without connecting to a WEB server?
Yep but I don’t have time to code it at the moment. Why not just use the Home Automation tutorial? It works without a web server.
Hello, first thank you, it is a great tutorial, question,
how many faces can be enrolled?
thank you.
Hi, I thought there was a limit of around 7 (this code: #define FACE_ID_SAVE_NUMBER 7) but maybe not… https://github.com/espressif/esp-who/issues/118
Ok, thanks a lot for the information.
Greetings is a great project.
hi
i want to ask why i get this message after delete the face
Camera Ready! Use ‘http://192.168.43.244’ to connect
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: -1, ERR: 104
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: -1, ERR: 104
[E][WiFiClient.cpp:392] write(): fail on fd 62, errno: 104, “Connection reset by peer”
[E][WiFiClient.cpp:392] write(): fail on fd 63, errno: 104, “Connection reset by peer”
this error will show when i reload the page
Hi, This is the Wi-FI library. It can be quite glitchy when streaming video on the ESP32-CAM. Does it delete the face from the system Ok?
assertion “l->size == size” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/components/esp-face/face_recognition/fr_flash.c”, line 85, function: read_face_id_from_flash
abort() was called at PC 0x400db8eb on core 1
Backtrace: 0x4008d888:0x3ffb1ed0 0x4008dab5:0x3ffb1ef0 0x400db8eb:0x3ffb1f10 0x400e93f6:0x3ffb1f40 0x400d0d50:0x3ffb1f80 0x400d1f33:0x3ffb1fb0 0x4008a53d:0x3ffb1fd0
i got this error when trying to delete face
if i upload the example code for camera it working find without any error
Are you using the code from the section above titled “Deleting Faces from the Memory” to delete the faces? Have you saved some faces successfully?
hi how to get it work without oppening the url everytime?
This version will work without the browser: https://robotzero.one/access-control-with-face-recognition/
Hi, im searching for all comments on all your projects in attend to find a proper version who work without a browser.
1.)First i found in your comment version Esp32camFacePremium.zip(its not working, a few errors and constant reeboting esp32), i was asked about this version in one comment, i dont know where.
2.)”This version will work without the browser: https://robotzero.one/access-control-with-face-recognition/” this your comment and link directing on a first main project, esp32cam with names and websocket, version that dont working when browser close
3.)I tryed a version for home automation, modified code, put partition, im using 1.05 hardware library. And when i run modified webserver to capture a face, and then we need to upload sketch for automation, but stucked on first webserver, because cant capture face, i dont know why, but code dont use flash memory proper to store faces, only main project esp32camFaceLook with websocket uses flash, store faces, unlocking,locking etc. But this version cant run without browser.
Its so confusing where to find proper tested version that working without web browser. Can you help with it.
Hi, The Premium version should work. I think I always tested with 1.0.4. Make sure you are using the same CameraWebServer example that comes with the hardware library you have installed.
It’s quite tricky to be able to disconnect and keep the detection running.
I wish I had more time to work on this all but unless someone wants to sponsor or employ me to do this I have to work on other things.
I can’t explain my error, my english is so bad, but i’m gonna try. When I go to http link camera server, i don’t see the “streaming” above the “Type the person’s name here” and the camera on the left side doesn’t appear.
Do any of the other tutorials work? Does the CameraWebServer example work?
Hello, I was trying this sketch but i allways get in the serial “Get one frame in xx ms”, any idea what is the problems. thak you very much.
Hi, Is that all you see in serial?
I have already used the code offline. but in serial monitor show “Camera capture failed”
Does the camera work with the CameraWebServer example?
Hi, First many thanks for a first class project, I have it working although it does freeze occasionally.
Have you ever thought of modifying this project to read and recognize car number plates (licence plates). I would like to do this in order to open gates when a correct number plate is read.
Thanks again
Peter
Great! I found that the cameras aren’t 100% reliable. Sometimes they go for ages and other times there are crashes and freezes. I think it’s Wi-Fi issues.
I would love to do object recognition but there’s no libraries or models apart from the face recognition for the ESP32. I imagine Espressif are working on this.
Hello I am working with esp32 and arduino connected together. I managed to sucsessfully upload everything and the system is working(the only difference I am using arduino uno , and insted of 3v on relay I use Vcc of esp32). I saved my face 3 times, the camera is recognized me but something is wrong, I do not get the right messages on serial monitor, and my electric lock is not opening.
14:12:05.831 -> MJPG: 5443B 191ms (5.2fps), AVG: 196ms (5.1fps), 130+53+0+0=184 0
14:12:06.102 -> MJPG: 5797B 268ms (3.7fps), AVG: 200ms (5.0fps), 131+121+0+0=253 0
14:12:07.055 -> Match Face ID: 2
14:12:07.395 -> MJPG: 11956B 1266ms (0.8fps), AVG: 254ms (3.9fps), 131+247+570+141=1089 DETECTED 2
14:12:08.280 -> No Match Found
14:12:08.618 -> MJPG: 11648B 1228ms (0.8fps), AVG: 306ms (3.3fps), 131+206+573+143=1054 DETECTED -1
14:12:09.535 -> Match Face ID: 2
14:12:09.840 -> MJPG: 13065B 1229ms (0.8fps), AVG: 358ms (2.8fps), 132+209+574+142=1058 DETECTED 2
14:12:10.687 -> Match Face ID: 2
14:12:11.057 -> MJPG: 13391B 1231ms (0.8fps), AVG: 410ms (2.4fps), 132+146+573+142=994 DETECTED 2
14:12:11.904 -> Match Face ID: 2
14:12:12.309 -> MJPG: 13085B 1227ms (0.8fps), AVG: 461ms (2.2fps), 131+143+572+142=989 DETECTED 2
14:12:13.191 -> Match Face ID: 2
14:12:13.531 -> MJPG: 13505B 1227ms (0.8fps), AVG: 513ms (1.9fps), 132+204+572+144=1053 DETECTED 2
14:12:14.476 -> Match Face ID: 2
14:12:14.950 -> MJPG: 13297B 1434ms (0.7fps), AVG: 575ms (1.7fps), 133+234+569+143=1081 DETECTED 2
14:12:15.864 -> Match Face ID: 2
14:12:16.200 -> MJPG: 13199B 1229ms (0.8fps), AVG: 627ms (1.6fps), 131+205+573+143=1052 DETECTED 2
14:12:17.118 -> Match Face ID: 2
14:12:17.424 -> MJPG: 13138B 1227ms (0.8fps), AVG: 679ms (1.5fps), 132+206+574+142=1055 DETECTED 2
14:12:18.338 -> Match Face ID: 2
14:12:18.643 -> MJPG: 13041B 1229ms (0.8fps), AVG: 731ms (1.4fps), 132+214+573+143=1064 DETECTED 2
14:12:19.556 -> Match Face ID: 2
14:12:19.895 -> MJPG: 13052B 1236ms (0.8fps), AVG: 783ms (1.3fps), 132+197+572+142=1044 DETECTED 2
14:12:20.776 -> Match Face ID: 2
14:12:21.114 -> MJPG: 12916B 1239ms (0.8fps), AVG: 835ms (1.2fps), 132+195+572+143=1043 DETECTED 2
14:12:22.025 -> Match Face ID: 2
14:12:22.533 -> MJPG: 12960B 1415ms (0.7fps), AVG: 896ms (1.1fps), 133+215+570+142=1062 DETECTED 2
14:12:22.733 -> MJPG: 4800B 186ms (5.4fps), AVG: 895ms (1.1fps), 126+53+0+0=180 0
14:12:22.904 -> MJPG: 4615B 191ms (5.2fps), AVG: 894ms (1.1fps), 130+53+0+0=184 0
14:12:23.108 -> MJPG: 3757B 186ms (5.4fps), AVG: 891ms (1.1fps), 127+53+0+0=180 0
14:12:23.276 -> MJPG: 3896B 186ms (5.4fps), AVG: 890ms (1.1fps), 127+53+0+0=180 0
14:12:23.480 -> MJPG: 4086B 186ms (5.4fps), AVG: 890ms (1.1fps), 125+53+0+0=178 0
14:12:23.685 -> MJPG: 3222B 209ms (4.8fps), AVG: 891ms (1.1fps), 125+77+0+0=203 0
14:12:23.854 -> MJPG: 3387B 184ms (5.4fps), AVG: 887ms (1.1fps), 123+53+0+0=177 0
14:12:24.057 -> MJPG: 3451B 186ms (5.4fps), AVG: 833ms (1.2fps), 126+53+0+0=179 0
14:12:24.259 -> MJPG: 3801B 186ms (5.4fps), AVG: 781ms (1.3fps), 127+53+0+0=180 0
14:12:24.429 -> MJPG: 3948B 187ms (5.3fps), AVG: 729ms (1.4fps), 128+53+0+0=182 0
14:12:24.632 -> MJPG: 4008B 186ms (5.4fps), AVG: 676ms (1.5fps), 124+57+0+0=181 0
14:12:24.799 -> MJPG: 3919B 183ms (5.5fps), AVG: 624ms (1.6fps), 124+54+0+0=179 0
14:12:24.970 -> MJPG: 4088B 182ms (5.5fps), AVG: 572ms (1.7fps), 124+53+0+0=178 0
14:12:25.176 -> MJPG: 4005B 185ms (5.4fps), AVG: 509ms (2.0fps), 123+53+0+0=177 0
Hi, that looks like the output from the normal CameraWebServer example and not my code?
Hi,
If you’re not not faint at heart and know how to use a soldering-iron…….
As output port you can use the LED port, connected to PIN 9 on the SOC. GPIO-33
I took out the tiny LED an soldered a thin wire on it… connecting it to a new header pin.
Now you have an independent output port.
Next – not absolutely reguired – step
Disconnect the flash LED by removing R13 (1K) and connect GPIO-33 via a new 1k resistor* to the base of the NPN transistor S8050 so ya can use the FLASH LED independant of the SD-card slot.
What does this gives you:
– You keep the full use of external RAM and SD card
– flash-LED does not blink when you write to SD card
– External port GPIO-33 for controlling your door opener,
IF FLASH LED is also connected toGPIO-33 – if door opener is activated, flash LED provides visible signal.
request: Can anyone pls modify code for using the SD-card as FR mem? than I have transferable ID-codes.
regards,
Ptr. (Jonker)
Interesting. Could it be used as an input as well?
Hey again, I followed every step of your tutorial, I am uploading it and I am getting these. Also, my face is saved after I remove the power and power it again, the messages I am getting seemed to be from your code, I can see messages like “No Match Found” , “Face Not Aligned” , Match Face ID: …
Hi, Do you still see : 12:08.618 -> MJPG: 11648B 1228ms (0.8fps), AVG: 306ms (3.3fps), 131+206+573+143=1054 DETECTED -1
In serial?
It’s this line in the CameraWebServer example:
Serial.printf("MJPG: %uB %ums (%.1ffps), AVG: %ums (%.1ffps), %u+%u+%u+%u=%u %s%d\n",
(uint32_t)(_jpg_buf_len),
(uint32_t)frame_time, 1000.0 / (uint32_t)frame_time,
avg_frame_time, 1000.0 / avg_frame_time,
(uint32_t)ready_time, (uint32_t)face_time, (uint32_t)recognize_time, (uint32_t)encode_time, (uint32_t)process_time,
(detected)?"DETECTED ":"", face_id
);
No its not in my CameraWebServerPermanent this part of the code.
Second tab (app_httpd.cpp) around line 439. It’s there.
Hello I’ve realized that something wrong was going on with my saved file for triggering since it was saved as a library (.h) and not as a sketch (.ino). I have changed the file to .ino and now when I try to upload I get this error.
Multiple libraries were found for “WiFi.h”
Used: C:\Users\antre\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi
Not used: C:\Program
exit status 1
‘mtmn_config_t’ does not name a type
OK. You don’t need any other files for this just the one .ino file.
Try replacing:
mtmn_config_t init_config()
{
mtmn_config_t mtmn_config = {0};
mtmn_config.min_face = 80;
mtmn_config.pyramid = 0.7;
mtmn_config.p_threshold.score = 0.6;
mtmn_config.p_threshold.nms = 0.7;
mtmn_config.r_threshold.score = 0.7;
mtmn_config.r_threshold.nms = 0.7;
mtmn_config.r_threshold.candidate_number = 4;
mtmn_config.o_threshold.score = 0.7;
mtmn_config.o_threshold.nms = 0.4;
mtmn_config.o_threshold.candidate_number = 1;
return mtmn_config;
}
with
static inline mtmn_config_t app_mtmn_config()
{
mtmn_config_t mtmn_config = {0};
mtmn_config.type = FAST;
mtmn_config.min_face = 80;
mtmn_config.pyramid = 0.707;
mtmn_config.pyramid_times = 4;
mtmn_config.p_threshold.score = 0.6;
mtmn_config.p_threshold.nms = 0.7;
mtmn_config.p_threshold.candidate_number = 20;
mtmn_config.r_threshold.score = 0.7;
mtmn_config.r_threshold.nms = 0.7;
mtmn_config.r_threshold.candidate_number = 10;
mtmn_config.o_threshold.score = 0.7;
mtmn_config.o_threshold.nms = 0.7;
mtmn_config.o_threshold.candidate_number = 1;
return mtmn_config;
}
mtmn_config_t mtmn_config = app_mtmn_config();
Yes this part of the file is in app_httpd.cpp
by replacing the code I am getting the same error…the triggering file must be .ino or .h? I have it as .ino……I can’t see what the problem is
these is the error message,
Arduino: 1.8.10 (Windows 10), Board: “ESP32 Wrover Module, Face Recognition (2621440 bytes with OTA), QIO, 40MHz, 115200, None”
Face_Rec:18:8: error: ‘mtmn_config_t’ does not name a type
static inline mtmn_config_t app_mtmn_config()
^
Face_Rec:41:15: error: redefinition of ‘mtmn_config_t mtmn_config’
mtmn_config_t mtmn_config = init_config();
^
C:\Users\antre\Documents\CameraWebServerPermanent\Face_Rec.ino:36:15: note: ‘mtmn_config_t mtmn_config’ previously declared here
mtmn_config_t mtmn_config = app_mtmn_config();
^
Face_Rec:41:41: error: ‘init_config’ was not declared in this scope
mtmn_config_t mtmn_config = init_config();
^
C:\Users\antre\Documents\CameraWebServerPermanent\Face_Rec.ino: In function ‘void setup()’:
Face_Rec:44:6: error: redefinition of ‘void setup()’
void setup() {
^
C:\Users\antre\Documents\CameraWebServerPermanent\CameraWebServerPermanent.ino:24:6: note: ‘void setup()’ previously defined here
void setup() {
^
C:\Users\antre\Documents\CameraWebServerPermanent\Face_Rec.ino: In function ‘void loop()’:
Face_Rec:153:6: error: redefinition of ‘void loop()’
void loop() {
^
C:\Users\antre\Documents\CameraWebServerPermanent\CameraWebServerPermanent.ino:105:6: note: ‘void loop()’ previously defined here
void loop() {
^
Multiple libraries were found for “WiFi.h”
Used: C:\Users\antre\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi
exit status 1
‘mtmn_config_t’ does not name a type
This report would have more information with
“Show verbose output during compilation”
option enabled in File -> Preferences.
I’ve tested the code on the page and it compiles for me. Here’s another version to try: https://pastebin.com/3sgh3ytj just copy and paste into a new Sketch (File > New)
Hello I managed to sucsessfully upload everything and the system was working fine for a week , suddenly two days ago i try to acess the webserver and i got the errors below and i can’t figure what dose it mean .. any help please ?
I rebuild every thing one more time and the error still there this error will show when i reload the page .
[D][esp32-hal-psram.c:47] psramInit(): PSRAM enabled
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 0 – WIFI_READY
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 2 – STA_START
[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 4 – STA_CONNECTED
…………..[D][WiFiGeneric.cpp:336] _eventCallback(): Event: 7 – STA_GOT_IP
[D][WiFiGeneric.cpp:379] _eventCallback(): STA IP: 192.168.1.28, MASK: 255.255.255.0, GW: 192.168.1.1
.
WiFi connected
httpd_start
Camera Ready! Use ‘http://192.168.1.28’ to connect
[D][WiFiClient.cpp:482] connected(): Disconnected: RES: 0, ERR: 128
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 104, “Connection reset by peer”
[E][WiFiClient.cpp:365] write(): fail on fd 60, errno: 104, “Connection reset by peer”
[E][WiFiClient.cpp:365] write(): fail on fd 61, errno: 104, “Connection reset by peer”
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 104, “Connection reset by peer”
Hi, I think that’s the error when the ESP32 thinks the browser has disconnected. I think it happens when the Wi-Fi connection isn’t very good.
thank you for the replay . i get a new board and i think i found the problem , every time i delete a face when i reset/restart the esp The list does not look properly arranged .
here exactly were the code disconnected :
i still don’t now the solution .
static esp_err_t send_face_list(WebsocketsClient &client)
{
Serial.println(st_face_list.count);
client.send(“delete_faces”); // tell browser to delete all faces
face_id_node *head = st_face_list.head;
char add_face[64];
for (int i = 0; i id_name);
Serial.println(add_face);
client.send(add_face); // add_face the problem
head = head->next;
}
}
That code is from a different tutorial. Are you sure you’re not mixing two tutorials up?
i feel silly now , i thought i post the reply at ” ESP-WHO Face Recognition with WebSocket Communication” tutorial .
Great work sir,
but, the function read face id from flash is rebooting esp32 cam again and again, help please.
assertion “l->size == size” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/components/esp-face/face_recognition/fr_flash.c”, line 85, function: read_face_id_from_flash
abort() was called at PC 0x400db8eb on core 1
Backtrace: 0x4008d888:0x3ffb1ed0 0x4008dab5:0x3ffb1ef0 0x400db8eb:0x3ffb1f10 0x400e93f6:0x3ffb1f40 0x400d0d50:0x3ffb1f80 0x400d1f33:0x3ffb1fb0 0x4008a53d:0x3ffb1fd0
Rebooting…
Which version of the ESP32 Hardware library are you using? Did you make the new partition?
Hi sir, I’m having the same problem. Did you solve this problem? If yes, can you tell me how to solve it?
Hi Could be the wrong version the ESP32 hardware library. Try 1.0.5. Let me know if it works.
I can’t explain my error, my english is so bad, but i’m gonna try. When I go to http link camera server, i don’t see the “streaming” above the “Type the person’s name here” and the camera on the left side doesn’t appear.
My answer = That’s same error my project (cant show camera Left side ) ,that work is first time (one time) when add 2Face and control access(LED) and… lose Camera.. idont know why // that same solution
WordBot
November 24, 2019 at 5:31 pm
Do any of the other tutorials work? Does the CameraWebServer example work?
My answer = Yes it’s work form example code
That’s same error my project .sorry my English is bad
Do you see something in the serial monitor?
no massage
last TEXT show IP address 198.162……….
my board ESP32-CAM (Circuit board antenna) ..
thanks for reply, this problem is solved by erasing flash(i used esp32 flash downloader) and then upload codes from start,(enrolling faces and then detection is working fine).
Door unlock tutorial is working but after a few Unlocks esp32 reboots after showing “Guru Meditation Error” in serial monitor. how it can be solved?
I made a change today with the code. Maybe that will fix it? If not do you have the ESP Exception Decoder installed?
[D][esp32-hal-psram.c:47] psramInit(): PSRAM enabled
[E][camera.c:1049] camera_probe(): Detected camera not supported.
[E][camera.c:1249] esp_camera_init(): Camera probe failed with error 0x20004
Camera capture failed
Camera capture failed
Are you using the ESP32-CAM? Does the example CameraWebServer work?
how to esp32 access control all time ,when I add usher i need to esp auto access control ?
some time when esp detect face …System freezes how to auto reset program
Hi there a little help if you have time I have this uploaded have the partition installed it saves the faces during power down no problem works great but when I run the trigger event code up above in the serial monitor all i get is the message (Get one frame in 35 ms ) time fluctuates some but it never detects the face or sets the relay pin high or says anything else in the monitor any help would be amazing I’ve read it multiple times tried it over and over even ran the sketch to delete the faces and tried again still saves the face just somethin with the trigger event code thanks in advance !!!!
Hi, It sounds like it’s not recognising faces. Does the CameraWebServer example work? Try using photos from a magazine or mobile and moving closer and further from the camera.
Yea it works flawlessly with the web server and saves the images after power down also have my face enrolled multiple times so it wouldn’t have any problems the web server recognizes me very quick every single time but the trigger event has not ever recognized me , i have tried close further away lighted room then I upload the web server again and it recognizes me perfect again
Are you using the ESP hardware libraries v1.0.4? There’s another script here you can try: https://pastebin.com/3sgh3ytj
Maybe something has changed somewhere. I’ll try to test this tutorial again tomorrow.
Ok thanks a million I will try that I think it is the latest version if I downgrade versions what would I have to change just the boards.txt file ? sorry for the multiple replies kept saying duplicate message did t realize it posted them !! Appreciate the help also wouldn’t the serial monitor at least say face not aligned or something in the serial monitor all I get is get one frame !! god bless and stay safe !!!
Worked perfect your awesome really appreciate the help
So say I had a hotspot and fixed a static up to the web server example with the persistent storage can I add the trigger event code to that web server code to where I can enroll faces on the go and also control the trigger offline without uploading two different sketches ? Just a thought that I would like to accomplish if it is possible !! Also will be sending some eth soon as Coinbase releases my transfer !! Not gonna retire but will show my appreciation for the help !!
Hello, I try you project error.
Using library WiFiClientSecure at version 1.0 in folder: C:\Users\Andriasjadi\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.1\libraries\WiFiClientSecure
exit status 1
‘face_id_name_list’ was not declared in this scope
Can you help me sir…
Hi, You need to update your ESP32 Hardware library to 1.0.4.
Dear Sir
very good work, it works for me After modifying partition to work with al-thinker esp32-cam board but the problem it is not saving recognized faces to partition. Each time power on we have to start recognition again.
I return back to the example from IDE “CameraWebServer” and found the same issue it is not save recognized faces to microsd-card. Each time power on we have to start recognition again.
So where is the problem ???
Hi, The faces are saved to the on-board memory not the SD card. Are you using version 1.0.4 of the ESP32 Hardware libraries?
yes I am using version 1.0.4 of the ESP32 and IDE rev 1.8.12 , and I follow your steps exactly but after modifying it for al-thinker esp32-cam board, and the partition appears under board in tools, But it is not saving recognized faces to partition as Each time power on we have to start recognition again.please help ??
I try again today and it works fine, many thanks to you.
Is there some guide to insert the Face_Recognition sketch in main
webcamserver so that we save faces and detect them in same sketch.
Great! Check out this tutorial for an all in one solution – https://robotzero.one/access-control-with-face-recognition/
Thank you very much
i want to do interfacing using esp eye using Arduino ide.do u have anything to do on it.
You can use an ESP-EYE in most of the projects on the site. I’ve not tried it looks like there are four SPI solder points on the board that you may be able to use as outputs.
Awesome tutorial! If all you wanted was to trigger an action if ANY face is detected, how would this change the code?? I.e. I don’t want to match faces, but I do want to know when any face is seen by the camera. Thx!!
There’s some more projects here that might help: https://robotzero.one/esp32-camera-projects/ You don’t need an interface for this because you are just doing face detection and not recognition.
Hi many thanks for your help but how can I turn on stream on boot of my esp 32 cam so it does not need to be activated all the time when I need to enter in my house(I have to enable stream from its web page every time).
Hi, the tutorial you’ve commented on doesn’t use a network connection to run the sketch. The first part, you modify the normal CameraWebServer example to make the face saves permanent. The second part of the tutorial, the sketch runs without a web connection. The other tutorial that is on YouTube as well, there’s a version that keeps running when you disconnect here – https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip I need to work on this some more.
Thanks but can you help me with this sketch I need to keep the stream always on upon boot on this one.
https://easyelectronicsproject.com/esp32-projects/esp32cam-face-recognition-lock/
You need some way of switching between the mode where you can capture faces and the mode where it just waits for a face detection. The code above uses two sketches for this but you could probably create one that does the same.. I don’t remember how without looking over it but this code does that – https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip If you want to make it simple you can drop all the network code and just use a button to save a face and use the LEDs to show when the face is successfully captured.
I do not know how arduino scripts work so can you kindly tell me how can I start stream on boot on this script.
https://easyelectronicsproject.com/esp32-projects/esp32cam-face-recognition-lock/
and will your given script be able to do what this script does
https://easyelectronicsproject.com/esp32-projects/esp32cam-face-recognition-lock/
Sorry, I don’t really have time to fix other people’s projects at the moment but my tutorial above does the same thing but with the advantage that faces are permanently saved so they aren’t lost when the ESP32 loses power.
Thank you I understand I will use your script as it does not require an internet connection to recognise faces.
Hello, any idea how to pass the face id manually, like say i want to store the face id (information) somewhere else (server) and upon restart i feed the list with my own face ids. Or at least could let me know, if I already know the face information which variables are needed ?
Hi. Do you mean storing the face data? I’m not sure how to get that from the ESP32 storage and move it elsewhere.
Hello sir ……Can i add more than one output
Hi, You want to use another pin to control something else? I think io12 might work but I don’t remember exactly which pins you can set high and low.
Tnx sir for repling me …
Sir iam making door lock system but i want to add another out put …I mean 2nd out put which may be a light bulb ..I want to turn on the 2nd out put with another face
Check out the other tutorial here: https://robotzero.one/access-control-with-face-recognition/
these lines:
sprintf(recognised_message, “DOOR OPEN FOR %s”, f->id_name);
open_door(client);
could be changed to do different things based on the the name of the person. Two different functions.
How can i hard reset esp32
Thank you for your article. I have followed the instructions and have detected the side that turns the Flash on when it gets correct, but why is it correct, it can’t output the signal to control the relay. I tried adjusting delay () but still failed, forwarding the socket to the signal pin then it could not give the LOW signal that I originally set up ().
Please explain and help me.
Thank you!
I don’t really understand what you mean. Are you trying to connect the relay to the correct pin? Try adding some print statements if you aren’t sure where it fails.
here is my demo.
#include “esp_camera.h”
#include “fd_forward.h”
#include “fr_forward.h”
#include “fr_flash.h”
#define CAMERA_MODEL_AI_THINKER
//khái báo kết nối relay và đèn
#define trueled 4
#define relayPin 12
unsigned long currentMillis = 0;
unsigned long openedMillis = 0;
long interval = 5000; // mở khóa trong …..ms
// khái báo chân kết nối với esp32_cam
#define PWDN_GPIO_NUM 32
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 0
#define SIOD_GPIO_NUM 26
#define SIOC_GPIO_NUM 27
#define Y9_GPIO_NUM 35
#define Y8_GPIO_NUM 34
#define Y7_GPIO_NUM 39
#define Y6_GPIO_NUM 36
#define Y5_GPIO_NUM 21
#define Y4_GPIO_NUM 19
#define Y3_GPIO_NUM 18
#define Y2_GPIO_NUM 5
#define VSYNC_GPIO_NUM 25
#define HREF_GPIO_NUM 23
#define PCLK_GPIO_NUM 22
#define ENROLL_CONFIRM_TIMES 5
#define FACE_ID_SAVE_NUMBER 7
// cấu hình của mô hình MTMN (nhận diện khuôn mặt kích thước ngưỡng ứng viên, kim tự tháp
static inline mtmn_config_t app_mtmn_config()
{
mtmn_config_t mtmn_config = {0};
mtmn_config.type = FAST;
mtmn_config.min_face = 80;
mtmn_config.pyramid = 0.707;
mtmn_config.pyramid_times = 4;
mtmn_config.p_threshold.score = 0.6;
mtmn_config.p_threshold.nms = 0.7;
mtmn_config.p_threshold.candidate_number = 20;
mtmn_config.r_threshold.score = 0.7;
mtmn_config.r_threshold.nms = 0.7;
mtmn_config.r_threshold.candidate_number = 10;
mtmn_config.o_threshold.score = 0.7;
mtmn_config.o_threshold.nms = 0.7;
mtmn_config.o_threshold.candidate_number = 1;
return mtmn_config;
}
mtmn_config_t mtmn_config = app_mtmn_config();
// khai báo danh sách các khuôn mặt đc ghi nhớ
static face_id_list id_list = {0};
dl_matrix3du_t *image_matrix = NULL;
camera_fb_t * fb = NULL;
// gắn ảnh mới đc căn chỉnh vào kích thước của face đc cài đặt
dl_matrix3du_t *aligned_face = dl_matrix3du_alloc(1, FACE_WIDTH, FACE_HEIGHT, 3);
//hàm khởi tạo
void setup() {
Serial.begin(115200); //tốc độ boaud
// đặt chế độ ban đầu vào cho chân relay và led
pinMode(relayPin, OUTPUT);
pinMode(trueled, OUTPUT);
pinMode(33 , OUTPUT) ; // hoạt động ở mức thấp
digitalWrite(relayPin,LOW);
digitalWrite(trueled, LOW);
// khởi tạo camera và khai báo chân kết nối với esp
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sscb_sda = SIOD_GPIO_NUM;
config.pin_sscb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000; // tần số ban camera sử dụng
config.pixel_format = PIXFORMAT_JPEG; // khai báo chất lượng ảnh
config.frame_size = FRAMESIZE_UXGA; // kích thước khung ảnh
config.jpeg_quality = 10; // khung hình và chất lượng ảnh là jpeg
config.fb_count = 2;
//khởi tạo camera
esp_err_t err = esp_camera_init(&config);
if (err != ESP_OK) {
Serial.printf(“Camera init failed with error 0x%x”, err);
return;
}
// giảm kích thước khung hình để có tốc độ cao hơn
sensor_t * s = esp_camera_sensor_get(); // lưu và cài đặt máy ảnh vào bộ nhớ không thay đổi
s->set_framesize(s, FRAMESIZE_QVGA); // kích thước khung
s->set_vflip(s, 1);
face_id_init(&id_list, FACE_ID_SAVE_NUMBER, ENROLL_CONFIRM_TIMES); // khởi tạo danh sách khuôn mặt đc lưu và đăng kí
read_face_id_from_flash(&id_list);// đọc dữ liệu khuôn mặt hiện tại từ flash trên bo mạch
}
// chương trình để mở khóa cửa và bật đèn led
void rzoCheckForFace() {
currentMillis = millis();
if (run_face_recognition()) {
// nó trả về giá trị true
Serial.println(“Face recognised”);
digitalWrite(relayPin, HIGH); //relay hoạt động
digitalWrite(trueled, HIGH); // led mở
openedMillis = millis(); //thời gian đóng lại của relay
}
if (currentMillis – interval > openedMillis) {
// thời gian >5s thì đóng cửa
digitalWrite(relayPin, LOW); // mở relay
digitalWrite(trueled, LOW); // tắt Flash
digitalWrite(33 , LOW); // bật led
}
}
// hàm xác định nhận dạng khuôn mặt
bool run_face_recognition() {
bool faceRecognised = false;
int64_t start_time = esp_timer_get_time();
fb = esp_camera_fb_get();
if (!fb) {
Serial.println(“Camera capture failed”);
return false;
}
int64_t fb_get_time = esp_timer_get_time();
Serial.printf(“Get one frame in %u ms.\n”, (fb_get_time – start_time) / 1000); // đưa Cam vào khuôn mặt muốn nhận diện
image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3); //ma trận khuôn mặt phát hiện so sánh kích thước của ảnh
uint32_t res = fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item);
if (!res) {
Serial.println(“to rgb888 failed”);
dl_matrix3du_free(image_matrix);
}
esp_camera_fb_return(fb);
box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);
if (net_boxes) {
// nếu thiết lập camera ok
if (align_face(net_boxes, image_matrix, aligned_face) == ESP_OK) {
int matched_id = recognize_face(&id_list, aligned_face); // khai báo biến trong list khuôn mặt
if (matched_id >= 0) {
Serial.printf(“Match Face ID: %u\n”, matched_id);
faceRecognised = true; // nhận diện dúng
} else {
Serial.println(“No Match Found”); // khong xác định được khuôn mặt
matched_id = -1;
}
} else {
Serial.println(“Face Not Aligned”); // khuôn mặt chưa đc căn chỉnh
}
free(net_boxes->box);
free(net_boxes->landmark);
free(net_boxes);
}
dl_matrix3du_free(image_matrix);
return faceRecognised;
}
void loop() {
rzoCheckForFace();
delay(1000);
}
Trueled works normally and the relay pin when loaded code and running, does not give a signal. I use esp32_cam.. Thank you for your interest.
Found this really good.
However tried to rebuild it today and seems that it does not work with the new 1.0.5 ESP32 Cam Board update. HAd to role Arduino IDE back to 1.0.4 version of the board library. Then installed okay
Looks like the new 1.0.5 allows uploading the partition with the sketch https://github.com/espressif/arduino-esp32/pull/3328 I’ve created a new partition scheme and added it to the tutorial. I just copied the existing Huge App scheme and replaced the SPIFFS partition with the fr one used to store the faces. If you have a moment can you see if it works for you?
Heii
I’ve got the same problem as some other users here. I get for line 14 the error: “‘mtmn_config_t’ does not name a type”
I tried this code alone and it worked 🙂 Btw thanks a lot for this cool tutorial. But then I added other parts of code, the CameraWebServer and OTA Update example and now it doesn’t work anymore. I thought it most be some interference with the other code snippets they both worked before putting together. Sadly I don’t know enough about arduino IDE to completely understand what this “static inline” command to know why it isn’t working anymore with the other parts of the code. Could you explain what “static inline mtmn_config_t app_mtmn_config()” is for and what it does?
Saphira
Hi,
The app_mtmn_config() is to set up the configuration settings for the face recognition. It’s like a array of configuration variables. If you get an error I think it’s because you are using a pre-1.0.4 version of the ESP32 hardware library or you have missed an included library in the top of the Sketch.
I am using Version 1.0.5 of the Esp32 hardware library. I did the Update so this tutorial would be easier:) and these are the librarys:
#include “esp_camera.h”
#include “fd_forward.h”
#include “fr_forward.h”
#include “fr_flash.h”
Is one missing?
Thank for your project, this complete my hobby project, to return, I’ve paid you a coffee, enjoy it!
I’m happy you found it useful. Thanks for the coffee!
Hello robotzero, it’s possible to use the other tutorial to save faces with their names and then use this one (or one similar) that displays the names? (For example display the name of the recognized person in a LCD)
i’ve got rebooting on cam wifi
i using board 1.0.3rc1
can you help me?
Need to use a more recent version of the hardware library
Hi Robotzero, I can’t see the code section in “Face Recognition Trigger Event” and some images, I think there’s a problem with your web page 🙁
Has it re-appeared? Seems there was a problem with the cache.
The images yes, but the code in that section doesn’t. After “Paste the following code into a new sketch” there’s no code ;(
Hi. Code is there but maybe you have something blocking it in your browser. You can also download it from here: https://pastebin.com/3sgh3ytj
Hello sir thank you for the great project you shared with us it was awesome I was challenged while I was creating that partition but I managed to solve that problem and I wanted to Embed GSM so that It can generate notification after locking or unlocking but I was unable to complete that so can you help me solve that by embedding GSM.
I hope to get your answers as soon as possible sir and again thank you for the great work you done.
Hi, Just add you ‘send notification’ code here:
digitalWrite(relayPin, HIGH); //close (energise) relay
This is the part that opens the door.
Does it seem to look like this?
void open_door(WebsocketsClient &client) {
if (digitalRead(relay_pin) == LOW) {
//Set Exact Baud rate of the GSM/GPRS Module.
Serial.begin(9600);
Serial.print(“\r”);
delay(1000);
Serial.print(“AT+CMGF=1\r”);
delay(1000);
Serial.print(“AT+CMGS=\”+250781147055\”\r”);
delay(1000);
//The text of the message to be sent.
Serial.print(“Door is opened sir”);
delay(1000);
Serial.write(0x1A);
delay(1000);
digitalWrite(relay_pin, HIGH); //close (energise) relay so door unlocks
Serial.println(“Door Unlocked”);
client.send(“door_open”);
door_opened_millis = millis(); // time relay closed and door opened
}
is it okay?
and I wonder if it is possible to add these line after the message
char recognised_message[64];
sprintf(recognised_message, “DOOR OPEN FOR %s”, f->id_name);
open_door(client);
client.send(recognised_message);
these code that will mention the one who entered through the door
ah ok.. you are looking at this tutorial https://github.com/robotzero1/esp32cam-access-control/blob/master/FaceDoorEntryESP32Cam.ino Should be able to do that. f->id_name is the name of the person recognised. Not sure what type of variable the GSM library expects but you can try.
Hi, is it possible to recognize faces via sd card?
Hi, Do you mean save the faces to an SD card or recognise faces saved on a card? The first one probably but I don’t have the code and it would be slow. Second one is probably easier, you just have to replace the camera code that gets the JPG with some code that reads files off the SD card.
Yes, can I have a link of the code? I really need it for my research project. thankyou
I don’t have code for either of those options. You would need to code it yourself.
oh, I’m actually a beginner. So I can’t code it myself. thankyou for replying
Hello,
Thank you for this great tutorial,
it seems things have changed
in the WebServerCam,
there is no #include “fr_forward.h”
and also no variable named left_sample_face
please advise
thank you
Hi, Yeah, you’ll need to use the versions mentioned (1.0.4 I think) for this to work. I’ve not worked with the 2.0.x library.
Hi, I tried to add the face id capacity to #define FACE_ID_SAVE_NUMBER 10, the code was successfully uploaded, but on the serial monitor, the esp32 cam continues to reboot showing:
Wi-Fi connected
Successfully configured static IP
assertion “l->size == size” failed: file “/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/components/esp-face/face_recognition/fr_flash.c”, line 85, function: read_face_id_from_flash
abort() was called at PC 0x40114563 on core 1
Rebooting…
Is there a way to solve the problem?
Hi which version of the ESP32 hardware library are you using?
hello sir please i am having an error similar to this .. please can you help this is my finalyear project pleasers sir:
0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:10944
load:0x40080400,len:6360
entry 0x400806b4
E (776) fr_flash: Not found
Hi, Which ESP32 Hardware library are you using?
hello, have you upgraded the FaceDoorEntryESP32CAMPremium version? I can’t find it. Help..
Hi, It’s here: https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip
Thank you sir for answering. I also checked the premium version but did not see the difference. I want to know why you have to activate the “access control” button when entering the yard.
Thank you sir for answering. I also checked the premium version but did not see the difference. I want to know why you have to activate the “access control” button every time you enter the yard.
hi, i can’t find new part partition scheme after i changed board.txt, do you know why?
HI, Which version of the Arduino IDE are you using? Might just need to restart it.
i am using version 1.0.6, i already restarted, but nothing happen
I’m not sure if 1.0.6 works. 1.0.5 should be good to go as explained here:
Hi
This work is very great
but I wonder if it is possible to make the esp32 cam recognize the face directly when it is turned on without entering the page?
I hope you will respond to me as soon as possible. Thank you in advance
Hi, This version works without having to connect to a browser: https://robotzero.one/esp32-face-door-entry/
I downloaded the code, but face recognition does not start unless you enter the web page
Sure you are using this one? https://robotzero.one/esp32-face-door-entry/ There’s no web interface in this version