Open a door when a face is recognised using the ESP32-CAM
This page contains code and other information for the the YouTube video found here: https://youtu.be/mu3-Sff0B9w
The code is a combination of the ESP32-CAM Face Recognition for Home Automation and ESP-WHO Face Recognition with WebSocket Communication tutorials.
As mentioned in the video, while this demonstration works well, the ESP32-CAM and associated IDF and Arduino libraries don’t match professional face based access control systems. It can be easily tricked and doesn’t check for ‘liveness’. It could be used as a secondary control system to open doors or activate systems for people that have already passed through another access control system.
If you want to see more like this then please say ‘thanks’ with a coffee below.
Download files from here: https://github.com/robotzero1/esp32cam-access-control
Buy Me A Coffee
If you found something useful above please say thanks by buying me a coffee here...
Hi !
Thankk you for the code , but i have a problem with esp32 cam when i enter access control it freezes then crashes and i have to restart it to work again , can it be a problem of overheating or something else ? any idea to fix this ?
Hi, I was sometimes seeing this. I’m not sure if it’s a bug in the Espressif libraries (I think someone else has reported this) or the camera is just overheating. It seemed random to me.
Hi !
I’m thai and love this project,so plz tell me how to detect intruder in code i’m beginner.
Hi, I’m not sure what you mean but check out the other tutorials on the site and maybe there’s something to help.
First of all congrats for this great work!
I’m facing an issue on this project…Everything was running perfect until the last couple of days i was no longer able to stream from the esp-32 cam, as a result it couldn’t detect or recognize faces anymore. I updated the boards and libraries, i even tried it from different computer and browser but still the same. I still can browse into the web UI but it won’t stream any image from the left window. I tested my esp32-cam board on other projects and it’s fully functional. So, any thoughts what’s causing this issue? Thanks!
Does the camerawebserver example work for you? If you load that up can see the video at http://your-ip-address:81/stream ? Might be the camera.
Success, installed 12/1/2019! Did the partition table changes and installed the code from the zip file. Worked great, no issues!
Tried to strip down the code for OTA but couldn’t get that to work, only had about 10 minutes to work on the OTA, I’ll try again later.
Thanks very much for posting this and the YouTube video!
Great! It’s always nice to hear when people have a project working.
Thanks for sharing this. I implemented it and tried compiling the code; but got the following error message:
‘ Sketch too big; see http://www.arduino.cc/en/Guide/Troubleshooting#size for tips on reducing it.
Error compiling for board ESP32 Wrover Module. ”
Please, what should I do to solve this? If I change partition scheme to ”Huge App (3MB No OTA…), it compiles.
Did you create a new partition following this tutorial – https://robotzero.one/esp32-face-door-entry/ ? It’s a big application but also needs the partition for storing faces.
Hi, I’ve followed what you’ve done but when I copy the code in I get the following error:
‘struct mtmn_config_t’ has no member named ‘type’
Any solutions to this?
Hi, Probably the wrong ESP32 Hardware library is installed. Which version are you using?
Hi, sorted that out, yup I was using the wrong hardware. Sorry to ask a question a little unrelated here, but if say i wanted to store the enrolled faces on a google firebase database, which function do I call that contains the blob information?
I’m quite new to this and I haven’t found any help online.
I’m not sure how you would save the face information somewhere different from the flash memory. I don’t know how well the recognition would work if it was testing against face data that wasn’t stored locally… it might be too slow.
hey i want ask about something
I have followed all the ways to be done but I found an error
Camera Ready! Use’http://192.168.1.68′ to connect
[D] [WiFiClient.cpp:509] connected(): Disconnected: RES: -1, ERR:104
[E] [WiFiClient.cpp:392] write(): fail on fd 60, errno: 104, “Connection reset by peer”
[E] [WiFiClient.cpp:392] write(): fail on fd 61, errno: 104, “Connection reset by peer”
can you help me? why show this error
Do you know which version of the ESP32 Hardware libraries you are using? Does it always happen?
Im using v1.0.4
Yes it always happens, and until now it’s still like that
Hmmm. I’m not sure. I thought it was a problem with a previous version of the Wi-Fi library. It works fine on my ESP-EYE so it should be OK on the Ai-Thinker.
I noticed that when I close the browser I see that message which means it comes up when the connection to the browser is lost for whatever reason.
Can you move closer to the router? Might be a power issue as well if the radio for the Wi-Fi doesn’t have enough juice.
whenever i click the ip given in serial monitor it shows like
WiFi connected
httpd_start
Camera Ready! Use ‘http://192.168.43.116’ to connect
[E][WiFiClient.cpp:392] write(): fail on fd 60, errno: 104, “Connection reset by peer”
IT shows an error.Please hep me out
Does it work at all? I guess you are using the 1.0.4 of the ESP Hardware libraries? Can you try moving closer to your router?
Hello, thanks for the great code snippet.
I also faced the same issue, when I entered access control the camera freezes and the system reboots.
I have made a small changes and the issue is resolved.
Under the recognition branch statement above the line of code “if(f)” the piece of code
“Serial.println(f->id_name);” is removed. Before validating the condition the name is printed in serial monitor causing load prohibited error and causing the system to reboot in my scenario.
Thanks again..
Ah.. thanks for pointing that out. I hadn’t noticed that line sitting there. I’ve removed it.
Please help..
Showing error during compliling. It shows “camrea_index.h” no such file or director ”..
“Multiple libraries were found ” for WiFi.h
You need to read the tutorial entirely. You’ve not got all the files you need in the directory.
Sir I am getting error on line 200 saying “static inline int do_enrollment(face_id_name_list *face_list, dl_matrix3d_t *new_id)”
can you help me.
hope to hear form you soon
Hi. What error do you see?
Hello Master
I have error
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 – WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 2 – STA_START
..[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 5 – STA_DISCONNECTED
[W][WiFiGeneric.cpp:353] _eventCallback(): Reason: 2 – AUTH_EXPIRE
..[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 4 – STA_CONNECTED
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 7 – STA_GOT_IP
[D][WiFiGeneric.cpp:381] _eventCallback(): STA IP: 192.168.43.148, MASK: 255.255.255.0, GW: 192.168.43.121
.
WiFi connected
httpd_start
Camera Ready! Use ‘http://192.168.43.148’ to connect
[D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128
Now. I use Board 1.0.4 and library manager arduinowebsockets ver. 0.4.10 for Arduino 1.8.12
Before the upgrade went well. But now it can’t.
Can You help me…
Hi, I thought it wasn’t working before updating to 1.0.4? It might be a poor Wi-Fi connection. Does the CameraWebServer example work well for you?
When I try to use the example camera webserver works fine, sir.
Hello, thanks for the great instructions…..
One question can I directly access the camera stream?
Hi. If you just want the stream the easiest way is to load up the CameraWebServer example and open http://ip.address.of.esp32:81/stream
Hello, is it possible to increase the frame size to svga???
Hi, You can stream at SVGA but the face detection etc has to be run at a lower resolution.
Hello, why has the face recognition has to be run at a lower resolution??
It’s a limit of the hardware but mostly face recognition doesn’t need high resolution images.
How can I change the resolution during runtime? Can you show me this?
You can change the resolution like this:
sensor_t * s = esp_camera_sensor_get();s->set_framesize(s, FRAMESIZE_SVGA);
delay(300);
camera_fb_t * fb = esp_camera_fb_get();
But not while grabbing frames. You would have to close the framebuffer with esp_camera_fb_return(fb); then change and restart.
Sorry, but can you help me? What happens is that when editing the html file that comes inside the zip, it does not make the changes, for example it does not change the text to Spanish, but always remains in English
Does anyone know why? or where it has to be modified?
HTML file is just for reference. Check this tutorial for how to edit the HTML – https://robotzero.one/esp32-cam-custom-html/
This is a great work and it works for me fine. Thanks very much
My question how to activate access control on start without clicking
its button from web server ?
This version you can disconnect and it still works – https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip I’m working on a version that works connected or not.
Thanks for your replay
But I do not want to connect to any external links as this is local home door.
I just want to activate access control at startup
I will wait for your new version.
I have another issue that esp32-cam board gets heated fast and then freeze so I have to power
off and on again every 15 minutes, Any one have this problem ??
# The heat concentrated in WiFi metal shield.
This project is perfect but not practically as if you use it for example for your home you should be connected with web server for start face-recognition.
This means if you out side and one of your family come home he can not come inside as the web server is not connected
Hi, I have the same issue as reported by Dimitris November 14, 2019 at 10:59 am. I’ve loaded the software and did work for a short period of time. I’ve loaded CameraWebServer example and the camera does work correctly. FaceDoorEntry is giving me the following screen:
https://drive.google.com/file/d/1QcbvuWPS8L-V1lLD91SnGca_Ur1sLygZ/view?usp=sharing
I’ve added an extra output in the code:
void loop() {
Serial.println(“Loop starts”);
auto client = socket_server.accept();
Serial.println(“Clinet created”);
The output gives me “Loop starts”, but not “Clinet created”
Any suggestions would be much appreciated.
Hi, It seems like you don’t have a very good connection to the Wi-Fi and the websocket connection isn’t being made. Do you have the internal aerial connected: https://robotzero.one/wp-content/uploads/2019/01/esp32-cam-ipex-antenna-jumper.jpg
Is there is a way for face-recognition and open door without web connect ??
You can use the first tutorial you tried. This works without the browser connection: https://robotzero.one/esp32-face-door-entry/ Otherwise wait a few weeks and I’ll have another tutorial like the one above that works with and without Wi-Fi
I have the same problem as radek marko
I used it for 2 weeks without any problem until yesterday, since I no longer let myself in to watch the transmission from the camera.
I get this error:
[D] [WiFiClient.cpp: 509] connected (): Disconnected: RES: 0, ERR: 128
I don’t understand why it happened, it just stopped working.
Has anyone solved this problem?
The first tutorial was working on my board good but now after I flash current tutorial and it works fine , then try to go back to first one but is not working.
The first tutorial now is keep restating, and I notice that when I remove this line ” read_face_id_from_flash(&id_list);”
it works, I think there is something with the partition it can not read from it.
Is there is any way to delete every thing on board and return to factory settings ??
I think you can do this by flashing another Sketch but choose the full partition to wipe the partitions and then go back to the first tutorial.
hi, Stunning work, there is a small problem if I disconnect it from electricity need to re-insert (no image saves), how can I save the pictures to SD card ?? Thanks
Hi, It doesn’t save images at the moment, it just saves a ‘map’ of the faces. These should be stored permanently. Did you set up the new partition?
Hi,
Great project and works very nicely, Thank you.
I am trying to access the gate and video from anywhere. any idea without having to use another computer. direct connection to the http site.
I also noticed the camera freezes and have to restart ??
Hi. You can probably run the ESP32 as an access point to access it directly or alternatively use port forwarding on your router so you can access the project from outside your house. I need to do some more on this project but don’t have much time at the moment.
Hi. I’m trying to do this project too, but after flash the Sketch, I will access the ESP32 IP in browser, the message below appears.
The camerawebserver works perfectly.
basic_string::_M_construct null not valid258EAFA5-E914-47DA-95CA-C5AB0DC85B110123456789abcdefABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/upgradeWebsocketconnectionSec-WebSocket-AcceptGET HTTP/1.1 Host: Sec-WebSocket-Key: UpgradeConnectionUpgrade: websocket Sec-WebSocket-VersionConnection: Upgrade User-AgentSec-WebSocket-Version: 13 OriginUser-Agent: TinyWebsockets Client Origin: https://github.com/gilmaimon/TinyWebsockets HTTP/1.1 101httpws://wswss://wsshttps://https8F @xF @H�@�; @�C @�; @P�@���������������U @V @websocket13Sec-WebSocket-KeyHTTP/1.1 101 Switching Protocols Sec-WebSocket-Accept: W @
intr_handle)/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/task_wdt.cesp_intr_free(twdt_config->intr_handle)esp_register_freertos_idle_hook_for_cpu(idle_hook_cb, i)esp_task_wdt_addesp_task_wdt_initcal_versioncal_macphy_initE (%d) %s: %s: calibration data MAC check failed: expected %
Hi, I have no idea! Something wrong with the websocket but I don’t know if it’s the library, the browser or something in the Wi-Fi connection.
Hi ! i tried the camerawebserver example and it works perfectly, but when i tried the door lock User Interface, particularly the STREAM CAMERA, it shows nothing, when i look up into the serial monitor, it says C⸮[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 – WIFI_READY, the ” ⸮ ” made me wondering about the websocket being not working, can you please give me some advice sir???
Hi, The only thing I can suggest is putting in a load of Serial.println(“something here”) statements in to see how far the code gets. You can also try one in the loop to see if something is happening in the loop.
I am getting a strange error while compiling. It says “In function ‘esp_err_t delete_all_faces(websockets::WebsocketsClient&)’:
FaceDoorEntryESP32Cam:227: error: no return statement in function returning non-void [-Werror=return-type]
”
I guess its expecting a return but im not sure what data type “esp_err_t ” is. using Arduinowebsocket s 0.4.18. How can I fix this?
The function should look like this:
static esp_err_t delete_all_faces(WebsocketsClient &client){
delete_face_all_in_flash_with_name(&st_face_list);
client.send("delete_faces");
}
It looks different in the error message.
hai im like the project and i stuck on error compiling for board esp32 wrover module…can you help me
What error message do you see?
‘struct mtmn_congig_t’ has no member named ‘type’
Do you know which version of the ESP32 Hardware Library you have installed?
Arduino: 1.8.13 (Windows 10), Board: “ESP32 Wrover Module, Face Recognition (2621440 bytes), QIO, 80MHz, 921600, Verbose”
C:\Users\natha\Documents\Arduino\esp32cam-access-control-master\FaceDoorEntryESP32Cam\FaceDoorEntryESP32Cam.ino: In function ‘mtmn_config_t app_mtmn_config()’:
FaceDoorEntryESP32Cam:52:15: error: ‘struct mtmn_config_t’ has no member named ‘type’
mtmn_config.type = FAST;
^
FaceDoorEntryESP32Cam:52:22: error: ‘FAST’ was not declared in this scope
mtmn_config.type = FAST;
^
FaceDoorEntryESP32Cam:55:15: error: ‘struct mtmn_config_t’ has no member named ‘pyramid_times’
mtmn_config.pyramid_times = 4;
^
Multiple libraries were found for “WiFi.h”
Used: C:\Users\natha\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.2\libraries\WiFi
Not used: e:\Program Files (x86)\Arduino\libraries\WiFi
exit status 1
‘struct mtmn_config_t’ has no member named ‘type’
This report would have more information with
“Show verbose output during compilation”
option enabled in File -> Preferences.
can you help me sir
You need to update the ESP32 Hardware library as seen here: https://robotzero.one/esp32-cam-arduino-ide/
Unfortunately, http://your-ip-address: 81/stream does not work. It is not integrated.
Can you easily add that?
your-ip-address is for the camerawebserver example it won’t work on this tutorial because it uses websockets.
Sir how can i delete the old code from esp 32
You don’t need to delete old code normally. When you upload (flash) the device it writes over everything.
Good tutorial and great work!
I’m facing an issue on this project…Everything was running perfect until the last couple of days i was no longer able to stream from the esp-32 cam, as a result it couldn’t detect or recognize faces anymore. I updated the boards and libraries, i even tried it from different computer and browser but still the same. I still can browse into the web UI but it won’t stream any image from the left window. I tested my esp32-cam board on other projects and it’s fully functional. So, any thoughts what’s causing this issue? Thanks!
Somebody else mentioned this happening. Did you change anything or add new faces before it stopped working? Do you see anything in the serial monitor or the browser console (F12)?
Hello sir iam facing a problem camera init failed with eror 0x30 after uploading code ..
This eror is givng at serial maniter
Possibly the wrong camera has been set. Which tutorial are you following?
I got this eror on ist day but after that it was solved now again i got this eror but this time iam unable to gitrid of this
Iam waiting for ur rply
Maybe it’s broken. If you are new to this I would recommend starting with more basic tutorials and maybe use a different board. The ESP32-CAMs are tricky to work with.
Code is uploading but camera didnt give ip adress ….If this is broken ….Why code is uploading
Maybe the camera itself is broken but the board is OK. The only thing you can do is keep testing things until you find the problem or buy another and see if that works.
Hello sir i have change the module ….Now code is uploading …Ip adress giving
But when i connect the board in circut and give aupply …….They does not connect to wifi
Hi, Are you connecting the relay or trying with just the board? Try Tools > Core Debug Mode > Verbose to possibly see more details in the serial monitor.
Iam trying with board …..In whih i have connect all the elements
Hi, wben the code runs, on serial monitor appair repetitive reset list. I use 1.0.4 esp32. What problem can it be? Thanks.
Hi, Is the ESP32 continually reseting? Does the CameraWebServer example work for you?
Hi, thanks for your reply, yes, ESP32 continually reseting. The CameraWebServer example work for me and also ESP32-CAM Face Recognition for Home Automation. I I just changed the original pertition scheme substituting 0x290000 instead of 0x280000 because I received a compilation error “sketch too big.”.
It’s probably crashing but the partitions aren’t correct. It should all work with the new partitions in the tutorial. Maybe it wasn’t selected when you flashed the device and then you saw the error.
Hi, after modifying the csv file, I have the following compilation error:
“C:\\Personal\\programmi-vari\\arduino-1.8.9-Portatile\\arduino-1.8.9\\hardware\\espressif\\esp32/tools/gen_esp32part.exe” -q “C:\\DOCUME~1\\fulvio\\IMPOST~1\\Temp\\arduino_build_640125/partitions.csv” “C:\\DOCUME~1\\fulvio\\IMPOST~1\\Temp\\arduino_build_640125/sketch_face_vers_2.ino.partitions.bin”
Error at line 2: CSV flag column contains unknown flag ‘;;;’ This is a strange error! in the csv file there are no “;” but only “,”.
I suspect there is a compatibility problem of my XP OS with the modified csv file. Beacause in the tutorial “Create a partition” it is not explained how to create the csv file, but it is shown only in notepad++, can you send me an original csv file by changing app0 from 0x280000 to 0x290000 (or higher)? Thanks.
You shouldn’t need to change the numbers. Here’s the csv file I used – https://robotzero.one/wp-content/uploads/2019/04/rzo_partitions.csv. There’s more info about this here – https://robotzero.one/arduino-ide-partitions/ But… take a look at this: https://github.com/francis94c/ESP32Partitions it might be easier for you.
Hi, so, the problem was when I saved the changes of the csv file, excell insert “;;;” at the end of each flags field. I solved this problem by editing this csv file with notepad ++:
# Name, Type, SubType, Offset, Size, Flags,
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x290000,
fr, 32, 32, 0x300000, 0xEF000,
eeprom, data, 0x99, 0x3ef000, 0x1000,
spiffs, data, spiffs, 0x3f0000, 0x2F000,
As you can see, I changed the size of app0 and consequently some offsets. Unfortunately, starting the code, from the serial montor I see that the reset occurs continuously, as it happened before. The compilation and upload were successful so I don’t know what else to do. Do you have any other suggestions before leaving this tutorial? I’ll still offer you a cup of coffee because I’m sure it’s a great program that cost you and still costs you a lot of effort. Thanks.
Are you sure you need to change the sizes? Are you changing something from the tutorial?
It gave me compilation error for “sketch too big”, so I increased the size of app0 from 280000 to 290000 and fixed the offsets, in fact then it went well compiling and uploading. Unfortunately when I start the code the card resets continuously. I think there is a problem with the csv file, but I don’t know which one.
I think you saw the too big error because you hadn’t selected the new partition or not set up the partition part correctly. The CSV partition sizes should work with the tutorial.
Hi, I ran the following steps from scratch: Camera Web Server example with Huge APP partition works fine;
Camera Web Server Permanent with the new partition csv file does not work, it does continuous autoreset;
Camera Web Server Permanent with the original csv file you sent me does not work, it does continuous autoreset.
I think the flash memory is corrupt, is it possible to restore all original partitions? Thanks.
I use esp32 CAM but after completing the above steps and loading the code, it does not recognize the face, it just says: Get one frame in 0 ms.
02: 44: 26.152 -> No Match Found
02: 44: 26.152 -> Get one frame in 0 ms.
02: 44: 26.396 -> dl_matrix3dqq_fc_with_bias, value> DL_QTP_MAX
02: 44: 26.396 -> dl_matrix3dqq_fc_with_bias, value <DL_QTP_MIN
. The worst thing is when declaring the pin connected to the relay pin, it cannot receive the signal from that pin. Can you give me advice. thank you
I think it’s because it has trouble detecting your face. Try with more light or moving further from the camera. Does the CameraWebServer example work for you?
dl_matrix3dqq_fc_with_bias, value> DL_QTP_MAX
02: 44: 26.396 -> dl_matrix3dqq_fc_with_bias, value <DL_QTP_MIN
Nó thông báo như thế này nghĩa là sao vậy.cảm ỏn
Hi, all the malfunctions were caused by the windows XP os. When I used Windows 7 it worked fine so I could verify that you did a great job !!! I’ll offer you a coffee. I take this opportunity to ask you, have you already made a tutorial on a video door phone?
Hi, I don’t know why XP would affect it. Maybe the USB driver. I know how to make the video door phone but one or two other things to do first. Thanks for the coffee!
hi sir, thanks for this project because it really helps me a lot. After I read the comment in these blogs, there have two problems which I faced also with the other people. The first problem is when start to access the webpage, the camera is not streaming and second problem that I faced is the flash memory of your code is exceed the partition that you created. In the end I found the problem. For first problem, I suspect that the problem happened in the arduino websockets and the network connection because in first few times, it can be accessed but not after several times of uploading the codes to the board. So, my solution is to reset your network in your pc or laptop. The solution for second problem is do not update your esp 32 to the latest version (1.5) because I suspect that the latest version may adding some new feature which the memory is exceed your partitions. So, 1.4 is fine for me to function the code. Sorry for my broken english and thanks again for the program that you created.
Hi, Thanks for letting me know the problems you encountered. The Websockets works really well but if the Wi-Fi isn’t perfect and the connection drops the Websocket closes and that’s it. I need to do something to handle this.
For the partitions, I’m not sure why it doesn’t work the same way but now there is a much easier method where you just put the partitions.csv file in the same folder as the sketch. I’ve created a different partition file and tested it and it all seems OK. The tutorials have been updated and I’ve added the new partition.csv to Github.
Thank you so much for the replied , I will try your solution which is putting csv file together with the sketch later.
Sir, I encountered with the problem of camera is not streaming on the webpage again. May I know why this problem occurs sometimes but not in other times?? Is it because of the websocket ? Because I am pretty sure the cam work in webserver example and my wifi connection is stable,
Sir, may i know how to store face data and the name to database( google sheet or excel online) instead of storing it in the board? thank you!
Sir, i am using your new partition.csv to run the project, but i still facing the same problem again. At the first few times, the project can work well but after i try delete all photo memory in the browser there, the camera will not stream on the left side of the webpage when i reuploading and rerun the program again.