Access Control with Face Recognition

featured.jpg

Open a door when a face is recognised using the ESP32-CAM

This page contains code and other information for the the YouTube video found here: https://youtu.be/mu3-Sff0B9w

The code is a combination of the ESP32-CAM Face Recognition for Home Automation and ESP-WHO Face Recognition with WebSocket Communication tutorials.

As mentioned in the video, while this demonstration works well, the ESP32-CAM and associated IDF and Arduino libraries don’t match professional face based access control systems. It can be easily tricked and doesn’t check for ‘liveness’. It could be used as a secondary control system to open doors or activate systems for people that have already passed through another access control system.

If you want to see more tutorials like this such as doorbell camera then please say ‘thanks’ with a coffee below.

Here’s the current code used. You can download this with the other files you need here: https://github.com/robotzero1/esp32cam-access-control


56 Replies to “Access Control with Face Recognition”

  1. Jasou says:

    Hi !
    Thankk you for the code , but i have a problem with esp32 cam when i enter access control it freezes then crashes and i have to restart it to work again , can it be a problem of overheating or something else ? any idea to fix this ?

    1. WordBot says:

      Hi, I was sometimes seeing this. I’m not sure if it’s a bug in the Espressif libraries (I think someone else has reported this) or the camera is just overheating. It seemed random to me.

  2. Noppanun says:

    Hi !
    I’m thai and love this project,so plz tell me how to detect intruder in code i’m beginner.

    1. WordBot says:

      Hi, I’m not sure what you mean but check out the other tutorials on the site and maybe there’s something to help.

  3. Dimitris says:

    First of all congrats for this great work!
    I’m facing an issue on this project…Everything was running perfect until the last couple of days i was no longer able to stream from the esp-32 cam, as a result it couldn’t detect or recognize faces anymore. I updated the boards and libraries, i even tried it from different computer and browser but still the same. I still can browse into the web UI but it won’t stream any image from the left window. I tested my esp32-cam board on other projects and it’s fully functional. So, any thoughts what’s causing this issue? Thanks!

    1. WordBot says:

      Does the camerawebserver example work for you? If you load that up can see the video at http://your-ip-address:81/stream ? Might be the camera.

  4. Keith Mack says:

    Success, installed 12/1/2019! Did the partition table changes and installed the code from the zip file. Worked great, no issues!
    Tried to strip down the code for OTA but couldn’t get that to work, only had about 10 minutes to work on the OTA, I’ll try again later.
    Thanks very much for posting this and the YouTube video!

    1. WordBot says:

      Great! It’s always nice to hear when people have a project working.

  5. Eva says:

    Thanks for sharing this. I implemented it and tried compiling the code; but got the following error message:
    ‘ Sketch too big; see http://www.arduino.cc/en/Guide/Troubleshooting#size for tips on reducing it.
    Error compiling for board ESP32 Wrover Module. ”
    Please, what should I do to solve this? If I change partition scheme to ”Huge App (3MB No OTA…), it compiles.

    1. WordBot says:

      Did you create a new partition following this tutorial – https://robotzero.one/esp32-face-door-entry/ ? It’s a big application but also needs the partition for storing faces.

  6. Preman Jeyabalan says:

    Hi, I’ve followed what you’ve done but when I copy the code in I get the following error:

    ‘struct mtmn_config_t’ has no member named ‘type’

    Any solutions to this?

    1. WordBot says:

      Hi, Probably the wrong ESP32 Hardware library is installed. Which version are you using?

  7. Preman Jeyabalan says:

    Hi, sorted that out, yup I was using the wrong hardware. Sorry to ask a question a little unrelated here, but if say i wanted to store the enrolled faces on a google firebase database, which function do I call that contains the blob information?

    I’m quite new to this and I haven’t found any help online.

    1. WordBot says:

      I’m not sure how you would save the face information somewhere different from the flash memory. I don’t know how well the recognition would work if it was testing against face data that wasn’t stored locally… it might be too slow.

  8. cavim says:

    hey i want ask about something
    I have followed all the ways to be done but I found an error

    Camera Ready! Use’http://192.168.1.68′ to connect
    [D] [WiFiClient.cpp:509] connected(): Disconnected: RES: -1, ERR:104
    [E] [WiFiClient.cpp:392] write(): fail on fd 60, errno: 104, “Connection reset by peer”
    [E] [WiFiClient.cpp:392] write(): fail on fd 61, errno: 104, “Connection reset by peer”

    can you help me? why show this error

    1. WordBot says:

      Do you know which version of the ESP32 Hardware libraries you are using? Does it always happen?

  9. cavim says:

    Im using v1.0.4
    Yes it always happens, and until now it’s still like that

    1. WordBot says:

      Hmmm. I’m not sure. I thought it was a problem with a previous version of the Wi-Fi library. It works fine on my ESP-EYE so it should be OK on the Ai-Thinker.

      I noticed that when I close the browser I see that message which means it comes up when the connection to the browser is lost for whatever reason.

      Can you move closer to the router? Might be a power issue as well if the radio for the Wi-Fi doesn’t have enough juice.

  10. +krish sharan says:

    whenever i click the ip given in serial monitor it shows like
    WiFi connected
    httpd_start
    Camera Ready! Use ‘http://192.168.43.116’ to connect
    [E][WiFiClient.cpp:392] write(): fail on fd 60, errno: 104, “Connection reset by peer”

    IT shows an error.Please hep me out

    1. WordBot says:

      Does it work at all? I guess you are using the 1.0.4 of the ESP Hardware libraries? Can you try moving closer to your router?

  11. Vichu says:

    Hello, thanks for the great code snippet.
    I also faced the same issue, when I entered access control the camera freezes and the system reboots.
    I have made a small changes and the issue is resolved.
    Under the recognition branch statement above the line of code “if(f)” the piece of code
    “Serial.println(f->id_name);” is removed. Before validating the condition the name is printed in serial monitor causing load prohibited error and causing the system to reboot in my scenario.
    Thanks again..

    1. WordBot says:

      Ah.. thanks for pointing that out. I hadn’t noticed that line sitting there. I’ve removed it.

  12. ANIRBAN GHOSAL says:

    Please help..
    Showing error during compliling. It shows “camrea_index.h” no such file or director ”..

    “Multiple libraries were found ” for WiFi.h

    1. WordBot says:

      You need to read the tutorial entirely. You’ve not got all the files you need in the directory.

  13. wasantha says:

    Sir I am getting error on line 200 saying “static inline int do_enrollment(face_id_name_list *face_list, dl_matrix3d_t *new_id)”
    can you help me.
    hope to hear form you soon

    1. WordBot says:

      Hi. What error do you see?

  14. Andri says:

    Hello Master
    I have error
    [D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 – WIFI_READY
    [D][WiFiGeneric.cpp:337] _eventCallback(): Event: 2 – STA_START
    ..[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 5 – STA_DISCONNECTED
    [W][WiFiGeneric.cpp:353] _eventCallback(): Reason: 2 – AUTH_EXPIRE
    ..[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 4 – STA_CONNECTED
    [D][WiFiGeneric.cpp:337] _eventCallback(): Event: 7 – STA_GOT_IP
    [D][WiFiGeneric.cpp:381] _eventCallback(): STA IP: 192.168.43.148, MASK: 255.255.255.0, GW: 192.168.43.121
    .
    WiFi connected
    httpd_start
    Camera Ready! Use ‘http://192.168.43.148’ to connect
    [D][WiFiClient.cpp:509] connected(): Disconnected: RES: 0, ERR: 128

    Now. I use Board 1.0.4 and library manager arduinowebsockets ver. 0.4.10 for Arduino 1.8.12
    Before the upgrade went well. But now it can’t.
    Can You help me…

    1. WordBot says:

      Hi, I thought it wasn’t working before updating to 1.0.4? It might be a poor Wi-Fi connection. Does the CameraWebServer example work well for you?

  15. Andri says:

    When I try to use the example camera webserver works fine, sir.

  16. You are cool says:

    Hello, thanks for the great instructions…..

    One question can I directly access the camera stream?

    1. WordBot says:

      Hi. If you just want the stream the easiest way is to load up the CameraWebServer example and open http://ip.address.of.esp32:81/stream

  17. Temp says:

    Hello, is it possible to increase the frame size to svga???

    1. WordBot says:

      Hi, You can stream at SVGA but the face detection etc has to be run at a lower resolution.

  18. Temp says:

    Hello, why has the face recognition has to be run at a lower resolution??

    1. WordBot says:

      It’s a limit of the hardware but mostly face recognition doesn’t need high resolution images.

  19. temp says:

    How can I change the resolution during runtime? Can you show me this?

    1. WordBot says:

      You can change the resolution like this:
      sensor_t * s = esp_camera_sensor_get();
      s->set_framesize(s, FRAMESIZE_SVGA);
      delay(300);
      camera_fb_t * fb = esp_camera_fb_get();

      But not while grabbing frames. You would have to close the framebuffer with esp_camera_fb_return(fb); then change and restart.

  20. sneyl says:

    Sorry, but can you help me? What happens is that when editing the html file that comes inside the zip, it does not make the changes, for example it does not change the text to Spanish, but always remains in English
    Does anyone know why? or where it has to be modified?

    1. WordBot says:

      HTML file is just for reference. Check this tutorial for how to edit the HTML – https://robotzero.one/esp32-cam-custom-html/

  21. Amgad Steen says:

    This is a great work and it works for me fine. Thanks very much
    My question how to activate access control on start without clicking
    its button from web server ?

    1. WordBot says:

      This version you can disconnect and it still works – https://robotzero.one/wp-content/uploads/2020/03/FaceDoorEntryESP32CAMPremium.zip I’m working on a version that works connected or not.

  22. Amgad Steen says:

    Thanks for your replay
    But I do not want to connect to any external links as this is local home door.
    I just want to activate access control at startup
    I will wait for your new version.

    I have another issue that esp32-cam board gets heated fast and then freeze so I have to power
    off and on again every 15 minutes, Any one have this problem ??
    # The heat concentrated in WiFi metal shield.

  23. Amgad Steen says:

    This project is perfect but not practically as if you use it for example for your home you should be connected with web server for start face-recognition.
    This means if you out side and one of your family come home he can not come inside as the web server is not connected

  24. radek_marko says:

    Hi, I have the same issue as reported by Dimitris November 14, 2019 at 10:59 am. I’ve loaded the software and did work for a short period of time. I’ve loaded CameraWebServer example and the camera does work correctly. FaceDoorEntry is giving me the following screen:

    https://drive.google.com/file/d/1QcbvuWPS8L-V1lLD91SnGca_Ur1sLygZ/view?usp=sharing

    I’ve added an extra output in the code:
    void loop() {
    Serial.println(“Loop starts”);
    auto client = socket_server.accept();
    Serial.println(“Clinet created”);

    The output gives me “Loop starts”, but not “Clinet created”

    Any suggestions would be much appreciated.

    1. WordBot says:

      Hi, It seems like you don’t have a very good connection to the Wi-Fi and the websocket connection isn’t being made. Do you have the internal aerial connected: https://robotzero.one/wp-content/uploads/2019/01/esp32-cam-ipex-antenna-jumper.jpg

  25. Amgad Steen says:

    Is there is a way for face-recognition and open door without web connect ??

    1. WordBot says:

      You can use the first tutorial you tried. This works without the browser connection: https://robotzero.one/esp32-face-door-entry/ Otherwise wait a few weeks and I’ll have another tutorial like the one above that works with and without Wi-Fi

  26. Carlosmtzv says:

    I have the same problem as radek marko
    I used it for 2 weeks without any problem until yesterday, since I no longer let myself in to watch the transmission from the camera.
    I get this error:
    [D] [WiFiClient.cpp: 509] connected (): Disconnected: RES: 0, ERR: 128
    I don’t understand why it happened, it just stopped working.
    Has anyone solved this problem?

  27. Amgad Steen says:

    The first tutorial was working on my board good but now after I flash current tutorial and it works fine , then try to go back to first one but is not working.
    The first tutorial now is keep restating, and I notice that when I remove this line ” read_face_id_from_flash(&id_list);”
    it works, I think there is something with the partition it can not read from it.
    Is there is any way to delete every thing on board and return to factory settings ??

    1. WordBot says:

      I think you can do this by flashing another Sketch but choose the full partition to wipe the partitions and then go back to the first tutorial.

  28. qaiss says:

    hi, Stunning work, there is a small problem if I disconnect it from electricity need to re-insert (no image saves), how can I save the pictures to SD card ?? Thanks

    1. WordBot says:

      Hi, It doesn’t save images at the moment, it just saves a ‘map’ of the faces. These should be stored permanently. Did you set up the new partition?

  29. BehMadi says:

    Hi,
    Great project and works very nicely, Thank you.
    I am trying to access the gate and video from anywhere. any idea without having to use another computer. direct connection to the http site.
    I also noticed the camera freezes and have to restart ??

    1. WordBot says:

      Hi. You can probably run the ESP32 as an access point to access it directly or alternatively use port forwarding on your router so you can access the project from outside your house. I need to do some more on this project but don’t have much time at the moment.

  30. Emerson Soares says:

    Hi. I’m trying to do this project too, but after flash the Sketch, I will access the ESP32 IP in browser, the message below appears.
    The camerawebserver works perfectly.

    basic_string::_M_construct null not valid258EAFA5-E914-47DA-95CA-C5AB0DC85B110123456789abcdefABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/upgradeWebsocketconnectionSec-WebSocket-AcceptGET HTTP/1.1 Host: Sec-WebSocket-Key: UpgradeConnectionUpgrade: websocket Sec-WebSocket-VersionConnection: Upgrade User-AgentSec-WebSocket-Version: 13 OriginUser-Agent: TinyWebsockets Client Origin: https://github.com/gilmaimon/TinyWebsockets HTTP/1.1 101httpws://wswss://wsshttps://https8F @xF @H�@�; @�C @�; @P�@���������������U @V @websocket13Sec-WebSocket-KeyHTTP/1.1 101 Switching Protocols Sec-WebSocket-Accept: W @
    intr_handle)/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/task_wdt.cesp_intr_free(twdt_config->intr_handle)esp_register_freertos_idle_hook_for_cpu(idle_hook_cb, i)esp_task_wdt_addesp_task_wdt_initcal_versioncal_macphy_initE (%d) %s: %s: calibration data MAC check failed: expected %

    1. WordBot says:

      Hi, I have no idea! Something wrong with the websocket but I don’t know if it’s the library, the browser or something in the Wi-Fi connection.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

scroll to top