Unlock a Door Using Face Detection and a Sonoff

ESP-EYE and Sonoff Basic

Learn how to re-program a Sonoff Basic to use on your WiFi network and control a device that is connected to mains electricity.

The Sonoff Basic is a WiFi enabled switch that can be used to remotely switch electrical devices. The ESP-WHO face detection library running on an ESP-EYE camera is used to detect a face and then send a signal over WiFi to activate the Sonoff.

This set up could be used to replace a motion sensor when you want to activate a device when a face is detected rather than only movement. It could be used to unlock a door or switch on a hall light when you don’t want movement from other things such as pets or wild animals to do the same. It would also work well when a person isn’t moving enough to trigger a motion sensor.

Updating the Sonoff Firmware

First the firmware needs to be replaced on the Sonoff with Tasmota to enable remote access. From the releases page download the sonoff-basic.bin version. You also need to download a tool to upload the bin file. Windows users should download the xxxx-x64.exe file from flasher tool page.

Use a USB to serial connector such as a CP2102 or FT232 module at 3.3v to connect as in the diagram below.

USB to Serial Connection for Sonoff Basic

As I only plan to connect to this once via USB I just used Dupont connectors poked through the holes in the circuit board. At no point should any power be connected to any other part of the device.

Sonoff Wiring With Dupont Connectors

Run the downloaded flasher tool and choose the following settings:

PyFlasher Settings

Browse to the downloaded sonoff-basic.bin binary file.

Browse Firmware

The Sonoff needs to be in be Flash Mode so you can upload to the device. To do this, hold the button down (the long black plastic thing) and plug the USB to Serial connector into your PC. After a few seconds you can release the button.

In the flasher tool, click the Flash NodeMCU button. After about 20 seconds you should see the message below. If it doesn’t work try again putting it into Flash Mode. Also check the wiring is correct and that the pins are making contact.

Console After Flashing Sonoff

Unplug the USB to serial connection from your PC and then reconnect it.

Adding the Sonoff to the WiFi Network

The Sonoff creates its own WiFi network (Access Point) so you can connect to it easily and configure it before adding it to the normal network. Look for the new network in your WiFi connections and connect your PC or phone.

Sonoff AP

If your browser doesn’t automatically open the configuration page, browse to http://192.168.4.1. You should see the Sonoff configurations screen below:

Sonoff Wifi Setup Screen

Scan for your actual WiFi network or just manually type in the information and click save.

The Sonoff can now be accessed on your normal WiFi network via its IP address. You can probably find this this in your router settings.

Sonoff in the Router

The admin screen looks like this:

Sonoff Admin Screen

Sonoff, Tasmota and Face Detection

The easiest way to control the Sonoff is to send commands as web requests similar to http://192.168.1.123/cm?cmnd=Power%20TOGGLE

In this tutorial we are only using face detection rather than face recognition so it’s a lot easier to set up than previous tutorials for controlling devices with a particular face.

If you haven’t set up or tested your camera before, you’ll need to set up the ESP32 in the Arduino IDE and test it works following this tutorial: https://robotzero.one/esp32-cam-arduino-ide/

If everything is OK, paste the following into a new Sketch, edit the WiFi connection details and the IP address of the Sonoff on your network. You also need to comment the correct camera. Save this as a new Sketch. Find the folder for this new Sketch and copy this file camera_index.h to the same folder. Upload it to your ESP32 Camera.

#include "esp_http_client.h"
#include <WiFi.h>
#include "esp_camera.h"
#include "Arduino.h"
#include "fd_forward.h"


const char* ssid = "NSA";
const char* password = "Orange";
char* sonoff_ip = "http://192.168.1.103";
long switch_time_out = 5000;

// Select camera model
//#define CAMERA_MODEL_WROVER_KIT
#define CAMERA_MODEL_ESP_EYE
//#define CAMERA_MODEL_M5STACK_PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE
//#define CAMERA_MODEL_AI_THINKER
#include "camera_pins.h"

long last_detected_millis = 0;
bool switch_status = 0;

static inline mtmn_config_t app_mtmn_config()
{
  mtmn_config_t mtmn_config = {0};
  mtmn_config.min_face = 80;
  mtmn_config.pyramid = 0.7;
  mtmn_config.p_threshold.score = 0.6;
  mtmn_config.p_threshold.nms = 0.7;
  mtmn_config.r_threshold.score = 0.7;
  mtmn_config.r_threshold.nms = 0.7;
  mtmn_config.r_threshold.candidate_number = 4;
  mtmn_config.o_threshold.score = 0.7;
  mtmn_config.o_threshold.nms = 0.4;
  mtmn_config.o_threshold.candidate_number = 1;
  return mtmn_config;
}
mtmn_config_t mtmn_config = app_mtmn_config();

void setup()
{
  Serial.begin(115200);
  Serial.setDebugOutput(true);
  Serial.println();

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  //init with high specs to pre-allocate larger buffers
  if (psramFound()) {
    config.frame_size = FRAMESIZE_UXGA;
    config.jpeg_quality = 10;
    config.fb_count = 2;
  } else {
    config.frame_size = FRAMESIZE_SVGA;
    config.jpeg_quality = 12;
    config.fb_count = 1;
  }

#if defined(CAMERA_MODEL_ESP_EYE)
  pinMode(13, INPUT_PULLUP);
  pinMode(14, INPUT_PULLUP);
#endif

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }

  sensor_t * s = esp_camera_sensor_get();
  s->set_framesize(s, FRAMESIZE_QVGA);

#if defined(CAMERA_MODEL_M5STACK_WIDE)
  s->set_vflip(s, 1);
  s->set_hmirror(s, 1);
#endif

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
}

static esp_err_t update_sonoff_status()
{
  char* full_sonoff_address = (char*)malloc(40);
  sprintf(full_sonoff_address, "%s/cm?cmnd=Power%%20TOGGLE", sonoff_ip);

  esp_http_client_config_t config = {
    .url = full_sonoff_address,
  };

  esp_http_client_handle_t client = esp_http_client_init(&config);
  esp_err_t err = esp_http_client_perform(client);

  if (err == ESP_OK) {
    ESP_LOGI(TAG, "Status = %d, content_length = %d",
             esp_http_client_get_status_code(client),
             esp_http_client_get_content_length(client));
  }
  esp_http_client_cleanup(client);
}

void loop()
{
  camera_fb_t * fb = NULL;
  dl_matrix3du_t *image_matrix = NULL;
  
  while (true) {

    fb = esp_camera_fb_get();
    image_matrix = dl_matrix3du_alloc(1, fb->width, fb->height, 3);

    fmt2rgb888(fb->buf, fb->len, fb->format, image_matrix->item);

    box_array_t *net_boxes = NULL;
    net_boxes = face_detect(image_matrix, &mtmn_config);

    if (net_boxes) {
      last_detected_millis = millis();
      if (switch_status == 0) { // switch is off
        Serial.println("Face Seen");
        update_sonoff_status();
        switch_status = 1;
      }
    }

    esp_camera_fb_return(fb);
    fb = NULL;
    dl_matrix3du_free(image_matrix);

    if (millis() - last_detected_millis > switch_time_out && switch_status == 1) { // No face detected and switch is on
      Serial.println("No Faces for switch_time_out milliseconds");
      switch_status = 0;
      update_sonoff_status();
    }

  }

}

When you run the Sketch and point the camera at a face and then point it away from the face you should hear the relay in the Sonoff click.

Wiring the Sonoff

To test it works when connected to mains electricity I used a desk lamp. I’ve replaced the normal lamp switch with the Sonoff device.

Sonoff and Switch

Remember, nothing should be connected to the mains when wiring the Sonoff device.

Sonoff Wired to Mains

Now you can plug in it into your mains electricity!

Video Demonstration

The video below demonstrates Sketch above. When the camera (bottom of the screen) sees a face, a signal (HTTP request) is sent to the Sonoff which toggles the power on. After 5 seconds, if no face is visible, the camera sends another request to the Sonoff which toggles the power off.

Further Guides

How to wire Sonoffs safely: https://youtu.be/8mz5sCAvDAY?t=9m23s
Tasmota install in much more detail: https://www.youtube.com/watch?v=IcOFeIcLFFo
One way of setting a static IP – https://youtu.be/UDnNI5wkNNY?t=11m21s

References

Hardware preparation: https://github.com/arendst/Sonoff-Tasmota/wiki/Hardware-Preparation
Flashing Tasmota: https://github.com/arendst/Sonoff-Tasmota/wiki/Flashing
Control commands: https://github.com/arendst/Sonoff-Tasmota/wiki/Commands#control
Espressif ESP-WHO library: https://github.com/espressif/esp-who

2 Replies to “Unlock a Door Using Face Detection and a Sonoff”

  1. Peter says:

    Hi, I watched some of your tutorials on esp32-cam which is my main board with which I am working right now and I just want to ask you a question if it is possible to hook up to this board sensors like RFID-RC522 or PN532 or all at the same time. The datasheet of esp32-cam mentions that it supports SPI,UART, I2C communication but I don’t seem to find the pins intended for these kind of communication. I don’t know who else to ask and any help would be appreciated. Thank you.
    ps: I guess uart is possible with the U0R U0T connections, not sure how to hook them up.

    https://loboris.eu/ESP32/ESP32-CAM%20Product%20Specification.pdf

    1. WordBot says:

      I couldn’t get anything to work as an input but I didn’t try connecting via serial to those two pins so maybe that’s possible. See this for more info: https://desire.giesecke.tk/index.php/2018/01/30/adc2-channel-cannot-be-used-when-wifi-is-in-use/

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

scroll to top