Building a face recognition system to recognize the faces of my friends and welcome them to a party.
There’s a few ways of building a system like this. Possibly the simplest approach is using this library – https://github.com/ageitgey/face_recognition. I’m using a Raspberry Pi Zero W with the Pi Zero Camera for the face recognition and a cheap Bluetooth module and speaker to announce friends with a greeting when a face is recognized. If the face is known to the system than the persons name is included in the greeting.
The instructions below are a modified version of the instructions here: https://gist.github.com/ageitgey/1ac8dbe8572f3f533df6269dab35df65 with corrections from the comments and other tips I’ve read elsewhere.
The instructions are much easier to work with if you connect to the Pi Zero remotely over SSH. Read the tutorial Headless Install of Raspberry Pi OS Lite for help. There are a lot of commands and it’s much easier to paste them from your browser rather than type them directly.
Raspberry Pi OS LITE Installation
Download the Raspberry Pi Imager from here: https://www.raspberrypi.org/software/ and create a new OS LITE install on a microSD card.
Set up SSH access as above or access the Pi directly with a keyboard and HDMI monitor.
Run:
sudo raspi-config
and configure the basics:
- Enable the Raspberry Pi camera (under Interface Options)
- Set gpu memory split to 16 (under Performance Options)
Reboot and then run the following commands to update the Pi:
sudo apt-get update
sudo apt-get upgrade
Install the necessary software. I’m not sure everything below is needed but install it all anyway! You can just copy and paste the entire text below into the shell.
sudo apt-get install build-essential \
cmake \
gfortran \
git \
wget \
curl \
graphicsmagick \
libgraphicsmagick1-dev \
libatlas-base-dev \
liblapack-dev \
libatlas3-base \
libavcodec-dev \
libavformat-dev \
libboost-all-dev \
libgtk2.0-dev \
libjpeg-dev \
liblapack-dev \
libswscale-dev \
pkg-config \
python3-dev \
python3-numpy \
python3-pip \
zip
sudo apt-get clean
This might make the image processing faster:
sudo apt-get install libopenblas-dev
Camera support:
sudo apt-get install python3-picamera
sudo pip3 install --upgrade picamera[array]
dlib Installation
The face recognition uses the dlib library – https://github.com/davisking/dlib . The version of dlib in previous Pi OS installs was out of date and didn’t work with this project so had to be compiled from source.
However, now dlib can be installed with a simple command:
sudo pip3 install dlib
dlib Installation – compiling from source
If you still want to compile from source the following instructions should work.
First you need to change the swapfile setting so dlib will compile:
sudo nano /etc/dphys-swapfile
In the text editor that opens, comment the line CONF_SWAPSIZE=100 and uncomment the line CONF_SWAPFACTOR=2 so it looks like this:
Press ctrl-x and then y to save and exit nano. Run the following command to reinitialize the swapfile:
sudo /etc/init.d/dphys-swapfile restart
Download dlib. First check the releases page to get the latest version – https://github.com/davisking/dlib/releases:
mkdir -p dlib
git clone -b 'v19.16' --single-branch https://github.com/davisking/dlib.git dlib/
cd ./dlib
Compile and install dlib:
sudo python3 setup.py install --compiler-flags "-mcpu=arm1176jzf-s -mfpu=vfp"
Depending on the microSD card, compiling takes from about 5 hours to over 12 hours on the Pi Zero. If you are connecting over SSH and lose the connection you can run pip3 freeze
to check dlib was successfully installed:
Python Library Installation
Install the python face recognition library (https://github.com/ageitgey/face_recognition)
sudo pip3 install --no-cache-dir face_recognition
If you compiled dlib from source, revert the swap file settings now that dlib is installed:
sudo nano /etc/dphys-swapfile
Edit the file so it looks like this:
Reinitialize the swapfile again:
sudo /etc/init.d/dphys-swapfile restart
Examples and Quick Test
Download the face recognition code examples:
git clone --single-branch https://github.com/ageitgey/face_recognition.git
Run a quick test:
cd ./face_recognition/examples
python3 facerec_on_raspberry_pi.py
The test script above (you can see it here) loads an jpg of Barack Obama and encodes it.
obama_image = face_recognition.load_image_file("obama_small.jpg")
obama_face_encoding = face_recognition.face_encodings(obama_image)[0]
The script then runs a loop that takes a frame from the camera stream and looks for faces in that frame. If it finds a face it then checks the encoded version of the face against the encoded face of Obama and prints a message. If you hold up an image of Obama on your phone to the camera you should see the I see someone named Barack Obama message!
Multiple Faces
To add more than one person to the system, put one image per person in a folder named ‘friends’ and upload this to /home/pi/face_recognition/examples
. Each file should use the persons name as the filename.
Create a new file called facerec_on_raspberry_pi_group.py in the examples directory. Paste the following into the new file.
#new variables
known_person=[]
known_image=[]
known_face_encoding=[]
#Loops to add images in friends folder
for file in os.listdir("friends"):
try:
#Extracting person name from the image filename eg: david.jpg
known_person.append(file.replace(".jpg", ""))
file=os.path.join("friends/", file)
known_image = face_recognition.load_image_file(file)
known_face_encoding.append(face_recognition.face_encodings(known_image)[0])
except Exception as e:
pass
#change recognition loop to
while True:
print("Capturing image.")
# Grab a single frame of video from the RPi camera as a numpy array
camera.capture(output, format="rgb")
# Find all the faces and face encodings in the current frame of video
face_locations = face_recognition.face_locations(output)
print("Found {} faces in image.".format(len(face_locations)))
face_encodings = face_recognition.face_encodings(output, face_locations)
# Loop over each face found in the frame to see if it's someone we know.
for face_encoding in face_encodings:
match = face_recognition.compare_faces(known_face_encoding, face_encoding)
matches=np.where(match)[0] #Checking which image is matched
if len(matches)>0:
name = str(known_person[matches[0]])
else:
name = ""
print("I see someone named {}!".format(name))
Or download the entire script from here: facerec_on_raspberry_pi_group.zip unzip and upload it to the same directory.
If you run this new script (python3 facerec_on_raspberry_pi_group.py
) and point the camera at the same people you just uploaded to the friends folder the Pi should recognize them and print a message. If there are lots of people it takes a while before the system is ready.
To make the system a little more fun you can add Bluetooth audio output to your headless Pi so it announces the entrance of your friends to the party, For a full explanation of connecting a Bluetooth speaker and voice synthesis see this post – https://robotzero.one/pi-zero-w-bluetooth-voice/
You can then update the script by adding the following line (replace the DEV ID with your own):
os.system("espeak 'Hello {}' -ven+f5 -k5 -s150 --stdout | aplay -D bluealsa:DEV=7B:08:72:E8:DD:13,PROFILE=a2dp ".format(name))
above the line:
print("I see someone named {}!".format(name))
and the Pi Zero will speak the name of anybody it recognizes.
Live Test
My friends aren’t too keen about appearing on YouTube so I bought a Spanish celebrity magazine and selected some pages to use in a live test of this system. I found other photos of the same people from the internet to train the face recognition.
Here’s the guest list for my party:
All the images above are random images taken from the internet. I uploaded these to the friends folder and ran the facerec_on_raspberry_pi_group.py script. In most of the tests, only one of the people wasn’t recognized.
More Information
This project on YouTube – https://www.youtube.com/watch?v=2qDCxVRbge0
Pi compiler options: https://www.raspberrypi.org/forums/viewtopic.php?t=11629
Pre-built Pi Zero W Img Download
If you just want to quickly test whether this face recognition system will work on your Pi Zero W here’s a previous version created as a Raspbian image with everything installed so that you can flash it to a blank microSD card with Etcher and be running in 10 minutes. Download the img file from here: face-recognition-zero-w-img.zip. The username is pi and the password is raspberry.
Buy Me A Coffee
If you found something useful above please say thanks by buying me a coffee here...
Hi,
Great article.
Would a USB camera work with this project and if yes roughly what steps would I need to change?
Thanks
Rigg
Hi,
I had a look and it seems a little complicated. You need to get this part:
# Grab a single frame of video from the RPi camera as a numpy array
camera.capture(output, format=”rgb”)
working with the feed from the USB camera. Maybe this will help? https://github.com/gebart/python-v4l2capture
Tried for a bit to get this to work…probably worth mentioning that the camera orientation matters! If you’re capturing images upside down it doesn’t work 🙂
Thanks for the Pi image, it really helped a lot. Appreciated.
Hello, what are the Raspberry login and Password of the image?
Hi, Should be the default – login is pi and the password raspberry.
Hi.
Could you please help. This is failing to install for me: “Package libatlas-dev is not available, but is referred to by another package.”
pi@raspberrypi:~ $ sudo apt-get install build-essential \
> cmake \
> gfortran \
> git \
> wget \
> curl \
> graphicsmagick \
> libgraphicsmagick1-dev \
> libatlas-dev \
> libatlas-base-dev \
> liblapack-dev \
> libatlas3-base \
> libavcodec-dev \
> libavformat-dev \
> libboost-all-dev \
> libgtk2.0-dev \
> libjpeg-dev \
> liblapack-dev \
> libswscale-dev \
> pkg-config \
> python3-dev \
> python3-numpy \
> python3-pip \
> zip
Reading package lists… Done
Building dependency tree
Reading state information… Done
Package libatlas-dev is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
libatlas-base-dev
E: Package ‘libatlas-dev’ has no installation candidate
Hi, What happens if you remove libatlas-dev from the list?. The error suggests that libatlas-base-dev replaces it. I guess you are using Buster? You could try another version of Raspbian: http://downloads.raspberrypi.org/raspbian/images/raspbian-2019-04-09/ Maybe this will help as well: https://askubuntu.com/questions/1056167/how-to-install-atlas-on-kubuntu-18-04bionic
Yes, I was using Buster… switched to Stretch and all is working . Thanks
This was pretty simple to set up. I ran the Obama test and it worked great. Thanks! I was wondering if I wanted to see what the camera see’s by showing it on the monitor how would I do that?
The easiest way is probably installing one of the Pi desktops. I’ve not done that so I’m not sure.
I did step by step I work well, but the detection of the person took about 15 seconds, okay ?
Should take 2 – 3 seconds to recognise a face. How many people did you add to the ‘friends’ folder? Do you have good lighting for the camera?
Hey! Thank you for the great project!
Is it somehow possible to fire up a MQTT message whenever a face is detected, so that you could connect the face recognition to IOT?
That would be so great!
Thanks for your help 🙂
Yep. it should be pretty easy. Just add an MQTT library to the code and change this line to send the message: print(“I see someone named {}!”.format(name))
thanks for your message! I’ll try that out 🙂
Good evening, first of all congratulations for the project, it’s very beautiful and interesting! I’m having trouble connecting the bluetooth speaker and giving me this error when I try to test with “aplay -D bluealsa …”
The raspberry is updated and the bluetooth speaker is connected correctly.
Thank you so much
What error are you seeing?
hi may i know what face detection method are you using? Hog, MMOD, Har Cascade or DNN?
it uses dlib for the face recognition. I’m not sure the name of the algorithm but it should be somewhere in their documentation.
is it possible to connect your image over vnc??
i try to enable vnc from config but after a several downloads it ends with 404 not found
Sorry.. I don’t know about this at all.
Hi! I don’t have a Bluetooth module rn, Can I still use the image?
Hi, It should still all work but you might have to remove a few lines of code so it doesn’t look for Bluetooth connections.
I ran into multiple errors while trying to install the various files. Several were not there. I then tried multiple times to download your pre-built img file and have yet to get the full download to work. Always get a network error. So far this has been a total bust. Is there another site that you can place the img.zip so that I can get a successful download.
Thanks,
Ken
You can try this download: https://easyupload.io/1kwqo2 It should download from my site.. it’s hosted on one of the biggest hosting companies in the world.
Are you using http://downloads.raspberrypi.org/raspbian/images/raspbian-2019-04-09/ as the base to work with?
Hi I am getting following error
os.system(“espeak ‘Hello {}’ -ven+f5 -k5 -s150 –stdout | aplay -D
SyntaxError: EOL while scanning string literal
Can you help me
Hi,
That part of the tutorial is one line like this:
os.system(“espeak ‘Hello {}’ -ven+f5 -k5 -s150 –stdout | aplay -D bluealsa:HCI=hci0,DEV=7B:08:72:E8:DD:13,PROFILE=a2dp “.format(name))
Hello, do you have any idea to recognize pets with this script? Its for a university project and i have tested with my dog but it didn’t work. Thanks!
Hi, for that you need object recognition. I’m sure there will be tutorials on the internet for this.
Hello everyone will I have project to the lest year in the university, my project is about a door that will be open if he know the person and I will use the raspberry pi zero w starter and camera rasberry pi, so my question is how can make e connections between the raspberry and the door to open if he know the person or stay clos if he doesn’t?
Check this tutorial for how to wire a relay to your Pi (tutorial uses an ESP32-CAM but it’s the same). Basically you set a pin high on the Pi and the lock opens. Be careful with the voltages.. don’t attach the lock directly to the Pi.
Hello,
For most people the precompiled dlib package will work so we don’t have to wait 5 to 12 hours… just use pip:
sudo pip3 install dlib
No need to modify the swap file or compile dlib..
Also I was running into some RAM problems and had to install “face_recognition” with this option:
sudo pip3 install face_recognition –no-cache-dir
Hi, It’s possible that Debian have updated the version of dlib in the newer versions of Raspbian. Which version did you use?
I used:
Raspberry Pi OS (32-bit) Lite
Minimal image based on Debian Buster
Version:August 2020
Release date:2020-08-20
Kernel version:5.4
And pip3 installed dlib version: 19.21
Oh thanks.. the dlib version is a little newer in your install so I’ll have to update the tutorial to make it easier for people.
Tutorial has now been updated for Pi OS LITE. Thanks go to Jason for his tips.
I did not do it with camera, I used this code:
import face_recognition
known_image = face_recognition.load_image_file (“biden.jpg”)
unknown_image = face_recognition.load_image_file (“unknown.jpg”)
biden_encoding = face_recognition.face_encodings (known_image) [0]
unknown_encoding = face_recognition.face_encodings (unknown_image) [0]
results = face_recognition.compare_faces ([biden_encoding], unknown_encoding)
Retrieved from here: https://pypi.org/project/face-recognition/
Sometimes it takes 15 seconds, other times 25 seconds
It should be noted that the multi-sided example code that this website copied (robotzero) if it works fast as in 3 seconds.
Maybe the dimensions for biden.jpg is too large? Try making it around 400×600 pixels. I probably could be smaller than that.
Hello
What kind of data does it receive:
face_recognition.compare_faces (known_face_encoding, face_encoding)
What kind of data is it: known_face_encoding
?
Hi, I’m not sure exactly but it’s the face described using maths. I think a 3D matrix or something but I don’t really know too much about how the faces are stored.
Hello,
Can you build one for me? What would be the price?
Thanks!
Hi, Where are based? Could you buy the Pi and camera and I just send the image to copy over to a card?
Hi, I’m still quite a newb when it comes to code so I’m sorry if this is super obvious but I can’t seem to figure it out. When I create “facerec_on_raspberry_pi_group.py” what variables should I be updating (a screen shot of a working code would definitely help me understand what needs to be updated) as I receive this error:
Traceback (most recent call last):
File “facerec_on_raspberry_pi_group.py”, line 7, in
for file in os.listdir(“friends”):
NameError: name ‘os’ is not defined
HI, Have you seen the video on YouTube? Might help with your issue. Read the full description under the video for changes – https://www.youtube.com/watch?v=2qDCxVRbge0
I have the same error. How did you fix it?
You probably need to download the whole script – https://robotzero.one/wp-content/uploads/2018/10/facerec_on_raspberry_pi_group.zip it’s in the article
Hi, How are you? First of all Thanks a lot for the Great & Informative Article & as well as the Image. I am Working on a Project Named Contactless DoorBell using RaspberryPi 3B+ & PiCam along with USB Sound Card for Mic. What I doing is to Detect the Face after which there will be a Notification sent to Owner via Blynk which has Authority to Control Gate after seeing Image of anyone at the Door. Second is that there is a Microphone for few Authorised People at Home for Sending Specific Voice Notes to Pi to Open the Door. I am facing an Issue while doing this all. I have Installed v2018 of RPi Image of Category Stretch from Official RaspberryPi Downloads. While doing any Command for Installation of Pi Package using sudo apt-get install or sudo apt install & pi install there comes so many Errors, most of the Errors among those are 404 Not Found & other are of Directory/Module not available. I am trying to Install OpenCV & as well as some other Packages including DLib but Still getting these Issues. My Pi is Updated & Upgraded Several Times but Still no Solution I got. Can you please guide in this Regard?
Thanks a lot.
Regards,
Muhammad Umar
Hi, I can’t really help you with general installation stuff but it might be the sources are missing or have been removed if it’s an old version of the OS. Here’s a new project for basic voice activation on the Pi Zero – https://robotzero.one/raspberry-pi-zero-voice-controller/
https://robotzero.one/wp-content/uploads/2018/10/face-recognition-zero-w-img.zip
This download link is not working..Please fixed it.i needit very badly.
I will be gratefully to you.
I don’t know why that file has gone from the server and I don’t seem to have a local copy so you will have to generate a new installation following the instructions. It won’t take long.