In this post I want to present my modest contribution to the war against the COVID19, the virus that has been changing the way we live for almost a year now.
In my latest post I explained how to connect a FLIR Lepton 3 thermal camera to a NVIDIA® Jetson™ Nano to acquire thermal images. With this post I want to explain how they can be used for an useful application for this strange period.
The “Jetson Fever Control” is an application that detects the 3D position of people, calculates the body temperature of each of them and emits an alarm if the nearest one has a temperature above 37.5°C, the well know fever threshold for COVID19 screening.
I added to the system a Stereolabs ZED2 3D camera to detect people and retrieve their 3D position.
Thanks to the powerful ZED SDK provided together with the ZED2 it is not necessary to train any neural network to be able to detect people, extract their bounding boxes and skeletons and track their position in the space, really straightforward C++ and Python APIs are available for this scope.
The HW prototype
To design and test the application I used the following components:
- 1x Nvidia Jetson Nano 2GB devkit
- 1x Flir Lepton3 sensor
- 1x GroupGets FLIR breakout board v1.4
- 1x prototype PCB board as FLIR carrier board
- 1x Stereolabs ZED2
- 1x tripod for ZED2
ZED2 and Nvidia Jetson Nano 2GB are an amazing couple, they seem made for each other. The ZED2 requires CUDA to extract depth information and to run deep learning algorithms to detect and track people. The Nvidia devices of the Jetson family are the only embedded boards that provide CUDA capabilities and the Jetson Nano 2GB costs only 59$.
The only doubt that can arise concerns the reduced amount of RAM: 2GB seem too few to run deep learning algorithms, but the tests that I made confirm that they are instead enough.
A swap disk is required to start the application, but as you can see in the video above, not all the 2GB RAM amount is used while the application is running.
The memory usage information are displayed below in the JTOP tool made by my friend Raffaello Bonghi: 1.2GB of CPU RAM, 557.4MB of GPU RAM for a total of 1.8GB.
Assembly
The FLIR Lepton 3 has been fixed above the left camera sensor of the ZED2 in order to horizontally align the RGB image of the ZED2 with the thermal image
This assembly is really rough, but it has the only scope to test the software. For a stable configuration a 3D printed case can be easily realized to integrate and keep all the components stably together.
Cable connections
The ZED2 can be connected easily to the Jetson Nano 2GB using the USB3 connector.
The FLIR Lepton 3 uses the I2C bus and the SPI bus to communicate with the host. A complete description of the connections is available in my previous post.
The software
The application has been written in Qt5 and the code is available on Github in my Lepton3_Jetson repository in the folder qt_check_fever
The application is designed to run on every Nvidia Jetson boards, not only on Nvidia Jetson Nano 2GB.
Graphic User Interface
The user interface is really simple. While normally running it displays the RGB image from the ZED2 camera. When the face of a person is detected, an overlay is drawn on it with colorized thermal information. Gray pixels are for all the thermal data that are not in the standard human body temperature, i.e. [34.5°C, 42°C]; green pixels are used for normal temperatures below 37°C; warning orange pixels are used for temperatures in the range [37°C, 37.49999999°C]; alarm red pixels are used for temperatures above 37.5°C, over the COVID fever threshold.
The faces of the tracked persons are highlighted by a bounding box, with information about the mean measured temperature and the distance from the camera.
Normal 
Warning 
Alert
Above the live view the temperature and the distance of the nearest person are displayed and a clear alarm status is available.
System calibration
To correctly draw the termal information from the Flir sensor over the RGB frames from the ZED2 a calibration is required. The FLIR Lepton 3 has a redured field of view respect to the ZED2 and a vertical offset is always present, it is not possible to mechanically match the optical axis of the two sensors.
For all these reasons the GUI can be expanded to show all the calibration controls.
In the top of the calibration interface the full grayscale thermal image from the FLIR Lepton 3 module is displayed.
A control to add a few celsius degrees to the body temperature and check that all the alarms are fully working is available just below the thermal image.
In the bottom a set of controls is available to modify the horizontal and vertical offsets and the scale of the thermal overlay image.
In the middle a checkbox is provided to enable the visualization of the skeleton of all the detected persons. This is useful to check how well people can be detected and tracked.
Conclusions
Thank you for your interest in my project, please do not hesitate to contact me using one of my social accounts to ask for more information or to request new features.
See you to the next project… and stay safe.
