3.4 KiB
CV-Project-1-BUSSIERE-BEAUD
This project has an objective of determine the heartrate of a person by analysing a 1 minute long video taken by a computer camera.
FIRST STEP - Get the video
To get the video, we need a normal camera ( like a laptop camera ) and record a ~1min long video of ourselves, in this case : Rémi. Then, we use the ffmpeg package* to convert this video into a 15fps, 640x480 video.
*sudo apt install ffmpeg on debian distribution
SECOND STEP - Get the ROI position
The ROI ( Region of interest ) is important here because we only need the average color of the person's head, the surrounding is considerate here as noise.
To get the ROI, we use a simple solution delivered by OpenCV* in python called haarcascades.
*pip install opencv-python
The haarcascades function will return 4 coordinates that will surround the person's head. The haarcascades is a pre-train model of machine learning developped for OpenCV where you can find here : https://github.com/opencv/opencv/tree/master/data/haarcascades
THIRD STEP - Calculate the average color
Then, we have the coordinates, and we compute the average between each red composents of each pixel inside the square, and we repeat it for blue and green. Then, we have as an output one color/frame.
You can find a picture of the haarcascades algotithm in the picture section.
We then export a .txt file with all the average color to use it in an octave code.
FOURTH STEP - Heart rate determination
Concerning this step, we decided to did it in octave, and it is devided into 5 main steps :
- Firstly, we have to extract the data from the txt file which is the output of the python code. The data in this file are actually the red, green and blue average of each sample of the 1 min video. In order to do that, We use functions to open, extract and close the file and store the data into 3 different vectors : R, G and B.
- Secondly, we normalize those three vectors by substracting the mean value of each and dividing by there standard deviation. Normalizing data aims to put into a standard form. It ensures that all of our normalized data have an average equal to zero and a standard deviation of 1.
- Thirdly, we use the algorithm FastICA (Independent Component Analysis) in order to transform our matrix of our normalized values (each row being each vector) into a new matrix composed of three channels of 450 elements each.
- Then, we compute the magnitude of the Fast Fourrier Transfrom of the new data and identify the index of the maximum of each channels so that we can relate it with its corresponding frequency.
- Finally, we display the maximum of those three frequencies multiplied by 60 so that we convert Hertz into BPM.
Conclusion & Results
To conclude, the output of our code is 8 BPM which is obviously false. There must be an error into the main code at the frequency identifying part. However, considering the fact that we struggled with FastICA, there might be an error into the algoritm that we don't understand for the moment.
POSSIBLE ALTENATIVES
In order to find the best possible ROI, we need some other algorithm than haarcascades. So that's why we used the mediapipe* library, wich is a way better algorithm, it's also a pre-trained model but much more complete. We can see here a picture of the results using the mediapipe library, where the ROI is more precise.
*pip install mediapipe more informations here : https://github.com/google/mediapipe