Repository navigation
2 Face Tracker
After we found faces in image , It's time to track faces.
If you have ever played with OpenCV face detection, you know that it works in real time and you can easily detect the face in every frame. So, why do you need tracking in the first place? Let’s explore the different reasons you may want to track objects in a video and not just do repeated detections.
1. Tracking is faster than Detection
Usually tracking algorithms are faster than detection algorithms. The reason is simple. When you are tracking an object that was detected in the previous frame, you know a lot about the appearance of the object. You also know the location in the previous frame and the direction and speed of its motion. So in the next frame, you can use all this information to predict the location of the object in the next frame and do a small search around the expected location of the object to accurately locate the object. A good tracking algorithm will use all information it has about the object up to that point while a detection algorithm always starts from scratch. Therefore, while designing an efficient system usually an object detection is run on every nth frame while the tracking algorithm is employed in the n-1 frames in between. Why don’t we simply detect the object in the first frame and track subsequently? It is true that tracking benefits from the extra information it has, but you can also lose track of an object when they go behind an obstacle for an extended period of time or if they move so fast that the tracking algorithm cannot catch up. It is also common for tracking algorithms to accumulate errors and the bounding box tracking the object slowly drifts away from the object it is tracking. To fix these problems with tracking algorithms, a detection algorithm is run every so often. Detection algorithms are trained on a large number of examples of the object. They, therefore, have more knowledge about the general class of the object. On the other hand, tracking algorithms know more about the specific instance of the class they are tracking.
2. Tracking can help when detection fails
If you are running a face detector on a video and the person’s face get’s occluded by an object, the face detector will most likely fail. A good tracking algorithm, on the other hand, will handle some level of occlusion. In the video below, you can see Dr. Boris Babenko, the author of the MIL tracker, demonstrate how the MIL tracker works under occlusion.
3. Tracking preserves identity
The output of object detection is an array of rectangles that contain the object. However, there is no identity attached to the object. For example, in the video below, a detector that detects red dots will output rectangles corresponding to all the dots it has detected in a frame. In the next frame, it will output another array of rectangles. In the first frame, a particular dot might be represented by the rectangle at location 10 in the array and in the second frame, it could be at location 17. While using detection on a frame we have no idea which rectangle corresponds to which object. On the other hand, tracking provides a way to literally connect the dots!
From Object Tracking using OpenCV
Here is a example of BOOSTING tracker in OpenCV , face_follower :
import cv2
import dlib
camera = cv2.VideoCapture(0)
detector = dlib.get_frontal_face_detector()
old_faces = []
while True:
ret, image = camera.read()
if not ret:
break
image = cv2.resize(image, (0, 0), fx=0.5, fy=0.5)
faces = detector(image, 1)
if len(old_faces) < len(faces):
old_faces = []
for face in faces:
tracker = cv2.Tracker_create('BOOSTING')
box = (face.left(), face.top(), face.width(), face.height())
tracker.init(image, box)
old_faces.append(tracker)
else:
for i, tracker in enumerate(old_faces):
ok, bbox = tracker.update(image)
if ok:
cv2.rectangle(image, (int(bbox[0]), int(bbox[1])), (int(bbox[0] + bbox[2]), int(bbox[1] + bbox[3])),
(100, 200, 100))
else:
old_faces.pop(i)
cv2.imshow("image", image)
key = cv2.waitKey(1) & 0xFF
if key == ord('q'):
break
camera.release()
cv2.destroyAllWindows()In this example , If you run it , And move your face fastly , moving in depth or many other motions , Tracker will fail !
After dlib 18.13 Davis King implemented a tracker method , This was a method described in the paper:
Danelljan, Martin, et al. "Accurate scale estimation for robust visual tracking." Proceedings of the British Machine Vision Conference BMVC. 2014.
We use correlation tracker in our Robot ( We named it Robina ) but why we choose it instead of OpenCV trackers ?
Correlation tracker has these features :
- Fast
- Trustworthy
- High accuracy ( winner of VOT2014 )
Here is an example of correlation tracker implemented in dlib :
import cv2
import dlib
camera = cv2.VideoCapture(0)
detector = dlib.get_frontal_face_detector()
old_faces = []
while True:
ret, image = camera.read()
if not ret:
break
image = cv2.resize(image, (0, 0), fx=0.5, fy=0.5)
faces = detector(image, 1)
if len(old_faces) < len(faces):
old_faces = []
for face in faces:
tracker = dlib.correlation_tracker()
tracker.start_track(image, face)
old_faces.append(tracker)
else:
for i, tracker in enumerate(old_faces):
quality = tracker.update(image)
if quality > 7:
pos = tracker.get_position()
pos = dlib.rectangle(
int(pos.left()),
int(pos.top()),
int(pos.right()),
int(pos.bottom()),
)
cv2.rectangle(image, (pos.left(), pos.top()), (pos.right(), pos.bottom()),
(100, 200, 100))
else:
old_faces.pop(i)
cv2.imshow("image", image)
key = cv2.waitKey(1) & 0xFF
if key == ord('q'):
break
camera.release()
cv2.destroyAllWindows()Now , Time to see differences :
| Dlib correlation tracker | OpenCV BOOSTING tracker |
|---|---|
 |
 |
You can read about Adaptive Correlation Filters on this paper.