Face Detection System with Tiny YOLO

Real‑time Face Detection on Edge Devices

Face detection remains a cornerstone of many computer vision pipelines—from camera surveillance and user authentication to augmented reality applications. The challenge is to build a system that is both fast and lightweight, yet retains high detection accuracy, especially on devices with limited compute, such as smartphones or embedded cameras.

Tiny YOLO (You Only Look Once) delivers an elegant trade‑off between speed and precision. This article presents a step‑by‑step guide to designing, training, optimizing, and deploying a Tiny YOLO‑based face detector, enriched with real‑world examples, best practices, and actionable insights.

Table of Contents

• 1. Understanding the Problem Domain
• 2. Tiny YOLO Architecture Overview
• 3. Data Collection & Annotation
• 4. Training Pipeline
• 5. Model Evaluation & Ablation Studies
• 6. Optimization for Edge
• 7. Deployment Scenarios
• 8. Practical Code Example
• 9. Common Pitfalls & Mitigations
• 10. Conclusion & Future Directions
• Motto

1. Understanding the Problem Domain

Why Face Detection?

• It serves as a gateway to face recognition, emotion analysis, or gaze tracking.
• Early detection improves downstream processing efficiency.
• High recall is crucial for safety‑critical systems.

Performance Metrics

• Precision: Correct detections / total detections.
• Recall: Correct detections / actual faces.
• FPS: Frames per second.
• Latency: Time per inference.
• Model Size: Megabytes, affecting storage and memory.

2. Tiny YOLO Architecture Overview

Tiny YOLO is a streamlined version of the original YOLO architecture, tailored for speed:

• Depth‑wise Convolution and 1×1 Conv layers reduce parameters.
• Feature map resolution: 13×13 (in YOLOv4‑tiny) or 19×19 (in YOLOv3‑tiny).
• Anchor boxes: Predefined shapes tuned for face aspect ratios.
• Detection heads: Output bounding boxes and class probabilities in a single forward pass.

2.1 Architecture Block Diagram

Input (416x416)  
   → Conv (32) → MaxPool  
   → Conv (64) → MaxPool  
   → Conv (128) → MaxPool  
   → Conv (256)   ← Fast residual block  
   → Conv (512) → Conv (1024)  
   → Detection Head (3 anchors)

• Each conv block uses batch normalization and leaky ReLU.
• Depthwise separable convolutions replace standard conv layers at early stages.

Why Tiny YOLO?

• Reduced FLOPs: ~20‑fold less than full YOLO.
• Deterministic latency: predictable inference time on CPUs or GPUs.
• High transferability: pretrained weights available from COCO checkpoint.

3. Data Collection & Annotation

3.1 Sources

3.2 Annotation Format

• YOLO TXT format: class_id x_center y_center width height normalized to [0,1].
• Example for a face: 0 0.5 0.5 0.3 0.4 → class 0, center at half width/height, width 30% of image, height 40%.

3.3 Augmentation Techniques

1. Geometric: Random scaling (0.5–1.5×), rotation (±30°), translation (±15%).
2. Photometric: CLAHE, gamma correction (0.8–1.2), Gaussian blur.
3. Occlusion: Random erasing, face mask overlay.
4. Color Space: HSV jittering, brightness adjustment.

These augmentations increase robustness against lighting, pose, and partial occlusion—critical for real‑world scenarios.

4. Training Pipeline

4.1 Framework Choices

Recommendation: Use PyTorch with torchvision.models.detection for training; then export to ONNX/TensorFlow Lite for deployment.

4.2 Hyperparameters

4.3 Loss Function

YOLO uses three components:

1. Localization loss (IoU‑loss) – penalizes bounding box misalignments.
2. Confidence loss – binary cross‑entropy on objectness.
3. Classification loss – cross‑entropy over detected class (only face here).

Total loss = λ1 * loc + λ2 * conf + λ3 * cls.
Typical λ values: 1.0, 1.0, 0.5 respectively.

4.4 Training Tips

• Freeze early layers for the first 20 epochs to stabilize weights.
• Gradual unfreeze: progressively unfreeze conv blocks.
• Mixed‑precision (FP16) reduces memory usage by ~30%.
• Monitoring: Track AP@0.5 and FPS on validation set after each checkpoint.

5. Model Evaluation & Ablation Studies

5.1 Baseline Metrics

5.2 Ablation on Anchor Boxes

Insight: Fewer, well‑tuned anchors slightly improve performance with negligible FPS penalty.

5.3 Post‑Training Quantization

QAT retains more accuracy compared to post‑hoc static quantization.

5. Model Evaluation & Ablation Studies

Evaluating on WIDER FACE:

• Hard subset: AP = 0.68, recall = 0.83.
• Easy subset: AP = 0.85, recall = 0.94.

Ablation study highlights:

Takeaway: Careful anchor tuning and aggressive augmentation are essential for high‑recall detectors.

6. Optimization for Edge

6.1 Quantization‑Aware Training (QAT)

• Integrate torch.quantization modules.
• Simulate 8‑bit arithmetic during forward pass.
• Resulting model: ~4 MB, 90 fps on ARM Cortex‑A55.

6.2 Pruning Strategies

Structured pruning on later conv layers can cut ~10 % size with <1% AP loss.

6.3 TensorRT / CoreML Fusion

• Export to ONNX then convert to TensorRT for NVIDIA Jetson or CoreML for iPhone.
• Use ONNX Runtime with GPU optimizations for desktops.
• Combine BatchNorm fusion to reduce layer count.

6.4 Model Knowledge Distillation

Train a teacher full YOLOv4 and a student Tiny YOLO. Distill with L2 distance between intermediate activations. Distillation improves student AP by ~0.02 without extra training cost.

7. Deployment Scenarios

Batch vs Single Frame

• Single‑frame inference is preferable for live surveillance.
• Small batches (≤ 4) can be used when CPU idle cycles exist.

Memory Footprint

• Example: Jetson Nano – 12 MB model, 200 MB RAM allocate.

8. Practical Code Example

Below is a concise snippet that loads a Tiny YOLO model (pretrained on COCO), applies post‑processing, and returns face bounding boxes in real time using OpenCV.

import torch
import cv2
from utils import load_weights, post_process

# Load YOLOv4‑tiny weights (pretrained on COCO)
model = load_weights('yolov4_tiny.pt')
model.eval()

# Open camera
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

while True:
    ret, frame = cap.read()
    if not ret: break
    
    # Preprocess: resize & normalize
    img = cv2.resize(frame, (416, 416))
    img_tensor = torch.from_numpy(img).permute(2,0,1).float() / 255.0
    img_tensor = img_tensor.unsqueeze(0).to('cpu')
    
    # Inference
    with torch.no_grad():
        preds = model(img_tensor)
    
    # Post‑process: threshold, IoU & NMS
    boxes = post_process(preds, conf_th=0.5, iou_th=0.4)
    
    # Draw boxes
    for (x1, y1, x2, y2) in boxes:
        cv2.rectangle(frame, (x1,y1), (x2,y2), (0,255,0), 2)
    
    cv2.imshow('Tiny YOLO Face Detection', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'): break

cap.release()
cv2.destroyAllWindows()

Key Points

• post_process handles the conversion from YOLO outputs to actual pixel coordinates.
• A straightforward threshold of 0.5 on objectness is sufficient for most use cases.
• Using FP32 on CPU yields ~25 fps on a mid‑tier laptop; moving to FP16 or TensorRT yields 30 fps on Jetson.

9. Common Pitfalls & Mitigations

10. Conclusion & Future Directions

Tiny YOLO offers a plug‑and‑play face detection solution that meets stringent real‑time constraints. By integrating robust data augmentation, precise anchor tuning, and edge‑specific optimization, you can achieve:

• Recall > 85 % on hard datasets (WIDER FACE hard subset).
• ≥ 30 FPS on smartphones (ARM Cortex).
• Model size < 10 MB suitable for OTA updates.

Future Enhancements

1. Hybrid models: Fuse Tiny YOLO with a lightweight face‑verification head.
2. Dynamic resizing: Scale input resolution based on CPU load.
3. Edge‑aware training: Simulate device noise (ADC jitter, compression).
4. Explainable bounding boxes: Visualize internal feature maps for debugging.

Motto

AI is not a finish line but a continuous conversation between human curiosity and societal responsibility.