Computer Vision: ArUco Detection & YOLOv8 Integration

A multi-stage computer vision pipeline combining camera calibration, ArUco marker detection, augmented reality projection, and real-time YOLOv8 object detection — all running from a standard webcam.

---

Features

• Camera intrinsic calibration using checkerboard patterns
• ArUco marker detection with 6-DoF pose estimation
• Augmented reality — projects 3D cubes (static and animated) onto detected markers
• Real-time marker path tracking with velocity and distance statistics
• YOLOv8 nano model integration for live object classification

Project Structure

AR_Yolo_Detection/
├── camera_calib.py        # Stage 1 — Camera calibration
├── aruco_detection.py     # Stage 2 — ArUco marker detection & pose estimation
├── ar_projection.py       # Stage 3 — AR cube projection onto markers
├── ar_yolo_tracking.py    # Stage 4 — Marker path tracking + YOLOv8 detection
├── camera_params.yaml     # Saved camera intrinsics (output of Stage 1)
├── calibration_result.jpg # Before/after distortion correction visualization
├── checkerboard_15mm_7x6.pdf  # Printable calibration target
└── markers/               # Generated ArUco marker images (marker_0 – marker_9)

Setup

Requirements

• Python 3.8+
• Webcam

Installation

pip install opencv-python ultralytics numpy pyyaml

The YOLOv8 nano weights (yolov8n.pt) are downloaded automatically by Ultralytics on first run.

Usage

Run the stages in order. Each stage builds on the output of the previous one.

Stage 1 — Camera Calibration

Print checkerboard_15mm_7x6.pdf, then capture calibration images from your webcam:

python camera_calib.py

Move the checkerboard in front of the camera at various angles. The script saves calibration parameters to camera_params.yaml.

Stage 2 — ArUco Marker Detection

Detect ArUco markers in a live webcam feed with 3D axis overlays and distance readout:

python aruco_detection.py

Print any of the markers in markers/ and hold them in front of the camera.

Stage 3 — AR Cube Projection

Project a 3D cube onto detected ArUco markers:

python ar_projection.py

Stage 4 — Path Tracking + YOLOv8

Track marker movement with fade-effect trails and run simultaneous YOLOv8 object detection:

python ar_yolo_tracking.py

Keyboard Controls

Key	aruco_detection.py	ar_projection.py	ar_yolo_tracking.py
q	Quit	Quit	Quit
s	Save screenshot	Save screenshot	Save screenshot
1	Static detection mode	—	—
2	Live video mode	—	—
y	—	—	Toggle YOLOv8 on/off
c	—	—	Clear path trails
p	—	—	Print path statistics

Calibration Results

Metric	Value
RMS Reprojection Error	0.1285 px
Mean Reprojection Error	0.0194 px
Focal Length (fx)	509.5 px
Focal Length (fy)	507.9 px
Principal Point (cx)	334.7 px
Principal Point (cy)	230.8 px
Calibration Images Used	30
Capture Resolution	640 × 480

A reprojection error below 0.5 px indicates a high-quality calibration.

Tech Stack

• OpenCV — camera calibration, marker detection, pose estimation, AR rendering
• Ultralytics YOLOv8 — real-time object detection
• NumPy — numerical computation
• PyYAML — camera parameter serialization