Lens2Apparel

This repository accompanies the MSc thesis “Lens2Apparel: Precision Anthropometry from Ray-Ban Meta Smart Glasses.”
It provides a lightweight, three-stage pipeline that converts egocentric images from Meta Ray-Ban smart glasses into garment-relevant body measurements.

---

Pipeline at a glance

1. Camera Calibration & Fisheye Correction → estimate intrinsics K and distortions D for your specific lens and evironment.
2. Batch Undistortion → rectify a folder of fisheye images using your calibration parameters.
3. Pose & Measurement → detect 2D landmarks (MediaPipe Pose) and compute shoulder, waist, and leg measures in real world units (cm) via single-reference scaling.

---

Requirements

• Python 3.9+
• OpenCV (with fisheye; opencv-python ≥ 4.x)
• MediaPipe (mediapipe)
• NumPy (numpy)
• Matplotlib (matplotlib) — optional, for visualization

Install:

pip install -r requirements.txt

---

Step 1 - Camera Calibration & Fisheye Correction

Script: bash calibration.py

Adapted from 2017 article: https://medium.com/@kennethjiang/calibrate-fisheye-lens-using-opencv-333b05afa0b0

Data Collection

1. Print calibration checkerboard. The script sets bash square_size = 35.0 mm (edit this to match your printed checkerboard square size).
2. Mount onto a a flat and firm surface to ensure consistency.
3. Using smart glasses, take 10-15 images at a fixed distance and location from the scene.
4. Make sure checkerboard is in different location and positions focusing on the periphery where the fisheye effect is most severe.
5. Ensure resulting folder of images are in .png format and all the same dimension (ex. 3024x4032) to ensure accurate results. Sometimes the results from the egocentric camera changes the image size during conversion.

How it works:

Detects checkerboard inner corners using cv2.findChessboardCorners.

Accumulates 2D–3D correspondences and runs cv2.fisheye.calibrate.

Prints your image size DIM, intrinsic matrix K, and distortion coefficients D to the console.

Usage:

1. Put calibration.py in the same folder as your .png checkerboard images (or pass a path if you extend the script).
2. Run it from that folder: bash python src/calibration.py

Copy console output into Step 2.

---

Step 2 - Undistort a Folder of Images

Script: bash undistort.py

How it works:

Uses your DIM, K, D from Step 1.

Builds undistortion maps via cv2.fisheye.initUndistortRectifyMap.

Remaps each .png to create a rectified image.

Saves outputs to a new subfolder: ./undistorted/.

Prepare the script: Paste the printed values from Step 1 at the top of bash undistort.py:

DIM = (3024, 4032) # (width, height) – must match your images exactly
K = np.array([...])
D = np.array([...])

Usage:

python src/undistort.py /path/to/your/image_folder

Output:

A new folder: bash /path/to/your/image_folder/undistorted/ containing rectified .png files ready for measurement.

---

Step 3 - MediaPipe Pose Detection & Measurement

Script: bash measure.py

How it works:

1. Loads each undistorted .png, runs **MediaPipe Pose **(static image mode), and extracts the 33 landmarks.

*Screenshot from MediaPipe Pose Landmarker guide. Source: Google AI Edge, “Pose landmark detection guide” (2025).*

2. Takes pixel distances between the landmarks.

3. Uses shoulder to wrist measure as a single real-world reference (entered by you in cm) to convert pixel distances → cm. (Prompts)

Computes:

• Shoulder width: distance between left/right shoulders (landmarks 11 & 12).

• Waist width: distance between left/right hips (landmarks 23 & 24).

• Inseam: hip (23/24) to ankle (29/30), averaged when both sides available.

-> Aggregates and prints mean measurements across the folder.

Usage:

python src/measure.py /path/to/undistorted

Final Output:

================== Final Averages ==================
Mean Shoulder Width: 41.32 cm
[per-image shoulder values in cm]


Mean Waist Width: 33.87 cm
[per-image waist values in cm]


Mean Inseam: 76.45 cm
[per-image inseam values in cm]

Tips:

• Use consistent posture (e.g., relaxed T-pose) with shoulders/wrists/hips/ankles visible.
• The script uses a visibility threshold 0.5; low-visibility landmarks are skipped.
• If shoulder→wrist is missing on one side, the script falls back to the other.

---

Troubleshooting

• “All images must share the same size.” Re-export .HEIC → .PNG with rotation/cropping disabled; ensure identical dimensions.

• Few/No landmarks detected. Improve lighting, reduce motion blur, ensure full body is in frame; optionally tweak the visibility threshold.

• Undistortion looks odd (stretched edges). Re-run calibration with more varied checkerboard angles, verify CHECKERBOARD inner-corner count, and confirm square_size.

---

Attribution

Fisheye calibration approach adapted from: Kenneth Jiang (2017), Calibrate Fisheye Lens using OpenCV. Source: https://medium.com/@kennethjiang/calibrate-fisheye-lens-using-opencv-333b05afa0b0