Structure from Motion (SfM)

📖 Project Overview

This project demonstrates Structure from Motion (SfM), a computer vision technique that reconstructs 3D structures from a sequence of 2D images or video frames. The goal is to generate a 3D point cloud representing the scene, leveraging feature detection, matching, and optimization techniques.

Our approach uses modern feature detectors like SuperPoint, feature matchers such as FlannBasedMatcher, and robust optimization techniques to refine 3D reconstruction. By calibrating the camera and processing the images, we create point clouds that visualize the scene's spatial structure.

---

🛠️ Approach

1. Feature Extraction: Detect keypoints in images using detectors like SIFT or SuperPoint.
2. Feature Matching: Match corresponding keypoints between consecutive images using BFMatcher or FlannBasedMatcher.
3. Camera Calibration: Use the provided calibration matrix for precise measurements.
4. Structure Reconstruction: Solve for camera poses and triangulate points to generate the 3D structure.
5. Optimization (Optional): Refine the reconstruction using techniques like bundle adjustment for accuracy.

---

🔍 Example Results

Below are some examples in the procces:

Feature detection of a toy duck using SIFT

Feature matching of a toy duck using BFM

---

🚀 How to Use

1. Shoot a video of an object or take pictures of it. If you have video, then go to frame_extractor.ipynb and preprocess the frames as you wish.
2. Follow the instructions in the presentation.ipynb file

⚙️ Libraries and Tools Used

For image processing, feature detection, and matching.

For numerical operations and data handling.

For visualizing features and results.

Used for least squares optimization.