StereoSense

Complete flow for 3D reconstruction form stereo cameras

Conda installation

# clone project
git clone https://github.com/lus105/StereoSense.git
# change directory
cd StereoSense
# create conda environment
conda create --name StereoSense python=3.11
# activate conda environment
conda activate StereoSense
# install pytorch
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
# install requirements
pip install -r requirements.txt

Download resources

# navigate to root
cd StereoSense
# camera configuration files
gdown https://drive.google.com/drive/folders/1mpDP1PHbM-xMh-Wwjx0DMOjTdmwnsyYU?usp=drive_link --folder
# data (if crashes, try to download folder via link)
gdown https://drive.google.com/drive/folders/1p_mfaFk4_bPl_JFaC41rpq9V0vVR02iw?usp=drive_link --folder
# models
gdown https://drive.google.com/drive/folders/15HT1PC70Jcmr9OSBmgRcslL31b8EEJld?usp=drive_link --folder
# output (generated pointclouds)
gdown https://drive.google.com/drive/folders/1N9eJ53YJbRBLYQ5ItHnJdAaVqU10zd-i?usp=drive_link --folder

Note: for using complete flow (main.py), install basler pylon.

Instructions (use with basler cameras)

1. Gather calibration data with basler cameras: python src/stereo_grab_basler.py
2. Run stereo camera calibration: notebooks/1.0_Calibrate.ipynb . Change constants to your specific ones.
3. Download model and place inside models/ directory.
4. Run python main.py
5. Once the configs are loaded, press 'c' to capture frames. Results will be saved in output/ directory.

Instructions (use without camera)

1. Create your own camera calibration files (refer to notebooks/1.0_Calibrate.ipynb )
2. Download model and place inside models/ directory.
3. Grab sample images (left and right) and place inside data/samples directory.
4. Run notebooks/2.0_Stereo_inference.ipynb

Notes

The input size of the model is 800x640 (hxw).

Expected result

Resources

• Model used: https://github.com/NVlabs/FoundationStereo
• Point Cloud Viewer using OpenCV Viz: https://github.com/rytisss/anaglyph3D