DiveXplore is an open-source software designed for interactive video retrieval. It has successfully competed in various international competitions, such as the "Video Browser Showdown (VBS)" and "Interactive Video Retrieval 4 Beginners (IVR4B)". After developing diveXplore since 2012, we decided to make diveXplore open-source for others to contribute to the software or use components of it in their own video retrieval systems.
This section of the project handles all data preprocessing. The processed data is then utilized by the middleware and frontend of diveXplore to deliver a seamless user experience.
To get the backend up and running, please follow the steps below.
Ensure you have the following software installed and running on your system:
- FFMPEG
- MongoDB (by default, the settings look for MongoDB running on "mongodb://localhost:27017". If that's not correct, change it in the combine_analysis_files.py, the create_texts_mongodb.py, and the local-config files)
- NPM v18+
- Angular CLI
Additionally, make sure your firewall does not block the necessary ports (8000, 8001) (e.g., sudo ufw allow 8000)
1. Clone the repository
git clone https://github.com/klschoef/divexplore.git2. Install the required Python dependencies:
Optional - Create a new Python environment and activate it:
python3 -m venv env
source env/bin/activateThen install the required dependencies:
pip install -r requirements.txtOnce the Setup is complete, you can start analyzing your videos.
1. Video Analysis
This step initiates the video processing pipeline. The provided shell script will automatically execute all the necessary analysis scripts in the recommended order. The analysis has to be performed for every video file you want to include in your system. From the root directory of the project, execute the following commands:
cd backend/analysis
chmod +x process_video.sh
./process_videos.sh [path_to_video]2. OpenCLIP Feature Extraction
After the initial analysis, this script generates OpenCLIP features for each keyframe that was extracted. Execute this script, when you finished processing all of your videos.
cd backend/analysis
python3 extract_openclip_image_features.py3. Database Integration
The final step is to collect all the generated information and add it to your MongoDB instance (make sure that the MongoDB service is running on localhost; otherwise, change the URLs in the following Python scripts and the local-configs in the middleware/frontend).
cd backend
python3 combine_analysis_files.py output/video_fps.txt output/scenes/ output/ocr/ output/asr/ output/summaries/
python3 create_texts_mongodb.pyYou have now successfully analyzed all your videos and populated the database!
The easiest way to host the keyframes/summaries/videos etc. is to move the videos to a folder /videos inside of backend/output/ or create a symlink to the videos from within backend/output/ with ln -s [video_path] videos. After that, host a simple server with Python from within the /output directory:
python3 -m http.serverIf you`re using your own hosting service, change the config files in the back-/frontend accordingly.
This section handles the FAISS index server, as well as the NodeJS server.
1. Start the FAISS Index Server
cd middleware/FAISS
chmod +x start_clipserver.sh
./start_clipserver.sh2. Start NodeJS server In order to configure the NodeJS server, you have to create a local-config.js in the middleware's root directory. An example local config (local-config-example.json) is in the directory. Just copy this file and remove the "-example" from the name.
cd middleware/Node
npm i
npm startThis section handles the frontend setup, as well as the connection with the DRES servers.
In order to launch the frontend, the DRES-relevant TypeScript files have to be generated. Additionally, like in the middleware, a config file has to be created.
These files are needed to allow for a connection to the DRES servers, used during competitions like the VBS and IVR4B. Execute these one after the other to set up the necessary prerequisites (You may need to execute this as SU, depending on your file permissions):
cd frontend
npm install @openapitools/openapi-generator-cli -g
npm install -g ng-openapi-genThen, generate the TypeScript files with these commands:
cd frontend
npm run-script gen-dres-client
npm run-script gen-dres-dev-clientTo connect the frontend to the backend, create a local-config.ts file under src/app/shared/config/. In this directory is a local-config-example.ts. You can just duplicate this file and rename it accordingly. If you followed the previous steps as described, the example config should work as is. Otherwise, you can change the settings by clicking the "Wheel" icon in the frontend or by changing the local-config accordingly.
Make sure you have installed NPM v18+ and the Angular CLI.
cd frontend
npm install
npm startRun ng serve or npm start to start the frontend server. Using the default configurations, diveXplore will launch on http://localhost:4200/. If you change any of the source code, the application will automatically reload.
All contributions are welcome! To contribute:
| 1. Fork the repository |
| 2. Create a new branch for your feature or bugfix |
| 3. Submit a pull request |