Doux

Doux is a research project on data storage systems for real-time analytics, containing implementations and optimizations of multiple LSM-tree storage engines.

Project Structure

This project contains the following main components:

📁 leveldb

Core implementation directory, containing our main methods:

• Doux: Decoupling Values from Keys for Real-Time Analytics - the core method for decoupling values from keys to enable real-time analytics
• LevelDB (SI): Secondary index implementation based on LevelDB
• RISE: Other related optimization methods

For detailed instructions on how to run and use these methods, please refer to leveldb/README.md.

📁 NEXT

Implementation of the NEXT paper - LSM-based Secondary Index framework implementation based on RocksDB.

NEXT is a new LSM-based secondary index framework that utilizes a two-level structure to improve non-key attribute query performance. For more information, please refer to NEXT/README.md.

📁 rocksdb

Standard RocksDB implementation - A persistent key-value store engine developed by Facebook.

RocksDB is a high-performance embedded key-value storage library, especially suited for flash and RAM storage environments. For more information, please refer to rocksdb/README.md.

Quick Start

Clone the Project

Since this project uses Git submodules, please clone with:

git clone --recursive https://github.com/YangHarvey/doux.git

If you have already cloned the project, you need to initialize the submodules:

git submodule update --init --recursive

Using LevelDB Implementation

Navigate to the leveldb directory and refer to leveldb/README.md for instructions on how to compile and run Doux, LevelDB (SI), RISE, and other methods.

Using NEXT Implementation

Navigate to the NEXT directory and refer to NEXT/README.md for instructions on how to compile and run the NEXT framework.

Using RocksDB

Navigate to the rocksdb directory and refer to rocksdb/README.md for instructions on how to compile and use RocksDB.

Submodules

This project uses Git submodules to manage dependent repositories:

• NEXT: Points to YangHarvey/NEXT
• rocksdb: Points to YangHarvey/rocksdb-10.1.3

License

Please refer to the LICENSE files in each subdirectory for license information of each subproject.

Related Papers

• Doux: Decoupling Values from Keys for Real-Time Analytics
• NEXT: A New Secondary Index Framework for LSM-based Data Storage (SIGMOD 2025)

Contributing

Issues and Pull Requests are welcome!