trade-ngin is a high-performance, modular quantitative trading system built in C++ designed for professional algorithmic traders and financial institutions. The system supports systematic trading strategies across multiple asset classes with a focus on futures, equities, and options markets.
- Multi-strategy portfolio management with dynamic capital allocation
- Comprehensive risk management with position limits, drawdown control, and VaR constraints
- High-performance backtesting engine with realistic execution simulation
- Transaction cost analysis for strategy optimization
- Modular architecture designed for extensibility and customization
- Professional-grade performance with C++ efficiency
trade-ngin is designed to handle both backtesting and live trading seamlessly, with a unified codebase that maintains consistency between simulation and production environments.
trade_ngin/
βββ apps/ # Application executables
β βββ backtest/ # Backtesting applications
β β βββ bt_trend.cpp # Trend following strategy backtest
β β βββ CMakeLists.txt # Build configuration for backtests
β βββ CMakeLists.txt # Build configuration for applications
βββ include/ # Public header files
β βββ trade_ngin/ # Main include directory
β βββ backtest/ # Backtesting components
β β βββ backtest_engine.hpp
β β βββ slippage_models.hpp
β β βββ transaction_cost_analysis.hpp
β βββ core/ # Core system components
β β βββ config_base.hpp
β β βββ config_manager.hpp
β β βββ config_version.hpp
β β βββ error.hpp
β β βββ logger.hpp
β β βββ state_manager.hpp
β β βββ types.hpp
β βββ data/ # Data management
β β βββ conversion_utils.hpp
β β βββ credential_store.hpp
β β βββ database_interface.hpp
β β βββ market_data_bus.hpp
β β βββ postgres_database.hpp
β βββ execution/ # Order execution
β β βββ execution_engine.hpp
β βββ instruments/ # Financial instruments
β β βββ equity.hpp
β β βββ futures.hpp
β β βββ instrument.hpp
β β βββ option.hpp
β βββ optimization/ # Portfolio optimization
β β βββ dynamic_optimizer.hpp
β βββ order/ # Order management
β β βββ order_manager.hpp
β βββ portfolio/ # Portfolio management
β β βββ portfolio_manager.hpp
β βββ risk/ # Risk management
β β βββ risk_manager.hpp
β βββ strategy/ # Strategy components
β βββ base_strategy.hpp
β βββ database_handler.hpp
β βββ regime_detector.hpp
β βββ strategy_interface.hpp
β βββ trend_following.hpp
β βββ types.hpp
βββ src/ # Implementation files
β βββ backtest/ # Backtesting implementations
β β βββ backtest_engine.cpp
β β βββ slippage_model.cpp
β β βββ transaction_cost_analysis.cpp
β βββ core/ # Core system implementations
β β βββ config_base.cpp
β β βββ config_manager.cpp
β β βββ config_version.cpp
β β βββ logger.cpp
β β βββ state_manager.cpp
β βββ data/ # Data management implementations
β β βββ conversion_utils.cpp
β β βββ credential_store.cpp
β β βββ market_data_bus.cpp
β β βββ postgres_database.cpp
β βββ execution/ # Order execution implementations
β β βββ execution_engine.cpp
β βββ instruments/ # Financial instruments implementations
β β βββ equity.cpp
β β βββ futures.cpp
β β βββ option.cpp
β βββ optimization/ # Portfolio optimization implementations
β β βββ dynamic_optimizer.cpp
β βββ order/ # Order management implementations
β β βββ order_manager.cpp
β βββ portfolio/ # Portfolio management implementations
β β βββ portfolio_manager.cpp
β βββ risk/ # Risk management implementations
β β βββ risk_manager.cpp
β βββ strategy/ # Strategy implementations
β βββ base_strategy.cpp
β βββ regime_detector.cpp
β βββ trend_following.cpp
βββ tests/ # Unit and integration tests
β βββ core/ # Core component tests
β β βββ test_config_base.cpp
β β βββ test_config_manager.cpp
β β βββ test_config_version.cpp
β β βββ test_logger.cpp
β β βββ test_result.cpp
β β βββ test_state_manager.cpp
β βββ CMakeLists.txt # Build configuration for tests
βββ CMakeLists.txt # Main build configuration
βββ config_template.json # Template for configuration files
trade-ngin follows a modular, component-based architecture with interfaces between system components. Here's a detailed breakdown of the major components:
- Defines custom error codes via
ErrorCodeenum - Implements
TradeErrorclass extendingstd::runtime_error - Provides
Result<T>template for error propagation throughout the system - Supports
make_error<T>()for creating standardized error responses
- Singleton implementation for system-wide logging
- Configurable log levels (TRACE, DEBUG, INFO, WARNING, ERROR, FATAL)
- Multiple output destinations (console, file, or both)
- Automatic log rotation based on file size
- Thread-safe implementation with mutex protection
- Tracks component states throughout the system
- Supports state transitions with validation
- Maintains component metrics and diagnostics
- Enables system-wide health monitoring
ConfigBase- Base class for serializable configurationsConfigManager- Centralized configuration managementConfigVersion- Version tracking and migration support- Environment-specific configuration overrides (dev, staging, prod, backtest)
- Abstract
DatabaseInterfacedefining common database operations - PostgreSQL implementation via
PostgresDatabase - Arrow-based data type support for efficient memory management
- Query building and parameter sanitization
- Publish-subscribe pattern for market data distribution
- Event-based architecture with callback registration
- Symbol and event type filtering
- Thread-safe implementation
- Arrow table to/from domain objects conversion
- Type-safe timestamp handling
- Error handling with Result pattern
- Abstract base class defining common instrument properties
- Asset-specific implementations:
EquityInstrument- Stocks, ETFsFuturesInstrument- Futures contractsOptionInstrument- Options with Greeks calculation
- Order lifecycle management (creation, submission, cancellation)
- Order book maintenance for status tracking
- Validation and error handling
- Commission calculation
- Algorithm-based order execution (TWAP, VWAP, PoV, etc.)
- Execution metrics tracking and analysis
- Position buffering to reduce unnecessary trading
- Custom algorithm extensibility
- Abstract interface for all trading strategies
- Lifecycle management (init, start, stop, pause, resume)
- Event handling (market data, executions, signals)
- Position and risk management
- Common implementation of strategy interface
- Position tracking and management
- Signal generation and execution
- Risk limit enforcement
TrendFollowingStrategy- Multi-timeframe trend following using EMA crossovers- Extensible framework for adding new strategies
flowchart TD
A[Receive market data] --> B{Enough history?}
B -->|No| C[Store price in history]
B -->|Yes| D[Calculate volatility]
D --> E[Calculate EMA crossovers for multiple timeframes]
E --> F[Generate raw forecasts]
F --> G[Scale forecasts by volatility]
G --> H[Combine forecasts with FDM]
H --> I[Calculate position size based on volatility-targeting formula]
I --> J{Use position buffering?}
J -->|Yes| K[Apply position buffer to reduce trading]
J -->|No| L[Use raw position]
K --> M[Update position]
L --> M
M --> N[Emit position signal]
subgraph Volatility Calculation
D1[Calculate returns] --> D2[Calculate EWMA std deviation]
D2 --> D3[Blend short and long-term volatility]
D3 --> D4[Apply volatility regime multiplier]
end
D -.-> D1
D4 -.-> D
subgraph Position Sizing
I1[Calculate target risk] --> I2[Apply IDM factor]
I2 --> I3[Scale by forecast strength]
I3 --> I4[Divide by price * volatility * point value]
end
I -.-> I1
I4 -.-> I
- Position-level and portfolio-level risk constraints
- Value at Risk (VaR) calculation
- Jump risk monitoring
- Maximum drawdown limits
- Leverage constraints
- Correlation risk monitoring
- Multi-strategy portfolio construction
- Dynamic capital allocation
- Position aggregation and netting
- Portfolio-level optimizations and constraints
- Historical market data simulation
- Event-driven architecture
- Realistic execution modeling
- Comprehensive performance metrics
- Transaction cost analysis
- Volume-based slippage
- Spread-based slippage
- Custom slippage model extensibility
- Execution quality evaluation
- Implementation shortfall calculation
- Benchmark comparison (VWAP, TWAP, arrival price)
- Cost breakdown (spread, market impact, timing, delay)
- Position optimization considering transaction costs
- Risk constraints enforcement
- Tracking error minimization
- Convex optimization techniques
classDiagram
class ConfigBase {
<<abstract>>
+save_to_file(filepath) Result~void~
+load_from_file(filepath) Result~void~
+to_json() json
+from_json(json) void
}
class StrategyConfig {
+capital_allocation double
+max_leverage double
+position_limits map
+max_drawdown double
+trading_params map
+to_json() json
+from_json(json) void
}
class PortfolioConfig {
+total_capital double
+reserve_capital double
+use_optimization bool
+use_risk_management bool
+to_json() json
+from_json(json) void
}
class RiskConfig {
+var_limit double
+jump_risk_limit double
+max_correlation double
+to_json() json
+from_json(json) void
}
class DynamicOptConfig {
+tau double
+capital double
+max_iterations int
+to_json() json
+from_json(json) void
}
class BacktestConfig {
+strategy_config StrategyConfig
+portfolio_config PortfolioConfig
+results_db_schema string
+to_json() json
+from_json(json) void
}
class DatabaseInterface {
<<interface>>
+connect() Result~void~
+disconnect() void
+is_connected() bool
+get_market_data(...) Result~Table~
+store_executions(...) Result~void~
+store_positions(...) Result~void~
+store_signals(...) Result~void~
+get_symbols(...) Result~string[]~
}
class PostgresDatabase {
-connection_string string
-connection pqxx::connection
+connect() Result~void~
+disconnect() void
+is_connected() bool
+get_market_data(...) Result~Table~
}
class StrategyInterface {
<<interface>>
+initialize() Result~void~
+start() Result~void~
+stop() Result~void~
+pause() Result~void~
+resume() Result~void~
+on_data(data) Result~void~
+on_execution(report) Result~void~
+on_signal(symbol, signal) Result~void~
+get_positions() Position[]
}
class BaseStrategy {
#id_ string
#config_ StrategyConfig
#positions_ map
#state_ StrategyState
+initialize() Result~void~
+on_data(data) Result~void~
+check_risk_limits() Result~void~
}
class TrendFollowingStrategy {
-trend_config_ TrendFollowingConfig
-price_history_ map
-volatility_history_ map
+on_data(data) Result~void~
-calculate_ewma(...) vector~double~
-get_raw_forecast(...) vector~double~
-calculate_position(...) double
}
class Instrument {
<<interface>>
+get_symbol() string
+get_type() AssetType
+is_tradeable() bool
+get_margin_requirement() double
+round_price(price) double
}
class SlippageModel {
<<interface>>
+calculate_slippage(...) double
+update(market_data) void
}
ConfigBase <|-- StrategyConfig
ConfigBase <|-- PortfolioConfig
ConfigBase <|-- RiskConfig
ConfigBase <|-- DynamicOptConfig
ConfigBase <|-- BacktestConfig
DatabaseInterface <|-- PostgresDatabase
StrategyInterface <|-- BaseStrategy
BaseStrategy <|-- TrendFollowingStrategy
SlippageModel <|-- VolumeSlippageModel
SlippageModel <|-- SpreadSlippageModel
-
Initialization
- Load configuration from files
- Initialize database connection
- Create strategy instances
- Initialize portfolio manager
- Configure risk management and optimization components
-
Market Data Loading
- Query historical data from database
- Convert to internal Bar representation
- Group data by timestamp for realistic simulation
-
Strategy Processing
- Feed market data to strategies chronologically
- Generate trading signals based on strategy logic
- Convert signals to position targets
-
Portfolio-Level Processing
- Aggregate positions across strategies
- Apply risk management constraints
- Apply position optimization if enabled
- Calculate trade sizes based on current vs. target positions
-
Order Execution Simulation
- Apply slippage models to execution prices
- Calculate transaction costs
- Generate execution reports
- Update positions based on fills
-
Performance Tracking
- Update equity curve
- Calculate drawdowns
- Track risk metrics over time
- Record trade statistics
-
Results Analysis
- Calculate performance metrics (Sharpe, Sortino, etc.)
- Analyze transaction costs
- Generate visualizations and reports
- Save results to database
sequenceDiagram
participant User as User
participant BE as BacktestEngine
participant DB as Database
participant PM as PortfolioManager
participant SI as Strategy
participant RM as RiskManager
participant DO as DynamicOptimizer
participant TCA as TCA
User->>BE: run_portfolio()
BE->>DB: load_market_data()
DB-->>BE: historical market data
loop For each timestamp
BE->>PM: process_market_data(bars)
PM->>SI: on_data(bars)
SI-->>PM: target_positions
opt If risk management enabled
PM->>RM: process_positions(positions)
RM-->>PM: risk_result
end
opt If optimization enabled
PM->>DO: optimize_single_period(positions)
DO-->>PM: optimized_positions
end
PM-->>BE: updated positions & executions
BE->>BE: apply_slippage(executions)
BE->>BE: calculate_transaction_costs(executions)
BE->>BE: update_equity_curve()
end
BE->>BE: calculate_metrics()
BE->>TCA: analyze_trade_sequence()
TCA-->>BE: transaction_cost_metrics
BE-->>User: backtest_results
-
Initialization
- Load configuration from files
- Initialize broker connections
- Create strategy instances
- Initialize portfolio manager
- Configure risk management and optimization components
-
Market Data Subscription
- Subscribe to real-time market data feeds
- Process incoming ticks/bars
- Update internal market data state
-
Strategy Processing
- Feed real-time data to strategies
- Generate trading signals based on strategy logic
- Convert signals to position targets
-
Portfolio-Level Processing
- Aggregate positions across strategies
- Apply risk management constraints
- Apply position optimization if enabled
- Calculate trade sizes based on current vs. target positions
-
Order Execution
- Submit orders to execution engine
- Select appropriate execution algorithm
- Route orders to broker/exchange
- Monitor order status and fills
-
Position Management
- Track current positions and exposures
- Calculate realized and unrealized P&L
- Enforce risk limits in real-time
-
Monitoring and Reporting
- Track system health and component states
- Generate performance reports
- Log diagnostics and metrics
- C++17 compatible compiler (GCC 8+, Clang 7+, MSVC 2019+)
- CMake 3.17 or higher
- Required libraries:
- nlohmann_json (for configuration handling)
- Arrow C++ (for data processing)
- libpqxx (for PostgreSQL connectivity)
- GoogleTest (for testing)
git clone https://github.com/your-organization/trade_ngin.git
cd trade_ngin- Create a PostgreSQL database for market data and strategy results
- Create the necessary schemas and tables
- Configure database credentials:
cp config_template.json config.json # Edit config.json with your database credentials
# Create build directory
mkdir build && cd build
# Configure CMake
cmake ..
# Build the library and applications
cmake --build . --config Release
# Run tests
ctest -C Release- Open the project folder in Visual Studio
- Select "Open CMake Project"
- Choose the desired configuration (Debug/Release)
- Build the solution
- Open the project folder in CLion
- CLion should automatically detect the CMake configuration
- Choose the desired build configuration
- Click "Build" to compile the project
# From the build directory
./bin/Release/bt_trendAll components in trade-ngin are configurable through JSON configuration files:
{
"strategy_config": {
"symbols": ["ES", "NQ", "CL", "GC"],
"asset_class": 0,
"data_freq": 0,
"data_type": "ohlcv",
"start_date": "1609459200",
"end_date": "1640995200",
"initial_capital": 1000000.0,
"commission_rate": 0.0005,
"slippage_model": 1.0,
"store_trade_details": true
},
"portfolio_config": {
"initial_capital": 1000000.0,
"use_risk_management": true,
"use_optimization": true,
"risk_config": {
"var_limit": 0.15,
"jump_risk_limit": 0.10,
"max_correlation": 0.7,
"max_gross_leverage": 4.0,
"max_net_leverage": 2.0,
"confidence_level": 0.99,
"lookback_period": 252,
"capital": 1000000.0
},
"opt_config": {
"tau": 1.0,
"capital": 1000000.0,
"asymmetric_risk_buffer": 0.1,
"cost_penalty_scalar": 10,
"max_iterations": 1000,
"convergence_threshold": 1e-6
}
},
"results_db_schema": "backtest_results"
}{
"capital_allocation": 1000000.0,
"max_leverage": 3.0,
"max_drawdown": 0.3,
"var_limit": 0.1,
"correlation_limit": 0.7,
"trading_params": {
"ES": 50.0,
"NQ": 20.0,
"CL": 1000.0,
"GC": 100.0
},
"position_limits": {
"ES": 100,
"NQ": 100,
"CL": 50,
"GC": 50
},
"save_executions": true,
"save_signals": true,
"save_positions": true
}To create a new strategy, follow these steps:
- Create a new header file in
include/trade_ngin/strategy/ - Create a new implementation file in
src/strategy/ - Inherit from
BaseStrategyor implementStrategyInterfacedirectly - Implement the required methods for your strategy logic
- Register your strategy with the portfolio manager
Example of a minimally viable strategy:
class MyStrategy : public BaseStrategy {
public:
MyStrategy(std::string id, StrategyConfig config, std::shared_ptr<DatabaseInterface> db)
: BaseStrategy(std::move(id), std::move(config), std::move(db)) {
// Initialize strategy-specific parameters
}
Result<void> on_data(const std::vector<Bar>& data) override {
// Call base class implementation first
auto base_result = BaseStrategy::on_data(data);
if (base_result.is_error()) return base_result;
// Implement your strategy logic here
// Generate signals and update positions
return Result<void>();
}
Result<void> initialize() override {
// Custom initialization logic
auto base_result = BaseStrategy::initialize();
if (base_result.is_error()) return base_result;
// Additional initialization steps
return Result<void>();
}
};To integrate a new data source:
- Implement a new class derived from
DatabaseInterface - Override all abstract methods to interact with your data source
- Implement conversion from your data source's format to trade-ngin's internal types
To add a new execution algorithm:
- Update the
ExecutionAlgoenum inexecution_engine.hpp - Implement a new method in
ExecutionEnginefor your algorithm - Update the
submit_executionmethod to handle your new algorithm
-
Code Style
- Follow the existing coding style and naming conventions
- Use snake_case for member variables and method names
- Use PascalCase for class names
- Use snake_case for file names
-
Error Handling
- Use the
Result<T>pattern for functions that can fail - Provide detailed error messages with component context
- Use the
-
Testing
- Write tests for all new functionality
- Ensure all tests pass before submitting a pull request
-
Documentation
- Document all public interfaces with clear descriptions
- Update relevant README sections for significant changes
ErrorCode::INVALID_ARGUMENT- Invalid parameter providedErrorCode::NOT_INITIALIZED- Component used before initializationErrorCode::DATABASE_ERROR- Database connection or query errorErrorCode::DATA_NOT_FOUND- Requested data not foundErrorCode::INVALID_DATA- Data validation failedErrorCode::CONVERSION_ERROR- Type conversion errorErrorCode::ORDER_REJECTED- Order rejected by the systemErrorCode::INSUFFICIENT_FUNDS- Insufficient capital for orderErrorCode::POSITION_LIMIT_EXCEEDED- Position limit exceededErrorCode::INVALID_ORDER- Invalid order parametersErrorCode::STRATEGY_ERROR- Error in strategy logicErrorCode::RISK_LIMIT_EXCEEDED- Risk limit exceeded
classDiagram
class Result~T~ {
-value_ T
-error_ unique_ptr~TradeError~
+Result(value T)
+Result(error unique_ptr~TradeError~)
+is_ok() bool
+is_error() bool
+value() T
+error() TradeError*
}
class TradeError {
-code_ ErrorCode
-component_ string
+TradeError(code, message, component)
+code() ErrorCode
+component() string
+to_string() string
}
class ErrorCode {
<<enumeration>>
NONE
UNKNOWN_ERROR
INVALID_ARGUMENT
NOT_INITIALIZED
DATABASE_ERROR
DATA_NOT_FOUND
INVALID_DATA
CONVERSION_ERROR
ORDER_REJECTED
INSUFFICIENT_FUNDS
POSITION_LIMIT_EXCEEDED
INVALID_ORDER
STRATEGY_ERROR
RISK_LIMIT_EXCEEDED
}
Result~T~ o-- TradeError
TradeError o-- ErrorCode
note for Result~T~ "Result<T>: Pattern for error propagation"
note for TradeError "TradeError: Extends std::runtime_error and provides error context"
If you encounter database connection issues:
- Verify your credentials in
config.json - Ensure the PostgreSQL server is running
- Check network connectivity and firewall settings
- Examine the error message from
PostgresDatabase::connect()
Example of handling database errors:
auto db = std::make_shared<PostgresDatabase>(connection_string);
auto connect_result = db->connect();
if (connect_result.is_error()) {
// Handle error case
std::cerr << "Database connection error: " << connect_result.error()->what() << std::endl;
std::cerr << "Error code: " << static_cast<int>(connect_result.error()->code()) << std::endl;
return 1;
}If a strategy fails to initialize:
- Check that the database connection is valid
- Verify that the strategy configuration is complete
- Look for specific error messages in the initialization chain
- Examine the strategy's
initialize()method
If backtests are running slowly:
- Reduce the number of symbols or date range
- Check for inefficient loops or calculations in strategies
- Optimize database queries to fetch data more efficiently
- Consider using release builds instead of debug builds
If you encounter memory issues:
- Look for resource leaks in components with manual memory management
- Ensure that destructors properly clean up allocated resources
- Check for memory-intensive operations in tight loops
- Consider using memory profiling tools like Valgrind
trade-ngin uses Apache Arrow for efficient in-memory data representation:
- Zero-copy data sharing between components
- Columnar memory layout for efficient processing
- Vectorized operations for performance
- Minimal memory allocations during data processing
- Components use thread-safe designs with mutex protection
- The market data bus implements a publish-subscribe pattern for efficient event distribution
- Strategy processing can be parallelized for multiple instruments
- RAII pattern throughout the codebase
- Smart pointers for automatic resource management
- Preallocated buffers for high-performance operations
- Limited heap allocations in performance-critical paths
To profile trade-ngin performance:
-
Build with profiling flags enabled
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_PROFILING=ON ..
-
Run with profiling tools
# Linux valgrind --tool=callgrind ./bin/bt_trend # Windows Visual Studio Profiler
- Classes: PascalCase (e.g.,
OrderManager) - Methods: snake_case (e.g.,
add_strategy) - Variables: snake_case (e.g.,
update_allocations) - Constants: UPPER_CASE (e.g.,
MAX_ORDERS) - Files: snake_case (e.g.,
order_manager.cpp)
- Use
Result<T>for functions that can fail - Avoid throwing exceptions across component boundaries
- Provide detailed error messages with context
- Check all return values for errors
Example of proper error handling:
Result<void> MyComponent::doSomething() {
auto result = dependentComponent_->operation();
if (result.is_error()) {
return make_error<void>(
result.error()->code(),
"Failed during operation: " + std::string(result.error()->what()),
"MyComponent"
);
}
// Continue with operation
return Result<void>();
}- Use smart pointers (
std::shared_ptr,std::unique_ptr) - Follow RAII principles
- Avoid raw
new/deleteoperations - Be mindful of ownership semantics
- Document all public interfaces
- Include parameter descriptions
- Document preconditions and postconditions
- Document error conditions and handling
trade-ngin is licensed under the GPL v3 License. See LICENSE file for details.
Third-party dependencies:
- nlohmann_json: MIT License
- Apache Arrow: Apache License 2.0
- libpqxx: BSD License
- GoogleTest: BSD 3-Clause License