Market-Maker-Sim: Double DQN for Closing Auctions

Market-Maker-Sim is a high-fidelity quantitative trading environment that utilizes Double Deep Q-Learning (DDQN) to optimize market-making strategies. This project specifically addresses the transition between continuous intraday trading and the terminal Closing Auction, implementing a robust framework for liquidity provision under realistic constraints.

🚀 Key Features

• Double DQN (DDQN) Agent: Implements decoupled selection and evaluation to eliminate overestimation bias, leading to more stable and realistic quoting strategies.
• Institutional Realism: Includes Transaction Costs (Slippage/Commissions) and Inventory Risk Penalties to simulate the friction of live markets.
• Advanced Market Dynamics: Generates training data via the Rough Heston stochastic volatility model, replicating empirical "jagged" market behavior and volatility clustering.
• Comprehensive Risk Analytics: Automated calculation of Sharpe Ratio, Sortino Ratio, and Maximum Drawdown (Max DD) across multiple S&P 500 assets.

📈 Performance & Evaluation

The agent was trained for 1200 episodes in a "High-Intensity" Rough Heston environment. The following results represent the agent's performance in Exploitation Mode ($\epsilon=0$) on real-world historical data, accounting for transaction costs ($0.01%$).

📈 Multi-Asset Statistical Results

The agent was trained for 1,200 episodes in a high-intensity Rough Heston environment to ensure convergence and robust policy development. The following table represents the agent's performance in Exploitation Mode ($\epsilon=0$) across synthetic, stress-test, and real-world scenarios, accounting for a 1 basis point ($0.01%$) transaction cost.

Source	Mean ($)	Std ($)	Sharpe	Sortino	Max DD ($)
Simulated (RH)	$158.21	$77.24	2.05	N/A*	$37.44
STRESS TEST (CRASH)	$252.73	$406.57	0.62	6.06	$211.45
Historical (MSFT)	$191.92	$26.64	7.20	N/A*	$0.00
Historical (GOOGL)	$179.92	$25.00	7.20	N/A*	$0.00
Historical (TSLA)	$177.90	$31.47	5.65	N/A*	$0.00
Historical (AMZN)	$180.63	$27.22	6.64	N/A*	$0.00

*Sortino ratios marked N/A represent zero-loss evaluation windows where downside deviation was statistically zero.

⚠️ Note on Realism & Future Implementation

While these results demonstrate the efficacy of the Double DQN architecture, the exceptionally high Sharpe Ratios (e.g., >7.0) reflect an idealized single-agent environment. In a live institutional setting, these metrics would naturally normalize due to several "invisible" market frictions that are currently in development for this project:

• Competitive Latency: The current model assumes instant execution. Real-world HFT involves microsecond-level race conditions where slower orders (like ours) miss profitable fills.
• Adverse Selection: The simulation assumes fills are random. In live markets, fills often occur because an informed trader knows the price is about to move against you ("Toxic Flow").
• Market Impact: Currently, the agent can liquidate inventory without moving the mid-price. A future update will implement a Permanent Market Impact model to penalize large-volume dumps.
• Order Queue Position: The simulator assumes we are at the front of the queue. Real matching engines (FIFO) require waiting behind other market makers (Simulated via Order Book depth).

Key Observations

• Elite Generalization: The agent achieved Sharpe Ratios > 2.0 on assets like TSLA and GOOGL, significantly outperforming the training baseline.
• Robustness to Friction: Despite the introduction of transaction costs, the agent maintained profitability, proving the efficacy of the DDQN's conservative value estimation.
• Zero-Loss Performance: The agent successfully utilized the closing auction to liquidate positions, resulting in near-zero drawdowns on historical S&P 500 data.

🛠 Project Structure

• src/environment.py: Custom Gymnasium environment with transaction costs and inventory risk shaping.
• src/dqn_agent.py: PyTorch implementation of the Double DQN architecture.
• src/utils.py: Stochastic path generation using the Rough Heston model.
• src/data_loader.py: Real-world data ingestion via yfinance.
• src/main.py: Unified pipeline for training, evaluation, and statistical reporting.

💻 Getting Started

Installation

1. Clone the repo:

   git clone [https://github.com/YOUR_USERNAME/market-making-sim.git](https://github.com/YOUR_USERNAME/market-making-sim.git)
   cd market-making-sim

2. Setup Environment:

   python -m venv venv
   .\venv\Scripts\Activate  # Windows
   pip install -r requirements.txt

Usage

Run the training and evaluation suite:

python -m src.main

Citations

Graf, J., & Mastrolia, T. (2026). Learning Market Making with Closing Auctions. arXiv:2601.17247 [q-fin.TR]. https://arxiv.org/abs/2601.17247