This is a simple game/ framework for demonstration purpose that I designed for CS7675. SQL-DQN is an interactive educational game that combines SQL learning with reinforcement learning and LLM in a electable themed environment. This game adapts SQL puzzle level based on player's mastery using a Deep Q-Network (DQN) reinforcement learning agent, and embeds every challenge in engaging narratives generated by LLM.
- Adaptive Learning: DQN agent selects appropriate puzzles based on player performance, namely, mastery of SQL concepts.
- Interactive SQL Challenges: Players write SQL queries against a thematic database.
- Diverse Concepts: Tasks range from basic SELECT statements to complex joins.
- Narrative Experience: Narrative elements are personalized, each player will unfold a unique storyline as they process SQL concepts.
- Extensible Framework: Support for custom themes, storylines, and database schemas.
- Backend: TypeScript, Node.js, Express
- Database: PostgreSQL
- Machine Learning: TensorFlow.js for DQN implementation
- LLM: DeepSeek API for generating narratives
- API: RESTful endpoints for game state management
Note: This project requires a PostgreSQL database and a frontend.
- Clone the repository and install the dependencies:
npm install- Create a
.envfile in the root directory with:
DB_USER=your_postgres_user
DB_PASSWORD=your_postgres_password
DB_HOST=your_postgres_host_address
DB_PORT=your_postgres_port
DB_DATABASE=your_postgres_database_name
DB_URL=postgresql://your_postgres_user:your_postgres_password@your_postgres_host_address:your_postgres_port/your_postgres_database_name
PORT=your_desired_port-
Create a PostgreSQL database and run the SQL scripts to set up the required schemas. You can use the provided SQL schemas below to create the necessary tables and insert sample data.
-
Run the server using
npm run start.
sql-dqn/
├── package.json # Project dependencies and scripts
├── tsconfig.json # TypeScript configuration
├── .env # Environment variables (DB credentials)
├── .gitignore # Git ignored files
│
├── src/ # Source code
│ ├── index.ts # Application entry point
│ │
│ ├── agent/ # Reinforcement learning components
│ │ ├── DQNAgent.ts # Deep Q-Network agent implementation
│ │ ├── QNetwork.ts # Neural network model for Q-learning
│ │ └── ReplayBuffer.ts # Experience replay memory
│ │
│ ├── environment/ # Game environment
│ │ └── SQLEnvironment.ts # Environment for SQL game interactions
│ │
│ ├── resources/ # Data and query resources
│ │ ├── data_generator.py # Python script to generate training data
│ │ └── generated_data.csv # Pre-generated training data
│ │
│ ├── server/ # Backend server code
│ │ ├── server.ts # Express server setup
│ │ ├── controllers/ # Request handlers
│ │ │ ├── setup.controller.ts # Initialize game/agent
│ │ │ └── query.controller.ts # Process SQL queries
│ │ ├── middleware/ # Express middleware
│ │ │ └── error.middleware.ts # Error handling
│ │ └── routes/ # API routes
│ │ └── api.routes.ts # Route definitions
│ │
│ ├── services/
│ │ └── agent.service.ts # Agent initialization services
| | ├── database.service.ts/ # Database connection and models
│ │ ├── training.service.ts # Agent training services
│ │
│ ├── types/ # Type definitions
│ │ └── index.ts # TypeScript interfaces
│ │
└── README.md # Project documentation- Check why the DQN agent will choose the same action once when the mastery is equal to 1.
- Check the other two themes
- External API to give informative error message
- Add more tasks to cybernetics’s case
- File upload part make it take in a file and parse it
- Add a user authentication for:
- Storing all previous narratives
- Summarize the narratives at the end and generate a report on the mastery of SQL concepts
- Different UIs for different themes
- Add a mapping service to avoid direct interactions with real databases
- Write tests, fix bugs and check for edge cases.
-
residuecolumn_name data_type name VARCHAR (50) status VARCHAR (50) name status Neo PotentialRebel Trinity PotentialRebel Morpheus Captain Jane Doe null Smith EliminationProtocol -
archivescolumn_name data_type mission_id INT mission_name VARCHAR (50) mission_description TEXT mission_id mission_name mission_description 1 Free The Mind An attempt to awaken humanity. 2 Locate The Key-maker Securing the Keymaker for the Source. 3 Defend Zion Protect the last human city from Sentinels. 4 Rescue Operator An operator has gone missing in the field. 5 Eliminate Virus Suspected virus detected within the Matrix code. -
mission_logscolumn_name data_type mission_name VARCHAR(50) agent_id INT reference TEXT mission_name agent_id reference Free The Mind 101 Operation started, Morpheus leads Free The Mind 101 Strange glitch observed in downtown Defend Zion 103 Sentinel swarm approaching main gate Eliminate Virus 102 Agent Smith anomaly flagged for investigation -
multi_agent_eventscolumn_name data_type agent_id INT timestamp TIMESTAMP location VARCHAR(50) agent_replication BOOLEAN agent_id timestamp location agent_replication 101 2023-01-01 09:15:00 Downtown FALSE 101 2023-01-01 09:30:00 Downtown FALSE 102 2023-01-01 10:00:00 Rooftop TRUE 102 2023-01-01 10:15:00 SubwayStation TRUE 103 2023-01-01 10:30:00 NULL FALSE
-
ringscolumn_name data_type name VARCHAR(50) profession VARCHAR(50) name profession Aragorn Ranger Frodo RingBearer Gandalf Wizard Gollum NULL Legolas ElvenArcher Samwise Gardener Bilbo RingBearer Sauron DarkLord -
chroniclescolumn_name data_type quest_name VARCHAR(50) quest_description TEXT quest_name quest_description Protect the Ring Ensure the One Ring never falls into evil hands. Assemble the Fellowship Gather heroes from across Middle‑earth. Defend Minas Tirith Hold the White City against Sauron’s armies. Crown the King Place the rightful king upon the throne. Restore the Shire Rebuild the Hobbits' homeland after the war. -
quest_logscolumn_name data_type quest_name VARCHAR(50) hero_id INT reference TEXT quest_name hero_id reference Protect the Ring 2 Frodo sets out from the Shire. Protect the Ring 6 Samwise pledges never to leave Frodo’s side. Assemble the Fellowship 3 Council of Elrond chooses nine companions. Assemble the Fellowship 5 Legolas swears loyalty to Aragorn. Defend Minas Tirith 3 Gandalf rallies the defenders at the gate. Defend Minas Tirith 1 Aragorn arrives with the Army of the Dead. Crown the King 1 Aragorn crowned Elessar Telcontar. Restore the Shire 2 Hobbits return to drive out the ruffians. -
battle_summariescolumn_name data_type hero_id INT timestamp TIMESTAMP location VARCHAR (50) victory BOOLEAN hero_id timestamp location victory 1 2025‑03‑05 09:15:00 PelennorFields TRUE 2 2025‑03‑05 09:30:00 MountDoom TRUE 4 2025‑03‑05 09:45:00 MountDoom NULL 3 2025‑03‑05 10:00:00 NULL TRUE 5 2025‑03‑05 10:15:00 HelmsDeep TRUE 6 2025‑03‑05 10:30:00 Shire FALSE
-
moviescolumn_name data_type movie_name VARCHAR (50) genre VARCHAR (50) movie_name genre Inception Sci-Fi La La Land Musical The Godfather Crime The Matrix Sci-Fi Finding Nemo Animation Unknown Project NULL The Lord of the Rings Fantasy -
reviewscolumn_name data_type movie_name VARCHAR (50) review_text TEXT rating INT movie_name review_text rating Inception “A mind‑bending blockbuster.” 9 Inception “Too complex on first watch.” 7 The Matrix “Classic cyber‑action that still holds up.” 9 The Matrix “Bullet‑time changed cinema forever.” 10 La La Land “Musical romance with modern flair.” 8 The Godfather “An offer you can’t refuse.” 10 Unknown Project “Still in production…” NULL -
actorscolumn_name data_type movie_name VARCHAR (50) user_id INT movie_name user_id Inception 101 Inception 102 The Matrix 103 The Matrix 104 The Matrix 105 La La Land 106 The Godfather 107 Finding Nemo 108 Unknown Project 109 -
collectionscolumn_name data_type movie_name VARCHAR (50) timestamp TIMESTAMP shared BOOLEAN movie_name timestamp shared The Matrix 2025‑03‑05 09:15:00 TRUE Inception 2025‑03‑05 09:30:00 TRUE La La Land 2025‑03‑05 10:00:00 FALSE The Matrix 2025‑03‑05 10:15:00 FALSE Finding Nemo 2025‑03‑05 10:45:00 TRUE Unknown Project 2025‑03‑05 11:00:00 NULL