Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@ A Fabric mod that implements a Model Context Protocol (MCP) server, enabling AI

## Overview

This mod creates an HTTP server within the Minecraft client that accepts MCP protocol requests, allowing Large Language Models to execute Minecraft commands safely and efficiently. The mod includes comprehensive safety validation to prevent destructive operations.
This mod creates an HTTP server within the Minecraft client or dedicated server that accepts MCP protocol requests, allowing Large Language Models to execute Minecraft commands safely and efficiently. The mod includes comprehensive safety validation to prevent destructive operations.

It is designed to be fully compatible with both Single-Player (Integrated Server) and Multiplayer Dedicated Servers.

## Features

- **Server and Client Support**: Works on both single-player and dedicated server environments.
- **MCP Protocol Support**: Full implementation of Model Context Protocol for AI interaction
- **Safety Validation**: Comprehensive command filtering and validation system
- **Asynchronous Execution**: Non-blocking command execution to maintain game performance
Expand All @@ -34,9 +37,17 @@ This mod creates an HTTP server within the Minecraft client that accepts MCP pro

The MCP server starts automatically when you launch Minecraft with the mod installed. By default, it runs on `localhost:8080`.

### Server vs Client Modes

The mod detects if it is running in a Client (Single Player) or a Dedicated Server environment:
- **Client Mode**: Full feature support, including the `take_screenshot` tool, which uses the local game window.
- **Dedicated Server Mode**: Has access to tools like `execute_commands`, `get_player_info`, and `get_blocks_in_area`, enabling full AI manipulation of the world without rendering. The `take_screenshot` tool is disabled in server mode since there is no rendering context. Note that `get_player_info` currently selects the first online player on the server to report its location.

If playing Single Player, the integrated server logic runs through the client-side MCP.

### Configuration

The mod creates a configuration file at `config/mcp-client.json`:
The mod creates a configuration file at `config/mcp.json`:

```json
{
Expand Down Expand Up @@ -267,7 +278,7 @@ Capture a screenshot of the current Minecraft game screen. Optionally, you can s

For debugging purposes, you can enable local saving of every screenshot captured by the MCP server.

1. Open `config/mcp-client.json`.
1. Open `config/mcp.json`.
2. Set `"save_screenshots_for_debug": true` in the `client` section.
3. Screenshots will be saved to the `mcp_debug_screenshots/` directory in your Minecraft instance folder.
4. Files are named using the pattern: `screenshot_YYYYMMDD_HHMMSS_SSS.png`.
Expand Down
8 changes: 7 additions & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ loom {
}
}

runs {
server {
source sourceSets.client
}
}

}

dependencies {
Expand Down Expand Up @@ -95,4 +101,4 @@ publishing {
// The repositories here will be used for publishing your artifact, not for
// retrieving dependencies.
}
}
}
12 changes: 9 additions & 3 deletions src/client/java/cuspymd/mcp/mod/MCPServerModClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,18 @@ public void onInitializeClient() {
String transport = config.getServer().getTransport();

if ("http".equals(transport)) {
httpServer = new HTTPMCPServer(config);
httpServer = new HTTPMCPServer(config,
new cuspymd.mcp.mod.command.CommandExecutor(config),
new cuspymd.mcp.mod.utils.PlayerInfoProvider(),
new cuspymd.mcp.mod.utils.BlockScanner(),
new cuspymd.mcp.mod.utils.ScreenshotUtils(),
true
);
httpServer.start();
LOGGER.info("HTTP MCP Server started on port {}", httpServer.getPort());
} else {
// Default to stdio transport
ipcServer = new IPCServer(config);
ipcServer = new IPCServer(config, new cuspymd.mcp.mod.command.CommandExecutor(config));
ipcServer.start();
LOGGER.info("IPC Server started on port {}", ipcServer.getPort());
}
Expand All @@ -52,4 +58,4 @@ public void onClientShutdown() {
LOGGER.info("HTTP MCP Server stopped");
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
import java.util.concurrent.TimeoutException;
import java.util.concurrent.TimeUnit;

public class CommandExecutor {
public class CommandExecutor implements cuspymd.mcp.mod.command.ICommandExecutor {
private static final Logger LOGGER = LoggerFactory.getLogger(CommandExecutor.class);
private static final long COMMAND_MESSAGE_WAIT_MS = 700L;
private static final long COMMAND_MESSAGE_IDLE_MS = 120L;
Expand Down
19 changes: 9 additions & 10 deletions src/client/java/cuspymd/mcp/mod/utils/BlockScanner.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,15 @@
import java.util.ArrayList;
import java.util.List;

public class BlockScanner {
public class BlockScanner implements cuspymd.mcp.mod.utils.IBlockScanner {
private static final Logger LOGGER = LoggerFactory.getLogger(BlockScanner.class);

public static JsonObject scanBlocksInArea(JsonObject fromPos, JsonObject toPos, int maxAreaSize) {
@Override
public JsonObject scanBlocksInArea(JsonObject fromPos, JsonObject toPos, int maxAreaSize) {
return scanBlocksInAreaStatic(fromPos, toPos, maxAreaSize);
}

public static JsonObject scanBlocksInAreaStatic(JsonObject fromPos, JsonObject toPos, int maxAreaSize) {
try {
MinecraftClient client = MinecraftClient.getInstance();
if (client.world == null) {
Expand Down Expand Up @@ -71,7 +76,7 @@ public static JsonObject scanBlocksInArea(JsonObject fromPos, JsonObject toPos,
result.add("area", areaInfo);

// Scan blocks
List<JsonObject> blockList = new ArrayList<>();
List<BlockCompressor.BlockData> blockList = new ArrayList<>();
int totalBlocks = 0;

for (int x = minX; x <= maxX; x++) {
Expand All @@ -85,13 +90,7 @@ public static JsonObject scanBlocksInArea(JsonObject fromPos, JsonObject toPos,
continue;
}

JsonObject blockInfo = new JsonObject();
blockInfo.addProperty("x", x);
blockInfo.addProperty("y", y);
blockInfo.addProperty("z", z);
blockInfo.addProperty("type", blockState.getBlock().toString());

blockList.add(blockInfo);
blockList.add(new BlockCompressor.BlockData(x, y, z, net.minecraft.registry.Registries.BLOCK.getId(blockState.getBlock()).toString()));
totalBlocks++;
}
}
Expand Down
9 changes: 7 additions & 2 deletions src/client/java/cuspymd/mcp/mod/utils/PlayerInfoProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@
import net.minecraft.util.math.Vec3d;
import net.minecraft.world.World;

public class PlayerInfoProvider {
public class PlayerInfoProvider implements cuspymd.mcp.mod.utils.IPlayerInfoProvider {

public static JsonObject getPlayerInfo() {
@Override
public JsonObject getPlayerInfo() {
return getPlayerInfoStatic();
}

public static JsonObject getPlayerInfoStatic() {
MinecraftClient client = MinecraftClient.getInstance();
ClientPlayerEntity player = client.player;
World world = client.world;
Expand Down
9 changes: 7 additions & 2 deletions src/client/java/cuspymd/mcp/mod/utils/ScreenshotUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,15 @@
import java.util.List;
import java.util.concurrent.CompletableFuture;

public class ScreenshotUtils {
public class ScreenshotUtils implements cuspymd.mcp.mod.utils.IScreenshotUtils {
private static final Logger LOGGER = LoggerFactory.getLogger(ScreenshotUtils.class);
static final List<DeferredTask> pendingDeferredTasks = Collections.synchronizedList(new ArrayList<>());

@Override
public CompletableFuture<String> takeScreenshot(JsonObject params) {
return takeScreenshotStatic(params);
}

static class DeferredTask {
Runnable runnable;
int remainingTicks;
Expand All @@ -43,7 +48,7 @@ static class DeferredTask {
* @param params JsonObject containing optional x, y, z, yaw, pitch
* @return A CompletableFuture that completes with the Base64 encoded PNG image data
*/
public static CompletableFuture<String> takeScreenshot(JsonObject params) {
public static CompletableFuture<String> takeScreenshotStatic(JsonObject params) {
MinecraftClient client = MinecraftClient.getInstance();
CompletableFuture<String> future = new CompletableFuture<>();

Expand Down
59 changes: 59 additions & 0 deletions src/main/java/cuspymd/mcp/mod/MCPServerModServer.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package cuspymd.mcp.mod;

import net.fabricmc.api.DedicatedServerModInitializer;
import net.fabricmc.fabric.api.event.lifecycle.v1.ServerLifecycleEvents;
import cuspymd.mcp.mod.bridge.HTTPMCPServer;
import cuspymd.mcp.mod.bridge.IPCServer;
import cuspymd.mcp.mod.config.MCPConfig;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class MCPServerModServer implements DedicatedServerModInitializer {
public static final Logger LOGGER = LoggerFactory.getLogger("mcp-server-mod");
private HTTPMCPServer httpServer;
private IPCServer ipcServer;

@Override
public void onInitializeServer() {
LOGGER.info("Initializing Minecraft MCP Dedicated Server");

ServerLifecycleEvents.SERVER_STARTED.register(server -> {
try {
MCPConfig config = MCPConfig.load();
if (config.getServer().isAutoStart()) {
String transport = config.getServer().getTransport();

if ("http".equals(transport)) {
httpServer = new HTTPMCPServer(config,
new cuspymd.mcp.mod.server.tools.ServerCommandExecutor(config, server),
new cuspymd.mcp.mod.server.tools.ServerPlayerInfoProvider(server),
new cuspymd.mcp.mod.server.tools.ServerBlockScanner(server),
new cuspymd.mcp.mod.server.tools.ServerScreenshotUtils(),
false
);
httpServer.start();
LOGGER.info("HTTP MCP Server started on port {}", httpServer.getPort());
} else {
// Default to stdio transport
ipcServer = new IPCServer(config, new cuspymd.mcp.mod.server.tools.ServerCommandExecutor(config, server));
ipcServer.start();
LOGGER.info("IPC Server started on port {}", ipcServer.getPort());
}
}
} catch (Exception e) {
LOGGER.error("Failed to start MCP Server", e);
}
});

ServerLifecycleEvents.SERVER_STOPPING.register(server -> {
if (ipcServer != null) {
ipcServer.stop();
LOGGER.info("IPC Server stopped");
}
if (httpServer != null) {
httpServer.stop();
LOGGER.info("HTTP MCP Server stopped");
}
});
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
import cuspymd.mcp.mod.command.CommandExecutor;
import cuspymd.mcp.mod.command.ICommandExecutor;
import cuspymd.mcp.mod.config.MCPConfig;
import cuspymd.mcp.mod.server.MCPProtocol;
import cuspymd.mcp.mod.utils.PlayerInfoProvider;
import cuspymd.mcp.mod.utils.BlockScanner;
import cuspymd.mcp.mod.utils.ScreenshotUtils;
import cuspymd.mcp.mod.utils.IPlayerInfoProvider;
import cuspymd.mcp.mod.utils.IBlockScanner;
import cuspymd.mcp.mod.utils.IScreenshotUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -31,14 +31,33 @@ public class HTTPMCPServer {
private static final Gson GSON = new Gson();

private final MCPConfig config;
private final CommandExecutor commandExecutor;
private final ICommandExecutor commandExecutor;
private final IPlayerInfoProvider playerInfoProvider;
private final IBlockScanner blockScanner;
private final IScreenshotUtils screenshotUtils;
private final boolean screenshotToolEnabled;
private final AtomicBoolean running = new AtomicBoolean(false);
private HttpServer httpServer;
private ExecutorService executor;

public HTTPMCPServer(MCPConfig config) {
public HTTPMCPServer(MCPConfig config, ICommandExecutor commandExecutor, IPlayerInfoProvider playerInfoProvider, IBlockScanner blockScanner, IScreenshotUtils screenshotUtils) {
this(config, commandExecutor, playerInfoProvider, blockScanner, screenshotUtils, true);
}

public HTTPMCPServer(
MCPConfig config,
ICommandExecutor commandExecutor,
IPlayerInfoProvider playerInfoProvider,
IBlockScanner blockScanner,
IScreenshotUtils screenshotUtils,
boolean screenshotToolEnabled
) {
this.config = config;
this.commandExecutor = new CommandExecutor(config);
this.commandExecutor = commandExecutor;
this.playerInfoProvider = playerInfoProvider;
this.blockScanner = blockScanner;
this.screenshotUtils = screenshotUtils;
this.screenshotToolEnabled = screenshotToolEnabled;
}

public void start() throws IOException {
Expand Down Expand Up @@ -275,7 +294,7 @@ private JsonObject handlePing() {

private JsonObject handleToolsList() {
JsonObject response = new JsonObject();
response.add("tools", MCPProtocol.getToolsListResponse(config));
response.add("tools", MCPProtocol.getToolsListResponse(config, screenshotToolEnabled));
return response;
}

Expand All @@ -295,6 +314,9 @@ private JsonObject handleToolsCall(JsonObject params) {
return handleGetBlocksInArea(arguments);
}
case "take_screenshot" -> {
if (!screenshotToolEnabled) {
return MCPProtocol.createErrorResponse("Tool not available in dedicated server mode", null);
}
return handleTakeScreenshot(arguments);
}
case null, default -> {
Expand All @@ -315,7 +337,7 @@ private JsonObject handleToolsCall(JsonObject params) {

private JsonObject handleGetPlayerInfo() {
try {
JsonObject playerInfo = PlayerInfoProvider.getPlayerInfo();
JsonObject playerInfo = playerInfoProvider.getPlayerInfo();

// Check if there was an error getting player info
if (playerInfo.has("error")) {
Expand Down Expand Up @@ -347,7 +369,7 @@ private JsonObject handleGetBlocksInArea(JsonObject arguments) {
}

int maxAreaSize = config.getServer().getMaxAreaSize();
JsonObject result = BlockScanner.scanBlocksInArea(fromPos, toPos, maxAreaSize);
JsonObject result = blockScanner.scanBlocksInArea(fromPos, toPos, maxAreaSize);

// Check if there was an error scanning blocks
if (result.has("error")) {
Expand Down Expand Up @@ -405,7 +427,7 @@ JsonObject awaitScreenshotResult(CompletableFuture<String> future) {
}

CompletableFuture<String> takeScreenshotAsync(JsonObject params) {
return ScreenshotUtils.takeScreenshot(params);
return screenshotUtils.takeScreenshot(params);
}

private JsonObject createSuccessResponse(JsonObject result, Integer requestId) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import cuspymd.mcp.mod.command.CommandExecutor;
import cuspymd.mcp.mod.command.ICommandExecutor;
import cuspymd.mcp.mod.config.MCPConfig;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
Expand All @@ -20,13 +20,13 @@ public class IPCServer {
private static final Gson GSON = new Gson();
private static final int IPC_PORT = 25565; // Default port for IPC
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid binding IPC listener to the game's default port

IPCServer hardcodes IPC_PORT to 25565, and this commit now starts that listener on dedicated servers when non-HTTP transport is selected. On servers using the normal gameplay port, this creates a bind collision and MCP fails to start for that transport mode. Make the IPC port configurable (or otherwise distinct from the gameplay listener) so dedicated deployments can actually use IPC mode.

Useful? React with 👍 / 👎.


private final CommandExecutor commandExecutor;
private final ICommandExecutor commandExecutor;
private final AtomicBoolean running = new AtomicBoolean(false);
private ServerSocket serverSocket;
private ExecutorService executor;

public IPCServer(MCPConfig config) {
this.commandExecutor = new CommandExecutor(config);
public IPCServer(MCPConfig config, ICommandExecutor commandExecutor) {
this.commandExecutor = commandExecutor;
}

public void start() throws IOException {
Expand Down
7 changes: 7 additions & 0 deletions src/main/java/cuspymd/mcp/mod/command/ICommandExecutor.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package cuspymd.mcp.mod.command;

import com.google.gson.JsonObject;

public interface ICommandExecutor {
JsonObject executeCommands(JsonObject arguments);
}
Loading