Skip to content

Add Cesium Copilot AI assistant to Sandcastle#13188

Open
jdehorty wants to merge 33 commits intomainfrom
cesium-copilot
Open

Add Cesium Copilot AI assistant to Sandcastle#13188
jdehorty wants to merge 33 commits intomainfrom
cesium-copilot

Conversation

@jdehorty
Copy link

@jdehorty jdehorty commented Feb 4, 2026

Summary

This PR introduces Cesium Copilot, an AI-powered coding assistant integrated directly into Sandcastle. Cesium Copilot helps developers write, modify, and debug CesiumJS code through an intuitive chat interface with intelligent code understanding and modification capabilities.

Key Features

  • Multi-Model Support: Seamlessly switch between Google Gemini and Anthropic Claude models, with secure client-side API key management
  • Intelligent Code Editing: Advanced diff parsing and application system that accurately modifies code based on natural language instructions
  • Streaming Responses: Real-time streaming of AI responses with support for extended thinking and tool calls
  • Error Context Extraction: Automatic extraction and formatting of runtime errors to provide contextual debugging assistance
  • Conversation History: Persistent conversation history with the ability to resume previous sessions
  • Customizable Settings: Configurable model parameters, UI preferences, and feature toggles

Technical Highlights

  • Robust Diff Engine: Custom diff parser and matcher with multiple matching strategies (exact, normalized, fuzzy) for reliable code modifications
  • Web Worker Architecture: Diff processing offloaded to web workers for responsive UI during complex operations
  • Comprehensive Test Coverage: Extensive unit and integration tests for AI clients, diff logic, and UI components
  • Type-Safe Implementation: Full TypeScript implementation with strict typing throughout

New Components

Component Description
ChatPanel Main chat interface with message history and input
DiffPreview Visual diff display with syntax highlighting
ApiKeyDialog Secure API key configuration modal
SettingsModal Comprehensive settings management
ModelPicker Model selection and configuration UI

Files Changed

  • AI Core (packages/sandcastle/src/AI/): Client implementations, diff parsing/application, error handling
  • UI Components (packages/sandcastle/src/): Chat interface, diff preview, settings panels
  • Contexts (packages/sandcastle/src/contexts/): State management for history and model selection
  • Tests: Comprehensive test suites with >90% coverage on critical paths

Test Plan

  • Verify API key configuration works for both Gemini and Anthropic
  • Test code generation with various prompts
  • Test code modification/editing workflows
  • Verify diff preview displays changes correctly
  • Test conversation history persistence
  • Verify settings changes are applied correctly
  • Run the full test suite: npm run test --workspace=@cesium/sandcastle
  • Manual testing in development environment

Screenshots

image

Note: This feature requires users to provide their own API keys for the AI services they wish to use. No API keys are stored on Cesium servers.

jdehorty and others added 7 commits February 3, 2026 03:52
Integrates an AI-powered coding assistant into Sandcastle that helps users
write and modify Cesium code. Features include:

- Multi-model support (Gemini, Anthropic Claude) with API key management
- Intelligent diff parsing and application for code modifications
- Chat interface with streaming responses and tool call handling
- Error context extraction for debugging assistance
- Comprehensive test coverage for AI client and diff logic
The MatchStrategy enum was imported only as a type, but it's used as a
value in the code. This change imports it as both value and type.
@github-actions
Copy link

github-actions bot commented Feb 4, 2026

Thank you for the pull request, @jdehorty!

✅ We can confirm we have a CLA on file for you.

The sandcastle package had "cesium": "*" as a dependency, which caused
npm to install cesium from npm registry into node_modules/cesium. This
conflicted with the local Source/Cesium.d.ts during TypeScript
definition generation in the deploy workflow, causing TS2649 and
TS6200 errors.

Sandcastle doesn't need cesium as a direct dependency - the cesium
types/code are loaded dynamically at runtime for the Monaco editor.
This commit adds "cesium": "*" as a dependency in the sandcastle package.json file. Additionally, it includes various formatting improvements in the ChatPanel component and related tests for better readability and consistency.
@jjspace jjspace self-assigned this Feb 4, 2026
@jjspace
Copy link
Contributor

jjspace commented Feb 6, 2026

Hey @jdehorty and @Guillar1, thanks for getting this started!

I had some general concerns before I dig too deep into the PR itself

  1. Can we break this PR up a bit? 35,000 lines of code is a lot to review all at once. Some ideas:
    • Testing setup for React as it's own PR (and then used in this PR)
    • Specific feature sets
      • example basic chat this PR > more advanced diff handling or other parts second PR
      • or single AI model this PR > add more clients in follow up
      • or break out the conversation history tracking/restoration into another PR
  2. It seems there was a bad merge somewhere along the way and some old sandcastle changes were reverted.
    • For example the banner linking to the "old" sandcastle which was removed back in November is back
    • I also believe some performance optimizations we've made are missing as the app feels more sluggish than it used too
    • (also please check the CI errors)
  3. It seems like none of the new code uses the Stratakit component library that Sandcastle was built with.
    • This leads to many visual inconsistencies between the copilot UI and the rest of the interface
    • It also means that only half of the new UI actually changes based on the page theme
    • I also see re-implementations of existing components like the Toggle
    • I realize the Stratakit package we use has recently been deprecated and replaced with a new MUI based system. Sandcastle has not switched over yet. We would like to but for now I think all the UI should use the existing library as much as possible
  4. There are a handful of UX changes/decisions we probably should talk through.
    • One of the biggest that stuck out to me is that we now have 2 (or 3 depending how you count) separate settings panels for the app with different layouts but that share some of the same app settings.
  5. The file organization on this branch is a little hard to follow
    • With so many new files and components I agree we should probably have better organization than the, mostly single, directory I was mostly using up until now. I'm open to discussion around the strategy here
      • (breaking up the PR itself will also probably help with this)
    • However it seems some organization was done but not completely or applied consistently. For example there are Diff related components at the top src directory but there are also DiffAccordion and DiffApprovalModal components in the components directory. Or components moved into components but the Gallery components didn't move?

There is also a general concern that I don't currently have access to API keys to actually test with the models that have been selected. We can continue that conversation in the other issues that will be opened.

@Guillar1 Guillar1 force-pushed the cesium-copilot branch 2 times, most recently from ca7df89 to 199658b Compare February 13, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants