Skip to content

Feature/muse#3

Open
VinceIngram07 wants to merge 34 commits intohtil:feature/eeg-vexfrom
VinceIngram07:feature/Muse
Open

Feature/muse#3
VinceIngram07 wants to merge 34 commits intohtil:feature/eeg-vexfrom
VinceIngram07:feature/Muse

Conversation

@VinceIngram07
Copy link
Copy Markdown

1.40 update

VinceIngram07 and others added 30 commits January 28, 2025 13:01
forward = Right
down = Left
Added new blocks (Move and Color)
Only Forward command is working with vex
WORK IN PROGRESS
Previous block functionality
still working on new block functions
Found problem: the number for the muscle energy window and the raw data doesn't match up
- need to connect the vex through AP before starting the application
- fixed colors
- no longer degrees for turn block (for kids)
- Shortcut is ready to be used (Neuroscope-EMG)
-Updated gitignore (pycache)
* src/main/index.js:
  - Fixed production build path from loadURL to loadFile with correct relative path
  - Added Python process spawning with virtual environment detection
  - Implemented WebSocket connection validation with 10s timeout
  - Added proper process cleanup on app quit and window close
  - Updated hardcoded references from Muse to Ganglion devices
  - Fixed port configuration from 3000 to 3002

* src/renderer/js/muse-client.js:
  - Fixed duplicate filters property that was overriding device name filters
  - Added Ganglion service (0xfe84) to optionalServices for proper discovery

* package.json:
  - Updated electron-builder config to include build/renderer directory
  - Added extraResources for Python executable bundling
  - Enhanced build scripts with python:build for PyInstaller integration
  - Updated serve script to use port 3002
  - Added build:local script for complete build process

* VEXServer.spec (new):
  - PyInstaller specification for Python server executable
  - Proper dependency handling for websockets and asyncio
  - Console executable configuration for debugging

* resources/python/VEXServer_dev.py (new):
  - Created simulation mode WebSocket server for development
  - Mock VEX robot with full command protocol (led_on, move, turn_left, turn_right)
  - Enables testing without physical VEX robot hardware
  - Runs on ws://127.0.0.1:8777 with JSON command interface

* requirements.txt:
  - Added PyInstaller>=5.0 for standalone executable creation

**Key Improvements:**
- Enables successful Windows executable creation with integrated Python server
- Two-process architecture provides reliable communication in both dev and production
- Fixed Bluetooth device scanning to properly detect Ganglion devices instead of Muse
- Automatic virtual environment detection in development mode
- Production builds now bundle Python server as standalone executable
- Enhanced development workflow with simulation server for hardware-free testing
…rking WebSocket management

**New Features:**
* Console UI: Replace frequency bands display with clean console output
* VEX Reconnect: Add functional reconnect button to restart VEX connection without app restart
* Enhanced Testing: Complete VEX simulation server with comprehensive test suite
* Robust WebSocket Management: Proper connection handling with reconnection support

**Console System (NEW):**
* src/renderer/js/console.js: Clean terminal-like console without timestamps/emojis
* src/renderer/index.html: Replace bands display with console container
* src/renderer/js/main.js: Integrate Console class replacing BandPowerVis
* src/renderer/js/wrapper-functions.js: Connect blockly_print to console output

**VEX Reconnect System (FULLY FUNCTIONAL):**
* src/main/index.js: Complete WebSocket connection management with proper scoping
  - Added stopPythonServer() and reconnectVEX() process management
  - Moved WebSocket variables and functions to global scope for proper access
  - Implemented createWebSocketConnection() with timeout and error handling
  - Fixed function scoping issues that prevented reconnection
* src/main/preload.js: Expose vexReconnect IPC method to renderer
* src/renderer/index.html: Add VEX reconnect button in sidebar (orange refresh icon)
* src/renderer/js/events.js: Implement reconnect button handler with visual feedback

**Enhanced VEX Testing:**
* resources/python/VEXServer_dev.py: Complete simulation server with full command support
* test_vex_server.py: Comprehensive WebSocket test suite for validation

**Key Technical Improvements:**
- Fixed variable scoping: ws and createWebSocketConnection now accessible globally
- Proper WebSocket lifecycle management: Close old connections before creating new ones
- Enhanced error handling with timeout protection and graceful fallbacks
- Clean console replaces cluttered frequency bands display
- Robust reconnection process with proper server restart sequence
- Enhanced print blocks accept any data type and output to console

**WebSocket Reconnection Flow:**
1.  Close existing WebSocket connection
2.  Stop Python VEX server gracefully (SIGTERM with SIGKILL fallback)
3.  Wait 2 seconds for cleanup
4.  Restart Python VEX server
5.  Wait 3 seconds for server initialization
6.  Re-establish WebSocket connection with timeout protection
7.  Resume VEX command functionality
**Block-to-Text Converter:**
* src/renderer/js/simple-text-view.js: Text coding environment using Blockly's built-in code generation
* src/renderer/index.html: Add toggle buttons for block/text modes and export functionality
* src/renderer/js/main.js: Integrate text view with existing Blockly system
… app rename

Python (resources/python):

Add resilient AIM connection loop; server stays alive and retries when robot is offline

Add reconnect_robot action to reset robot connection without restarting server

Add disconnect_robot cleanup to close websocket threads and sockets before re-init

Add status action reporting robot_connected for UI

Improve connection logs (attempting / connected / retry)

Electron main (src/main):

Detect and spawn bundled VEXServer.exe in production

Replace waitOn websocket probe with lightweight TCP port polling

Guard reconnectVEX against overlap and clear force-kill timer to avoid killing new process

Periodically poll server status and relay to renderer via vex-status

Fix false-positive connected state by distinguishing wsConnected vs robotConnected

Add lazy-init UDP sockets for Tello (no bind to 9000/8890 until needed)

Renderer:

Add connectivity status dot showing wsConnected (grey/yellow) and robotConnected (green)

Add onVexStatus and requestVexStatus to preload for UI updates

Packaging/build:

Ensure PyInstaller bundle includes vex module/resources and fix spec path resolution

Use process.resourcesPath/python/VEXServer.exe in production and improve spawn/path logging

Naming/branding:

Rename app to "NeuroBlock EMG for VEX" (productName/appId, window title, HTML title)

Docs:

Update README with "Adding a New Blockly Block (Example: VEX Kicker)" section and build notes

Notes:

Reconnect button sends reconnect_robot when local websocket is up; only restarts Python process if websocket is down

Status dot: grey = websocket down, yellow = websocket up/robot down, green = robot connected

Avoid binding to port 9000 unless Tello is actively used
…X integration

### Application Branding & Identity
- Rename app from 'NeuroBlock EMG for VEX' → 'NeuroBlock EEG for VEX'
- Update package.json productName, window title, and HTML title
- Update appId to reflect EEG focus: com.htil.neuroblock-emg-vex

### EEG Signal Processing (Muse Headset)
- Restore Signal class for real-time EEG data buffering (512-sample buffer)
- Restore BandPowerVis for frequency band visualization (delta, theta, alpha, beta, gamma)
- Restore FeatureExtractor for band power computation from raw EEG samples
- Restore BLE (Bluetooth Low Energy) integration for Muse headset connection
- Wire Muse eegReadings callback → Signal.add_data() for live data streaming

### UI Layout - Two-Panel Design
- Left panel (45% width):
  * Top card (40vh): 4 EEG channel graphs (channels 0-3 from Muse)
  * Bottom card (40vh): Band power frequency spectrum visualization
- Right panel (55% width):
  * Blockly visual programming editor for robot control
  * VEX AIM robot command center and status display
  * Console output and device connection status
…band power metrics and fixing controls_if handler
Copilot AI review requested due to automatic review settings March 24, 2026 17:25
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the app to v1.4.0 with VEX AIM integration (Python WebSocket server + Electron IPC), new VEX/EMG Blockly capabilities, and a new blocks↔text viewing/editing experience.

Changes:

  • Add Python VEX server (prod + dev mock), Electron main-process lifecycle management, IPC bridges, and UI reconnect/status controls.
  • Add new Blockly blocks/categories (VEX movement/kicker, EMG muscle energy) plus wrapper/interpreter support.
  • Add text-mode tooling (Monaco-based editor, simple text view, block-to-text converter demos) and refresh docs/build configuration.

Reviewed changes

Copilot reviewed 38 out of 51 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
test_vex_server.py Adds a manual Python WebSocket test script for the VEX server.
test-reconnect.js Adds a Node-based WebSocket reconnect smoke test.
test-converter.html Adds a browser test harness for the block-to-text converter.
src/renderer/js/wrapper-functions.js Adds safer band-power getters and VEX wrapper functions (incl. kicker).
src/renderer/js/text-editor.js Adds Monaco-based text editor component loaded from CDN.
src/renderer/js/simple-text-view.js Adds a basic blocks→text display mode with export/toast messages.
src/renderer/js/muse-client.js Changes BLE requestDevice options for Muse service access.
src/renderer/js/main.js Initializes global band powers and adds a command helper + sanitize logic.
src/renderer/js/interpreter-api.js Exposes new VEX and EMG functions to the Blockly interpreter.
src/renderer/js/feature-extractor.js Hardens relative band power calculation against divide-by-zero.
src/renderer/js/events.js Adds exportCode and VEX reconnect UI events.
src/renderer/js/customblock.js Adds VEX/muscle blocks and changes some drone block labels/colors (but introduces syntax/runtime issues).
src/renderer/js/console.js Adds a custom console UI inside the “bands” panel.
src/renderer/js/coding-mode-manager.js Adds a Blockly↔text mode manager using Monaco + JS generator.
src/renderer/js/categories.js Adds VEX category and adds muscle_energy to Data category.
src/renderer/js/blockly-main.js Registers new toolbox categories/modules (incl. VEX + muscle_energy).
src/renderer/js/block-to-text-converter.js Adds a converter from Blockly AST to Python-like text.
src/renderer/index.html Updates UI layout and adds mode toggle, export, VEX status/reconnect controls.
src/main/tello.js Switches Tello initialization to lazy creation.
src/main/preload.js Adds VEX IPC API and exposes sendCommand to renderer.
src/main/index.js Adds Python server lifecycle + WebSocket bridge, VEX IPC handlers, packaging path updates, status polling, cleanup hooks.
src/main/index copy.js Minor formatting tweak.
setup-venv.sh Adds a helper script to create/activate a Python venv and install deps.
resources/python/vex/vex_types.py Adds VEX AIM types/enums (but contains a critical constant bug).
resources/python/vex/vex_messages.py Adds VEX AIM WebSocket message builders.
resources/python/vex/vex_globals.py Adds VEX-style global constants.
resources/python/vex/settings.py Adds JSON settings loader for VEX host config.
resources/python/vex/settings.json Adds default VEX host configuration.
resources/python/vex/init.py Exposes the VEX package API.
resources/python/VexTest.py Adds a Python VEX WebSocket test script.
resources/python/VEXServer_dev.py Adds a mock VEX server for development without hardware.
resources/python/VEXServer.py Adds the production VEX WebSocket server with reconnect/status/kicker actions.
requirements.txt Adds Python dependencies for VEX server and packaging.
programs/maze_program_emg.xml Adds a sample Blockly program using muscle_energy and VEX actions.
package.json Bumps to 1.4.0; changes dev server port; adds ws/wait-on; adds Python build step and packaging resources.
block-text-demo.html Adds a standalone block-to-text demo page.
VEXServer.spec Adds PyInstaller spec to bundle the Python server + VEX package.
README.md Expands documentation for usage, adding blocks, and production builds.
Neuroscope.bat Adds a Windows helper to run yarn serve with a venv.
.gitignore Adds ignores for build artifacts, venv, and pycache.
Comments suppressed due to low confidence (8)

src/renderer/js/customblock.js:1

  • This line is syntactically invalid JavaScript and will prevent the renderer bundle from loading. Restore a valid initialization (e.g., let drone_blocks_color = 70;) and remove the stray text.
    src/renderer/js/customblock.js:1
  • The generated code calls electronAPI.sendCommand(...), but electronAPI is not a global in the js-interpreter runtime (and in the renderer it’s typically window.electronAPI). These blocks will fail at execution time. Generate code that calls an interpreter-exposed function (e.g., a wrapper like sendCommand(...) / a dedicated wrapper function) rather than referencing electronAPI directly.
    src/renderer/js/customblock.js:1
  • The generated code calls electronAPI.sendCommand(...), but electronAPI is not a global in the js-interpreter runtime (and in the renderer it’s typically window.electronAPI). These blocks will fail at execution time. Generate code that calls an interpreter-exposed function (e.g., a wrapper like sendCommand(...) / a dedicated wrapper function) rather than referencing electronAPI directly.
    src/renderer/js/coding-mode-manager.js:1
  • Blockly is used but not imported in this module, which will throw a ReferenceError when switching modes. Add an explicit import (e.g., import * as Blockly from \"blockly/core\";) or route resizing through an existing component that already imports Blockly.
    src/renderer/js/muse-client.js:1
  • The navigator.bluetooth.requestDevice(...) options object now lacks filters and acceptAllDevices, which are required by the Web Bluetooth API. Using only optionalServices will cause requestDevice to reject. Keep a filters entry (e.g., by service UUID and/or namePrefix) and use optionalServices only to request additional services.
    src/renderer/index.html:1
  • Both exportCode and bluetooth buttons are positioned at the exact same coordinates (top: 260px; right: 5px) with the same z-index, so one will overlap the other and become unclickable. Adjust positioning (different top values) or group the buttons into a vertical stack container.
    src/renderer/js/block-to-text-converter.js:1
  • The converter is reading input names that don’t match the blocks defined in customblock.js (e.g., the print block uses name: \"val\", and drone movement blocks use name: \"value\"). This will cause generated text to fall back to defaults instead of reflecting the workspace. Update the converter to use the actual field/input names used by your blocks (or centralize block field-name constants shared by both definitions and converter).
    src/renderer/js/text-editor.js:1
  • Loading Monaco from a remote CDN inside an Electron renderer increases supply-chain risk and typically conflicts with Electron hardening guidance (CSP, remote content, offline use). Prefer bundling Monaco with the app build or pin+verify integrity (SRI) with a restrictive CSP; at minimum, document why remote loading is acceptable for this app.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +153 to +160
robot.turn_for(vex.TurnType.LEFT, degrees) # Correct VEX method
# Send a response back to the client
await websocket.send(json.dumps({"status": "success", "action": "turn_left", "degrees": degrees}))

elif action == "turn_right":
degrees = command.get("degrees", 90)
print(f"Turning robot right: {degrees} degrees")
robot.turn_for(vex.TurnType.RIGHT, degrees) # Correct VEX method
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vex is not defined in this module (you import from vex import *, which does not create a vex namespace). This will raise NameError on turn actions. Use TurnType.LEFT/RIGHT (or the imported global LEFT/RIGHT) instead of vex.TurnType.*.

Suggested change
robot.turn_for(vex.TurnType.LEFT, degrees) # Correct VEX method
# Send a response back to the client
await websocket.send(json.dumps({"status": "success", "action": "turn_left", "degrees": degrees}))
elif action == "turn_right":
degrees = command.get("degrees", 90)
print(f"Turning robot right: {degrees} degrees")
robot.turn_for(vex.TurnType.RIGHT, degrees) # Correct VEX method
robot.turn_for(TurnType.LEFT, degrees) # Use imported TurnType enum
# Send a response back to the client
await websocket.send(json.dumps({"status": "success", "action": "turn_left", "degrees": degrees}))
elif action == "turn_right":
degrees = command.get("degrees", 90)
print(f"Turning robot right: {degrees} degrees")
robot.turn_for(TurnType.RIGHT, degrees) # Use imported TurnType enum

Copilot uses AI. Check for mistakes.
'''A percentage unit that represents a value from 0% to 100%'''
LEFT = TurnType.LEFT
'''A turn unit that is defined as left turning.'''
RIGHT = TurnType.LEFT
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RIGHT is incorrectly assigned to TurnType.LEFT, which will invert/merge semantics for any consumer importing this global. Change it to RIGHT = TurnType.RIGHT.

Suggested change
RIGHT = TurnType.LEFT
RIGHT = TurnType.RIGHT

Copilot uses AI. Check for mistakes.
Comment on lines +125 to +128
angle = data.get("angle", 90)
response = self.robot.turn_left(angle)
elif action == "turn_right":
angle = data.get("angle", 90)
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Electron main process sends { action: \"turn_left\", degrees: ... } / {...degrees...} but the dev server reads angle. This means turns will always default to 90° in dev mode. Accept degrees as well (e.g., data.get(\"degrees\") fallback to angle).

Suggested change
angle = data.get("angle", 90)
response = self.robot.turn_left(angle)
elif action == "turn_right":
angle = data.get("angle", 90)
angle = data.get("degrees", data.get("angle", 90))
response = self.robot.turn_left(angle)
elif action == "turn_right":
angle = data.get("degrees", data.get("angle", 90))

Copilot uses AI. Check for mistakes.
Comment on lines 459 to 465
ipcMain.on("drone-up", (event, response) => {
let recent_val = parseInt(response);
let upVal = recent_val > maxSpeed ? maxSpeed : recent_val < minSpeed ? minSpeed : recent_val;
console.log("drone up", upVal, "sent", response);
tello.up(upVal);
let rightVal = recent_val > maxSpeed ? maxSpeed : recent_val < minSpeed ? minSpeed : recent_val;
console.log("Sphero right", rightVal, "sent", response);
const moveCommand = { action: "move", distance: response, heading: 90 };//For Vex
sendCommand(moveCommand);
});
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You compute a clamped numeric value (rightVal) but send distance: response (the unclamped original, potentially a non-numeric string). Send the clamped number instead to ensure speed/limits are actually enforced (same pattern appears in other movement handlers).

Copilot uses AI. Check for mistakes.

# Activate the venv
echo "Activating virtual environment..."
source venv/Scripts/activate
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This activation path is Windows-specific. For bash on macOS/Linux, the virtualenv activation script is typically venv/bin/activate. If this script is intended to be cross-platform, detect the OS and choose the correct activation path (or provide separate scripts).

Copilot uses AI. Check for mistakes.
Comment on lines +32 to +34
"python:build": "C:/dev/repos/VexEMG/neuroscope/.venv/Scripts/pyinstaller.exe VEXServer.spec --clean --noconfirm",
"electron:build": "electron-builder build -c.mac.target=dir",
"build:local": "yarn build && yarn electron:build",
"build:local": "yarn build && yarn python:build && yarn electron:build",
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two packaging blockers:\n1) python:build uses an absolute, machine-specific path to PyInstaller, which will fail on other environments/CI.\n2) extraResources.from points at dist/VEXServer.exe, but Electron Builder also uses dist/ as its output directory by default—this risks overwriting/removing the PyInstaller output during electron:build.\nRecommendation: call pyinstaller via the active venv (.venv/Scripts/pyinstaller or python -m PyInstaller) and configure PyInstaller to output to a non-conflicting directory (e.g., python-dist/) or set --distpath.

Copilot uses AI. Check for mistakes.
Comment on lines +66 to +70
"extraResources": [
{
"from": "dist/VEXServer.exe",
"to": "python/VEXServer.exe"
},
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two packaging blockers:\n1) python:build uses an absolute, machine-specific path to PyInstaller, which will fail on other environments/CI.\n2) extraResources.from points at dist/VEXServer.exe, but Electron Builder also uses dist/ as its output directory by default—this risks overwriting/removing the PyInstaller output during electron:build.\nRecommendation: call pyinstaller via the active venv (.venv/Scripts/pyinstaller or python -m PyInstaller) and configure PyInstaller to output to a non-conflicting directory (e.g., python-dist/) or set --distpath.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants