Add throughput testing infrastructure and fix Python telemetry performance

examples/throughput_server/README.md

@@ -0,0 +1,179 @@
+# COSMOS Throughput Testing Server
+
+A standalone Python TCP/IP server for measuring COSMOS command and telemetry throughput.
+
+## Overview
+
+This server provides a lightweight test environment for measuring COSMOS throughput performance. It implements:
+
+- Dual-port TCP server (7778 for INST, 7780 for INST2)
+- CCSDS command packet parsing
+- CCSDS telemetry packet generation
+- LengthProtocol framing (compatible with COSMOS interfaces)
+- Configurable telemetry streaming rates
+- Real-time throughput metrics
+
+## Architecture
+
+```
+COSMOS Container                      Throughput Server (Python)
++-------------------+                 +-------------------+
+| INST (Ruby)       |----TCP/7778---->| Port 7778         |
+|   TcpipClient     |                 |   CCSDS Commands  |
+|   LengthProtocol  |<---TCP/7778----.|   CCSDS Telemetry |
++-------------------+                 +-------------------+
+| INST2 (Python)    |----TCP/7780---->| Port 7780         |
+|   TcpipClient     |                 |   CCSDS Commands  |
+|   LengthProtocol  |<---TCP/7780----.|   CCSDS Telemetry |
++-------------------+                 +-------------------+
+```
+
+## Requirements
+
+- Python 3.10 or higher
+- No external dependencies (uses only standard library)
+
+## Usage
+
+### Starting the Server
+
+```bash
+# Default ports (7778 for INST, 7780 for INST2)
+python throughput_server.py
+
+# Custom ports
+python throughput_server.py --inst-port 8778 --inst2-port 8780
+
+# Debug logging
+python throughput_server.py --debug
+```
+
+### Command Line Options
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--inst-port` | 7778 | Port for INST (Ruby) target |
+| `--inst2-port` | 7780 | Port for INST2 (Python) target |
+| `--debug` | False | Enable debug logging |
+
+## Supported Commands
+
+The server responds to the following CCSDS commands:
+
+| Command | PKTID | Description |
+|---------|-------|-------------|
+| START_STREAM | 200 | Start telemetry streaming at specified rate |
+| STOP_STREAM | 201 | Stop telemetry streaming |
+| GET_STATS | 202 | Request THROUGHPUT_STATUS telemetry packet |
+| RESET_STATS | 203 | Reset all throughput statistics |
+
+### START_STREAM Payload
+
+| Field | Type | Description |
+|-------|------|-------------|
+| RATE | 32-bit UINT | Packets per second (1-100000) |
+| PACKET_TYPES | 32-bit UINT | Bitmask of packet types (default: 0x01) |
+
+## Telemetry Packets
+
+### THROUGHPUT_STATUS (APID 100)
+
+| Field | Type | Description |
+|-------|------|-------------|
+| CMD_RECV_COUNT | 32-bit UINT | Total commands received |
+| CMD_RECV_RATE | 32-bit FLOAT | Commands per second |
+| TLM_SENT_COUNT | 32-bit UINT | Total telemetry packets sent |
+| TLM_SENT_RATE | 32-bit FLOAT | Telemetry packets per second |
+| TLM_TARGET_RATE | 32-bit UINT | Configured streaming rate |
+| BYTES_RECV | 64-bit UINT | Total bytes received |
+| BYTES_SENT | 64-bit UINT | Total bytes sent |
+| UPTIME_SEC | 32-bit UINT | Server uptime in seconds |
+
+## CCSDS Packet Format
+
+### Command Header (8 bytes)
+
+```
+Bits 0-2:   CCSDSVER (3 bits) = 0
+Bit 3:      CCSDSTYPE (1 bit) = 1 (command)
+Bit 4:      CCSDSSHF (1 bit) = 0
+Bits 5-15:  CCSDSAPID (11 bits)
+Bits 16-17: CCSDSSEQFLAGS (2 bits) = 3
+Bits 18-31: CCSDSSEQCNT (14 bits)
+Bits 32-47: CCSDSLENGTH (16 bits) = packet_length - 7
+Bits 48-63: PKTID (16 bits)
+```
+
+### Telemetry Header (14 bytes)
+
+```
+Bits 0-2:   CCSDSVER (3 bits) = 0
+Bit 3:      CCSDSTYPE (1 bit) = 0 (telemetry)
+Bit 4:      CCSDSSHF (1 bit) = 1
+Bits 5-15:  CCSDSAPID (11 bits)
+Bits 16-17: CCSDSSEQFLAGS (2 bits) = 3
+Bits 18-31: CCSDSSEQCNT (14 bits)
+Bits 32-47: CCSDSLENGTH (16 bits)
+Bits 48-79: TIMESEC (32 bits)
+Bits 80-111: TIMEUS (32 bits)
+Bits 112-127: PKTID (16 bits)
+```
+
+## COSMOS Integration
+
+### Installing the Modified DEMO Plugin
+
+To use the throughput server with COSMOS, install the modified DEMO plugin with throughput server enabled:
+
+```bash
+./openc3.sh cli load openc3-cosmos-demo-*.gem \
+  use_throughput_server=true \
+  throughput_server_host=host.docker.internal
+```
+
+### Plugin Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `use_throughput_server` | false | Enable throughput server mode |
+| `throughput_server_host` | host.docker.internal | Hostname of throughput server |
+| `inst_throughput_port` | 7778 | Port for INST connection |
+| `inst2_throughput_port` | 7780 | Port for INST2 connection |
+
+### Running Throughput Tests
+
+1. Start the throughput server:
+   ```bash
+   python throughput_server.py
+   ```
+
+2. Install the DEMO plugin with throughput mode enabled
+
+3. Run the Ruby throughput test:
+   - Open Script Runner in COSMOS
+   - Load `INST/procedures/throughput_test.rb`
+   - Execute and observe results
+
+4. Run the Python throughput test:
+   - Open Script Runner in COSMOS
+   - Load `INST2/procedures/throughput_test.py`
+   - Execute and observe results
+
+5. Monitor via the THROUGHPUT screen in Telemetry Viewer
+
+## File Structure
+
+```
+examples/throughput_server/
+├── throughput_server.py    # Main entry point
+├── ccsds.py                # CCSDS packet encoding/decoding
+├── metrics.py              # Throughput statistics
+├── config.py               # Configuration constants
+├── requirements.txt        # Dependencies (none required)
+└── README.md               # This file
+```
+
+## License
+
+Copyright 2026 OpenC3, Inc.
+Licensed under the GNU Affero General Public License v3.

examples/throughput_server/ccsds.py

@@ -0,0 +1,197 @@
+# Copyright 2026 OpenC3, Inc.
+# All Rights Reserved.
+#
+# This program is free software; you can modify and/or redistribute it
+# under the terms of the GNU Affero General Public License
+# as published by the Free Software Foundation; version 3 with
+# attribution addendums as found in the LICENSE.txt
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+
+"""CCSDS packet encoding and decoding utilities."""
+
+import struct
+import time
+from dataclasses import dataclass
+from typing import Tuple
+
+from config import (
+    CCSDS_CMD_HEADER_SIZE,
+    CCSDS_TLM_HEADER_SIZE,
+    LENGTH_ADJUSTMENT,
+)
+
+
+@dataclass
+class CcsdsCommand:
+    """Parsed CCSDS command packet."""
+
+    version: int
+    packet_type: int
+    secondary_header_flag: int
+    apid: int
+    sequence_flags: int
+    sequence_count: int
+    length: int
+    pktid: int
+    payload: bytes
+
+
+@dataclass
+class CcsdsTelemetry:
+    """CCSDS telemetry packet structure."""
+
+    apid: int
+    sequence_count: int
+    pktid: int
+    payload: bytes
+    time_sec: int = 0
+    time_us: int = 0
+
+
+def parse_ccsds_command(data: bytes) -> CcsdsCommand:
+    """Parse a CCSDS command packet.
+
+    CCSDS Command Header (8 bytes):
+    Bits 0-2:   CCSDSVER (3 bits) = 0
+    Bit 3:      CCSDSTYPE (1 bit) = 1 (command)
+    Bit 4:      CCSDSSHF (1 bit) = 0
+    Bits 5-15:  CCSDSAPID (11 bits)
+    Bits 16-17: CCSDSSEQFLAGS (2 bits) = 3
+    Bits 18-31: CCSDSSEQCNT (14 bits)
+    Bits 32-47: CCSDSLENGTH (16 bits) = packet_length - 7
+    Bits 48-63: PKTID (16 bits)
+
+    Args:
+        data: Raw packet bytes (must be at least CCSDS_CMD_HEADER_SIZE bytes)
+
+    Returns:
+        Parsed CcsdsCommand object
+    """
+    if len(data) < CCSDS_CMD_HEADER_SIZE:
+        raise ValueError(
+            f"Command packet too short: {len(data)} < {CCSDS_CMD_HEADER_SIZE}"
+        )
+
+    # Parse first two bytes (big-endian): version(3) + type(1) + shf(1) + apid(11)
+    word0 = struct.unpack(">H", data[0:2])[0]
+    version = (word0 >> 13) & 0x07
+    packet_type = (word0 >> 12) & 0x01
+    secondary_header_flag = (word0 >> 11) & 0x01
+    apid = word0 & 0x07FF
+
+    # Parse next two bytes: seqflags(2) + seqcnt(14)
+    word1 = struct.unpack(">H", data[2:4])[0]
+    sequence_flags = (word1 >> 14) & 0x03
+    sequence_count = word1 & 0x3FFF
+
+    # Parse length field (2 bytes)
+    length = struct.unpack(">H", data[4:6])[0]
+
+    # Parse PKTID (2 bytes)
+    pktid = struct.unpack(">H", data[6:8])[0]
+
+    # Extract payload (after 8-byte header)
+    payload = data[CCSDS_CMD_HEADER_SIZE:]
+
+    return CcsdsCommand(
+        version=version,
+        packet_type=packet_type,
+        secondary_header_flag=secondary_header_flag,
+        apid=apid,
+        sequence_flags=sequence_flags,
+        sequence_count=sequence_count,
+        length=length,
+        pktid=pktid,
+        payload=payload,
+    )
+
+
+def build_ccsds_telemetry(
+    tlm: CcsdsTelemetry, use_current_time: bool = True
+) -> bytes:
+    """Build a CCSDS telemetry packet.
+
+    CCSDS Telemetry Header (16 bytes):
+    Bits 0-2:   CCSDSVER (3 bits) = 0
+    Bit 3:      CCSDSTYPE (1 bit) = 0 (telemetry)
+    Bit 4:      CCSDSSHF (1 bit) = 1 (secondary header present)
+    Bits 5-15:  CCSDSAPID (11 bits)
+    Bits 16-17: CCSDSSEQFLAGS (2 bits) = 3
+    Bits 18-31: CCSDSSEQCNT (14 bits)
+    Bits 32-47: CCSDSLENGTH (16 bits)
+    Bits 48-79: TIMESEC (32 bits)
+    Bits 80-111: TIMEUS (32 bits)
+    Bits 112-127: PKTID (16 bits)
+
+    Args:
+        tlm: Telemetry packet data
+        use_current_time: If True, use current time; otherwise use tlm.time_sec/time_us
+
+    Returns:
+        Raw packet bytes
+    """
+    if use_current_time:
+        now = time.time()
+        time_sec = int(now)
+        time_us = int((now - time_sec) * 1_000_000)
+    else:
+        time_sec = tlm.time_sec
+        time_us = tlm.time_us
+
+    # Calculate total packet length
+    # CCSDSLENGTH = (packet_length - 7), where packet_length includes primary header
+    # Packet = primary header (6) + secondary header (8: TIMESEC+TIMEUS) + PKTID (2) + payload
+    total_length = 6 + 8 + 2 + len(tlm.payload)
+    ccsds_length = total_length - LENGTH_ADJUSTMENT
+
+    # Build first word: version(3) + type(1) + shf(1) + apid(11)
+    # version=0, type=0 (telemetry), shf=1 (secondary header present)
+    word0 = (0 << 13) | (0 << 12) | (1 << 11) | (tlm.apid & 0x07FF)
+
+    # Build second word: seqflags(2) + seqcnt(14)
+    # seqflags=3 (standalone packet)
+    word1 = (3 << 14) | (tlm.sequence_count & 0x3FFF)
+
+    # Pack header
+    header = struct.pack(
+        ">HHHIIH",
+        word0,
+        word1,
+        ccsds_length,
+        time_sec,
+        time_us,
+        tlm.pktid,
+    )
+
+    return header + tlm.payload
+
+
+def read_packet_from_stream(data: bytes) -> Tuple[bytes, bytes]:
+    """Extract a complete packet from a data stream using LengthProtocol.
+
+    Uses the CCSDSLENGTH field to determine packet boundaries.
+
+    Args:
+        data: Raw data buffer
+
+    Returns:
+        Tuple of (complete_packet, remaining_data)
+        If no complete packet is available, returns (b"", data)
+    """
+    if len(data) < 6:
+        # Not enough data for primary header
+        return b"", data
+
+    # Read length field at bytes 4-5
+    ccsds_length = struct.unpack(">H", data[4:6])[0]
+    total_length = ccsds_length + LENGTH_ADJUSTMENT
+
+    if len(data) < total_length:
+        # Not enough data for complete packet
+        return b"", data
+
+    return data[:total_length], data[total_length:]