Skip to content

Conversation

@dmduran12
Copy link

Summary

Pre-calculate and store airtime_ms when packets are received, eliminating repeated client-side airtime calculations on every chart render.

Problem

Currently, the frontend calculates airtime for every packet each time a chart is rendered:

Backend stores packet → Frontend fetches packet → Frontend calculates airtime per packet × N
                                                   ↑ Repeated on every time range change

For large packet histories (e.g., 75K packets), this results in:

  • Chart render times of 100-500ms
  • High CPU usage on resource-constrained devices (Raspberry Pi)
  • Redundant calculations for unchanged data

Solution

Calculate airtime once at receive time and store it with the packet:

Backend stores packet + airtime_ms → Frontend fetches packet with airtime_ms → Just sum/bucket
                                     ↑ Calculate once, use forever

Changes

Database (sqlite_handler.py)

  • Migration 4: Add airtime_ms REAL column to packets table
  • Update store_packet() INSERT to include airtime_ms
  • Update get_recent_packets(), get_filtered_packets(), get_packet_by_hash() to return airtime_ms

Packet Processing (engine.py)

  • Calculate airtime_ms using existing AirtimeManager.calculate_airtime() when building packet record
  • Uses the actual radio config (SF, BW, CR) at receive time for accurate calculations

API Response

Packet responses now include the pre-calculated value:

{
  "timestamp": 1705728000,
  "type": 4,
  "raw_packet": "...",
  "airtime_ms": 226.3
}

Migration Path

  • New packets: Get airtime_ms calculated and stored automatically
  • Old packets: Return airtime_ms = NULL; frontend falls back to client-side calculation
  • Self-healing: As old data ages out via 7-day cleanup, fallback code becomes unnecessary

Performance Impact

Metric Before After
Chart render (75K pkts) ~100-500ms ~10-50ms
CPU usage on RPi High (formula × N) Low (just sum)
Accuracy Must sync formula Always matches backend

Co-Authored-By: Warp agent@warp.dev

Calculate and store airtime_ms when packets are received, eliminating
the need for frontend to recalculate airtime for every packet on
every chart render.

Changes:
- Add airtime_ms REAL column to packets table (migration 4)
- Store airtime_ms in packet record at receive time using AirtimeManager
- Include airtime_ms in all packet query responses

This reduces chart render time from O(n * calculation) to O(n) for
airtime aggregation, particularly beneficial on resource-constrained
devices like Raspberry Pi with large packet histories.

Old packets without airtime_ms return NULL; frontend can fall back to
client-side calculation until data ages out via normal cleanup.

Co-Authored-By: Warp <agent@warp.dev>
@dmduran12
Copy link
Author

we can also implement a one-time database backfill that adds this to all packets... but... it was starting to feel complex. If you'd like, I can submit it, but I think the front can take the brunt of that work with fallback methods to calculate airtime based on current radio settings IF the API doesn't return airtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant