Expose usable API

[jsdw]

People like the 1kv program need to be able to programmatically access data about nodes. Currently, they do this via the "feed" data that's sent to the telemetry UI to populate it. This feed interface sucks; it's not supposed to be a "public" interface and we don't want to commit to it being stable. Much better would be to programmatically expose specific data that's of interest (via some sort of subscriptions) in a more accessible and easy to work with format. This is then a format we can endavour to maintain some level of support for and clearly signal any breaking changes to.

It's come up a number of times where people have requested such an interface. See eg #562 more recently. @ggwpez was also asking about extracting validator version info.

[dcolley]

The same issue now for Decentralised Nodes, we do need a predictable API.
One example:

ChainStats:

substrate-telemetry/frontend/src/common/types.ts

Line 123 in d4a557c

export type ChainStats = {

JS frontend says this is a single structure:

export type NodeSysInfo = {
  cpu: string;
  memory: number;
  core_count: number;
  linux_kernel: string;
  linux_distro: string;
  is_virtual_machine: boolean;
};

export type ChainStats = {
  version: Maybe<Ranking<string>>;
  target_os: Maybe<Ranking<string>>;
  target_arch: Maybe<Ranking<string>>;
  cpu: Maybe<Ranking<string>>;
  core_count: Maybe<Ranking<number>>;
  memory: Maybe<Ranking<Range>>;
  is_virtual_machine: Maybe<Ranking<boolean>>;
  linux_distro: Maybe<Ranking<string>>;
  linux_kernel: Maybe<Ranking<string>>;
  cpu_hashrate_score: Maybe<Ranking<Range>>;
  memory_memcpy_score: Maybe<Ranking<Range>>;
  disk_sequential_write_score: Maybe<Ranking<Range>>;
  disk_random_write_score: Maybe<Ranking<Range>>;
  cpu_vendor: Maybe<Ranking<string>>;
};

The telemetry feed is providing this - the above fields are provided over 2 dicts:

 [
759,
  [
    'Stakeflow',
    'Parity Polkadot',
    '1.16.2-dba2dd59101',
    'CxvXU7VeqanWnCh78r9G4FCYkvCvnEgZHPxT2ps5JmRF548',
    '12D3KooWFVRbiB7dVCVBVzNWfGtDyf1X2aPMr8Nn6BqNCc2oT99L',
    '91.242.215.213',
// could be NodeSysInfo?
    {
      cpu: 'Intel(R) Xeon(R) E-2276G CPU @ 3.80GHz',
      memory: 67322474496,
      core_count: 6,
      linux_kernel: '6.8.0-47-generic',
      linux_distro: 'Ubuntu 24.04.1 LTS',
      is_virtual_machine: false
    },
// could be incomplete ChainStats?
    {
      cpu_hashrate_score: 1432,
      memory_memcpy_score: 15497,
      disk_sequential_write_score: 1438,
      disk_random_write_score: 524
    }
  ],