AstroVIPER Architecture

[Jan-Willem]

A deployment of the Radio Astronomy Data Processing System (RADPS) will be designed to efficiently execute multiple independent workflows with differing computational requirements, while producing science-ready data products such as deconvolved spectral-cube images. Figure 1 illustrates the RADPS software layers. The smallest unit is a domain function, which performs a single transformation or query on in-memory data. Multiple domain functions, together with data loading and caching, constitute a node task. Graph tasks are created by combining node tasks into a map-reduce graph that exploits the available parallelisms in the data and domain functions. For example, spectral-cube imaging can be performed independently across frequency channels, allowing a node task to image a subset of channels. The graph of node tasks is where large-scale parallelism occurs.

Groups of related tasks form stages for control, maintenance, and orchestration. For example, a data-import and preparation stage may consist of graph tasks that download data from the archive, apply flagging, and query antenna tables. Each layer will have a documented and tested API. This layered design was chosen to ensure a clear separation of concerns.