diff --git a/.gitignore b/.gitignore index 4e62a69..f72b9ba 100644 --- a/.gitignore +++ b/.gitignore @@ -131,3 +131,6 @@ dmypy.json # Generally ignore VS Code files .vscode + +# Generally ignore PyCharm files +.idea diff --git a/architectures/index.rst b/architectures/index.rst new file mode 100644 index 0000000..636d85b --- /dev/null +++ b/architectures/index.rst @@ -0,0 +1,19 @@ +.. experiment_control_protocol documentation master file, created by + sphinx-quickstart on Wed Jan 18 20:57:47 2023. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +.. include:: README.md + :parser: myst_parser.sphinx_ + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + pymeasure_v0.11 + template + +## Indices and tables + +* :ref:`genindex` +* :ref:`search` diff --git a/architectures/pymeasure_v0.11.md b/architectures/pymeasure_v0.11.md new file mode 100644 index 0000000..160017e --- /dev/null +++ b/architectures/pymeasure_v0.11.md @@ -0,0 +1,79 @@ +# Architecture: Pymeasure v0.11 + +This is a description of the Pymeasure architecture at version 0.11 + +## Components + +### Manager + +- Type: [Director](comp:director), [Coordinator](comp:coordinator) +- Interface: {term}`Queue.py` +- Input: procedure, status +- Output: status +- Description: + - The pymeasure Manager is an optional component that is used only with the pymeasure graphic user interface. It + manages a queue of experiment runs for a given Procedure. + +### Procedure + +- Type: [Procedure](comp:procedure), [Actor](comp:actor) +- Interface: {term}`object` +- Input: N/A +- Output: N/A +- Description: + - A pymeasure Procedure defines the control sequence for an experiment. As an `object`, it is instantiated and + controlled in code by the Worker component. It uses pymeasure Drivers to connect and communicate with different + instruments. + +### Recorder + +- Type: [Observer](comp:observer) +- Interface: {term}`queue.py` +- Input: result +- Output: N/A +- Description: + - The pymeasure Recorder records ongoing datapoints from an experiment to an output file. It utilizes the builtin + python `logging` library for recording output to a file handle. + +### Results +- Type: [Observer](comp:observer) +- Interface: {term}`object` +- Input: status +- Output: N/A +- Description: + - The pymeasure Results creates the output file for an experiment run, although it doesn't write data + points to it, see Recorder. It is typically instantiated and controlled by the Manager to periodically read the + output file data points to graph datapoints in the GUI. + +### Worker + +- Type: [Director](comp:director), [Coordinator](comp:coordinator) +- Interface: {term}`queue.py`, {term}`zmq` +- Input: procedure, status +- Output: result, status +- Description: + - A pymeasure Worker is the main component that executes an experiment run, directing other components such as + Procedure and Recorder. The `zmq` interface is not utilized by default. + +## Diagrams +GUI Implementation + +```{mermaid} +graph TD + Manager-->Results + Results-->Worker + Worker-->Recorder + Worker-->Procedure + Worker-->Manager + Procedure-->Worker +``` + +Script Implementation + +```{mermaid} +graph TD + Results-->Worker + Worker-->Recorder + Worker-->Procedure + Procedure-->Worker +``` \ No newline at end of file diff --git a/architectures/template.md b/architectures/template.md new file mode 100644 index 0000000..0c58db8 --- /dev/null +++ b/architectures/template.md @@ -0,0 +1,23 @@ +# Architecture: template + +This is a template for a proposed or current experiment control architecture. + +## Components + +### Component Name + +- Type: List of types from [Components](comp) +- Interface: List of Interface types from [Interface](iface) glossary +- Input: List of Message types from [Messages](messages) glossary +- Output: List of Message types from [Messages](messages) glossary +- Description: + - A description of what the component does in relation to other components. + +## Diagrams + +Diagram illustrating different implementations of the architecture. + +```{mermaid} +graph TD + Component-->Component +``` diff --git a/components.md b/components.md index b6d8b95..2f385d9 100644 --- a/components.md +++ b/components.md @@ -1,7 +1,9 @@ +(comp)= # Components This page provides details on the main component/entity types that make up a deployment of the ECP. +(comp:director)= ## Director A Director manages a part of or the entire setup, orchestrating the Components according to the needs of the experiment. It can, among other things, @@ -11,6 +13,7 @@ It can, among other things, Potentially a GUI could be attached here too. +(comp:actor)= ## Actor An Actor is a component that interfaces with a (hardware) Device and that has a specific API on the ECP side. An Actor must contain a separate Driver object which communicates with the Device. @@ -59,6 +62,7 @@ Recent values may be cached in the Actor. It may correspond closely to _attributes_ or Python (or PyMeasure) _properties_ of the Driver. It may have unit information that is used when sending data over the Network. +(comp:procedure)= ## Procedures Sequences of steps that make up experiment runs, e.g. PyMeasure procedures. These instructions could be consumed by a Director and trigger a sequence of commands (ramps, loops, conditionals,...). @@ -93,6 +97,7 @@ sequenceDiagram Processor1->>Observer1: "Average oven temperature" 14.47 degC ::: +(comp:coordinator)= ## Coordinator A component primarily concerned with routing/coordinating the message flow between other Components. It represents the intermediate zmq brokers, proxies or somesuch. @@ -100,6 +105,7 @@ Multiple coordinator instances may be necessary for large deployments, but a sin The presence of a coordinator should avoid the complexity/scaling of a purely point-to-point messaging approach. +(comp:observer)= ## Observer A Component that receives data from other Components, e.g. for logging, storage, or plotting, either directly in a streaming fashion, batched, or delayed. It only consumes message streams, but does not command `Actors`. diff --git a/glossary.md b/glossary.md index 83bbb5b..e132980 100644 --- a/glossary.md +++ b/glossary.md @@ -1,37 +1,50 @@ # Glossary + An alphabetical list of terms with a short description. -To help distinguish between the plain English meaning of these terms, and our more specific use, we Capitalize terms from this glossary when we refer to the ECP-specific meaning. +To help distinguish between the plain English meaning of these terms, and our more specific use, we Capitalize terms +from this glossary when we refer to the ECP-specific meaning, similar to a Python class. +(glossary)= :::{glossary} -Actor - An Actor offers a standardized interface to the ECP network to communicate with some Device. This happens via a Driver contained in the Actor, see {ref}`components.md#actor`. An Actor implements the mapping/translation between ECP messages and the Driver's interface. +Actor + A Component that offers a standardized interface to the ECP network to communicate with some Device. This + happens via a Driver contained in the Actor, see {ref}`components.md#actor`. An Actor implements the + mapping/translation between ECP messages and the Driver's interface. -Component - A type of entity, a set of which make up the ECP communication Network, see {ref}`components.md#components`. +Component + A Component is the logical block that makes up the block diagram of an experiment control network. The different + component types, subclasses of Component, are listed in see {ref}`components.md#components`. A Component + communicates with other Components over an Interface to pass Messages. Coordinator A Component primarily concerned with routing/coordinating the message flow, see {ref}`components.md#coordinator`. Device - Some piece of hardware controlled by a Driver. + Hardware controlled by a Driver, for instance a test instrument. Director - A Component which takes part in orchestrating a (i.e. ECP-controlled) measurement setup, see {ref}`components.md#director`. + A Component which takes part in orchestrating a (i.e. ECP-controlled) measurement setup, see + {ref}`components.md#director`. Driver - An object that takes care of communicating with a Device. This object is external to ECP, for example coming from and instrument control library like `pymeasure`, `instrumentkit` or `yaq`. See {ref}`components.md#driver`. + An object that takes care of communicating with a Device. This object is external to ECP, for example coming from and + instrument control library like `pymeasure`, `instrumentkit` or `yaq`. See {ref}`components.md#driver`. ECP The **E**xperiment **C**ontrol **P**rotocol framework. +Interface + The communication methods that a Component can receive or transmit to other Components. + Network The web of Components communicating with each other in an ECP deployment. Observer - A Component that receives data from other Components, e.g. for logging, storage, or plotting, see {ref}`components.md#observer`. + A Component that receives data from other Components, e.g. for logging, storage, or plotting, see + {ref}`components.md#observer`. Processor - A Component on the ECP network which runs some kind of processing operation on one or more inputs and produces one or more outputs. Can be stateful or stateless. See {ref}`components.md#processor`. - + A Component on the ECP network which runs some kind of processing operation on one or more inputs and produces one or + more outputs. Can be stateful or stateless. See {ref}`components.md#processor`. ::: diff --git a/index.rst b/index.rst index a6238b3..e7582ed 100644 --- a/index.rst +++ b/index.rst @@ -13,10 +13,12 @@ introduction components glossary + interfaces + messages + architectures/index Hello_world -Indices and tables -================== +## Indices and tables * :ref:`genindex` * :ref:`search` diff --git a/interfaces.md b/interfaces.md new file mode 100644 index 0000000..2e63eea --- /dev/null +++ b/interfaces.md @@ -0,0 +1,23 @@ +(iface)= +# Interfaces + +An Interface is the way Components can communicate with each other. + + +:::{glossary} + +zmq + [Zeromq](https://zeromq.org/) is a message passing framework that can communicate over multiple + network communication protocols, such as TCP, UDP, and websockets. It implements several common message passing + patterns, such as the "Publish / Subscribe" pattern. + +queue.py + The queue.py interface utilizes the builtin Python [queue](https://docs.python.org/3/library/queue.html) system for + passing messages between Python threads. + +object + The object interface is interacted with inline of the codebase, in the same thread as the Component that + initializes it. For example, if the output of `Procedure` was only accessible in code `result = Procedure.output()`, + the interface of `Procedure` would be labelled as an "object interface". + +::: diff --git a/messages.md b/messages.md new file mode 100644 index 0000000..3fe79f2 --- /dev/null +++ b/messages.md @@ -0,0 +1,19 @@ +(messages)= +# Messages + +These are the different types of messages. Can be thought of as publisher topics. + +`result` + The `result` message type contains data that should be displayed or recorded + +`experiment` + The `experiment` message type + +`control` + The `control` message type + +`status` + The `status` message type contains status information of the ECP + +`procedure` + The `procedure` message type