Merge pull request #155 from labthings/simplify-property-and-add-type-hints

Simplify property and add type hints

This is a big code tidy-up that removes a lot of confusing and unused behaviour, and makes it possible to statically type `Thing` code. See the "properties" tutorial page on readthedocs for examples.

Author: rwb27

.codespellrc

@@ -7,6 +7,8 @@ skip = *.git,
        ./build,
        ./dist,
        ./docs/_build,
+       ./docs/build,
+       ./docs/source/autoapi,
        ./htmlcov,
        ./src/openflexure_microscope_server/static,
        .venv,

.github/workflows/test.yml

@@ -75,6 +75,9 @@ jobs:
 
       - name: Analyse with MyPy
         run: mypy src
+      
+      - name: Type tests with MyPy
+        run: mypy --warn-unused-ignores typing_tests
 
   test-with-unpinned-deps:
     runs-on: ubuntu-latest

dev-requirements.txt

@@ -34,6 +34,12 @@ click==8.2.1
     #   uvicorn
 codespell==2.4.1
     # via labthings-fastapi (pyproject.toml)
+colorama==0.4.6
+    # via
+    #   click
+    #   pytest
+    #   sphinx
+    #   uvicorn
 coverage==7.9.2
     # via pytest-cov
 dnspython==2.7.0
@@ -49,6 +55,10 @@ email-validator==2.2.0
     # via
     #   fastapi
     #   pydantic
+exceptiongroup==1.3.0
+    # via
+    #   anyio
+    #   pytest
 fastapi==0.116.1
     # via labthings-fastapi (pyproject.toml)
 fastapi-cli==0.0.8
@@ -58,13 +68,10 @@ fastapi-cloud-cli==0.1.4
 flake8==7.3.0
     # via
     #   labthings-fastapi (pyproject.toml)
-    #   flake8-docstrings
     #   flake8-pyproject
     #   flake8-rst
     #   flake8-rst-docstrings
     #   pydoclint
-flake8-docstrings==1.7.0
-    # via labthings-fastapi (pyproject.toml)
 flake8-pyproject==1.2.3
     # via labthings-fastapi (pyproject.toml)
 flake8-rst==0.8.0
@@ -152,8 +159,6 @@ pydantic-settings==2.10.1
     # via fastapi
 pydoclint==0.6.6
     # via labthings-fastapi (pyproject.toml)
-pydocstyle==6.3.0
-    # via flake8-docstrings
 pyflakes==3.4.0
     # via flake8
 pygments==2.19.2
@@ -210,9 +215,7 @@ shellingham==1.5.4
 sniffio==1.3.1
     # via anyio
 snowballstemmer==3.0.1
-    # via
-    #   pydocstyle
-    #   sphinx
+    # via sphinx
 sphinx==8.1.3
     # via
     #   labthings-fastapi (pyproject.toml)
@@ -239,6 +242,14 @@ sphinxcontrib-serializinghtml==2.0.0
     # via sphinx
 starlette==0.47.1
     # via fastapi
+tomli==2.2.1
+    # via
+    #   coverage
+    #   flake8-pyproject
+    #   mypy
+    #   pydoclint
+    #   pytest
+    #   sphinx
 typer==0.16.0
     # via
     #   fastapi-cli
@@ -249,16 +260,20 @@ typing-extensions==4.14.1
     # via
     #   labthings-fastapi (pyproject.toml)
     #   anyio
+    #   astroid
+    #   exceptiongroup
     #   fastapi
     #   mypy
     #   pydantic
     #   pydantic-core
     #   pydantic-extra-types
     #   referencing
+    #   rich
     #   rich-toolkit
     #   starlette
     #   typer
     #   typing-inspection
+    #   uvicorn
 typing-inspection==0.4.1
     # via pydantic-settings
 ujson==5.10.0
@@ -272,8 +287,6 @@ uvicorn==0.35.0
     #   fastapi
     #   fastapi-cli
     #   fastapi-cloud-cli
-uvloop==0.21.0
-    # via uvicorn
 watchfiles==1.1.0
     # via uvicorn
 websockets==15.0.1

docs/source/documentation.rst

@@ -0,0 +1,33 @@
+.. _gen_docs:
+
+Generated documentation
+=======================
+
+LabThings describes its HTTP API in two ways: with a :ref:`wot_td` and with an OpenAPI_ document.
+
+.. _openapi:
+
+OpenAPI
+-------
+
+OpenAPI_ is a standard way to describe an HTTP interface. It lists all of the possible HTTP requests that may be made, along with a description of each one, and a description of the possible responses.
+
+.. _gen_td:
+
+Thing Description
+-----------------
+
+Each :ref:`wot_thing` is documented by a Thing Description, which is a JSON document describing all of the ways to interact with that Thing (:ref:`wot_affordances`\ ). The WoT_ standard defines the `Thing Description`_ and includes a JSON Schema against which it may be validated.
+
+Thing Description documents are higher-level than OpenAPI_ and focus on the capabilities of the Thing. For example, they include a list of properties, where each action is described only once. LabThings treats the Thing Description as your public API, and as a general rule anything not described in the Thing Description is not available over HTTP or to a `.DirectThingClient`\ .
+
+Comparison of Thing Description and OpenAPI
+-------------------------------------------
+
+Thing Description aims to be a neat way to describe the capabilities of a Thing, while OpenAPI focuses on detailed documentation of every possible interaction with the server. Thing Description is a newer and less well adopted standard that's specific to the Web of Things, while OpenAPI has been around for a while and is widely used and understood as a general-purpose API description.
+
+OpenAPI describes each HTTP endpoint individually. There are usually more HTTP endpoints than there are :ref:`wot_affordances` because a Property may have two endpoints, one to read its value and one to write it. Actions usually correspond to several endpoints; one to invoke the action, one to check the action's status, one to cancel it, and another to retrieve its output once it has finished. In principle, client code based on a Thing Description should be more meaningful because related endpoints can be grouped together into properties or actions that correspond to structures in the programming language (like methods and properties in Python). However, OpenAPI is a much more widely adopted standard and so both forms of documentation are generated by LabThings-FastAPI.
+
+.. _WoT: https://www.w3.org/WoT/
+.. _Thing Description: https://www.w3.org/TR/wot-thing-description/
+.. _OpenAPI: https://www.openapis.org/

docs/source/index.rst

@@ -15,6 +15,7 @@ Documentation for LabThings-FastAPI
    blobs.rst
    concurrency.rst
    using_things.rst
+   see_also.rst
 
    autoapi/index
 

docs/source/quickstart/counter.py

@@ -22,9 +22,8 @@ def slowly_increase_counter(self) -> None:
             time.sleep(1)
             self.increment_counter()
 
-    counter = lt.ThingProperty(
-        model=int, initial_value=0, readonly=True, description="A pointless counter"
-    )
+    counter: int = lt.property(default=0, readonly=True)
+    "A pointless counter"
 
 
 if __name__ == "__main__":

docs/source/see_also.rst

@@ -0,0 +1,48 @@
+See Also
+========
+
+LabThings-FastAPI makes quite heavy use of a few key concepts from external libraries, including `fastapi`, `pydantic`, and of course Python's core library. This page attempts to summarise these, and also acts as a useful place for docstrings to link to, so we can avoid repetition.
+
+.. _descriptors:
+
+Descriptors
+-----------
+
+Descriptors are a way to intercept attribute access on an object. By default, attributes of an object are just variables - so an object called ``foo`` might have an attribute called ``bar``, and you may read its value with ``foo.bar``, write its value with ``foo.bar = "baz"``, and delete the attribute with ``del foo.bar``. If ``foo`` is a descriptor, Python will call the ``__get__`` method of that descriptor when it's read and the ``__set__`` method when it's written to. You have quite probably used a descriptor already, because the built-in `~builtins.property` creates a descriptor object: that's what runs your getter method when the property is accessed. The descriptor protocol is described with plenty of examples in the `Descriptor Guide`_ in the Python documentation.
+
+In LabThings-FastAPI, descriptors are used to implement :ref:`wot_actions` and :ref:`wot_properties` on `.Thing` subclasses. The intention is that these will function like standard Python methods and properties, but will also be available over HTTP, along with automatic documentation in the :ref:`wot_td` and OpenAPI documents.
+
+There are a few useful notes that relate to many of the descriptors in LabThings-FastAPI:
+
+* Descriptor objects **may have more than one owner**. As a rule, a descriptor object
+    (e.g. an instance of `.DataProperty`) is assigned to an attribute of one `.Thing` subclass. There may, however, be multiple *instances* of that class, so it is not safe to assume that the descriptor object corresponds to only one `.Thing`. This is why the `.Thing` is passed to the ``__get__`` method: we should ensure that any values being remembered are keyed to the owning `.Thing` and are not simply stored in the descriptor. Usually, this is done using `.WeakKeyDictionary` objects, which allow us to look up values based on the `.Thing`, without interfering with garbage collection.
+
+    The example below shows how this can go wrong.
+
+    .. code-block:: python
+
+        class BadProperty:
+            "An example of a descriptor that has unwanted behaviour."
+            def __init__(self):
+                self._value = None
+            
+            def __get__(self, obj):
+                return self._value
+
+            def __set__(self, obj, val):
+                self._value = val
+
+        class BrokenExample:
+            myprop = BadProperty()
+
+        a = BrokenExample()
+        b = BrokenExample()
+
+        assert a.myprop is None
+        b.myprop = True
+        assert a.myprop is None  # FAILS because `myprop` shares values between a and b
+
+* Descriptor objects **may know their name**. Python calls ``__set_name__`` on a descriptor if it is available. This allows the descriptor to know the name of the attribute to which it is assigned. LabThings-FastAPI uses the name in the URL and in the Thing Description. When ``__set_name__`` is called, the descriptor **is also passed the class that owns it**. This allows us to check for type hints and docstrings that are part of the class, rather than part of the descriptor.
+* There is a convention that descriptors return their value when accessed as an instance attribute, but return themselves when accessed as a class attribute (as done by `builtins.property`). LabThings adheres to that convention.
+
+.. _`Descriptor Guide`: https://docs.python.org/3/howto/descriptor.html

docs/source/tutorial/index.rst

@@ -7,6 +7,8 @@ LabThings-FastAPI tutorial
 
    installing_labthings.rst
    running_labthings.rst
+   writing_a_thing.rst
+   properties.rst
 
 ..
    In due course, these pages should exist...

docs/source/tutorial/properties.rst

@@ -0,0 +1,136 @@
+.. _tutorial_properties:
+
+Properties
+=========================
+
+
+
+Properties are values that can be read from and written to a Thing. They are used to represent the state of the Thing, such as its current temperature, brightness, or status. :ref:`wot_properties` are a key concept in the Web of Things standard.
+
+LabThings implements properties in a very similar way to the built-in Python `~builtins.property`. The key difference is that defining an attribute as a `.property` means that the property will be listed in the :ref:`gen_td` and exposed over HTTP. This is important for two reasons:
+
+* Only properties declared using `.property` (usually imported as ``lt.property``) can be accessed over HTTP. Regular attributes or properties using `builtins.property` are only available to your `.Thing` internally, except in some special cases.
+* Communication between `.Thing`\ s within a LabThings server should be done using a `.DirectThingClient` class. The purpose of `.DirectThingClient` is to provide the same interface as a `.ThingClient` over HTTP, so it will also only expose functionality described in the Thing Description.
+
+You can add properties to a `.Thing` by using `.property` (usually imported as ``lt.property``).
+
+Data properties
+-------------------------
+
+Data properties behave like variables: they simply store a value that is used by other code on the `.Thing`. They are defined similarly to fields in `dataclasses` or `pydantic` models:
+
+.. code-block:: python
+
+    import labthings_fastapi as lt
+
+    class MyThing(lt.Thing):
+        my_property: int = lt.property(default=42)
+
+The example above defines a property called `my_property` that has a default value of `42`. Note the type hint `int` which indicates that the property should hold an integer value. This is important, as the type will be enforced when the property is written to via HTTP, and it will appear in :ref:`gen_docs`. By default, this property may be read or written to by HTTP requests. If you want to make it read-only, you can set the `readonly` parameter to `True`:
+
+.. code-block:: python
+
+    class MyThing(lt.Thing):
+        my_property: int = lt.property(default=42, readonly=True)
+
+Note that the ``readonly`` parameter only affects *client* code, i.e. it may not be written to via HTTP requests or `.DirectThingClient` instances. However, the property can still be modified by the Thing's code, e.g. in response to an action or another property change as ``self.my_property = 100``.
+
+It is a good idea to make sure there is a docstring for your property. This will be used in the :ref:`gen_docs`, and it will help users understand what the property is for. You can add a docstring to the property by placing a string immediately after the property definition:
+
+.. code-block:: python
+
+    class MyThing(lt.Thing):
+        my_property: int = lt.property(default=42, readonly=True)
+        """A property that holds an integer value."""
+
+You don't need to include the type in the docstring, as it will be inferred from the type hint. However, you can include additional information about the property, such as its units or any constraints on its value.
+
+Data properties may be *observed*, which means notifications will be sent when the property is written to (see below).
+
+Functional properties
+-------------------------
+
+It is also possible to have properties that run code when they are read or written to. These are called functional properties, and they are defined using the `lt.FunctionalProperty` class. They might communicate with hardware (for example to read or write a setting on an instrument), or they might perform some computation based on other properties. They are defined with a decorator, very similarly to the  built-in `property` function:
+
+.. code-block:: python
+
+    import labthings_fastapi as lt
+
+    class MyThing(lt.Thing):
+        my_property: int = lt.property(default=42)
+        """A property that holds an integer value."""
+
+        @lt.property
+        def twice_my_property(self) -> int:
+            """Twice the value of my_property."""
+            return self.my_property * 2
+
+The example above defines a functional property called `twice_my_property` that returns twice the value of `my_property`. The type hint `-> int` indicates that the property should return an integer value. When this property is read via HTTP, the code in the method will be executed, and the result will be returned to the client. As with `property`, the docstring of the property is taken from the method's docstring, so you can include additional information about the property there.
+
+Functional properties may also have a "setter" method, which is called when the property is written to via HTTP. This allows you to perform some action when the property is set, such as updating a hardware setting or performing some computation. The setter method should take a single argument, which is the new value of the property:
+
+.. code-block:: python
+
+    import labthings_fastapi as lt
+
+    class MyThing(lt.Thing):
+        my_property: int = lt.property(default=42)
+        """A property that holds an integer value."""
+
+        @lt.property
+        def twice_my_property(self) -> int:
+            """Twice the value of my_property."""
+            return self.my_property * 2
+
+        @twice_my_property.setter
+        def twice_my_property(self, value: int):
+            """Set the value of twice_my_property."""
+            self.my_property = value // 2
+
+Adding a setter makes the property read-write (if only a getter is present, it must be read-only). 
+
+It is possible to make a property read-only for clients by setting its ``readonly`` attribute: this has the same behaviour as for data properties.
+
+.. code-block:: python
+
+    import labthings_fastapi as lt
+
+    class MyThing(lt.Thing):
+        my_property: int = lt.property(default=42)
+        """A property that holds an integer value."""
+
+        @lt.property
+        def twice_my_property(self) -> int:
+            """Twice the value of my_property."""
+            return self.my_property * 2
+
+        @twice_my_property.setter
+        def twice_my_property(self, value: int):
+            """Set the value of twice_my_property."""
+            self.my_property = value // 2
+
+        # Make the property read-only for clients
+        twice_my_property.readonly = True
+
+In the example above, ``twice_my_property`` may be set by code within ``MyThing`` but cannot be written to via HTTP requests or `.DirectThingClient` instances.
+
+Functional properties may not be observed, as they are not backed by a simple value. If you need to notify clients when the value changes, you can use a data property that is updated by the functional property. In the example above, ``my_property`` may be observed, while ``twice_my_property`` cannot be observed. It would be possible to observe changes in ``my_property`` and then query ``twice_my_property`` for its new value.
+
+HTTP interface
+--------------
+
+LabThings is primarily controlled using HTTP. Mozilla have a good `Overview of HTTP`_ that is worth a read if you are unfamiliar with the concept of requests, or what ``GET`` and ``PUT`` mean.
+
+Each property in LabThings will be assigned a URL, which allows it to be read and (optionally) written to. The easiest way to explore this is in the interactive OpenAPI documentation, served by your LabThings server at ``/docs``\ . Properties can be read using a ``GET`` request and written using a ``PUT`` request.
+
+LabThings follows the `HTTP Protocol Binding`_ from the Web of Things standard. That's quite a detailed document: for a gentle introduction to HTTP and what a request means, see 
+
+.. _`Overview of HTTP`: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Overview
+.. _`HTTP Protocol Binding`: https://w3c.github.io/wot-binding-templates/bindings/protocols/http/index.html
+
+Observable properties
+-------------------------
+
+Properties can be made observable, which means that clients can subscribe to changes in the property's value. This is useful for properties that change frequently, such as sensor readings or instrument settings. In order for a property to be observable, LabThings must know whenever it changes. Currently, this means only data properties can be observed, as functional properties do not have a simple value that can be tracked.
+
+Properties are currently only observable via websockets: in the future, it may be possible to observe them from other `.Thing` instances or from other parts of the code.