diff --git a/documentation/api/change_log.rst b/documentation/api/change_log.rst index e110cfc802..48fc6c7628 100644 --- a/documentation/api/change_log.rst +++ b/documentation/api/change_log.rst @@ -5,6 +5,12 @@ API change log .. note:: The FlexMeasures API follows its own versioning scheme. This is also reflected in the URL (e.g. `/api/v3_0`), allowing developers to upgrade at their own pace. +v3.0-24 | 2025-06-10 +"""""""""""""""""""" +- New API endpoint `[POST] /assets/(id)/schedules/trigger `_ to schedule a site with multiple flexible devices. +- Updated message for 404 Not Found on endpoints for managing assets: `/assets` (GET, POST) and `/assets/` (GET, PATCH, DELETE). + + v3.0-23 | 2025-04-08 """""""""""""""""""" diff --git a/documentation/changelog.rst b/documentation/changelog.rst index 2b2c870095..db123167fc 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -14,6 +14,7 @@ v0.27.0 | August XX, 2025 New features ------------- +* New API endpoint `[POST] /assets/(id)/schedules/trigger `_ to schedule a site with multiple flexible devices [see `PR #1065 `_] * Add form to upload sensor data to the database [see `PR #1481 `_] * Allow editing users in the UI [see `PR #1502 `_] * Move various warnings to toast notifications [see `PR #1529 `_] diff --git a/documentation/features/scheduling.rst b/documentation/features/scheduling.rst index c0c78cffdc..5c00eef970 100644 --- a/documentation/features/scheduling.rst +++ b/documentation/features/scheduling.rst @@ -44,7 +44,7 @@ For more details on the possible formats for field values, see :ref:`variable_qu Where should you set these fields? Within requests to the API or by editing the relevant asset in the UI. -If they are not sent in via the API (the endpoint triggering schedule computation), the scheduler will look them up on the `flex-context` field of the asset. +If they are not sent in via the API (one of the endpoints triggering schedule computation), the scheduler will look them up on the `flex-context` field of the asset. And if the asset belongs to a larger system (a hierarchy of assets), the scheduler will also search if parent assets have them set. @@ -156,7 +156,7 @@ The process scheduler is suitable for shiftable, breakable and inflexible loads, We describe the respective flex models below. -At the moment, they have to be sent through the API (the endpoint to trigger schedule computation, or using the FlexMeasures client) or through the CLI (the command to add schedules). +At the moment, they have to be sent through the API (one of the endpoints to trigger schedule computation, or using the FlexMeasures client) or through the CLI (the command to add schedules). We will soon work on the possibility to store (a subset of) these fields on the data model and edit them in the UI. @@ -275,7 +275,7 @@ depending on the first target state of charge and the capabilities of the asset. Of course, we also log a failure in the scheduling job, so it's important to take note of these failures. Often, mis-configured flex models are the reason. -For a hands-on tutorial on using some of the storage flex-model fields, head over to :ref:`tut_v2g` use case and `the API documentation for triggering schedules <../api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_. +For a hands-on tutorial on using some of the storage flex-model fields, head over to :ref:`tut_v2g` use case and `the API documentation for triggering schedules <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_. Finally, are you interested in the linear programming details behind the storage scheduler? Then head over to :ref:`storage_device_scheduler`! diff --git a/documentation/tut/forecasting_scheduling.rst b/documentation/tut/forecasting_scheduling.rst index f7a1cfb1f9..fecd9c265b 100644 --- a/documentation/tut/forecasting_scheduling.rst +++ b/documentation/tut/forecasting_scheduling.rst @@ -99,7 +99,7 @@ It usually involves a linear program that combines a state of energy flexibility There are two ways to queue a scheduling job: First, we can add a scheduling job to the queue via the API. -We already learned about the `[POST] /schedules/trigger <../api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_ endpoint in :ref:`posting_flex_states`, where we saw how to post a flexibility state (in this case, the state of charge of a battery at a certain point in time). +We already learned about the `[POST] /schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_ endpoint in :ref:`posting_flex_states`, where we saw how to post a flexibility state (in this case, the state of charge of a battery at a certain point in time). Here, we extend that (storage) example with an additional target value, representing a desired future state of charge. @@ -108,6 +108,7 @@ Here, we extend that (storage) example with an additional target value, represen { "start": "2015-06-02T10:00:00+00:00", "flex-model": { + "sensor": 15, "soc-at-start": "12.1 kWh", "soc-targets": [ { @@ -130,7 +131,7 @@ A second way to add scheduling jobs is via the CLI, so this is available for peo .. code-block:: bash - $ flexmeasures add schedule for-storage --sensor 1 --consumption-price-sensor 2 \ + $ flexmeasures add schedule for-storage --sensor 15 --consumption-price-sensor 2 \ --start 2022-07-05T07:00+01:00 --duration PT12H \ --soc-at-start 50% --roundtrip-efficiency 90% --as-job @@ -154,7 +155,7 @@ A prognosis can be requested at a URL looking like this: .. code-block:: html - https://company.flexmeasures.io/api//sensors/data + https://company.flexmeasures.io/api/v3_0/sensors/data This example requests a prognosis for 24 hours, with a rolling horizon of 6 hours before realisation. @@ -180,11 +181,23 @@ We saw above how FlexMeasures can create optimised schedules with control signal .. code-block:: html - https://company.flexmeasures.io/api//sensors//schedules/ + https://company.flexmeasures.io/api/v3_0/sensors//schedules/ -Here, the schedule's Universally Unique Identifier (UUID) should be filled in that is returned in the `[POST] /schedules/trigger <../api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_ response. +Here, the schedule's Universally Unique Identifier (UUID) should be filled in that is returned in the `[POST] /schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_ response. Schedules can be queried by their UUID for up to 1 week after they were triggered (ask your host if you need to keep them around longer). Afterwards, the exact schedule can still be retrieved through the `[GET] /sensors/data <../api/v3_0.html#get--api-v3_0-sensors-data>`_, using precise filter values for ``start``, ``prior`` and ``source``. +Besides the UUID, the endpoint for retrieving schedules takes a sensor ID, which is the sensor ID of one of the power sensors that was referenced in the flex model. + +.. note:: If a ``state-of-charge`` sensor was referenced in the flex model (like in the example below), the scheduled state of charge can be retrieved using the same endpoint and UUID, but then using the state-of-charge sensor ID. + + .. code-block:: json + + "flex-model": { + "sensor": 15, + "state-of-charge": {"sensor": 16} + } + + For instance, if the above snippet represents the flex model used by FlexMeasures to compute the schedule, then to fetch the scheduled state of charge you simply replace the power sensor ID in the URL of the `[GET] /sensors/data <../api/v3_0.html#get--api-v3_0-sensors-data>`_ endpoint with the state-of-charge sensor ID. The following example response indicates that FlexMeasures planned ahead 45 minutes for the requested battery power sensor. The list of consecutive power values represents the target consumption of the battery (negative values for production). @@ -212,3 +225,8 @@ However, because the targets values represent averages over 15-minute time inter For example, the battery might start to consume with 2.1 MW at 10.00am and increase its consumption to 2.25 at 10.10am, increase its consumption to 5 MW at 10.15am and decrease its consumption to 2 MW at 10.20am. That should result in the same average values for each quarter-hour. + +Likewise, the control signals can be used to schedule devices that only run at specific power levels. +For example, let's assume the battery only supports running at an integer number of MW. +In that case, the battery could start to consume with 2 MW at 10.00am, increase its consumption to 3 MW at (15 seconds before) 10.12am, and decrease its consumption to 2 MW at 10.30am. +Again, this results in the same average values for each quarter-hour. diff --git a/documentation/tut/posting_data.rst b/documentation/tut/posting_data.rst index 6eb63b6419..3452a9a671 100644 --- a/documentation/tut/posting_data.rst +++ b/documentation/tut/posting_data.rst @@ -46,7 +46,7 @@ The exact URL will depend on your domain name, and will look approximately like .. code-block:: html - [POST] https://company.flexmeasures.io/api//sensors/data + [POST] https://company.flexmeasures.io/api/v3_0/sensors/data This example "PostSensorDataRequest" message posts prices for hourly intervals between midnight and midnight the next day for the Korean Power Exchange (KPX) day-ahead auction, registered under sensor 16. @@ -271,14 +271,18 @@ There is one more crucial kind of data that FlexMeasures needs to know about: Wh For example, a battery has a certain state of charge, which is relevant to describe the flexibility that the battery currently has. In our terminology, this is called the "flex model" and you can read more at :ref:`describing_flexibility`. -Owners of such devices can post the flex model along with triggering the creation of a new schedule, to `[POST] /schedules/trigger <../api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_. +Owners of such devices can post the flex model along with triggering the creation of a new schedule, to one of two endpoints: + +1. `[POST] /assets//schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_ - for scheduling multiple devices +2. `[POST] /sensors//schedules/trigger <../api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_ - for scheduling a single device (which can also be done with the first endpoint) + The URL might look like this: .. code-block:: html - https://company.flexmeasures.io/api//sensors/10/schedules/trigger + https://company.flexmeasures.io/api/v3_0/assets/10/schedules/trigger -The following example triggers a schedule for a power sensor (with ID 10) of a battery asset, asking to take into account the battery's current state of charge. +The following example triggers a schedule for a power sensor (with ID 15) of a battery asset (with ID 10), asking to take into account the battery's current state of charge. From this, FlexMeasures derives the energy flexibility this battery has in the next 48 hours and computes an optimal charging schedule. The endpoint also allows to limit the flexibility range and also to set target values. @@ -286,13 +290,27 @@ The endpoint also allows to limit the flexibility range and also to set target v { "start": "2015-06-02T10:00:00+00:00", - "flex-model": { - "soc-at-start": "12.1 kWh" - } + "flex-model": [ + { + "sensor": 15, + "soc-at-start": "12.1 kWh" + } + ] } .. note:: More details on supported flex models can be found in :ref:`flex_models_and_schedulers`. -.. note:: Flexibility states are persisted on sensor attributes. To record a more complete history of the state of charge, set up a separate sensor and post data to it using `[POST] /sensors/data <../api/v3_0.html#post--api-v3_0-sensors-data>`_ (see :ref:`posting_sensor_data`). +.. note:: + Flexibility states posted in trigger messages are only stored temporarily to describe the scheduling job. + To record a more complete history of the flexibility state, set up separate sensors and post data to them using `[POST] /sensors/data <../api/v3_0.html#post--api-v3_0-sensors-data>`_ (see :ref:`posting_sensor_data`). + Then reference those sensors in your flex model. + For example, say you use sensor 82 to record the power-to-heat efficiency of a heating system, then use this sensor reference in your flex model: + + .. code-block:: json + + { + "charging-efficiency": {"sensor": 82} + } + In :ref:`how_queue_scheduling`, we'll cover what happens when FlexMeasures is triggered to create a new schedule, and how those schedules can be retrieved via the API, so they can be used to steer assets. \ No newline at end of file diff --git a/documentation/tut/toy-example-expanded.rst b/documentation/tut/toy-example-expanded.rst index 5f084bb056..7a66fd5fad 100644 --- a/documentation/tut/toy-example-expanded.rst +++ b/documentation/tut/toy-example-expanded.rst @@ -63,7 +63,7 @@ Setting the data source type to "forecaster" helps FlexMeasures to visually dist $ flexmeasures add beliefs --sensor 3 --source 4 solar-tomorrow.csv --timezone Europe/Amsterdam Successfully created beliefs -The one-hour CSV data is automatically resampled to the 15-minute resolution of the sensor that is recording solar production. We can see solar production in the `FlexMeasures UI `_ : +The one-hour CSV data is automatically resampled to the 15-minute resolution of the sensor that is recording solar production. We can see solar production in the `FlexMeasures UI `_: .. image:: https://github.com/FlexMeasures/screenshots/raw/main/tut/toy-schedule/sensor-data-production.png :align: center @@ -75,17 +75,139 @@ The one-hour CSV data is automatically resampled to the 15-minute resolution of Trigger an updated schedule ---------------------------- -Now, we'll reschedule the battery while taking into account the solar production. This will have an effect on the available headroom for the battery, given the ``site-power-capacity`` limit discussed earlier. - -.. code-block:: bash - - $ flexmeasures add schedule for-storage --sensor 2 --consumption-price-sensor 1 \ - --inflexible-device-sensor 3 \ - --start ${TOMORROW}T07:00+02:00 --duration PT12H \ - --soc-at-start 50% --roundtrip-efficiency 90% - New schedule is stored. - -We can see the updated scheduling in the `FlexMeasures UI `_ : +Now, we'll reschedule the battery while taking into account the solar production (forecast) as an inflexible device. +This will have an effect on the available headroom for the battery, given the ``site-power-capacity`` limit discussed earlier. + +.. tabs:: + + .. tab:: CLI + + .. code-block:: bash + :emphasize-lines: 3 + + $ flexmeasures add schedule for-storage \ + --sensor 2 \ + --inflexible-device-sensor 3 \ + --start ${TOMORROW}T07:00+01:00 \ + --duration PT12H \ + --soc-at-start 50% \ + --roundtrip-efficiency 90% + New schedule is stored. + + .. tab:: API + + Example call: `[POST] http://localhost:5000/api/v3_0/assets/2/schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_ (update the start date to tomorrow): + + .. code-block:: json + :emphasize-lines: 11-13 + + { + "start": "2025-06-11T07:00+01:00", + "duration": "PT12H", + "flex-model": [ + { + "sensor": 2, + "soc-at-start": "50%", + "roundtrip-efficiency": "90%" + } + ], + "flex-context": { + "inflexible-device-sensors": [3] + } + } + + Alternatively, if the solar production is curtailable, move the solar production to the flex-model. + There, we tell the scheduler to pick any production value between 0 and the production forecast recorded on sensor 3, and to store the resulting schedule on sensor 3 as well (the FlexMeasures UI will still be able to distinguish forecasts from schedules): + + .. code-block:: json + :emphasize-lines: 10-14,16 + + { + "start": "2025-06-11T07:00+01:00", + "duration": "PT12H", + "flex-model": [ + { + "sensor": 2, + "soc-at-start": "50%", + "roundtrip-efficiency": "90%" + }, + { + "sensor": 3, + "consumption-capacity": "0 kW", + "production-capacity": {"sensor": 3}, + } + ], + "flex-context": {} + } + + .. tab:: FlexMeasures Client + + Using the `FlexMeasures Client `_: + + .. code-block:: bash + + pip install flexmeasures-client + + .. code-block:: python + :emphasize-lines: 22-24 + + import asyncio + from datetime import date + from flexmeasures_client import FlexMeasuresClient as Client + + async def client_script(): + client = Client( + email="toy-user@flexmeasures.io", + password="toy-password", + host="localhost:5000", + ) + schedule = await client.trigger_and_get_schedule( + asset_id=2, # Toy building (asset ID) + start=f"{date.today().isoformat()}T07:00+01:00", + duration="PT12H", + flex_model=[ + { + "sensor": 2, # battery power (sensor ID) + "soc-at-start": "50%", + "roundtrip-efficiency": "90%", + }, + ], + flex_context={ + "inflexible-device-sensors": [3], # solar production (sensor ID) + }, + ) + print(schedule) + await client.close() + + asyncio.run(client_script()) + + Alternatively, if the solar production is curtailable, move the solar production to the flex-model: + + .. code-block:: python + :emphasize-lines: 11-15,17 + + schedule = await client.trigger_and_get_schedule( + asset_id=2, # Toy building (asset ID) + start=f"{date.today().isoformat()}T07:00+01:00", + duration="PT12H", + flex_model=[ + { + "sensor": 2, # battery power (sensor ID) + "soc-at-start": "50%", + "roundtrip-efficiency": "90%", + }, + { + "sensor": 3, # solar production (sensor ID) + "consumption-capacity": "0 kW", + "production-capacity": {"sensor": 3}, + }, + ], + flex_context={}, + ) + + + +We can see the updated scheduling in the `FlexMeasures UI `_: .. image:: https://github.com/FlexMeasures/screenshots/raw/main/tut/toy-schedule/sensor-data-charging-with-solar.png :align: center @@ -117,7 +239,7 @@ In the case of the scheduler that we ran in the previous tutorial, which did not .. note:: You can add arbitrary sensors to a chart using the asset UI or the attribute ``sensors_to_show``. See :ref:`view_asset-data` for more. -A nice feature is that you can check the data connectivity status of your building asset. Now that we have made the schedule, both lamps are green. You can also view it in `FlexMeasures UI `_ : +A nice feature is that you can check the data connectivity status of your building asset. Now that we have made the schedule, both lamps are green. You can also view it in `FlexMeasures UI `_: .. image:: https://github.com/FlexMeasures/screenshots/raw/main/tut/toy-schedule/screenshot_building_status.png :align: center diff --git a/documentation/tut/toy-example-from-scratch.rst b/documentation/tut/toy-example-from-scratch.rst index 30fc907f4e..5ee37867aa 100644 --- a/documentation/tut/toy-example-from-scratch.rst +++ b/documentation/tut/toy-example-from-scratch.rst @@ -19,14 +19,77 @@ Make a schedule After going through the setup, we can finally create the schedule, which is the main benefit of FlexMeasures (smart real-time control). -We'll ask FlexMeasures for a schedule for our (dis)charging sensor (ID 2). We also need to specify what to optimize against. Here we pass the Id of our market price sensor (ID 1). +We'll ask FlexMeasures for a schedule for our (dis)charging sensor (ID 2). To keep it short, we'll only ask for a 12-hour window starting at 7am. Finally, the scheduler should know what the state of charge of the battery is when the schedule starts (50%) and what its roundtrip efficiency is (90%). -.. code-block:: bash +.. tabs:: + + .. tab:: CLI + + .. code-block:: bash + + $ flexmeasures add schedule for-storage \ + --sensor 2 \ + --start ${TOMORROW}T07:00+01:00 \ + --duration PT12H \ + --soc-at-start 50% \ + --roundtrip-efficiency 90% + New schedule is stored. + + .. tab:: API + + Example call: `[POST] http://localhost:5000/api/v3_0/assets/2/schedules/trigger <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_ (update the start date to tomorrow): + + .. code-block:: json + + { + "start": "2025-06-11T07:00+01:00", + "duration": "PT12H", + "flex-model": [ + "sensor": 2, + "soc-at-start": "50%", + "roundtrip-efficiency": "90%" + ] + } + + .. tab:: FlexMeasures Client + + Using the `FlexMeasures Client `_: + + .. code-block:: bash + + pip install flexmeasures-client + + .. code-block:: python + + import asyncio + from datetime import date + from flexmeasures_client import FlexMeasuresClient as Client + + async def client_script(): + client = Client( + email="toy-user@flexmeasures.io", + password="toy-password", + host="localhost:5000", + ) + schedule = await client.trigger_and_get_schedule( + asset_id=2, # Toy building (asset ID) + start=f"{date.today().isoformat()}T07:00+01:00", + duration="PT12H", + flex_model=[ + { + "sensor": 2, # battery power (sensor ID) + "soc-at-start": "50%", + "roundtrip-efficiency": "90%", + }, + ], + ) + print(schedule) + await client.close() + + asyncio.run(client_script()) - $ flexmeasures add schedule for-storage --sensor 2 --start ${TOMORROW}T07:00+01:00 --duration PT12H \ - --soc-at-start 50% --roundtrip-efficiency 90% - New schedule is stored. +.. note:: We already specified what to optimize against by having set the consumption price sensor in the flex-context of the battery (see :ref:`tut_load_data`). Great. Let's see what we made: diff --git a/documentation/tut/toy-example-process.rst b/documentation/tut/toy-example-process.rst index bcf9a93376..3fcf9ce440 100644 --- a/documentation/tut/toy-example-process.rst +++ b/documentation/tut/toy-example-process.rst @@ -27,7 +27,7 @@ Moreover, we'll touch upon the use of time restrictions to avoid scheduling a pr Setup -..... +----- Before moving forward, we'll add the `process` asset and three sensors to store the schedules resulting from following three different policies. @@ -36,15 +36,15 @@ Before moving forward, we'll add the `process` asset and three sensors to store $ flexmeasures add toy-account --kind process - User with email toy-user@flexmeasures.io already exists in account Docker Toy Account. - The sensor recording day-ahead prices is day-ahead prices (ID: 1). - Created - Created - Created - Created - The sensor recording the power of the inflexible load is Power (Inflexible) (ID: 4). - The sensor recording the power of the breakable load is Power (Breakable) (ID: 5). - The sensor recording the power of the shiftable load is Power (Shiftable) (ID: 6). + User with email toy-user@flexmeasures.io already exists in account Docker Toy Account. + The sensor recording day-ahead prices is day-ahead prices (ID: 1). + Created + Created + Created + Created + The sensor recording the power of the inflexible load is Power (Inflexible) (ID: 4). + The sensor recording the power of the breakable load is Power (Breakable) (ID: 5). + The sensor recording the power of the shiftable load is Power (Shiftable) (ID: 6). Trigger an updated schedule @@ -59,7 +59,7 @@ Now we are ready to schedule a process. Let's start with the INFLEXIBLE policy, .. code-block:: bash - flexmeasures add schedule for-process --sensor 4 --consumption-price-sensor 1\ + $ flexmeasures add schedule for-process --sensor 4 \ --start ${TOMORROW}T00:00:00+02:00 --duration PT24H --process-duration PT4H \ --process-power 0.2MW --process-type INFLEXIBLE \ --forbid "{\"start\" : \"${TOMORROW}T15:00:00+02:00\", \"duration\" : \"PT1H\"}" @@ -70,7 +70,7 @@ Following the INFLEXIBLE policy, we'll schedule the same 4h block using a BREAKA .. code-block:: bash - flexmeasures add schedule for-process --sensor 5 --consumption-price-sensor 1\ + $ flexmeasures add schedule for-process --sensor 5 \ --start ${TOMORROW}T00:00:00+02:00 --duration PT24H --process-duration PT4H \ --process-power 0.2MW --process-type BREAKABLE \ --forbid "{\"start\" : \"${TOMORROW}T15:00:00+02:00\", \"duration\" : \"PT1H\"}" @@ -81,7 +81,7 @@ Finally, we'll schedule the process using the SHIFTABLE policy. .. code-block:: bash - flexmeasures add schedule for-process --sensor 6 --consumption-price-sensor 1\ + $ flexmeasures add schedule for-process --sensor 6 \ --start ${TOMORROW}T00:00:00+02:00 --duration PT24H --process-duration PT4H \ --process-power 0.2MW --process-type SHIFTABLE \ --forbid "{\"start\" : \"${TOMORROW}T15:00:00+02:00\", \"duration\" : \"PT1H\"}" diff --git a/documentation/tut/toy-example-reporter.rst b/documentation/tut/toy-example-reporter.rst index 9fb895f9fb..bcf968e784 100644 --- a/documentation/tut/toy-example-reporter.rst +++ b/documentation/tut/toy-example-reporter.rst @@ -22,7 +22,7 @@ In the second part, we'll use the `ProfitOrLossReporter` to compute the costs of Before getting to the meat of the tutorial, we need to set up up all the entities. Instead of having to do that manually (e.g. using commands such as ``flexmeasures add sensor``), we have prepared a command that does that automatically. Setup -..... +----- Just as in previous sections, we need to run the command ``flexmeasures add toy-account``, but this time with a different value for *kind*: @@ -48,29 +48,29 @@ Run the command below to show the values for our newly-created `grid connection $ flexmeasures show beliefs --sensor 7 --start ${TOMORROW}T00:00:00+02:00 --duration PT24H --resolution PT1H Beliefs for Sensor 'grid connection capacity' (ID 7). - Data spans a day and starts at 2023-08-14 00:00:00+02:00. - The time resolution (x-axis) is an hour. - ┌────────────────────────────────────────────────────────────┐ - │ │ - │ │ - │ │ - │ │ - │ │ 1.0MW - │ │ - │ │ - │ │ - │▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀│ 0.5MW - │ │ - │ │ - │ │ - │▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁│ 0.0MW - │ │ - │ │ - │ │ - │ │ -0.5MW - └────────────────────────────────────────────────────────────┘ - 5 10 15 20 - ██ grid connection capacity + Data spans a day and starts at 2025-06-13 00:00:00+02:00. + The time resolution (x-axis) is an hour. + ┌────────────────────────────────────────────────────────────┐ + │ │ + │ │ + │ │ + │ │ + │ │ 1.0MW + │ │ + │ │ + │ │ + │▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄│ 0.5MW + │ │ + │ │ + │ │ + │▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁│ 0.0MW + │ │ + │ │ + │ │ + │ │ -0.5MW + └────────────────────────────────────────────────────────────┘ + 06:00 12:00 18:00 + ██ grid connection capacity (toy-building) Moreover, we can check the freshly created source ``, which defines the `ProfitOrLossReporter` with the required configuration. @@ -81,19 +81,19 @@ That's because reporters belong to a bigger category of classes that also contai $ flexmeasures show data-sources --show-attributes --id 6 - type: reporter - ======== - - ID Name User ID Model Version Attributes - ---- ------------ --------- -------------------- --------- ------------------------------------------ - 6 FlexMeasures ProfitOrLossReporter { - "data_generator": { - "config": { - "consumption_price_sensor": 1, - "loss_is_positive": true - } + type: reporter + ======== + + ID Name User ID Model Version Attributes + ---- ------------ --------- -------------------- --------- ------------------------------------------ + 6 FlexMeasures ProfitOrLossReporter { + "data_generator": { + "config": { + "consumption_price_sensor": 1, + "loss_is_positive": true } } + } Compute headroom diff --git a/documentation/tut/toy-example-setup.rst b/documentation/tut/toy-example-setup.rst index 36209ae353..0b98e836e5 100644 --- a/documentation/tut/toy-example-setup.rst +++ b/documentation/tut/toy-example-setup.rst @@ -19,7 +19,7 @@ Below are the ``flexmeasures`` CLI commands we'll run, and which we'll explain s # setup an account with a user, assets for battery & solar and an energy market (ID 1) $ flexmeasures add toy-account - # load prices to optimise the schedule against + # load prices to optimize schedules against $ flexmeasures add beliefs --sensor 1 --source toy-user prices-tomorrow.csv --timezone Europe/Amsterdam @@ -131,6 +131,8 @@ Install Flexmeasures and the database .. include:: ../notes/macOS-port-note.rst +.. _tut_load_data: + Add some structural data --------------------------------------- @@ -195,6 +197,7 @@ If you want, you can inspect what you created: 4 toy-solar solar 2 (52.374, 4.88969) .. code-block:: bash + :emphasize-lines: 30 $ flexmeasures show asset --id 2 @@ -210,7 +213,7 @@ If you want, you can inspect what you created: Child assets of toy-building (ID: 2) ==================================== - Id Name Type + ID Name Type ------- ----------------- ---------------------------- 3 toy-battery battery 4 toy-solar solar diff --git a/flexmeasures/api/common/schemas/generic_assets.py b/flexmeasures/api/common/schemas/generic_assets.py index cdd5fe81b3..e69de29bb2 100644 --- a/flexmeasures/api/common/schemas/generic_assets.py +++ b/flexmeasures/api/common/schemas/generic_assets.py @@ -1,25 +0,0 @@ -from __future__ import annotations - -from flask import abort -from marshmallow import fields -from sqlalchemy import select - -from flexmeasures.data import db -from flexmeasures.data.models.generic_assets import GenericAsset - - -class AssetIdField(fields.Integer): - """ - Field that represents a generic asset ID. It de-serializes from the asset id to an asset instance. - """ - - def _deserialize(self, asset_id: int, attr, obj, **kwargs) -> GenericAsset: - asset: GenericAsset = db.session.execute( - select(GenericAsset).filter_by(id=int(asset_id)) - ).scalar_one_or_none() - if asset is None: - raise abort(404, f"GenericAsset {asset_id} not found") - return asset - - def _serialize(self, asset: GenericAsset, attr, data, **kwargs) -> int: - return asset.id diff --git a/flexmeasures/api/common/utils/args_parsing.py b/flexmeasures/api/common/utils/args_parsing.py index 32383ff7f3..4bc50567f2 100644 --- a/flexmeasures/api/common/utils/args_parsing.py +++ b/flexmeasures/api/common/utils/args_parsing.py @@ -38,10 +38,28 @@ def validation_error_handler(error: FMValidationError): @parser.location_loader("args_and_json") def load_data(request, schema): """ - We allow parameters to come from either GET args or POST JSON, - as validators can be attached to either. + We allow parameters to come from URL path, GET args and POST JSON, + as validators can be attached to any of them. """ + + # GET args (i.e. query parameters, such as https://flexmeasures.io/?id=5) newdata = request.args.copy() + + # View args (i.e. path parameters, such as the `/assets/` endpoint) + path_params = request.view_args + # Avoid clashes such as visiting https://flexmeasures.io/assets/4/?id=5 on the /assets/ endpoint + for key in path_params: + if key in newdata: + raise FMValidationError(message=f"{key} already set in the URL path") + newdata.update(path_params) + if request.mimetype == "application/json" and request.method == "POST": - newdata.update(request.get_json()) + json_params = request.get_json() + # Avoid clashes + for key in json_params: + if key in newdata: + raise FMValidationError( + message=f"{key} already set in the URL path or query parameters" + ) + newdata.update(json_params) return MultiDictProxy(newdata, schema) diff --git a/flexmeasures/api/tests/utils.py b/flexmeasures/api/tests/utils.py index 61b71f8fe7..ed2a719e33 100644 --- a/flexmeasures/api/tests/utils.py +++ b/flexmeasures/api/tests/utils.py @@ -1,5 +1,7 @@ from __future__ import annotations +from pytest import UsageError + import json from flask import url_for, current_app, Response @@ -71,7 +73,12 @@ class UserContext(object): """ def __init__(self, user_email: str): - self.the_user = find_user_by_email(user_email) + user = find_user_by_email(user_email) + if user is None: + raise UsageError( + f"no user with email {user_email} found - test is possible missing a fixture that sets up this user", + ) + self.the_user = user def __enter__(self): return self.the_user diff --git a/flexmeasures/api/v3_0/assets.py b/flexmeasures/api/v3_0/assets.py index f4bbfadbb7..3cbbcb449d 100644 --- a/flexmeasures/api/v3_0/assets.py +++ b/flexmeasures/api/v3_0/assets.py @@ -1,5 +1,7 @@ from __future__ import annotations import json +from datetime import datetime, timedelta +from http import HTTPStatus from humanize import naturaldelta from flask import current_app, request @@ -9,7 +11,7 @@ from flask_json import as_json from flask_sqlalchemy.pagination import SelectPagination -from marshmallow import fields +from marshmallow import fields, ValidationError import marshmallow.validate as validate from webargs.flaskparser import use_kwargs, use_args @@ -26,8 +28,19 @@ from flexmeasures.data.models.generic_assets import GenericAsset from flexmeasures.data.queries.generic_assets import query_assets_by_search_terms from flexmeasures.data.schemas import AwareDateTimeField -from flexmeasures.data.schemas.generic_assets import GenericAssetSchema as AssetSchema -from flexmeasures.api.common.schemas.generic_assets import AssetIdField +from flexmeasures.data.schemas.generic_assets import ( + GenericAssetSchema as AssetSchema, + GenericAssetIdField as AssetIdField, +) +from flexmeasures.data.schemas.scheduling import AssetTriggerSchema +from flexmeasures.data.services.scheduling import ( + create_sequential_scheduling_job, + create_simultaneous_scheduling_job, +) +from flexmeasures.api.common.responses import ( + invalid_flex_config, + request_processed, +) from flexmeasures.api.common.schemas.search import SearchFilterField from flexmeasures.api.common.schemas.users import AccountIdField from flexmeasures.utils.coding_utils import flatten_unique @@ -427,7 +440,14 @@ def post(self, asset_data: dict): return asset_schema.dump(asset), 201 @route("/", methods=["GET"]) - @use_kwargs({"asset": AssetIdField(data_key="id")}, location="path") + @use_kwargs( + { + "asset": AssetIdField( + data_key="id", status_if_not_found=HTTPStatus.NOT_FOUND + ) + }, + location="path", + ) @permission_required_for_context("read", ctx_arg_name="asset") @as_json def fetch_one(self, id, asset): @@ -463,7 +483,14 @@ def fetch_one(self, id, asset): @route("/", methods=["PATCH"]) @use_args(partial_asset_schema) - @use_kwargs({"db_asset": AssetIdField(data_key="id")}, location="path") + @use_kwargs( + { + "db_asset": AssetIdField( + data_key="id", status_if_not_found=HTTPStatus.NOT_FOUND + ) + }, + location="path", + ) @permission_required_for_context("update", ctx_arg_name="db_asset") @as_json def patch(self, asset_data: dict, id: int, db_asset: GenericAsset): @@ -546,7 +573,14 @@ def patch(self, asset_data: dict, id: int, db_asset: GenericAsset): return asset_schema.dump(db_asset), 200 @route("/", methods=["DELETE"]) - @use_kwargs({"asset": AssetIdField(data_key="id")}, location="path") + @use_kwargs( + { + "asset": AssetIdField( + data_key="id", status_if_not_found=HTTPStatus.NOT_FOUND + ) + }, + location="path", + ) @permission_required_for_context("delete", ctx_arg_name="asset") @as_json def delete(self, id: int, asset: GenericAsset): @@ -575,7 +609,11 @@ def delete(self, id: int, asset: GenericAsset): @route("//chart", strict_slashes=False) # strict on next version? see #1014 @use_kwargs( - {"asset": AssetIdField(data_key="id")}, + { + "asset": AssetIdField( + data_key="id", status_if_not_found=HTTPStatus.NOT_FOUND + ) + }, location="path", ) @use_kwargs( @@ -606,7 +644,11 @@ def get_chart(self, id: int, asset: GenericAsset, **kwargs): "//chart_data", strict_slashes=False ) # strict on next version? see #1014 @use_kwargs( - {"asset": AssetIdField(data_key="id")}, + { + "asset": AssetIdField( + data_key="id", status_if_not_found=HTTPStatus.NOT_FOUND + ) + }, location="path", ) @use_kwargs( @@ -884,3 +926,146 @@ def update_default_asset_view(self, **kwargs): return { "message": "Default asset view updated successfully.", }, 200 + + @route("//schedules/trigger", methods=["POST"]) + @use_args(AssetTriggerSchema(), location="args_and_json", as_kwargs=True) + # Simplification of checking for create-children access on each of the flexible sensors, + # which assumes each of the flexible sensors belongs to the given asset. + @permission_required_for_context("create-children", ctx_arg_name="asset") + def trigger_schedule( + self, + asset: GenericAsset, + start_of_schedule: datetime, + duration: timedelta, + belief_time: datetime | None = None, + flex_model: dict | None = None, + flex_context: dict | None = None, + sequential: bool = False, + **kwargs, + ): + """ + Trigger FlexMeasures to create a schedule for a collection of flexible and inflexible devices. + + .. :quickref: Schedule; Trigger scheduling job for any number of devices + + Trigger FlexMeasures to create a schedule for this asset. + The flex-model references the power sensors of flexible devices, which must belong to the given asset, + either directly or indirectly, by being assigned to one of the asset's (grand)children. + + In this request, you can describe: + + - the schedule's main features (when does it start, what unit should it report, prior to what time can we assume knowledge) + - the flexibility models for the asset's relevant sensors (state and constraint variables, e.g. current state of charge of a battery, or connection capacity) + - the flexibility context which the asset operates in (other sensors under the same EMS which are relevant, e.g. prices) + + For details on flexibility model and context, see :ref:`describing_flexibility`. + Below, we'll also list some examples. + + .. note:: This endpoint supports scheduling an EMS with multiple flexible devices at once. + It can do so jointly (the default) or sequentially + (considering previously scheduled sensors as inflexible). + To use sequential scheduling, use ``sequential=true`` in the JSON body. + + The length of the schedule can be set explicitly through the 'duration' field. + Otherwise, it is set by the config setting :ref:`planning_horizon_config`, which defaults to 48 hours. + If the flex-model contains targets that lie beyond the planning horizon, the length of the schedule is extended to accommodate them. + Finally, the schedule length is limited by :ref:`max_planning_horizon_config`, which defaults to 2520 steps of each sensor's resolution. + Targets that exceed the max planning horizon are not accepted. + + The appropriate algorithm is chosen by FlexMeasures (based on asset type). + It's also possible to use custom schedulers and custom flexibility models, see :ref:`plugin_customization`. + + If you have ideas for algorithms that should be part of FlexMeasures, let us know: https://flexmeasures.io/get-in-touch/ + + **Example request** + + This message triggers a schedule for a storage asset (with power sensor 931), + starting at 10.00am, when the state of charge (soc) should be assumed to be 12.1 kWh, + and also schedules a curtailable production asset (with power sensor 932), + whose production forecasts are recorded under sensor 760. + + Aggregate consumption (of all devices within this EMS) should be priced by sensor 9, + and aggregate production should be priced by sensor 10, + where the aggregate power flow in the EMS is described by the sum over sensors 13, 14, 15, + and the two power sensors (931 and 932) of the flexible devices being optimized (referenced in the flex-model). + + The battery consumption power capacity is limited by sensor 42 and the production capacity is constant (30 kW). + Finally, the site consumption capacity is limited by sensor 32. + + .. code-block:: json + + { + "start": "2015-06-02T10:00:00+00:00", + "flex-model": [ + { + "sensor": 931, + "soc-at-start": 12.1, + "soc-unit": "kWh", + "power-capacity": "25kW", + "consumption-capacity" : {"sensor": 42}, + "production-capacity" : "30 kW" + }, + { + "sensor": 932, + "consumption-capacity": "0 kW", + "production-capacity": {"sensor": 760}, + } + ], + "flex-context": { + "consumption-price-sensor": 9, + "production-price-sensor": 10, + "inflexible-device-sensors": [13, 14, 15], + "site-power-capacity": "100kW", + "site-production-capacity": "80kW", + "site-consumption-capacity": {"sensor": 32} + } + } + + **Example response** + + This message indicates that the scheduling request has been processed without any error. + A scheduling job has been created with some Universally Unique Identifier (UUID), + which will be picked up by a worker. + The given UUID may be used to obtain the resulting schedule for each flexible device: see /sensors//schedules/. + + .. sourcecode:: json + + { + "status": "PROCESSED", + "schedule": "364bfd06-c1fa-430b-8d25-8f5a547651fb", + "message": "Request has been processed." + } + + :reqheader Authorization: The authentication token + :reqheader Content-Type: application/json + :resheader Content-Type: application/json + :status 200: PROCESSED + :status 400: INVALID_DATA + :status 401: UNAUTHORIZED + :status 403: INVALID_SENDER + :status 405: INVALID_METHOD + :status 422: UNPROCESSABLE_ENTITY + """ + end_of_schedule = start_of_schedule + duration + + scheduler_kwargs = dict( + start=start_of_schedule, + end=end_of_schedule, + belief_time=belief_time, # server time if no prior time was sent + flex_model=flex_model, + flex_context=flex_context, + ) + if sequential: + f = create_sequential_scheduling_job + else: + f = create_simultaneous_scheduling_job + try: + job = f(asset=asset, enqueue=True, **scheduler_kwargs) + except ValidationError as err: + return invalid_flex_config(err.messages) + except ValueError as err: + return invalid_flex_config(str(err)) + + response = dict(schedule=job.id) + d, s = request_processed() + return dict(**response, **d), s diff --git a/flexmeasures/api/v3_0/sensors.py b/flexmeasures/api/v3_0/sensors.py index 7a1e69f546..14fb16166c 100644 --- a/flexmeasures/api/v3_0/sensors.py +++ b/flexmeasures/api/v3_0/sensors.py @@ -444,9 +444,9 @@ def trigger_schedule( **kwargs, ): """ - Trigger FlexMeasures to create a schedule. + Trigger FlexMeasures to create a schedule for a single flexible device, possibly taking into account inflexible devices. - .. :quickref: Schedule; Trigger scheduling job + .. :quickref: Schedule; Trigger scheduling job for one device Trigger FlexMeasures to create a schedule for this sensor. The assumption is that this sensor is the power sensor on a flexible asset. @@ -460,9 +460,8 @@ def trigger_schedule( For details on flexibility model and context, see :ref:`describing_flexibility`. Below, we'll also list some examples. - .. note:: This endpoint does not support to schedule an EMS with multiple flexible sensors at once. This will happen in another endpoint. - See https://github.com/FlexMeasures/flexmeasures/issues/485. Until then, it is possible to call this endpoint for one flexible endpoint at a time - (considering already scheduled sensors as inflexible). + .. note:: To schedule an EMS with multiple flexible sensors at once, + use `this endpoint <../api/v3_0.html#post--api-v3_0-assets-(id)-schedules-trigger>`_ instead. The length of the schedule can be set explicitly through the 'duration' field. Otherwise, it is set by the config setting :ref:`planning_horizon_config`, which defaults to 48 hours. @@ -500,8 +499,8 @@ def trigger_schedule( Storage efficiency is set to 99.99%, denoting the state of charge left after each time step equal to the sensor's resolution. Aggregate consumption (of all devices within this EMS) should be priced by sensor 9, and aggregate production should be priced by sensor 10, - where the aggregate power flow in the EMS is described by the sum over sensors 13, 14 and 15 - (plus the flexible sensor being optimized, of course). + where the aggregate power flow in the EMS is described by the sum over sensors 13, 14, 15, + and the power sensor of the flexible device being optimized (referenced in the endpoint URL). The battery consumption power capacity is limited by sensor 42 and the production capacity is constant (30 kW). @@ -628,7 +627,7 @@ def get_schedule( # noqa: C901 ): """Get a schedule from FlexMeasures. - .. :quickref: Schedule; Download schedule from the platform + .. :quickref: Schedule; Download schedule for one device **Optional fields** @@ -688,7 +687,6 @@ def get_schedule( # noqa: C901 ) return unrecognized_event(job.meta["fallback_job_id"], "fallback-job") - scheduler_info_msg = "" scheduler_info = job.meta.get("scheduler_info", dict(scheduler="")) scheduler_info_msg = f"{scheduler_info['scheduler']} was used." @@ -761,7 +759,9 @@ def get_schedule( # noqa: C901 ) sign = 1 - if sensor.get_attribute("consumption_is_positive", True): + if sensor.measures_power and sensor.get_attribute( + "consumption_is_positive", True + ): sign = -1 # For consumption schedules, positive values denote consumption. For the db, consumption is negative diff --git a/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py b/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py new file mode 100644 index 0000000000..db5f82fcdc --- /dev/null +++ b/flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py @@ -0,0 +1,289 @@ +from __future__ import annotations + +from flask import url_for +import pytest +from isodate import parse_datetime, parse_duration + +from numpy.testing import assert_almost_equal +import pandas as pd +from rq.job import Job + +from flexmeasures import Sensor +from flexmeasures.api.v3_0.tests.utils import message_for_trigger_schedule +from flexmeasures.data.models.planning.tests.utils import check_constraints +from flexmeasures.data.tests.utils import work_on_rq +from flexmeasures.data.services.scheduling import ( + handle_scheduling_exception, + get_data_source_for_job, +) +from flexmeasures.data.services.utils import sort_jobs +from flexmeasures.utils.unit_utils import ur + + +@pytest.mark.parametrize( + "message_without_targets, message_with_targets, asset_name", + [ + ( + message_for_trigger_schedule(), + message_for_trigger_schedule(with_targets=True), + "Test battery", + ), + ], +) +@pytest.mark.parametrize("sequential", [True, False]) +@pytest.mark.parametrize( + "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True +) +def test_asset_trigger_and_get_schedule( + app, + add_market_prices_fresh_db, + setup_roles_users_fresh_db, + add_charging_station_assets_fresh_db, + keep_scheduling_queue_empty, + message_without_targets, + message_with_targets, + asset_name, + sequential, + requesting_user, +): # noqa: C901 + # Include the price sensor and site-power-capacity in the flex-context explicitly, to test deserialization + price_sensor_id = add_market_prices_fresh_db["epex_da"].id + message_without_targets["flex-context"] = { + "consumption-price": {"sensor": price_sensor_id}, + "production-price": {"sensor": price_sensor_id}, + "site-power-capacity": "1 TW", # should be big enough to avoid any infeasibilities + } + + # Set up flex-model for CP 1 + CP_1_flex_model = message_without_targets["flex-model"].copy() + bidirectional_charging_station = add_charging_station_assets_fresh_db[ + "Test charging station (bidirectional)" + ] + sensor_1 = bidirectional_charging_station.sensors[0] + assert sensor_1.name == "power", "expecting to schedule a power sensor" + + # Set up flex-model for CP 2 + charging_station = add_charging_station_assets_fresh_db["Test charging station"] + CP_2_flex_model = message_with_targets["flex-model"].copy() + sensor_2 = charging_station.sensors[0] + assert sensor_2.name == "power", "expecting to schedule a power sensor" + + uni_soc_sensor = add_charging_station_assets_fresh_db["uni-soc"] + bi_soc_sensor = add_charging_station_assets_fresh_db["bi-soc"] + CP_1_flex_model["state-of-charge"] = {"sensor": bi_soc_sensor.id} + CP_2_flex_model["state-of-charge"] = {"sensor": uni_soc_sensor.id} + + # Convert the two flex-models to a single multi-asset flex-model + CP_1_flex_model["sensor"] = sensor_1.id + CP_2_flex_model["sensor"] = sensor_2.id + message = message_without_targets.copy() + message["flex-model"] = [ + CP_1_flex_model, + CP_2_flex_model, + ] + message["sequential"] = sequential + + # trigger a schedule through the /assets//schedules/trigger [POST] api endpoint + assert len(app.queues["scheduling"]) == 0 + with app.test_client() as client: + print(message) + print(message["flex-model"]) + trigger_schedule_response = client.post( + url_for( + "AssetAPI:trigger_schedule", id=sensor_1.generic_asset.parent_asset.id + ), + json=message, + ) + print("Server responded with:\n%s" % trigger_schedule_response.json) + assert trigger_schedule_response.status_code == 200 + job_id = trigger_schedule_response.json["schedule"] + + # look for scheduling jobs in queue + scheduled_jobs = app.queues["scheduling"].jobs + deferred_job_ids = app.queues["scheduling"].deferred_job_registry.get_job_ids() + deferred_jobs = sort_jobs(app.queues["scheduling"], deferred_job_ids) + + assert len(scheduled_jobs) == 1, "one scheduling job should be queued" + if sequential: + assert len(deferred_jobs) == len( + message["flex-model"] + ), "a scheduling job should be made for each sensor flex model (1 was already queued, but there is also 1 wrap-up job that should be triggered when the last scheduling job is done" + done_job_id = deferred_jobs[-1].id + else: + assert ( + len(deferred_jobs) == 0 + ), "the whole scheduling job is handled as a single job (simultaneous scheduling)" + done_job_id = scheduled_jobs[0].id + scheduling_job = scheduled_jobs[0] + + print(scheduling_job.kwargs) + if sequential: + assert ( + scheduling_job.kwargs["asset_or_sensor"]["id"] == sensor_1.id + ), "first queued job is for scheduling the first sensor" + else: + assert ( + scheduling_job.kwargs["asset_or_sensor"]["id"] + == sensor_1.generic_asset.parent_asset.id + ), "first queued job is the one for the top-level asset" + assert scheduling_job.kwargs["start"] == parse_datetime(message["start"]) + assert done_job_id == job_id + + # process the scheduling queue + work_on_rq(app.queues["scheduling"], exc_handler=handle_scheduling_exception) + assert ( + Job.fetch(job_id, connection=app.queues["scheduling"].connection).is_finished + is True + ) + + # Derive some expectations from the POSTed message + resolution = sensor_1.event_resolution + expected_length_of_schedule = parse_duration(message["duration"]) / resolution + + # check results are in the database + + # First, make sure the scheduler data source is now there + scheduling_job.refresh() # catch meta info that was added on this very instance + scheduler_source = get_data_source_for_job(scheduling_job) + assert scheduler_source is not None + + def compute_expected_length( + message: dict, sensors: list[Sensor], sequential: bool + ) -> list[int]: + """We expect a longer schedule if the targets exceeds the original duration in the trigger. + + If the planning happens sequentially, individual schedules may be extended to accommodate for far-away targets. + If the planning happens jointly, we expect all schedules to be extended. + """ + expected_durations = [pd.Timedelta(message["duration"])] * len(sensors) + for d, (sensor, flex_model) in enumerate(zip(sensors, message["flex-model"])): + assert ( + flex_model["sensor"] == sensor.id + ), "make sure we are dealing with the assumed sensor" + if "soc-targets" in flex_model: + for t in flex_model["soc-targets"]: + duration = pd.Timestamp(t["datetime"]) - pd.Timestamp( + message["start"] + ) + if duration > expected_durations[d]: + if sequential: + expected_durations[d] = duration + else: + expected_durations = [duration] * len(sensors) + + # Convert duration to number of steps in the sensor's resolution + expected_lengths = [ + expected_durations[d] / sensor.event_resolution + for d, sensor in enumerate(sensors) + ] + + return expected_lengths + + sensors = [sensor_1, sensor_2] + soc_sensors = [bi_soc_sensor, uni_soc_sensor] + expected_length_of_schedule = compute_expected_length(message, sensors, sequential) + + # The 72nd quarterhour is the first quarterhour within the cheapest hour. + # That's when we expect all charging for the uni-directional CP. + expected_uni_schedule = [0] * 188 + expected_uni_schedule[72] = 0.053651 + + # try to retrieve the schedule for each sensor through the /sensors//schedules/ [GET] api endpoint + for d, (sensor, soc_sensor, flex_model) in enumerate( + zip(sensors, soc_sensors, message["flex-model"]) + ): + + # Fetch power schedule + sensor_id = flex_model["sensor"] + get_schedule_response = client.get( + url_for("SensorAPI:get_schedule", id=sensor_id, uuid=scheduling_job.id), + query_string={"duration": "PT48H"}, + ) + print("Server responded with:\n%s" % get_schedule_response.json) + assert get_schedule_response.status_code == 200 + assert ( + get_schedule_response.json["unit"] == "MW" + ), "by default, the schedules are expected in the sensor unit" + # assert get_schedule_response.json["type"] == "GetDeviceMessageResponse" + power_schedule = get_schedule_response.json["values"] + assert len(power_schedule) == expected_length_of_schedule[d] + + check_constraints( + sensor=sensor, + schedule=pd.Series( + data=power_schedule, + index=pd.date_range( + start=get_schedule_response.json["start"], + periods=len(power_schedule), + freq=sensor.event_resolution, + ), + ), + soc_at_start=flex_model["soc-at-start"], + soc_min=flex_model["soc-min"], + soc_max=flex_model["soc-max"], + roundtrip_efficiency=ur.Quantity(flex_model["roundtrip-efficiency"]) + .to("dimensionless") + .magnitude, + storage_efficiency=ur.Quantity(flex_model["storage-efficiency"]) + .to("dimensionless") + .magnitude, + ) + + # Fetch SoC schedule + get_schedule_response = client.get( + url_for("SensorAPI:get_schedule", id=soc_sensor.id, uuid=scheduling_job.id), + query_string={"duration": "PT48H"}, + ) + print("Server responded with:\n%s" % get_schedule_response.json) + assert get_schedule_response.status_code == 200 + assert ( + get_schedule_response.json["unit"] == "MWh" + ), "by default, the schedules are expected in the sensor unit" + soc_schedule = get_schedule_response.json["values"] + assert ( + len(soc_schedule) + == expected_length_of_schedule[d] + + 1 # +1 because the SoC schedule is end-inclusive + ) + assert soc_schedule[0] * 1000 == flex_model["soc-at-start"] + + # Check for cycling and final state + if sensor_id == sensor_1.id: + # We expect cycling fully for the bi-directional Charge Point + assert any( + [s == flex_model["soc-min"] / 1000 for s in soc_schedule] + ), "we should reach soc-min at least once, because we expect at least one full cycle" + assert any( + [s == flex_model["soc-max"] / 1000 for s in soc_schedule] + ), "we should reach soc-max at least once, because we expect at least one full cycle" + assert ( + soc_schedule[-1] * 1000 == flex_model["soc-min"] + ), "we should end empty" + else: + # We expect no cycling for the uni-directional Charge Point + s = pd.Series(soc_schedule) + soc_as_percentage_of_previous_soc = 1 + s.diff() / s + assert ( + soc_as_percentage_of_previous_soc.min() + <= ur.Quantity(flex_model["storage-efficiency"]).to("").magnitude + ), "all downwards SoC should be attributable to storage losses" + assert ( + soc_schedule[-1] * 1000 == flex_model["soc-targets"][0]["value"] + ), "we should end on target" + + prices = add_market_prices_fresh_db["epex_da"].search_beliefs( + event_starts_after=message["start"], + event_ends_before=pd.Timestamp(message["start"]) + + pd.Timedelta(message["duration"]), + ) + cheapest_hour = prices.values.argmin() + if sequential and sensor_id == sensor_2.id: + assert ( + sum(power_schedule[cheapest_hour * 4 : (cheapest_hour + 1) * 4]) > 0 + ), "we expect to charge in the cheapest hour" + assert_almost_equal(power_schedule, expected_uni_schedule) + elif not sequential and sensor_id == sensor_2.id: + assert ( + sum(power_schedule[cheapest_hour * 4 : (cheapest_hour + 1) * 4]) > 0 + ), "we expect to charge in the cheapest hour" + assert_almost_equal(power_schedule, expected_uni_schedule) diff --git a/flexmeasures/api/v3_0/tests/test_assets_api.py b/flexmeasures/api/v3_0/tests/test_assets_api.py index 86aaa8ebd3..dbd190b002 100644 --- a/flexmeasures/api/v3_0/tests/test_assets_api.py +++ b/flexmeasures/api/v3_0/tests/test_assets_api.py @@ -35,35 +35,37 @@ def test_get_assets_badauth(client, setup_api_test_data, requesting_user, status assert get_assets_response.status_code == status_code +@pytest.mark.parametrize( + ("whose_asset", "exp_status"), + [ + # okay to look at assets in own account + ("test_supplier_user_4@seita.nl", 200), + # not okay to see assets owned by other accounts + ("test_prosumer_user@seita.nl", 403), + # proper 404 for non-existing asset + (None, 404), + ], +) @pytest.mark.parametrize( "requesting_user", ["test_supplier_user_4@seita.nl"], indirect=True ) -def test_get_asset_nonaccount_access(client, setup_api_test_data, requesting_user): +def test_get_asset_nonaccount_access( + client, setup_api_test_data, whose_asset, exp_status, requesting_user +): """Without being on the same account, test correct responses when accessing one asset.""" - with UserContext("test_prosumer_user@seita.nl") as prosumer1: - prosumer1_assets = prosumer1.account.generic_assets - with UserContext("test_supplier_user_4@seita.nl") as supplieruser4: - supplieruser4_assets = supplieruser4.account.generic_assets + if isinstance(whose_asset, str): + with UserContext(whose_asset) as owner: + asset_id = owner.account.generic_assets[0].id + else: + asset_id = 8171766575 # non-existent asset ID - # okay to look at assets in own account - asset_response = client.get( - url_for("AssetAPI:fetch_one", id=supplieruser4_assets[0].id), - follow_redirects=True, - ) - assert asset_response.status_code == 200 - # not okay to see assets owned by other accounts - asset_response = client.get( - url_for("AssetAPI:fetch_one", id=prosumer1_assets[0].id), - follow_redirects=True, - ) - assert asset_response.status_code == 403 - # proper 404 for non-existing asset asset_response = client.get( - url_for("AssetAPI:fetch_one", id=8171766575), + url_for("AssetAPI:fetch_one", id=asset_id), follow_redirects=True, ) - assert asset_response.status_code == 404 - assert "not found" in asset_response.json["message"] + assert asset_response.status_code == exp_status + if exp_status == 404: + assert asset_response.json["message"] == "No asset found with ID 8171766575." @pytest.mark.parametrize( diff --git a/flexmeasures/cli/data_add.py b/flexmeasures/cli/data_add.py index dced19df6a..e5865bdd4c 100755 --- a/flexmeasures/cli/data_add.py +++ b/flexmeasures/cli/data_add.py @@ -2178,18 +2178,21 @@ def create_asset_with_one_sensor( "toy-process", "process", "Power (Inflexible)", + flex_context={"consumption-price": {"sensor": day_ahead_sensor.id}}, ) breakable_power = create_asset_with_one_sensor( "toy-process", "process", "Power (Breakable)", + flex_context={"consumption-price": {"sensor": day_ahead_sensor.id}}, ) shiftable_power = create_asset_with_one_sensor( "toy-process", "process", "Power (Shiftable)", + flex_context={"consumption-price": {"sensor": day_ahead_sensor.id}}, ) db.session.flush() diff --git a/flexmeasures/cli/data_show.py b/flexmeasures/cli/data_show.py index fb61435192..e93891f32b 100644 --- a/flexmeasures/cli/data_show.py +++ b/flexmeasures/cli/data_show.py @@ -280,7 +280,7 @@ def show_generic_asset(asset): click.echo(f"Child assets of {asset.name} (ID: {asset.id})") click.echo(f"======{len(asset.name) * '='}===================\n") if child_asset_data: - click.echo(tabulate(child_asset_data, headers=["Id", "Name", "Type"])) + click.echo(tabulate(child_asset_data, headers=["ID", "Name", "Type"])) else: click.secho("No child assets ...", **MsgStyle.WARN) diff --git a/flexmeasures/conftest.py b/flexmeasures/conftest.py index b444ac100b..2ffb5e356b 100644 --- a/flexmeasures/conftest.py +++ b/flexmeasures/conftest.py @@ -1000,11 +1000,21 @@ def create_charging_station_assets( """Add uni- and bi-directional charging station assets, set their capacity value and their initial SOC.""" oneway_evse = GenericAssetType(name="one-way_evse") twoway_evse = GenericAssetType(name="two-way_evse") + charging_hub = GenericAssetType(name="charging_hub") + + charging_hub = GenericAsset( + name="Test charging hub", + owner=setup_accounts["Prosumer"], + generic_asset_type=charging_hub, + latitude=10, + longitude=100, + ) charging_station = GenericAsset( name="Test charging station", owner=setup_accounts["Prosumer"], generic_asset_type=oneway_evse, + parent_asset=charging_hub, latitude=10, longitude=100, flex_context={ @@ -1040,6 +1050,7 @@ def create_charging_station_assets( name="Test charging station (bidirectional)", owner=setup_accounts["Prosumer"], generic_asset_type=twoway_evse, + parent_asset=charging_hub, latitude=10, longitude=100, flex_context={ @@ -1070,9 +1081,25 @@ def create_charging_station_assets( ), ) db.session.add(bidirectional_charging_station_power_sensor) + bi_soc = Sensor( + name="bi-soc", + generic_asset=bidirectional_charging_station, + unit="MWh", + event_resolution=timedelta(minutes=0), + ) + uni_soc = Sensor( + name="uni-soc", + generic_asset=charging_station, + unit="MWh", + event_resolution=timedelta(minutes=0), + ) + db.session.add(bi_soc) + db.session.add(uni_soc) return { "Test charging station": charging_station, "Test charging station (bidirectional)": bidirectional_charging_station, + "bi-soc": bi_soc, + "uni-soc": uni_soc, } diff --git a/flexmeasures/data/models/planning/storage.py b/flexmeasures/data/models/planning/storage.py index 499e1594e5..cc19833db7 100644 --- a/flexmeasures/data/models/planning/storage.py +++ b/flexmeasures/data/models/planning/storage.py @@ -686,6 +686,9 @@ def _prepare(self, skip_validation: bool = False) -> tuple: # noqa: C901 soc_max[d], soc_min[d], ) + else: + # No need to validate non-existing storage constraints + skip_validation = True power_capacity_in_mw[d] = get_continuous_series_sensor_or_quantity( variable_quantity=power_capacity_in_mw[d], @@ -1049,14 +1052,18 @@ def deserialize_flex_config(self): ) for d, sensor_flex_model in enumerate(self.flex_model): self.flex_model[d] = StorageFlexModelSchema( - start=self.start, sensor=sensor_flex_model["sensor"] + start=self.start, + sensor=sensor_flex_model["sensor"], + default_soc_unit=sensor_flex_model["sensor_flex_model"].get( + "soc-unit" + ), ).load(sensor_flex_model["sensor_flex_model"]) self.flex_model[d]["sensor"] = sensor_flex_model["sensor"] # Extend schedule period in case a target exceeds its end self.possibly_extend_end( - soc_targets=sensor_flex_model.get("soc_targets"), - sensor=sensor_flex_model["sensor"], + soc_targets=self.flex_model[d].get("soc_targets"), + sensor=self.flex_model[d]["sensor"], ) else: @@ -1086,51 +1093,42 @@ def possibly_extend_end(self, soc_targets, sensor: Sensor = None): else: self.end = max_target_datetime - def get_min_max_targets( - self, deserialized_names: bool = True - ) -> tuple[float | None, float | None]: + def get_min_max_targets(self) -> tuple[float | None, float | None]: + """This happens before deserializing the flex-model.""" min_target = None max_target = None - soc_targets_label = "soc_targets" if deserialized_names else "soc-targets" # if the SOC targets are defined as a Sensor, we don't get min max values - if isinstance(self.flex_model.get(soc_targets_label), dict): + if isinstance(self.flex_model.get("soc-targets"), dict): return None, None - if ( - soc_targets_label in self.flex_model - and len(self.flex_model[soc_targets_label]) > 0 - ): + if "soc-targets" in self.flex_model and len(self.flex_model["soc-targets"]) > 0: min_target = min( - [target["value"] for target in self.flex_model[soc_targets_label]] + [target["value"] for target in self.flex_model["soc-targets"]] ) max_target = max( - [target["value"] for target in self.flex_model[soc_targets_label]] + [target["value"] for target in self.flex_model["soc-targets"]] ) return min_target, max_target - def get_min_max_soc_on_sensor( - self, adjust_unit: bool = False, deserialized_names: bool = True - ) -> tuple[float | None, float | None]: + def get_min_max_soc_on_sensor(self) -> tuple[float | None, float | None]: + """This happens before deserializing the flex-model.""" soc_min_sensor: float | None = self.sensor.get_attribute("min_soc_in_mwh") soc_max_sensor: float | None = self.sensor.get_attribute("max_soc_in_mwh") - soc_unit_label = "soc_unit" if deserialized_names else "soc-unit" - if adjust_unit: - if soc_min_sensor and self.flex_model.get(soc_unit_label) == "kWh": - soc_min_sensor *= 1000 # later steps assume soc data is kWh - if soc_max_sensor and self.flex_model.get(soc_unit_label) == "kWh": - soc_max_sensor *= 1000 + if soc_min_sensor and self.flex_model.get("soc-unit") == "kWh": + soc_min_sensor *= 1000 # later steps assume soc data is kWh + if soc_max_sensor and self.flex_model.get("soc-unit") == "kWh": + soc_max_sensor *= 1000 return soc_min_sensor, soc_max_sensor def ensure_soc_min_max(self): """ Make sure we have min and max SOC. If not passed directly, then get default from sensor or targets. + This happens before deserializing the flex-model. """ - _, max_target = self.get_min_max_targets(deserialized_names=False) - soc_min_sensor, soc_max_sensor = self.get_min_max_soc_on_sensor( - adjust_unit=True, deserialized_names=False - ) + _, max_target = self.get_min_max_targets() + soc_min_sensor, soc_max_sensor = self.get_min_max_soc_on_sensor() if "soc-min" not in self.flex_model or self.flex_model["soc-min"] is None: # Default is 0 - can't drain the storage by more than it contains self.flex_model["soc-min"] = soc_min_sensor if soc_min_sensor else 0 diff --git a/flexmeasures/data/models/planning/tests/utils.py b/flexmeasures/data/models/planning/tests/utils.py index b72e5dd590..323b722c8e 100644 --- a/flexmeasures/data/models/planning/tests/utils.py +++ b/flexmeasures/data/models/planning/tests/utils.py @@ -27,6 +27,8 @@ def check_constraints( roundtrip_efficiency: float = 1, storage_efficiency: float = 1, tolerance: float = 0.00001, + soc_min: float | None = None, + soc_max: float | None = None, ) -> pd.Series: soc_schedule = integrate_time_series( schedule, @@ -45,8 +47,12 @@ def check_constraints( assert min(schedule.values) >= capacity * -1 - tolerance assert max(schedule.values) <= capacity + tolerance for soc in soc_schedule.values: - assert soc >= sensor.get_attribute("min_soc_in_mwh") - assert soc <= sensor.get_attribute("max_soc_in_mwh") + assert soc >= ( + soc_min if soc_min is not None else sensor.get_attribute("min_soc_in_mwh") + ) + assert soc <= ( + soc_max if soc_max is not None else sensor.get_attribute("max_soc_in_mwh") + ) return soc_schedule diff --git a/flexmeasures/data/models/planning/utils.py b/flexmeasures/data/models/planning/utils.py index ee893f3cce..82e5046613 100644 --- a/flexmeasures/data/models/planning/utils.py +++ b/flexmeasures/data/models/planning/utils.py @@ -69,7 +69,7 @@ def initialize_index( def add_tiny_price_slope( - orig_prices: pd.DataFrame, col_name: str = "event_value", d: float = 10**-3 + orig_prices: pd.DataFrame, col_name: str = "event_value", d: float = 10**-4 ) -> pd.DataFrame: """Add tiny price slope to col_name to represent e.g. inflation as a simple linear price increase. This is meant to break ties, when multiple time slots have equal prices, in favour of acting sooner. diff --git a/flexmeasures/data/schemas/generic_assets.py b/flexmeasures/data/schemas/generic_assets.py index a03a38f311..f2fcc15d1d 100644 --- a/flexmeasures/data/schemas/generic_assets.py +++ b/flexmeasures/data/schemas/generic_assets.py @@ -1,7 +1,9 @@ from __future__ import annotations import json +from http import HTTPStatus +from flask import abort from marshmallow import validates, ValidationError, fields, validates_schema from flask_security import current_user from sqlalchemy import select @@ -14,7 +16,6 @@ from flexmeasures.data.schemas.utils import ( FMValidationError, MarshmallowClickMixin, - with_appcontext_if_needed, ) from flexmeasures.auth.policy import user_has_admin_access from flexmeasures.cli import is_running as running_as_cli @@ -267,16 +268,24 @@ class Meta: class GenericAssetIdField(MarshmallowClickMixin, fields.Int): """Field that deserializes to a GenericAsset and serializes back to an integer.""" - @with_appcontext_if_needed() - def _deserialize(self, value, attr, obj, **kwargs) -> GenericAsset: + def __init__(self, status_if_not_found: HTTPStatus | None = None, *args, **kwargs): + self.status_if_not_found = status_if_not_found + super().__init__(*args, **kwargs) + + def _deserialize(self, value: int | str, attr, obj, **kwargs) -> GenericAsset: """Turn a generic asset id into a GenericAsset.""" - generic_asset = db.session.get(GenericAsset, value) + generic_asset: GenericAsset = db.session.execute( + select(GenericAsset).filter_by(id=int(value)) + ).scalar_one_or_none() if generic_asset is None: - raise FMValidationError(f"No asset found with id {value}.") - # lazy loading now (asset is somehow not in session after this) - generic_asset.generic_asset_type + message = f"No asset found with ID {value}." + if self.status_if_not_found == HTTPStatus.NOT_FOUND: + raise abort(404, message) + else: + raise FMValidationError(message) + return generic_asset - def _serialize(self, asset, attr, data, **kwargs): + def _serialize(self, asset: GenericAsset, attr, data, **kwargs) -> int: """Turn a GenericAsset into a generic asset id.""" return asset.id diff --git a/flexmeasures/data/schemas/scheduling/__init__.py b/flexmeasures/data/schemas/scheduling/__init__.py index 7458e5c77d..3bc9eb6538 100644 --- a/flexmeasures/data/schemas/scheduling/__init__.py +++ b/flexmeasures/data/schemas/scheduling/__init__.py @@ -486,6 +486,7 @@ class AssetTriggerSchema(Schema): data_key="flex-model", ) flex_context = fields.Dict(required=False, data_key="flex-context") + sequential = fields.Bool(load_default=False) @validates_schema def check_flex_model_sensors(self, data, **kwargs): diff --git a/flexmeasures/data/schemas/sensors.py b/flexmeasures/data/schemas/sensors.py index 3b63e667ca..c746d213b3 100644 --- a/flexmeasures/data/schemas/sensors.py +++ b/flexmeasures/data/schemas/sensors.py @@ -208,9 +208,17 @@ class Meta: class SensorIdField(MarshmallowClickMixin, fields.Int): """Field that deserializes to a Sensor and serializes back to an integer.""" - def __init__(self, *args, unit: str | ur.Quantity | None = None, **kwargs): + def __init__( + self, + asset: GenericAsset | None = None, + unit: str | ur.Quantity | None = None, + *args, + **kwargs, + ): super().__init__(*args, **kwargs) + self.asset = asset + if isinstance(unit, str): self.to_unit = ur.Quantity(unit) elif isinstance(unit, ur.Quantity): @@ -236,6 +244,16 @@ def _deserialize(self, value: int, attr, obj, **kwargs) -> Sensor: sensor.generic_asset sensor.generic_asset.generic_asset_type + # if the asset is defined, check if the sensor belongs to it (or to its offspring) + if ( + self.asset is not None + and sensor.generic_asset != self.asset + and sensor.generic_asset not in self.asset.offspring + ): + raise FMValidationError( + f"Sensor {value} must be assigned to asset {self.asset} (or to one of its offspring)" + ) + # if the units are defined, check if the sensor data is convertible to the target units if self.to_unit is not None and not units_are_convertible( sensor.unit, str(self.to_unit.units) diff --git a/flexmeasures/data/services/scheduling.py b/flexmeasures/data/services/scheduling.py index fe607ed4c8..7c85756150 100644 --- a/flexmeasures/data/services/scheduling.py +++ b/flexmeasures/data/services/scheduling.py @@ -123,6 +123,17 @@ def trigger_optional_fallback(job, connection, type, value, traceback): scheduler_kwargs = job.meta["scheduler_kwargs"] + # Deserialize start and end + timezone = "UTC" + if hasattr(asset_or_sensor, "timezone"): + timezone = asset_or_sensor.timezone + scheduler_kwargs["start"] = pd.Timestamp(scheduler_kwargs["start"]).tz_convert( + timezone + ) + scheduler_kwargs["end"] = pd.Timestamp(scheduler_kwargs["end"]).tz_convert( + timezone + ) + if ("scheduler_specs" in job.kwargs) and ( job.kwargs["scheduler_specs"] is not None ): @@ -201,7 +212,7 @@ def create_scheduling_job( """ # We first create a scheduler and check if deserializing works, so the flex config is checked # and errors are raised before the job is enqueued (so users get a meaningful response right away). - # Note: We are putting still serialized scheduler_kwargs into the job! + # Note: We should put only serializable scheduler_kwargs into the job! if sensor is not None: current_app.logger.warning( @@ -248,6 +259,15 @@ def create_scheduling_job( job.meta["asset_or_sensor"] = asset_or_sensor job.meta["scheduler_kwargs"] = scheduler_kwargs + + # Serialize start and end + job.meta["scheduler_kwargs"]["start"] = job.meta["scheduler_kwargs"][ + "start" + ].isoformat() + job.meta["scheduler_kwargs"]["end"] = job.meta["scheduler_kwargs"][ + "end" + ].isoformat() + job.save_meta() # in case the function enqueues it @@ -339,7 +359,7 @@ def create_sequential_scheduling_job( previous_sensors ) current_scheduler_kwargs["resolution"] = sensor.event_resolution - current_scheduler_kwargs["sensor"] = sensor + current_scheduler_kwargs["asset_or_sensor"] = sensor job = create_scheduling_job( **current_scheduler_kwargs, @@ -625,19 +645,10 @@ def find_scheduler_class(asset_or_sensor: Asset | Sensor) -> type: else: asset = asset_or_sensor - if asset.generic_asset_type.name in ( - "battery", - "one-way_evse", - "two-way_evse", - ): - scheduler_class = StorageScheduler - elif asset.generic_asset_type.name in ("process", "load"): + if asset.generic_asset_type.name in ("process", "load"): scheduler_class = ProcessScheduler else: - raise ValueError( - "Scheduling is not (yet) supported for asset type %s." - % asset.generic_asset_type - ) + scheduler_class = StorageScheduler return scheduler_class diff --git a/flexmeasures/data/services/utils.py b/flexmeasures/data/services/utils.py index 2f8efbfffe..689190f370 100644 --- a/flexmeasures/data/services/utils.py +++ b/flexmeasures/data/services/utils.py @@ -10,6 +10,7 @@ import click from sqlalchemy import JSON, String, cast, literal from flask import current_app +from rq import Queue from rq.job import Job from sqlalchemy import select @@ -38,7 +39,11 @@ def get_scheduler_instance( def get_asset_or_sensor_ref(asset_or_sensor: Asset | Sensor) -> dict: - return {"id": asset_or_sensor.id, "class": asset_or_sensor.__class__.__name__} + class_name = asset_or_sensor.__class__.__name__ + # todo: remove these two lines after renaming the GenericAsset class to Asset + if class_name == "GenericAsset": + class_name = "Asset" + return {"id": asset_or_sensor.id, "class": class_name} def get_asset_or_sensor_from_ref(asset_or_sensor: dict): @@ -60,13 +65,13 @@ def get_asset_or_sensor_from_ref(asset_or_sensor: dict): Sensor(id=2) """ - if asset_or_sensor["class"] == Asset.__name__: + if asset_or_sensor["class"] in (Asset.__name__, "Asset"): klass = Asset elif asset_or_sensor["class"] == Sensor.__name__: klass = Sensor else: raise ValueError( - f"Unrecognized class `{asset_or_sensor['class']}`. Please, consider using GenericAsset or Sensor." + f"Unrecognized class `{asset_or_sensor['class']}`. Please, consider using Asset or Sensor." ) return db.session.get(klass, asset_or_sensor["id"]) @@ -244,3 +249,13 @@ def wrapper(*args, **kwargs): return wrapper return decorator + + +def sort_jobs(queue: Queue, jobs: list[str | Job]) -> list[Job]: + """Sort jobs in chronological order of creation, and return Job objects.""" + jobs = [queue.fetch_job(job) if isinstance(job, str) else job for job in jobs] + jobs = [ + job for job in jobs if job is not None + ] # Remove any None entries (in case some jobs don’t exist) + sorted_jobs = sorted(jobs, key=lambda job: job.created_at) + return sorted_jobs diff --git a/flexmeasures/data/tests/test_scheduling_simultaneous.py b/flexmeasures/data/tests/test_scheduling_simultaneous.py index 95f39cf145..5e31e598f1 100644 --- a/flexmeasures/data/tests/test_scheduling_simultaneous.py +++ b/flexmeasures/data/tests/test_scheduling_simultaneous.py @@ -38,7 +38,7 @@ def test_create_simultaneous_jobs( # The EV is scheduled firstly. assert job.kwargs["asset_or_sensor"] == { "id": assets["Test Site"].id, - "class": "GenericAsset", + "class": "Asset", } # It uses the inflexible-device-sensors that are defined in the flex-context, exclusively. assert job.kwargs["flex_context"]["inflexible-device-sensors"] == [