Add playground experiments

.github/workflows/test.yml

@@ -52,5 +52,5 @@ jobs:
         env:
           DATABASE__HOST: localhost
           DATABASE__PORT: 5432
-          EXECUTOR__HOST: localhost
-          EXECUTOR__PORT: 6379
+          ENGINE__HOST: localhost
+          ENGINE__PORT: 6379

.vscode/settings.json

@@ -14,6 +14,9 @@
         "Flou",
         "websockets"
     ],
+    "cSpell.enabledLanguageIds": [
+        "markdown",
+    ],
     "python.testing.pytestArgs": [
     ],
     "python.testing.cwd": "${workspaceFolder}/tests",

compose.yml

@@ -4,7 +4,7 @@ volumes:
 
 services:
   engine:
-    image: "${REGISTRY-}flouai/flou:latest"
+    image: "${REGISTRY-}flouai/flou:${REGISTRY_TAG-latest}"
     depends_on:
       - db
       - cache
@@ -18,12 +18,12 @@ services:
       - path: .env
         required: false
     healthcheck:
-      test: celery -A flou.executor.celery.app status || exit 1
+      test: celery -A flou.engine.celery.app status || exit 1
       interval: 1s
       timeout: 10s
       retries: 10
   api:
-    image: "${REGISTRY-}flouai/flou:latest"
+    image: "${REGISTRY-}flouai/flou:${REGISTRY_TAG-latest}"
     environment:
       - PORT=8000
     ports:
@@ -44,7 +44,7 @@ services:
       - path: .env
         required: false
   studio:
-    image: "${REGISTRY-}flouai/studio:latest"
+    image: "${REGISTRY-}flouai/studio:${REGISTRY_TAG-latest}"
     # env_file: studio/.env.docker
     environment:
       - PORT=8001
@@ -85,7 +85,7 @@ services:
       - ALLOW_EMPTY_PASSWORD=yes
 
   docs:
-    image: "${REGISTRY-}flouai/docs:latest"
+    image: "${REGISTRY-}flouai/docs:${REGISTRY_TAG-latest}"
     restart: on-failure
     ports:
       - "8002:8002"

docs/docs/documentation/changelog.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## Flou v0.2 (in development) "Experiments"
+
+This focuses on the bases for Flou's Experiments.
+
+- Changed `state` to `store` when referring to the data the states hold (2024-03-21)
+- Added pydantic schemas to the transitions *experimental* (2024-03-20)
+- Added store ui widgets *experimental* (2024-03-20)
+- Refactor `Executor` to `Engine`. You may need to change your `.env` values.
+
 ## Flou v0.1
 
 Initial public release.

docs/docs/documentation/contributing/release-procedure.md

@@ -21,15 +21,26 @@ Flou follows [Semantic Versioning](https://semver.org/) (SemVer) versioning.
 
 You can find the current version in the `pyproject.toml` file.
 
-Flou will be released in `stable` and `dev`. Every release with a `-dev.X`
+Flou will be released in `stable` and `dev`. Every release with a `.devX`
 suffix means it's in active development and should not be used except to try
 experimental features. Before releasing a `stable` release, remove the `dev`
 suffix, publish the changes and then advance the minor or patch version and add
 the `dev` suffix again.
 
-## Updating the docker images
+## Releasing a new version
 
-        % export FLOU_VERSION=0.1.0-dev.1
+You can release a new version by running:
+
+        % flou release --push --pypi
+
+This will take care of updating the version, building the docker images, pushing
+them to the registry and publishing the python package to PyPI.
+
+## Manually releasing a new version
+
+### Updating the docker images
+
+        % export FLOU_VERSION=0.1.0.dev1
         % docker compose -f compose.yml -f compose.dev.yml build
 
         % docker tag flou-engine:latest flouai/flou:latest
@@ -46,7 +57,7 @@ the `dev` suffix again.
         % docker push flouai/studio:$FLOU_VERSION
         % docker push flouai/docs:$FLOU_VERSION
 
-## Updating the python package in PyPI
+### Updating the python package in PyPI
 
 Make sure you have your PyPI credentials set in `~/.pypirc`.
 

docs/docs/documentation/engine/api-cli.md

@@ -138,7 +138,7 @@ You will need the instance ID and call:
     ```json title="Sample output"
     {
         "name": "sample_ltm",
-        "state": {
+        "store": {
             "_status": "active",
             ...
         },
@@ -172,7 +172,7 @@ You will need the instance ID and call:
 
     ```bash title="Sample output"
     ┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
-    ┃ name       ┃ fqn                ┃ state   ┃ params ┃ structur┃ concurrent_instances ┃ created_at          ┃ uploaded_at         ┃
+    ┃ name       ┃ fqn                ┃ store   ┃ params ┃ structur┃ concurrent_instances ┃ created_at          ┃ uploaded_at         ┃
     ┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
     │ sample_ltm │ flou.app.SampleLTM │ {       │ None   │ {       │ {                    │ 2024-09-01 21:32:43 │ 2024-09-01 21:32:43 │
     │            │                    │     ... │        │     ... │     ...              │                     │                     │

docs/docs/documentation/engine/dependencies.md

@@ -56,13 +56,13 @@ VSCode extension
 1. Select the docker container that ends in `...-engine-1`
 1. A new VSCode window will appear using the engine Python environment -->
 
-### Executor settings
+### Engine settings
 
 #### Max retries
 
 - **Type**: Int
 - **Default**: 1
 - **Description**: You can set the maximum amount of retries for a certain
-execution by setting `EXECUTOR__MAX_RETRIES`. Each retry will be done with a
+execution by setting `ENGINE__MAX_RETRIES`. Each retry will be done with a
 jittery exponential backoff. For more information see [Celery's
 documentation](https://docs.celeryq.dev/en/stable/userguide/tasks.html#Task.retry_backoff).

docs/docs/documentation/network/concurrency.md

@@ -99,7 +99,7 @@ launched with names: `processing_file_1111` and `processing_file_2222`.
 * You can have as many parameters as needed in a parameterized transition.
 * Use `self.params` to get the params of the current executing LTM.
 * You can concurrently launch either States or sub State Machines.
-* Each parameterized LTM has it's own local store accessible via `self.state`.
+* Each parameterized LTM has it's own local store accessible via `self.store`.
 
 ## Joining concurrent flows
 
@@ -110,14 +110,14 @@ store.
 ### Concurrently Updating the Store
 
 When using concurrency you need to be very careful about updates to the store.
-Internally `update_state` updates the local memory store immediately but waits
+Internally `update_store` updates the local memory store immediately but waits
 until the State execution finishes to update the database store atomically with
 just one call. This makes it difficult to work with concurrent stores when you
 have several LTMs updating it at the same time.
 
-For this use case you can use `LTM.atomic_state_append(key, value)` that
+For this use case you can use `LTM.atomic_store_append(key, value)` that
 atomically and immediately appends `value` to a pre initialized list `key` in
-`self.state` and returns the updated list. This can be used when joining
+`self.store` and returns the updated list. This can be used when joining
 concurrent forks.
 
 ### Joining forked workflows
@@ -143,7 +143,7 @@ class ConcurrentJoinMachine(LTM):
         { 'label': 'done', 'from': JoinTasks, 'to': Finished},
     ]
 
-    def get_initial_state(self):
+    def get_initial_store(self):
         return {'executed_tasks': []}
 ```
 
@@ -167,7 +167,7 @@ class TaskA(LTM):
 ```
 
 Because `JoinTasks` will be called 3 times and possible at the same time we need
-to use `atomic_state_append` that will add an item to `executed_tasks` and
+to use `atomic_store_append` that will add an item to `executed_tasks` and
 return the new value atomically. This way we can guarantee that in only 1 of the
 3 executions `executed_tasks` will have three items, hence only transitioning
 `done` once.
@@ -180,7 +180,7 @@ class JoinTasks(LTM):
     name = 'join_tasks'
 
     def run(self):
-        executed_tasks = self.parent.atomic_state_append('executed_tasks', payload)
+        executed_tasks = self.parent.atomic_store_append('executed_tasks', payload)
         if set(executed_tasks) == set(('A', 'B', 'C')):
             self.transition('done')
 ```
@@ -204,7 +204,7 @@ class LaunchFilesProcessing(LTM):
     def run(self, payload):
         uploaded_file_ids = payload['uploaded_file_ids']
 
-        self.parent.update_state('launched_params', uploaded_file_ids)
+        self.parent.update_store('launched_params', uploaded_file_ids)
 
         self.transition(
             "start_{file_id}",
@@ -219,7 +219,7 @@ class JoinProcessing(LTM):
     name = 'join_processing'
 
     def run(self):
-        processed_files = self.parent.atomic_state_append('processed_files', payload)
-        if set(processed_files) == set(self.parent.state['launched_params']):
+        processed_files = self.parent.atomic_store_append('processed_files', payload)
+        if set(processed_files) == set(self.parent.store['launched_params']):
             self.transition('done')
 ```

docs/docs/documentation/network/nesting.md

@@ -78,8 +78,8 @@ store](#using-nested-stores).
 You can access the `root` or `parent` stores to share information between State
 Machines.
 
-* `self.parent.state` & `self.parent.update_state(...)`
-* `self.root.state` & `self.root.update_state(...)`
+* `self.parent.store` & `self.parent.update_store(...)`
+* `self.root.store` & `self.root.update_store(...)`
 
 ## Transition Namespaces
 

docs/docs/documentation/network/states-machines.md

@@ -186,6 +186,47 @@ chatbot.transition(`message_received`, payload={'message': "A message"})
 Payloads are great for data that is strictly related to a single transition
 since they aren't stored in a permanent fashion.
 
+### Transition Payload Schemas (Experimental)
+
+!!! experimental "This feature is experimental and subject to change"
+
+You can define JSON schemas for transition payloads using Pydantic models. This allows for:
+
+1. Validation of payload data
+2. Automatic form generation in the Studio UI
+3. Type hints and documentation
+
+To define payload schemas, add a `transition_payloads` dictionary to your LTM class that maps
+transition labels to Pydantic models:
+
+```python
+from pydantic import BaseModel, Field
+
+class WritingInstructions(BaseModel):
+    writing_instructions: str = Field(..., description="Instructions for writing")
+
+class MyStateMachine(LTM):
+    name = "my_state_machine"
+    init = [WaitingState]
+
+    transitions = [
+        {
+            "from": WaitingState,
+            "label": "update_instructions",
+            "to": ProcessingState
+        },
+        # Other transitions...
+    ]
+
+    # Define payload schemas
+    transition_payloads = {
+        "update_instructions": WritingInstructions
+    }
+```
+
+When these transitions are displayed in the Studio UI, a form will be generated based on the 
+Pydantic model's fields, with appropriate input types and validation.
+
 ## Store Management
 
 Flou provides a **data store** out-of-the-box. Each State Machine has a store
@@ -195,68 +236,66 @@ that any State can access and each State has it's own local store.
 class MyState(LTM):
     name = 'my_state'
 
-    def get_initial_state(self):
+    def get_initial_store(self):
         return {
             "key1": "example",
             "key2": "other",
         }
 
     def run(self, payload=None):
-        self.state  # local State store
+        self.store  # local State store
         # returns {
         #     "key1": "example",
         #     "key2": "other",
         # }
 
-        self.root.state
+        self.root.store
         # returns the State Machine global store
 ```
 
-!!! warning "`state` will be renamed `store` in upcoming releases"
-
-Every `LTM` has a function `get_initial_state` that is used during the State
+Every `LTM` has a function `get_initial_store` that is used during the State
 initialization to create it's local store. It must return a dictionary. Keys
 cannot start with `_` (underscore) as they are reserved for Flou internal
 properties.
 
-Using `self.state` you can retrieve the store of the current State.
+Using `self.store` you can retrieve the store of the current State.
 
-Using `self.root.state` you can access the global store of the State Machine.
+Using `self.root.store` you can access the global store of the State Machine.
 
 ### Modifying the store
 
-!!! warning "Don't attempt to directly modify `.state`, use `.update_state()` instead"
+!!! warning "Don't attempt to directly modify `.store`, use `.update_store()` instead"
     As Flou is prepared for concurrency in order to update the store you need to
-    use the special function `update_state`.
+    use the special function `update_store`.
 
-To update the store use the special function `update_state` which expects a
+To update the store use the special function `update_store` which expects a
 dictionary. This creates or overwrites the desired keys.
 
 To update a nested key in the store you need to use the Qualified Name of the
 path to access that key separated by `.` (dots). For example:
 
 ``` python
 def run(self, payload=None):
-    self.state
+    self.store
     # returns {}
 
-    self.state['key'] = {'a': 1, 'b': 1}
-    # **INVALID**, use `update_state`
+    self.store['key'] = {'a': 1, 'b': 1}
+    # **INVALID**, use `update_store`
 
-    self.update_state({'key': {'a': 1, 'b': 2}})
-    # updates the state
+    self.update_store({'key': {'a': 1, 'b': 2}})
+    # updates the store
 
-    self.state
+    self.store
     # returns {'key': {'a': 1, 'b': 1}}
 
-    self.update_state({'key': {'a': 3})
+    self.update_store({'key': {'a': 3})
     # replaces `key` with `{'a': 1}`
 
     # use instead
-    self.update_state({'key.a': {'a': 3})  # replaces ['key']['a'] with {'a': 1}
+    self.update_store({'key.a': {'a': 3})  # replaces ['key']['a'] with {'a': 1}
 ```
 
 !!! abstract "Changes to the store are not immediately committed to the database"
-    When calling `update_state` the in memory store get's updated but the
+    When calling `update_store` the in memory store get's updated but the
     database isn't updated until the State code is executed successfully. This
     keeps the State execution in an atomic transaction.

docs/docs/documentation/quicksheet.md

@@ -25,11 +25,11 @@ class MyState(LTM):
         params = {}
         self.transition(label, **params)
 
-        self.update_state({'internal_state_key': 'state_value'})
-        self.root.update_state({'global_key': 'global_value'})
+        self.update_store({'internal_store_key': 'store_value'})
+        self.root.update_store({'global_key': 'global_value'})
 
-        print(self.state['internal_state_key'])  # prints 'state_value'
-        print(self.root.state['global_key'])  # prints 'global_value'
+        print(self.store['internal_store_key'])  # prints 'store_value'
+        print(self.root.store['global_key'])  # prints 'global_value'
 
 class EndState(LTM):
     def run(self, **params):
@@ -53,7 +53,7 @@ registry.register(MyNetwork)
 * `LTM.start()`: start the root LTM, should be called only once
 * `LTM.root`: returns the `root` LTM
 * `LTM.parent`: returns the `parent` LTM
-* `LTM.state`: returns a dict with the current LTM state, you can use `self.root.state` for global state. Don't assign directly, see `LTM.update_state`
-* `LTM.update_state(update_list)`: pass a list of `{key: value}` to update the state. Use `self.root.update_state` to update the global state
-* `LTM.atomic_state_append(key, value)`: atomically and immediately append `value` to a pre initialized list `key` in `self.state`. Use only in concurrent states.
+* `LTM.store`: returns a dict with the current LTM store, you can use `self.root.store` for global store. Don't assign directly, see `LTM.update_store`
+* `LTM.update_store(update_list)`: pass a list of `{key: value}` to update the store. Use `self.root.update_store` to update the global store
+* `LTM.atomic_store_append(key, value)`: atomically and immediately append `value` to a pre initialized list `key` in `self.store`. Use only in concurrent states.
 * `LTM.transition(label, payload, params, namespace)`: transition all LTMS with `label` transition