-
Notifications
You must be signed in to change notification settings - Fork 41
Final changes for trigger beta 0.0.3b1 #435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
NiveditJain
merged 10 commits into
exospherehost:main
from
NiveditJain:triggers-finalization
Sep 29, 2025
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
cd257ea
Enhance documentation and add triggers functionality
NiveditJain 706aa0c
Update README.md to clarify triggers feature versioning
NiveditJain 4f31163
adding beta status
NiveditJain ca7ac50
Update docs/docs/exosphere/graph-components.md
NiveditJain 0d0700e
Refactor trigger handling and improve error management in graph templ…
NiveditJain edbbfcf
improving triggers
NiveditJain 26ed57a
Update docs/docs/exosphere/triggers.md
NiveditJain f6994e1
Refactor trigger handling and improve error management in tests
NiveditJain 6539073
Merge branch 'triggers-finalization' of https://github.com/NiveditJai…
NiveditJain 2d77783
Refactor import statements in verify_graph.py
NiveditJain File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,211 @@ | ||
| # Triggers | ||
|
|
||
| !!! warning "Beta Feature" | ||
| Triggers functionality is currently in beta and available under the `beta-latest` Docker tag and SDK version `0.0.3b1`. The API may change in future versions. | ||
NiveditJain marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Triggers allow you to schedule automatic execution of your graphs using cron expressions. When a trigger is defined, Exosphere will automatically execute your graph at the specified times without requiring manual intervention. | ||
|
|
||
| ## Overview | ||
|
|
||
| Triggers provide **scheduled execution** for your workflows, enabling automation for: | ||
|
|
||
| - **Regular data processing** (daily reports, hourly syncs) | ||
| - **Maintenance tasks** (cleanup jobs, backups) | ||
| - **Monitoring workflows** (health checks, alerts) | ||
| - **Business processes** (invoice generation, notifications) | ||
|
|
||
| ```mermaid | ||
| graph TB | ||
| A[Cron Trigger] --> B[Scheduled Time Reached] | ||
| B --> C[Graph Execution Triggered] | ||
| C --> D[Workflow Runs Automatically] | ||
| E[Multiple Triggers] -.->|Different schedules| A | ||
| style A fill:#e8f5e8 | ||
| style E fill:#e8f5e8 | ||
| ``` | ||
|
|
||
| ## How Triggers Work | ||
|
|
||
| ### Trigger Lifecycle | ||
|
|
||
| 1. **Definition**: Triggers are defined when creating/updating a graph template | ||
| 2. **Scheduling**: Exosphere schedules the next execution based on cron expression | ||
| 3. **Execution**: At the scheduled time, the graph is triggered automatically | ||
| 4. **Rescheduling**: After execution, the next occurrence is automatically scheduled | ||
|
|
||
NiveditJain marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ### Trigger Types | ||
|
|
||
| Currently, **CRON** is the only supported trigger type, using standard 5-field cron expressions: | ||
|
|
||
| ``` | ||
| * * * * * | ||
| │ │ │ │ │ | ||
| │ │ │ │ └── Day of Week (0-7, Sunday = 0 or 7) | ||
| │ │ │ └──── Month (1-12) | ||
| │ │ └────── Day of Month (1-31) | ||
| │ └──────── Hour (0-23) | ||
| └────────── Minute (0-59) | ||
| ``` | ||
NiveditJain marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Implementation | ||
|
|
||
| ### JSON Configuration | ||
|
|
||
| Define triggers in your graph template: | ||
|
|
||
| ```json | ||
| { | ||
| "triggers": [ | ||
| { | ||
| "type": "CRON", | ||
| "value": { | ||
| "expression": "0 9 * * 1-5" | ||
| } | ||
| }, | ||
| { | ||
| "type": "CRON", | ||
| "value": { | ||
| "expression": "0 0 * * 0" | ||
| } | ||
| } | ||
| ], | ||
| "nodes": [ | ||
| // ... your graph nodes | ||
| ] | ||
| } | ||
NiveditJain marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| ### Python SDK Example | ||
|
|
||
| ```python | ||
| from exospherehost import StateManager, GraphNodeModel, CronTrigger | ||
|
|
||
| async def create_scheduled_graph(): | ||
| state_manager = StateManager( | ||
| namespace="DataPipeline", | ||
| state_manager_uri=EXOSPHERE_STATE_MANAGER_URI, | ||
| key=EXOSPHERE_API_KEY | ||
| ) | ||
|
|
||
| # Define your graph nodes | ||
| graph_nodes = [ | ||
| GraphNodeModel( | ||
| node_name="DataExtractorNode", | ||
| namespace="DataPipeline", | ||
| identifier="extractor", | ||
| inputs={"source": "initial"}, | ||
| next_nodes=["processor"] | ||
| ), | ||
| GraphNodeModel( | ||
| node_name="DataProcessorNode", | ||
| namespace="DataPipeline", | ||
| identifier="processor", | ||
| inputs={"raw_data": "${{ extractor.outputs.data }}"}, | ||
| next_nodes=[] | ||
| ) | ||
| ] | ||
|
|
||
| # Define triggers for automatic execution | ||
| triggers = [ | ||
| CronTrigger(expression="0 2 * * *"), # Daily at 2:00 AM | ||
| CronTrigger(expression="0 */4 * * *") # Every 4 hours | ||
| ] | ||
|
|
||
| # Create the graph with triggers | ||
| result = await state_manager.upsert_graph( | ||
| graph_name="data-pipeline", | ||
| graph_nodes=graph_nodes, | ||
| secrets={"api_key": "your-api-key"}, | ||
| triggers=triggers | ||
| ) | ||
|
|
||
| print(f"Graph created with {len(triggers)} triggers") | ||
| return result | ||
|
|
||
| # Run the function | ||
| import asyncio | ||
| asyncio.run(create_scheduled_graph()) | ||
| ``` | ||
|
|
||
| ## Common Cron Expressions | ||
|
|
||
| ### Basic Patterns | ||
|
|
||
| | Expression | Description | Example Use Case | | ||
| |------------|-------------|------------------| | ||
| | `"0 9 * * 1-5"` | Every weekday at 9:00 AM | Business reports | | ||
| | `"0 */6 * * *"` | Every 6 hours | Data synchronization | | ||
| | `"0 0 * * 0"` | Every Sunday at midnight | Weekly cleanup | | ||
| | `"*/15 * * * *"` | Every 15 minutes | Health checks | | ||
| | `"0 2 * * *"` | Daily at 2:00 AM | Nightly batch jobs | | ||
| | `"0 0 1 * *"` | First day of every month | Monthly reports | | ||
|
|
||
| ### Advanced Patterns | ||
|
|
||
| | Expression | Description | | ||
| |------------|-------------| | ||
| | `"0 9-17 * * 1-5"` | Every hour from 9 AM to 5 PM, weekdays only | | ||
| | `"0 0 * * 1,3,5"` | Monday, Wednesday, Friday at midnight | | ||
| | `"30 8 * * 1-5"` | Weekdays at 8:30 AM | | ||
| | `"0 12 1,15 * *"` | 1st and 15th of each month at noon | | ||
| | `"0 0 1 1,7 *"` | January 1st and July 1st at midnight | | ||
|
|
||
| ## Best Practices | ||
|
|
||
| ### Scheduling Considerations | ||
|
|
||
| 1. **Avoid Peak Times**: Schedule resource-intensive workflows during off-peak hours | ||
| 2. **Stagger Executions**: If you have multiple graphs, stagger their execution times | ||
| 3. **Consider Time Zones**: Cron expressions use server time (UTC by default) | ||
| 4. **Resource Planning**: Ensure your infrastructure can handle scheduled workloads | ||
|
|
||
| ### Error Handling | ||
|
|
||
| - **Retry Policies**: Combine triggers with retry policies for resilient automation | ||
| - **Monitoring**: Set up alerts for failed scheduled executions | ||
| - **Logging**: Ensure adequate logging for troubleshooting scheduled runs | ||
|
|
||
| ### Example with Retry Policy | ||
|
|
||
| ```python | ||
| from exospherehost import RetryPolicyModel, RetryStrategyEnum | ||
|
|
||
| # Define retry policy for scheduled executions | ||
| retry_policy = RetryPolicyModel( | ||
| max_retries=3, | ||
| strategy=RetryStrategyEnum.EXPONENTIAL, | ||
| backoff_factor=2000, | ||
| exponent=2 | ||
| ) | ||
|
|
||
| # Create graph with both triggers and retry policy | ||
| result = await state_manager.upsert_graph( | ||
| graph_name="robust-pipeline", | ||
| # Assuming `graph_nodes`, `secrets`, and `triggers` are defined as in previous examples | ||
| graph_nodes=graph_nodes, | ||
| secrets=secrets, | ||
| triggers=triggers, | ||
NiveditJain marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| retry_policy=retry_policy # Handles failures in scheduled runs | ||
| ) | ||
| ``` | ||
|
|
||
| ## Limitations | ||
|
|
||
| - **CRON Only**: Currently only cron-based scheduling is supported | ||
| - **No Manual Override**: Scheduled executions cannot be manually cancelled once triggered | ||
| - **Time Zone**: All cron expressions are evaluated in server time (UTC) | ||
| - **Minimum Interval**: Avoid scheduling more frequently than every minute | ||
|
|
||
| ## Next Steps | ||
|
|
||
| - **[Create Graph](./create-graph.md)** - Learn about graph creation | ||
| - **[Retry Policy](./retry-policy.md)** - Add resilience to scheduled executions | ||
| - **[Store](./store.md)** - Persist data across scheduled runs | ||
| - **[Dashboard](./dashboard.md)** - Monitor scheduled executions | ||
|
|
||
| ## Related Concepts | ||
|
|
||
| - **[Graph Components](./graph-components.md)** - Complete overview of graph features | ||
| - **[Python SDK](./python-sdk-graph.md)** - Type-safe graph creation with triggers | ||
NiveditJain marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.