Skip to content

Conversation

@amadeovezz
Copy link
Contributor

@amadeovezz amadeovezz commented Aug 25, 2025

NEW FEATURES:

  • Return job dag and file dag
  • Return completed jobs in save-config
  • Return completed, planned and unscheduled files in save-config
  • Actually implement get_config

CODE

  • Refactor state management into a new class
  • Config object stores state
  • Run endpoint no longer calls find_buildable_targets - gets them from the config. This keeps targets consistent from calls to config and run
  • Use snakemake to determine completed files/jobs instead of searching through the dag

OTHER NOTES

-We always generate a new config even if the params + available files are the same. We let snakemake determine this.

API CHANGES:

Modification to save-config

Now returns

success_json(
        {
          config_id: config.id,
          files: config.state['files'],
          jobs: config.state['jobs'],
          params: JSON.parse(workflow_params_json)
       }

where the state object looks like:

 {
        available_files: available_files,
        files: {
          completed: completed_files,
          planned: files_planned,
          unscheduled: unscheduled_files
        },
        jobs: {
          completed: completed_jobs,
          planned: jobs_planned,
          unscheduled: unscheduled_jobs
        }
      }

Modification to create-workspace

New param type: dag: obj.dag returns an adjacency list that represents the job dag (in json)
New param: file_dag: which is the file dag - also in json

New endpoint get-config

new endpoint get-config - api/v2/:project_name/workspace/:workspace_id/config/:config_id

…emake method to determine files/job scheduled
…ndency graph, unify downstream nodes for both data structures
…wnstream nodes, allow future_state to receive files, add new endpoint to retrieve state
@amadeovezz amadeovezz force-pushed the vulcan-state-management branch from 1a9b3a4 to c4d780a Compare September 15, 2025 21:10
@amadeovezz amadeovezz changed the title WIP: Improved state management Improved state management Oct 2, 2025
@dtm2451 dtm2451 merged commit 395bc98 into master Dec 2, 2025
10 checks passed
@dtm2451 dtm2451 deleted the vulcan-state-management branch December 2, 2025 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants