Dynamic worker execution and CLI command refactoring

[berggren]

Summary

Refactored the CLI command structure, implemented dynamic worker execution with workflow support, and improved API client request handling and error reporting.

This PR introduces the run command, a powerful new way to execute OpenRelik workers directly from the CLI. It enables users to build and execute complex workflows using familiar command-line patterns, including support for sequential chaining, parallel execution, and automated file handling.

Key Features

• Dynamic Subcommands: Worker commands are automatically generated based on the server's registered workers.
• Sequential Chaining (--then): Execute workers in a chain where each step processes the output of the previous one.
• Parallel Execution (--and): Run multiple workers concurrently on the same set of input files.
• Automatic File Upload: Local file paths passed as arguments are automatically uploaded to a "CLI Uploads" folder before execution.
• Live Monitoring: Real-time terminal UI with spinners and progress bars to track workflow and task status.
• Result Management: Configurable download policies (all, final, none) to automatically fetch output files.

---

Examples

Execute OpenRelik workers on files.
Subcommands are dynamically generated based on registered workers.

Command chaining is supported using --then:
openrelik run strings --then grep --regex "foo" 123

Parallel execution is supported using --and:
openrelik run strings --and grep 123

Usage:
  openrelik run [command]

Available Commands:
  capa            Capa Malware Analysis
  extract_archive Extract files from archives
  grep            Grep
  image_export    Extract files from disk images
  log2timeline    Plaso Log2Timeline
  psort           Plaso Psort
  strings         Strings

Flags:
      --and string                  Run workers in parallel (use as delimiter)
      --download string             Download policy (final, all) (default "final")
      --dry-run                     Generate and display workflow spec without executing
  -h, --help                        help for run
      --no-download                 Do not download any results
  -o, --output-dir string           Output directory for downloads (default ".")
      --task-folders                Organize downloads into task folders
      --then string                 Chain workers (use as delimiter)
      --upload-folder-id int        Folder ID to upload local files to
      --upload-folder-name string   Folder name to upload local files to (default: CLI Uploads (username))

Sequential Chaining with --then
Extract strings from a file and then search for a specific regex pattern:

openrelik run strings --then grep --regex "password" 12345

Parallel Execution with --and
Run multiple analysis tools on a local file simultaneously:

openrelik run strings --and hash --and exiftool ./evidence.bin

Advanced Workflow with Downloads
Run a chain on a local file and download all intermediate and final results to a specific directory:

openrelik run strings --then grep --regex "FLAG{.*}" ./mem.dmp -o ./analysis_results --download all

Dry Run
Generate and inspect the workflow JSON specification without starting the execution:

openrelik run strings --then grep --regex "admin" 123 --dry-run

Technical Details:

• API Client Refinement: Updated NewRequest to split endpoints at the query string separator, preventing JoinPath from escaping the ? character. Modified error string generation to strip query parameters and fragments from URLs for cleaner error messages.
• CLI Command Restructuring: Renamed plural command groups (users, folders, files, workers) to singular form (user, folder, file, worker) to follow standard CLI conventions.
• Dynamic Worker Execution: Introduced a new run command that dynamically generates subcommands based on a local worker cache. This supports command chaining via --then and parallel execution via --and.
• Workflow Management: Added a dedicated workflow command for creating, running, and inspecting workflow metadata and status. Updated the WorkflowsService.Get method to use the direct /workflows/{id} endpoint, removing the folder ID requirement.
• Terminal UI Improvements: Implemented a new ProgressTracker and WorkflowMonitor using Unicode symbols and colors. This provides real-time status updates, transfer speeds, and ETA estimates for file operations and task execution.
• Schema Updates: Modified the Task struct to include a recursive Tasks field, enabling support for nested task structures within workflows.
• Worker Caching: Added local caching logic for registered workers in the user's config directory to enable dynamic CLI autocomplete and subcommand generation without repeated API calls.