Skip to content

Add async client support for improved performance #45

@anjor

Description

@anjor

Description

The foundry-platform-python SDK provides AsyncFoundryClient for asynchronous operations. Adding async support to the CLI could significantly improve performance for batch operations and large data processing tasks.

Benefits

  • Improved Performance: Concurrent operations for batch processing
  • Better Resource Utilization: Non-blocking I/O operations
  • Enhanced User Experience: Progress indicators during long-running operations
  • Scalability: Handle larger datasets and bulk operations more efficiently

Use Cases

  • Batch dataset operations (upload/download multiple files)
  • Bulk ontology object processing
  • Large-scale SQL query execution
  • Mass admin operations (user/group management)
  • Concurrent folder operations

Implementation Considerations

  • Add async variants of existing commands (e.g., --async flag)
  • Implement progress bars for long-running async operations
  • Handle async error management and cleanup
  • Consider thread pool configuration for optimal performance
  • Maintain backward compatibility with synchronous operations

Technical Requirements

  • AsyncFoundryClient integration
  • Progress tracking and reporting
  • Proper error handling for concurrent operations
  • Configuration options for concurrency limits
  • Testing framework for async operations

Example Commands

# Async batch file upload
pltr dataset upload --async --files *.csv --dataset-rid ri.foundry.main.dataset.123

# Concurrent folder operations
pltr folder create --async --batch folders.json

# Bulk ontology object queries
pltr ontology objects list --async --batch-size 1000

References

  • SDK AsyncFoundryClient documentation
  • Current synchronous implementations in src/pltr/services/

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions