A comprehensive Streamlit-based application for task invocation, design, and workflow management within the STELAR KLMS (Knowledge and Learning Management System).
This application provides a user-friendly interface for managing and executing various data processing tasks in the STELAR ecosystem. It supports workflow processes, dataset management, S3 storage integration, and specialized tools for agricultural and meteorological data processing.
- Secure token-based authentication with STELAR API
- Session management for authenticated users
- Browse and filter available workflow processes
- Process selection with detailed task information
- Advanced filtering and column visibility controls
- Real-time process status monitoring
- Paginated dataset browsing with concurrent loading
- Multi-dataset selection capabilities
- Advanced filtering by various data types
- Organization-based dataset categorization
- MinIO-based S3 storage browser
- Bucket exploration with folder navigation
- File management and path selection
- Secure credential handling
- Fertilizers Mode: Matches NPK values with fertilizer datasets
- Pesticides Mode: Matches active substances with pesticide databases
- Multi-language support (Italian, English, French, German)
- Interpolates missing meteorological station data
- Processes meteorological files with coordinate data
- Generates complete datasets for analysis
- Agricultural suitability scoring for raster data
- Configurable classification parameters
- Temperature and climate-based scoring
- Clone the repository:
git clone <repository-url>
cd task-execution-gui- Install dependencies:
pip install -r requirements.txt- Run the application:
streamlit run app.pyKey dependencies include:
streamlit>=1.45.1- Web application frameworkpandas>=2.3.0- Data manipulation and analysisrequests>=2.32.4- HTTP library for API callsminio>=7.2.15- S3-compatible object storage clientopenpyxl>=3.1.5- Excel file processing
See requirements.txt for the complete list of dependencies.
STELAR_BASE_URL: Base URL for STELAR API (default:https://klms.stelar.gr/stelar)
Tool configurations are defined in utils/tool_configs.py and include:
- Process IDs and tool specifications
- Input/output dataset mappings
- Parameter configurations
- S3 storage paths
- Launch the application and log in with your STELAR credentials
- The system will authenticate and store your session token
- Navigate to "Workflow Processes" tab
- Browse available processes with filtering options
- Select a process to view associated tasks
- Process details include execution state and organization info
- Use the "Datasets" tab to browse available datasets
- Apply filters to narrow down results
- Select single or multiple datasets as needed
- Navigate through paginated results (up to 3,750 datasets)
- Access the "S3 Menu" to browse storage buckets
- Navigate folder structures with breadcrumb navigation
- Select paths for task output destinations
- View file details and manage storage resources
- Go to "Tool Selection" tab and choose from available tools
- Configure tool-specific parameters:
- Agri Products Match: Select mode (fertilizers/pesticides) and language preferences
- Missing Data Interpolation: Configure meteorological data processing
- VSR Tool: Set up raster classification parameters
- Complete the workflow by selecting:
- Process ID from workflow management
- Input datasets from dataset browser
- Output S3 path from storage navigator
- Execute tasks with generated JSON configurations
The application integrates with STELAR KLMS API endpoints:
/api/v1/users/token- Authentication/api/v2/processes.fetch- Workflow processes/api/v2/datasets.fetch- Dataset management/api/v2/tools.fetch- Available tools/api/v2/organizations.fetch- Organization data/api/v2/task/{id}- Task details/api/v1/users/s3/credentials- S3 credentials
{
"process_id": "f9645b89-34e4-4de2-8ecd-dc10163d9aed",
"name": "Agri Products Match",
"tool": "agri-products-match",
"inputs": {
"npk_values": ["dataset-id-1"],
"fertilizer_dataset": ["dataset-id-2"]
},
"parameters": { "mode": "fertilizers" },
"outputs": {
"matched_fertilizers": {
"url": "s3://bucket/path/output.csv",
"dataset": "output-dataset-id"
}
}
}{
"process_id": "f9645b89-34e4-4de2-8ecd-dc10163d9aed",
"name": "VSR Classification",
"tool": "vocational-score-raster",
"inputs": { "rasters": "d0::owned" },
"parameters": {
"Tmax_max_summer_2011_2021.tif": {
"val_min": 26,
"val_max": 28.5,
"new_val": 1
}
},
"outputs": {
"scored_files": {
"url": "s3://bucket/VSR/output",
"dataset": "output-dataset-id"
}
}
}- Fork the repository
- Create a feature branch (
git checkout -b feature/new-feature) - Commit your changes (
git commit -am 'Add new feature') - Push to the branch (
git push origin feature/new-feature) - Create a Pull Request
This project is part of the STELAR project ecosystem. Please refer to the project's licensing terms.
For issues and questions:
- Check the application logs for error details
- Ensure proper authentication credentials
- Verify API endpoint availability
- Contact the STELAR development team for system-specific issues
- The application uses caching (
@st.cache_data) for performance optimization - Concurrent data loading is implemented for large dataset operations
- Session state management maintains user selections across page navigation
- Error handling includes retry logic for API requests
- The interface supports both single and multi-selection workflows