feat: IBM COS connector with HMAC and S3 Connector by edwinjosechittilappilly · Pull Request #1077 · langflow-ai/openrag

edwinjosechittilappilly · 2026-03-09T15:47:47Z

This pull request adds support for configuring and managing AWS S3 and IBM Cloud Object Storage (COS) connectors through new settings dialogs and API hooks, improving the user experience for direct-auth connectors. It introduces new React Query hooks for fetching and updating connector configuration and status, updates the connector card UI to support configuration dialogs, and adds icon support for these connectors.

Connector configuration and management enhancements:

Added new React Query hooks for configuring AWS S3 and IBM COS connectors, allowing users to set credentials, endpoints, and bucket filters directly from the UI (useS3ConfigureMutation.ts, useIBMCOSConfigureMutation.ts). [1] [2]
Introduced hooks to fetch default configuration and bucket status for both AWS S3 and IBM COS, enabling the UI to display current connector state and available buckets (useS3DefaultsQuery.ts, useIBMCOSDefaultsQuery.ts, useS3BucketStatusQuery.ts, useIBMCOSBucketStatusQuery.ts, useIBMCOSBucketsQuery.ts). [1] [2] [3] [4] [5]

UI and UX improvements:

Updated the connector card UI to support an "Edit configuration" button for connectors that support direct configuration (e.g., IBM COS, AWS S3), and to show a settings dialog instead of OAuth flow when appropriate (connector-card.tsx, connector-cards.tsx). [1] [2] [3] [4] [5] [6] [7]
Added icon support for AWS S3 and IBM COS connectors throughout the app, improving visual clarity (knowledge/page.tsx, connector-cards.tsx). [1] [2] [3]

Connector sync and feedback improvements:

Enhanced the connector sync mutation to support new options for syncing all files or filtering by bucket names, supporting more granular ingestion for object storage connectors (useSyncConnector.ts).
Improved the connect mutation to handle direct-auth connectors by refreshing connector status and displaying success toasts without requiring OAuth redirects (useConnectConnectorMutation.ts).

Screen.Recording.2026-03-10.at.1.39.55.AM.mov

Command to run MinIO:

podman run -p 9000:9000 -p 9001:9001 \
    -e MINIO_ROOT_USER=minioadmin \
    -e MINIO_ROOT_PASSWORD=minioadmin \
    quay.io/minio/minio server /data --console-address ":9001"

src/api/connectors.py

Introduce a full Amazon S3 / S3-compatible connector and integrate it into the app. Backend: add new S3 connector implementation and auth helpers (src/connectors/aws_s3), register API routes for defaults/configure/list/bucket-status, wire S3 into connection manager and connector registry, and include AWS S3 in sync_all. Frontend: add S3 settings form/dialog, React Query hooks (defaults, configure, bucket status), connect S3 UI into connector cards, cloud picker, and upload flow with an S3 bucket view and direct-sync support. CLI/TUI: add S3-related env fields and config prompts. Misc: small UI icon usages and query invalidation added to keep state in sync.

src/api/connectors.py

Add env-var fallback getters for S3 credentials and clearer errors: implement get_client_id and get_client_secret to read from config or AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY and raise ValueError when missing. Frontend fixes: treat a connected connector as available in the connector card UI, and surface S3 bucket loading errors on the upload page by including the query error in the response and rendering a descriptive error message when bucket fetch fails.

Connector card: use isConnected alone for active state, treat a connector as configured when isConnected or connector.available, and show a "Configure" action (with Settings2 icon) when onConfigure is provided; also keep existing loading state. S3 settings dialog: import useEffect and add an effect to sync buckets and selectedBuckets when defaults.bucket_names load asynchronously so defaults are applied after dialog mount. AWS logo: replace fill="currentColor" with an explicit color (#232F3E) for consistent rendering.

Analuisa1a · 2026-03-12T18:56:47Z

Lates file: Figma

IBM cloud storage icons by theme:
IBM cloud icons.zip

Updates for buckets to allow multi-select

…_SPIKE

…g user_id parameter

…o IBM_COS_S3_SPIKE

This reverts commit 9fee2e8.

mfortman11 · 2026-03-16T20:58:58Z

frontend/contexts/auth-context.tsx

These changes mirror those in #1090 and are tied to the the IBM_AUTH_ENABLED env

edwinjosechittilappilly · 2026-03-16T21:16:36Z

@mfortman11 can you also resolve the conflicts too?

…_SPIKE

Copilot

Pull request overview

Copilot reviewed 50 out of 52 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

frontend/app/settings/_components/ibm-cos-settings-form.tsx

+    auth_mode: "iam" | "hmac";
+    endpoint: string;
+    // IAM
+    api_key: string;
+    service_instance_id: string;
+    // HMAC
+    hmac_access_key: string;
+    hmac_secret_key: string;
+}
+
+interface IBMCOSSettingsFormProps {
+    /** Available buckets after a successful test — null means not yet tested */
+    buckets: string[] | null;
+    selectedBuckets: string[];
+    onSelectedBucketsChange: (buckets: string[]) => void;
+    isFetchingBuckets: boolean;
+    bucketsError: string | null;
+    onTestConnection: () => void;
+    apiKeySet?: boolean;
+    hmacAccessKeySet?: boolean;
+    hmacSecretKeySet?: boolean;
+    formError?: string | null;
+    /** When true, IAM tab is greyed out and HMAC is the only selectable option */
+    disableIam?: boolean;
+}
+
+export function IBMCOSSettingsForm({
+    buckets,
+    selectedBuckets,
+    onSelectedBucketsChange,
+    isFetchingBuckets,
+    bucketsError,
+    onTestConnection,
+    apiKeySet,
+    hmacAccessKeySet,
+    hmacSecretKeySet,
+    formError,
+    disableIam = false,
+}: IBMCOSSettingsFormProps) {
+    const {
+        register,
+        control,
+        formState: { errors },
+    } = useFormContext<IBMCOSFormData>();


src/utils/file_utils.py

 def get_file_extension(mimetype: str) -> str:
-    """Get file extension based on MIME type"""
+    """Get file extension based on MIME type. Returns None if the type is unknown."""
    mime_to_ext = {


src/connectors/ibm_cos/api.py

+    try:
+        cfg = connection.config
+        cos = create_ibm_cos_resource(cfg)
+        all_buckets = [b.name for b in cos.buckets.all()]
+    except Exception:


src/connectors/connection_manager.py

+                "name": S3Connector.CONNECTOR_NAME,
+                "description": S3Connector.CONNECTOR_DESCRIPTION,
+                "icon": S3Connector.CONNECTOR_ICON,
+                "available": os.environ.get("IBM_AUTH_ENABLED", "").lower() in ("1", "true", "yes"),
            },


frontend/app/upload/[provider]/page.tsx

+    refetch,
+  } = useIBMCOSBucketStatusQuery(connector.connectionId, { enabled: true });
+  return (
+    <BucketView
+      connector={connector}
+      buckets={buckets}
+      isLoading={isLoading}


src/connectors/aws_s3/connector.py

+    async def authenticate(self) -> bool:
+        """Validate credentials by listing accessible buckets."""
+        try:
+            resource = self._get_resource()
+            list(resource.buckets.all())


src/connectors/ibm_cos/models.py

+class IBMCOSConfigureBody(BaseModel):
+    auth_mode: str  # "iam" or "hmac"
+    endpoint: str


src/connectors/connection_manager.py

+                "name": IBMCOSConnector.CONNECTOR_NAME,
+                "description": IBMCOSConnector.CONNECTOR_DESCRIPTION,
+                "icon": IBMCOSConnector.CONNECTOR_ICON,
+                "available": os.environ.get("IBM_AUTH_ENABLED", "").lower() in ("1", "true", "yes"),
+            },


frontend/app/settings/_components/s3-settings-form.tsx

+    access_key: string;
+    secret_key: string;
+    endpoint_url: string;
+    region: string;
+}
+
+interface S3SettingsFormProps {
+    /** Available buckets after a successful test — null means not yet tested */
+    buckets: string[] | null;
+    selectedBuckets: string[];
+    onSelectedBucketsChange: (buckets: string[]) => void;
+    isFetchingBuckets: boolean;
+    bucketsError: string | null;
+    onTestConnection: () => void;
+    accessKeySet?: boolean;
+    secretKeySet?: boolean;
+    formError?: string | null;
+}
+
+export function S3SettingsForm({
+    buckets,
+    selectedBuckets,
+    onSelectedBucketsChange,
+    isFetchingBuckets,
+    bucketsError,
+    onTestConnection,
+    accessKeySet,
+    secretKeySet,
+    formError,
+}: S3SettingsFormProps) {
+    const { register } = useFormContext<S3FormData>();
+
+    const toggleBucket = (name: string, checked: boolean) => {
+        if (checked) {
+            onSelectedBucketsChange([...selectedBuckets, name]);
+        } else {
+            onSelectedBucketsChange(selectedBuckets.filter((b) => b !== name));
+        }
+    };
+
+    const toggleAll = (checked: boolean) => {
+        onSelectedBucketsChange(checked ? (buckets ?? []) : []);
+    };


ricofurtado · 2026-03-17T18:58:20Z

src/connectors/ibm_cos/connector.py

+    CLIENT_ID_ENV_VAR = "IBM_COS_API_KEY"
+    CLIENT_SECRET_ENV_VAR = "IBM_COS_SERVICE_INSTANCE_ID"
+
+    def get_client_id(self) -> str:


We'll need to deal with API returning credentials to client, this is a credential disclosure bug.
The status payload includes connector.get_client_id() take a look in src/api/connectors.py(352) the JSON returns the raw api_key, so simply loading the connectors page fetches the secret via frontend/app/api/queries/useGetConnectorsQuery.ts line 66). T

ricofurtado

I believe it is safe to merge. Have take some notes we should rework.

ricofurtado · 2026-03-17T19:10:20Z

src/api/connectors.py


        # Cloud connector types to sync
-        cloud_connector_types = ["google_drive", "onedrive", "sharepoint"]
+        cloud_connector_types = ["google_drive", "onedrive", "sharepoint", "ibm_cos", "aws_s3"]


The new connectors will be skipped if there are not files in Opensearch right? Does it need sync_all / bucket_filter for initial ingest?

feat: Integrate IBM COS connector with HMAC as default connection method

0e25241

Copilot AI review requested due to automatic review settings March 9, 2026 15:47

github-actions bot added frontend 🟨 Issues related to the UI/UX backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) enhancement 🔵 New feature or request labels Mar 9, 2026

Copilot started reviewing on behalf of edwinjosechittilappilly March 9, 2026 15:48 View session

github-advanced-security bot found potential problems Mar 9, 2026

View reviewed changes

src/api/connectors.py Fixed Show fixed Hide fixed

src/api/connectors.py Fixed Show fixed Hide fixed

src/api/connectors.py Fixed Show fixed Hide fixed

This comment was marked as outdated.

Sign in to view

Merge branch 'main' into IBM_COS_S3_SPIKE

7e05cad