Skip to content

Sync mode selection should validate against stream's supported_sync_modes #917

@devin-ai-integration

Description

@devin-ai-integration

Summary

When building the configured catalog, the _get_sync_mode helper function unconditionally falls back to SyncMode.incremental, even for streams that only support full_refresh. This can cause connector failures for streams that don't support incremental sync.

Background

This issue was identified during code review of PR #899 (force_full_refresh fix). The current behavior predates that PR, so it's not a regression - but it's a valid edge case that should be addressed.

Requested by: @aaronsteers (AJ Steers)

Current Behavior

def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
    supported_modes = getattr(stream, "supported_sync_modes", None)

    if force_full_refresh:
        if supported_modes and SyncMode.full_refresh in supported_modes:
            return SyncMode.full_refresh
        return SyncMode.incremental  # BUG: stream might not support incremental

    return SyncMode.incremental  # BUG: stream might only support full_refresh

Expected Behavior

The selected sync mode should always be validated against supported_sync_modes when that attribute is present:

def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
    supported_modes = getattr(stream, "supported_sync_modes", None)

    if force_full_refresh:
        if supported_modes and SyncMode.full_refresh in supported_modes:
            return SyncMode.full_refresh
        if supported_modes and SyncMode.incremental in supported_modes:
            return SyncMode.incremental
        return SyncMode.full_refresh  # fallback for unknown/empty

    # Default: prefer incremental if supported, otherwise full_refresh
    if supported_modes and SyncMode.incremental in supported_modes:
        return SyncMode.incremental
    if supported_modes and SyncMode.full_refresh in supported_modes:
        return SyncMode.full_refresh
    return SyncMode.incremental  # preserve backwards compat for unknown

Affected Code

  • airbyte/sources/base.py - _get_sync_mode helper function (around line 449)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions