Skip to content

Develop I/O Client #3

@jaehyeon-kim

Description

@jaehyeon-kim

The sql pyio connector should have a composite transform that executes a sql query in one or more partitions and outputs the results

Acceptance Criteria

  • Create two DoFns
    • GenerateQueryFn generates a sql query into partitions
    • ProcessQueryFn executes partitioned queries as a splittable DoFn
  • Create a composite transform that links the DoFns
    • It should be triggered at the beginning
    • Note to Reshuffle() so that each query is executed separately by preventing fusion

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions