Skip to content

[Bug]: Resource leak in KafkaIO GCS truststore file download #37680

@PDGGK

Description

@PDGGK

What happened?

In KafkaReadSchemaTransformProvider.identityOrGcsToLocalFile, when downloading a GCS file (e.g., truststore) to the local filesystem, ReadableByteChannel, FileOutputStream, and WritableByteChannel are opened before the copy loop but closed manually after the loop. If an IOException occurs during the copy (e.g., disk full, network error), all three resources leak because the close calls are never reached.

Additionally, the catch block wraps the error in an IllegalArgumentException but drops the original IOException cause, making debugging harder.

Location:
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaReadSchemaTransformProvider.java, method identityOrGcsToLocalFile

Fix:

  • Convert to try-with-resources to ensure resources are closed even on exceptions
  • Chain the original IOException as the cause of IllegalArgumentException

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions