Apache Beam is a great library that enables you to build batch/streaming pipelines in the unified way. Then again, there are some pitfalls and technical challenges when writing practical applications.
This project aims to collect know-hows and best practices about Apache Beam.
| example | description |
|---|---|
| word-count-beam | This is most likely your first Apache Beam project. It includes fundamental features and best practices of Beam. |
| multi-storage | This shows how to push data to multiple storage locations. |
| gcs-triggered | This shows how to start processing data in an event-driven way with GCS notification. |
| slowly-changing-lookup-cache | Implementation of Beam SideInput which updates periodically |
| dead-letter | Implementation of Dealing with bad data pattern which deals invalid data with dead-letter. |
You can add an example project by sending PRs.
Please follow the below procedure:
- Please write a
README.mdusing the template. - Update "Example summary" section of this file.
- @Kenji-H
- @byam