You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+2-6Lines changed: 2 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -70,16 +70,12 @@ _(Instructions to be added)_
70
70
- This design assumes a "happy path". In a distributed system, any network call can fail and any message can be delivered more than once.
71
71
- A flood of traffic to Manager Service can lead to massive scalability issues, especially with large files, in which the service is designed to stream one file at a time.
72
72
- How the heck can I even debug this asynchronous architecture?
73
-
- Google managed services are not 100% SLA.
73
+
- Google managed services are not 100% SLA. (obviously)
74
74
- Services can access GCS and Pub/Sub with no known rate limits, quotas.
75
75
76
76
## Todo
77
-
- Write unit tests and integration tests for manager and worker services.
78
-
- Containerize services with Docker.
79
-
- Use Kubernetes (GKE) to manage, scale, and orchestrate the services.
80
-
- Build Continuous Integration (CI) pipeline with Github Actions.
81
-
- Build Continuous Deployment (CD) pipeline with ArgoCD.
82
77
- Integrate Google Cloud Operations Suite for monitoring, logging, and tracing.
78
+
- Build Continuous Deployment (CD) pipeline with ArgoCD.
83
79
- Implement retries logic at streaming data, sending/receiving messages to/from Message Queue, etc.
84
80
- Create a budget plan to estimate the cost of running this system on Google Cloud Platform (GCP).
85
81
- Split file (>= 50GB) in chunks for parallel compression. **Requires architecture redesign**
0 commit comments