Airflow task to invalidate cloudfront cache

[eveleighoj]

Overview
Want a task in Airflow that invalidates the cache from Cloudfront.

Tech Approach

• Do we want to use a bash or python script for this? (easier to test in Python)
• Do we want to invalidate all of the urls or just a proportion of them?
  • Can we be specific about which ones we want to invalidate?
  • Should it run at the end of everything? - Simpler but some issues
  • Should it be part of each collection? - This seems preferable especially when it comes to failure scenarios.
• Who has the IAM permissions to run / edit this?

Acceptance Criteria/Tests
These do depend on the tech approach taken, e.g. does the cache get invalidated after each collection has run or after the whole thing has run.

Resourcing & Dependencies
AWS CLI link
Boto library link