Handle external dependencies #27
Description
At Carbonfact, we use Observable for our dashboards and data viz. We have no way to know if a notebook is broken or not. So when we edit a view, we cross fingers it won't break too many notebooks. We fix the notebooks when someone complains on Slack. The same was true when I worked at Alan with Metabase dashboards. Obviously we could have a better setup.
dbt has a nice way to handle this called exposures. You basically declare resources and their dependencies. These resources are then part of the DAG and can be used with the dbt toolchain. I think this is a huge step forward, but its a bit too cumbersome to maintain.
I'm thinking it should be possible to grab all the notebooks from Observable, list the cells that have SQL in them, dump the SQL queries to a directory, and continue from there. Users would have their freedom to decide which queries they want to put in that directory. We could provide utilities to consume from Observable and Metabase.