This repository was archived by the owner on Oct 9, 2018. It is now read-only.

Description
Per irc today :) Maybe put a snippit / git link near the page with the on-going spark job, or wiki this up a bit?
GL
07:13 < gregglind> peeps, what's the recommended path for using a spark job as the basis for a dashboard?
07:14 < gregglind> 1. run it as a cron... then get the data from S3? Is there permissions and stuff to deal with?
07:14 < gregglind> (i.e., wanting to setup a json or csv that other inside-moz dashboards can use)
07:15 <@mreid> gregglind: public dashboards are easy. moz-internal dashboards take a little more effort, but it's doable.
07:20 < gregglind> tell me how to make the easy :)
07:27 <@mreid> gregglind: run a scheduled spark job, set it to public, and write out the data you want for your dashboard to
$PWD (or just make the notebook itself be the dashboard)
07:27 <@mreid> gregglind: then your output will be web-accessible at a URL like this:
07:27 <@mreid> https://analysis-output.telemetry.mozilla.org/mreid-test-new-dev/data/test.txt
07:30 * gregglind confused, doesn't the machine die?
07:30 <@mreid> gregglind: if you want to just have a notebook that updates (with embedded viz and stuff), it would appear at
https://analysis-output.telemetry.mozilla.org/<job_name>/data/MyAwesomeNotebook.ipynb
07:30 <@mreid> gregglind: it's magic, don't worry :P
07:31 < gregglind> I am trying to avoid 'the notebook is the dashboard.
07:31 < gregglind> Okay :)
is $PWD) get auto-uploaded to S3 for posterity
07:31 < gregglind> Can you maybe just copy that to a little gist :) I will file a bug against telemetry dash :)
07:32 <@mreid> so you could also output "my_fancy_dataset.csv[.gz]" and it would become available via a similar URL