Produce auspice compatible JSONs

[trvrb]

@famulare ---

It's really exciting to see push for auspice in simulated data. However, I need to ask for some small additions to make this work. There are two possible entry points here:

1. Generate node data JSON files for branch lengths and for internal traits. Each node in the tree including internal nodes needs a date attached to it, as well as branch_length and clock_length. To map locations, we also need a node data JSON that lists internal node states. We'd then stitch this together into an auspice-compatible JSON via augur export command.

2. Directly generate auspice compatible JSON from simulation. This shouldn't be any harder really than generating the currently metadata tsv + timetree.nwk.

I'd definitely recommend 2. I've just made a stripped down minimal example of what to target here: https://github.com/nextstrain/auspice/tree/master/examples.

This should be pretty self-explanatory. You can replace country with puma or neighborhood. A couple relevant details:

1. clade just needs to be an integer iterating through nodes in the tree. This should go away, but is here for now.
2. meta.json needs geo to be populated with lat/longs. I believe @jotasolano has already calculated these centroids here somewhere: https://github.com/seattleflu/seattle-flu-js-prototype

[famulare]

I confess I was hoping this was an easy one for you guys... (Without having thought about it carefully), is it possible to call augur::export.py and get what you need? I tried to format these with that in mind, from my limited understanding of the intermediate artifacts of an augur build.

[famulare]

Updating my previous comment after reading yours more carefully.... I guess I don't fully understand what you need to get to augur export. That's the easier path for me as I'm not familiar with the tools for working with phylogenetic trees in JSON.

[jameshadfield]

The easiest way to understand it is to clone https://github.com/nextstrain/zika and run the snakefile. The only dependency would be the nextstrain cli tool (you'd run nextstrain build .) or augur itself (run snakemake) and you can then examine the intermediate files which are used by augur export. Not sure which is easier: to create the JSONs yourself, or create the files needed for augur export!

[famulare]

I can do that, so my question is timing. Do you want to use these for prototyping soon, or are you both just generically interested in the ability to push models to auspice? If the first, I’ll find a few hours in the next few days. If the second, I’ll do it in May.

[trvrb]

@famulare ---

I wanted this mainly for demo purposes. There is no huge hurry. As @jameshadfield says, easiest way to see what augur export needs is to run https://github.com/nextstrain/zika-tutorial. However, I do believe you're essentially writing out a Newick timetree directly from your simulation, yes? If this is the indeed the case, I would again recommend directly writing JSONs. If you aren't directly writing Newick, then, maybe some other approach is better.

[trvrb]

@famulare / @jameshadfield --- I just took another look at this. The core issue is that simulatedSFS_flu_seasonal_h3n2_ha_timetree.nwk has all the information we need, but that augur export expects branch lengths and dates to be encoded in a separate branch-lengths.json file, that lists data per node. This is normally produced by augur refine when it runs TreeTime: https://nextstrain.org/docs/tutorials/zika#get-a-time-resolved-tree. I believe it would be a fairly hairy script to take the simulatedSFS_flu_seasonal_h3n2_ha_timetree.nwk and simulatedSFS_flu_seasonal_h3n2_ha_metadata.tsv and create the branch-lengths.json file.