Query tool

[Farof]

I'd like to add a simple query tool that returns values of a certain property in a given file across multiple replays. Probably nothing that replaces a real parsing and analysis tool but a quick way to view some data that would be useful to build the reference.

[jnovack]

jq is a lightweight and flexible command line JSON processor.

The language is a BEAST to learn, but it's extremely sexy.

All examples assume you are in the directory containing the extracted replays.

** ... all m_hero values (echo to a file and then you can | uniq)

for p in */
do
    cat "$p"/replay.details.json | jq ".m_playerList[].m_hero"
done

** ... all m_string values where another field equals something (can't cross files)

for p in */
do
    cat "$p"/replay.message.events.json | jq ".[]|select(._userid.m_userId == 7) | .m_string"
done

** Print directory name where TickleMeOzmo was on Jaina and won.

for p in */
do
    cat "$p"/replay.details.json | jq "if path(.m_playerList[]|select(.m_name == \"TickleMeOzmo\" and .m_hero == \"Jaina\" and .m_result == 1)) then \"$p\" else \"\" end"
done

[Farof]

Ah, with you saying that this repo does too much, that seems a bit of an overkill :)

I've started on it on an unpushed branch, just to see what I wanted out if it. Right now I can query direct path like prop.subprop.subsubprop and I'd like to be able to query thought arrays and aggregate data from across multiple replays.

I hadn't thought about more complex queries with filters yet, and I can see how it could be useful. I'll look into that lib.

Just to remind my initial statement though, while usage will dictate were this tool goes, I don't see it useful for more than basic data extraction. For analysis, I hope more powerful tools and applications and websites emerge. Writing my own is certainly a goal, separate from this repository, and this query tool probably won't be adapted for it.

My typical usecase for this tool would be to extract all internal unit names or mount or skin to incorporate into the reference.

[jnovack]

Oh, I thought your reasoning for this was to fill out the enums in the reference material. (e.g. "find all values of 'x' so we can document 'x'). I was not suggesting it as a tool to be added for the repo, but a tool for filling out the documentation.

[jnovack]

This is how I'm getting all values for enums.

for p in */; do cat "$p"replay.attributes.events.json | jq ".scopes[].\"500\"[].value" 2>/dev/null; done | sort | uniq -c | sort -r

This should get you all the values for .scopes.NUM.500.value

 552 "Humn"
   8 "Comp"

[jnovack]

Someone already did most of the work on the heroes.

http://heroesjson.com/

[Farof]

Pushed something basic, probably won't do more and will be moved to another repo at some point.

Get the names of all players that are not bots:

$ node bin/query.js replays/ -f details -q "m_playerList[m_toon.m_id != 0].m_name"

Get the internal name of every unit that is born:

$  node bin/query.js replays/ -f tracker.events -q "[_event = NNet.Replay.Tracker.SUnitBornEvent].m_unitTypeName"

Need to document it better somewhere.