Enhancement: add jval(1) tool that uses the XPath for JSON to analyse, transform and selectively extract data from JSON

[lcn2]

Is there an existing issue for this?

• I have searched for existing issues and did not find anything like this

Describe the enhancement

We recommend adding a jval(1) tool to allow one to analyse, transform and selectively extract data from a JSON document.

This tool should use the XPath for JSON model.

FYI: XPath is a general model that allows one to analyse, transform and selectively extract data from XML. The above mentioned XPath for JSON applies that model to JSON: often called JSONPath.

There exist several implementations JSONPath. For example, the "outer repo" uses JSONPath.sh as a bash(1) implementation of JSONPath. And while that implementation of very slow, it is used by tools such as bin/jval-wrapper.sh and bin/jfmt-wrapper.sh and bin/jprint-wrapper.sh.

To speed up the above mentioned wrapper tools, a jval(1) tool needs to be created that uses the same XPath for JSON model used by the very slow JSONPath.sh.

Relevant images, screenshots or other files

We recommend the following command line for jval(1):

usage: jval [-h] [-v level] [-V] [--use-jparse] file.json pattern

    -h		    print help message and exit
    -v level	    set verbosity level (def level: 0)
    -V		    print version string and exit

    --use-jparse    Do nothing: this option is a simple way to verify this tool came from the jparse toolset.

    NOTE: Other command line options are being considered.  This command line model will be updated later.

    file.json           JSON input file
                        If file.json is - (dash) read from standard input.
                        If file.json is . (dot) do not read any file, perform an internal test and exit accordingly.

    pattern          JPATH Expression

A good resource for JSONPath Syntax and Semantics is RFC9535.

NOTE: We are still considering additional command line options. One such possibility comes from the jsonpath-cli facility and some of its jpp(1) options. See also the jsonpath-cli repo.

Another XPath for JSON implementation and possible command line model is jsp.

Relevant links

See RFC9535 for an overview of JSONPath Syntax and Semantics.

The XPath of JSON is one of the original descriptions for how XPath would apply to JSON.

See How to use JSON Path for examples of how to use jpp(1) (see jsonpath-cli).

This issue was inspired by the obsolete issue #523 from the mkiocccentry repo.

Anything else?

We will fill out this section as the above command line model is improved.

Stay tuned for more updates!

[xexyl]

Thanks! I will look at this in more detail later on today.

I have some other things I have to do and then I will look at the other issues and then this before I take care of some other things.

I know you are contemplating other tools but do we want the others from before so we have all three: jval, jnamval and jfmt ?

[xexyl]

Ah: I see! That other issue suggests that some of those tools might not be needed.

Okay well we will discuss these things before we work on the actual tools.

Back later.

[xexyl]

Oh as far as XPath and XML: I have used XML extensively before. I used the library libxml2 though I know that nowadays some of the functions I used back then are considered deprecated.

Well anyway I have a number of things to do before I can get to this issue but I think it's great so far.

[xexyl]

BTW: please feel free to open a pull request with the updates to the JSON util README file.

[xexyl]

Please remind me about this if I do not get to it sometime soon: say in the next day or at most two days. I need to take care of some other things first and depending on how long that takes and how I feel I might not be able to address this (maybe I'll be able to look at the prep.sh script in the mkiocccentry repo or this one too, for instance).

Do you wish to be assigned this too? I figure we probably will both work on this tool but it doesn't matter to me if you're assigned or not: I leave that up to you. I'll assign it to me in any case and then have things I have to do I'm afraid.

[lcn2]

Do you wish to be assigned this too?

While we are happy to develop the concept (and more is needed), we can leave the coding to you.

UPDATE 0

And will be happy to advise and test.

[xexyl]

Do you wish to be assigned this too?

While we are happy to develop the concept (and more is needed), we can leave the coding to you.

UPDATE 0

And will be happy to advise and test.

Sure. But if you want at some point or it seems necessary (for some reason that's unforeseen at this time) you can help out (of course). Even so regardless of that I'll put in the top comments the usual about the parser and the tools, the co-developed part, as it is a joint effort.

But yes please do further the concept and then later on (after IOCCC28 I guess) we can start to work on this and the other tool.

Which reminds me. Do you want to bring in jprint(1) here too or keep each tool to its own issue?

[lcn2]

Which reminds me. Do you want to bring in jprint(1) here too or keep each tool to its own issue?

Our guess is that jprint(1) will just be code that calls the jparse function and then walks the parse tree printing stuff.

This is probably rather different than jval(1) which needs to do xpath for JSON work. As such it is likely to have a rather different approach.

Thus, we advise to keep these issues separate.

[xexyl]

Which reminds me. Do you want to bring in jprint(1) here too or keep each tool to its own issue?

Our guess is that jprint(1) will just be code that calls the jparse function and then walks the parse tree printing stuff.

This is probably rather different than jval(1) which needs to do xpath for JSON work. As such it is likely to have a rather different approach.

Thus, we advise to keep these issues separate.

Just wanted to be sure. I don't mind either way.

[lcn2]

We recommend this JSONPath Outline Evaluator as a tool to evaluate/test/try the XPath for JSON.

[xexyl]

We recommend this JSONPath Outline Evaluator as a tool to evaluate/test/try the XPath for JSON.

I guess we'll be talking about this in more details later on but it's nice to have a tool to evaluate things too. Thanks.