Future of HTML Interface

[ebolyen]

Hey All,

Let's start the discussion of where the HTML Interface should go, and what it is going to be used for. Where do we draw the line of functionality and how much overlap is acceptable between QiiTa if any?

https://github.com/qiime/QiiTa

[josenavas]

My $0.02: I think that QiiTa will be able to take advantage of the functionality that pyqi will have already implemented. However, I think that the goals will be different: while piqy's HTML interface aims for a standalone HTML interface, in the case of QiiTa we need a interface that can be embedded in a more complex one. I don't know how this will impact the future of the HTML interface, but I think this is something that has to be cleared up from the very beginning, so we can re-design the HTML interface in a way that both projects can take advantage.

@squirrelo, your input here will be really useful, since you are the one that have design the user interface of QiiTa-pet. Having a list of what you really need to include in your interface can highly derive the design.

[antgonza]

I think the overlap is on how the output of the workflows will be shown but
nothing else, right? So if that is the case the only thing we need is to
have pyqi/Qiita be aware of the session variables (or whatever we use to
control sessions). In fact, it will be great if there were an easy to plug
output from pyqi. The rest of the Qiita HTML/web-services shouldn't change.

[josenavas]

@antgonza I don't get your point of pyqi/Qiita be aware of session variables. Can you elaborate?

[antgonza]

Basically, you will only display the results from userA to userA and userB
shouldn't be aware of them, right? So for that to happen the pages should
be "aware" of the session and who created who. Note that this doesn't mean
databases or whatever, simply using a validate display function that
returns a boolean that can be plugged to any other code should suffice.

[squirrelo]

From my current working with the pyqi web interface, it seems there are two fundamentally different goals going on here: a standalone interface built into pyqi and the ability to generate html pages from pyqi that can be used for another webserver.
In an ideal setup, the pyqi script would generate a page that contains form inputs for all options, automatically set to default values if available. I don't know if this is currently possible, but would be nice for the future.

[josenavas]

@antgonza I see your point but I think this is something that Qiita-pet has to take care of, and pyqi shouldn't know anything about it. IMO, Qiita will take only advantage of the HTML forms generation, as pointed by @squirrelo

Being able to show the results directly on Qiita will depend on the output of the executed command. If it already output an html page (as emperor) it'll be trivial. Other kind of outputs will need a bit of work.

[antgonza]

That's fine so really what pyqi needs to create is something similar
to what core_diversity_analyses.py does now, right?

[josenavas]

For the output I think that'd work.

I was more concerned about the parameter input for the commands, which I think its the challenging part of Qiita-pet and where it can really take advantage of the HTML interface.

[squirrelo]

That exactly. With this ideal setup I can just regenerate the options page any time the backend pyqi stuff is changed with little fuss.

[josenavas]

Well, the key thing of pyqi is that it generates the HTML on the fly, so you won't need to regenerate anything. You'll be always requesting them to pyqi.

[squirrelo]

I keep forgetting that. Even better.
How much overhead does that add though?

[josenavas]

I think that the overhead will be negligible. I was able to generate the XML pages for Galaxy (which essentially performs the same steps) in ms for all QIIME scripts.

[ebolyen]

So from what I am gathering, QiiTa has some use for generating input forms from a pyqi config. However, I am not clear on how QiiTa intends to 'intercept' the values from pyqi, as pyqi also generally acts as a driver for the commands.

Just so we are clear, any current implementation of the HTML Interface isn't particularly valuable to reference. Because as it stands, it cannot survive a router between host and browser. At the very least, it needs a way to handle a job queue and provide data on when jobs are finished.

In my mind, I imagine the ultimate form of the HTML Interface would look like a visual drag and drop unix shell, where commands can be sequenced to run. This would require either dependency injection or native support for the following:

• Storage for execution state
• Parallelization of an Execution Context
• A data-service, rather than direct html as output

Now, this sounds a lot like what QiiTa attempts to accomplish, so lets suppose that this is an unreasonable goal for the HTML Interface

Instead we might choose to allow the HTML Interface to exist as a 'simple' version in which there is no sequencing of commands. This would still require:

• Storage of execution state
• A concept of Execution Context

Neither of these things make a great deal of sense in the context of PyQi unless we chose to allow PyQi to fundamentally handle parallelization in which case, the ultimate form is not necessarily any more difficult.

Assuming we decide that it is not PyQi's job to care about where or how commands are executed, then we are left with an HTML Interface which is similar to what we have now. Additions that make sense would be:

• Templating Support

It should probably be noted that if execution state cannot be stored, then it will never be possible for the HTML Interface to operate beyond localhost except for trivial operations as a timeout delay would occur, or if heartbeats were sent, the user will invariable close the browser anyways.

[josenavas]

For Qiita, I think that the only functionality that we really need is a 'make_html' command, in which we pass the command that we are looking for and it returns the HTML interface. Then Qiita will call the commands directly, providing the inputs as needed.

AFAIK, one of the goals of pyqi is to provide a standard API, which is encoded under the commands folder. That means that we don't really need to use pyqi to call them. Is that right or I'm misunderstanding something?