Awaitables

[fleming79]

This PR is a subset of #160

This PR enables asynchronous code execution as follows:

• Python methods are coroutines (async def)
• Each request sent to the frontend is assigned a Pending (just like an asyncio.Future). When the frontend replies, the pending is set done and the result is returned.
• A JSON representation is returned if the result can be converted to JSON.
• A Connection object is provided if the object returned in the frontend is a disposable (or when the transform is specified as a connection).
• The connection can be used to:
  • access the methods and properties
  • close the object
  • use the object as a reference in the frontend using 'toObject'
  • conveted to a lumino widget using 'toLuminoWidget'

Demo running all cells at once

Awaiting in each cell ensures the result is available for the next cell. The gif below is 0.5x playback speed.

Menus

This gif demonstrates the usage of menus and restoration when the page is refreshed.

Async execution is enabled in notebooks with async-kernel.

Subshells

Requires async-kernel

Leverages on the concept of kernel subshells to provide isolated user_ns and common global_user_ns.

resolves #11

Replaces #160

[github-actions]

👈 Try it on Binder (branch fleming79/ipylab/ipylab-awaitable)

[fleming79]

described_by was added in this commit. It provides a similar interface to the first part of #162 (printing args only).

This illustrates how awaitables can be used to add in more functionality.

[bollwyvl]

Yeh, the goal over on #162 was to add some more optional features without breaking the public APIs (even though that took a good deal of mangling). The execute(..., handler) is likely even more interesting than describe.

Until there is a proper upstream ipywidgets async (iterable) story, I think having additive, alternate APIs here would be nice, for sure.

[fleming79]

Yeh, the goal over on #162 was to add some more optional features without breaking the public APIs (even though that took a good deal of mangling). The execute(..., handler) is likely even more interesting than describe.

Until there is a proper upstream ipywidgets async (iterable) story, I think having additive, alternate APIs here would be nice, for sure.

Your comment "Until there is a proper upstream ipywidgets async" made me realize that the patched libraries aren't required for this PR; so I've removed them. At the moment it still requires the kernel to be async-kernel, which was developed specifically to make ipylab work nicely in notebooks.

In this PR:

• CommandRegistry.execute returns the result of execution in the frontend (or a Connection to an object).
• The 'described_by' feature (without validation) is configurable on the CommandConnection which gets returned when a command is created.
• Icons can be included in CommandRegistry.add_command
• Multiple command registries are supported; a new registry can be created with CommandRegistry(name="my new registry").

[fleming79]

I've been working on a Jupyterlite plugin for 'async-kernel' which now mostly working. For the moment the code is being hosted here: https://github.com/fleming79/echo-kernel.

The interface is simple in design and should be pretty robust. It uses a web worker to run pyodide which loads and runs the kernel. Messages are transferred using the builtin post functions and kernel messages are passed via a single callback function on each side, the messages are serialized to json for javascript <--> python conversion.

The kernel is started with a script which is configurable, by default script searches for wheels locally, and installs them prior to starting the kernel. This helps to make it easier to bundle wheels in jupyterlite.

There are a few associated PRs about interrupts, which is the main missing feature.

• Added interrupt_mode to the Kernel Spec interface jupyterlab/jupyterlab#18307
• Added interrupt_mode handling to LiteKernelClient.interrupt. jupyterlite/jupyterlite#1834

I've adapted ipylab readthedocs to use the new kernel. You can access the most recent docs here.