Execute notebook programatically has different behavior from running it on a Jupyter Notebook UI

[noklam]

Context

I am creating a Jupyter magic that will create multiple cells. When I run it manually (i.e. Ctrl + Enter), I can create new cells successfully. When I run it programatically, the code get run without error but the cells are not created.

A gif to show how this magic should behave.

See: kedro-org/kedro#3510

I am trying to create automated test for this feature, thus it is important to execute this in an automated fashion. I wonder how this is tested

import nbformat
from nbconvert.preprocessors import ExecutePreprocessor

ep = ExecutePreprocessor(timeout=600, kernel_name="python3")
print(f"Executing Notebook {notebook_filename}")
try:
    # Execute the notebook from the same directory
    ep.preprocess(nb,
                  {'metadata': {'path': Path(notebook_filename).parent.resolve()}})

I wonder if you have any idea why it doesn't work, or you can give me some pointers to some relevant tests in this repository. Thank you in advance!

[SajidAlamQB]

Hey @noklam, this might be related: kedro-org/kedro#4369 (comment)