`jhack tail` produces lots of "not found in history... simulating one" lines with Juju 4

[astrojuanlu]

Bug Description

Listening for events...ERROR:jhack./home/ubuntu/.local/share/uv/tools/jhack/lib/python3.11/site-packages/jhack/utils/tail_charms/core/processor.py:mysql-k8s/0:mysql_pebble_ready not found in history... simulating one.
ERROR:jhack./home/ubuntu/.local/share/uv/tools/jhack/lib/python3.11/site-packages/jhack/utils/tail_charms/core/processor.py:mysql-k8s/1:mysql_pebble_ready not found in history... simulating one.
ERROR:jhack./home/ubuntu/.local/share/uv/tools/jhack/lib/python3.11/site-packages/jhack/utils/tail_charms/core/processor.py:mysql-k8s/2:mysql_pebble_ready not found in history... simulating one.
ERROR:jhack./home/ubuntu/.local/share/uv/tools/jhack/lib/python3.11/site-packages/jhack/utils/tail_charms/core/processor.py:mysql-k8s/0:config_changed not found in history... simulating one.
ERROR:jhack./home/ubuntu/.local/share/uv/tools/jhack/lib/python3.11/site-packages/jhack/utils/tail_charms/core/processor.py:mysql-k8s/2:config_changed not found in history... simulating one.

To Reproduce

(TBC)

Environment

                                 juju info v0.1                                 
┌──────────────┬───────────────────────────────────────────────────────────────┐
│ jhack        │ 0.4.4.0.24                                                    │
│ python       │ 3.11.14 (/home/ubuntu/.local/share/uv/tools/jhack/bin/python) │
│ juju-* snaps │  juju │ 4.0.0 - 33191 (4.0/stable)                            │
│ microk8s     │ Not Installed.                                                │
│ lxd          │ 5.21.4 LTS                                                    │
│ multipass    │ Not Installed.                                                │
│ multipassd   │ Not Installed.                                                │
│ os           │ Ubuntu 24.04.3 LTS                                            │
│ kernel       │ Linux 6.8.0-88-generic x86_64                                 │
└──────────────┴───────────────────────────────────────────────────────────────┘

Relevant log output

(See above)

Additional context

No response

[PietroPasotti]

Can you give me more context about the situation in which this happens?
This is usually because you did jhack fire or some event has failed (dispatch exited nonzero), and jhack reads a log that says event foo has errored out but it can't find in its history a log line that tells us that a foo event was ever emitted in the first place. So it injects a 'fake' one. But that usually happens when the juju history is cropped at some point in time after the event was emitted, so the logline we're searching for is not in jhack's cached history.

2 things I can imagine might be happening:

• juju 4 changed the log format, so jhack's regexes fail to identify the 'event emitted' logline as such
• juju 4 is dropping some key logs

Can you attach (or pastebin) a full juju debug-log --replay? And verify that the same issue occurs when you do cat ./juju-debug.log | jhack tail? that's a good repro to start debugging from.

[astrojuanlu]

(Haven't forgotten about this but I'm juggling several things at the same time and I might need a bit more time to give a proper reproducer)