Replace old monitor workflow with sciline-based implementation #655
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement a sciline-based monitor workflow that runs in the data_reduction namespace alongside the existing monitor_data service workflow. This uses ess.reduce.streaming.StreamProcessor for accumulation. Key components: - WindowAccumulator: clears after each finalize via on_finalize() hook - histogram_monitor_data: handles both event-mode (binned events) and histogram-mode (already-histogrammed) monitor data - StreamProcessorWorkflow wrapper for sciline Pipeline integration - Auto-registration of monitor_view spec via Instrument.__post_init__ The workflow produces the same outputs as MonitorStreamProcessor: cumulative histogram, current window histogram, and ratemeter counts. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add time, start_time, end_time coords to monitor workflow outputs to match behavior of MonitorStreamProcessor. The StreamProcessorWorkflow now tracks the time range during accumulate() and adds coords to specified window outputs in finalize(). Window outputs (current, counts_total, counts_in_toa_range) get the coords; cumulative output does not since it spans all time. Prompt: The previous commit adds an alternative monitor workflow implementation. Please look into adding missing time/start_time/end_time coords to match behavior of the old MonitorStreamProcessor. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Match the title and description of the new data_reduction/monitor_view workflow spec with the existing monitor_data/monitor_histogram spec, since monitor_view is intended to eventually replace it. Also remove "V2" temporal reference from test docstring. --- Prompt: Please review changes since `add-start-end-time-coords`. Is everything in good shape and ready for a PR? Follow-up: Aside from what you found, I think we should also make the new workflow title and description match the old one, as we plan to eventually replace it? Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Strip the old MonitorStreamProcessor and MonitorHandlerFactory, move the new StreamProcessor-based monitor workflow from data_reduction namespace to monitor_data namespace, and adjust preprocessor setup accordingly. Changes: - Delete monitor_data_handler.py (contained only removed classes) - Rename create_monitor_view_workflow to create_monitor_workflow - Move monitor workflow spec registration from data_reduction to monitor_data namespace - Use ReductionHandlerFactory in monitor_data service instead of MonitorHandlerFactory - Remove CollectTOA accumulator (only used by removed MonitorHandlerFactory) - Update tests: move TestMonitorDataParams, update spec registration tests The new workflow uses ToNXevent_data preprocessor for events and Cumulative for histogram data, supporting both event-mode and histogram-mode monitor inputs. Integration tests confirm all monitor data service configurations work correctly. User request: (1) strip the old one and (2) make the new workflow part of the `monitor_data` namespace (and adjust preprocessor setup)
Re-export MonitorEvents from accumulators.py to maintain API compatibility with message_adapter.py which imports it from there. Fix test import in monitor_workflow_test.py that incorrectly used the sciline NewType TOAEdges instead of the Pydantic model from parameter_models. Prompt: Please review changes since `add-start-end-time-coords` - we have replaced a bespoke monitor workflow with the more generic approach based on StreamProcessorWorkflow. Is everything consistent and in good shape? Are tests adequate? Does the functionality look identical? Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: Needs update of
essreduce. Need to also think some more in #633.Summary
MonitorStreamProcessorwith the more genericStreamProcessorWorkflowwrapper around scilineToNXevent_datapreprocessor instead ofCollectTOAfor monitor events, producing standard binned scipp DataArraysWindowAccumulatorfor window outputs that clear after each finalize cycleMonitorEventsfrom accumulators.py)The new approach produces identical outputs (
cumulative,current,counts_total,counts_in_toa_range) with the same time coordinate handling.Test plan
🤖 Generated with Claude Code