Skip to content

Conversation

@AshokThangavel
Copy link
Contributor

Feature: Implement Automatic History Purge

Overview

This PR introduces an automated history retention policy in IPM. It allows users to configure how many days of history should be kept, preventing the history table from growing.

fixes #977

Key Changes

  • New Configuration Key: Added HistoryRetain to the IPM configuration.
    • config set HistoryRetain 10: Retains the last 10 days of history.
    • config set HistoryRetain 0: Disables the auto-purge.

Implemented AutoPurgeOldHistory. This method checks for and removes records older than the configured threshold.

  • No Task Manager Dependency: To keep IPM easy to manage, It integrated the purge directly into the shell lifecycle rather than relying on the Task Manager.

CLI Usage

Users can manage the retention policy directly via the IPM shell:

  • Set Policy: zpm:USER> config set HistoryRetain 10
  • Check Policy: zpm:USER> config get HistoryRetain
  • Disable Policy: zpm:USER> config set HistoryRetain 0

Testing & Validation

Unit tests have been implemented to verify the full lifecycle of the configuration and the purge engine:

ID Action Status Description
1 LogMessage Passed Run command: config set HistoryRetain 10
2 LogMessage Passed Run command: config get HistoryRetain
3 LogMessage Passed Successfully updated retention from 10 days to 1 day
4 LogMessage Passed Run command: config set HistoryRetain 1
5 LogMessage Passed Verified purge timestamp: 67553
6 LogMessage Passed Reset HistoryRetain to default by deleting config
7 LogMessage Passed Run command: config delete HistoryRetain
8 Performance Passed Execution duration: 0.011236 sec.

@isc-tleavitt
Copy link
Contributor

@AshokThangavel - thank you again!

@isc-kiyer @isc-dchui I'm curious if you think this meets the expectations from the conversation behind #977 - the ideal would be a scheduled task, but I think this approach is also valid. (Since you're not going to get history without running zpm commands.)

Personally, I strongly believe we should maintain all history by default and have purge be opt-in (as this does). Most of the time it will be very lightweight unless there is a ton of activity on the system - and if there is a ton of activity, the history becomes more essential/valuable. For long-running production systems that may or may not have good change management procedures, the history could be a lifeline for understanding when/why something broke.

Copy link
Contributor

@isc-tleavitt isc-tleavitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits + needs a changelog. Also needs confirmation from other devs who were deeper in discussion on history - but I like this approach, FWIW.

}

/// Automatically purge old history based on retention settings
ClassMethod AutPurgeOldHistory()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: typo, should be AutoPurgeOldHistory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!. I've fixed the typo in the method

quit
}
set retainDays = ##class(%IPM.Repo.UniversalSettings).GetHistoryRetain()
quit:(+retainDays <=0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General code style comment:
We generally dislike postconditionals for code readability. I'd personally make an exception for:

  • Loop exit conditions with simple tests (e.g., quit:key="" or quit:AtEnd)
  • Write:verbose !,"Not worth putting this in it's own if block"

Adding this to my list of things to formalize in https://github.com/intersystems/ipm/blob/main/CONTRIBUTING.md.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!. I've modified the post condition

@AshokThangavel
Copy link
Contributor Author

Hi @isc-tleavitt
Thanks for the feedback! My initial thought was indeed to use a scheduled task. However, I decided on this approach to avoid the overhead of explicit task management during the package lifecycle. Since the purge workload is minimal, implementing it within the command ensures the system remains clean without extra maintenance.

I also completely agree with your point on making this opt-in. It ensured the default behavior remains unchanged so history is preserved unless a user explicitly configures HistoryRetain. This keeps that 'lifeline' intact for production systems.
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

History Log time purging

2 participants