Refreshing tickets, cutoff date, logging updates, bug fixes #365

juanpablosalas · 2025-12-06T22:07:48Z

This PR introduces a series of changes and bug fixes mainly to the handling of tickets.

Tickets

Introduces cutoff_date for JIRA and Redmine tickets. If specified, A2rchi will pull only tickets created before that date. One possible issue with this is that the ticket changes after the creation date will still be fetched. (See Introduce a cutoff date for JIRA #362)
Introduces frequency for JIRA and Redmine tickets. If specified, A2rchi will pull tickets updated/created since the last collection date within that frequency (in days). This update is triggered by a call to ChatWrapper. If the frequency is set to 0 it will pull the tickets again on every call. (See Refresh JIRA tickets since last pull #359)

Logging

Reduced logging in persistence.py, vectorstore/manager.py. A list of stored resources was being logged as info but I find this to make a lot of noise (specially considering there are hundreds of tickets in each project, not to mention the crawled pages). I reduced the logging level to DEBUG so it's not completely gone.
Reduced logging for httpx module

Bug fixes/Code cleaning

Deletes utils section from base config template as it has been superseded by sources
Removes outdated utils_config
Fixed circular import issues in document index and incorrect display of JIRA tickets

haozturk

A bug identified, changes needed

src/interfaces/chat_app/app.py

src/data_manager/collectors/tickets/ticket_manager.py

pmlugato · 2025-12-08T17:13:39Z

I think we need to think a bit more about the document viewing/uploader. One reason to keep the account creation central was to control who was given the ability to add/delete documents (like you mentioned is not handled now). In general, I can think of a few permutations in which you might want the viewer to be accessible to everyone or only some, and then the same with actual uploading permissions (deploying to admins vs users, for example).

Is it a huge pain to move the account creation code to a different PR? Since ticket updating is more pressing, whereas the uploader is not, and can move this discussion/additional changes to the new PR? Let me know @juanpablosalas

With respect to the updating of the tickets, thanks a lot for getting this done quick, it is definitely essential for the deployment!

juanpablosalas · 2025-12-09T01:38:03Z

Hi @pmlugato, I agree with you! I think there are still some things we need to discuss regarding the document viewer so I switched those commits to #367

haozturk · 2025-12-09T14:42:09Z

Thanks Juan, my biggest concern with the current way of refreshing tickets is that update_tickets is a synchronous call which blocks the chat response until its termination. I think it's better to make it an async process that runs in the background. One way is to keep your implementation (trigger via chat call) and make it async or create a standalone service which periodically updates tickets. The second is a cleaner approach in my opinion and I don't know how much extra work it requires. What do you think? @juanpablosalas @pmlugato

haozturk · 2025-12-10T11:15:41Z

Getting the following error now @juanpablosalas

`(2025-12-10 11:10:23,531) [src.interfaces.chat_app.app] ERROR: Failed to update tickets - 'datetime.timedelta' object has no attribute 'minutes'`

Use .total_seconds() / 60 or seconds.
Also it'd be useful to add a debug logging line which tells if any ticket is updated, if so how many and so on.

src/data_manager/collectors/tickets/ticket_manager.py

src/cli/templates/base-config.yaml

src/interfaces/chat_app/app.py

haozturk · 2025-12-10T13:09:19Z

There's a bug and a various points to improve @juanpablosalas . Let me know when they are addressed and I'll review again

juanpablosalas · 2025-12-10T19:18:36Z

Thanks for the thorough feedback @haozturk , I pushed the changes

src/data_manager/collectors/tickets/integrations/jira.py

src/data_manager/collectors/tickets/ticket_manager.py

pmlugato · 2025-12-12T14:42:18Z

src/interfaces/chat_app/app.py

+        try:
+            self.data_manager.update_tickets()
+        except Exception as e:
+            #NOTE: Error is logged but a failure to update tickets does not necessarily mean A2rchi cannot process and respond to the message
+            logger.error(f"Failed to update tickets - {str(e)}")
+


do we want to have an indicator should the ticket updating go wrong? for example if someone asks about a new ticket that came in but archi fails to grab it, to let the user know this so they know it's not an issue of retrieval/query or something? e.g., "failed to grab tickets since x time" or "last ticket update at x time"... what do you think? @juanpablosalas @haozturk

It's a good idea to show last data update, but without communicating what data we're talking about, i don't think it would be useful. Tickets aren't the only data we have, so ideally we want to show the last doc update as well. I wouldn't include this feature in this PR, but rather make an issue and tackle it later. I will let Juan comment as well. If he agrees, we should make an issue for this.

pmlugato

@juanpablosalas thanks a lot for the good work and the quick iteration with @haozturk . I've left a small comment which I'll leave up to you both as operators whether it would be useful to include or not. Other, than that, the rest looks great to me, Hasan feel free to merge when you are ready!

haozturk · 2025-12-15T09:37:34Z

src/data_manager/collectors/tickets/integrations/jira.py

            return

        self.client = client
+        self.timezone = None


It should be self.jira_timezone = None, o/w it fails :(

(2025-12-15 09:29:40,199) [src.interfaces.chat_app.app] ERROR: Failed to update tickets - 'JiraClient' object has no attribute 'jira_timezone'

Done and ran it again with no issues

haozturk · 2025-12-16T07:21:45Z

Thanks @juanpablosalas can you please resolve the conflicts that arose after #354 is merged

config changes

haozturk · 2025-12-18T14:49:26Z

Tested locally, for some reason refreshing capability doesn't work anymore. Not ready to be merged. Discussing with Juan what's wrong.

added documentation

fixed bug when no frequency is passed in the config always update fixed when jira frequency is 0 undo document index changes

fixed requested changes improved debug message and timezone errors fixed jira created and updated dates

fixed jira_timezone declaration fix bug fixed issues when rebasing removed utils again re added changes

lucalavezzo · 2025-12-19T16:49:15Z

Hi @juanpablosalas I am afraid that my bad coding practices means there are going to be a lot of conflicts with my PR #377. Do you mind if we close this, and I will merge the relevant changes into #377?

juanpablosalas · 2025-12-19T17:07:35Z

Not at all @lucalavezzo, in any case I was aware the changes will need to be reconciled given the introduction of the cronjob infrastructure you are developing!

haozturk · 2025-12-19T18:21:25Z

I was about to test this and merge if it was okay. Should I not touch it then?

lucalavezzo · 2025-12-19T18:23:32Z

Hi @haozturk, if it's okay, I am working on harmonizing Juan Pablo's work in the updated Data Manager I am developing in #377: it would be easier to avoid merging this to avoid explicit conflicts.

haozturk · 2025-12-19T18:30:20Z

Hi @lucalavezzo ofc, please let me know when yours is ready for a review. I'm curious to check it.

juanpablosalas changed the title ~~Feature/refresh jira tickets~~ Refreshing tickets, cutoff date, document index account creation, logging updates, bug fixes Dec 6, 2025

juanpablosalas requested review from haozturk and pmlugato December 6, 2025 22:45

juanpablosalas marked this pull request as ready for review December 6, 2025 22:45

This was linked to issues Dec 8, 2025

Introduce a cutoff date for JIRA #362

Open

Refresh JIRA tickets since last pull #359

Open

handle accounts for document uploader and revisit obsolete secrets #329

Open

haozturk requested changes Dec 8, 2025

View reviewed changes

src/interfaces/chat_app/app.py Outdated Show resolved Hide resolved

pmlugato reviewed Dec 8, 2025

View reviewed changes

src/data_manager/collectors/tickets/ticket_manager.py Show resolved Hide resolved

juanpablosalas changed the title ~~Refreshing tickets, cutoff date, document index account creation, logging updates, bug fixes~~ Refreshing tickets, cutoff date, logging updates, bug fixes Dec 8, 2025

juanpablosalas force-pushed the feature/refresh_jira_tickets branch from 1a81925 to 0ee17ec Compare December 9, 2025 01:30

haozturk reviewed Dec 10, 2025

View reviewed changes

src/data_manager/collectors/tickets/ticket_manager.py Outdated Show resolved Hide resolved

haozturk reviewed Dec 10, 2025

View reviewed changes

src/data_manager/collectors/tickets/ticket_manager.py Show resolved Hide resolved

haozturk reviewed Dec 10, 2025

View reviewed changes

src/cli/templates/base-config.yaml Outdated Show resolved Hide resolved

haozturk reviewed Dec 10, 2025

View reviewed changes

src/interfaces/chat_app/app.py Outdated Show resolved Hide resolved

haozturk reviewed Dec 12, 2025

View reviewed changes

src/data_manager/collectors/tickets/integrations/jira.py Outdated Show resolved Hide resolved

haozturk reviewed Dec 12, 2025

View reviewed changes

src/data_manager/collectors/tickets/ticket_manager.py Outdated Show resolved Hide resolved

juanpablosalas requested a review from pmlugato December 12, 2025 14:00

pmlugato reviewed Dec 12, 2025

View reviewed changes

pmlugato approved these changes Dec 12, 2025

View reviewed changes

haozturk reviewed Dec 15, 2025

View reviewed changes

Juan Pablo Salas Galindo added 2 commits December 16, 2025 08:03

initial changes

bf6b661

config changes

remove outdated utils configuration

3939b73

Juan Pablo Salas Galindo added 5 commits December 16, 2025 08:03

working version of refreshing tickets on chat call

51300af

inclusion of cutoff date for jira

817345e

reduced logging for persistence and manager to DEBUG level

b5af2e6

bug fix caused by chat dockerfile

0d3373c

included create account endpoint

bb9deff

juanpablosalas force-pushed the feature/refresh_jira_tickets branch 2 times, most recently from 4bb031a to de5cf66 Compare December 16, 2025 16:33

juanpablosalas force-pushed the feature/refresh_jira_tickets branch 2 times, most recently from 11eb37d to a1e03fa Compare December 19, 2025 14:37

Juan Pablo Salas Galindo and others added 5 commits December 19, 2025 09:48

added redmine cutoff date

9dadf24

added documentation

comments before cutoff date are not included now

2291c39

fixed bug when no frequency is passed in the config always update fixed when jira frequency is 0 undo document index changes

separate try-except for ticket updates

96e6a69

switched frequency unit to minutes

e52a740

fixed requested changes improved debug message and timezone errors fixed jira created and updated dates

added Hasan's changes

f6a7e85

fixed jira_timezone declaration fix bug fixed issues when rebasing removed utils again re added changes

juanpablosalas force-pushed the feature/refresh_jira_tickets branch from a1e03fa to f6a7e85 Compare December 19, 2025 14:49

Refreshing tickets, cutoff date, logging updates, bug fixes #365

Are you sure you want to change the base?

Refreshing tickets, cutoff date, logging updates, bug fixes #365

Conversation

juanpablosalas commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haozturk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pmlugato commented Dec 8, 2025

Uh oh!

juanpablosalas commented Dec 9, 2025

Uh oh!

haozturk commented Dec 9, 2025

Uh oh!

haozturk commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

haozturk commented Dec 10, 2025

Uh oh!

juanpablosalas commented Dec 10, 2025

Uh oh!

Uh oh!

Uh oh!

pmlugato Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

haozturk Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

pmlugato left a comment

Choose a reason for hiding this comment

Uh oh!

haozturk Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

juanpablosalas Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

haozturk commented Dec 16, 2025

Uh oh!

haozturk commented Dec 18, 2025

Uh oh!

lucalavezzo commented Dec 19, 2025

Uh oh!

juanpablosalas commented Dec 19, 2025

Uh oh!

haozturk commented Dec 19, 2025

Uh oh!

lucalavezzo commented Dec 19, 2025

Uh oh!

haozturk commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

juanpablosalas commented Dec 6, 2025 •

edited

Loading

haozturk commented Dec 10, 2025 •

edited

Loading