ics import: respect recurrence id

[ftrojahn]

Hello,
we maintain an instance with more than 500 domains. So, there are a lot of ics imports (see verdigado/calendarize_external).

During integration of calendarize we noticed

• each ics import must respect the pid, see my PR Restrict ics-import to pid per calendar #773 -
  otherwise, ics imports using the same source link "steal" events from each other
• the SEQUENCE key in ics files is not respected.

This leads to recurring events, which will be created (or old ones overwritten): but without the frequency ...
resulting in missing events.

Only workaround for now is, in the source ics calendar:
deleting the recurring event (which had a calendar exception) and recreating the recurring event from scratch,
which is not always possible (not all users own that calendar they export from).

I debugged it like this:

• have a Google calendar export with recurring events => usually works
• then, someone changes one event of the recurring series. In the ics export, you now have two events with the same UID
  and different SEQUENCE like 0 and 1.
• now, the next ics import runs like this:
  • the event with SEQUENCE:0 has the RRULE => frequency is set for the importid.
  • the event with SEQUENCE:1 has no RRULE, but same UID => frequency is unset.
  • we get in Log: Invalid frequency="{frequency}" in RRULE errors
• sometimes you see other events in the export having e.g. SEQUENCE:11, but no other event with the same UID / importId

IMHO we could fix this by attaching the sequence id to the importid like here:
master...verdigado:calendarize:add-sequenceid-to-importid
(no PR, just asking)

But this creates two other problems:

• existing, recurring events (remember: up to now, we did not yet add sequenceid to importId)
  will be reimported using importId+sequenceid
• other, (non recurring?) events which have a sequenceid will be reimported, too

That is due to: old imported events will not be deleted on ics import, if deleted in ics source. See #620 too.

So we would have to deal with finding all "old" importId events which must be deleted, if we have a new import with importId+sequenceid. ATM I see at least no slot/eventlistener where I can hook in to do this.

My test calendar link contains both of the above mentioned recurring events.

Any idea?
Thanx anyway for this great extension!
Falko

[okmiim]

Hi @ftrojahn,

one big painpoint with the ICS import is that is is only an "import" (add + update) and not really a "synchronization" (... + delete). I been thinking/planning (for a long time) to do a new implementation of the ICS handling, but did not get around to that.

The modification of specific instance of a recurring series in iCalendar is actually reflected by the RECURRENCE-ID and not the SEQUENCE. The RECURRENCE-ID value contains the date when original recurrence instance would occur, which is "overwritten".
The SEQUENCE number is just "sequence of revisions" and not (really) relevant for recurring series.

You could handle the import of specific recurring instances as follows:

1. The importId is a combination of the UID with the recurrence-id (if existing)
2. Normal import
3. Additional steps on events containing recurrence-id:
   • Search the base record (without recurrence-Id) and add an Configuration with handling=exclude with the recurrence date

A hacky way to get rid of non-existing ics-events:

1. Delete all events in a specific folder (e.g. cleanup command)
2. Do the import as normal
   This has a lot of overhead.

Regards