Handling events with UID not working

[masscream]

Hello, I'm trying to handle with some events from my Calendar. I'm using DavMail gateway 6.5.1-3873 on Debian 13, pulling my Office 365 mailbox/cal. I usually pull the desired events with IDs, but when I try to make any operation with a specific event based on uid, I get this error.

iamuser@linuxvm:~/Applications/plann/bin$ ./plann select --event --uid="040000008200E00074C5B7101A82E0080000000090B535C6607ADC01000000000000000010000000146EF6E628784548917BDC1C738ED418" list
Traceback (most recent call last):
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/caldav/search.py", line 479, in search
    (response, objects) = calendar._request_report_build_resultlist(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        xml, self.comp_class, props=props
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/caldav/collection.py", line 784, in _request_report_build_resultlist
    response = self._query(xml, 1, "report")
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/caldav/davobject.py", line 239, in _query
    raise error.exception_by_method[query_method](errmsg(ret))
caldav.lib.error.ReportError: ReportError at '503 Service Unavailable

Invalid Caldav request: Unexpected character 'V' (code 86) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]', reason no reason

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iamuser/Applications/plann/bin/./plann", line 33, in <module>
    sys.exit(load_entry_point('plann==0.0.0', 'console_scripts', 'plann')())
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1485, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1406, in main
    rv = self.invoke(ctx)
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1873, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1870, in invoke
    super().invoke(ctx)
    ~~~~~~~~~~~~~~^^^^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1269, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/core.py", line 824, in invoke
    return callback(*args, **kwargs)
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/plann/cli.py", line 189, in select
    return _select(*largs, **kwargs)
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/plann/commands.py", line 44, in _select
    __select(ctx, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/plann/commands.py", line 101, in __select
    objs.append(c.object_by_uid(uid_, comp_filter=comp_filter))
                ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/caldav/collection.py", line 1079, in object_by_uid
    items_found = searcher.search(
        self, xml=comp_filter, _hacks="insist", post_filter=True
    )
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/caldav/search.py", line 491, in search
    return self._search_with_comptypes(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        calendar,
        ^^^^^^^^^
    ...<5 lines>...
        _hacks,
        ^^^^^^^
    )
    ^
  File "/home/iamuser/Applications/plann/lib/python3.13/site-packages/caldav/search.py", line 199, in _search_with_comptypes
    raise NotImplementedError(
        "full xml given, and it has to be patched to include comp_type"
    )
NotImplementedError: full xml given, and it has to be patched to include comp_type
iamuser@linuxvm:~/Applications/plann/bin$ python3 --version
Python 3.13.5

From DavMail, I get this in the log:

2026-01-30 14:43:25,086 ERROR [CaldavConnection-33550] davmail  - Invalid Caldav request: Unexpected character 'V' (code 86) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]
davmail.exception.DavMailException: Invalid Caldav request: Unexpected character 'V' (code 86) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]
	at davmail.caldav.CaldavConnection$CaldavRequest.parseXmlBody(CaldavConnection.java:1590)
	at davmail.caldav.CaldavConnection$CaldavRequest.<init>(CaldavConnection.java:1395)
	at davmail.caldav.CaldavConnection.handleRequest(CaldavConnection.java:224)
	at davmail.caldav.CaldavConnection.run(CaldavConnection.java:180)

Thank you for any hints.

[tobixen]

That's bizarre ... and too little information to understand what could be wrong.

If you set an environmental variable PYTHON_CALDAV_COMMDUMP=1 then the caldav library will dump the communication under /tmp/

[masscream]

Received 3 logs from 1 attempt

================================================================================
2026-03-11T11:46:14
====>
PROPFIND http://localhost:1080/users/user@domain/calendar/
Depth: 0

<?xml version='1.0' encoding='utf-8'?>
<D:propfind xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <D:current-user-principal/>
  </D:prop>
</D:propfind>
<====
207 Multi-StatusServer: DavMail Gateway 6.5.1-3873
DAV: 1, calendar-access, calendar-auto-schedule, calendarserver-private-events, addressbook
Date: Wed, 11 Mar 2026 10:46:14 +0000
Expires: Wed, 11 Mar 2026 10:46:14 +0000
Cache-Control: private, max-age=0
Transfer-Encoding: chunked
Content-Type: text/xml;charset=UTF-8
Connection: keep-alive

<D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:E="urn:ietf:params:xml:ns:carddav">
  <D:response>
    <D:href>/users/user@domain/calendar/</D:href>
    <D:propstat>
      <D:prop/>
      <D:status>HTTP/1.1 200 OK</D:status>
    </D:propstat>
  </D:response>
</D:multistatus>

================================================================================
2026-03-11T11:46:15
====>
PROPFIND http://localhost:1080/users/user@domain/calendar/
Depth: 0

<?xml version='1.0' encoding='utf-8'?>
<D:propfind xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-home-set/>
  </D:prop>
</D:propfind>
<====
207 Multi-StatusServer: DavMail Gateway 6.5.1-3873
DAV: 1, calendar-access, calendar-auto-schedule, calendarserver-private-events, addressbook
Date: Wed, 11 Mar 2026 10:46:15 +0000
Expires: Wed, 11 Mar 2026 10:46:15 +0000
Cache-Control: private, max-age=0
Transfer-Encoding: chunked
Content-Type: text/xml;charset=UTF-8
Connection: keep-alive

<D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:E="urn:ietf:params:xml:ns:carddav">
  <D:response>
    <D:href>/users/user@domain/calendar/</D:href>
    <D:propstat>
      <D:prop/>
      <D:status>HTTP/1.1 200 OK</D:status>
    </D:propstat>
  </D:response>
</D:multistatus>

Largest

caldavcommyihx96er.log

[tobixen]

Received 3 logs from 1 attempt

================================================================================
<D:propfind (...) <D:current-user-principal/>
(...)
<D:status>HTTP/1.1 200 OK</D:status>

This indicates that current-user-principal works, it's a bit strange that there was a warning in the other issue about it not working. Hm.

[tobixen]

So the URL for the principal (user) and "calendar home set" is the same. The third is apparently a list of all calendars it can find (one with the name "Calendar" and lots of them with junk names). The search itself seems to be missing.

The NotImplementedError exists in the master branch of caldav too, and the error is thrown before communication is done with the caldav server - so it is actually a bug in the caldav library. I will look into it, but it may take some few days

[masscream]

Sorry, I had the old version, I updated my python env libraries and got a new errors...

user@server:~/Applications/plann/bin$ pip list
Package               Version
--------------------- -----------
caldav                3.0.1
caldav-server-tester  0.1.0
charset-normalizer    3.4.5
click                 8.3.1
dnspython             2.8.0
h11                   0.16.0
icalendar             7.0.3
icalendar-searcher    1.0.5
jh2                   5.0.10
lxml                  6.0.2
niquests              3.18.1
pip                   26.0.1
plann                 1.0.0
python-dateutil       2.9.0.post0
PyYAML                6.0.3
qh3                   1.6.0
recurring-ical-events 3.8.1
setuptools            82.0.1
six                   1.17.0
sortedcontainers      2.4.0
tzdata                2025.3
tzlocal               5.3.1
urllib3-future        2.17.901
wassima               2.0.5
x-wr-timezone         2.0.1

user@server:~/Applications/plann/bin$ ./plann select --event --uid="040000008200E00074C5B7101A82E008000000004C685F4221F5D701000000000000000010000000D44994B661243343AFC991F8123019C9" list --template '{UID} {SUMMARY}'
Traceback (most recent call last):
  File "/home/user/Applications/plann/bin/./plann", line 6, in <module>
    sys.exit(cli())
             ~~~^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1485, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1406, in main
    rv = self.invoke(ctx)
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1873, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1870, in invoke
    super().invoke(ctx)
    ~~~~~~~~~~~~~~^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/core.py", line 1269, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/core.py", line 824, in invoke
    return callback(*args, **kwargs)
  File "/home/user/Applications/plann/lib/python3.13/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/user/Applications/plann/lib/python3.13/site-packages/plann/cli.py", line 173, in select
    return _select(*largs, **kwargs)
  File "/home/user/Applications/plann/lib/python3.13/site-packages/plann/commands.py", line 20, in _select
    __select(ctx, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/plann/commands.py", line 77, in __select
    objs.append(c.object_by_uid(uid_, comp_filter=comp_filter))
                ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/caldav/collection.py", line 1549, in object_by_uid
    return self.get_object_by_uid(*largs, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/caldav/collection.py", line 1504, in get_object_by_uid
    items_found = self.search(
        uid=uid, comp_class=comp_class, xml=comp_filter, post_filter=True, _hacks="insist"
    )
  File "/home/user/Applications/plann/lib/python3.13/site-packages/caldav/collection.py", line 1379, in search
    return my_searcher.search(
           ~~~~~~~~~~~~~~~~~~^
        self, server_expand, split_expanded, props, xml, post_filter, _hacks
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/user/Applications/plann/lib/python3.13/site-packages/caldav/search.py", line 666, in search
    result = cal._request_report_build_resultlist(xm, comp_cls, props=prp)
  File "/home/user/Applications/plann/lib/python3.13/site-packages/caldav/collection.py", line 1113, in _request_report_build_resultlist
    response = self._query(xml, 1, "report")
  File "/home/user/Applications/plann/lib/python3.13/site-packages/caldav/davobject.py", line 246, in _query
    raise error.exception_by_method[query_method](errmsg(ret))
caldav.lib.error.ReportError: ReportError at '503 Service Unavailable

Invalid Caldav request: Unexpected character 'V' (code 86) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]', reason no reason

2026-03-11 12:32:18,293 ERROR [CaldavConnection-48742] davmail  - Invalid Caldav request: Unexpected character 'V' (code 86) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]
davmail.exception.DavMailException: Invalid Caldav request: Unexpected character 'V' (code 86) in prolog; expected '<'
 at [row,col {unknown-source}]: [1,1]
	at davmail.caldav.CaldavConnection$CaldavRequest.parseXmlBody(CaldavConnection.java:1590)
	at davmail.caldav.CaldavConnection$CaldavRequest.<init>(CaldavConnection.java:1395)
	at davmail.caldav.CaldavConnection.handleRequest(CaldavConnection.java:224)
	at davmail.caldav.CaldavConnection.run(CaldavConnection.java:180) 

[tobixen]

A 503. Hm. Sometimes a 503 will disappear by simply trying again - and if not, it would be nice to see the communication.

[masscream]

It's not going away by re-trying. And I can't make the caldav lib vomit the logs again, did they change something with the new version?

PYTHON_CALDAV_COMMDUMP=1 ./plann ........

Trying by above or the export PYTHON_CALDAV_COMMDUMP=1 command, but it's not working.

[tobixen]

Indeed, the functionality has been accidentally removed. I'll fix it ... but I can't guarantee it will be today.

[masscream]

np, take your time.

[tobixen]

caldav 3.0.2 was released now, the debug hook for dumping the communcation is reintroduced.

caldav-server-tester 1.0.0 was released now.

perhaps I'll get around to release a new version of plann tomorrow.