Error "Tried to query ’:closed’ for non-existent ’:closed’" with recent org-ml

[MackieLoeffel]

Calling org-sql-user-push with a recent org-ml leads to the error: "Tried to query ’:closed’ for non-existent ’:closed’". I tried debugging it a bit but did not get very far. It seems like something in the output of org-ml-headline-get-planning changed such that it does not return a node, but the contents of a node and thus cannot be passed to org-ml-get-property?

My versions are:
org-sql: 4.0.0
org-ml: 6.0.2
org: Org mode version 9.8-pre (release_9.7.29-363-g3f1ea8 @ ...)

Here is the output of the debugger:

Debugger entered--Lisp error: (arg-type-error "Tried to query ’:closed’ for non-existent ’:closed’")
  org-ml--arg-error("Tried to query '%s' for non-existent '%s'" :closed :closed)
  org-ml--get-property-decoder(:closed :closed)
  org-ml-get-property(:closed (:closed nil :scheduled (2025 5 13 nil nil) :deadline nil))
  #f(compiled-function (acc type) #<bytecode 0x1064871aff7bbd15>)((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) :closed)
  -reduce-from(#f(compiled-function (acc type) #<bytecode 0x1064871aff7bbd15>) (:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) (:closed :deadline :scheduled))
  org-sql--insert-alist-add-headline-planning((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) (:outline-hash "9121fd63206b83af2f70f784fa3fc19b" :lb-config ... :log-note-matcher ... :headline ... :parent-ids ...))
  org-sql--insert-alist-add-headline((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) 24 (:outline-hash "9121fd63206b83af2f70f784fa3fc19b" :lb-config ... :log-note-matcher ... :headline ... :parent-ids ...))
  #f(compiled-function (acc hstate index hl) #<bytecode -0x8b7a095adce2fe1>)((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) (:outline-hash "9121fd63206b83af2f70f784fa3fc19b" :lb-config ... :log-note-matcher ... :headline ... :parent-ids ...) 24 (headline ... ...))
  #f(compiled-function (acc hstate index hl) #<bytecode -0x8b7a095adce2fe1>)((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) (:outline-hash "9121fd63206b83af2f70f784fa3fc19b" :lb-config ... :log-note-matcher ... :headline ... :parent-ids ...) 0 (headline ... ... ... ... ... ... ... ... ... ... ... ... ... ...))
  #f(compiled-function (acc hstate index hl) #<bytecode -0x8b7a095adce2fe1>)((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) nil 3 (headline ... ... ... ... ... ... ... ... ... ... ... ... ... ...))
  org-sql--insert-alist-add-headlines((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) (:outline-hash "9121fd63206b83af2f70f784fa3fc19b" :paths-with-attributes ... :size 627625 :lines 11135 :top-section ... :headlines ... :lb-config ... ...))
  org-sql--outline-config-to-insert-alist((:inserts ... :headline-id 1079 :timestamp-id 431 :entry-id 1 :link-id 439 :property-id 9 :clock-id 206) (:outline-hash "9121fd63206b83af2f70f784fa3fc19b" :paths-with-attributes ... :size 627625 :lines 11135 :top-section ... :headlines ... :lb-config ... ...))

[jarofromel]

I think the main cause is here ndwarshuis/org-ml#46. There are breaking changes in org-mode>=9.7 and while org-ml was adapted to accomodate these changes by ndwarshuis in the version 6.0, org-sql is not adapted yet.

In my case org-sql worked flawlessly with org-mode 9.6 and org-ml 5.8.8.
But I like to have the latest org-mode (in version 9.7 with org-ml 5.8.8) and in this case the org-sql works partially. E.g main structures (files, headings and its metadata) have its data stored correctly but the heading content and derived elements (links, timestamps etc) not -> the relevant columns are empty. It is probably the effect of org-mode=9.7 change which causes elements to be deferred and lazily calculated on the request (that's my explanation).

[ndwarshuis]

@jarofromel Sorry for not replying sooner. I've been slowly working on this and its actually more complex than I thought it would be. There's lots of fixes scattered throughout org-sql that need to happen to make this work. The new org-element api is only part of the problem; I made lots of independent breaking improvements in org-ml along with the new org-element api adaptations to avoid two major version bumps.

[jarofromel]

I am happy to test once you say it is ready. Thanks a lot for your work.