Assemblies: daps fails to detect undefined entities in tasks, references, concepts or glues

[taroth21]

Problem description

If I misspell an entity in the assembly file (*.asm), daps will detect the undefined entity and return an error message:

✔ /local/git/GH/doc-modular [taroth/test_entity L|✚ 2] 
17:19 $ daps -d DC-minimal-vm validate
/local/git/GH/doc-modular/articles/sle-minimal-vm.asm.xml:54: parser error : Entity 'taroth' not defined
              Complete revision incorporation feedback and learnings by &taroth;
                                                                               ^
Fatal error:
/local/git/GH/doc-modular/build/.assembly/minimal-vm.xml:
    CRITICAL: Entity 'taroth' not defined (line 16 column 81)
                                                                               ^
Fatal error:
/local/git/GH/doc-modular/build/.assembly/minimal-vm.xml:
    CRITICAL: Entity 'taroth' not defined (line 16 column 81)

However, if an entity is misspelled in a reference, concepts, task or glue topic, daps does not detect the undeclared entity and will (erroneously) report: Document is valid.

Expected behavior

daps should detect missing entity declarations anywhere within snippets used for the assembly.

[fsundermeyer]

@tomschr : Would you please have a look? This is a stylesheet issue. The assembly stylesheets silently remove any unkown entities in files included in the assembly.

[tomschr]

I did some research and I'm afraid that seems be a "problem" of the libxslt library...

The assembly upstream stylesheet loads the referenced XML files via the document() function. By design, its document() function can load external XML even if it has undefined entities. Rather than halting on unresolved entities, libxml2/libxslt silently ignores them and proceeds, which is why you're seeing the assemble process continue despite unknown entity references.

In pure specification terms, it may not violate XSLT 1.0 rules, but many users consider this behavior suboptimal—it hides entity-related issues.

There are some workaround which we could consider:

• Switch to a more rigorous XSLT processor
  Theoretically we could switch to Saxon >12. However, that would introduce XSLT 3.0 and I'm not completely sure how compatible is that to all existing stylesheets. That would require more testing. Additionally, we could face a potential performance issue as it's written in Java (although once it's loaded, it's quite fast).
  The benefit is that XSLT 3.0 allows workflows that weren't possible with XSLT 1.0.

• Use a hybrid well-formedness check before calling the assemble stylesheet
  If we want to stick with libxml2/libxslt, prior to calling the assemble process we could iterate through the XML files and check them with:

    $ xmllint --noout --nonet --noent XMLFILE
  

  Or use the daps-wellformed Python script. Regardless what you use, check the return code. If there are a problem, break. Otherwise proceed.
  Both commands can take more than one XML files. This makes it a bit easier to process all in one go.

=> Probably the best for the time being is to integrate option 2.

---

Apart from the above issues, libxml2/libxslt has a problem with maintenance. Although the current maintainer does a great job, it seems he's the only one doing that. That does not seem like a healthy project when all the work is on the shoulder of one person. Especially if the two libraries have complicated issues.

[fsundermeyer]

Probably the best for the time being is to integrate option 2.

Totally agree, but that will require to get a list of files that are "included" in an assembly. The files listed in the reources section are not necessarily "included". Is there a way to extract only the ones that are really used (they are referenced via ID in the structure section).
If that fails we could go with the full list of resources...

[tomschr]

Probably the best for the time being is to integrate option 2.

Totally agree, but that will require to get a list of files that are "included" in an assembly. The files listed in the resources section are not necessarily "included". Is there a way to extract only the ones that are really used (they are referenced via ID in the structure section). If that fails we could go with the full list of resources...

I think, this can be done with some XSLT magic. 😉 Should not be too complicated.