[DOXIA-767] Clarify block vs inline semantics with regards to whitespaces in Sink API

[jira-importer]

Konrad Windszus opened DOXIA-767 and commented

The javadoc of https://maven.apache.org/doxia/doxia/doxia-sink-api/apidocs/org/apache/maven/doxia/sink/Sink.html should be clarified with regards to trimming whitespace behaviour for block/inline elements. Not necessarily each block Sink element is a block element in the desired output format (e.g. XHTML). Most prominently this is the case with

Sink.verbatim(Decoration:source) which is a block element in Sink API but opens an inline element (<pre><code>) in XHTML.

See https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Whitespace.

---

Issue Links:

• DOXIA-764 XDoc source has a spurious new line at the beginning

[jira-importer]

Konrad Windszus commented

Compare with https://www.w3.org/TR/2000/WD-xhtml-modularization-20000105/conformance.html

In elements where the 'xml:space' attribute is set to 'preserve', the user agent must leave all whitespace characters intact (with the exception of leading and trailing whitespace characters, which should be removed). Otherwise, whitespace is handled according to the following rules:

All whitespace surrounding block elements should be removed.
Comments are removed entirely and do not affect whitespace handling. One whitespace character on either side of a comment is treated as two white space characters.
Leading and trailing whitespace inside a block element must be removed.
Line feed characters within a block element must be converted into a space (except when the 'xml:space' attribute is set to 'preserve').
A sequence of white space characters must be reduced to a single space character (except when the 'xml:space' attribute is set to 'preserve').
With regard to rendition, the User Agent should render the content in a manner appropriate to the language in which the content is written. In languages whose primary script is Latinate, the ASCII space character is typically used to encode both grammatical word boundaries and typographic whitespace; in languages whose script is related to Nagari (e.g., Sanskrit, Thai, etc.), grammatical boundaries may be encoded using the ZW 'space' character, but will not typically be represented by typographic whitespace in rendered output; languages using Arabiform scripts may encode typographic whitespace using a space character, but may also use the ZW space character to delimit 'internal' grammatical boundaries (what look like words in Arabic to an English eye frequently encode several words, e.g. 'kitAbuhum' = 'kitAbu-hum' = 'book them' == their book); and languages in the Chinese script tradition typically neither encode such delimiters nor use typographic whitespace in this way.

[jira-importer]

Konrad Windszus commented

The above quote does no longer apply to HTML5. It really depends on the actual element, e.g. for <pre> only the leading newline is stripped the rest of whitespace characters is emitted as is.

[jira-importer]

Konrad Windszus commented

It seems also that newlines within Sink.text(String text) are not properly escaped in all Sinks. In order to make them appear as newlines in HTML one has to use <br> or backslash at the end of a line for Markdown, however this is not the case in

maven-doxia/doxia-core/src/main/java/org/apache/maven/doxia/sink/impl/Xhtml5BaseSink.java

Line 1460 in 99e7e1f

content(text);

. Similarly newlines in HTML text need to be converted to spaces in

maven-doxia/doxia-core/src/main/java/org/apache/maven/doxia/parser/AbstractXmlParser.java

Line 474 in 99e7e1f

protected String getText(XmlPullParser parser) {

(which isn't the case for XHTML at least).

Michael Osipov What do you think about cleanup here? Right now the semantics is incorrect or at least too fuzzy in the Sink API from my PoV...

[jira-importer]

Michael Osipov commented

Yep, the current behavior is wrong. Line breaks need to be handled according to the output specification. In other words: Make it consistent!

[jira-importer]

Konrad Windszus commented

The problem is that there is no Sink method for linebreaks within the Markup language of the Sink.
Once both Sink.text("\n") and Sink.lineBreak() (https://maven.apache.org/doxia/doxia/doxia-sink-api/apidocs/org/apache/maven/doxia/sink/Sink.html#lineBreak--) behave the same, there is no way to keep linebreaks in the source only (not having an effect on rendering result) when converting from e.g. APT to Markdown or rendering Markdown to HTML.

[jira-importer]

Michael Osipov commented

Then we need to look at case by case, I see no other way.