Add CJK-friendly Emphasis Extension #921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

tats-u wants to merge 10 commits into xoofx:master from tats-u:cjk-friendly

readme.md

-Original file line number
+Diff line change
@@ Expand Up @@
       - [**Diagrams**](src/Markdig.Tests/Specs/DiagramsSpecs.md) extension whenever a fenced code block contains a special keyword, it will be converted to a div block with the content as-is (currently, supports [`mermaid`](https://mermaid.js.org) and [`nomnoml`](https://github.com/skanaar/nomnoml) diagrams)
       - [**YAML Front Matter**](src/Markdig.Tests/Specs/YamlSpecs.md) to parse without evaluating the front matter and to discard it from the HTML output (typically used for previewing without the front matter in MarkdownEditor)
       - [**JIRA links**](src/Markdig.Tests/Specs/JiraLinks.md) to automatically generate links for JIRA project references (Thanks to @clarkd: https://github.com/clarkd/MarkdigJiraLinker)
+      - [**CJK-friendly Emphasis**](src/Markdig.Tests/Specs/CJKFriendlyEmphasis.md) to mitigate a CommonMark specification issue in CJK languages (Thanks to @tats-u: https://github.com/tats-u/markdown-cjk-friendly)
     - Starting with Markdig version `0.20.0+`, Markdig is compatible only with `NETStandard 2.0`, `NETStandard 2.1`, `NETCoreApp 2.1` and `NETCoreApp 3.1`.
     If you are looking for support for an old .NET Framework 3.5 or 4.0, you can download Markdig `0.18.3`.
@@ Expand Down @@

src/Markdig.Tests/Specs/CJKFriendlyEmphasis.generated.cs

-Original file line number
+Diff line change
@@ -0,0 +1,67 @@
+    // --------------------------------
+    //       CJK-friendly Emphasis
+    // --------------------------------
+    using System;
+    using NUnit.Framework;
+    namespace Markdig.Tests.Specs.CJKFriendlyEmphasis
+    {
+        [TestFixture]
+        public class TestCJKFriendlyEmphasisExtension
+        {
+            // ## CJK-friendly Emphasis Extension
+            //
+            // See https://github.com/tats-u/markdown-cjk-friendly/blob/main/specification.md for details about the spec of this extension.
+            //
+            // This extension drastically mitigates [the long-standing issue (specification flaw)](https://github.com/commonmark/commonmark-spec/issues/650) in CommonMark that emphasis in CJK languages is often not parsed as expected.
+            //
+            // The plain CommonMark cannot recognize even the following emphasis in CJK languages:
+            [Test]
+            public void CJKFriendlyEmphasisExtension_Example001()
+            {
+                // Example 1
+                // Section: CJK-friendly Emphasis Extension
+                //
+                // The following Markdown:
+                //     **この文を強調できますか（Can I emphasize this sentence）？**残念ながらこの文のせいでできません（Unfortunately not possible due to this sentence）。
+                //
+                // Should be rendered as:
+                //     <p><strong>この文を強調できますか（Can I emphasize this sentence）？</strong>残念ながらこの文のせいでできません（Unfortunately not possible due to this sentence）。</p>
+                TestParser.TestSpec("**この文を強調できますか（Can I emphasize this sentence）？**残念ながらこの文のせいでできません（Unfortunately not possible due to this sentence）。", "<p><strong>この文を強調できますか（Can I emphasize this sentence）？</strong>残念ながらこの文のせいでできません（Unfortunately not possible due to this sentence）。</p>", "cjk-friendly-emphasis", context: "Example 1\nSection CJK-friendly Emphasis Extension\n");
+            }
+            // ````````````````````````````````` example
+            // 我可以强调**这个`code`**吗（Can I emphasize **this `code`**）？
+            // .
+            // <p>我可以强调<code>这个`code`</code>吗（Can I emphasize <strong>this <code>code</code></strong>）？</p>
+            // `````````````````````````````````
+            [Test]
+            public void CJKFriendlyEmphasisExtension_Example002()
+            {
+                // Example 2
+                // Section: CJK-friendly Emphasis Extension
+                //
+                // The following Markdown:
+                //     **이 용어(This term)**를 강조해 주세요. (Please emphasize **this term**.)
+                //
+                // Should be rendered as:
+                //     <p><strong>이 용어(This term)</strong>를 강조해 주세요. (Please emphasize <strong>this term</strong>.)</p>
+                TestParser.TestSpec("**이 용어(This term)**를 강조해 주세요. (Please emphasize **this term**.)", "<p><strong>이 용어(This term)</strong>를 강조해 주세요. (Please emphasize <strong>this term</strong>.)</p>", "cjk-friendly-emphasis", context: "Example 2\nSection CJK-friendly Emphasis Extension\n");
+            }
+            // You can compare the results with and without this extension: https://tats-u.github.io/markdown-cjk-friendly/?sc8=KirjgZPjga7mlofjgpLlvLfoqr_jgafjgY3jgb7jgZnjgYvvvIhDYW4gSSBlbXBoYXNpemUgdGhpcyBzZW50ZW5jZe-8ie-8nyoq5q6L5b-144Gq44GM44KJ44GT44Gu5paH44Gu44Gb44GE44Gn44Gn44GN44G-44Gb44KT77yIVW5mb3J0dW5hdGVseSBub3QgcG9zc2libGUgZHVlIHRvIHRoaXMgc2VudGVuY2XvvInjgIIKCuaIkeWPr-S7peW8uuiwgyoq6L-Z5LiqYGNvZGVgKirlkJfvvIhDYW4gSSBlbXBoYXNpemUgKip0aGlzIGBjb2RlYCoq77yJ77yfCgoqKuydtCDsmqnslrQoVGhpcyB0ZXJtKSoq66W8IOqwleyhsO2VtCDso7zshLjsmpQuIChQbGVhc2UgZW1waGFzaXplICoqdGhpcyB0ZXJtKiouKQo&gfm=1&engine=markdown-it
+            //
+            // You will find how poor the plain CommonMark is for CJK languages.
+            //
+            // To use this extension, configure the pipeline as follows:
+            //
+            // ```csharp
+            // var pipeline = new MarkdownPipelineBuilder()
+            //     .UseCJKFriendlyEmphasis() // Add this
+            //     .Build();
+            // ```
+        }
+    }

src/Markdig.Tests/Specs/CJKFriendlyEmphasis.md

-Original file line number
+Diff line change
@@ -0,0 +1,37 @@
+    ## CJK-friendly Emphasis Extension
+    See https://github.com/tats-u/markdown-cjk-friendly/blob/main/specification.md for details about the spec of this extension.
+    This extension drastically mitigates [the long-standing issue (specification flaw)](https://github.com/commonmark/commonmark-spec/issues/650) in CommonMark that emphasis in CJK languages is often not parsed as expected.
+    The plain CommonMark cannot recognize even the following emphasis in CJK languages:
+    ```````````````````````````````` example
+    **この文を強調できますか（Can I emphasize this sentence）？**残念ながらこの文のせいでできません（Unfortunately not possible due to this sentence）。
+    .
+    <p><strong>この文を強調できますか（Can I emphasize this sentence）？</strong>残念ながらこの文のせいでできません（Unfortunately not possible due to this sentence）。</p>
+    ````````````````````````````````
+    ````````````````````````````````` example
+    我可以强调**这个`code`**吗（Can I emphasize **this `code`**）？
+    .
+    <p>我可以强调<code>这个`code`</code>吗（Can I emphasize <strong>this <code>code</code></strong>）？</p>
+    `````````````````````````````````
+    ```````````````````````````````` example
+    **이 용어(This term)**를 강조해 주세요. (Please emphasize **this term**.)
+    .
+    <p><strong>이 용어(This term)</strong>를 강조해 주세요. (Please emphasize <strong>this term</strong>.)</p>
+    ````````````````````````````````
+    You can compare the results with and without this extension: https://tats-u.github.io/markdown-cjk-friendly/?sc8=KirjgZPjga7mlofjgpLlvLfoqr_jgafjgY3jgb7jgZnjgYvvvIhDYW4gSSBlbXBoYXNpemUgdGhpcyBzZW50ZW5jZe-8ie-8nyoq5q6L5b-144Gq44GM44KJ44GT44Gu5paH44Gu44Gb44GE44Gn44Gn44GN44G-44Gb44KT77yIVW5mb3J0dW5hdGVseSBub3QgcG9zc2libGUgZHVlIHRvIHRoaXMgc2VudGVuY2XvvInjgIIKCuaIkeWPr-S7peW8uuiwgyoq6L-Z5LiqYGNvZGVgKirlkJfvvIhDYW4gSSBlbXBoYXNpemUgKip0aGlzIGBjb2RlYCoq77yJ77yfCgoqKuydtCDsmqnslrQoVGhpcyB0ZXJtKSoq66W8IOqwleyhsO2VtCDso7zshLjsmpQuIChQbGVhc2UgZW1waGFzaXplICoqdGhpcyB0ZXJtKiouKQo&gfm=1&engine=markdown-it
+    You will find how poor the plain CommonMark is for CJK languages.
+    To use this extension, configure the pipeline as follows:
+    ```csharp
+    var pipeline = new MarkdownPipelineBuilder()
+        .UseCJKFriendlyEmphasis() // Add this
+        .Build();
+    ```

src/Markdig.Tests/Specs/readme.md

-Original file line number
+Diff line change
@@ Expand Up @@
       - [**Diagrams**](DiagramsSpecs.md)
       - [**YAML frontmatter**](YamlSpecs.md)
       - [**JIRA links**](JiraLinks.md)
+      - [**CJK-friendly Emphasis**](CJKFriendlyEmphasis.md)
        > Notice that the links above are not yet the final documentation but are "specification" files used for testing the correctness of markdig for each extension

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add CJK-friendly Emphasis Extension #921

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Add CJK-friendly Emphasis Extension #921

Are you sure you want to change the base?

Uh oh!

Add CJK-friendly Emphasis Extension #921

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!