WIP: Add Persian and English mkdocs by Cmatrix1 · Pull Request #30 · fast-crawler/fastcrawler

Cmatrix1 · 2023-07-14T16:30:41Z

I have added Persian and English mkdocs to the project, but they need to be cleaner and better from now on.

ManiMozaffar · 2023-07-15T06:36:41Z

That is not our TOP priority, but we'll work on this branch.
let's make it to 'DRAFT' and WIP

ManiMozaffar · 2023-08-25T09:41:59Z

docs/en/data/people.yml

+maintainers:
+- login: tiangolo
+  answers: 1844
+  prs: 430
+  avatarUrl: https://avatars.githubusercontent.com/u/1326112?u=740f11212a731f56798f558ceddb0bd07642afa7&v=4
+  url: https://github.com/tiangolo
+experts:
+- login: Kludex
+  count: 434
+  avatarUrl: https://avatars.githubusercontent.com/u/7353520?u=62adc405ef418f4b6c8caa93d3eb8ab107bc4927&v=4
+  url: https://github.com/Kludex
+- login: dmontagu
+  count: 237
+  avatarUrl: https://avatars.githubusercontent.com/u/35119617?u=540f30c937a6450812628b9592a1dfe91bbe148e&v=4
+  url: https://github.com/dmontagu
+- login: Mause
+  count: 220
+  avatarUrl: https://avatars.githubusercontent.com/u/1405026?v=4
+  url: https://github.com/Mause
+- login: ycd


can you get rid of these people? they're FastAPI's people not us haha
Maybe you can add us as maintainers :)

ManiMozaffar · 2023-08-25T09:42:11Z

docs/en/data/github_sponsors.yml

+sponsors:
+- - login: cryptapi
+    avatarUrl: https://avatars.githubusercontent.com/u/44925437?u=61369138589bc7fee6c417f3fbd50fbd38286cc4&v=4
+    url: https://github.com/cryptapi


We don't have sponsers!

ManiMozaffar · 2023-08-25T09:42:28Z

docs/en/data/sponsors_badge.yml

+logins:
+  - jina-ai
+  - deta
+  - investsuite
+  - mikeckennedy
+  - deepset-ai
+  - cryptapi
+  - xoflare
+  - DropbaseHQ
+  - VincentParedes
+  - BLUE-DEVIL1134
+  - ObliviousAI
+  - Doist
+  - nihpo
+  - svix
+  - armand-sauzay
+  - databento-bot
+  - nanram22


We don't have sponsers

ManiMozaffar · 2023-08-25T09:42:42Z

docs/en/docs/img/github-social-preview.png

Don't use FastAPI logo

ManiMozaffar · 2023-08-25T09:42:59Z

docs/en/docs/index.md

+* **Fast**: Very high performance, on par with **NodeJS** and **Go** (thanks to Starlette and Pydantic). [One of the fastest Python frameworks available](#performance).
+* **Fast to code**: Increase the speed to develop features by about 200% to 300%. *
+* **Fewer bugs**: Reduce about 40% of human (developer) induced errors. *
+* **Intuitive**: Great editor support. <abbr title="also known as auto-complete, autocompletion, IntelliSense">Completion</abbr> everywhere. Less time debugging.
+* **Easy**: Designed to be easy to use and learn. Less time reading docs.
+* **Short**: Minimize code duplication. Multiple features from each parameter declaration. Fewer bugs.
+* **Robust**: Get production-ready code. With automatic interactive documentation.
+* **Standards-based**: Based on (and fully compatible with) the open standards for APIs: <a href="https://github.com/OAI/OpenAPI-Specification" class="external-link" target="_blank">OpenAPI</a> (previously known as Swagger) and <a href="https://json-schema.org/" class="external-link" target="_blank">JSON Schema</a>.
+


Add our own library text

ManiMozaffar · 2023-08-25T09:46:53Z

scripts/mkdocs_hooks.py

+from functools import lru_cache
+from pathlib import Path
+from typing import Any, List, Union
+
+import material
+from mkdocs.config.defaults import MkDocsConfig
+from mkdocs.structure.files import File, Files
+from mkdocs.structure.nav import Link, Navigation, Section
+from mkdocs.structure.pages import Page
+
+
+@lru_cache()
+def get_missing_translation_content(docs_dir: str) -> str:
+    docs_dir_path = Path(docs_dir)
+    missing_translation_path = docs_dir_path.parent.parent / "missing-translation.md"
+    return missing_translation_path.read_text(encoding="utf-8")
+
+
+@lru_cache()
+def get_mkdocs_material_langs() -> List[str]:
+    material_path = Path(material.__file__).parent
+    material_langs_path = material_path / "partials" / "languages"
+    langs = [file.stem for file in material_langs_path.glob("*.html")]
+    return langs
+
+
+class EnFile(File):
+    pass
+
+
+def on_config(config: MkDocsConfig, **kwargs: Any) -> MkDocsConfig:
+    available_langs = get_mkdocs_material_langs()
+    dir_path = Path(config.docs_dir)
+    lang = dir_path.parent.name
+    if lang in available_langs:
+        config.theme["language"] = lang
+    if not (config.site_url or "").endswith(f"{lang}/") and not lang == "en":
+        config.site_url = f"{config.site_url}{lang}/"
+    return config
+
+
+def resolve_file(*, item: str, files: Files, config: MkDocsConfig) -> None:
+    item_path = Path(config.docs_dir) / item
+    if not item_path.is_file():
+        en_src_dir = (Path(config.docs_dir) / "../../en/docs").resolve()
+        potential_path = en_src_dir / item
+        if potential_path.is_file():
+            files.append(
+                EnFile(
+                    path=item,
+                    src_dir=str(en_src_dir),
+                    dest_dir=config.site_dir,
+                    use_directory_urls=config.use_directory_urls,
+                )
+            )
+
+
+def resolve_files(*, items: List[Any], files: Files, config: MkDocsConfig) -> None:
+    for item in items:
+        if isinstance(item, str):
+            resolve_file(item=item, files=files, config=config)
+        elif isinstance(item, dict):
+            assert len(item) == 1
+            values = list(item.values())
+            if not values:
+                continue
+            if isinstance(values[0], str):
+                resolve_file(item=values[0], files=files, config=config)
+            elif isinstance(values[0], list):
+                resolve_files(items=values[0], files=files, config=config)
+            else:
+                raise ValueError(f"Unexpected value: {values}")
+
+
+def on_files(files: Files, *, config: MkDocsConfig) -> Files:
+    resolve_files(items=config.nav or [], files=files, config=config)
+    if "logo" in config.theme:
+        resolve_file(item=config.theme["logo"], files=files, config=config)
+    if "favicon" in config.theme:
+        resolve_file(item=config.theme["favicon"], files=files, config=config)
+    resolve_files(items=config.extra_css, files=files, config=config)
+    resolve_files(items=config.extra_javascript, files=files, config=config)
+    return files
+
+
+def generate_renamed_section_items(
+    items: List[Union[Page, Section, Link]], *, config: MkDocsConfig
+) -> List[Union[Page, Section, Link]]:
+    new_items: List[Union[Page, Section, Link]] = []
+    for item in items:
+        if isinstance(item, Section):
+            new_title = item.title
+            new_children = generate_renamed_section_items(item.children, config=config)
+            first_child = new_children[0]
+            if isinstance(first_child, Page):
+                if first_child.file.src_path.endswith("index.md"):
+                    # Read the source so that the title is parsed and available
+                    first_child.read_source(config=config)
+                    new_title = first_child.title or new_title
+            # Creating a new section makes it render it collapsed by default
+            # no idea why, so, let's just modify the existing one
+            # new_section = Section(title=new_title, children=new_children)
+            item.title = new_title
+            item.children = new_children
+            new_items.append(item)
+        else:
+            new_items.append(item)
+    return new_items
+
+
+def on_nav(
+    nav: Navigation, *, config: MkDocsConfig, files: Files, **kwargs: Any
+) -> Navigation:
+    new_items = generate_renamed_section_items(nav.items, config=config)
+    return Navigation(items=new_items, pages=nav.pages)
+
+
+def on_pre_page(page: Page, *, config: MkDocsConfig, files: Files) -> Page:
+    return page
+
+
+def on_page_markdown(
+    markdown: str, *, page: Page, config: MkDocsConfig, files: Files
+) -> str:
+    if isinstance(page.file, EnFile):
+        missing_translation_content = get_missing_translation_content(config.docs_dir)
+        header = ""
+        body = markdown
+        if markdown.startswith("#"):
+            header, _, body = markdown.partition("\n\n")
+        return f"{header}\n\n{missing_translation_content}\n\n{body}"
+    return markdown


Again doc missing

Add doc string to each function, explaining what it does

Make sure it's not fastapi related

ManiMozaffar · 2023-08-25T09:47:07Z

scripts/netlify-docs.sh

+python3.6 -m pip install --user flit
+# Install with Flit
+python3.6 -m flit install --user --extras doc
+# Finally, run mkdocs
+python3.6 -m mkdocs build


why should we use python 3.6?

ManiMozaffar · 2023-08-25T09:47:24Z

scripts/publish.sh

+
+set -e
+
+flit publish


i don't think this is how we manage our publish

ManiMozaffar · 2023-08-25T09:47:36Z

scripts/test-cov-html.sh

+bash scripts/test.sh ${@}
+coverage combine
+coverage report --show-missing
+coverage html


contact with amir bahador regarding this.

ManiMozaffar · 2023-08-25T09:47:47Z

scripts/test.sh

+#!/usr/bin/env bash
+
+set -e
+set -x
+
+export PYTHONPATH=./docs_src
+coverage run -m pytest tests ${@}


hmm again this is CI/CD

Add Persian and English mkdocs

79e2687

Cmatrix1 added the documentation Improvements or additions to documentation label Jul 14, 2023

Cmatrix1 requested review from ManiMozaffar and amirbahador-hub July 14, 2023 16:30

Cmatrix1 self-assigned this Jul 14, 2023

ManiMozaffar changed the title ~~Add Persian and English mkdocs~~ WIP: Add Persian and English mkdocs Jul 15, 2023

ManiMozaffar marked this pull request as draft July 15, 2023 06:37

ManiMozaffar linked an issue Jul 15, 2023 that may be closed by this pull request

Adding docs for website's doc service #12

Open

14 tasks

aerosadegh force-pushed the develop branch from 5c63bcf to 7009996 Compare July 16, 2023 14:08

🎉 Add title for docs

7400a7d

ManiMozaffar requested changes Aug 25, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Add Persian and English mkdocs#30

WIP: Add Persian and English mkdocs#30
Cmatrix1 wants to merge 2 commits intodevelopfrom
feature/docs

Cmatrix1 commented Jul 14, 2023

Uh oh!

ManiMozaffar commented Jul 15, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

ManiMozaffar Aug 25, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Cmatrix1 commented Jul 14, 2023

Uh oh!

ManiMozaffar commented Jul 15, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants