Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Contribution Guide

## Pull Request
We welcome anyone who can adhere to our [code of conduct.](CODE_OF_CONDUCT_JP.md)

## Test
### Code Style
Please adhere to the `black` style guide. Please adhere to the `flake8` lint rules.
1 change: 0 additions & 1 deletion Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ verify_ssl = true
name = "pypi"

[packages]
tqdm = "*"
dataclasses-json = "*"

[dev-packages]
Expand Down
59 changes: 21 additions & 38 deletions Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,31 @@
# linkstat
[![test-lint-format](https://github.com/DogFortune/linkstat/actions/workflows/lint-test-format.yml/badge.svg?branch=main)](https://github.com/DogFortune/linkstat/actions) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)

_linkstat_ is a script that verifies the connectivity of links documented in the documentation. By detecting broken links early, it maintains the integrity of the documentation.
Currently, only Markdown files (*.md) are supported.

## Caution
This library accesses services during runtime, so executing it in large quantities will cause load on the target service. When performing functional verification or integrating into CI/CD, please ensure the load on the linked service is minimized as much as possible.

## Install

```sh
pip install linkstat
```

## Usage

```sh
linkstat {source_file_or_directory}
```

## Output

You can output reports in JSON format by using the option.

```sh
linkstat --report-json {path} {source_file_or_directory}
```

## Contribute
[Guideline](CONTRIBUTING.md)
11 changes: 6 additions & 5 deletions README_JP.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# linkstat
<div align="center">
[![test-lint-format](https://github.com/DogFortune/linkstat/actions/workflows/lint-test-format.yml/badge.svg?branch=main)](https://github.com/DogFortune/linkstat/actions) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
</div>

[![test-lint-format](https://github.com/DogFortune/linkstat/actions/workflows/lint-test-format.yml/badge.svg)](https://github.com/DogFortune/linkstat/actions/workflows/lint-test-format.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)

_linkstat_ はドキュメントに記載されているリンクの疎通確認を行うスクリプトです。リンク切れの早期発見を行う事でドキュメントの健全性を保ちます。
現在対応しているのはMarkdownファイル(*.md)のみです。

## 注意
本ライブラリは実行時にサービスへアクセスするため、大量に実行すると相手サービスに負荷が発生します。動作確認及びCI/CDに取り込む場合は、リンク先のサービス負荷は限りなく少なくして下さい。

## インストール

```sh
Expand All @@ -18,8 +21,6 @@ pip install linkstat
linkstat {source_file_or_directory}
```

パスを指定しない場合はカレントディレクトリを検査対象とします。

## 出力
オプションを使用することでJSON形式のレポートを出力できます。

Expand Down
5 changes: 0 additions & 5 deletions THIRD-PARTY-LICENSES.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,6 @@ The MIT License (MIT)
Copyright (c) 2018 Łukasz Langa
https://github.com/psf/black/blob/43135e9fafccbca723910a45aa62f0f182e85e5f/LICENSE

## tqdm
MPL v2.0 and MIT
Copyright (c) 2013 noamraph
https://github.com/tqdm/tqdm/blob/0ed5d7f18fa3153834cbac0aa57e8092b217cc16/LICENCE

## Dataclasses JSON
The MIT License
Copyright (c) 2019 Charles Li and contributors
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ description = "Perform connectivity checks on URLs listed in the Markdown"
license-files = ["LICENSE"]
readme = "README.md"
requires-python = ">=3.10"
version = "0.0.2"
dependencies = ["tqdm", "dataclasses-json"]
version = "1.0.0"
dependencies = ["dataclasses-json"]
keywords = ["check", "url", "link"]
classifiers = [
"Development Status :: 3 - Alpha",
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3 :: Only",
Expand Down
34 changes: 16 additions & 18 deletions src/linkstat/analyzer.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
from linkstat.reporter import ReportData
from dataclasses import dataclass
import re
from tqdm import tqdm

URL_PATTERN = r'https?://[^\s\)\]>"]+'
URL_RE = re.compile(URL_PATTERN)
Expand Down Expand Up @@ -50,29 +49,28 @@ def request(url: str) -> AnalyzeResponse:


def check_links(links: dict[str, URLInfo]) -> list[ReportData]:
"""URLの疎通確認を行います。確認を行うのは重複していないものだけです
"""URLの疎通確認を行います。確認を行うのは重複していないものだけ

:param links: URLリスト
:type links: dict[str, URLInfo]
:return: 確認結果
:return: 確認結果(重複したURLは除く)
:rtype: list[ReportData]
"""
results = []
with tqdm(links.items()) as links_prog:
for file_path, link_items in links_prog:
links_prog.set_description(file_path)
for item in tqdm(link_items):
if not item.duplicate:
res = request(item.url)
data = ReportData(
file_path,
item.line,
item.url,
res.result,
res.code,
res.reason,
)
results.append(data)
for file_path, link_items in links.items():
for item in link_items:
if not item.duplicate:
res = request(item.url)
data = ReportData(
file_path,
item.line,
item.url,
res.result,
res.code,
res.reason,
)
print(f"{data.url}: {data.result}")
results.append(data)
return results


Expand Down
7 changes: 4 additions & 3 deletions src/linkstat/app.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import os
from pathlib import Path
from . import analyzer
from . import reporter
Expand All @@ -19,7 +18,7 @@ def __output(data: list[ReportData], format: OutputType, args):
"""
match format:
case OutputType.Console:
line = reporter.console(data)
line = reporter.get_summary_message(data)
print(line)
case OutputType.Json:
output_path = args.report_json
Expand All @@ -46,7 +45,7 @@ def __format_setting(args) -> OutputType:

def create_parser():
parser = argparse.ArgumentParser()
parser.add_argument("src", default=os.environ.get("SRC_DIR", "."))
parser.add_argument("src")
parser.add_argument(
"--report-json", type=str, help="Create json report file at given path"
)
Expand All @@ -60,6 +59,8 @@ def main(args=None):
format = __format_setting(parsed_args)
src = parsed_args.src

start_msg = reporter.get_fill_plain_message(" linkstat start ")
print(start_msg)
files = analyzer.search(src)
links = analyzer.extract_url(files)
report_data_list = analyzer.check_links(links)
Expand Down
83 changes: 76 additions & 7 deletions src/linkstat/reporter.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
from dataclasses import dataclass
from dataclasses_json import dataclass_json
from pprint import pformat
from linkstat.enums import Result
from typing import List
import json
import os
import shutil


@dataclass_json
Expand All @@ -12,9 +13,9 @@ class ReportData:
file: str
line: int
url: str
result: str
result: Result
code: int
reason: str
reason: str | None


@dataclass_json
Expand All @@ -31,10 +32,78 @@ def default(self, obj):
return super().default(obj)


def console(data: list[ReportData]):
# TODO: 出力形式は仮でpformatを設定中。
line = pformat(data)
return line
class Colors:
"""カラーコード"""

RED = "\033[91m"
GREEN = "\033[92m"
YELLOW = "\033[93m"
RESET = "\033[0m"


def get_fill_plain_message(msg: str) -> str:
"""区切り線が入ったメッセージテキストを作成します。
メソッド自体はメッセージを作るだけで、出力は呼び出し側が行って下さい。

:param msg: _description_
:type msg: str
:return: _description_
:rtype: str
"""
fill_char = "="
terminal_width = shutil.get_terminal_size(fallback=(80, 24)).columns
if terminal_width < 40:
terminal_width = 80

total_fill = terminal_width - len(msg)
left_fill = total_fill // 2
right_fill = total_fill - left_fill

start_message = f"{fill_char*left_fill}{msg}{fill_char*right_fill}"
return start_message


def get_summary_message(data: list[ReportData]):
"""レポート内容を元にサマリーを作成します。
チェックしたURLの数、OK,NGの数、NGのものはURLを出す。

:param data: _description_
:type data: list[ReportData]
:return: _description_
:rtype: _type_
"""
total_count = len(data)
ok_count = sum(item.result == Result.OK for item in data)
ng_items = [item for item in data if item.result == Result.NG]

total_part = f"{Colors.GREEN}{total_count} Total{Colors.RESET}"
ok_part = f"{Colors.GREEN}{ok_count} OK{Colors.RESET}"

color_message = f" {total_part}, {ok_part}"
plain_message = f" {total_count} Total, {ok_count} OK"
summary_message = ""

if (ng_count := len(ng_items)) == 0:
fill_char = f"{Colors.GREEN}={Colors.RESET}"
else:
print(get_fill_plain_message(" FAILURES "))
ng_detail = "\n".join([f"{item.url}: {item.reason}" for item in ng_items])
summary_message += f"{ng_detail}" + "\n"
ng_part = f"{Colors.RED}{ng_count} NG{Colors.RESET}"
fill_char = f"{Colors.RED}={Colors.RESET}"
color_message += f", {ng_part} "
plain_message += f", {ng_count} NG "

terminal_width = shutil.get_terminal_size(fallback=(80, 24)).columns
if terminal_width < 40:
terminal_width = 80

total_fill = terminal_width - len(plain_message)
left_fill = total_fill // 2
right_fill = total_fill - left_fill

summary_message += f"{fill_char*left_fill}{color_message}{fill_char*right_fill}"
return summary_message


def dump_json(data: list[ReportData], output_path: str):
Expand Down
4 changes: 2 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
from urllib.error import HTTPError, URLError


@pytest.fixture(scope="session", autouse=True)
def check_mock_server():
@pytest.fixture()
def use_mock_server():
"""テスト実行前にモックサーバーの起動を確認"""
mock_server_url = "http://localhost:8000/get"

Expand Down
Loading