Improvements to changelog

[ErikBjare]

Just a few things I thought about which would improve the changelog and its generation.

• Generate it in markdown format, with subheaders + links to repo for submodules. (fixed in docs: added new script to build changelog #381)
• Assign commits to issues/PRs, and include correct links to them. Right now GitHub assumes PRs/issues in subrepos are in the main repo, leading to it generating bad links. (fixed in docs: added new script to build changelog #381)
• With the guideline changes in Improve commit messages #391 we should additionally group changes by type (feat, fix, docs, etc.) to make it easy to read at a glance. (fixed in feat: improved generation of changelog #505)
• Make the changelog page in the docs include a md file with the actual changelog, so we don't need to maintain it in two markup languages. Will have to go with future improvements.

[xylix]

Seems good to me. I'll look into this.

[ErikBjare]

I started looking into this, began writing a python script that generates it nicely. Here's what I have so far:

from subprocess import run as _run, STDOUT, PIPE
import re


def run(cmd) -> str:
    return _run(cmd.split(" "), stdout=PIPE, stderr=STDOUT, encoding="utf8").stdout


def process_line(s: str, repo: str) -> str:
    """Generates links from commit and issue references (like 0c14d77, #123) to correct repo and such"""
    s = re.sub(r"#([0-9]+)", rf"[#\1](https://github.com/ActivityWatch/{repo}/issues/\1)", s)
    return s


def commit_linkify(commitid: str, repo: str) -> str:
    return f"[`{commitid}`](https://github.com/ActivityWatch/{repo}/commit/{commitid})"


def build():
    prev_release = run("git describe --abbrev=0").strip()
    summary_bundle = run(f"git log {prev_release}...master --oneline --decorate")
    print("### activitywatch (bundle repo)")
    for line in summary_bundle.split("\n"):
        if line:
            commit = line.split(" ")[0]
            line = ' '.join(line.split(' ')[1:])
            commit_link = commit_linkify(commit, 'activitywatch')
            line = f" - {line} ({commit_link})"
            print(process_line(line, "activitywatch"))

    summary_subrepos = run(f"git submodule summary {prev_release}")
    for s in summary_subrepos.split("\n\n"):
        lines = s.split("\n")
        header = lines[0]
        if header.strip():
            _, name, commitrange, count = header.split(" ")
            name = name.strip(".").strip("/")
            print(f"\n### {name} {commitrange}")
            commits = [process_line(" - " + l.strip(" ").strip(">").strip(" "), name) for l in lines[1:]]
            print("\n".join(commits))


if __name__ == "__main__":
    build()

I also updated the changelog for the v0.9 release to make it look a bit more like what I had in mind.

[ErikBjare]

Updated it a bit (will update the script I created above).

Here's an example output:

activitywatch (bundle repo)

• (tag: v0.9.0, origin/master, master) Merge pull request #375 from xylix/dev/fix-.dmg-naming (bdf4fa1)
• Add platform and arch info to .dmg (3525d0b)
• [travis] switched back to skip_cleanup (55c4c90)
• Merge pull request #371 from xylix/dev/use-new-travis-cleanup (bf789e5)
• Merge pull request #373 from xylix/dev/release-note-script (0ae4632)
• Add release note script to main repo (198bfa1)
• Actually use travi's new cleanup boolean (54178f2)
• Merge pull request #369 from xylix/submodule-update-fixedt-test (51a484a)
• Add node_modules fix in appveyor back to aw-server, update version number in aw.spec (a6bf09b)
• Merge pull request #365 from xylix/master (cd75c71)
• Update submodules (depends on aw-server-rust submodule update) (c4b0be1)
• Merge pull request #356 from ActivityWatch/dev/less-breaking-travis-improvements (4d8dfc5)
• Update submodule aw-server-rust (0a62d83)
• Merge pull request #354 from ActivityWatch/dev/aw-server-rust-app (b1e9ead)
• Add caching Fix travis warnings (9109c63)
• Add aw-server-rust binary to pyinstaller bundles (a567003)
• Merge pull request #347 from ActivityWatch/dev/flask-restx (d709526)
• (origin/dev/flask-restx, dev/flask-restx) updated aw-server submodule (fefb479)
• migrated to flask-restx, updated submodules (e2d57b7)
• Update stale.yml (dc89d60)
• Create stale.yml (d73b97b)
• Merge pull request #331 from ActivityWatch/dev/aw-server-rust (6762125)
• (origin/dev/aw-server-rust) Update aw-qt submodule (2ca1e6e)
• updated aw-server-rust (865b7e4)
• correctly placed aw-server-rust module in subdirectory, updated aw-server-rust submodule to correctly name executable as aw-server-rust (50346bd)
• readded aw-server-python (cbdc3b4)
• Replace aw-server with aw-server-rust (2ddeef5)
• aw-watcher-window: Update submodule (5bfdb3d)
• Merge pull request #336 from ActivityWatch/dev/macos-app (f8cb9fd)
• removed unused requirements.txt (c123849)
• fixed .dmg building in CI (d189007)
• Fix wrong filename in merged aw.spec (d5495a4)
• Merge branch 'dev/macos-app' of https://github.com/ActivityWatch/activitywatch (90183cd)
• Merge https://github.com/ActivityWatch/activitywatch (e23d750)
• Add dmg building (3657247)
• Add version info to aw.spec, update dev version of pyinstaller (eca1c03)
• fixed building .app, tried fixing error in Ubuntu CI: 'libQt5Xml.so.5: cannot open shared object file' (5e4de1e)
• (origin/xylix-macos-app-2) fixed icon and version for .app, added .app building to CI (0bbddfb)
• Merge branch 'master' of https://github.com/xylix/activitywatch into xylix-macos-app (3d01b39)
• updated submodule (3eb6349)
• updated submodules (90ec663)
• Working .app package require version 4.0 of pyinstaller for building mega-bundle (3c19049)
• Start building mega spec (1c68175)
• updated submodule (f0bfde4)
• updated submodules (9f3b91e)
• updated submodules with poetry/pyproject.toml changes (609429a)

aw-client a8c9fd7...9e4cdb5

• switched to only using poetry install in Makefile
• changed Makefile use of pip to pip3
• fixed version in pyproject.toml
• updated poetry.lock
• fixed make build DEV=true
• Merge pull request #48 from ActivityWatch/dev/poetry
• Merge pull request #47 from JustinDDavis/232-event-limit-default-change

aw-core f6bcbd9...a62a299

• switched to only using poetry install in Makefile
• changed Makefile use of pip to pip3
• fixed make build DEV=true
• moved from Pipfile/setup.py/pipenv to pyproject.toml/poetry (#85)

aw-qt 3f8d416...3de8095

• switched to only using poetry install in Makefile
• Merge pull request #55 from xylix/dev/improved-exit-handling
• Merge pull request #53 from ActivityWatch/dev/windows-regex-fix
• Merge branch 'master' of github.com:ActivityWatch/aw-qt
• changed logging level for message
• switched to using pyproject/poetry (#51)
• Merge pull request #52 from xylix/monochrome-logos

aw-server f95ec5e...908e0e3

• switched to only using poetry install in Makefile
• Merge pull request #67 from xylix/dev/add-back-rm-node-modules
• Merge pull request #65 from xylix/dev/stop-removing-node-modules
• kept werkzeug version at <1.0 to prevent bug: Werkzeug 1.0 breaks CI #346
• Merge pull request #63 from ActivityWatch/dev/flask-restx
• changed Makefile use of pip to pip3
• updated poetry.lock
• fixed make build DEV=true
• Merge pull request #62 from ActivityWatch/dev/poetry

aw-server-rust 0000000...d91a4a6

• aw-transform: Fix clippy warnings

aw-watcher-afk 55528c3...4d357eb

• added installation of pyinstaller to before_install
• switched to only using poetry install in Makefile
• switched to using pyproject.toml/poetry (#38)

aw-watcher-window 0c14d77...751e682

• added installation of pyinstaller to before_install
• switched to only using poetry install in Makefile
• readded script that disappeared in previous PR
• switched to using pyproject/poetry (#39)
• Merge pull request #37 from cunidev/master

[ErikBjare]

• What's notably missing there are commit hashes for submodules. I'll see if I can fix that somehow. (fixed in feat: improved generation of changelog #505)

[tulir]

Would something like towncrier be an acceptable solution? It requires a bit more work when making pull requests (creating the newsfiles), but changelogs should be better when they're compiled from newsfiles written with changelogs in mind rather than from commit messages (which often contain more technical details).

[ErikBjare]

Using towncrier would be in addition the changes this issue is about, not really a substitute.

I've looked into it in the past but I'm not so sure it's a great fit with git submodules, and I'd also rather not introduce additional requirements for contributors who submit PRs.

[ErikBjare]

Just pushed something that could become final in #505

[ErikBjare]

Closing since #505 was merged.