Discussion on package maintainer notification

[gasche]

Currently opam-builder is a niche tool used by only a few people -- I suspect I may be the only active user, Damien has his script that does more or less the same thing. I would like to make it a more central part of the opam-repository QA work, by increasing its adoption among package maintainers, so that they would themselves fix the issues that Damien and myself are currently alone working on.

There are minor things that can be improved with opam-builder (such as adding a bit more information in the build logs to make them a single place to visit to act upon a failure, instead of having to cross-reference them with the opam file etc.), and some more important changes that require more work (such as having a way to link to upstream issues that discuss specific failures from the report page, to coordinate response and discussions). But I think that the most important change that needs to happen to increase maintainer adoption is notification. We need to alert the right people when we find a build failure, moving from a "pull" model where only interested people come look the reports to a "push" model where we directly incite people action.

I would like here to discuss the way forward for maintainer notification, with the goal of eventually implementing something. My time to work on this is very limited, so the simplest the consensual design, the more chances of an implementation.

Right now I have to potential designs in mind:

1. An email-centric design. Each time the status of a build (a one case in the full report, that is a (package, version, compiler version)) changes, send an email to the package maintainer (as indicated in the OPAM metadata) with a link to the build log, and let them handle it from there. More precisely, the status changes that would result in an email are ({Fail,Deps} -> Builds, Builds -> {Fails,Deps}, and Unknown -> {Fail,Deps}) -- Fail is a build failure, Deps is uninstallability, and Unknown is a non-existing build (non-existing package or package version or compiler version).
2. A github-centric design. Create a specific opam-builder-status repository. Each time the status of a build changes to Fail or Deps, open an issue there, link the issue from the HTML report. The issue would contain a link to the log, and a cc: mention of the package maintainer if we know their username; otherwise we would send an email to the maintainer(s) with a link to the issue.

(I also considered an syndication-centric design that would export an RSS or Atom feed per package and per maintainer, containing items for status changes. But that is still a pull-based model, and I don't think we should require maintainer actions to start notifying them. It may be a nice complement of the other models, but I don't it works alone.)

In terms of user convenience the github-centric design seems better. In particular, it makes it easy for people to create a link between the opam-builder report and an eventual upstream issue (on the faliing package's bugtracker), by just linking to that github issue. In particular if the upstream is also using github we get the nice cross-reference support. The fact that it opens a discussion space, besides being a mere notification, is also useful, as it's not always easy to orient oneself and decide how to fix a specific failure so multi-actor discussion may be required.

There is also the question of giving maintainers control over which notification reaches them. We could think of per-package control to enable or disable notifications, but that sounds too much work for a little-used feature -- I'm not sure we want to make it easy for maintainers to stop hearing about failures. On the other hand, maintaining a blacklist of people that do not wish to be contacted by these automated means could be a gentle thing to do. This blacklist could simply be a file listing emails of opted-out maintainers, stored either in the opam-builder sources themselves, or on that potential opam-builder-status repository. People wishing to opt out of notifications would have to send a pull-request against that list.

If we decide to go the github-centric route, a last issue is that I don't think we have the github usernames for my cc: idea. It may be useful to have maintainers' github usernames as part of package metadata (and we could think of a bulk change to add that based on a crowd-sourced mapping); or we could maintain this mapping only for opam-builder for now, as a list on the side.

@lefessan, do you have an email server that could run to send email notifications? Is there a convenient way to use it from the OCaml side, as part of opam-builder report generation?

[dbuenzli]

But I think that the most important change that needs to happen to increase maintainer adoption is notification.

Even before that if I could simply put my maintainer name or the org:erratique tag in the search box on the page, that would make the tool useful to me. As is now, I simply can't get a bird eyes view of the ~25 packages I distribute which renders the thing useless for me.

[gasche]

@dbuenzli it should be reasonably easy to implement, given that the search feature is implemented in OCaml (and compiled with js_of_ocaml): search.ml. We would need to add the information (maintainer, tags) in (hidden) parts of the HTML report, and then enhance the searching code to also include them in the searched data.

I don't know what would be more natural, having hidden columns for this metadata, or putting it as attributes of the title td cell.

[lefessan]

@gasche Unfortunately, the current server is not yet configured as an email server, but I will discuss with Grégoire who has much more expertise than me about how I can improve that.

@dbuenzli On a side note, maybe you could namespace your projects ? For example, most OCamlPro's software either starts with ocp- or with typerex-, which makes it easy to find them in opam-builder. If you choose a nice prefix, it would make it easy for you to find them, but also for users to immediately know your family of projects.

[dbuenzli]

On a side note, maybe you could namespace your projects ?

I'm really not fond about this idea.

First I don't have a family of projects (except for the unicode stuff which is prefixed with U), each project stands on its own and most have quite different scopes. In the future these projects may not be developed or handled by me and I don't think they should switch name because of this. The fact that these ~25 projects are currently handled by me is an accident.

Second I find the system much more usable for end users if my website project page name, source repo name, github mirror name, tarball name, opam name, ocamlfind prefix name, OCaml library prefix name and OCaml module prefix name all coincide, and an Erratique_react or Erratique_cmdliner module would be really be awful.

Given these points, using an org: tag in package metadata to encode administrative ownership is a much better solution than to encode this in the name of packages which in my opinion should be left to an accurate or poetic description of what the software actually does.

Tags give me a super easy way of bulk handle my packages in opam. For example on a new OCaml release I simply do an opam install $(opam search org:erratique -s) to (most of the time) get peace of mind.