Split into separate repos?

[sellout]

As mentioned by @conal in #13 (comment), this could potentially be split into separate repos. I support this idea, for basically the same reasons as I made #91.

Basically, we use concat-classes and concat-examples in https://github.com/con-kitty/categorifier, but not concat-plugin. #91 solves the main issue, which is having concat-plugin in the graph. However, there are other changes we’d like to make to concat-classes/-examples (#99) that are easier if we don’t have to push them through the plugin as well.

I’m happy to make this happen if there is agreement about it. Also, I would like #91 to happen first, as it’s nicer history-wise to move things around in one repo than to move them across repos.

[conal]

I'd like to get a better (than almost no) understanding of what con-kitty/categorifier is about. Is the BOB talk up yet?

[sellout]

Only a few months late here. The BOB talk is up, but AFAIK, it’s only part of the larger full-track video. This should jump you to the correct part: https://www.youtube.com/watch?t=10758&v=e74TE0b4xEM

Here’s my quick summary of categorifier: it’s a rewrite/fork of concat-plugin with a goal of compiling pre-existing Haskell to categories. We had 30k+ lines of Haskell that we wanted to transform without having to rewrite it. Some of the things that have been added: fine-grained failure reporting, support for sum types, automatic HasRep generation via TH, support for alternative category class hierarchies (e.g., Kmett’s categories library), support for recursion, extensible “RULES” definitions (it does all rewrites internal to the plugin, rather than using actual RULES pragmas), etc. There’s also a similar list at https://github.com/con-kitty/categorifier/blob/master/plugin/README.md#concat, which is a bit out of date and could use updating.

[mikesperber]

I support the idea of splitting out the plugin into a separate repo. (I'm a little uneasy about changes to examples that can't be pushed through the ConCat plugin, as we're still using that combo, but we can cross that bridge once we get to it.)

[sellout]

It’s not about not being able to push the changes through ConCat plugin, but more the amount of effort required. And the changes I have in mind wouldn’t break the relationship between -examples and -plugin. E.g., supporting GHC 9.0 in -examples is fairly trivial, but because all of the GHC API is restructured in 9.0, supporting GHC 9.0 in -plugin is much more complicated. I would like to add GHC 9 support to -examples without having to do all the work for -plugin, which we don’t use. These kinds of changes shouldn’t break the dependency in any way when compiling with GHC 8.

[mikesperber]

Ah, thanks for the explanation!

[mikesperber]

@conal You OK with splitting? I expect it'll take a few days to shake it all out, but I'll make sure everything will work as before once we're done. (And we're very actively using the ConCat plugin, so I'm highly motivated to make it work.)

[conal]

What is the set of repos we'd end up with? How would we keep the separate repos consistent?

[sellout]

I haven't looked at the dependency graph recently, but I think I'd be happy with anywhere between “every package is its own repo” and three repos that look like

• classes / known / inline / satisfy
• examples / graphics / hardware
• plugin

[mikesperber]

For the plugin, I don't think there's a significant dependency on the version of ConCat.Category and ConCat.AltCat.

Eventually, it would help making the dependencies on the functions known to the plugin explicit in some place other than its source code. @sellout has already done that for this plugin, which can work with different underlying sets of type classes. What exactly that means will become more apparent once we're done with our current set of changes.

(And, as I said, we have vested interest in keeping them consistent for the foreseeable future.)

[conal]

I don't have a picture of how we would keep the separate repos consistent.

[mikesperber]

@conal Would it be OK to proceed with this? (I see that Github has reordered your question and my answer to it.)

[conal]

Currently, if a change to the classes or plugin breaks the examples, the person who makes the change will find out immediately and the change won't get committed and merged before the examples all work again. I'm still unclear on the plan for maintaining consistency after splitting.

I'm also unclear on the motivation for splitting.