Move the reference dic and template files into separate repositories

[vaitkus]

Originally most of the dictionaries resided in this single GitHub repository. At some point, they were split off into separate repositories (#156) which, IMHO, improved the handling and versioning of these dictionaries. I would like to suggest that we do the same for the reference dictionary and/or the template files as these have their own independent version numbers.

If it is decided to split the dictionaries, it should best be done before the next release.

[nautolycus]

I haven't thought through all the implications, but at first glance I think this is a good idea.

[jamesrhester]

I agree this is a good idea. It is just a bit involved to divide the git history between the repositories, something which I've now forgotten how to do, but there are online instructions on how to do it.

I think the ddl.dic file clearly should be in a separate repository, and I'm inclined to put the template/enum files together in a separate repository. I think we should be able to create a Github release artifact that includes core/template/enum files as a single download.

[vaitkus]

I would suggest going even further and splitting template and enum files into their own repositories. My main reasoning is that these files carry separate version numbers therefore the version given in a tag release will automatically not match at least one of them. Alternatively, we could give these two files the same version number and update it in both files upon release even if only one of them was changed.

[jamesrhester]

It turns out that there is a relatively simple solution, using git-filter-repo. I will create new repositories for each of the above-mentioned files, then clean out their history from the cif core repository.

[jamesrhester]

I have now created three new repositories for DDL, templ_enum.cif and templ_attr.cif. The next step is to remove these files from cif_core. There are a couple of options:

1. Do a normal deletion as a pull request. This seems safest.
2. Do a git filter-repo on cif_core. It will be as if the files were never there.

I favour (1) as there seems no harm in having duplicate history in the cif_core and new repositories for the given files.

A final step will need to be updating our checking workflows to import the dictionary and template files from their respective repositories, and creating releases where they are all together in one directory.

[vaitkus]

I have now created three new repositories for DDL, templ_enum.cif and templ_attr.cif. The next step is to remove these files from cif_core.

Thank you for doing this and sorry for not responding sooner.

There are a couple of options:
1. Do a normal deletion as a pull request. This seems safest.

2. Do a `git filter-repo` on cif_core. It will be as if the files were never there.

I favour (1) as there seems no harm in having duplicate history in the cif_core and new repositories for the given files.

I am also in support of (1) as it preserves the actual history of the repository. This approach has now been implemented by merging PR #578.

A final step will need to be updating our checking workflows to import the dictionary and template files from their respective repositories, and creating releases where they are all together in one directory.

I have issued PRs with updates to the checking workflows, hopefully, I did not miss any.

I am not sure I understand why it is needed to creat[e] releases where they are all together in one directory. It seem more flexible for each repository to have their own separate versioned tag. Of course, I might be overlooking something.

One final thing that needs to be done is to transfer some of the issues from this repository to the new ones.

[jamesrhester]

By "create releases where they are all together in one directory" I meant a release of core CIF that bundled the auxilliary files together with core CIF, so that they could be downloaded from a single location, just for convenience.

[jamesrhester]

Closing this as largely done, any further issues we'll deal with as they arise.

[vaitkus]

By "create releases where they are all together in one directory" I meant a release of core CIF that bundled the auxilliary files together with core CIF, so that they could be downloaded from a single location, just for convenience.

Understood, thanks.