bpeabody · bpeabody · Mar 26, 2019 · May 22, 2019 · May 22, 2019 · Jun 4, 2019
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -0,0 +1,17 @@
+name: CI
+on: [push, pull_request, workflow_dispatch]
+jobs:
+  Test-Git-Meta:
+    runs-on: ubuntu-20.04
+    steps:
+      - run: echo "The job was automatically triggered by a ${{ github.event_name }} event."
+      - run: sudo apt-get install libkrb5-dev
+      - name: Check out repository code
+        uses: actions/checkout@v2
+      - run: git config --global user.name "A U Thor"
+      - run: git config --global user.email author@example.com
+      - run: yarn install
+        working-directory: ${{ github.workspace }}/node
+      - run: yarn test
+        working-directory: ${{ github.workspace }}/node
+      - run: echo "This job's status is ${{ job.status }}."
diff --git a/.jshintrc b/.jshintrc
@@ -12,7 +12,8 @@
     "undef"    : true,
     "node"     : true,
     "unused"   : true,
-    "esnext"   : true,
+    "esversion": 8,
+    "noyield"  : true,
     "varstmt"  : true,
     "predef": [
         "Intl"

diff --git a/.travis.yml b/.travis.yml
diff --git a/README.md b/README.md
@@ -33,6 +33,8 @@
 <img src="/doc/git-meta-logo.png" width="600">
 </p>
 
+[![Build Status](https://travis-ci.org/twosigma/git-meta.svg?branch=master)](https://travis-ci.org/twosigma/git-meta)
+
 # What is git-meta?
 
 Git-meta allows developers to work with extremely large codebases --

diff --git a/doc/architecture.md b/doc/architecture.md
@@ -108,23 +108,29 @@ products that take a similar approach:
 
 - [Gitslave](http://gitslave.sourceforge.net)
 - [myrepos](https://myrepos.branchable.com)
-- [Android Repo](https://source.android.com/source/using-repo.html)
+- [git-repo](https://gerrit.googlesource.com/git-repo/)
 - [gclient](http://dev.chromium.org/developers/how-tos/depottools#TOC-gclient)
 - [Git subtrees](https://git-scm.com/book/en/v1/Git-Tools-Subtree-Merging)
 - [Git submodules](https://git-scm.com/docs/git-submodule)
 
 All of these tools overlap with the problems git-meta is trying to solve, but
-none of them are sufficient:
+none of them is sufficient:
 
-- most don't provide a way to reference the state of all repositories
-  (Gitslave, Android Repo, Myrepos)
-- some require a custom server (Android Repo)
-- many are strongly focused on supporting a specific software platform (Android
-  Repo, gclient)
+- Most don't provide a way to reference the state of all repositories (Gitslave
+  and Myrepos). git-repo has the ability to reference the state of all repos,
+  but not in a way that can be used easily with normal Git commands (the state
+  is tracked in an XML file in a separate repository).
+- or are strongly focused on supporting a specific piece of software (gclient)
 - doesn't fully solve the scaling issue (Git subtrees)
 - prohibitively difficult to use (Git submodules)
 - lack scalable collaboration (e.g., pull request) strategies
 
+The git-repo project uses an approach that is structurally similar to the one
+used by git-meta: a (remote) meta-repo tracks the state of the sub-repos in an
+XML file.  It does not generally try to provide a full suite of
+cross-repository operations (such as `rebase`, `cherry-pick`, etc.) and assumes
+the use of the Gerrit code review tool.
+
 Git submodules come the closest: they do provide the technical ability to solve
 the problem, but are very difficult to use and lack some of the desired
 features.  With git-meta, we build on top of Git submodules to provide the
@@ -292,13 +298,13 @@ explain our high-level branching and tagging strategy.  Then, we define
 synthetic-meta-refs and discuss how they fit into git-meta workflows.  Finally,
 we present two variations on synthetic-meta-refs to help illustrate how the
 concept evolved; one of these variations, *mega-refs*, may prove necessary for
-old version of Git.
+old versions of Git.
 
 ## Branches and Tags
 
 In git-meta, branches and tags are applied only to meta-repos.  Because each
 commit in a meta-repo unambiguously describes the state of all sub-repos, it is
-unnecessary to apply branches and tags to sup-repos.  Furthermore, as described
+unnecessary to apply branches and tags to sub-repos.  Furthermore, as described
 in the "Design and Evolution" section below, schemes relying on sub-repo
 branches proved to be impractical.
 
@@ -322,7 +328,7 @@ of this strategy on client-side checkouts.
 
 A synthetic-meta-ref is a ref in a sub-repo whose name includes the commit ID
 of the commit to which it points, such as:
-`refs/meta/929e8afc03fef8d64249ad189341a4e8889561d7`.  The term is derived from
+`refs/commits/929e8afc03fef8d64249ad189341a4e8889561d7`.  The term is derived from
 the fact that such a ref is:
 
 1. _synthetic_ -- generated by a tool
@@ -341,35 +347,35 @@ A mono-repo has two invariants with respect to synthetic-meta-refs:
 Some mono-repos in valid states:
 
 ```
-'-------------------`  '-------------------`
-| meta-repo         |  |         a         |
-| - - - - - - - - - |  | - - - - - - - - - |
-|  master  | a [a1] |  | refs/meta/a1 [a1] |
-`-------------------,  `-------------------,
+'-------------------`  '-----------------------`
+| meta-repo         |  |           a           |
+| - - - - - - - - - |  | - - - - - - - - - - - |
+|  master  | a [a1] |  | refs/commits/a1 [a1]  |
+`-------------------,  `-----------------------,
 
 The 'master' branch in the meta-repo indicates commit 'a1' for repo 'a' and a
 valid synthetic-meta-ref exists.
 ```
 
 ```
-'-------------------`  '-------------------`
-| meta-repo         |  |         a         |
-| - - - - - - - - - |  | - - - - - - - - - |
-|  master  | a [a1] |  | refs/meta/a1 [a1] |
-| - - - - -+- - - - |  | refs/meta/ab [ab] |
-|  release | a [ab] |  `-------------------,
+'-------------------`  '-----------------------`
+| meta-repo         |  |           a           |
+| - - - - - - - - - |  | - - - - - - - - - - - |
+|  master  | a [a1] |  | refs/commits/a1 [a1]  |
+| - - - - -+- - - - |  | refs/commits/ab [ab]  |
+|  release | a [ab] |  `-----------------------,
 `-------------------,
 
 The meta-repo has another branch, 'release', indicating commit 'ab' in 'a',
 which also has a valid synthetic-meta-ref.
 ```
 
 ```
-'-----------------------`  '-------------------`
-| meta-repo             |  |         a         |
-| - - - - - - - - - - - |  | - - - - - - - - - |
-|  master  | a [a1]     |  | refs/meta/a2 [a2] |
-| - - - - -+- - - - - - |  `-------------------,
+'-----------------------`  '-----------------------`
+| meta-repo             |  |           a           |
+| - - - - - - - - - - - |  | - - - - - - - - - - - |
+|  master  | a [a1]     |  | refs/commits/a2 [a2]  |
+| - - - - -+- - - - - - |  `-----------------------,
 |  release | a [a2->a1] |
 `-----------------------,
 
@@ -381,31 +387,31 @@ for 'a1'.
 A few mono-repos in invalid states:
 
 ```
-'-------------------`  '-------------------`
-| meta-repo         |  |         a         |
-| - - - - - - - - - |  | - - - - - - - - - |
-|  master  | a [a1] |  | refs/meta/a1 [a2] |
-`-------------------,  `-------------------,
+'-------------------`  '-----------------------`
+| meta-repo         |  |           a           |
+| - - - - - - - - - |  | - - - - - - - - - - - |
+|  master  | a [a1] |  | refs/commits/a1 [a2]  |
+`-------------------,  `-----------------------,
 
 The synthetic-meta-ref for 'a1' does not point to 'a1'.
 ```
 
 ```
-'-------------------`  '-------------------`
-| meta-repo         |  |         a         |
-| - - - - - - - - - |  | - - - - - - - - - |
-|  master  | a [a1] |  | refs/meta/ab [ab] |
-`-------------------,  `-------------------,
+'-------------------`  '-----------------------`
+| meta-repo         |  |           a           |
+| - - - - - - - - - |  | - - - - - - - - - - - |
+|  master  | a [a1] |  | refs/commits/ab [ab]  |
+`-------------------,  `-----------------------,
 
 No synthetic-meta-ref for commit 'a1'.
 ```
 
 ```
-'-----------------------`  '-------------------`
-| meta-repo             |  |         a         |
-| - - - - - - - - - - - |  | - - - - - - - - - |
-|  master  | a [a1]     |  | refs/meta/a1 [a1] |
-| - - - - -+- - - - - - |  `-------------------,
+'-----------------------`  '-----------------------`
+| meta-repo             |  |           a           |
+| - - - - - - - - - - - |  | - - - - - - - - - - - |
+|  master  | a [a1]     |  | refs/commits/a1 [a1]  |
+| - - - - -+- - - - - - |  `-----------------------,
 |  release | a [a2->a1] |
 `-----------------------,
 
@@ -452,32 +458,32 @@ local
 `---------------------------------,
 
 remote
-'---------------------`  '--------------`  '--------------`
-| meta-repo  |        |  | a            |  | b            |
-| master     | a [a1] |  | refs/meta/a1 |  | refs/meta/b1 |
-| [m1]       | b [b1] |  |  [a1]        |  |  [b1]        |
-`---------------------,  `--------------,  `--------------,
+'---------------------`  '-----------------`  '-----------------`
+| meta-repo  |        |  | a               |  | b               |
+| master     | a [a1] |  | refs/commits/a1 |  | refs/commits/b1 |
+| [m1]       | b [b1] |  |  [a1]           |  |  [b1]           |
+`---------------------,  `-----------------,  `-----------------,
 ```
 Where we have new commits, `a2` and `b2` in repos `a` and `b`, respectively,
 and a new meta-repo commit, `m2` that references them.  Note that `a1` and `b1`
-have appropriate synthetic-meta-refs 
+have appropriate synthetic-meta-refs
 
 After invoking `git meta push`, the remote repos would look like:
 
 ```
-'---------------------`  '--------------`  '--------------`
-| meta-repo  |        |  | a            |  | b            |
-| master     | a [a2] |  | refs/meta/a1 |  | refs/meta/b1 |
-| [m2]       | b [b2] |  |  [a1]        |  |  [b1]        |
-`---------------------,  | refs/meta/a2 |  | refs/meta/b2 |
-                         |  [a2]        |  |  [b2]        |
-                         `--------------,  `--------------,
+'---------------------`  '-----------------`  '-----------------`
+| meta-repo  |        |  | a               |  | b               |
+| master     | a [a2] |  | refs/commits/a1 |  | refs/commits/b1 |
+| [m2]       | b [b2] |  |  [a1]           |  |  [b1]           |
+`---------------------,  | refs/commits/a2 |  | refs/commits/b2 |
+                         |  [a2]           |  |  [b2]           |
+                         `-----------------,  `-----------------,
 ```
 Note that `git meta push` created meta-refs in the sub-repos for the new
 commits before it updated the meta-repo.  If the process had been interrupted,
-for example, after pushing `refs/meta/a2` but before pushing `refs/meta/b2`,
+for example, after pushing `refs/commits/a2` but before pushing `refs/commits/b2`,
 the mono-repo would still be in a valid state.  If no meta-repo commit ever
-referenced `a2`, the synthetic-meta-ref `refs/meta/a2` would eventually be
+referenced `a2`, the synthetic-meta-ref `refs/commits/a2` would eventually be
 cleand up.
 
 ### Client-side access to sub-repo commits

diff --git a/doc/stitch.md b/doc/stitch.md
@@ -0,0 +1,69 @@
+<!--
+    Copyright (c) 2021, Two Sigma Open Source
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions are met:
+
+    * Redistributions of source code must retain the above copyright notice,
+      this list of conditions and the following disclaimer.
+
+    * Redistributions in binary form must reproduce the above copyright notice,
+      this list of conditions and the following disclaimer in the documentation
+      and/or other materials provided with the distribution.
+
+    * Neither the name of git-meta nor the names of its
+      contributors may be used to endorse or promote products derived from
+      this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+    AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+    IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+    ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+    LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+    CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+    SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+    INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+    CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+    ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+    POSSIBILITY OF SUCH DAMAGE.
+-->
+
+# Overview
+
+The stitcher stitches a git-meta repository into a single unified
+repository (perhaps leaving behind some submodules, depending on
+configuration).
+
+# Motivation
+
+You might want this because you are migrating away from git-meta.  Or
+you might just want a unified repo to power code search or code review
+tooling.
+
+There's also a destitched, which reverses the process.  Of course,
+that's a little trickier: if you create a new subdirectory which
+itself contains two subdirectories, how many sub*modules* do you
+create?
+
+# An implementation note
+
+The most efficient repository layout that we have yet discovered for
+stitching has three repositories:
+1. just the meta objects
+2. just the submodule objects
+3. the stitched commits, which has objects/info/alternates pointed at
+(1) and (2).
+
+This makes fetches from the meta repo faster, without making other
+fetches slower.
+
+If you can managed to remove the alternates for meta commits during
+pushes from the local unity repo to a remote one, that's even faster.
+
+The configuration variable gitmeta.stitchSubmodulesRepository can hold
+the path to the submodule-only repo; if it's present, the stitcher
+assumes that the alternates configuration is set up correctly.
+
+We have not yet considered the efficiency of destitching, and it does
+not support this configuration variable.
diff --git a/node/.gitignore b/node/.gitignore
@@ -1,2 +1,5 @@
 /node_modules/
 /coverage
+/yarn/
+.vscode
+
diff --git a/node/bin/destitch b/node/bin/destitch
@@ -0,0 +1,34 @@
+#!/usr/bin/env node
+/*
+ * Copyright (c) 2018, Two Sigma Open Source
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ *   this list of conditions and the following disclaimer.
+ *
+ * * Redistributions in binary form must reproduce the above copyright notice,
+ *   this list of conditions and the following disclaimer in the documentation
+ *   and/or other materials provided with the distribution.
+ *
+ * * Neither the name of git-meta nor the names of its
+ *   contributors may be used to endorse or promote products derived from
+ *   this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+require("./../lib/destitch.js");
+