Braid

Braid is a simple tool to help track vendor branches in a
Git repository.

Motivation

Vendoring allows you take the source code of an external library and ensure it’s
version controlled along with the main project. This is in contrast to including
a reference to a packaged version of an external library that is available in a
binary artifact repository such as Maven Central, RubyGems or NPM.

Vendoring is useful when you need to patch or customize the external libraries
or the external library is expected to co-evolve with the main project. The
developer can make changes to the main project and patch the library in a single
commit.

The problem arises when the external library makes changes that you want to
integrate into your local vendored version or the developer makes changes to the
local version that they want integrated into the external library.

A typical “implementation” of vendoring is to simply download or checkout the
source for the external library, remove the .git or .svn directories and
commit it to the main source tree. However this approach makes it very difficult
to update the library. When you want to update the library do you re-apply your
local changes onto a new copy of the vendored library or do you re-apply the
changes from the external library to local version? Both cases involve manual
generation and application of patch files to source trees.

This is where Braid comes into play. Braid makes it easy to vendor in remote git
repositories and use an automated mechanism for updating the external library
and generating patches to upgrade the external library.

Braid creates a file .braids.json in the root of your repository that contains
references to external libraries or mirrors. The configuration allows you to control
aspects of the mirroring process such as;

• whether the mirror is locked to a particular version of the external library.
• whether the mirror is tracking a tag or a branch.
• whether the mirror includes the entire external library or just a subdirectory.

Installation

gem install braid

Quick usage - ruby project

Let’s assume we’re writing the project myproject that needs grit in lib/grit. Initialize the repo (nothing braid related here):

git init myproject
cd myproject
touch README
git add README
git commit -m "initial commit"

Now let’s vendor grit:

braid add git://github.com/mojombo/grit.git lib/grit

And you’re done! Braid vendored grit into lib/grit. Feel free to inspect the changes with git log or git show.

If further down the line, you want to bring new changes from grit into your repository, just update the mirror:

braid update lib/grit

If you make changes to the grit library and want to generate a patch file so that you can submit the patch file
to the grit project:

braid diff lib/grit > grit.patch

Alternatively you can push changes back to the source directory directly using the following command. The command
will push the changes to the branch myproject_customizations that has been branched off the source branch (master
in this example). Omit the --branch argument to push directly to the source branch.

braid push lib/grit --branch myproject_customizations

Once those changes have been applied to grit you probably want to update your local version of grit again.

braid update lib/grit

More usage

Use the built in help system to find out about all commands and options:

braid help
braid help add # or braid add --help

Examples

Adding a mirror

braid add git://github.com/rails/rails.git vendor/rails

Adding a subdirectory from a mirror

This is useful if you want to add a subdirectory from a mirror into your own project.

braid add --path dist https://github.com/twbs/bootstrap.git vendor/assets/bootstrap

Adding a mirror based on a branch

braid add --branch 5-0-stable https://github.com/rails/rails.git vendor/rails

Adding a mirror based on a tag

braid add --tag v1.0 https://github.com/realityforge/backpack.git vendor/tools/backpack

Adding mirror locked to a revision

braid add --revision bf1b1e0 git://github.com/rails/rails.git vendor/rails

Updating mirrors

# Update a specific mirror
braid update vendor/plugins/cache_fu
# Update all mirrors
braid update

Updating mirrors with conflicts

If a braid update creates a conflict, braid will stop execution and leave the partially committed
files in your working copy, just like a normal git merge conflict would.

You will then have to resolve all conflicts and manually run git commit. The commit message is
already prepared.

If you want to cancel the braid update and the merge, you’ll have to reset your working copy and
index with git reset --hard.

Locking and unlocking mirrors

Lock to a particular version in the mirror.

braid update --revision 6c1c16b vendor/rails

Go back to tracking a particular branch.

braid update --branch master vendor/rails

Showing local changes made to mirrors

braid diff vendor/rails

License compliance for vendored content

When you distribute a repository that includes vendored content (managed by
Braid or any other tool), you are responsible for complying with the license for
that content. Usually, one of the requirements is to include a copy of the
license in your repository. If you vendor the entire upstream repository and the
license is in there, then you’ve already met this requirement. However, if you
vendor a subdirectory or single file from the upstream repository, you may need
to add the license separately. One solution is to add the license file as its
own Braid mirror. (Here is an example .braids.json
file
of a project that took this approach.)

Note that if the upstream license ever changes, it’s your responsibility to keep
your copy of the license in sync with the version of the content in your
repository. Normally, that would mean keeping the content and license mirrors at
the same upstream revision. If your workflow is to run braid update on all
mirrors, that should keep the two mirrors in sync, barring a race condition in
which the upstream repository is updated while braid update is running.
However, if you update the content mirror individually, it would be easy to
forget to update the license mirror. We have an idea about how to solve this
problem in Braid, but it would add significant complexity, so we’re not
implementing it until there’s enough evidence of demand from users. See this
issue for more information or
to get involved.

Supported environments

As of this writing (2022-01-20), we try to keep Braid working at least on Linux,
OSX and Windows with recent versions of its dependencies (Git, Ruby, gems, etc.).
Your mileage on other operating systems or with other versions of dependencies
may vary. We don’t have a procedure in place to systematically test Braid in
multiple environments; typically, Braid developers just run the test suite on
their own systems with whatever is installed. So breakages may sometimes occur.
If you run into an environment-related problem, please report it and we’ll fix
it if feasible. Contributions to improve testing of Braid would be welcome.

Environment-specific notes

• In our testing on Windows with RubyInstaller as of 2024-11-03, we ran into an
  issue where running bundle install in the Braid repository failed to install
  the native extension of the psych gem. We found two workarounds that appear
  to work: (1) use gem install -g instead, or (2) run ridk exec pacman -Syu mingw-w64-ucrt-x86_64-libyaml to install the missing native library (this may
  need to be run twice if the first run exits early due to an MSYS2 core system
  upgrade). We don’t have a good understanding of what’s going on here and
  don’t want to spend the time to research it, but we’re providing this
  information in case it helps anyone. If you know more about this issue,
  please tell us.

Braid version compatibility

Since Braid has been regularly changing the configuration format and adding new
features that some projects may choose to rely on, and somewhat less often
making breaking changes in how the configuration is handled, problems can arise
if different developers work on the same project using different versions of
Braid. Since version 1.1.0, Braid refuses to operate if it detects potentially
problematic version skew. If this happens, Braid will tell you what you can do.
If you’d like an overview of what to expect, read on.

Roughly speaking, the .braids.json configuration file contains a configuration
version number that corresponds to a range of compatible Braid minor versions
(x.y). “Patch” upgrades to Braid (i.e., x.y.z -> x.y.(z+1)) will never
(intentionally!) have configuration compatibility implications and are always
recommended as they may fix critical bugs.

If you use a Braid version too old for your configuration file, Braid will
direct you to the configuration version history page with
instructions to upgrade Braid. If you use a Braid version too new, Braid will
tell you how you can upgrade your configuration file or find a compatible older
Braid version to use. (As an exception, a newer version of Braid can run
read-only commands on an older configuration file without upgrading it if there
are no breaking changes.) If you upgrade your configuration file, then other
developers on the project may need to upgrade Braid. Braid does not support
downgrading a configuration file, though you can revert the commit that upgraded
it if you haven’t made any subsequent changes to the configuration.

If you work on multiple projects, you may need to install multiple versions of
Braid and manually run the correct version for each project. Fortunately, the
RubyGems system makes this reasonably straightforward.

Another approach is to standardize the Braid version for a project by listing
Braid in a Gemfile (either checking in Gemfile.lock or using a version
constraint in the Gemfile) and run the project’s version of Braid via
Bundler with bundle exec braid. Even non-Ruby projects
can do this if it’s acceptable to have a Gemfile and Gemfile.lock. Ruby
projects that don’t want Braid to interact with their other gems can potentially
put the Gemfile in a subdirectory and provide a wrapper script for bundle
that sets the BUNDLE_GEMFILE environment variable. We do not yet have enough
experience with this approach to make a firm recommendation for or against it.

This is the best design we could find to prevent surprises and adequately
support normal development processes while minimizing the additional maintenance
cost of the version compatibility mechanism. We want to have a scheme in place
that is robust enough to make it reasonable to encourage serious adoption of
Braid, yet we don’t want to spend extra work adding conveniences until there’s
evidence of sufficient demand for them.

Contributing

We appreciate any patches, error reports and usage ideas you may have. Please
submit an issue or pull request on GitHub.

Subversion

While preparing to release Braid v1.0 the support for subversion repositories was removed as
there was no active maintainers and inadequate test coverage. If there is anyone motivated to
re-add and maintain the Subversion support, please contact the authors.

Authors

• Cristi Balan
• Norbert Crombach
• Peter Donald
• Matt McCutchen

Contributors (alphabetically)

• Alan Harper
• Brad Durrow
• Christoph Sturm
• Dennis Muhlestein
• Ferdinand Svehla
• Michael Klishin
• Roman Heinrich
• Travis Tilley
• Tyler Rick