johnny-deps alternatives and similar packages
Based on the "Package Management" category.
Alternatively, view johnny-deps alternatives based on common mentions on social networks and blogs.
-
godep
DISCONTINUED. dependency tool for go, godep helps build packages reproducibly by fixing their dependencies. -
gvt
DISCONTINUED. gvt is a simple vendoring tool made for Go native vendoring (aka GO15VENDOREXPERIMENT), based on gb-vendor. -
gigo
DISCONTINUED. PIP-like dependency tool for golang, with support for private repositories and hashes. -
go modules
Modules are the unit of source code interchange and versioning. The go command has direct support for working with modules, including recording and resolving dependencies on other modules.
SaaSHub - Software Alternatives and Reviews
Do you think we are missing an alternative of johnny-deps or a related project?
README
Johnny Deps
Johnny Deps is a small tool from VividCortex that
provides minimalistic dependency versioning for Go repositories using Git. Its
primary purpose is to help create reproducible builds when many import paths in
various repositories are required to build an application. It's based on a Perl
script that provides subcommands for retrieving or building a project, or
updating a dependencies file (called Godeps
), listing first-level imports for
a project.
Getting started
Install Johnny Deps by cloning the project's Github repository and running the provided scripts, like this:
git clone https://github.com/VividCortex/johnny-deps.git
cd johnny-deps
./configure --prefix=/your/path
make install
The --prefix
option to configure
is not mandatory; it defaults to
/usr/local
if not provided (but you'd have to install as root in that case).
The binary will end up at the bin
subdirectory, under the prefix you choose;
make sure you have that location specified in your PATH
.
Note that Perl is required, although that's probably provided by your system already. Also Go, Git and (if you're using makefiles) Make.
Dependencies
Johnny Deps is all about project dependencies. Each project should have a file called Godeps at its root, listing the full set of first-level dependencies; i.e., all repositories with Go packages imported directly by this project. The file may be omitted when empty and looks like this:
github.com/VividCortex/godaemon 2fdf3f9fa715a998e834f09e07a8070d9046bcfd
github.com/VividCortex/log 1ffbbe58b5cf1bcfd7a80059dd339764cc1e3bff
github.com/VividCortex/mysql f82b14f1073afd7cb41fc8eb52673d78f481922e
The first column identifies the dependency. The second is the commit identifier for the exact revision the current project depends upon. You can use any identifier Git would accept to checkout, including abbreviated commits, tags and branches. Note, however, that the use of branches is discouraged, cause it leads to non-reproducible builds as the tip of the branch moves forward.
Introducing the tool
jd
is Johnny Deps' main binary. It's a command line tool to retrieve projects
from Github, check dependencies, reposition local working copies according to
each project's settings, building and updating. It accepts subcommands much like
go
or git
do:
jd [global-options] [command] [options] [project]
Global options apply to all commands. Some allow you to change the external
tools that are used (go, git and make) in case you don't have them in your path,
or otherwise want to use a different version. There's also a -v
option to
increase verbosity, that you can provide twice for extra effect. (Note that the
tool runs silently by default, only displaying errors, if any.)
It's worth noting that all parameters are optional. If you don't specify a
command, it will default to build
(see "Building" below). If you don't specify
a project, jd
will try to infer the project based on your current working
path, and your setting for GOPATH
. If you're in a subdirectory of any of the
GOPATH
components, and you're also in a Git working tree, jd
would be happy
to fill up the project for you.
When in doubt, check jd help
.
Retrieving projects
Retrieving a Go application with Johnny Deps is just as easy as retrieving a
single base project. Run jd get
and the full application, with all transitive
dependencies, will be set up in your environment. Here's what we'd type for one
of our applications:
jd get github.com/VividCortex/api-hosts
Johnny Deps will look for all required projects in your GOPATH
, and download
those missing to the first component of GOPATH
. It will even create the
directory stated in your GOPATH
if it doesn't yet exist. As jd
traverses the
graph of dependencies, it checks whether version conflicts exist. If it happens
to detect one, it will abort with a message like this:
Version mismatch detected for github.com/VividCortex/core
561c9e9798307b875b8f90b89b7888eae4a983ce referenced by:
github.com/VividCortex/api-hosts
github.com/VividCortex/config
github.com/VividCortex/nap
but dfe3ff5362d778214272b56e2afcca0d96651911 referenced by github.com/VividCortex/shard
Here, the tool is telling you that two different versions of core
are being
included. The first is the commit identifier at the top, that is shared as a
dependency for the three projects that follow. But shard
, on the other hand,
is including a different commit for core
, shown at the last line. If no
version mismatch is found, you'll end up with all projects required to build the
application you were interested in (api-hosts
in the example above).
Besides retrieving required projects, jd get
will reposition local copies
(whether they existed already or were just cloned) to the version stated in
Godeps
files. Furthermore, if you're aiming at a specific commit (as
recommended), jd
does an extra effort trying to checkout a branch whose tip
matches that commit, as opposed to leaving you in a detached HEAD state. That's
most probably what you want, cause it's probably a work in progress and you'll
be adding commits to that branch. (If you prefer a detached HEAD instead,
provide the -d
flag to get
.)
When choosing a branch to checkout for a given commit identifier, jd
will
first search among all locals. If there's none whose tip matches the commit,
jd
will try remote tracking branches instead. Among those matching, jd
selects one with a local branch by the same name, having the remote as an
upstream branch. If there's one available, that remote branch is merged into the
local, and the latter is checked out for use. Otherwise, jd
keeps one of the
matching remotes with no local branch by the same name, and checks out a new
local branch with that remote set as upstream. (If local branches existed for
all candidate remotes, but none of them had the remote by the same name set as
upstream, then jd
would abort with an appropriate message. In that case you
should either review your local branches, cause there's possibly an upstream
setting missing, or otherwise use -d
to checkout in detached HEAD mode.) In
any case, if there's more than one choice and you're running with double -v
,
you'd get a message displaying the other options as well.
It's worth noting that jd
favors local operations as much as possible, to
avoid long round-trips to remote repositories. Hence, remotes won't be fetched
if the required revision is found locally. (That's particularly relevant when
including a branch name at the Godeps
file cause, if found locally, the branch
will not be updated with remote changes.) Note also that, unless it actually
needs to move to a different release, jd
will not insist in that your working
copy is clean. This is good from a developer's point of view, cause it allows
you to play with the application, trying modifications or fixes in the whole
code base, without jd
complaining.
If the project you're interested in is not present in GOPATH
, jd get
will
clone it from the remote repository and checkout the master branch. But once you
have a local copy, jd
will never checkout a different revision. (It will
change revisions for dependencies, but not for the main project you provide to
jd get
.) You may reposition the working copy to your liking using Git
commands; jd
will be happy to adjust dependencies accordingly. However, if you
want to force your main project into a specific revision, even before you have a
local copy, you can use the -r
parameter to jd get
, like so:
jd get github.com/VividCortex/api-hosts -r my-release
where the argument to -r
can be anything you can checkout from Git: a commit
identifier (abbreviated or not), a branch or a tag.
After working copies for all projects in the application are set, jd get
runs
a check on first level dependencies for the main project (i.e., the one either
you specified on the command line, or jd
inferred from your current
directory). The check is run against the result of go list
. jd
will complain
if the sets don't match exactly, displaying both missing and not required
projects. If that's the case, you need to fix your Godeps
file (see "Updating"
below).
Building
Since building is what you'll be doing most of the time, jd
conveniently
defaults to build
if no command is provided. Furthermore, jd
may be able to
retrieve the project out of your current working directory (see "Introducing the
tool" above). Hence, you'd typically be able to compile by typing only jd
at
the command prompt. Not even your location within the project tree matters; the
tool works equally fine if run from deep inside the hierarchy.
Before the actual building process, jd
runs the equivalent of a jd get
command. That's how it makes sure that you're actually using the correct
versions of all dependencies. (Keep in mind, though, that if your local copies
were already set to the correct revisions, it's okay to have local changes; even
in Godeps
files.) The implicit get
run by build
, and the choice of build
as the default command, make the tool particularly easy to use to build projects
you don't even have. The following command retrieves the full dependencies for
the application and builds:
jd github.com/VividCortex/api-hosts
Furthermore, since the -r
option to build
is actually passed along to the
implicit get
, you can readily set up a specific version by appending the
appropriate -r
to the command above. (The same behavior goes to the -d
option to build
.)
Johnny Deps calls go build
at the project's root to build. But, in order to
accommodate special needs, jd
checks for custom instructions first, resorting
to go build
only if there are none. The highest priority goes to the Make
utility. If there's a file called Makefile
or makefile
at the project root,
then make
is run. If an executable file called build
is found, then that
will be invoked instead. Otherwise the tool resorts to go build ./...
.
There's no need for a script if you need to go install ./...
instead. Johnny
Deps will use the later if you ask it to install
rather than build
. However,
keep in mind that this has to be explicitly asked for; jd
defaults to build
if no command is provided. The rest of the process works exactly like it does
for build
, including the attempts at make
and the build
script. Even
though jd
makes no difference between build
and install
when custom
scripts are used, the command name is made available in case different actions
are required in scripts themselves. This is published as the JD_ACTION
environment variable.
Updating dependencies
Johnny Deps can't decide which releases to use from the project's you import.
But it can help writing the Godeps
file. By running jd update
, jd
will
disregard the current dependencies in the Godeps
file, overwriting it with the
latest master release for each project you depend upon, after pulling. Of
course, that may or may not work. Using the latest release for each dependency
could potentially lead to inconsistencies (version mismatches), that would make
jd
complain. The dependencies file would have been changed anyway. It's your
responsibility to decide which projects to upgrade or withhold.
It's worth noting that jd update
does not rely on the Godeps
file to check
current dependencies; it takes them from go list
instead. A nice consequence
is that new imports are automatically detected from the code and added to
Godeps
with no manual intervention required. (And no longer needed imports
will be removed as well.) Note also that, although the old dependencies file is
overwritten, the new copy is not committed or even staged for commit in Git.
(Rationale: you should test that everything still works properly!) You can do
that with the rest of your changes, without leaving traces in history if you run
the update multiple times before you're done.
Return codes
These are the return codes for jd
:
- 0 - Success
- 1 - Error with parameters
- 2 - Bad dependencies or unable to read them
- 3 - Version mismatch detected
- 4 - External command failed
- 5 - Unable to checkout requested revision
Workflows
Johnny Deps is intentionally agnostic about the specific workflow used. In practice, people seem to fall into one of two camps that reflect how they think about dependency management, and their differing goals.
The first category, roughly speaking, is those who would like to build from the
tip of their source control repositories all the time, but have a need for
pinning some things to a specific version. Those users may use branch names in
Godeps
as opposed to commit identifiers, and change to a specific commit when
they need to pin a version. (Nevertheless, jd
will not automatically fetch the
latest changes. See "Retrieving projects".)
The second school of thought holds that the Godeps
file should contain
external dependencies and their exact versions, so that checking out a
particular revision of an application's repository and running jd
will result
in exactly the same versions of all of the code used to build the application,
every time.
At VividCortex, we want to be able to reproduce a binary for debugging or other
purposes. All of our builds have a command-line flag called --build-version
that, when present, will result in the binary printing out the Git revision from
which it was built. We can thus easily reproduce any version by calling jd
build
with that revision as the -r
parameter. To embed the revision in the
binary, we use a specific shell script called build
(see "Building" above)
that runs something like:
go build -ldflags "-X main.Godeps '$(git rev-parse HEAD)'"
At the application we set things up so that --build-version
displays the
contents of the main.Godeps
variable set by the compiler.
Contributing
We welcome issue reports, suggestions, and especially pull requests:
- Fork the project
- Write your code in a feature branch
- Add tests (if applicable)
- Run tests (always!)
- Commit, push and send Pull Request
Because this is a VividCortex internal tool that we're sharing publicly, we may not want to implement some features or fixes.
TODO
Add tests. Those we previously had are not appropriate for the new tool.
Optionally add support for other repositories, like Mercurial. This tool is now targeted at Git on github.com, that is what we use at VividCortex.
License
Copyright (c) 2013 VividCortex. Released under the MIT License. Read the LICENSE file for details.
Contributors
Johnny Deps is the combination of several different thought processes from multiple authors, with inspiration from tools such as Ruby's Bundler and dep gem, Python's pip, and others. Give credit to @xaprb and @gkristic.
*Note that all licence references and agreements mentioned in the johnny-deps README section above
are relevant to that project's source code only.