migrate

Database migrations. CLI and Golang library.

Programming language: Go

License: GNU General Public License v3.0 or later

Tags: Database    

Latest version: v4.14.1

README

migrate

Database migrations written in Go. Use as CLI or import as library.

• Migrate reads migrations from sources and applies them in correct order to a database.
• Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
• Database drivers don't assume things or try to correct user input. When in doubt, fail.

Forked from mattes/migrate

Databases

Database drivers run migrations. [Add a new database?](database/driver.go)

• [PostgreSQL](database/postgres)
• [PGX](database/pgx)
• [Redshift](database/redshift)
• [Ql](database/ql)
• [Cassandra](database/cassandra)
• [SQLite](database/sqlite)
• [SQLite3](database/sqlite3) (todo #165)
• [SQLCipher](database/sqlcipher)
• [MySQL/ MariaDB](database/mysql)
• [Neo4j](database/neo4j)
• [MongoDB](database/mongodb)
• [CrateDB](database/crate) (todo #170)
• [Shell](database/shell) (todo #171)
• [Google Cloud Spanner](database/spanner)
• [CockroachDB](database/cockroachdb)
• [YugabyteDB](database/yugabytedb)
• [ClickHouse](database/clickhouse)
• [Firebird](database/firebird)
• [MS SQL Server](database/sqlserver)

Database URLs

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: !, #, $, %, &, ', (, ), *, +, ,, /, :, ;, =, ?, @, [, ]

It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")'
String to encode: FAKEpassword!#$%&'()*+,/:;=?@[]
FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D
$

Migration Sources

Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go)

• [Filesystem](source/file) - read from filesystem
• [io/fs](source/iofs) - read from a Go io/fs
• [Go-Bindata](source/go_bindata) - read from embedded binary data (jteeuwen/go-bindata)
• [pkger](source/pkger) - read from embedded binary data (markbates/pkger)
• [GitHub](source/github) - read from remote GitHub repositories
• [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories
• [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories
• [Gitlab](source/gitlab) - read from remote Gitlab repositories
• [AWS S3](source/aws_s3) - read from Amazon Web Services S3
• [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage

CLI usage

• Simple wrapper around this library.
• Handles ctrl+c (SIGINT) gracefully.
• No config search paths, no config files, no magic ENV var injections.

[CLI Documentation](cmd/migrate)

Basic usage

$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2

Docker usage

$ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate
    -path=/migrations/ -database postgres://localhost:5432/database up 2

Use in your Go project

• API is stable and frozen for this release (v3 & v4).
• Uses Go modules to manage dependencies.
• To help prevent database corruptions, it supports graceful stops via GracefulStop chan bool.
• Bring your own logger.
• Uses io.Reader streams internally for low memory overhead.
• Thread-safe and no goroutine leaks.

Go Documentation

import (
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/github"
)

func main() {
    m, err := migrate.New(
        "github://mattes:personal-access-token@mattes/migrate_test",
        "postgres://localhost:5432/database?sslmode=enable")
    m.Steps(2)
}

Want to use an existing database client?

import (
    "database/sql"
    _ "github.com/lib/pq"
    "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
)

func main() {
    db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable")
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    m, err := migrate.NewWithDatabaseInstance(
        "file:///migrations",
        "postgres", driver)
    m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run
}

Getting started

Go to [getting started](GETTING_STARTED.md)

Tutorials

• [CockroachDB](database/cockroachdb/TUTORIAL.md)
• [PostgreSQL](database/postgres/TUTORIAL.md)

(more tutorials to come)

Migration files

Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration)

1481574547_create_users_table.up.sql
1481574547_create_users_table.down.sql

[Best practices: How to write migrations.](MIGRATIONS.md)

Versions

Version	Supported?	Import	Notes
master	:white_check_mark:	import "github.com/golang-migrate/migrate/v4"	New features and bug fixes arrive here first
v4	:white_check_mark:	import "github.com/golang-migrate/migrate/v4"	Used for stable releases
v3	:x:	import "github.com/golang-migrate/migrate" (with package manager) or import "gopkg.in/golang-migrate/migrate.v3" (not recommended)	DO NOT USE - No longer supported

Development and Contributing

Yes, please! [Makefile](Makefile) is your friend, read the [development guide](CONTRIBUTING.md).

Also have a look at the [FAQ](FAQ.md).

---

Looking for alternatives? https://awesome-go.com/#database.