Popularity
1.3
Declining
Activity
1.4
-
9
2
1

Programming language: Go
Tags: Utilities     Configuration     Go Tools     CLI     Json     Golang     Yaml     Toml    
Latest version: v1.1.0

CONFLATE alternatives and similar packages

Based on the "Configuration" category

Do you think we are missing an alternative of CONFLATE or a related project?

Add another 'Configuration' Package

README

CONFLATE

Library providing routines to merge and validate JSON, YAML, TOML files and/or structs (godoc)

Typical use case: Make your application configuration files multi-format, modular, templated, sparse, location-independent and validated

Build Status Coverage Status Go Report Card

Description

Conflate is a library and cli-tool, that provides the following features :

  • merge data from multiple formats (JSON/YAML/TOML/go structs) and multiple locations (filesystem paths and urls)
  • validate the merged data against a JSON schema
  • apply any default values defined in a JSON schema to the merged data
  • expand environment variables inside the data
  • marshal merged data to multiple formats (JSON/YAML/TOML/go structs)

It supports draft-04, draft-06 and draft-07 of JSON Schema. If the key $schema is missing, or the draft version is not explicitly set, a hybrid mode is used which merges together functionality of all drafts into one mode. Improvements, ideas and bug fixes are welcomed.

Getting started

Run the following command, which will build and install the latest binary in $GOPATH/bin

go get github.com/miracl/conflate/...

Alternatively, you can install one of the pre-built release binaries from https://github.com/miracl/conflate/releases

Usage of Library

Please refer to the godoc and the [example code](./example/main.go)

Usage of CLI Tool

Help can be obtained in the usual way :

$conflate --help
Usage of conflate:
  -data value
        The path/url of JSON/YAML/TOML data, or 'stdin' to read from standard input
  -defaults
        Apply defaults from schema to data
  -expand
        Expand environment variables in files
  -format string
        Output format of the data JSON/YAML/TOML
  -includes string
        Name of includes array. Blank string suppresses expansion of includes arrays (default "includes")
  -noincludes
        Switches off conflation of includes. Overrides any --includes setting.
  -schema string
        The path/url of a JSON v4 schema file
  -validate
        Validate the data against the schema
  -version
        Display the version number

To conflate the following file ... :

$cat ./testdata/valid_parent.json
{
  "includes": [
    "valid_child.json",
    "valid_sibling.json"
  ],
  "parent_only" : "parent",
  "parent_child" : "parent",
  "parent_sibling" : "parent",
  "all": "parent"
}

...run the following command, which will merge valid_parent.json, valid_child.json, valid_sibling.json :

$conflate -data ./testdata/valid_parent.json -format JSON
{
  "all": "parent",
  "child_only": "child",
  "parent_child": "parent",
  "parent_only": "parent",
  "parent_sibling": "parent",
  "sibling_child": "sibling",
  "sibling_only": "sibling"
}

Note how the includes are loaded remotely as relative paths.

Also, note values in a file override values in any included files, and that an included file overrides values in any included file above it in the includes list.

If you instead host a file somewhere else, then just use a URL :

$conflate -data https://raw.githubusercontent.com/miracl/conflate/master/testdata/valid_parent.json -format JSON
{
  "all": "parent",
  "child_only": "child",
  "parent_child": "parent",
  "parent_only": "parent",
  "parent_sibling": "parent",
  "sibling_child": "sibling",
  "sibling_only": "sibling"
}

The includes here are also loaded as relative urls and follow exactly the same merging rules.

To output in a different format use the -format option, e.g. TOML :

$conflate -data ./testdata/valid_parent.json -format TOML
all = "parent"
child_only = "child"
parent_child = "parent"
parent_only = "parent"
parent_sibling = "parent"
sibling_child = "sibling"
sibling_only = "sibling"

To additionally use defaults from a JSON schema and validate the conflated data against the schema, use -defaults and -validate respectively :

$cat ./testdata/blank.yaml

$conflate -data ./testdata/blank.yaml -schema ./testdata/test.schema.json -validate -format YAML
Schema validation failed : The document is not valid against the schema : Invalid type. Expected: object, given: null (#)

$conflate -data ./testdata/blank.yaml -schema ./testdata/test.schema.json -defaults -validate -format YAML
all: parent
child_only: child
parent_child: parent
parent_only: parent
parent_sibling: parent
sibling_child: sibling
sibling_only: sibling

Note any defaults are applied before validation is performed, as you would expect.

If you don't want to intrusively embed an "includes" array inside your JSON, you can instead provide multiple data files which are processed from left-to-right :

$conflate -data ./testdata/valid_child.json -data ./testdata/valid_sibling.json -format JSON
{
  "all": "sibling",
  "child_only": "child",
  "parent_child": "child",
  "parent_sibling": "sibling",
  "sibling_child": "sibling",
  "sibling_only": "sibling"
}

Or alternatively, you can create a top-level JSON file containing only the includes array. For fun, lets choose to use YAML for the top-level file, and output TOML :

$cat toplevel.yaml
includes:
  - testdata/valid_child.json
  - testdata/valid_sibling.json

$conflate -data toplevel.yaml -format TOML
all = "sibling"
child_only = "child"
parent_child = "child"
parent_sibling = "sibling"
sibling_child = "sibling"
sibling_only = "sibling"

If you want to read a file from stdin you can do the following. Here we pipe in some TOML to override a value to demonstrate :

$echo 'all="MY OVERRIDDEN VALUE"' |  conflate -data ./testdata/valid_parent.json -data stdin  -format JSON
{
  "all": "MY OVERRIDDEN VALUE",
  "child_only": "child",
  "parent_child": "parent",
  "parent_only": "parent",
  "parent_sibling": "parent",
  "sibling_child": "sibling",
  "sibling_only": "sibling"
}

Note that in all cases -data sources are processed from left-to-right, with values in right files overriding values in left files, so the following doesn't work :

$echo 'all="MY OVERRIDDEN VALUE"' |  conflate -data stdin -data ./testdata/valid_parent.json  -format JSON
{
  "all": "parent",
  "child_only": "child",
  "parent_child": "parent",
  "parent_only": "parent",
  "parent_sibling": "parent",
  "sibling_child": "sibling",
  "sibling_only": "sibling"
}

You can optionally expand environment variables in the files like this :

$export echo MYVALUE="some value"
$export echo MYJSONMAP='{ "item1" : "value1" }'
$echo '{ "my_value": "$MYVALUE", "my_map": $MYJSONMAP }' | conflate -data stdin -expand -format JSON
{
  "my_map": {
    "item1": "value1"
  },
  "my_value": "some value"
}

Acknowledgements

Images derived from originals by Renee French https://golang.org/doc/gopher/