jsonl-graph

🏝 JSONL Graph Tools

Programming language: Go

License: MIT License

Tags: Science And Data Analysis    

README

JSONL Graph Tools

Convenient to use with jq

# get https://graphviz.org/download/ 
$ go install github.com/nikolaydubina/jsonl-graph@latest

What is JSONL graph? Node has id. Edge has from and to.

{
    "id": "github.com/gin-gonic/gin",
    "can_get_github": true,
    "github_url": "https://github.com/gin-gonic/gin",
    "git_last_commit": "2021-04-21",
    "git_num_contributors": 321,
    ...
}
...
{
    "from": "github.com/gin-gonic/gin",
    "to": "golang.org/x/tools",
    ...
}

Examples

Kubernetes Pod Owners

$ kubectl get pods -o json | jq '.items[] | {to: (.kind + ":" + .metadata.name), from: (.metadata.ownerReferences[].kind + ":" + .metadata.ownerReferences[].name)}' | jsonl-graph | dot -Tsvg > k8s_pod_owners.svg

[k8s_pod_owners](./testdata/k8s_pod_owners.svg)

Large nodes and color scheme

$ cat '
{"id":"github.com/gin-gonic/gin","can_get_git":true, ... }
{"id":"github.com/gin-contrib/sse","can_get_git":true,"can_run_tests":true ... }
...
{"from":"github.com/gin-gonic/gin","to":"golang.org/x/tools"}
{"from":"github.com/gin-gonic/gin","to":"github.com/go-playground/validator/v10"}
' | jsonl-graph -color-scheme=file://$PWD/testdata/colors.json | dot -Tsvg > colored.svg

[gin-color](./testdata/gin_color.svg)

Small nodes or only edges

$ cat '
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"bufio"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"bytes"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"encoding/json"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"errors"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"fmt"}
...
' | jsonl-graph | dot -Tsvg > small.svg

[small](./testdata/small.svg)

All Kubernetes Pod Owners with details

# add edges
$ kubectl get pods -o json | jq '.items[] | {to: .metadata.name, from: .metadata.ownerReferences[].name}' > k8s_pod_owners_details.jsonl
# add node details
$ kubectl get rs -o json | jq '.items[] | .id += .metadata.name' >> k8s_pod_owners_details.jsonl
$ kubectl get pods -o json | jq '.items[] | .id += .metadata.name' >> k8s_pod_owners_details.jsonl
# flatten objects and render
$ cat k8s_pod_owners_details.jsonl | jq '. as $in | reduce leaf_paths as $path ({}; . + { ($path | map(tostring) | join(".")): $in | getpath($path) })' | jsonl-graph | dot -Tsvg > k8s_pod_owners.svg

[k8s_pod_owners_details](./testdata/k8s_pod_owners_details.svg)

Rendering

Currently only Graphviz is supported. Follow progress of native Go graph rendering in github.com/nikolaydubina/go-graph-layout. Once it is ready, it will be integrated into this project.