Popularity
6.5
Growing
Activity
5.5
Declining
401
7
80
Programming language: Go
License: Apache License 2.0
Tags:
Database
Latest version: v1.10
clickhouse-bulk alternatives and similar packages
Based on the "Database" category.
Alternatively, view clickhouse-bulk alternatives based on common mentions on social networks and blogs.
-
prometheus
The Prometheus monitoring system and time series database. -
tidb
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://tidbcloud.com/free-trial -
influxdb
Scalable datastore for metrics, events, and real-time analytics -
cockroach
CockroachDB - the open source, cloud-native distributed SQL database. -
Milvus
Vector database for scalable similarity search and AI applications. -
vitess
Vitess is a database clustering system for horizontal scaling of MySQL. -
groupcache
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases. -
TinyGo
Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM. -
rqlite
The lightweight, distributed relational database built on SQLite -
VictoriaMetrics
VictoriaMetrics: fast, cost-effective monitoring solution and time series database -
go-cache
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications. -
immudb
immudb - immutable database based on zero trust, SQL and Key-Value, tamperproof, data change history -
bytebase
Database CI/CD for DevOps teams. https://www.bytebase.com -
go-mysql-elasticsearch
Sync MySQL data into elasticsearch -
buntdb
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support -
rosedb
🚀 A high performance NoSQL database based on bitcask, supports string, list, hash, set, and sorted set. -
pREST
PostgreSQL ➕ REST, low-code, simplify and accelerate development, ⚡ instant, realtime, high-performance on any Postgres application, existing or new -
go-memdb
Golang in-memory database built on immutable radix trees -
xo
Command line tool to generate idiomatic Go code for SQL databases supporting PostgreSQL, MySQL, SQLite, Oracle, and Microsoft SQL Server -
tiedot
A rudimentary implementation of a basic document (NoSQL) database in Go -
nutsdb
A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set. -
cache2go
Concurrency-safe Go caching library with expiration capabilities and access counters -
GCache
An in-memory cache library for golang. It supports multiple eviction policies: LRU, LFU, ARC -
gocraft/dbr (database records)
Additions to Go's database/sql for super fast performance and convenience. -
fastcache
Fast thread-safe inmemory cache for big number of entries in Go. Minimizes GC overhead -
CovenantSQL
A decentralized, trusted, high performance, SQL database with blockchain features
Build time-series-based applications quickly and at scale.
InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.
Promo
www.influxdata.com
Do you think we are missing an alternative of clickhouse-bulk or a related project?
README
ClickHouse-Bulk
Simple Yandex ClickHouse insert collector. It collect requests and send to ClickHouse servers.
Installation
Download binary for you platorm
or
or from sources (Go 1.13+):
git clone https://github.com/nikepan/clickhouse-bulk
cd clickhouse-bulk
go build
Features
- Group n requests and send to any of ClickHouse server
- Sending collected data by interval
- Tested with VALUES, TabSeparated formats
- Supports many servers to send
- Supports query in query parameters and in body
- Supports other query parameters like username, password, database
- Supports basic authentication
For example:
INSERT INTO table3 (c1, c2, c3) VALUES ('v1', 'v2', 'v3')
INSERT INTO table3 (c1, c2, c3) VALUES ('v4', 'v5', 'v6')
sends as
INSERT INTO table3 (c1, c2, c3) VALUES ('v1', 'v2', 'v3')('v4', 'v5', 'v6')
Options
- -config - config file (json); default config.json
Configuration file
{
"listen": ":8124",
"flush_count": 10000, // check by \n char
"flush_interval": 1000, // milliseconds
"clean_interval": 0, // how often cleanup internal tables - e.g. inserts to different temporary tables, or as workaround for query_id etc. milliseconds
"remove_query_id": true, // some drivers sends query_id which prevents inserts to be batched
"dump_check_interval": 300, // interval for try to send dumps (seconds); -1 to disable
"debug": false, // log incoming requests
"dump_dir": "dumps", // directory for dump unsended data (if clickhouse errors)
"clickhouse": {
"down_timeout": 60, // wait if server in down (seconds)
"connect_timeout": 10, // wait for server connect (seconds)
"tls_server_name": "", // override TLS serverName for certificate verification (e.g. in cases you share same "cluster" certificate across multiple nodes)
"insecure_tls_skip_verify": false, // INSECURE - skip certificate verification at all
"servers": [
"http://127.0.0.1:8123"
]
}
}
Environment variables (used for docker image)
CLICKHOUSE_BULK_DEBUG
- enable debug loggingCLICKHOUSE_SERVERS
- comma separated list of serversCLICKHOUSE_FLUSH_COUNT
- count of rows for insertCLICKHOUSE_FLUSH_INTERVAL
- insert intervalCLICKHOUSE_CLEAN_INTERVAL
- internal tables clean intervalDUMP_CHECK_INTERVAL
- interval of resend dumpsCLICKHOUSE_DOWN_TIMEOUT
- wait time if server is downCLICKHOUSE_CONNECT_TIMEOUT
- clickhouse server connect timeoutCLICKHOUSE_TLS_SERVER_NAME
- server name for TLS certificate verificationCLICKHOUSE_INSECURE_TLS_SKIP_VERIFY
- skip certificate verification at all
Quickstart
./clickhouse-bulk
and send queries to :8124
Metrics
manual check main metrics
curl -s http://127.0.0.1:8124/metrics | grep "^ch_"
ch_bad_servers 0
- actual count of bad serversch_dump_count 0
- dumps saved from launchch_queued_dumps 0
- actual dump files id directorych_good_servers 1
- actual good servers countch_received_count 40
- received requests count from launchch_sent_count 1
- sent request count from launch
Tips
For better performance words FORMAT and VALUES must be uppercase.