Description
This repository was originally built for tracedb database and later moved to a separate repository to make general use of it.
A large application that process million of requests may often run out of memory and you will notice performance degradation or in worst case scenario application will crash. So working on large scale application you need an accurate measurement of memory allocation. The bpool repository is built to limit excess memory usage and prevent CPU trashing.
bpool forked from tracedb alternatives and similar packages
Based on the "Utilities" category.
Alternatively, view bpool forked from tracedb alternatives based on common mentions on social networks and blogs.
-
hub
A command-line tool that makes git easier to use with GitHub. -
delve
Delve is a debugger for the Go programming language. -
项目文档
基于vite+vue3+gin搭建的开发基础平台(已完成setup语法糖版本),集成jwt鉴权,权限管理,动态路由,显隐可控组件,分页封装,多点登录拦截,资源权限,上传下载,代码生成器,表单生成器等开发必备功能,五分钟一套CURD前后端代码。 -
excelize
Go language library for reading and writing Microsoft Excel™ (XLAM / XLSM / XLSX / XLTM / XLTX) spreadsheets -
go-torch
Stochastic flame graph profiler for Go programs. -
goreleaser
Deliver Go binaries as fast and easily as possible -
xlsx
Go (golang) library for reading and writing XLSX files. -
Task
A task runner / simpler Make alternative written in Go -
godropbox
Common libraries for writing Go services/applications. -
godotenv
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.) -
hystrix-go
Netflix's Hystrix latency and fault tolerance library, for Go -
goreporter
A Golang tool that does static analysis, unit testing, code review and generate code quality report. -
go-funk
A modern Go utility library which provides helpers (map, find, contains, filter, ...) -
gorequest
GoRequest -- Simplified HTTP client ( inspired by nodejs SuperAgent ) -
mc
MinIO Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage. -
gojson
Automatically generate Go (golang) struct definitions from example JSON -
grequests
A Go "clone" of the great and famous Requests library -
Kopia
Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included. -
spinner
Go (golang) package with 90 configurable terminal spinner/progress indicators. -
filetype
Fast, dependency-free Go package to infer binary file types based on the magic numbers header signature -
create-go-app
✨ Create a new production-ready project with backend, frontend and deploy automation by running one CLI command! -
sling
A Go HTTP client library for creating and sending API requests -
mole
CLI application to create ssh tunnels focused on resiliency and user experience. -
boilr
:zap: boilerplate template manager that generates files or directories from template repositories -
beaver
💨 A real time messaging system to build a scalable in-app notifications, multiplayer games, chat apps in web and mobile apps. -
coop
Cheat sheet for some of the common concurrent flows in Go -
go-underscore
Helpfully Functional Go - A useful collection of Go utilities. Designed for programmer happiness. -
jump
Jump helps you navigate faster by learning your habits. ✌️ -
JobRunner
Framework for performing work asynchronously, outside of the request flow -
gentleman
Plugin-driven, extensible HTTP client toolkit for Go -
git-time-metric
Simple, seamless, lightweight time tracking for Git -
goreq
Minimal and simple request library for Go language. -
csvtk
A cross-platform, efficient and practical CSV/TSV toolkit in Golang -
mimetype
A fast Golang library for media type and file extension detection, based on magic numbers -
immortal
⭕ A *nix cross-platform (OS agnostic) supervisor
Less time debugging, more time building
Do you think we are missing an alternative of bpool forked from tracedb or a related project?
Popular Comparisons
README
Buffer pool prevent from excess memory usage and CPU trashing.
Quick Start
To import bpool from source code use go get command.
go get -u github.com/unit-io/bpool
Usage
Use buffer pool for writing incoming requests to buffer such as Put or Batch operations or use buffer pool while writing data to log file (during commit operation). The objective of creating BufferPool library with capacity is to perform initial writes to buffer without backoff until buffer pool reaches its target size. Buffer pool does not discard any Get or Write requests but it add gradual delay to it to limit the memory usage that can used for other operations such writing to log or db sync operations.
Detailed API documentation is available using the godoc.org service.
Make use of the client by importing it in your Go client source code. For example,
import "github.com/unit-io/bpool"
Following code snippet if executed without buffer capacity will consume all system memory and will cause a panic.
buf := bytes.NewBuffer(make([]byte, 0, 2))
defer func() {
if r := recover(); r != nil {
fmt.Println("panics from blast")
}
}()
for {
_, err := buf.Write([]byte("create blast"))
if err != nil {
fmt.Println(err.Error())
return
}
}
Code snippet to use BufferPool with capacity will limit usage of system memory by adding gradual delay to the requests and will not cause a panic.
pool := bpool.NewBufferPool(1<<20, &bpool.Options{MaxElapsedTime: 1 * time.Minute, WriteBackOff: true}) // creates BufferPool of 16MB target size
buf := pool.Get()
defer pool.Put(buf)
for {
_, err := buf.Write([]byte("create blast"))
if err != nil {
fmt.Println(err.Error())
return
}
}
New Buffer Pool
Use bpool.NewBufferPool() method and pass BufferSize parameter to create new buffer pool.
const (
BufferSize = 1<<30 // (1GB size)
)
pool := bpool.NewBufferPool(BufferSize, nil)
Get Buffer
To get buffer from buffer pool use BufferPool.Get(). When buffer pool reaches its capacity Get method runs with gradual delay to limit system memory usage.
....
var buffer *bpool.Buffer
buffer = pool.Get()
Writing to Buffer
To write to buffer use Buffer.Write() method.
var scratch [8]byte
binary.LittleEndian.PutUint64(scratch[0:8], uint64(buffer.Size()))
b.buffer.Write(scratch[:])
....
Reading from Buffer
To read buffer use Buffer.Bytes() method. This operation returns underline data slice stored into buffer.
data := buffer.Bytes()
...
Put Buffer to Pool
To put buffer to the pool when finished using buffer use BufferPool.Put() method, this operation resets the underline slice. It also resets the buffer pool interval that was used to delay the Get operation if capacity is below the target size.
pool.Put(buffer)
...
To reset the underline slice stored to the buffer and continue using the buffer use Buffer.Reset() method instead of using BufferPool.Put() operation.
buffer.Reset()
....
Contributing
If you'd like to contribute, please fork the repository and use a feature branch. Pull requests are welcome.
Licensing
This project is licensed under MIT License.
*Note that all licence references and agreements mentioned in the bpool forked from tracedb README section above
are relevant to that project's source code only.