Popularity
9.3
Stable
Activity
7.2
-
5,876
469
596

Description

Serf is a decentralized solution for service discovery and orchestration that is lightweight, highly available, and fault tolerant.

Serf runs on Linux, Mac OS X, and Windows. An efficient and lightweight gossip protocol is used to communicate with other nodes. Serf can detect node failures and notify the rest of the cluster. An event system is built on top of Serf, letting you use Serf's gossip protocol to propagate events such as deploys, configuration changes, etc. Serf is completely masterless with no single point of failure.

Here are some example use cases of Serf, though there are many others:

Programming language: Go
License: Mozilla Public License 2.0
Latest version: v0.9.5

Serf alternatives and similar packages

Based on the "Distributed Systems" category.
Alternatively, view serf alternatives based on common mentions on social networks and blogs.

  • go-micro

    DISCONTINUED. A Go microservices framework [Moved to: https://github.com/micro/go-micro]
  • grpc-go

    The Go language implementation of gRPC. HTTP/2 based RPC
  • Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
    Promo coderabbit.ai
    CodeRabbit Logo
  • Nomad

    Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
  • go-zero

    DISCONTINUED. go-zero is a web and rpc framework written in Go. It's born to ensure the stability of the busy sites with resilient design. Builtin goctl greatly improves the development productivity. [Moved to: https://github.com/zeromicro/go-zero]
  • micro

    An API first development platform
  • rpcx

    Best microservices framework in Go, like alibaba Dubbo, but with more features, Scale easily. Try it. Test it. If you feel it's better, use it! ๐‰๐š๐ฏ๐šๆœ‰๐๐ฎ๐›๐›๐จ, ๐†๐จ๐ฅ๐š๐ง๐ ๆœ‰๐ซ๐ฉ๐œ๐ฑ! build for cloud!
  • raft

    Golang implementation of the Raft consensus protocol
  • tendermint

    โŸ Tendermint Core (BFT Consensus) in Go
  • ringpop-go

    Scalable, fault-tolerant application-layer sharding for Go applications
  • Encore

    Open Source Development Platform for building robust type-safe distributed systems with declarative infrastructure
  • Kitex

    Go RPC framework with high-performance and strong-extensibility for building micro-services.
  • KrakenD

    Ultra performant API Gateway with middlewares. A project hosted at The Linux Foundation
  • torrent

    Full-featured BitTorrent client package and utilities
  • dht

    Full-featured BitTorrent client package and utilities
  • dragonboat

    A feature complete and high performance multi-group Raft library in Go.
  • Dkron

    Dkron - Distributed, fault tolerant job scheduling system https://dkron.io
  • emitter-io

    High performance, distributed and low latency publish-subscribe platform.
  • gleam

    Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
  • DHT

    BitTorrent DHT Protocol && DHT Spider.
  • glow

    Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
  • Olric

    Distributed, in-memory key/value store and cache. It can be used as an embedded Go library and a language-independent service.
  • Dragonfly

    Dragonfly is an open source P2P-based file distribution and image acceleration system. It is hosted by the Cloud Native Computing Foundation (CNCF) as an Incubating Level Project.
  • liftbridge

    Lightweight, fault-tolerant message streams.
  • gocelery

    Celery Distributed Task Queue in Go
  • go-doudou

    go-doudou๏ผˆdoudou pronounce /dษ™udษ™u/๏ผ‰is OpenAPI 3.0 (for REST) spec and Protobuf v3 (for grpc) based lightweight microservice framework. It supports monolith service application as well.
  • hprose

    Hprose is a cross-language RPC. This project is Hprose for Golang.
  • redis-lock

    Simplified distributed locking implementation using Redis
  • go-health

    Library for enabling asynchronous health checks in your service
  • arpc

    More effective network communication, two-way calling, notify and broadcast supported.
  • rain

    ๐ŸŒง BitTorrent client and library in Go
  • trpc-go

    A pluggable, high-performance RPC framework written in golang
  • Temporal

    Temporal Go SDK
  • gorpc

    Simple, fast and scalable golang rpc library for high load
  • resgate

    A Realtime API Gateway used with NATS to build REST, real time, and RPC APIs, where all your clients are synchronized seamlessly.
  • consistent

    Consistent hashing with bounded loads in Golang
  • digota

    ecommerce microservice
  • go-sundheit

    A library built to provide support for defining service health for golang services. It allows you to register async health checks for your dependencies and the service itself, provides a health endpoint that exposes their status, and health metrics.
  • go-jump

    go-jump: Jump consistent hashing
  • sleuth

    A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services
  • jsonrpc

    The jsonrpc package helps implement of JSON-RPC 2.0
  • dynamolock

    DynamoDB Lock Client for Go
  • outboxer

    A library that implements the outboxer pattern in go
  • Maestro

    Take control of your data, connect with anything, and expose it anywhere through protocols such as HTTP, GraphQL, and gRPC.
  • doublejump

    A revamped Google's jump consistent hash
  • dot

    distributed data sync with operational transformation/transforms
  • celeriac

    Golang client library for adding support for interacting and monitoring Celery workers, tasks and events.
  • drmaa

    Compute cluster (HPC) job submission library for Go (#golang) based on the open DRMAA standard.
  • go-mysql-lock

    MySQL Backed Locking Primitive
  • flowgraph

    Flowgraph package for scalable asynchronous system development

Do you think we are missing an alternative of Serf or a related project?

Add another 'Distributed Systems' Package

README

Serf CircleCI Join the chat at https://gitter.im/hashicorp-serf/Lobby

Serf is a decentralized solution for service discovery and orchestration that is lightweight, highly available, and fault tolerant.

Serf runs on Linux, Mac OS X, and Windows. An efficient and lightweight gossip protocol is used to communicate with other nodes. Serf can detect node failures and notify the rest of the cluster. An event system is built on top of Serf, letting you use Serf's gossip protocol to propagate events such as deploys, configuration changes, etc. Serf is completely masterless with no single point of failure.

Here are some example use cases of Serf, though there are many others:

  • Discovering web servers and automatically adding them to a load balancer
  • Organizing many memcached or redis nodes into a cluster, perhaps with something like twemproxy or maybe just configuring an application with the address of all the nodes
  • Triggering web deploys using the event system built on top of Serf
  • Propagating changes to configuration to relevant nodes.
  • Updating DNS records to reflect cluster changes as they occur.
  • Much, much more.

Quick Start

First, download a pre-built Serf binary for your operating system, compile Serf yourself, or install using go get -u github.com/hashicorp/serf/cmd/serf.

Next, let's start a couple Serf agents. Agents run until they're told to quit and handle the communication of maintenance tasks of Serf. In a real Serf setup, each node in your system will run one or more Serf agents (it can run multiple agents if you're running multiple cluster types. e.g. web servers vs. memcached servers).

Start each Serf agent in a separate terminal session so that we can see the output of each. Start the first agent:

$ serf agent -node=foo -bind=127.0.0.1:5000 -rpc-addr=127.0.0.1:7373
...

Start the second agent in another terminal session (while the first is still running):

$ serf agent -node=bar -bind=127.0.0.1:5001 -rpc-addr=127.0.0.1:7374
...

At this point two Serf agents are running independently but are still unaware of each other. Let's now tell the first agent to join an existing cluster (the second agent). When starting a Serf agent, you must join an existing cluster by specifying at least one existing member. After this, Serf gossips and the remainder of the cluster becomes aware of the join. Run the following commands in a third terminal session.

$ serf join 127.0.0.1:5001
...

If you're watching your terminals, you should see both Serf agents become aware of the join. You can prove it by running serf members to see the members of the Serf cluster:

$ serf members
foo    127.0.0.1:5000    alive
bar    127.0.0.1:5001    alive
...

At this point, you can ctrl-C or force kill either Serf agent, and they'll update their membership lists appropriately. If you ctrl-C a Serf agent, it will gracefully leave by notifying the cluster of its intent to leave. If you force kill an agent, it will eventually (usually within seconds) be detected by another member of the cluster which will notify the cluster of the node failure.

Documentation

Full, comprehensive documentation is viewable on the Serf website:

https://www.serf.io/docs

Developing Serf

If you wish to work on Serf itself, you'll first need Go installed (version 1.10+ is required). Make sure you have Go properly installed, including setting up your GOPATH.

Next, clone this repository into $GOPATH/src/github.com/hashicorp/serf and then just type make. In a few moments, you'll have a working serf executable:

$ make
...
$ bin/serf
...

NOTE: make will also place a copy of the executable under $GOPATH/bin/

Serf is first and foremost a library with a command-line interface, serf. The Serf library is independent of the command line agent, serf. The serf binary is located under cmd/serf and can be installed stand alone by issuing the command go get -u github.com/hashicorp/serf/cmd/serf. Applications using the Serf library should only need to include github.com/hashicorp/serf.

Tests can be run by typing make test.

If you make any changes to the code, run make format in order to automatically format the code according to Go standards.