Popularity
4.3
Growing
Activity
8.7
-
99
6
6

Programming language: Go
License: MIT License
Tags: Stream Processing    
Latest version: v0.9.8

machine alternatives and similar packages

Based on the "Stream Processing" category.
Alternatively, view machine alternatives based on common mentions on social networks and blogs.

Do you think we are missing an alternative of machine or a related project?

Add another 'Stream Processing' Package

README

Go PkgGoDev GoDoc Go Report Card Codacy Badge Codacy Badge Version Badge

Machine is a library for creating data workflows. These workflows can be either very concise or quite complex, even allowing for cycles for flows that need retry or self healing mechanisms.

It supports opentelemetry spans and metrics out of the box

It also supports building dynamic pipelines using

Components is a repository of different vertex and plugin implementations


Installation

Add the primary library to your project

  go get -u github.com/whitaker-io/machine

Data is a library for getting and setting values in a map[string]interface{}


Documentation

[Docs](./docs)

  • [Stream](./docs/01_Stream.md)
  • [Map](./docs/02_Map.md)
  • [Fork](./docs/03_Fork.md)
  • [Fold](./docs/04_Fold.md)
  • [Loop](./docs/07_Loop.md)
  • [Sort](./docs/09_Sort.md)
  • [Remove](./docs/10_Remove.md)
  • [Publish](./docs/06_Publisher.md)
  • [Plugins](./docs/08_Plugins.md)

Example

Basic receive -> process -> send Flow

  stream := NewStream("unique_id1", 
    func(c context.Context) chan []Data {
      channel := make(chan []Data)

      // setup channel to collect data as long as 
      // the context has not completed

      return channel
    },
    &Option{FIFO: boolP(false)},
    &Option{Metrics: boolP(true)},
    &Option{Span: boolP(false)},
  )

  stream.Builder().Map("unique_id2", 
      func(m Data) error {
        var err error

        // ...do some processing

        return err
      },
    ).Publish("publish_left_id", publishFN(func(d []data.Data) error {
      // send the data somewhere

      return nil
    }),
  )

  if err := stream.Run(context.Background()); err != nil {
    // Run will return an error in the case that 
    // one of the paths is not terminated (i.e. missing a Publish)
    panic(err)
  }

🤝 Contributing

Contributions, issues and feature requests are welcome. Feel free to check issues page if you want to contribute. [Check the contributing guide](./CONTRIBUTING.md).

Author

👤 Jonathan Whitaker

Show your support

Please ⭐️ this repository if this project helped you!


License

Machine is provided under the MIT License.

The MIT License (MIT)

Copyright (c) 2020 Jonathan Whitaker


*Note that all licence references and agreements mentioned in the machine README section above are relevant to that project's source code only.