Gollum v0.5.0 Release Notes
Release Date: 2017-12-21 // over 6 years ago-
Gollum 0.5.0 contains major breaking changes in all areas. ๐ง Configuration files working with Gollum 0.4.x will not work with this verison unless changed. ๐ Please have a look at the transition guide for details.
Important note: When switching a pipline from 0.4.x to 0.5.0, make sure all spooling data has been read. Messages serialized to disk with 0.4.x are not compatible with 0.5.0.
๐ New with 0.5.0
- ๐ Filters and Formatters have been merged into one list
- ๐ You can now use a filter or formatter more than once in the same plugin
- Consumers can now do filtering and formatting, too
- ๐ Messages can now store metadata. Formatters can affect the payload or a metadata field
- ๐ All plugins now have an automatic log scope
- ๐ฐ Message payloads are now backed by a memory pool
- ๐ฐ Messages now store the original message, i.e. a backup of the payload state after consumer processing
- Gollum now provides per-stream metrics
- ๐ Plugins are now able to implement health checks that can be queried via http
- ๐ New base types for producers: Direct, Buffered, Batched
- ๐ Plugin configurations now support nested structures
- ๐ง The configuration process has been simplified a lot by adding automatic error handling and struct tags
- ๐ง All plugin configuration keys are now case insensitive
- โ Added a new formatter format.GrokToJSON
- โ Added a new formatter format.JSONToInflux10
- โ Added a new formatter format.Double
- โ Added a new formatter format.MetadataCopy
- โ Added a new formatter format.Trim
- ๐ Consumer.File now supports filesystem events
- Consumers can now define the number of go routines used for formatting/filtering
- ๐ All AWS plugins now support role switching
- ๐ All AWS plugins are now based on the same credentials code
๐ Fixed with 0.5.0
- ๐ The plugin lifecycle has been reimplemented to avoid gollum being stuck waiting for plugins to change state
- โ Integration test suite added
- ๐ Producer.HTTPRequest port handling fixed
- โ The test-config command will now produce more meaningful results
- Duplicating messages now properly duplicates the whole message and not just the struct
- ๐ Several race conditions have been fixed
- Producer.ElasticSearch is now based on a more up-to-date library
- Producer.AwsS3 is now behaving more like producer.File
- Gollum metrics can now bind to a specific address instead of just a port
๐ฅ Breaking changes with 0.5.0
- The config format has changed to improve automatic processing
- ๐ A lot of plugins have been renamed to avoid confusion and to better reflect their behavior
- ๐ A lot of plugins parameters have been renamed
- ๐ The instances plugin parameter has been removed
- Most of gollum's metrics have been renamed
- ๐ Plugin base types have been renamed
- All message handling function signatures have changed to use pointers
- All formatters don't daisy chain anymore as they can now be listed in proper order
- ๐ Stream plugins have been renamed to Router plugins
- Routers are not allowed to modify message content anymore
- ๐ filter.All and format.Forward have been removed as they are not required anymore
- ๐ Producer formatter listss dedicated to format a key or similar constructs have been removed
- ๐ฒ Logging framework switched to logrus
- ๐ฆ The package gollum.shared has been removed in favor of trivago.tgo
- ๐ Fuses have been removed from all plugins
- ๐ The general message sequence number has been removed
- The term "drop" has been replaced by the term "fallback" to emphasise it's use
- โฌ๏ธ The _DROPPED_ stream has been removed. Messages are discarded if no fallback is set
- Formatters can still the stream of a message but cannot trigger routing by themselves
- ๐ Compiling contrib plugins now requires a specific loader.go to be added
- ๐ณ The docker file on docker hub is now a lot smaller and only contains the gollum binary
- The message serialization format has been changed