Gollum v0.5.0 Release Notes

Release Date: 2017-12-21 // over 6 years ago
  • Gollum 0.5.0 contains major breaking changes in all areas. ๐Ÿ”ง Configuration files working with Gollum 0.4.x will not work with this verison unless changed. ๐Ÿš€ Please have a look at the transition guide for details.

    Important note: When switching a pipline from 0.4.x to 0.5.0, make sure all spooling data has been read. Messages serialized to disk with 0.4.x are not compatible with 0.5.0.

    ๐Ÿ†• New with 0.5.0

    • ๐Ÿ”€ Filters and Formatters have been merged into one list
    • ๐Ÿ”Œ You can now use a filter or formatter more than once in the same plugin
    • Consumers can now do filtering and formatting, too
    • ๐Ÿ“‡ Messages can now store metadata. Formatters can affect the payload or a metadata field
    • ๐Ÿ”Œ All plugins now have an automatic log scope
    • ๐Ÿ›ฐ Message payloads are now backed by a memory pool
    • ๐Ÿ›ฐ Messages now store the original message, i.e. a backup of the payload state after consumer processing
    • Gollum now provides per-stream metrics
    • ๐Ÿ”Œ Plugins are now able to implement health checks that can be queried via http
    • ๐Ÿ†• New base types for producers: Direct, Buffered, Batched
    • ๐Ÿ”Œ Plugin configurations now support nested structures
    • ๐Ÿ”ง The configuration process has been simplified a lot by adding automatic error handling and struct tags
    • ๐Ÿ”ง All plugin configuration keys are now case insensitive
    • โž• Added a new formatter format.GrokToJSON
    • โž• Added a new formatter format.JSONToInflux10
    • โž• Added a new formatter format.Double
    • โž• Added a new formatter format.MetadataCopy
    • โž• Added a new formatter format.Trim
    • ๐Ÿ‘ Consumer.File now supports filesystem events
    • Consumers can now define the number of go routines used for formatting/filtering
    • ๐Ÿ”Œ All AWS plugins now support role switching
    • ๐Ÿ”Œ All AWS plugins are now based on the same credentials code

    ๐Ÿ›  Fixed with 0.5.0

    • ๐Ÿ”Œ The plugin lifecycle has been reimplemented to avoid gollum being stuck waiting for plugins to change state
    • โœ… Integration test suite added
    • ๐Ÿ›  Producer.HTTPRequest port handling fixed
    • โœ… The test-config command will now produce more meaningful results
    • Duplicating messages now properly duplicates the whole message and not just the struct
    • ๐Ÿ›  Several race conditions have been fixed
    • Producer.ElasticSearch is now based on a more up-to-date library
    • Producer.AwsS3 is now behaving more like producer.File
    • Gollum metrics can now bind to a specific address instead of just a port

    ๐Ÿ’ฅ Breaking changes with 0.5.0

    • The config format has changed to improve automatic processing
    • ๐Ÿ”Œ A lot of plugins have been renamed to avoid confusion and to better reflect their behavior
    • ๐Ÿ”Œ A lot of plugins parameters have been renamed
    • ๐Ÿšš The instances plugin parameter has been removed
    • Most of gollum's metrics have been renamed
    • ๐Ÿ”Œ Plugin base types have been renamed
    • All message handling function signatures have changed to use pointers
    • All formatters don't daisy chain anymore as they can now be listed in proper order
    • ๐Ÿ”Œ Stream plugins have been renamed to Router plugins
    • Routers are not allowed to modify message content anymore
    • ๐Ÿšš filter.All and format.Forward have been removed as they are not required anymore
    • ๐Ÿšš Producer formatter listss dedicated to format a key or similar constructs have been removed
    • ๐ŸŒฒ Logging framework switched to logrus
    • ๐Ÿ“ฆ The package gollum.shared has been removed in favor of trivago.tgo
    • ๐Ÿšš Fuses have been removed from all plugins
    • ๐Ÿšš The general message sequence number has been removed
    • The term "drop" has been replaced by the term "fallback" to emphasise it's use
    • โฌ‡๏ธ The _DROPPED_ stream has been removed. Messages are discarded if no fallback is set
    • Formatters can still the stream of a message but cannot trigger routing by themselves
    • ๐Ÿ”Œ Compiling contrib plugins now requires a specific loader.go to be added
    • ๐Ÿณ The docker file on docker hub is now a lot smaller and only contains the gollum binary
    • The message serialization format has been changed