Benthos changelog

Changelog History

Page 1

• v3.65.0 Changes

  March 07, 2022

  ➕ Added

  • 🆕 New sql_raw processor and output.

  🛠 Fixed

  • Corrected a case where nested parallel processors that result in emptied batches (all messages filtered) would propagate an unack rather than an acknowledgement.

  🔄 Changed

  • 🚚 The sql processor and output are no longer marked as deprecated and will therefore not be removed in V4. This change was made in order to provide more time to migrate to the new sql_raw processor and output.

• v3.64.0 Changes

  February 23, 2022

  ➕ Added

  • Field nack_reject_patterns added to the amqp_0_9 input.
  • 🆕 New experimental mongodb input.
  • 📜 Field cast added to the xml processor and parse_xml bloblang method.
  • New experimental gcp_bigquery_select processor.
  • 🆕 New assign bloblang method.
  • 👍 The protobuf processor now supports Any fields in protobuf definitions.
  • The azure_queue_storage input field queue_name now supports interpolation functions.

  🛠 Fixed

  • 🛠 Fixed an issue where manually clearing errors within a catch processor would result in subsequent processors in the block being skipped.
  • The cassandra output should now automatically match float columns.
  • 🛠 Fixed an issue where the elasticsearch output would collapse batched messages of matching ID rather than send as individual items.
  • 🚚 Running streams mode with --no-api no longer removes the /ready endpoint.

  🔄 Changed

  • 🗄 The throttle processor has now been marked as deprecated.

• v3.63.0 Changes

  February 08, 2022

  ➕ Added

  • 👍 Field cors added to the http_server input and output, for supporting CORS requests when custom servers are used.
  • Field server_side_encryption added to the aws_s3 output.
  • Field use_histogram_timing and histogram_buckets added to the prometheus metrics exporter.
  • 🆕 New duration string and back off field types added to plugin config builders.
  • Experimental field multipart added to the http_client output.
  • Codec regex added to inputs.
  • ⏱ Field timeout added to the cassandra output.
  • New experimental gcp_bigquery_select input.
  • Field ack_wait added to the nats_jetstream input.

  🔄 Changed

  • 💅 The old map-style resource config fields (resources.processors.<name>, etc) are now marked as deprecated. Use the newer list based fields (processor_resources, etc) instead.

  🛠 Fixed

  • 👍 The generate input now supports zeroed duration strings (0s, etc) for unbounded document creation.
  • The aws_dynamodb_partiql processor no longer ignores the endpoint field.
  • Corrected duplicate detection for custom cache implementations.
  • 🛠 Fixed panic caused by invalid bounds in the range function.
  • ✅ Resource config files imported now allow (and ignore) a tests field.
  • 🛠 Fixed an issue where the aws_kinesis input would fail to back off during unyielding read attempts.
  • 🛠 Fixed a linting error with zmq4 input/output urls fields that was incorrectly expecting a string.

• v3.62.0 Changes

  January 21, 2022

  ➕ Added

  • 🔀 Field sync added to the gcp_pubsub input.
  • 🆕 New input, processor, and output config field types added to the plugin APIs.
  • ➕ Added new experimental parquet processor.
  • 🆕 New Bloblang method format_json.
  • 👍 Field collection in mongodb processor and output now supports interpolation functions.
  • Field output_raw added to the jq processor.
  • The lambda distribution now supports a BENTHOS_CONFIG_PATH environment variable for specifying a custom config path.
  • 📇 Field metadata added to http and http_client components.
  • Field ordering_key added to the gcp_pubsub output.
  • A suite of new experimental geoip_ methods have been added.
  • ➕ Added flag --deprecated to the benthos lint subcommand for detecting deprecated fields.

  🔄 Changed

  • The sql processor and output have been marked deprecated in favour of the newer sql_insert, sql_select alternatives.

  🛠 Fixed

  • The input codec chunked is no longer capped by the packet size of the incoming streams.
  • The schema_registry_decode and schema_registry_encode processors now honour trailing slashes in the url field.
  • 🔧 Processors configured within pipeline.processors now share processors across threads rather than clone them.
  • 🔌 Go API: Errors returned from input/output plugin Close methods no longer cause shutdown to block.
  • 🔧 The pulsar output should now follow authentication configuration.
  • 🛠 Fixed an issue where the aws_sqs output might occasionally retry a failed message send with an invalid empty message body.

• v3.61.0 Changes

  December 28, 2021

  ➕ Added

  • Field json_marshal_mode added to the MongoDB processor.
  • Fields extract_headers.include_prefixes and extract_headers.include_patterns added to the http_client input and output and to the http processor.
  • 📇 Fields sync_response.metadata_headers.include_prefixes and sync_response.metadata_headers.include_patterns added to the http_server input.
  • The http_client input and output and the http processor field copy_response_headers has been deprecated in favour of the extract_headers functionality.
  • ➕ Added new cli flag --no-api for the streams subcommand to disable the REST API.
  • 🆕 New experimental kafka_franz input and output.
  • ➕ Added new Bloblang function ksuid.
  • 👍 All codec input fields now support custom csv delimiters.

  🛠 Fixed

  • Streams mode paths now resolve glob patterns in all cases.
  • 🌲 Prevented the nats input from error logging when acknowledgments can't be fulfilled due to the lack of message replies.
  • 🛠 Fixed an issue where GCP inputs and outputs could terminate requests early due to a cancelled client context.
  • 💅 Prevented more parsing errors in Bloblang mappings with windows style line endings.

• v3.60.1 Changes

  December 03, 2021

  🛠 Fixed

  • 🛠 Fixed an issue where the mongodb output would incorrectly report upsert not allowed on valid operators.

• v3.60.0 Changes

  December 01, 2021

  ➕ Added

  • 👍 The pulsar input and output now support oauth2 and token authentication mechanisms.
  • 📇 The pulsar input now enriches messages with more metadata.
  • Fields message_group_id, message_deduplication_id, and metadata added to the aws_sns output.
  • Field upsert added to the mongodb processor and output.

  🛠 Fixed

  • The schema_registry_encode and schema_registry_decode processors now honour path prefixes included in the url field.
  • The mqtt input and output keepalive field is now interpreted as seconds, previously it was being erroneously interpreted as nanoseconds.
  • 🔀 The header Content-Type in the field http_server.sync_response.headers is now detected in a case insensitive way when populating multipart message encoding types.
  • The nats_jetstream input and outputs should now honour auth.* config fields.

• v3.59.0 Changes

  November 22, 2021

  ➕ Added

  • New Bloblang method parse_duration_iso8601 for parsing ISO-8601 duration strings into an integer.
  • 📇 The nats input now supports metadata from headers when supported.
  • Field headers added to the nats output.
  • Go API: Optional field definitions added for config specs.
  • 🆕 New (experimental) sql_select input.
  • New (experimental) sql_select and sql_insert processors, which will supersede the existing sql processor.
  • 🆕 New (experimental) sql_insert output, which will supersede the existing sql output.
  • Field retained_interpolated added to the mqtt output.
  • Bloblang now allows optional carriage returns before line feeds at line endings.
  • 🆕 New CLI flag -w/-watcher added for automatically detecting and applying configuration file changes.
  • Field avro_raw_json added to the schema_registry_encode processor.
  • 🆕 New (experimental) msgpack processor.
  • New parse_msgpack and format_msgpack Bloblang methods.

  🛠 Fixed

  • Fixed an issue where the azure_table_storage output would attempt to send >100 size batches (and fail).
  • 🛠 Fixed an issue in the subprocess input where saturated stdout streams could become corrupted.

• v3.58.0 Changes

  November 02, 2021

  ➕ Added

  • amqp_0_9 components now support TLS EXTERNAL auth.
  • Field urls added to the amqp_0_9 input and output.
  • New experimental schema_registry_encode processor.
  • ⏱ Field write_timeout added to the mqtt output, and field connect_timeout added to both the input and output.
  • 🔧 The websocket input and output now support custom tls configuration.
  • 🆕 New output broker type fallback added as a drop-in replacement for the now deprecated try broker.

  🛠 Fixed

  • ✂ Removed a performance bottleneck when consuming a large quantity of small files with the file input.

• v3.57.0 Changes

  October 14, 2021

  ➕ Added

  • Go API: New config field types StringMap, IntList, and IntMap.
  • 🔊 The http_client input, output and processor now include the response body in request error logs for more context.
  • Field dynamic_client_id_suffix added to the mqtt input and output.

  🛠 Fixed

  • Corrected an issue where the sftp input could consume duplicate documents before shutting down when ran in batch mode.

• 1
• 2
• 3
• 4
• 5
• Next ›
• Last »