RabbitMQ 3.9.0 发布-Linuxeden开源社区

The most important changes are Streams, support for Erlang 24, and improved integration with our RabbitMQ Kubernetes Operator. Other changes worth mentioning are JSON logging, per-protocol and queue type message rates as Prometheus metrics and faster initial cluster formation on Kubernetes.

Highlights

Streams

Streams are a new persistent and replicated data structure (“queue type”) in RabbitMQ which models an append-only log with non-destructive consumer semantics.

They can be used as a regular AMQP 0.9.1 queue or through a new binary protocol plugin and associated client(s).

Streams enable messaging patterns that were not possible or practical before, for example, repeatable non-destructive consumption and mass fanouts to a very large number of consumers.

Erlang 24 Support

Erlang 24 provides 20%-50% higher throughput for many workloads tested by the RabbitMQ core team and community members. Supporting the latest runtime version also allowed RabbitMQ to replace a few external dependencies with libraries now provided by the runtime.

Erlang 24 is now used by default in the community RabbitMQ Docker image.

Kubernetes

RabbitMQ Cluster Operator automates provisioning, management, and operations of RabbitMQ clusters running on Kubernetes. It is open source and is developed by the RabbitMQ Core team at VMware, Inc. This Operator is not specific to 3.9 and can be used with latest 3.8.x release series.

Messaging Topology Operator makes it possible to define RabbitMQ resources (virtual hosts, users, permissions, topologies, policies, etc) as Kubernetes objects. This Operator is also open source and can be used with latest 3.8.x release series.

Logging in JSON

Switch from Lager to the new Erlang Logger API for logging introduces an option for structured logging in JSON. This is relevant for the Prometheus-based alerting feature that we have been investing in this year: Notify me when RabbitMQ has a problem.

Obtaining Packages

RabbitMQ releases are distributed via GitHub, Cloudsmith, and PackageCloud.

Erlang/OTP Compatibility Notes

This release requires Erlang 23.2 or later. Erlang 24 is recommended as it offers 20%-50% throughput improvements for many workloads.

Provisioning Latest Erlang Releases explains what package repositories and tools can be used to provision latest patch versions of Erlang 23.3.x and 24.x.

Upgrading to 3.9

See the Upgrading guide for documentation on upgrades and RabbitMQ change log for release notes of other releases.

RabbitMQ 3.9.0 nodes can run alongside 3.8.x nodes. However, 3.8.16+ as well as 3.9.x both require Erlang 23.2 or later (and support 24). 3.9.x-specific features can only be made available when all nodes in the cluster upgrade to 3.9.0 or any other patch release in the series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes is covered below. Once all nodes are upgraded to 3.9.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended periods of time (no more than a few hours).

Quorum Queue Declaration

In a mixed version cluster, declaration of new quorum queues will fail if executed on a node in the version minority. For example, in a five node cluster with two nodes running 3.8, the three 3.9 nodes are in the version majority and the remaining are in the version minority.

Existing quorum queues will operate just like before.

Classic Queue Metrics

HTTP API responses will provide a limited number of metrics or metrics for a limited number of queues (for operations such as GET /api/queues/{vhost}) when operating in mixed version mode.

Compatibility Notes

Client Library Compatibility

Client libraries that were compatible with RabbitMQ 3.8.x will be compatible with 3.9.0.

Upgrading to Erlang 24.x

RabbitMQ 3.8.16 and later require Erlang 23.2 and support Erlang 24 (recommended). Users of RabbitMQ versions that do not support Erlang 23.2 should first upgrade to the latest 3.8.x patch release on Erlang 23.2, then upgrade to 3.9.0, and ideally Erlang 24.

Getting Help

Any questions about this release, upgrades or RabbitMQ in general are welcome on the RabbitMQ mailing list.

Changes Worth Mentioning

Core Server

Enhancements

Streams, a new durable replicated data structure that allows for repeatable (non-destructive) consumption. RabbitMQ 3.9 introduces a new binary protocol for working with streams. While streams can be used by AMQP 0-9-1 clients, in most cases it will be used via a RabbitMQ stream binary protocol client. The new protocol exposes all the operations specific to this queue type and is heavily optimized for them, resulting in much higher read and write throughput.
Java and Go clients
Structural logging in JSON. Nodes can now log in JSON format.
GitHub issue: #2861
Peer discovery no longer uses randomized startup delay to ensure only one node starts as a seed one during initial (parallel) cluster formation. Instead it relies on an internal runtime locking library.
GitHub issue: #3075
Binding recovery on node startup is now more efficient. This means in clusters with a lot of queues and/or bindings, nodes start quicker.
GitHub issue: #3137
Classic mirrored queue replcas are now started synchronously.
GitHub issue: #2751
journald notifications are now performed using a library and not an external tool (socat)
GitHub issue: #2940
Startup banner now provides more information some of which can be customised.
GitHub issues: #2247, #2777
Per-protocol and queue type message rates are now available as Prometheus metrics, as well as Streams-specific errors. This also addresses a long-standing 3.8.x counters issue for most metrics of this type, as captured in #2783.
GitHub issues: #3127, #3136, #3157

Internal API and Other Changes

Standard runtime logger is now used for logging instead of Lager, a 3rd party dependency. In modern Erlang versions, the module offers at least a comparable feature set and efficiency of Lager.
Users who relied on Lager-specific configuration, e.g. custom extensions or sinks, would have to rebuild them on top of the standard Erlang logger module.

GitHub issue: #2861
Different Raft-based features now can have different defaults and settings, with more configurability coming at a later point. For example, client ID tracking in the MQTT plugin can use settings for lower memory footprint and more frequent compaction while stream and quorum queues use settings more optimal for higher throughput on the same node.
GitHub issue: #2909
Continuous integration of open source RabbitMQ has switched to Bazel, GitHub Actions and BuildBuddy, resulting in much faster and incremental test runs.
Bazel support for Erlang is a new project open sourced by the RabbitMQ Core team as a result.
Process group membership now uses pg.
GitHub issue: #2850

Bug Fixes

powershell.exe now can run without profile
GitHub issue: #2201
Queue index recovery now can continue after a forced node shutdown during recovery (e.g. killed)
GitHub issue: #3099
Correct an issue where the num_acceptors.ssl configuration value was applied incorrectly
GitHub issue: #3180
Correct an issue where the --tags flag was ignored if --description was not specified in the rabbitmqctl add_vhost command

Kubernetes

Enhancements

RabbitMQ Cluster Operator automates provisioning, management, and operations of RabbitMQ clusters running on Kubernetes.
Messaging Topology Operator makes it possible to define RabbitMQ resources (virtual hosts, users, permissions, topologies, policies, etc) as Kubernetes objects.
Kubernetes peer discovery mechanism no longer uses randomized delays to avoid the inherent race condition during initial cluster formation. Instead it relies on an internal distributed locking mechanism available in modern Erlang releases.
While randomized startup delay was reasonably effective at ensuring only one cluster was formed even during parallel node startup, it by definitely assumes that with some low probability more than one node can still start as seed nodes, and be joined by different groups of cluster peers. The locking mechanism does not have this downside. Now that RabbitMQ requires Erlang 23.2, the internal locking library becomes an option for peer discovery.

cluster_formation.randomized_startup_delay_range.min and cluster_formation.randomized_startup_delay_range.max configuration keys will no longer have any effect.

GitHub issue: #3075

AWS Peer Discovery Plugin

Enhancements

AWS peer discovery mechanism now supports Instance Metadata Service v2. In case it is not available, requests will fall back to the original metadata service endpoint.
The v2 endpoint offers substantial security improvements and is one of the AWS best practices.

Contributed by @thuandb (AWS).

GitHub issue: #2952

Federation Plugin

Enhancements

Quorum queues now can be federated.
GitHub issue: #2804

Management Plugin

Bug Fixes

Management UI now honors queue name length limits
GitHub issue: #3202
Management UI now sets a correct www-authenticate response header on authentication failures
GitHub issue: #3205