"Dead-letter for deleted queues" - Message evacuation on queue deletion for anonymous/ephemeral queues (publish to designated exchange/routing key)

[luizcarlosfaria]

Context

Anonymous (server‑named) queues bound under x-consistent-hash exchanges are common in sharded/weighted work distribution. Each consumer instance creates its own ephemeral queue (often exclusive, auto‑deleted, non‑durable) and binds it to the consistent‑hash exchange, letting the broker hash the original routing key to pick exactly one of the available queues. This pattern scales well and keeps ordering per hash key, but it also concentrates responsibility: if a consumer (and thus its anonymous queue) disappears, any in‑flight or enqueued messages mapped to that shard can be stranded and then discarded when the queue is deleted.

By design, these ephemeral queues lack resilience settings, and the deletion path of a queue (consumer ends, connection drops, explicit delete, or autoDelete) does not trigger DLX. Even when DLX arguments are present, they are often impractical on server‑named shard queues, and they don’t cover the specific case of queue deletion. The end result is avoidable message loss in consistent‑hash topologies where the business cannot tolerate dropping leftovers from a disappearing shard.

---

Is your feature request related to a problem? Please describe.

Currently, when a queue is deleted—especially anonymous/ephemeral queues—there is no native mechanism to evacuate unconsumed messages to a safe destination.

This causes message loss and reduces reliability for temporary flows that, due to business requirements, must not discard their residual messages.

---

Describe the solution you'd like

Introduce a native on‑delete evacuation mechanism: when a queue is deleted, any remaining messages are re‑published to a specified exchange with a specified routing key.

Suggested Queue Arguments

• on-delete-dead-letter-exchange: string (required to enable)
• on-delete-dead-letter-routing-key: string (optional; default "")

Expected Behavior

1. Upon queue.delete (or autoDelete removal), the broker:

   • Drains remaining messages from the queue.

   • For each message, re‑publishes it to on-delete-dead-letter-exchange using on-delete-dead-letter-routing-key.

   • Attaches standard headers, for example:

     • x-original-queue: deleted queue name
     • x-evacuated-at: timestamp (UTC)
     • x-evacuated-reason: "queue-delete" / "autoDelete"

2. Ordering is not guaranteed (same caveats as DLX).

3. TTL, priority, headers, and delivery mode are preserved when feasible.

4. Internal confirmation: the broker completes deletion after evacuation (with reasonable back‑pressure and limits).

5. Observability: emit an audit event (e.g., queue.evacuated) including count of evacuated messages.

---

Technical Considerations

• Compatibility: opt‑in via queue args and/or policy; no impact on existing queues unless enabled.

• Queue types: support classic and quorum; document differences (e.g., drain cost, throughput).

• Back‑pressure / limits: internal parameters for batch size, rate limiting, and maximum evac size to avoid long pauses.

• Publish failures: if the target exchange is missing or republish fails:

  • Option A (simple): drop remaining messages and log/metric error.
  • Option B (robust): retry counter + fallback to configurable amq.fallback exchange.

• Observability: metrics (e.g., messages_evacuated_total, latency, errors) and an audit event.

• Interplay with DLX/Queue TTL/Message TTL: document precedence; on‑delete runs only on the queue deletion path (does not replace DLX for expiration/rejection).

---

Impact

This feature reduces message loss in ephemeral flows, improves operational reliability, and avoids fragile consumer‑side workarounds—while preserving the simplicity and benefits of anonymous queues when desired.

---

Describe alternatives you've considered

Use same dead-letter configuration and mechanism to add the behavior of move messages when queue is deleted, if dead-letter configuration is present.

[michaelklishin]

@luizcarlosfaria transient queues by definition do not guarantee anything in terms of data safety.

Users of consistent hashing exchanges with N transient queues for workload distribution these days can use a stream or a partitioned stream (which are durable, replicated, efficient and allow for parallel consumption from followers) instead of asking our tiny team to invest a lot of time into introducing "data safety features" for transient queues.

[michaelklishin]

@luizcarlosfaria @LeonardoDaLuz @ruifgdias @reinaldocoelho @emensonlima01 @r-leafar @ruifgdias also worth mentioning: our team really does not appreciate upvoting fests.

You are not on Reddit. Upvoting fests are very obvious to any long time maintainer, and make it extra clear that your team is the only one affected, which does not make your case any stronger.

At the very least don't all upvote on the same day.

[michaelklishin]

Introduce a native on‑delete evacuation mechanism: when a queue is deleted,
any remaining messages are re‑published to a specified exchange with a specified routing key

This has been discussed to death before, on the mailing list and elsewhere. The reason why RabbitMQ does not dead letter messages in a deleted queue is the following: there can be tens of millions of messages in a deleted queues, and re-publishing them can take a lot of time and resources, confuse the user, cause channel operation (queue.delete) timeouts and force applications to retry deletes.

None of those things are very helpful when a user tries to delete a queue, possibly a queue that has accidentally accumulated a backlog. In fact, nearly all users want the exact opposite of those things when they delete a queue with a massive backlog.

RabbitMQ is long done with the "One True Queue Type™ with a bajillion features based on settings" approach. That's why we have queue types, and soon enough will likely have queue sub-types.

We are long done with flags that require so many internal changes that you might as well develop a new queue type.

Speaking of which, queue types these days can be implemented by plugins, so you are welcome to develop one and open source it (or not) without convincing our team or waiting for us to get to this highly controversial idea.

Queue types: support classic and quorum

Quorum queues do have DLX safety features that no other queue types support.

Quorum queues are not also not mean to be semantically transient, cannot be non-durable, ignore the transient property on the messages routed to them, and so on.

So yeah, that's a lot of text but not a lot of considerations for what all other RabbitMQ users need. Sorry.