How to Stop Slack Event Retries from Duplicating AI Agent Actions

Claim the message as well as the delivery. Acknowledge Slack within three seconds, then claim two keys before any tool runs: Slack's event_id, which absorbs retries of the same callback, and the message's own (team, channel, ts), which absorbs a second envelope for the same message. Then key every mutating tool call on the run, the step and the arguments, so one run cannot write twice. The first key stops Slack's retries, the second stops fan-out and edits, and the third stops a single run from doubling its own writes.

Why Slack retries create duplicate agent work

A Slack AI agent that works in a demo can double-post in production. A teammate mentions the agent once, the endpoint takes too long to answer Slack, and a retry arrives while the first run is still opening knowledge, drafting a support reply, or creating a ticket. The thread gets two answers and the helpdesk gets two tickets.

Slack Developer Docs on the Events API are explicit about delivery. Your app should respond with an HTTP 2xx within three seconds. If it does not, Slack considers the delivery failed and retries three times with exponential backoff: nearly immediately, after one minute, and after five minutes. Retry attempts include an x-slack-retry-num header with values 1, 2, or 3, and an x-slack-retry-reason header such as http_timeout. Slack calls the Events API a best-effort system, so retries are not a delivery guarantee either: by default Slack does not deliver an event that is more than two hours late.

Retries keep Slack reliable when endpoints stall. They become dangerous once an agent creates tickets, posts replies, or edits knowledge. A handler that verifies the signature, then searches Kipwise, calls Intercom, opens GitHub, and posts a long reply inside the request path will regularly miss the three-second window. Slack retries, and if your code treats every POST as a fresh job, the model runs again.

QASkills' guide on testing Slack Events API retries opens with exactly this scenario: a handler posts the same incident alert twice, both requests carry the same Slack event ID, and the second also carries x-slack-retry-num: 1 and http_timeout. The duplicate delivery is expected platform behavior. The duplicate side effect is an application defect. ResumeLens' Slack Events API deep dive gives the standard fix, a conditional insert or Redis SET NX on event_id before processing, and notes that read-only logging survives duplicates while creating tickets, posting messages, or updating knowledge does not. Both are write-ups about specific designs, not measured demand.

Claim two keys, not one

Every interactive Slack agent run needs three identities recorded up front:

1. Delivery envelope: Slack's outer event_id, which is globally unique for that callback.
2. Logical user message: coordinates that stay stable across fan-out, typically (team_id, channel, message ts).
3. Agent run: your internal run ID that owns tool spans, citations, approvals, and delivery status.

Slack's Events API docs show event_id on the outer callback and note that one request carries one event. Agents need more than that field. The kortix-ai/suna Slack dedup.ts module explains why: one user message can reach an app as several envelopes with different event IDs, because Slack delivers a channel @mention as both an app_mention and a message event, and redeliveries can land on different replicas. Every one of them carries the same (team, channel, ts), so claiming that identity once guarantees a single agent run per message however it arrived.

Implement two atomic claims with the same store pattern (database INSERT ... ON CONFLICT DO NOTHING, Redis SET key value NX EX, or equivalent):

• Envelope claim on event_id. Absorbs Slack's own redelivery of the identical callback. If the insert loses, return 2xx and do not enqueue.
• Message claim on (team, channel, ts). Absorbs fan-out where one human mention produces multiple envelopes. If the message claim loses, return 2xx even when the event_id was never seen before.

The dedup.ts comments describe the incident class this prevents: a redelivery that lands after a thread-to-session mapping exists is treated as a brand-new follow-up and runs the agent a second time. That is how duplicate support drafts and duplicate wiki writes appear in Slack threads.

Take the ts from the message, not from the event. A message_changed event, sent when a message is edited, has a new event_id and a new top-level ts, with the original message nested inside it as message.ts. Slack's docs add that the event can also fire when Slack's automatic language detection updates a message's metadata, with no human edit at all. If your handler keys on the top-level ts, an edit or a metadata update looks like a new message. Key on message.ts, and treat an edit as new intent only when it adds the mention. If a user edits the question mid-run, say so in the thread and ask whether to rerun, since a rerun can repeat tool writes.

Choose fail-open versus fail-closed deliberately. The Suna module fails open on claim-store errors for inbound messages, so a database blip does not drop a user's message, and it accepts a rare duplicate during the outage. It fails closed for its one-time error notice, because spam is what that claim exists to prevent. Document your choice. Silent drops during an outage look like "the agent ignored me," which is harder to detect than a rare double post.

Kipwise Agent fits this model as a thread-native work agent. A teammate mentions it once and continues with ordinary replies. Applicable actions use the requesting user's permissions. It cites opened sources, refuses to guess, and keeps consequential writes explicit. Those product behaviors only stay trustworthy if one Slack delivery path maps to one run.

Acknowledge fast, claim before tools, process asynchronously

Build the request path in this order:

1. Verify Slack's signing secret on the raw body.
2. Handle url_verification if present.
3. Return HTTP 2xx quickly after a durable claim decision.
4. Enqueue or continue the agent run only when this process won the claim.
5. Never start mutating tools until the claim is held.

Slack's Events API guidance recommends responding with a 200 as soon as you can, not processing events in the same process that answers Slack, and using a queue. Model and tool latency routinely exceeds three seconds, so this is required for AI agents rather than optional polish.

ResumeLens describes the production tradeoff. If you acknowledge first, Slack will not retry even when later processing fails, so you need internal retries, dead-letter handling, and alerting. If you process first, Slack will retry on failure, but you must still be idempotent and you will usually still miss the three-second budget for a multi-tool agent. Most production Slack AI agents should acknowledge first, claim durably, and own recovery themselves.

QASkills adds a second rule operators often miss: an already-accepted duplicate should usually receive 2xx. Returning an error invites another retry and spends your failure budget without creating new value. Slack temporarily disables event subscriptions when an app fails to answer successfully for more than 95% of delivery attempts over 60 minutes, and retries count toward that limit. Your claim store is the source of truth, not the HTTP status on the duplicate.

Log the retry headers when present. Slack's Governance and trust guide asks builders to record who triggered the request, tools called, retry attempts, and outcome, so the retry headers belong on the same audit row as the claim result.

Make mutating tools idempotent under the same run

Delivery dedup stops a second run. It does not stop a single run from double-writing when a tool succeeds and the response times out. For every mutating tool call, derive a stable idempotency key from the agent run ID, the step or tool name, and a hash of the normalized arguments that matter for uniqueness. Before creating a ticket, posting a customer-visible reply, editing a knowledge page, or opening a GitHub change, check whether that key already has a stored result. If yes, reuse it. If no, execute once and store the result with the key.

This is one of several separate layers. Human approval decides whether a write may proceed. Scheduling decides when an unattended job fires and how delivery is proven. Retry idempotency decides whether Slack's retries can create two writes from one interactive mention. When a run has started and a tool then times out, mid-workflow recovery takes over. You usually need all of them, but they solve different jobs.

Keep read-only tools cheaper to retry. Knowledge search and citation gathering can often be retried inside one run. Mutating tools cannot. Separate them in your tool policy so the model cannot "helpfully" recreate a ticket after a timeout.

Worked example: a support mention that must not double-file

A support operations lead mentions the agent in #customer-escalations:

"Draft an answer from the help center for ticket 18422 and open an internal follow-up if the refund policy is unclear."

Expected path:

1. Slack POSTs app_mention (and possibly a message event).
2. Endpoint verifies signature, claims event_id, claims (team, channel, ts), returns 200.
3. Worker loads requester permissions, searches approved help-center and Kipwise sources, cites what it opened, and refuses to invent policy.
4. If a knowledge write or ticket create is needed, the write waits on your approval gate and uses a run/step idempotency key.
5. A Slack retry with x-slack-retry-num: 1 finds the claims and exits without a second draft or second ticket.
6. Audit shows one run, retry headers observed, tools called once, and delivery state for the Slack reply recorded separately from model generation.

Failure modes to rehearse in staging:

• Claim succeeds, enqueue fails: recover from an outbox or pending-claim state, or you will acknowledge duplicates while the work never runs.
• Tool write succeeds, HTTP to the tool times out: without a step key, a worker retry creates a second ticket.
• Only event_id is claimed: app_mention and message both run.
• Duplicate returns 500: Slack keeps retrying and your failure ratio climbs toward the disablement threshold.

Audit fields that make retries diagnosable

Slack's governance guide recommends logging who triggered the request, tools called, model, retries, tokens, and outcome (success, partial, or failure). For retry-safe agents, add:

• event_id
• message key (team, channel, ts)
• claim results for envelope and message
• x-slack-retry-num and x-slack-retry-reason when present
• agent run ID
• mutating-tool idempotency keys and whether the result was reused
• Slack reply delivery state, separate from generation state

Without those fields, observability can show two successful generations without explaining that Slack redelivered the same mention. The guide to monitoring AI agent workflows in Slack covers reconstructing tool calls, approvals, and delivery. Add claim-key fields to those traces so retries are visible.

What to check before rollout

Measure p95 handler acknowledgement time, and if it is near three seconds, move work off the request path. Replay signed duplicate deliveries in staging, including http_timeout retries, as QASkills recommends. If you subscribe to both app_mention and message, test the fan-out, and edit a mentioning message to confirm it does not start a second run. Alert on claim-store errors, duplicate claims observed, and mutating-tool result reuse. Pilot on one high-volume support or ops channel before workspace-wide rollout.

If your Slack AI agent still processes Events API callbacks synchronously, or keeps recent event IDs only in memory on one replica, you do not yet have a production idempotency plan. Start with the two durable claims and the mutating-tool keys.