Queue messages reappearing (visibility timeout)
Why pgmq messages come back after you read them, how the visibility timeout and read_ct work, and how to delete, archive or extend messages so each one is handled once.
What you see
A worker reads a message from a pgmq queue and processes it, but the same message shows up again a little later. read_ct on the message keeps going up, and your worker handles the same job two or three times.
Why it happens
Reading a message doesn't remove it. read hides the message for a set number of seconds, the visibility timeout (vt). If the message is still there when the timeout ends, any reader can pick it up again. That's what makes queues safe when a worker crashes halfway: the message comes back instead of being lost.
It reappears when:
- You never delete or archive it. Processing finished, but the message is still in the queue.
- The timeout is shorter than the work. A 10-second timeout on a job that takes 30 seconds lets a second worker grab it.
- The worker fails before acknowledging. An exception, a function timeout or a deploy cuts it off. This is the queue doing its job.
How to fix it
Acknowledge every message you finish. Use
deleteto drop it, orarchiveto move it to the queue's archive table for auditing:select * from pgmq.read('invoice_emails', 60, 10); -- hide 10 messages for 60 s -- ...send the emails... select pgmq.archive('invoice_emails', 1234); -- or pgmq.delete(...)Set the timeout longer than your slowest run. The second argument to
readis the timeout in seconds. Allow plenty of headroom.For work that runs long, extend the timeout while you're still busy:
select * from pgmq.set_vt('invoice_emails', 1234, 120); -- visible again in 120 sMake handlers safe to run twice. Retries happen, so key side effects on the message or a business ID. For example, record
invoice_idin asent_emailstable with a unique constraint and skip it if it's already there.Stop poison messages from looping forever. Check
read_ctand move a message aside after a few tries:select msg_id, read_ct, message from pgmq.read('invoice_emails', 60, 10); -- if read_ct > 5: log it, then pgmq.archive(...)
From an Edge Function
The pgmq_public wrappers let you work with queues over the Data API. On SupaHost only service_role can call them by default, so use the service key on the server:
import { createClient } from "npm:@supabase/supabase-js@2";
const supabase = createClient(
Deno.env.get("SUPABASE_URL")!,
Deno.env.get("SUPABASE_SERVICE_ROLE_KEY")!,
);
Deno.serve(async () => {
const q = supabase.schema("pgmq_public");
const { data: messages, error } = await q.rpc("read", {
queue_name: "invoice_emails",
sleep_seconds: 60, // the visibility timeout
n: 10,
});
if (error) return new Response(error.message, { status: 500 });
for (const m of messages ?? []) {
await sendInvoiceEmail(m.message); // your own code
await q.rpc("archive", { queue_name: "invoice_emails", message_id: m.msg_id });
}
return Response.json({ handled: messages?.length ?? 0 });
});
In pgmq_public.read, sleep_seconds is the visibility timeout, not a wait. A function has about 150 seconds to start responding, so keep each batch well inside that and the timeout above it.
On SupaHost
- Turn queues on with Enable Queues on Dashboard → your project → Queues tab. That creates
pgmqand thepgmq_publicwrappers (send,send_batch,read,pop,archive,delete). - The Queues tab shows Waiting, Total sent and Oldest for each queue, and Messages lists the newest 25. A message that keeps showing up there after your worker has run is one that was never deleted or archived.
popreads and deletes in one step. Use it only when losing a message on a crash is acceptable.