The following is copy/pasted from backend repo Any changes made here should be copied back.

manageQueue Edge Function

REST API for document upload queue management. Provides endpoints for listing, filtering, and managing queue items.

Base URL

POST/GET/PATCH/DELETE https://<supabase-project>.supabase.co/functions/v1/manageQueue/<endpoint>

Authentication

All endpoints require a valid JWT token in the Authorization header:

Authorization: Bearer <jwt_token>

The JWT must contain app_claims with access_level for authorization.

Endpoints

Queue Items

GET /items/:id

Get a single queue item by ID. Useful for polling status in fire-and-forget workflows.

Response:

{
  "id": "uuid",
  "status": "completed",
  "priority": 0,
  "attempts": 1,
  "max_attempts": 3,
  "request_body": { ... },
  "created_at": "2025-01-01T00:00:00Z",
  "processed_at": "2025-01-01T00:01:30Z",
  "error_message": null,
  "edge_function_response": {
    "success": true,
    "statusCode": 200,
    "processingStats": { ... }
  },
  "transaction_id": "uuid"
}

Error Response (404):

{
  "error": "Queue item not found"
}

GET /items

List queue items with optional filters. Returns full objects including request_body and edge_function_response JSONB fields.

Note: For table views, consider using /items-summary instead for faster loading.

GET /items-summary

Lightweight version of /items for table views. Returns only fields needed for display, excluding heavy JSONB fields (request_body, edge_function_response). Use /items/:id to fetch full details when a row is expanded.

Query Parameters: Same as /items (status, table_name, search, user_id, limit, offset)

Response:

{
  "items": [
    {
      "id": "uuid",
      "status": "pending",
      "table_name": "documents_fpp",
      "priority": 0,
      "attempts": 1,
      "max_attempts": 3,
      "created_at": "2025-01-01T00:00:00Z",
      "updated_at": "2025-01-01T00:00:00Z",
      "processed_at": null,
      "last_attempt_at": null,
      "error_message": null,
      "user_id": "user-uuid",
      "transaction_id": null,
      "source_url": "https://example.com/doc.pdf",
      "source_title": "Document Title",
      "action_type": "add"
    }
  ],
  "total": 100
}

action_type Values:

Value	Condition
`"add"`	`addDocs=true`, `delDocs=false`
`"delete"`	`delDocs=true`, `addDocs=false`
`"replace"`	`addDocs=true` AND `delDocs=true`
`"update"`	`updDocs=true`
`"process"`	None of the above flags set

Benefits:

Much smaller payload (no JSONB fields)
Faster initial table render
source_url, source_title, and action_type extracted and flattened for easy display
Use /items/:id for on-demand detail fetch when user expands a row

GET /items (Full)

List queue items with optional filters.

Query Parameters:

Parameter	Type	Description
`status`	string	Filter by status: `pending`, `submitted`, `processing`, `completed`, `failed`, `cancelled`, `expired`, `held`, `deleted`
`table_name`	string	Filter by document table (e.g., `documents_fpp`)
`search`	string	Search in sourceUrl or sourceTitle
`user_id`	string	Filter by user ID. Use `mine` to filter by authenticated user's submissions
`limit`	number	Max items to return (default: 100)
`offset`	number	Pagination offset (default: 0)

Response:

{
  "items": [
    {
      "id": "uuid",
      "status": "pending",
      "priority": 0,
      "attempts": 0,
      "max_attempts": 3,
      "request_body": {
        "sourceUrl": "https://example.com/doc.pdf",
        "sourceDate": "2025-01-01T00:00:00Z",
        "addDocs": true,
        "delDocs": false,
        "metadata": {}
      },
      "created_at": "2025-01-01T00:00:00Z",
      "updated_at": "2025-01-01T00:00:00Z",
      "processed_at": null,
      "error_message": null,
      "edge_function_response": null,
      "transaction_id": null,
      "table_name": "documents_fpp"
    }
  ],
  "total": 42
}

PATCH /items/:id

Update a queue item.

Body:

{
  "requestBody": {}, // Update request_body JSONB
  "priority": 10, // Update priority (0-100)
  "tableName": "...", // Update table_name
  "status": "pending", // Update status
  "resetAttempts": true // Reset attempts/error for retry
}

DELETE /items/:id

Permanently delete a queue item.

Queue Statistics

GET /stats

Get queue statistics.

Query Parameters:

Parameter	Type	Description
`table_name`	string	Filter stats by table. If ends with `_testing`, uses testing stats view

Testing vs Production Stats:

If table_name ends with _testing OR STORE_TESTING_SUFFIX=true, uses upload_queue_stats_testing view
Otherwise uses upload_queue_stats view (excludes testing items)

Response:

{
  "pending": 5,
  "processing": 1,
  "completed": 100,
  "failed": 2,
  "cancelled": 0,
  "expired": 0,
  "held": 3,
  "avgProcessingSeconds": 12.5,
  "avgQueueToCompletionSeconds": 45.2
}

Field	Description
`avgProcessingSeconds`	Pure processing time (from edge function duration)
`avgQueueToCompletionSeconds`	Total time from queue creation to completion (includes wait time)

Queue Submission

POST /submit

Submit a new document to the queue.

Body:

{
  "requestBody": {
    "sourceUrl": "https://example.com/document.pdf",
    "sourceDate": "2025-01-01T00:00:00Z",
    "sourceTitle": "Document Title",
    "addDocs": true,
    "delDocs": false,
    "updDocs": false,
    "active": true,
    "access_level": 1,
    "metadata": {
      "dc_creator": "Author Name",
      "dc_rights": "CC BY 4.0"
    }
  },
  "priority": 0,
  "tableName": "documents_fpp"
}

Response:

{
  "message": "Queue item created",
  "id": "uuid"
}

Queue Processing

POST /process-background (Recommended)

Trigger the Netlify background processor to handle queue items. This is the recommended approach as it avoids frontend timeout issues and can handle large document operations (up to 15 minutes per item).

Ownership Enforcement:

Non-admin users can only process their own queue items
If no queueIds provided: Auto-selects the user's own pending items (non-admin) or any pending items (admin)
If queueIds provided: Validates ownership before processing
Attempting to process items owned by other users returns a 403 error
Admins (accessLevel >= 9) can process any items

JWT Auto-Refresh: The stored user_jwt is automatically refreshed with the caller's current token in the database. Additionally, the fresh JWT is passed directly to Netlify in the request body, ensuring the processor always uses a valid token.

Body (optional):

{
  "queueId": "uuid", // Optional: process specific item
  "queueIds": ["uuid1", "uuid2"], // Optional: process multiple specific items
  "batchSize": 1, // Optional: number of items to process (default: 1)
  "tableName": "documents_fpp" // Optional: filter by table
}

What gets sent to Netlify:

{
  "queue_id": "uuid",
  "queue_ids": ["uuid1", "uuid2"],
  "batch_size": 1,
  "table_name": "documents_fpp",
  "user_jwt": "eyJ..." // Fresh JWT from caller's Authorization header
}

Response (202 Accepted):

{
  "message": "Background processing started",
  "success": true,
  "items_queued": 1,
  "queue_ids": ["uuid"]
}

Error Response (403 Forbidden):

{
  "error": "You can only process your own queue items. 3 item(s) belong to other users."
}

Environment Variables Required:

NETLIFY_QUEUE_PROCESSOR_URL: URL of the Netlify background function
NETLIFY_BACKGROUND_API_KEY: API key for authentication

Netlify Integration: The Netlify function should prefer using the user_jwt from the request body (if provided) over fetching from the database. This ensures the freshest possible token is used.

Error Handling: If the Netlify call fails (network error or non-2xx response), items are automatically marked as failed with the error message captured. This increments the attempt counter and may auto-expire items that exceed max attempts.

POST /process (Legacy)

Get the next item to process (marks as 'processing'). Returns the item for frontend-based processing. Note: Frontend processing may timeout for large operations (>500 documents).

JWT Auto-Refresh: The stored user_jwt is automatically replaced with the caller's current JWT token. This prevents "JWT expired" errors when processing queued items.

Body (optional):

{
  "statusFilter": "pending" // Default: "pending"
}

Response:

{
  "item": {
    /* queue item object with refreshed user_jwt */
  }
}

Or if no items available:

{
  "item": null,
  "message": "No items to process"
}

POST /refresh-jwt

Refresh JWT tokens for expired/failed queue items and reset them to pending status. Uses the caller's current Authorization header JWT to update stored tokens.

Body (optional):

{
  "itemIds": ["uuid1", "uuid2"], // Optional: specific items to refresh
  "statusFilter": ["expired", "failed"] // Optional: filter by status (default)
}

Response:

{
  "message": "Refreshed JWT and reset 3 items to pending",
  "refreshed": 3,
  "itemIds": ["uuid1", "uuid2", "uuid3"]
}

Usage: When queue items fail due to JWT expiration, call this endpoint while logged in to refresh them with your current session token, then trigger background processing.

POST /complete/:id

Mark item as completed.

Body:

{
  "transactionId": "uuid",
  "response": {
    "success": true,
    "processingStats": {}
  }
}

POST /fail/:id

Mark item as failed (will auto-retry if under max attempts).

Body:

{
  "errorMessage": "Error description",
  "response": {}
}

POST /cancel/:id

Cancel a queue item (sets status to 'cancelled').

Bulk Operations

All bulk endpoints accept:

{
  "itemIds": ["uuid1", "uuid2", "uuid3"]
}

POST /bulk/status

Bulk update status.

Body:

{
  "itemIds": ["uuid1", "uuid2"],
  "newStatus": "pending",
  "resetAttempts": true
}

POST /bulk/hold

Set selected items to 'held' status. Only affects items in pending, failed, or expired status.

POST /bulk/unhold

Release held items back to 'pending' status. Resets attempts.

POST /bulk/remove

Permanently delete selected items.

POST /bulk/replace

Set items for document replacement (delDocs=true, addDocs=true). Resets to pending.

POST /bulk/delete

Set items for document deletion only (delDocs=true, addDocs=false). Resets to pending.

POST /bulk/metadata

Bulk update metadata fields.

Body:

{
  "itemIds": ["uuid1", "uuid2"],
  "active": true,
  "accessLevel": 5,
  "metadata": {
    "dc_creator": "New Author"
  },
  "mergeMode": "merge",
  "onBehalfOfUserId": "target-user-uuid"
}

Field	Description
`active`	Set document active status
`accessLevel`	Set access level (0-100, admin only)
`metadata`	Metadata fields to update
`mergeMode`	`"merge"` (add/update fields) or `"replace"` (overwrite all)
`onBehalfOfUserId`	Transfer ownership to user UUID

Ownership Transfer

POST /transfer/:id

Transfer ownership of a queue item to another user.

Body:

{
  "targetUserId": "user-uuid"
}

POST /lookup-user

Look up a user by email (for ownership transfer).

Body:

{
  "email": "user@example.com"
}

Response:

{
  "authUser": {
    "id": "auth-uuid",
    "email": "user@example.com"
  },
  "user": {
    "id": "public-user-uuid",
    "full_name": "User Name",
    "access_level": 1,
    "org_id": "fpp"
  }
}

Source URL Check

POST /check-source

Check if a source URL already exists in the vector store. Use this before submitting a new document to determine whether to use "Add", "Update", or "Replace".

Body:

{
  "sourceUrl": "https://example.com/document.pdf",
  "orgId": "ctrl-shift" // Optional, falls back to JWT claims
}

Response (documents exist):

{
  "exists": true,
  "count": 200,
  "sourceUrl": "https://example.com/document.pdf",
  "orgId": "ctrl-shift",
  "tableName": "vectors_ctrl_shift",
  "metadata": {
    "title": "Document Title",
    "date": "2025-10-06",
    "type": "HTML",
    "userId": "a52a44dc-8faa-4b76-b645-1945fcbe4314",
    "orgId": "ctrl-shift"
  }
}

Response (no documents):

{
  "exists": false,
  "count": 0,
  "sourceUrl": "https://example.com/new-document.pdf",
  "orgId": "ctrl-shift",
  "tableName": "vectors_ctrl_shift",
  "metadata": null
}

Response (org has no vector store):

{
  "exists": false,
  "count": 0,
  "message": "No vector store found for org: new-org"
}

Use Case: Call this endpoint when the user enters a source URL to check if documents already exist. If exists: true, prompt the user to choose:

Update (updDocs: true) - Update metadata only
Replace (delDocs: true, addDocs: true) - Delete existing and re-add
Cancel - Don't proceed

This prevents the "Cannot add documents. Documents already exist" error.

Lightweight Status Polling

POST /status-check

Lightweight endpoint for polling status of multiple queue items. Returns only the id→status map, avoiding the overhead of fetching full item objects with request_body, edge_function_response, metadata, etc.

Use Case: Frontend polling during background processing. Instead of fetching full item objects every 15 seconds, use this endpoint to check only status changes.

Body:

{
  "ids": ["uuid-1", "uuid-2", "uuid-3"]
}

Response:

{
  "statuses": {
    "uuid-1": "processing",
    "uuid-2": "completed",
    "uuid-3": "failed"
  }
}

Notes:

Maximum 100 IDs per request
Non-admin users only see statuses for their own items (items they don't own are omitted from the response)
IDs not found in the database are omitted from the response
Much smaller payload than GET /items - ideal for frequent polling

Example Usage:

// Poll status of items being processed
async function pollStatuses(queueIds, jwt) {
  const response = await fetch(`${MANAGE_QUEUE_URL}/status-check`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${jwt}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ ids: queueIds }),
  });

  const { statuses } = await response.json();

  // Check for completed/failed items
  for (const [id, status] of Object.entries(statuses)) {
    if (status === "completed") {
      showToast(`Item ${id} completed!`);
    } else if (status === "failed") {
      showToast(`Item ${id} failed`, "error");
    }
  }

  return statuses;
}

// Periodic polling (e.g., every 15 seconds)
const processingIds = ["uuid-1", "uuid-2"];
setInterval(() => pollStatuses(processingIds, jwt), 15000);

Queue Item Status Values

Status	Description
`pending`	Waiting to be processed
`submitted`	Handed off to Netlify background processor (awaiting pickup)
`processing`	Currently being processed by Netlify
`completed`	Successfully processed
`failed`	Processing failed (may auto-retry)
`cancelled`	Manually cancelled by user
`expired`	Exceeded max attempts
`held`	Manually held, won't process
`deleted`	Soft deleted

Status Flow

pending → submitted → processing → completed
                   ↘           ↘
                    failed → expired (after max attempts)

pending → submitted: Set when /process-background is called (before Netlify pickup)
submitted → processing: Set by Netlify when it starts processing the item
processing → completed: Set when processing succeeds
processing → failed: Set when processing fails (may retry)
failed → expired: Set when max retry attempts exceeded

Request Body Fields

The request_body JSONB contains the document processing parameters:

Field	Type	Required	Description
`sourceUrl`	string	Yes	URL of document to process
`sourceDate`	string	Yes	ISO 8601 date of document
`sourceTitle`	string	No	Document title
`addDocs`	boolean	No	Add new documents (default: true)
`delDocs`	boolean	No	Delete existing documents
`updDocs`	boolean	No	Update metadata only
`active`	boolean	No	Document active status
`access_level`	number	No	Access level 0-10 (admin only)
`onBehalfOfUserId`	string	No	User UUID for ownership transfer
`metadata`	object	No	Dublin Core and custom metadata

Error Responses

All errors return:

{
  "error": "Error message description"
}

Status	Description
400	Bad request (invalid parameters)
401	Unauthorized (missing/invalid JWT)
404	Not found (unknown endpoint or item)
500	Server error

Example: Frontend Integration (JavaScript)

const SUPABASE_URL = "https://your-project.supabase.co";
const MANAGE_QUEUE_URL = `${SUPABASE_URL}/functions/v1/manageQueue`;

// Get auth token from your auth system
const jwt = await getAuthToken();

// List pending items
const response = await fetch(
  `${MANAGE_QUEUE_URL}/items?status=pending&limit=50`,
  {
    headers: {
      Authorization: `Bearer ${jwt}`,
    },
  },
);
const { items, total } = await response.json();

// List only my queue items (fire-and-forget submissions)
const myItems = await fetch(`${MANAGE_QUEUE_URL}/items?user_id=mine`, {
  headers: {
    Authorization: `Bearer ${jwt}`,
  },
});
const { items: myQueueItems } = await myItems.json();

// Submit new document
const submitResponse = await fetch(`${MANAGE_QUEUE_URL}/submit`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    requestBody: {
      sourceUrl: "https://example.com/doc.pdf",
      sourceDate: new Date().toISOString(),
      addDocs: true,
      metadata: {
        dc_creator: "Author Name",
      },
    },
    priority: 0,
  }),
});

// Bulk hold items
await fetch(`${MANAGE_QUEUE_URL}/bulk/hold`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    itemIds: ["uuid1", "uuid2"],
  }),
});

Fire-and-Forget Workflow

For long-running document processing, use the async pattern:

1. Submit with async mode

// Submit document and get queueId immediately
const response = await fetch(`${SUPABASE_URL}/functions/v1/maintainSource`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    sourceUrl: "https://example.com/large-document.pdf",
    sourceDate: new Date().toISOString(),
    addDocs: true,
    async: true, // Fire-and-forget mode
  }),
});

const { queueId } = await response.json();
// Returns immediately with HTTP 202

2. Check status (choose one approach)

Option A: Manual refresh (recommended)

Use the user_id=mine filter to show the user's submissions. User clicks refresh to see updates.

// List user's queue items with current status
const response = await fetch(
  `${SUPABASE_URL}/functions/v1/manageQueue/items?user_id=mine`,
  { headers: { Authorization: `Bearer ${jwt}` } },
);

const { items } = await response.json();
// items contains all user's submissions with their current status
// completed items have edge_function_response with results

Option B: Active polling (optional)

For real-time updates (toast notifications, progress indicators):

async function pollQueueStatus(queueId, jwt, maxWaitMs = 600000) {
  let delayMs = 1000; // Start with 1s
  const maxDelayMs = 30000; // Max 30s between polls
  const startTime = Date.now();

  while (Date.now() - startTime < maxWaitMs) {
    const response = await fetch(
      `${SUPABASE_URL}/functions/v1/manageQueue/items/${queueId}`,
      { headers: { Authorization: `Bearer ${jwt}` } },
    );

    const item = await response.json();

    if (item.status === "completed") {
      return { success: true, result: item.edge_function_response };
    }
    if (item.status === "failed") {
      return { success: false, error: item.error_message };
    }

    // Still processing - wait and retry
    await new Promise((r) => setTimeout(r, delayMs));
    delayMs = Math.min(delayMs * 1.5, maxDelayMs); // Exponential backoff
  }

  throw new Error("Queue processing timeout");
}

// Usage: poll in background after submission
const { queueId } = await submitAsync(sourceUrl);
pollQueueStatus(queueId, jwt).then((result) => {
  if (result.success) {
    showToast("Document processed successfully");
    refreshQueueList();
  }
});

3. Handle expired items

Note: As of 2026-01-12, POST /process automatically refreshes JWTs when fetching items. The /refresh-jwt endpoint is still useful for bulk-resetting expired/failed items to "pending" status.

// Reset all expired/failed items to pending (with fresh JWT)
const refreshResponse = await fetch(`${MANAGE_QUEUE_URL}/refresh-jwt`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    statusFilter: ["expired", "failed"], // Default, can omit
  }),
});

const { refreshed } = await refreshResponse.json();
console.log(`Reset ${refreshed} items to pending`);

// Items are now pending and will get fresh JWT when processed

4. Trigger processing (admin)

Recommended: Use background processing

// Trigger Netlify background processor (handles large operations)
const response = await fetch(`${MANAGE_QUEUE_URL}/process-background`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    batchSize: 5, // Process up to 5 items
  }),
});

const { items_queued, queue_ids } = await response.json();
console.log(`Started processing ${items_queued} items`);

Legacy frontend processing:

Processing via the UI "Process Queue" button:

Calls POST /process to get next item (JWT is auto-refreshed)
Uses returned item.user_jwt to call maintainSource
Updates status via mark_upload_queue_completed or mark_upload_queue_failed

// Frontend processing loop (JWT auto-refresh happens automatically)
async function processNextItem() {
  const { item } = await fetch(`${MANAGE_QUEUE_URL}/process`, {
    method: "POST",
    headers: { Authorization: `Bearer ${jwt}` },
  }).then((r) => r.json());

  if (!item) return null;

  // item.user_jwt is already refreshed with caller's current token
  const result = await fetch(`${SUPABASE_URL}/functions/v1/maintainSource`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${item.user_jwt}`, // Fresh JWT
      "Content-Type": "application/json",
    },
    body: JSON.stringify(item.request_body),
  }).then((r) => r.json());

  return result;
}

Large Document Note

For documents over 200 pages or 1500+ chunks, the Edge Function may hit CPU/timeout limits during embedding generation.

Recommendations

Convert PDF to Markdown before submission (reduces chunks by 50-70%)
Use skipParentStorage: true in the request body
Split into smaller documents (by chapter or section)

Error Symptoms

If processing fails with these errors, the document may be too large:

Error	Meaning
`520 Origin Error`	Edge Function crashed (CPU/memory)
`504 Gateway Timeout`	Wall-clock time exceeded
`WORKER_LIMIT`	CPU quota exhausted

See Large Document Processing Limits for detailed benchmarks and the planned Netlify embedding offload solution.

maintainSource - Processes documents (called by queue processor)
rag - RAG endpoint that queries processed documents
Netlify process-queue-background - Background function for long-running operations

Frontend UI Requirements

Access Control for Non-Admin Users (accessLevel < 9)

The following UI elements should be hidden for non-admin users:

Element	Reason
"My Submissions" checkbox	Non-admins should only see their own items (filter enforced)
`testDbToggleContainer`	Testing database toggle is admin-only

Queue Item Visibility (Backend Enforced)

User Type	Queue Items Visible
Non-admin (accessLevel < 9)	Only their own items (enforced server-side, cannot be bypassed)
Admin (accessLevel >= 9)	All items (can optionally use `?user_id=mine` to filter)

Note: Non-admin users cannot see other users' queue items even if they try to pass a different user_id parameter. The backend ignores this parameter for non-admins and always filters by the authenticated user's ID.

Example Implementation

// After getting user claims from JWT
const isAdmin = claims.accessLevel >= 9;

// My Submissions checkbox - hide for non-admin, always filter by user_id
const mySubmissionsContainer = document.querySelector(
  ".my-submissions-container",
);
const mySubmissionsCheckbox = document.getElementById("mySubmissionsCheckbox");

if (!isAdmin) {
  mySubmissionsContainer?.classList.add("hidden");
  mySubmissionsCheckbox.checked = true; // Force user_id filter
} else {
  mySubmissionsContainer?.classList.remove("hidden");
}

// Test DB toggle - admin only
const testDbToggleContainer = document.getElementById("testDbToggleContainer");
if (!isAdmin) {
  testDbToggleContainer?.classList.add("hidden");
}

Netlify Function Enhancement (Recommended)

Since /process-background now sets items to submitted status before calling Netlify, the Netlify function should:

Set status to processing when it starts working on an item
Filter for submitted status if fetching items from DB (though queue_ids are now passed directly)

// In processQueueBackground, when starting to process an item:
await supabase
  .from("document_upload_queue")
  .update({ status: "processing", updated_at: new Date().toISOString() })
  .eq("id", queueId);

Change Log

Date	Description
2026-01-16	`action_type` in `/items-summary` now computed for ALL items (previously returned null for terminal statuses)
2026-01-15	`/lookup-user` now uses `users_with_email` view for O(1) indexed lookup (replaces slow `listUsers()` API)
2026-01-15	Added `action_type` field to `/items-summary` (derived from addDocs/delDocs/updDocs flags)
2026-01-15	Added `GET /items-summary` endpoint for lightweight table view (no JSONB fields, extracts source_url/source_title)
2026-01-15	Added `POST /status-check` endpoint for lightweight status polling (returns only id→status map)
2026-01-14	Added "Large Document Note" section with recommendations for documents over 200 pages or 1500+ chunks
2026-01-14	`/process-background` marks items as `failed` (not `pending`) if Netlify call fails - captures error message
2026-01-14	`/process-background` now auto-selects user's own pending items when no `queueIds` provided
2026-01-14	Added `submitted` status for tracking Edge Function → Netlify handoff
2026-01-14	Backend user filtering: Non-admin users now only see their own queue items (enforced server-side)
2026-01-14	`/submit` now sets `user_id` column for new items
2026-01-14	`/process-background` now sets status to `submitted` before calling Netlify
2026-01-14	Added ownership validation to `/process-background` - users can only process their own items (admins exempt)
2026-01-14	Added `queueIds` (plural) support to `/process-background` for batch processing
2026-01-14	Added Frontend UI Requirements section for access control
2026-01-13	Added `POST /check-source` endpoint to check if source URL exists in vector store before submission
2026-01-12	`POST /process-background` now refreshes JWT in database AND passes fresh JWT directly to Netlify
2026-01-12	`POST /process` now always refreshes JWT with caller's token (prevents expired token errors)
2026-01-01	Added `POST /refresh-jwt` endpoint to refresh expired/failed queue items with fresh JWT
2026-01-01	Added `POST /process-background` endpoint for Netlify-based background processing (avoids timeout issues)
2026-01-01	Changed `avgProcessingSeconds` to pure processing time; added `avgQueueToCompletionSeconds` for total queue time
2026-01-01	Added `user_id` filter to `GET /items` endpoint (use `?user_id=mine` for user's own submissions)
2025-12-31	Added `GET /items/:id` endpoint for single-item status polling
2025-12-31	Added fire-and-forget workflow documentation
2025-12-31	Initial README for frontend team integration

manageQueue Edge Function

Base URL

Authentication

Endpoints

Queue Items

GET /items/:id

GET /items

GET /items-summary

GET /items (Full)

PATCH /items/:id

DELETE /items/:id

Queue Statistics

GET /stats

Queue Submission

POST /submit

Queue Processing

POST /process-background (Recommended)

POST /process (Legacy)

POST /refresh-jwt

POST /complete/:id

POST /fail/:id

POST /cancel/:id

Bulk Operations

POST /bulk/status

POST /bulk/hold

POST /bulk/unhold

POST /bulk/remove

POST /bulk/replace

POST /bulk/delete

POST /bulk/metadata

Ownership Transfer

POST /transfer/:id

POST /lookup-user

Source URL Check

POST /check-source

Lightweight Status Polling

POST /status-check

Queue Item Status Values

Status Flow

Request Body Fields

Error Responses

Example: Frontend Integration (JavaScript)

Fire-and-Forget Workflow

1. Submit with async mode

2. Check status (choose one approach)

Option A: Manual refresh (recommended)

Option B: Active polling (optional)

3. Handle expired items

4. Trigger processing (admin)

Large Document Note

Recommendations

Error Symptoms

Related Functions

Frontend UI Requirements

Access Control for Non-Admin Users (accessLevel < 9)

Queue Item Visibility (Backend Enforced)

Example Implementation

Netlify Function Enhancement (Recommended)

Change Log