Frontend API Updates - January 2026

New features for handling large document uploads, fire-and-forget processing, and queue management. Modified: 2026-Jan-19 20:29:29 UTC

Frontend API Updates - January 2026

Summary: New features for handling large document uploads, fire-and-forget processing, and queue management.

Quick Reference

Feature Endpoint/Parameter Purpose
Fire-and-forget POST /maintainSource with async: true Returns immediately with queueId
Skip parent storage POST /maintainSource with skipParentStorage: true Faster processing for large files
Background processing POST /manageQueue/process-background Triggers Netlify processor (15 min timeout)
Refresh expired items POST /manageQueue/refresh-jwt Refresh JWT and reset to pending
My queue items GET /manageQueue/items?user_id=mine List current user's submissions
Single item status GET /manageQueue/items/:id Poll specific queue item

1. Fire-and-Forget Document Upload

Submit documents and get an immediate response without waiting for processing.

Request

const response = await fetch(`${SUPABASE_URL}/functions/v1/maintainSource`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    sourceUrl: "https://example.com/document.pdf",
    sourceDate: "2026-01-01",
    addDocs: true,
    async: true, // Fire-and-forget mode
  }),
});

const { queueId } = await response.json();
// HTTP 202 - Returns immediately

Response (HTTP 202)

{
  "success": true,
  "statusCode": 202,
  "message": "Document queued for processing",
  "queueId": "uuid-here",
  "userId": "user-uuid",
  "orgId": "org-id",
  "sourceUrl": "https://example.com/document.pdf"
}

2. Faster Processing for Large Files

Use skipParentStorage: true to bypass parent document storage. This significantly speeds up processing for large files (10MB+).

{
  sourceUrl: 'https://example.com/large-document.pdf',
  sourceDate: '2026-01-01',
  addDocs: true,
  skipParentStorage: true  // Skip parent doc storage
}

Trade-offs:

  • Faster processing (no Storage bucket overhead)
  • No parent document retrieval capability (vector chunks only)
  • Standard 400-char chunks instead of larger parent chunks

3. Checking Queue Status

const response = await fetch(
  `${SUPABASE_URL}/functions/v1/manageQueue/items?user_id=mine`,
  { headers: { Authorization: `Bearer ${jwt}` } }
);

const { items } = await response.json();
// items = [{ id, status, created_at, edge_function_response, ... }]

Option B: Poll Specific Item

const response = await fetch(
  `${SUPABASE_URL}/functions/v1/manageQueue/items/${queueId}`,
  { headers: { Authorization: `Bearer ${jwt}` } }
);

const item = await response.json();
// item.status: 'pending' | 'processing' | 'completed' | 'failed'
// item.edge_function_response: results when completed

Polling with Exponential Backoff

async function pollQueueStatus(queueId, jwt, maxWaitMs = 600000) {
  let delayMs = 1000;
  const maxDelayMs = 30000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxWaitMs) {
    const response = await fetch(
      `${SUPABASE_URL}/functions/v1/manageQueue/items/${queueId}`,
      { headers: { Authorization: `Bearer ${jwt}` } }
    );

    const item = await response.json();

    if (item.status === "completed") {
      return { success: true, result: item.edge_function_response };
    }
    if (item.status === "failed") {
      return { success: false, error: item.error_message };
    }

    await new Promise((r) => setTimeout(r, delayMs));
    delayMs = Math.min(delayMs * 1.5, maxDelayMs);
  }

  throw new Error("Queue processing timeout");
}

4. Triggering Background Processing (Admin)

For admin UIs that need to trigger queue processing:

const response = await fetch(
  `${SUPABASE_URL}/functions/v1/manageQueue/process-background`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${jwt}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      batchSize: 5, // Process up to 5 items
    }),
  }
);

const { items_queued, queue_ids } = await response.json();
// HTTP 202 - Processing started in background

5. Refreshing Expired Queue Items

When queue items fail due to JWT expiration, refresh them with the current user's token and re-process.

Refresh and Reset to Pending

// Refresh all expired/failed items with current user's JWT
const response = await fetch(
  `${SUPABASE_URL}/functions/v1/manageQueue/refresh-jwt`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${jwt}`, // Fresh JWT from current session
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      // Optional: refresh specific items by ID
      // itemIds: ['uuid-1', 'uuid-2'],
      // Optional: filter by status (default: ['expired', 'failed'])
      // statusFilter: ['expired']
    }),
  }
);

const { refreshed, itemIds } = await response.json();
// { message: "Refreshed JWT and reset 3 items to pending", refreshed: 3, itemIds: [...] }

Typical Flow for Handling Expired Items

// 1. User sees "expired" items in queue management UI
// 2. User clicks "Refresh & Retry" button
// 3. Call refresh-jwt endpoint with current session token
const refreshResult = await fetch(
  `${SUPABASE_URL}/functions/v1/manageQueue/refresh-jwt`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${jwt}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ statusFilter: ["expired", "failed"] }),
  }
);

// 4. Items are now "pending" with fresh JWT
// 5. Trigger background processing
await fetch(`${SUPABASE_URL}/functions/v1/manageQueue/process-background`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${jwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ batchSize: 10 }),
});

Note: The endpoint uses the caller's current Authorization header JWT to update the stored user_jwt in queue items.

6. Queue Item Status Values

Status Description
pending Waiting to be processed
processing Currently being processed
completed Successfully processed
failed Processing failed (may auto-retry)
cancelled Manually cancelled
expired Exceeded max retry attempts
held Manually held, won't process

6. Completed Item Response

When status === 'completed', the edge_function_response contains:

{
  "success": true,
  "statusCode": 200,
  "message": "Successfully processed...",
  "duration": 12500,
  "processingStats": {
    "parentDocs": { "deleted": 0, "added": 25 },
    "vectorDocs": { "deleted": 0, "added": 150 },
    "sourceDocsLoaded": 45,
    "fileType": "pdf",
    "loadMode": "remote"
  }
}

Note: When skipParentStorage: true, parentDocs.added will always be 0.

Full Documentation