RAG API Breaking Changes

Last Updated: 2025-10-22 Impact: Breaking changes to authentication and SSE event payloads Migration Required: Client-side changes needed

🔐 Authentication Breaking Changes (JWT v3 Migration)

Date: 2025-10-22 Status: ⚠️ BREAKING - Immediate action required

Summary

The RAG function now uses JWT v3 authentication with claims extracted from the app_claims namespace. Client applications MUST NOT send org_id, userId, or access_level in the request body anymore - these are now automatically extracted from the JWT token.

What Changed

❌ OLD WAY (v2) - Request Body Parameters

const response = await fetch("/functions/v1/rag", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${userJwt}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    org_id: "fpp", // ❌ Remove this
    userId: "user-uuid", // ❌ Remove this
    access_level: 9, // ❌ Remove this
    sessionId: "chat-123",
    chatMessages: {
      user: "What is AI?",
      history: [],
    },
    modelVersion: "gpt-4o",
    // ... other params
  }),
});

✅ NEW WAY (v3) - JWT-Based Authentication

const response = await fetch("/functions/v1/rag", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${userJwt}`, // ✅ JWT contains all auth info
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    // org_id, userId, access_level are NOT needed!
    // They're extracted from JWT automatically
    sessionId: "chat-123",
    chatMessages: {
      user: "What is AI?",
      history: [],
    },
    modelVersion: "gpt-4o",
    // ... other params
  }),
});

JWT Structure (v3)

The server now expects JWTs with this structure:

{
  "sub": "auth-id-uuid", // Supabase Auth ID
  "email": "user@example.com",
  "role": "authenticated",
  "app_claims": {
    "user_id": "user-table-id-uuid", // Users table primary key
    "access_level": 9, // Data visibility level
    "org_id": "fpp", // Organization ID
    "role": "authenticated"
  }
}

Required Request Parameters (Updated)

Parameter	Type	Required	Default	Description
Authorization header	`Bearer ${jwt}`	✅ Yes	-	JWT token with v3 `app_claims`
`sessionId`	`string`	❌ No	`"streaming-test-rag"`	Session ID for conversation history
`chatMessages`	`object`	✅ Yes	-	Chat messages (see below)
`chatMessages.user`	`string`	✅ Yes	-	User's question
`chatMessages.history`	`array`	❌ No	`[]`	Previous chat history
`modelVersion`	`string`	❌ No	`OPENAI_DEFAULT_MODEL`	OpenAI model to use
`metadataFilter`	`object`	❌ No	`undefined`	Metadata filtering
`apiKey`	`string`	❌ No	`OPENAI_API_KEY`	OpenAI API key (override)
`specificity`	`string`	❌ No	`"1"`	Query specificity
`useScored`	`boolean`	❌ No	`USE_SCORED_SEARCH`	Use scored search
`verbose`	`boolean`	❌ No	`false`	Enable verbose logging
~~`org_id`~~	-	❌ REMOVED	-	Now extracted from JWT
~~`userId`~~	-	❌ REMOVED	-	Now extracted from JWT
~~`access_level`~~	-	❌ REMOVED	-	Now extracted from JWT

Error Responses (Authentication)

Missing or Invalid JWT

{
  "error": "Unauthorized: Missing or invalid Bearer token",
  "status": 401
}

JWT Missing app_claims Namespace

{
  "type": "error",
  "message": "JWT missing 'app_claims' namespace. Please sign in again to get a fresh token.",
  "timestamp": "2025-10-22T10:30:00.000Z"
}

JWT Missing org_id

{
  "type": "error",
  "message": "JWT app_claims missing 'org_id'. Please contact your administrator.",
  "timestamp": "2025-10-22T10:30:00.000Z"
}

Insufficient Access Level

{
  "type": "error",
  "message": "Access denied: Insufficient access level (0). Please contact your administrator.",
  "timestamp": "2025-10-22T10:30:00.000Z"
}

Migration Checklist (Authentication)

[ ] Remove org_id from request body
[ ] Remove userId from request body
[ ] Remove access_level from request body
[ ] Ensure JWT token is sent in Authorization: Bearer ${token} header
[ ] Verify JWT contains app_claims namespace with org_id, user_id, and access_level
[ ] Users may need to sign in again to get fresh v3 tokens
[ ] Update error handling to catch 401 authentication errors
[ ] Test with fresh JWT tokens

Additional Resources

See CLIENT_MIGRATION_GUIDE.md for complete JWT v3 migration instructions including:

How to decode and extract JWT claims client-side
Distinction between authId (Supabase Auth) and userId (users table)
TypeScript type definitions
Complete code examples

📊 SSE Event Payload Changes (HTML to JSON Migration)

Date: 2025-10-13 Status: ⚠️ BREAKING - Client rendering changes required

Summary

The RAG function has been refactored to follow REST API best practices by returning structured JSON data instead of pre-rendered HTML. This separates concerns between server (data) and client (presentation).

Changed Event Types

1. `events-sources-chart` (REMOVED)

Old Behavior:

{
  type: "events-sources-chart",
  chunk: "pie showData \n title Source Contribution (%)\n\n  \"#1\" : 45\n  \"#2\" : 35\n  \"#3\" : 20"
}

Returned Mermaid chart syntax as a string
Required Mermaid.js on client to render

Replacement: Use events-sources-chart-js instead (already available)

2. `events-sources-table` → `events-sources-data`

Old Behavior:

{
  type: "events-sources-table",
  chunk: "<table class=\"table table-striped\">...</table>"
}

Returned fully rendered HTML table with Bootstrap classes
Client simply inserted HTML into DOM

New Behavior:

{
  type: "events-sources-data",
  chunk: JSON.stringify({
    sources: [
      { index: 1, source: "https://example.com/doc1", percent: 45 },
      { index: 2, source: "https://example.com/doc2", percent: 35 },
      { index: 3, source: "https://example.com/doc3", percent: 20 }
    ],
    usedAttention: true,
    totalSources: 3
  })
}

Client Migration:

// Parse the JSON
const data = JSON.parse(event.chunk);

// Render table yourself
const tableHTML = `
  <table class="sources-table table table-striped">
    <thead>
      <tr>
        <th>#</th>
        <th>Source</th>
        <th>Percent</th>
      </tr>
    </thead>
    <tbody>
      ${data.sources
        .map(
          (s) => `
        <tr>
          <td>${s.index}</td>
          <td>${s.source}</td>
          <td>${s.percent}%</td>
        </tr>
      `
        )
        .join("")}
    </tbody>
  </table>
`;

3. `events-llm-sources-no-contribution` → `events-sources-no-contribution`

Old Behavior:

{
  type: "events-llm-sources-no-contribution",
  chunk: `
    <div class="events-llm-sources-no-contribution text-bg-warning-outline">
      <div class="p-3 text-center">
        <h6>🤖 LLM Self-Assessment: Sources Did Not Contribute</h6>
        <p class="mb-1">The system retrieved 5 documents, but...</p>
        <small class="text-muted">Confidence: 85% | ...</small>
      </div>
    </div>
  `
}

Returned fully styled HTML with Bootstrap classes

New Behavior:

{
  type: "events-sources-no-contribution",
  chunk: JSON.stringify({
    sourcesContributed: false,
    documentsRetrieved: 5,
    confidence: 85,
    reasoning: "Documents did not contain relevant information",
    message: "The system retrieved 5 documents, but the AI determined they did not provide relevant information for answering your question."
  })
}

Client Migration:

// Parse the JSON
const data = JSON.parse(event.chunk);

// Render your own styled component
const messageHTML = `
  <div class="alert alert-warning">
    <h6>🤖 LLM Self-Assessment: Sources Did Not Contribute</h6>
    <p>${data.message}</p>
    <small>Confidence: ${data.confidence}% | ${data.reasoning}</small>
  </div>
`;

4. `events-answer-confidence-display` (REMOVED)

Old Behavior:

{
  type: "events-answer-confidence-display",
  chunk: `<div class="confidence-display" data-confidence="85">
    <div class="confidence-value">85%</div>
    <div class="progress">...</div>
    <div class="confidence-details">...</div>
  </div>`
}

Returned fully rendered HTML with Bootstrap classes and inline styles
Client simply inserted HTML into DOM

Replacement: Use events-answer-confidence JSON data instead

New Behavior: The events-answer-confidence event already provides all the data needed:

{
  type: "events-answer-confidence",
  chunk: JSON.stringify({
    confidence: 85,
    reasoning: "Sources provided relevant information that contributed to the answer",
    sourcesFound: 5,
    sourcesContributed: true,
    llmAssessmentConfidence: 90
  })
}

Client Migration:

// Parse the JSON
const data = JSON.parse(event.chunk);

// Render your own confidence display
const confidenceHTML = renderConfidenceDisplay(data);
document.getElementById("confidence").innerHTML = confidenceHTML;

function renderConfidenceDisplay(data) {
  const {
    confidence,
    reasoning,
    sourcesFound,
    sourcesContributed,
    llmAssessmentConfidence,
  } = data;

  // Determine confidence level and styling
  const getConfidenceLevel = (conf) => {
    if (conf >= 75) return { level: "High", accent: "success" };
    if (conf >= 50) return { level: "Medium", accent: "warning" };
    return { level: "Low", accent: "danger" };
  };

  const { level, accent } = getConfidenceLevel(confidence);

  // Determine confidence note
  const getConfidenceNote = (contributed) => {
    if (contributed) return "Sources provided relevant information";
    return "Answer based on general knowledge";
  };

  const note = getConfidenceNote(sourcesContributed);

  return `
    <div class="confidence-display" data-confidence="${confidence}" data-level="${level}">
      <div class="confidence-value text-${accent} fw-bold">${confidence}%</div>

      <div class="progress" role="progressbar" aria-label="Answer confidence">
        <div class="progress-bar bg-${accent}" style="width:${confidence}%"></div>
      </div>

      ${
        llmAssessmentConfidence
          ? `
        <div class="badge bg-${accent} bg-opacity-25 mt-2">
          LLM self-assessment: ${llmAssessmentConfidence}%
        </div>
      `
          : ""
      }

      <div class="confidence-details">
        <div class="confidence-level">
          <i class="bi bi-shield-check"></i>
          <span>${level} confidence</span>
        </div>
        <div class="confidence-sources">
          <i class="bi bi-collection"></i>
          <span>${sourcesFound} sources</span>
        </div>
      </div>

      <div class="confidence-note text-muted">
        ${note}
      </div>

      <div class="visually-hidden" aria-live="polite">
        ${reasoning}
      </div>
    </div>
  `;
}

Unchanged Event Types (Still Working)

These events were already sending JSON and continue to work:

✅ events-sources-chart-js - Chart.js configuration (JSON) ✅ events-metadata-summary - Metadata summaries (JSON) ✅ events-answer-confidence - Confidence scores (JSON) ✅ events-sources-attention - Attention weights (JSON) ✅ results - Answer text chunks (string) ✅ error - Error messages (JSON)

Benefits of This Change

Separation of Concerns - Server handles data, client handles presentation
Flexibility - Client can style/render data however they want
Reusability - Same data can be rendered differently in different contexts
Smaller Payloads - JSON is typically smaller than HTML
Type Safety - Structured data is easier to validate and type-check
Testability - Easier to test data vs. testing HTML strings

Migration Checklist (Complete)

Authentication (JWT v3) ✅

[ ] Remove org_id, userId, access_level from request body
[ ] Ensure Authorization header includes Bearer token
[ ] Verify JWT has app_claims namespace
[ ] Update error handling for 401 responses
[ ] Users sign in again for fresh v3 tokens
[ ] Test with real authenticated requests

SSE Event Rendering ✅

[ ] Update SSE event handlers to recognize new event types
[ ] Parse JSON from events-sources-data instead of inserting HTML
[ ] Create client-side table rendering function
[ ] Parse JSON from events-sources-no-contribution instead of inserting HTML
[ ] Create client-side warning/alert rendering function
[ ] Parse JSON from events-answer-confidence and create client-side confidence display rendering function
[ ] Remove handling for deprecated events-sources-chart (Mermaid)
[ ] Remove handling for deprecated events-sources-table
[ ] Remove handling for deprecated events-llm-sources-no-contribution
[ ] Remove handling for deprecated events-answer-confidence-display
[ ] Test with real RAG responses
[ ] Update any documentation referencing old event types

Questions?

Contact the backend team for assistance with migration or if you need additional data fields in the JSON responses.

For JWT v3 authentication issues, see CLIENT_MIGRATION_GUIDE.md for detailed migration instructions.

📖 Complete Migration Example

Before (v2) - Old Implementation

// ❌ OLD: Send auth params in body, use HTML event types
async function callRAG_OLD(question, jwt) {
  // Extract auth info manually from JWT
  const payload = decodeJWT(jwt);
  const orgId = payload.org_id; // From top-level (v2)
  const userId = payload.userId; // From top-level (v2)
  const accessLevel = payload.accessLevel; // From top-level (v2)

  const response = await fetch("/functions/v1/rag", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${jwt}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      org_id: orgId, // ❌ Sent in body
      userId: userId, // ❌ Sent in body
      access_level: accessLevel, // ❌ Sent in body
      sessionId: "chat-123",
      chatMessages: {
        user: question,
        history: [],
      },
      modelVersion: "gpt-4o",
    }),
  });

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    const chunk = decoder.decode(value);
    const lines = chunk.split("\n\n");

    for (const line of lines) {
      if (!line.startsWith("data: ")) continue;

      const data = JSON.parse(line.slice(6));

      // ❌ OLD: Direct HTML insertion
      if (data.type === "events-sources-table") {
        document.getElementById("sources").innerHTML = data.chunk;
      }

      if (data.type === "events-answer-confidence-display") {
        document.getElementById("confidence").innerHTML = data.chunk;
      }

      if (data.type === "events-llm-sources-no-contribution") {
        document.getElementById("warning").innerHTML = data.chunk;
      }

      if (data.type === "results") {
        document.getElementById("answer").textContent += data.chunk;
      }
    }
  }
}

After (v3) - New Implementation

// ✅ NEW: JWT-based auth, JSON event rendering
async function callRAG_NEW(question, jwt) {
  // No need to extract auth info - server does it automatically!

  const response = await fetch("/functions/v1/rag", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${jwt}`, // ✅ JWT contains everything
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      // ✅ No auth params in body!
      sessionId: "chat-123",
      chatMessages: {
        user: question,
        history: [],
      },
      modelVersion: "gpt-4o",
    }),
  });

  // Handle authentication errors
  if (response.status === 401) {
    console.error("Authentication failed - user needs to sign in again");
    showError("Please sign in again to continue");
    return;
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    const chunk = decoder.decode(value);
    const lines = chunk.split("\n\n");

    for (const line of lines) {
      if (!line.startsWith("data: ")) continue;

      const data = JSON.parse(line.slice(6));

      // ✅ NEW: Parse JSON and render with your own templates
      if (data.type === "events-sources-data") {
        const sourceData = JSON.parse(data.chunk);
        const html = renderSourcesTable(sourceData);
        document.getElementById("sources").innerHTML = html;
      }

      if (data.type === "events-answer-confidence") {
        const confidenceData = JSON.parse(data.chunk);
        const html = renderConfidenceDisplay(confidenceData);
        document.getElementById("confidence").innerHTML = html;
      }

      if (data.type === "events-sources-no-contribution") {
        const warningData = JSON.parse(data.chunk);
        const html = renderWarning(warningData);
        document.getElementById("warning").innerHTML = html;
      }

      if (data.type === "results") {
        document.getElementById("answer").textContent += data.chunk;
      }

      if (data.type === "error") {
        console.error("RAG error:", data.message);
        showError(data.message);
      }
    }
  }
}

// ✅ NEW: Custom rendering functions
function renderSourcesTable(data) {
  return `
    <div class="sources-container">
      <h4>Sources (${data.totalSources})</h4>
      <table class="table table-striped">
        <thead>
          <tr><th>#</th><th>Source</th><th>Contribution</th></tr>
        </thead>
        <tbody>
          ${data.sources
            .map(
              (s) => `
            <tr>
              <td>${s.index}</td>
              <td><a href="${s.source}">${s.source}</a></td>
              <td>${s.percent}%</td>
            </tr>
          `
            )
            .join("")}
        </tbody>
      </table>
    </div>
  `;
}

function renderConfidenceDisplay(data) {
  const level =
    data.confidence >= 75 ? "High" : data.confidence >= 50 ? "Medium" : "Low";
  const color =
    data.confidence >= 75
      ? "success"
      : data.confidence >= 50
      ? "warning"
      : "danger";

  return `
    <div class="confidence-display">
      <div class="text-${color} fw-bold">${data.confidence}%</div>
      <div class="progress">
        <div class="progress-bar bg-${color}" style="width:${data.confidence}%"></div>
      </div>
      <small>${level} confidence • ${data.sourcesFound} sources</small>
    </div>
  `;
}

function renderWarning(data) {
  return `
    <div class="alert alert-warning">
      <h6>🤖 Sources Did Not Contribute</h6>
      <p>${data.message}</p>
      <small>Confidence: ${data.confidence}%</small>
    </div>
  `;
}

Key Differences

Aspect	v2 (Old)	v3 (New)
Authentication	Send `org_id`, `userId`, `access_level` in body	Automatic extraction from JWT
JWT Structure	Top-level claims	`app_claims` namespace
Request Body	Includes auth params	Only business logic params
Error Handling	SSE errors only	401 status + SSE errors
Event Types	HTML strings	JSON data
Rendering	Direct HTML insertion	Parse JSON + custom templates
Flexibility	Fixed server styling	Full client control
Type Safety	HTML strings	Structured data

Need Help?

JWT v3 Migration: See CLIENT_MIGRATION_GUIDE.md
Edge Function Updates: See PHASE_4_COMPLETE.md
Questions: Contact the backend team

// Old approach
eventSource.addEventListener("message", (event) => {
  const data = JSON.parse(event.data);

  if (data.type === "events-sources-table") {
    // Just insert HTML
    document.getElementById("sources").innerHTML = data.chunk;
  }
});

// New approach
eventSource.addEventListener("message", (event) => {
  const data = JSON.parse(event.data);

  if (data.type === "events-sources-data") {
    // Parse structured data
    const sourceData = JSON.parse(data.chunk);

    // Render with your own template
    const html = renderSourcesTable(sourceData);
    document.getElementById("sources").innerHTML = html;
  }
});

function renderSourcesTable(data) {
  return `
    <div class="sources-container">
      <h4>Sources (${data.totalSources})</h4>
      ${
        data.usedAttention
          ? '<span class="badge">Attention Weighted</span>'
          : ""
      }
      <table class="table">
        <thead>
          <tr>
            <th>#</th>
            <th>Source</th>
            <th>Contribution</th>
          </tr>
        </thead>
        <tbody>
          ${data.sources
            .map(
              (s) => `
            <tr>
              <td>${s.index}</td>
              <td><a href="${s.source}" target="_blank">${s.source}</a></td>
              <td>
                <div class="progress">
                  <div class="progress-bar" style="width: ${s.percent}%">
                    ${s.percent}%
                  </div>
                </div>
              </td>
            </tr>
          `
            )
            .join("")}
        </tbody>
      </table>
    </div>
  `;
}

Questions?

Contact the backend team for assistance with migration or if you need additional data fields in the JSON responses.

RAG API Breaking Changes

🔐 Authentication Breaking Changes (JWT v3 Migration)

Summary

What Changed

❌ OLD WAY (v2) - Request Body Parameters

✅ NEW WAY (v3) - JWT-Based Authentication

JWT Structure (v3)

Required Request Parameters (Updated)

Error Responses (Authentication)

Missing or Invalid JWT

JWT Missing app_claims Namespace

JWT Missing org_id

Insufficient Access Level

Migration Checklist (Authentication)

Additional Resources

📊 SSE Event Payload Changes (HTML to JSON Migration)

Summary

Changed Event Types

1. events-sources-chart (REMOVED)

2. events-sources-table → events-sources-data

3. events-llm-sources-no-contribution → events-sources-no-contribution

4. events-answer-confidence-display (REMOVED)

Unchanged Event Types (Still Working)

Benefits of This Change

Migration Checklist (Complete)

Authentication (JWT v3) ✅

SSE Event Rendering ✅

Questions?

📖 Complete Migration Example

Before (v2) - Old Implementation

After (v3) - New Implementation

Key Differences

Need Help?

Questions?

1. `events-sources-chart` (REMOVED)

2. `events-sources-table` → `events-sources-data`

3. `events-llm-sources-no-contribution` → `events-sources-no-contribution`

4. `events-answer-confidence-display` (REMOVED)