Server-Sent Events (SSE)

Overview

Yes, this setup is implementing Server-Sent Events (SSE) to stream data from the server to the client. Here's how it works in the context of our setup:

Server-Sent Events (SSE)

Server-Sent Events is a standard describing how servers can initiate data transmission towards browser clients once an initial client connection has been established. It is commonly used to send real-time updates to a web application in a way that is more efficient than traditional polling.

Implementation in Our Setup

Client-Side JavaScript (consoleClient.js): The client-side script makes a request to the Netlify Edge Function and listens for events using the EventSource interface. In our setup, we're using the fetch API with a ReadableStream to manually handle the SSE data, but you could also use EventSource for a more straightforward implementation.
Netlify Edge Function (simple_Edge_Test.js): This function acts as a proxy, forwarding the client's request to the Netlify Serverless Function. It doesn't directly deal with SSE, but it ensures that the response from the Serverless Function is passed back to the client correctly.
Netlify Serverless Function (simple_Serverless.js): This is where the SSE implementation is crucial. The function generates a stream of data and sends it back to the client in the SSE format. Each message is prefixed with data: and suffixed with two newline characters (\n\n) as per the SSE specification. The Content-Type header is set to text/event-stream to indicate that the response is an SSE stream.

How SSE Works in This Setup

The client initiates a connection by making a request to the Netlify Edge Function.
The Edge Function forwards the request to the Serverless Function.
The Serverless Function starts generating data and sends it back in the SSE format. The response is streamed, meaning that data is sent as soon as it's available without waiting for the entire response to be generated.
The client receives the streamed data and processes it. In the case of SSE, the client typically listens for messages and updates the UI based on the received data.
The connection remains open, allowing the server to send additional data at any time until the client closes the connection or an error occurs.

This SSE implementation allows for real-time data streaming from the server to the client, making it ideal for applications that require constant data updates, such as live feeds, notifications, or real-time analytics.

Detailed Explanation

Certainly! Here's a detailed documentation of the entire setup for streaming data from a Netlify Serverless Function through a Netlify Edge Function to a client-side application:

Client-Side JavaScript (consoleClient.js)

This script sends a POST request to the Netlify Edge Function and processes the streamed response.

const btnSubmit = document.getElementById("btnSubmit");
const dataDiv = document.getElementById("streamed-data");

btnSubmit.addEventListener("click", async function (event) {
  event.preventDefault();

  const topic = document.getElementById("topic").value;
  const url = `${window.location.origin}/simple`;

  try {
    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ topic: topic }),
    });
    console.log(`consoleClient response.status: ${response.status}`);
    if (!response.ok) {
      throw new Error("Failed to get response from server");
    }

    const reader = response.body.getReader();
    const decoder = new TextDecoder("utf-8");
    let content = "";

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      content += decoder.decode(value, { stream: true });

      const messages = content.split("\n\n");
      for (const message of messages.slice(0, -1)) {
        const data = message.replace("data: ", "");
        try {
          const parsedData = JSON.parse(data);
          dataDiv.innerText += parsedData.chunk;
        } catch (error) {
          console.error("Error parsing JSON:", error);
        }
      }
      content = messages[messages.length - 1];
    }
  } catch (error) {
    console.error("Error:", error);
  }
});

Netlify Edge Function (simple_Edge_Test.js)

This Edge Function forwards the request to the Serverless Function and streams the response back to the client.

export default async (request, context) => {
  let protocol, topic;

  try {
    if (request.method == "GET") {
      const url = new URL(request.url);
      topic = url.searchParams.get("topic") || "dog";
    } else if (request.method == "POST") {
      const body = await request.json();
      topic = body.topic || "dog";
    } else {
      return new Response("Invalid Method", { status: 405 });
    }

    const host = request.headers.get("host");
    protocol = host.includes("localhost") ? "http" : "https";
    const serverlessFunctionUrl = `${protocol}://${host}/.netlify/functions/simple_Serverless`;

    console.log("simple_Edge_test topic", topic);

    const response = await fetch(serverlessFunctionUrl, {
      method: "POST",
      body: JSON.stringify({ topic }),
      headers: {
        "Content-Type": "application/json",
      },
    });

    console.log(`simple_Edge_test response.status: ${response.status}`);

    return new Response(response.body, {
      status: response.status,
      headers: response.headers,
    });
  } catch (error) {
    console.error("Error:", error);
    return new Response("Internal Server Error", { status: 500 });
  }
};

export const config = {
  path: "/simple",
};

Netlify Serverless Function (simple_Serverless.js)

This Serverless Function generates a stream of data and sends it back to the Edge Function.

import { stream } from "@netlify/functions";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatOpenAI } from "@langchain/openai";

export const handler = stream(async function (event) {
  let topic;
  try {
    if (event.httpMethod == "POST") {
      const body = JSON.parse(event.body);
      topic = body.topic || "squirrel";
    } else if (event.httpMethod === "GET") {
      topic = event.queryStringParameters.topic || "squirrel";
    } else {
      throw new Error("Invalid HTTP Method");
    }

    console.log(`simple_Serverless_test topic: ${topic}`);

    const model = new ChatOpenAI({});
    const prompt = ChatPromptTemplate.fromTemplate(
      "Write a short poem about {topic}"
    );
    const parser = new StringOutputParser();
    const chain = prompt.pipe(model).pipe(parser);
    const stream = await chain.stream({ topic: topic });

    const responseBody = new ReadableStream({
      async start(controller) {
        for await (const chunk of stream) {
          controller.enqueue(`data: ${JSON.stringify({ chunk })}\n\n`);
        }
        controller.close();
      },
    });

    return {
      headers: {
        "content-type": "text/event-stream",
      },
      statusCode: 200,
      body: responseBody,
    };
  } catch (error) {
    console.error("error", error);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: "Error calling simple_Serverless_test" }),
    };
  }
});

How It Works

The client-side JavaScript (consoleClient.js) sends a POST request with a topic to the Netlify Edge Function (simple_Edge_Test.js).
The Edge Function forwards the request to the Netlify Serverless Function (simple_Serverless.js), which generates a stream of data based on the provided topic.
The Serverless Function sends the streamed data back to the Edge Function, which then streams it back to the client.
The client-side JavaScript processes the streamed data and updates the webpage accordingly.

This setup allows for efficient handling of long-running tasks and streaming responses, reducing the risk of timeouts and ensuring a smooth user experience.