Creating an API with hono.js to stream OpenAI API responses.

hono.js v3.7.0 now supports SSE (Server-Sent Events)!

I am truly happy. I have been waiting for this.

Using this, I will try streaming chunks that are (mostly) compatible with the OpenAI API from Cloudflare Workers.

Code

	const openai = new OpenAI({ apiKey: c.env.OPENAI_API_KEY })
	const chatStream = await openai.chat.completions.create({
		messages: PROMPT(body.messages),
		model: 'gpt-3.5-turbo-0613',
		stream: true
	})

	c.header('Content-Type', 'text/event-stream')

	return c.streamText(async (stream) => {
		for await (const message of chatStream) {
			await stream.write(`data: ${JSON.stringify(message)}\n\n`);
		}
		await stream.write(`data: [DONE]\n\n`);
	});

Also, the PROMPT looks something like this.

const PROMPT = (messages: OpenAI.Chat.ChatCompletionMessage[]) => [
  {
    role: 'system' as const,
    content:
      '＜システムプロンプト＞'
  },
  ...messages,
]

Don't forget the imports as well.

import { Hono } from 'hono'
import { OpenAI } from 'openai'

I feel like I might be doing things in a roundabout way, but this works.

Wrapping the API with Cloudflare Workers is great because it allows you to keep your Secret Key hidden.

Please also follow me on X.
https://twitter.com/31pi_