概覽
串流允許您在 token 生成時即時接收,而不是等待完整回應。這大幅改善了使用者感知到的延遲。何時使用串流
- 聊天介面 — 顯示打字指示器和即時文字
- 長回應 — 不讓使用者等待完整回應
- 程式碼生成 — 即時顯示正在撰寫的程式碼
運作方式
- 在請求中設定
stream: true - OpenModex 開啟 SSE(Server-Sent Events)連線
- Token 以
data:事件的形式在生成時發送 - 串流以
data: [DONE]結束
Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
透過 Server-Sent Events 即時逐 token 傳遞回應。
stream: truedata: 事件的形式在生成時發送data: [DONE] 結束const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Tell me a story.' }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Tell me a story."}],
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
print(content, end="", flush=True)
stream, _ := client.Chat.Completions.CreateStream(ctx, &openmodex.ChatCompletionParams{
Model: "gpt-4o",
Messages: []openmodex.ChatMessage{{Role: "user", Content: "Tell me a story."}},
})
for chunk, err := range stream.Iter() {
if err != nil { break }
fmt.Print(chunk.Choices[0].Delta.Content)
}
import { useChat } from '@openmodex/react';
function Chat() {
const { messages, input, setInput, sendMessage, isLoading } = useChat({
apiKey: 'omx_sk_...',
model: 'gpt-4o',
});
return (
<div>
{messages.map((m) => (
<div key={m.id}>{m.role}: {m.content}</div>
))}
<input value={input} onChange={(e) => setInput(e.target.value)} />
<button onClick={() => sendMessage()}>Send</button>
</div>
);
}
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Hello!' }],
stream: true,
routing: {
fallback: ['claude-3.5-sonnet'],
},
});