OpenAI 兼容性
2024 年 2 月 8 日
Ollama 现在内置了与 OpenAI Chat Completions API 的兼容性,使得在本地使用更多工具和应用程序与 Ollama 成为可能。
设置
首先下载 Ollama 并拉取一个模型,例如 Llama 2 或 Mistral
ollama pull llama2
用法
cURL
要调用 Ollama 兼容 OpenAI 的 API 端点,请使用相同的 OpenAI 格式 并将主机名更改为 https://127.0.0.1:11434
curl https://127.0.0.1:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "llama2",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Hello!"
}
]
}'
OpenAI Python 库
from openai import OpenAI
client = OpenAI(
base_url = 'https://127.0.0.1:11434/v1',
api_key='ollama', # required, but unused
)
response = client.chat.completions.create(
model="llama2",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
{"role": "assistant", "content": "The LA Dodgers won in 2020."},
{"role": "user", "content": "Where was it played?"}
]
)
print(response.choices[0].message.content)
OpenAI JavaScript 库
import OpenAI from 'openai'
const openai = new OpenAI({
baseURL: 'https://127.0.0.1:11434/v1',
apiKey: 'ollama', // required but unused
})
const completion = await openai.chat.completions.create({
model: 'llama2',
messages: [{ role: 'user', content: 'Why is the sky blue?' }],
})
console.log(completion.choices[0].message.content)
示例
Vercel AI SDK
Vercel AI SDK 是一个用于构建对话式流应用程序的开源库。要开始使用,请使用 create-next-app
克隆示例仓库
npx create-next-app --example https://github.com/vercel/ai/tree/main/examples/next-openai example
cd example
然后,在 app/api/chat/route.ts
中进行以下两处编辑,以更新聊天示例以使用 Ollama
const openai = new OpenAI({
baseURL: 'https://127.0.0.1:11434/v1',
apiKey: 'ollama',
});
const response = await openai.chat.completions.create({
model: 'llama2',
stream: true,
messages,
});
接下来,运行应用程序
npm run dev
最后,在浏览器中打开示例应用程序,地址为 https://127.0.0.1:3000
Autogen
Autogen 是微软流行的开源框架,用于构建多代理应用程序。对于此示例,我们将使用 Code Llama 模型
ollama pull codellama
安装 Autogen
pip install pyautogen
然后创建一个 Python 脚本 example.py
,以将 Ollama 与 Autogen 结合使用
from autogen import AssistantAgent, UserProxyAgent
config_list = [
{
"model": "codellama",
"base_url": "https://127.0.0.1:11434/v1",
"api_key": "ollama",
}
]
assistant = AssistantAgent("assistant", llm_config={"config_list": config_list})
user_proxy = UserProxyAgent("user_proxy", code_execution_config={"work_dir": "coding", "use_docker": False})
user_proxy.initiate_chat(assistant, message="Plot a chart of NVDA and TESLA stock price change YTD.")
最后,运行该示例,让助手编写代码来绘制图表
python example.py
更多内容即将推出
这是对 OpenAI API 的初始实验性支持。正在考虑的未来改进包括
- Embeddings API(嵌入 API)
- Function calling(函数调用)
- Vision support(视觉支持)
- Logprobs(对数概率)