Deploy and run
AI models through CLI
One OpenAI-compatible endpoint for every model the operator offers, paid from project credit: keys for your apps, prices per model, usage and a monthly cap.
Your apps can use AI models (chat, embeddings) through CLI instead of signing up with each AI company. CLI gives them one address that works like OpenAI's, charges each call to your project's credit for the tokens actually used, and shows what each model cost you.
How to do it: in project settings, open AI models, make a key for the app, and paste the two values it shows into the app's variables. Any library that works with OpenAI works unchanged.
"Give my app access to AI models through CLI and set a $10 a day limit."
There are no AI company accounts, cards or keys to manage, and you can cap spending per key.
Your apps can call AI models through CLI instead of through a provider account of their own. CLI gives them one OpenAI-compatible endpoint for every model on offer, charges each call to the project's credit at the tokens it actually used, and shows what each model cost. There are no provider accounts, cards or keys for you or your agent to manage.
Use it from an app
Create an AI key for the app (project settings, AI models, or POST /v1/projects/:id/ai/keys). You get two variables to set on the app:
OPENAI_BASE_URL=https://<your CLI>/v1/ai
OPENAI_API_KEY=clai_…Any OpenAI SDK picks those up. Ask for a model by the name in the list:
const r = await openai.chat.completions.create({ model: "fast", messages: [{ role: "user", content: "Hi" }] });The answer is the usual chat completion, with usage, plus cli.cost (micro-USDC) and an X-CLI-Cost header.
Agents can call it with their project key (or a signed-in session) and an X-CLI-Project: <project id> header, or with cli_ai.
Models and prices
GET /v1/ai/models lists the models with what they cost per million input and output tokens, in micro-USDC. Those are the provider's prices plus the operator's markup. Each call costs exactly the tokens used; nothing is charged for calls the provider fails.
Spending limits
- Before a call goes out CLI works out the most it could cost (the prompt's size and
max_tokens) and refuses it if the project's balance can't cover that (INSUFFICIENT_FUNDS). - Monthly cap: an admin can cap a project's AI spending per calendar month (
PUT /v1/projects/:id/ai/cap {"monthlyCap": <micro-USDC>}, ornullfor none). A call that could pass it is refused withAI_CAP_REACHED. Lowermax_tokenslowers the worst case. - Rate: a project can make a limited number of calls a minute (120 unless the operator changes it).
Usage
Project settings, AI models, shows this month's spending and a breakdown by model. GET /v1/projects/:id/ai has the same, with the keys. Every call is also in the ledger as AI usage.
Keys
Keys are per project, shown once, and can be revoked at any time (DELETE /v1/projects/:id/ai/keys/:keyId). Give each app its own key so you can see and cut off one without touching the others.
Streaming and tool calls
stream: true works the way OpenAI's API does: chat.completion.chunk events, then data: [DONE]. The last chunk carries usage and cli.cost; the call is charged when the stream ends. Checks (credit, cap, model) happen before anything is streamed, so a refused call is an ordinary error response.
Tools work too: send tools and tool_choice as you would to OpenAI, and the answer has tool_calls; send the results back as tool messages. For Anthropic models CLI translates both ways, including tool calls in a stream.
Embeddings
POST /v1/ai/embeddings {"model": "<an embedding model>", "input": "…" | ["…"]} returns OpenAI-style vectors, charged per input token. Store them in Postgres with vector search on (/docs/vector-search).
Without an account: pay per call
An agent with a wallet and no CLI account can call the gateway too. Without an Authorization header CLI answers 402 with an x402 challenge for the call's worst case (from the prompt size and max_tokens); paying it gets the answer. That price is the call's price; each payment buys one call.
For operators
Set CLI_AI_PROVIDERS (each provider's kind, openai or anthropic, base URL and key) and CLI_AI_MODELS (each model's name, provider, the provider's model name and its price per million tokens). CLI_AI_MARKUP_BPS sets the markup (1000 is 10%). See .env.example.