Inference

Overview

Call X-Serve from your own code. The API is OpenAI-compatible, so an existing OpenAI client works once you change the base URL and the key. This page covers streaming, controlling how much the model reasons, calling your own tools, the Anthropic-compatible path that some clients expect, and what prompt caching saves you.

If you haven't sent a request yet, start with the Quickstart.

Before you start

You need an API key in your environment and a positive credit balance. The Quickstart covers both.

export NETPREME_API_KEY="sk-..."

Every example below points at https://api.netpreme.com/v1.

Step-by-step guide

Step 1: stream a response

Set stream to true and the reply arrives as server-sent events instead of one object.

curl -N https://api.netpreme.com/v1/chat/completions \
  -H "Authorization: Bearer $NETPREME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.2-FP8",
    "stream": true,
    "messages": [{"role": "user", "content": "count to three"}]
  }'

Each event carries a delta. Reasoning arrives in delta.reasoning_content and the answer in delta.content, so a client that only reads content shows nothing until the reasoning finishes:

data: {"choices":[{"delta":{"reasoning_content":"1","role":"assistant"},"index":0}],"model":"zai-org/GLM-5.2-FP8","object":"chat.completion.chunk"}

The stream ends with data: [DONE].

Step 2: control how much the model reasons

Both models reason by default and return it separately from the answer. Send reasoning_effort to change that.

curl https://api.netpreme.com/v1/chat/completions \
  -H "Authorization: Bearer $NETPREME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.2-FP8",
    "reasoning_effort": "none",
    "messages": [{"role": "user", "content": "In one sentence, what can you help me with?"}]
  }'

Accepted values, lowest to highest:

none, minimal, low, medium, high, xhigh, max

At none the model answers directly, so the model fills content and leaves reasoning_content null. At every other level the model reasons first. An unrecognized value returns 400 and names the whole set:

{"message":"Failed to deserialize the JSON body into the target type: unknown variant `bogus`, expected one of `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`","type":"Bad Request","code":400}

Not every model serves every level, and the Console shows which. Under each model on the Coding Agents page there is a reasoning ladder with the unsupported rungs greyed out:

Model	Serves
`zai-org/GLM-5.2-FP8`	`none`, `high`, `max`
`zai-org/GLM-5.3`	`none`, `low`, `high`, `max`
`moonshotai/Kimi-K3`	`none`, `low`, `high`, `max`

none turns reasoning off and works on both. Stay on the levels listed, because a model handles an unlisted one poorly rather than falling back to its nearest rung. One model may happen to accept an unlisted level today, but that's incidental and the two models differ.

Reasoning isn't free. Those tokens count as output, so a high setting costs more and can consume your whole max_tokens budget before the answer starts. If a reply comes back with finish_reason of length and an empty content, raise max_tokens or lower the effort.

Step 3: call a tool

Pass tools and the model can ask you to run one. This is a two-turn exchange: the model returns the call it wants, you run it, and you send the result back.

curl https://api.netpreme.com/v1/chat/completions \
  -H "Authorization: Bearer $NETPREME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.2-FP8",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string", "description": "City name"}},
          "required": ["city"]
        }
      }
    }]
  }'

The model answers with finish_reason of tool_calls and no content. Run the function yourself, then send the conversation back with a tool message carrying the result. The chat completions reference has both response shapes.

The second call comes back with finish_reason of stop and the answer written from your result. Both models handle this the same way.

Step 4: use the Anthropic-compatible endpoint

Clients built for the Anthropic Messages API can talk to X-Serve without translation. Post to /v1/messages instead of /v1/chat/completions.

curl https://api.netpreme.com/v1/messages \
  -H "Authorization: Bearer $NETPREME_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.2-FP8",
    "max_tokens": 2048,
    "messages": [{"role": "user", "content": "In one sentence, what can you help me with?"}]
  }'

Authentication is the one thing that differs from the Anthropic API. X-Serve reads Authorization: Bearer, and an x-api-key header returns 401, so a client that defaults to x-api-key needs pointing at the other one. This is why the Claude Code setup uses ANTHROPIC_AUTH_TOKEN rather than ANTHROPIC_API_KEY.

The response uses the Anthropic shape. Reasoning arrives as a thinking content block rather than a separate field, and max_tokens is mandatory:

{
  "id": "msg_...",
  "type": "message",
  "role": "assistant",
  "model": "zai-org/GLM-5.2-FP8",
  "content": [{"type": "thinking", "thinking": "..."}, {"type": "text", "text": "..."}],
  "stop_reason": "end_turn",
  "usage": {"input_tokens": 23, "output_tokens": 655}
}

Be generous with max_tokens here. Thinking draws on the same budget, so the model can spend a value that looks ample before the answer starts. The one-sentence question in the preceding request needs around 650 output tokens. At 512 the reply comes back with stop_reason of max_tokens and a thinking block but no text at all.

To count tokens without running a completion, post the same body to /v1/messages/count_tokens. It returns {"input_tokens": 15} for the preceding request.

Step 5: switch models

Both models take the same request shape, so changing model is a one-word change.

curl https://api.netpreme.com/v1/models \
  -H "Authorization: Bearer $NETPREME_API_KEY"

See Available Models for what each one accepts.

Step 6: what you have now

You can stream, control reasoning cost, and reach the service from either an OpenAI client or an Anthropic one. Every request appears in Usage in the Console and draws down your credit balance.

What caching saves you

Repeating a prefix is cheap. The gateway caches what it has already processed, and every response reports how much of your input it reused:

"usage": {"prompt_tokens": 2011, "prompt_tokens_details": {"cached_tokens": 1920}}

That's a real measurement. Sending the same 2,011-token prompt three times in a row returned cached_tokens of 0, then 1,920, then 1,920: after the first call, 95% of the prompt came from cache.

This matters because cached input carries its own rate, separate from ordinary input. Anything that repeats, a long system prompt, a document you keep asking about, a conversation that grows one turn at a time, gets cheaper on the second call onward. Put the stable part of your prompt first so the cache covers more of it.