Chat Completion

model

true

string

Model to use for this request. Available models are gpt-4.1, gpt-4o, gpt-4o-audio-preview, gpt-5, gpt-5-mini, gpt-4o-mini, o3, o3-mini, o4-mini-2025-04-16, o1, jais-30b, Llama-3.3-70B, llama3-70b, mixtral-8x7b, mistral-7b, gpt-oss-120b-crerebras, gpt-oss-120b-core42, gpt-oss-120b-azure, gpt-oss-120b-core42-amd, gpt-oss-20b-core42-amd, gpt-oss-120b-qualcomm, gpt-oss-20b-qualcomm, gpt-oss-20b-core42, claude-sonnet-4, qwen3-14b, command-a-03-2025, and cohere-command-r-08-2024.

Note: GPT-4o model supports image and text inputs and generates text output.

GPT-4o mini supports text input and text outputs.

Note: GPT-4o supports the following model versions:

• gpt-4o: defaults to gpt-4o-2024-08-06

• gpt-4o-2024-08-06

• gpt-4o-2024-11-20

messages[].role

true

string

The message’s author role.

messages[].content

true

string

Contents of the message.
Array of content parts:
An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the GPT-4o model.
The supported formats for images include PNG (.png), JPEG (.jpeg, .jpg), WEBP (.webp), and non-animated GIF (.gif).
For image, sample request include:
{ "model": "", "stream": false, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "" }, { "type": "image_url", "image_url": { "url": "" } } ] } ] }

messages[].name

false

string

User’s name in a multi-user chat.

audio

false

object or null

Parameters for audio output. Required when audio output is requested with modalities: ["audio"].

audio[].format

true

string

Specifies the output audio format. Supported formats are wav, mp3, flac, opus, or pcm16.

When using gpt-4o-audio-preview and stream is set to true, the only supported audio format is pcm16.

audio[].voice

true

string

The voice the model uses to respond. Supported voices are alloy, echo,
fable, onyx, nova, and shimmer.

modalities

false

array or null

Output types you would like the model to generate. The default is

["text"].

The gpt-4o-audio-preview model can also be used to generate audio.

To request that this model generate both text and audio responses, use

["text", "audio"].

temperature

false

number

The temperature controls randomness. The range is from 0 to 2. Lowering results in less random completion. If the value approaches 0, the model tends to produce more predictable and deterministic responses. If the value approaches to 2, it produces more randomness, resulting in responses that are less predictable and more diverse.

top_p

false

number

An alternative to the temperature parameter, also called nucleus sampling, is where the model considers the results of the tokens with top_p probability mass. 0.1 means only the tokens comprising the top 10% probability mass are considered. It is recommended to alter this or the temperature, but not both. Setting top_p to 0.9 increases the selection of words, allowing the model to consider a wider range of options. This results in more varied and original responses, potentially leading to more imaginative and engaging suggestions for the adventure book title.

n

false

integer

Number of chat completion choices to generate for each input message. The user will be charged based on the number of generated tokens across all the choices. Keep n as 1 to minimize cost.

stream

false

boolean

Facilitates the generation of responses in a streaming fashion. The default value is set as false. When set to false, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. When streaming is enabled, messages are sent in real-time and in a partial manner. The server sends chunks of data as they become available, and the client receives these incrementally. This streaming of data continues until the server decides to close the connection or the complete response has been sent.
Note: This does not apply to the Jais 30B model.

stop

false

string, array

Up to four sequences where the API will stop generating further tokens.
Note: This parameter applies only to GPT-4o, GPT-4.1, GPT-4o Audio, GPT-4o mini, o3-mini, o1, Llama3.3 70B, Llama 3 70B, Mixtral 8x7B, and Mistral 7B models.

max_tokens

false

integer

The maximum number of tokens that can be generated during the chat completion. The model's context length limits the total length of input tokens and generated tokens.

max_completion_tokens

false

integer or null

The maximum number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.

Note: This parameter applies only to GPT-4o, GPT-4o mini, o3, o3-mini, and o1 models.

presence_penalty

false

number

Number between -2.0 and 2.0. Positive values encourage the model to include new tokens based on their appearance in the text so far to talk about new topics.

Note: This does not apply to the Jais 30B model.

frequency_penalty

false

number

Allowed values are between -2.0 and 2.0. Positive values discourage the model from repeating the same text, words, or phrases based on their existing frequency.
Note: This does not apply to the Jais 30B model.

repitition_penalty

false

number

Set the value to 1.1 to stop repetition of the data.
Note: This parameter applies only to the Mistral 7B model.

logit_bias

false

object

Modifies the chances of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase the likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
Note: This does not apply to the Jais 30B, Llama 3.3 70B, and Llama 3 70B models.

user

false

string

A unique identifier that represents the user.

safety_identifier

false

string

A stable identifier used to help detect users of your application that may be violating OpenAI's usage policies. The IDs should be a string that uniquely identifies each user. Compass recommends hashing their username or email address to avoid sending us any identifying information. Learn more about safety_identifier.

prompt_cache_key

false

string

Used by OpenAI to cache responses for similar requests to optimize your cache hit rates. Replaces the user field. Learn more about prompt_cache_key.

store

false

boolean or null

Whether or not to store the output of this chat completion request for use in our model distillation or evals products. The default value is false

Supports text and image inputs.

Note: Image inputs over 8MB in size will be dropped.

service_tier

false

string or null

Specifies the processing type used for serving the request.

• If set to auto, then the request will be processed with the service tier configured in the Project settings. Unless otherwise configured, the Project will use default.

• If set to default, then the request will be processed with the standard pricing and performance for the selected model.

• If set to flex or priority, then the request will be processed with the corresponding service tier.

• When not set, the default behavior is auto.

When the service_tier parameter is set, the response body will include the service_tier value based on the processing mode actually used to serve the request. This response value may be different from the value set in the parameter.

top_logprobs

false

integer or null

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

top_k

nullable=true

integer

Number of highest probability vocabulary tokens to keep for top-k-filtering. The default value is null.
Note: This parameter is only applicable to the Jais 30B, Llama 3.3 70B, and Llama 3 70B models.

do_sample

nullable=true

boolean

If set to True, this parameter enables decoding strategies such as multinomial sampling, beam-search multinomial sampling, top-k sampling, and top-p sampling. All these strategies select the next token from the probability distribution over the entire vocabulary with various strategy-specific adjustments.
Note: This parameter is only applicable to the Llama 3.3 70B, and Llama 3 70B models.

max_seconds

nullable=true

integer, int32

Signed 32-bit integers (commonly used integer type).

num_beams

nullable=true

integer, int32

By specifying a number of beams higher than 1, you are effectively switching from greedy search to beam search.

ts

false

string

Timestamp of the request. For example, 2024-03-18T05:40:18.264Z

tools

false

array

A list of tools the model can call. As of now, only functions are supported as a tool. The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.

For more information, see tools parameter. The tool has type and function properties.

tools[].type

true

string

Defines the tool type. Supports only function as of now.
Note: The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.

tools[].function

true

object

The function contains description, name, and parameters fields to be provided.
Note: The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.

tools[].function[].description

false

string

A description of what the function does, used by the model to choose when and how to call the function.

tools[].function[].name

true

string

The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.

tools[].function[].parameters

true

object

The parameters the functions accept are described as a JSON Schema object. See the guide for examples, and the JSON Schema reference for documentation about the format.

Omitting parameters defines a function with an empty parameter list.

tools[].function[].strict

false

boolean or null

Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the parameters field. Only a subset of JSON Schema is supported when strict is true.

tool_choice

false

string

Controls if any tool is called by the model. The available values are none, auto, and required. If set to none, the model will not call any tool and generate a message. This is the default value when no tool is present. If set to auto, the model can pick between generating a message or calling one or more tools. If set to required, the model must call one or more tools by specifying a particular tool with the following format. Default when tools are present.
{ "type": "function", "function": { "name": "my_function" } }
Note: The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.

verbosity

false

string or null

Constrains the verbosity of the model's response. Lower values will result in more concise responses, while higher values will result in more verbose responses. Currently supported values are low, medium, and high.

safe_prompt

false

boolean

Specifies whether to inject a safety prompt before all conversations.

Note: This parameter is only available for Mistral 7B model.

reasoning_effort

false

string or null

Constrains effort on reasoning for reasoning models. Currently supported values are minimal, low, medium, and high. The default value is medium.

Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.

Note: This parameter is available only with GPT-5, GPT-5 mini, gpt-oss-120b, gpt-oss-20b, o1, o3, o3-mini, and o4-mini models.

random_seed

nullable=true

integer

Seed to use for random sampling.
Note: This parameter is only available for Mistral 7B model.

thinking

false

object

Configuration for enabling extended thinking.

When enabled, responses include thinking content blocks that show Claude's thought process before the final answer. Requires a minimum budget of 1,024 tokens and counts towards your max_tokens limit.

See extended thinking for details.

Note: This parameter is available only with the Claude Sonnet 4, and Qwen 3 14B models.

thinking_budget

false

integer

Specifies the maximum length of the thinking process. This parameter is applicable only when enable_thinking is set to true and applies to Qwen3 commercial and open-source models. For more information, see Limit the thinking length.

When you call the API using the Python SDK, configure this parameter using extra_body. The configuration is as follows: extra_body={"thinking_budget": xxx}.

Note: This parameter is available only with the Qwen 3 14B model.

documents

false

list of strings or object

A list of relevant documents that the model can cite to generate a more accurate reply. Each document is either a string or a document object with content and metadata.

Note: This parameter is available only with the Cohere Command R model.

documents[].data

true

object

A relevant document that the model can cite to generate a more accurate reply. Each document is a string or a dictionary.

Note: This parameter is available only with the Cohere Command R model.

documents[].id

false

string

A unique identifier for this document, which will be referenced in citations. If not provided, an ID will be automatically generated.

Note: This parameter is available only with the Cohere Command R model.

citation_options

false

object

Options for controlling citation generation.

mode

false

enum

Defaults to "accurate". Dictates the approach taken to generating citations as part of the RAG flow by allowing the user to specify whether they want "accurate" results, "fast" results, or no results.

The default is "fast".

Allowed values: FAST, ACCURATE, OFF

Note: This parameter is available only with the Cohere Command R model.

response_format

false

object

Configuration for forcing the model output to adhere to the specified format. Supported on Command R models.

The model can be forced into outputting JSON objects by setting { "type": "json_object" }.

A JSON Schema can be optionally provided to ensure a specific structure.

Note: When using { "type": "json_object" } your message should always explicitly instruct the model to generate a JSON (eg: “Generate a JSON …”) .

Otherwise, the model may end up getting stuck, generating an infinite stream of characters, and eventually run out of context length.

Note: When json_schema is not specified, the generated object can have up to 5 layers of nesting.

Limitation: The parameter is not supported when used in combination with the documents or tools parameters.

Note: This parameter is available only with the Cohere Command R model.

min_p

false

number

Dynamic filtering threshold that adapts based on token probabilities. The default value is 0.05.

Note: This parameter applies only to Qwen3-Reranker-8B, and Qwen3-14B models.