Chat Completion

When given a list of messages, the models return a response.

Create a Chat Completion

Creates a completion for the chat message

Azure OpenAI

Request

POST https://api.core42.ai/openai/deployments/{deployment-id}/chat/completions

Name	In	Required	Type	Description
deployment-id	path	true	string	Model to use for this request. Available models are gpt-4.1, gpt-4o, gpt-4o-audio-preview, gpt-4o-mini, o3, o3-mini, o1, jais-30b, Llama-3.3-70B, llama3-70b, mixtral-8x7b, and mistral-7b. Note: GPT-4o model supports image and text inputs and generates text output. GPT-4o mini supports text input and text outputs. Note: GPT-4o supports the following model versions: gpt-4o: defaults to gpt-4o-2024-08-06. gpt-4o-2024-08-06 gpt-4o-2024-11-20
api-version	query	true	string	For the available API versions, refer to Microsoft Azure documentation.

OpenAI

Request

POST https://api.core42.ai/v1/chat/completions

Request Parameters

Name	Required	Type	Description
model	true	string	Model to use for this request. Available models are gpt-4.1, gpt-4o, gpt-4o-audio-preview, gpt-4o-mini, o3, o3-mini, o1, jais-30b, Llama-3.3-70B, llama3-70b, mixtral-8x7b, and mistral-7b. Note: GPT-4o model supports image and text inputs and generates text output. GPT-4o mini supports text input and text outputs. Note: GPT-4o supports the following model versions: gpt-4o: defaults to gpt-4o-2024-08-06. gpt-4o-2024-08-06 gpt-4o-2024-11-20
messages[].role	true	string	The message’s author role.
messages[].content	true	string	Contents of the message. Array of content parts: An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the GPT-4o model. The supported formats for images include PNG (.png), JPEG (.jpeg, .jpg), WEBP (.webp), and non-animated GIF (.gif). For image, sample request include: { "model": "", "stream": false, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "" }, { "type": "image_url", "image_url": { "url": "" } } ] } ] }
messages[].name	false	string	User’s name in a multi-user chat.
audio	false	object or null	Parameters for audio output. Required when audio output is requested with modalities: ["audio"].
audio[].format	true	string	Specifies the output audio format. Supported formats are wav, mp3, flac, opus, or pcm16.
audio[].voice	true	string	The voice the model uses to respond. Supported voices are alloy, ash, ballad, coral, echo, sage, and shimmer.
modalities	false	array or null	Output types you would like the model to generate. The default is ["text"]. The gpt-4o-audio-preview model can also be used to generate audio. To request that this model generate both text and audio responses, use ["text", "audio"].
temperature	false	number	The temperature controls randomness. The range is from 0 to 2. Lowering results in less random completion. If the value approaches 0, the model tends to produce more predictable and deterministic responses. If the value approaches to 2, it produces more randomness, resulting in responses that are less predictable and more diverse.
top_p	false	number	An alternative to the temperature parameter, also called nucleus sampling, is where the model considers the results of the tokens with top_p probability mass. 0.1 means only the tokens comprising the top 10% probability mass are considered. It is recommended to alter this or the temperature, but not both. Setting top_p to 0.9 increases the selection of words, allowing the model to consider a wider range of options. This results in more varied and original responses, potentially leading to more imaginative and engaging suggestions for the adventure book title.
n	false	integer	Number of chat completion choices to generate for each input message. The user will be charged based on the number of generated tokens across all the choices. Keep n as 1 to minimize cost.
stream	false	boolean	Facilitates the generation of responses in a streaming fashion. The default value is set as false. When set to false, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. When streaming is enabled, messages are sent in real-time and in a partial manner. The server sends chunks of data as they become available, and the client receives these incrementally. This streaming of data continues until the server decides to close the connection or the complete response has been sent. Note: This does not apply to the Jais 30B model.
stop	false	string, array	Up to four sequences where the API will stop generating further tokens. Note: This parameter applies only to GPT-4o, GPT-4.1, GPT-4o Audio, GPT-4o mini, o3-mini, o1, Llama3.3 70B, Llama 3 70B, Mixtral 8x7B, and Mistral 7B models.
max_tokens	false	integer	The maximum number of tokens that can be generated during the chat completion. The model's context length limits the total length of input tokens and generated tokens.
max_completion_tokens	false	integer or null	The maximum number of tokens that can be generated for a completion, including visible output tokens. Note: This parameter applies only GPT-4o, GPT-4o mini, o3, o3-mini, and o1 models.
presence_penalty	false	number	Number between -2.0 and 2.0. Positive values encourage the model to include new tokens based on their appearance in the text so far to talk about new topics. Note: This does not apply to the Jais 30B model.
frequency_penalty	false	number	Allowed values are between -2.0 and 2.0. Positive values discourage the model from repeating the same text, words, or phrases based on their existing frequency. Note: This does not apply to the Jais 30B model.
repitition_penalty	false	number	Set the value to 1.1 to stop repetition of the data. Note: This parameter applies only to the Mistral 7B model.
logit_bias	false	object	Modifies the chances of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase the likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token. Note: This does not apply to the Jais 30B, Llama 3.3 70B, and Llama 3 70B models.
user	false	string	A unique identifier that represents the user.
top_k	nullable=true	integer	Number of highest probability vocabulary tokens to keep for top-k-filtering. The default value is null. Note: This parameter is only applicable to the Jais 30B, Llama 3.3 70B, and Llama 3 70B models.
do_sample	nullable=true	boolean	If set to True, this parameter enables decoding strategies such as multinomial sampling, beam-search multinomial sampling, top-k sampling, and top-p sampling. All these strategies select the next token from the probability distribution over the entire vocabulary with various strategy-specific adjustments. Note: This parameter is only applicable to the Llama 3.3 70B, and Llama 3 70B models.
max_seconds	nullable=true	integer, int32	Signed 32-bit integers (commonly used integer type).
num_beams	nullable=true	integer, int32	By specifying a number of beams higher than 1, you are effectively switching from greedy search to beam search.
ts	false	string	Timestamp of the request. For example, 2024-03-18T05:40:18.264Z
tools	false	array	A list of tools the model can call. As of now, only functions are supported as a tool. The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models. For more information, see tools parameter. The tool has type and function properties.
tools[].type	true	string	Defines the tool type. Supports only function as of now. Note: The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.
tools[].function	true	object	The function contains description, name, and parameters fields to be provided. Note: The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.
tools[].function[].description	false	string	A description of what the function does, used by the model to choose when and how to call the function.
tools[].function[].name	true	string	The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.
tools[].function[].parameters	true	object	The parameters the functions accept are described as a JSON Schema object. See the guide for examples, and the JSON Schema reference for documentation about the format. Omitting parameters defines a function with an empty parameter list.
tools[].function[].strict	false	boolean or null	Whether to enable strict schema adherence when generating the function call. If set to true, the model will follow the exact schema defined in the parameters field. Only a subset of JSON Schema is supported when strict is true.
tool_choice	false	string	Controls if any tool is called by the model. The available values are none, auto, and required. If set to none, model will not call any tool and generate a message. This is the default value when no tool is present. If set to auto, model can pick between generating a message or calling one or more tools. If set to required, model must call one or more tools by specifying a particular tool with the following format. Default when tools are present. { "type": "function", "function": { "name": "my_function" } } Note: The supported models are GPT-4.1, GPT-4o, GPT-4o Audio, o3-mini, and o1 models.
safe_prompt	false	boolean	Specifies whether to inject a safety prompt before all conversations. Note: This parameter is available only for Mistral 7B model.
random_seed	nullable=true	integer	Seed to use for random sampling. Note: This parameter is available only for Mistral 7B model.
web_search.mode	false	string	Specifies whether to enable Web Search in the response. Supported values are always and false. always: enables the Web Search. false: does not enable the Web Search. Note: This parameter is available only for the Web Search model and is currently available with GPT-4o, GPT-4o mini, Llama 3.3 70B, Llama 3 70B, Mistral 7B, and Mixtral 8x7B models.
web_search.top_results	false	integer	The number of results from Web Search. The default value is 5. Note: This parameter is available only for the Web Search model and is currently available with GPT-4o, GPT-4o mini, Llama 3.3 70B, Llama 3 70B, Mistral 7B, and Mixtral 8x7B models.
web_search.lang	false	string	The response language when Web Search is enabled. If the value is not set, the response will be in the same language as the user input. Valid values are auto, en (English), and ar (Arabic). The default value is auto. Note: This parameter is available only for the Web Search model and is currently available with GPT-4o, GPT-4o mini, Llama 3.3 70B, Llama 3 70B, Mistral 7B, and Mixtral 8x7B models.
web_search.search_results	false	boolean	Specifies whether to include the actual text from Web Search in the citations. Note: This parameter is available only for the Web Search model and is currently available with GPT-4o, GPT-4o mini, Llama 3.3 70B, Llama 3 70B, Mistral 7B, and Mixtral 8x7B models.

API Parameter Examples

stream

The stream parameter facilitates the generation of responses in a streaming fashion, allowing for real-time interaction and continuous output.

When stream = false

Sample Request Format

1  {

2    "model": "gpt-4o",

3    "stream": false,

4    "messages": [{

5        "role": "user",

6        "content": "Hello"

7    }]

8   }

9    }

Sample Response Format

Hello How are you Today?

With Streaming Disabled:

The entire response is generated at once and returned as a single block of text.
The entire narrative is available immediately, suitable for scenarios where immediate access to the complete response is preferred.

This method is useful when you need the entire generated text for further processing or analysis without waiting for it to be generated incrementally.

When stream=true

Sample Request Format

1 {

2    "model": "gpt-4o",

3    "stream": true,

4    "messages": [{

5        "role": "user",

6        "content": "Hi"

7    }]

8 }

9    }

Sample Response Format

Server: Sends a data chunk, "Hello"

Client: Receives, "Hello"

Server: Sends another data chunk, "how are you?"

Client: Receives, "how are you?"

Server: Sends the final data chunk, "today?"

Client: Receives, "today?"

With streaming enabled:

In the context of Server-Sent Events (SSE) using EventSource, messages are sent in real-time and in a partial manner. This means that the server sends pieces of data as they become available, and the client receives these pieces incrementally. This streaming of data continues until the server decides to close the connection or the complete response has been sent.
With a streaming API call, the response is sent back incrementally in chunks via an event stream. In Python, you can iterate over these events with a for loop.

The below example shows the time elapsed for each data chunk:

This method is suitable to display text as it's generated, creating a sense of anticipation and engagement.

temperature

The temperature parameter in the Compass API influences the level of randomness in the generated text. It controls how conservative or exploratory the model is when generating responses.

Examples of Request and Responses

1. temperature = 0

Request

1 {

2    "model": "gpt-4o",

3    "messages": [

4        {

5            "role": "user",

6            "content": "What are the benefits of exercising?"

7        }

8    ],

9    "temperature": 0,

10    "max_tokens": 50

11 }

12 }

Response

"Exercising improves heart health and muscle strength, lowers the chance of chronic diseases, and helps manage weight."

At a temperature of 0, the model selects the most likely words, resulting in a highly predictable and factual response. This is ideal for scenarios where accuracy is paramount, such as providing information on health benefits.

2. temperature = 1

Request

1 {

2    "model": "gpt-4o",

3    "messages": [

4        {

5            "role": "user",

6            "content": "What are the benefits of exercising?"

7        }

8    ],

9    "temperature": 1,

10    "max_tokens": 50

11 }

12 }

Response

"Exercise is the alchemist turning sweat into a miracle cure, a ritual dancing in the flames of effort and reward."

At a temperature of 1, the model takes more creative liberties and chooses less likely words, resulting in a more imaginative but unpredictable response. This level of temperature encourages the generation of diverse and creative answers, which can be suitable for contexts where engaging the audience with unique perspectives is desired.

top_p

Also known as nucleus sampling, it constrains the probability distribution to the top_p tokens. This helps in maintaining response diversity by limiting the selection of tokens based on their likelihood. Higher values of top_p allow for a wider selection of tokens, potentially leading to more varied responses.

Examples of Request and Responses

1. top_p = 0.5

Request

1 {
2    "model": "gpt-4o",
3    "messages": [
4        {
5            "role": "user",
6            "content": "Please suggest a title for an adventure book."
7        }
8    ],
9    "top_p": 0.5,
10    "max_tokens": 50
11 }
12 }

Response

"The Enigma of Emerald Island."

With a top_p value of 0.5, the model selects words that collectively represent at least 50% of the total probability distribution. This ensures a diverse range of responses while maintaining coherence.

2. top_p = 0.9

Request

1 {

2    "model": "gpt-4o",

3    "messages": [

4        {

5            "role": "user",

6            "content": "Please suggest a title for an adventure book."

7        }

8    ],

9    "top_p": 0.9,

10    "max_tokens": 50

11 }

12 }

Response

"Echoes from the Forgotten Jungle: Chronicles of the Fearless Explorer."

Explanation

Setting top_p to 0.9 expands the selection of words, allowing the model to consider a wider range of options. This results in more varied and original responses, potentially leading to more imaginative and engaging suggestions for the adventure book.

tools

In the below example, a function, get_current_weather is defined and called the model. The model can use this function to retrieve information based on the input location. When a user asks question regarding weather, this is something that model cannot tell because of lack of information, instead, it can detect the available functions to provide this kind of information. So, the model will build input parameters based on user questions and indicate which function needs to be called. Once the function is called as per the model suggestion, the response is returned. The model will eventually be able to answer the user's question regarding the weather. you can do the following with the response:

1. Send the user conversation and available functions to the model.
2. Based on the model response, call the particular functions.
3. Send information for each function call and function response to the model.

1  from openai import OpenAI

2  import json

3  client = OpenAI(

4      base_url = "https://api.core42.ai/v1",

5      default_headers={"api-key": "<API_KEY>"},

6      api_key="XXX"

7  )

8  # Example dummy function hard coded to return the same weather

9  # In production, this could be your backend API or an external API

10 def get_current_weather(location, unit="fahrenheit"):

11    """Get the current weather in a given location"""

12    if "tokyo" in location.lower():

13        return json.dumps({"location": "Tokyo", "temperature": "10", "unit": unit})

14    elif "san francisco" in location.lower():

15        return json.dumps({"location": "San Francisco", "temperature": "72", "unit": unit})

16    elif "paris" in location.lower():

17        return json.dumps({"location": "Paris", "temperature": "22", "unit": unit})

18    else:

19        return json.dumps({"location": location, "temperature": "unknown"})

20 def run_conversation():

21    # Step 1: send the conversation and available functions to the model

22    messages = [{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}]

23    tools = [

24        {

25            "type": "function",

26            "function": {

27                "name": "get_current_weather",

28                "description": "Get the current weather in a given location",

29                "parameters": {

30                    "type": "object",

31                    "properties": {

32                        "location": {

33                            "type": "string",

34                            "description": "The city and state, e.g. San Francisco, CA",

35                        },

36                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},

37                    },

38                    "required": ["location"],

39                },

40            },

41        }

42    ]

43    response = client.chat.completions.create(

44        model="gpt-4o",

45        messages=messages,

46        tools=tools,

47        tool_choice="auto",  # auto is default, but we'll be explicit

48    )

49    response_message = response.choices[0].message

50    tool_calls = response_message.tool_calls

51    # Step 2: check if the model wanted to call a function

52    if tool_calls:

53        # Step 3: call the function

54        # Note: the JSON response may not always be valid; be sure to handle errors

55        available_functions = {

56            "get_current_weather": get_current_weather,

57        }  # only one function in this example, but you can have multiple

58        messages.append(response_message)  # extend conversation with assistant's reply

59        # Step 4: send the info for each function call and function response to the model

60        for tool_call in tool_calls:

61            function_name = tool_call.function.name

62            function_to_call = available_functions[function_name]

63            function_args = json.loads(tool_call.function.arguments)

64            function_response = function_to_call(

65                location=function_args.get("location"),

66                unit=function_args.get("unit"),

67            )

68            messages.append(

69                {

70                    "tool_call_id": tool_call.id,

71                    "role": "tool",

72                    "name": function_name,

73                    "content": function_response,

74                }

75            )  # extend conversation with function response

76        second_response = client.chat.completions.create(

77            model="gpt-4o",

78            messages=messages,

79        )  # get a new response from the model where it can see the function response

80        return second_response

81  print(run_conversation())

82 }

Web Search Example

Following is the sample request and response for Web Search.

Request

1 {

2    "model": "gpt-4o",

3    "stream": false,

4    "messages": [

5        {

6            "role": "user",

7            "content": "How to make chocolate cake",

8            "max_tokens": 100,

9            "temperature": 0.7,

10            "top_p": 0.95

11        }

12    ],

13    "web_search": {

14        "mode": "always",

15        "top_results":5,

16        "lang": "en",

17        "search_results": true

18    }

19 }

Response

1  {

2    "choices": [

3        {

4            "finish_reason": "stop",

5            "index": 0,

6            "logprobs": null,

7            "message": {

8                "content": "To make a chocolate cake, follow these general steps, combining insights from various sources:

9

10 1. **Preheat and Prepare Pans**: Preheat your oven to 180°C (160°C Fan/Gas 4) and grease and line your cake pans, usually two 20cm/8in sandwich tins [url1].

11 

12 2. **Gather Ingredients**: You will typically need butter, sugar, eggs, flour, baking cocoa, and baking soda. For a specific recipe, you'll require 2/3 cup softened butter, 1-2/3 cups sugar, 3 large eggs, 2 cups all-purpose flour, and 2/3 cup baking cocoa, among other ingredients [url3].

13

14 3. **Mix Dry Ingredients**: In a large bowl, mix all your dry ingredients together [url4].

15

16 4. **Combine Wet Ingredients**: In a separate container, combine your liquid ingredients like vanilla, milk, and coffee. Add these to the dry ingredients all at once. Mix on low speed for about a minute, stop and scrape the bowl, then mix for 30 seconds more [url4].

17

18 5. **Add Boiling Water**: If specified, add boiling water to the mixture to ensure the batter is smooth and evenly mixed [url5].

19

20 6. **Bake**: Pour the batter into the prepared pans and bake in the preheated oven. Follow the designated baking time in your recipe, usually around 30-35 minutes, or until a toothpick inserted into the center comes out clean [url5].

21

22 7. **Cool and Frost**: Once baked, remove the cakes from the oven, let them cool completely, and then apply a chocolate frosting of your choice [url2].

23

24 This is a general guide on making chocolate cake. Adjustments can be made based on personal taste preferences or specific recipes.",

25                "role": "assistant"

26            }

27        }

28    ],

29    "created": 1732260038,

30    "id": "chatcmpl-AWIFyaT15sNqXU1IMV9oZ8TKjzooR",

31    "model": "gpt-4o-2024-08-06",

32    "object": "chat.completion",

33    "system_fingerprint": "fp_d54531d9eb",


34    "usage": {

35        "completion_tokens": 380,

36        "prompt_tokens": 952,

37        "total_tokens": 1332

38    },

39    "web_search_citations": {

40        "url1": {

41            "citation": "https://www.allrecipes.com/article/easiest-chocolate-cake-from-scratch/",

42            "content": "Try this 5-star recipe for the Best Chocolate Frosting. It whips up quickly and takes only 5 ingredients to make. Bonus Tip: If you're making a layered sheet cake, you'll want to double the recipe. To remove the sheet cake from the pan, cover it with a sheet of parchment paper, lay a cutting board over the pan, then invert the whole thing."

43        },

44        "url2": {

45            "citation": "https://scientificallysweet.com/the-best-moist-chocolate-cake/",


46            "content": "The BEST Moist Chocolate Cake recipe is so unbelievably EASY, made in just one bowl with no mixer required and a fudgy chocolate frosting! ... INGREDIENTS FOR THE BEST CHOCOLATE CAKE. Here are some notes about the ingredients. For a full list of ingredients, check out the recipe card at the bottom of the page. ... If I wanted to use 4-inch pans ..."

47        },

48        "url3": {

49            "citation": "https://www.tasteofhome.com/recipes/classic-chocolate-cake/",

50            "content": "Ingredients. 2/3 cup butter, softened; 1-2/3 cups sugar; 3 large eggs, room temperature; 2 cups all-purpose flour; 2/3 cup baking cocoa; 1-1/4 teaspoons baking soda"

51        },

52        "url4": {

53            "citation": "https://www.kingarthurbaking.com/recipes/chocolate-cake-recipe",

54            "content": "In a large measuring cup, combine the vanilla with the milk and coffee (or water), and add all at once to the dry ingredients. Mix for 1 minute at low speed, stop and scrape the sides and bottom of the bowl, then mix for 30 seconds more."


55        },

56        "url5": {

57            "citation": "https://www.bbc.co.uk/food/recipes/easy_chocolate_cake_31070",

58            "content": "Method. Preheat the oven to 180C/160C Fan/Gas 4. Grease and line two 20cm/8in sandwich tins. For the cake, place all of the cake ingredients, except the boiling water, into a large mixing bowl."

59        }

60    }

61 }