Your First AI Interaction

Every API call to a modern LLM follows the same basic structure: send a request containing a model name, a list of messages, and optional parameters; the model generates a response; you receive the response and extract the text. Understanding each component of this request is the foundation for building agents that behave reliably in production.

The Anthropic SDK for Python wraps this structure in client.messages.create(). The required parameters are model (which specific model to call),max_tokens (the maximum number of tokens to generate), andmessages (a list of dictionaries with role andcontent keys). The response object contains the generated text atmessage.content[0].text and token usage information atmessage.usage.

The equivalent call structure in the OpenAI SDK usesclient.chat.completions.create() with the same conceptual parameters. Most agent frameworks abstract over both APIs using a common interface, which is why understanding the underlying request structure matters more than memorising any one SDK's syntax.

A token is the basic unit of text that a language model processes. In English, a token is roughly 4 characters or 0.75 words. "Unbelievable" is about 3 tokens; "AI" is 1 token. Pricing, context limits, and rate limits are all measured in tokens. Developers who do not track token usage in development are routinely surprised by production costs.

A single LLM call costs a small fraction of a cent. An agent that makes 20 calls in a loop, each with a long context window (system prompt plus conversation history plus tool results), can accumulate costs quickly. The Anthropic SDK returns token usage with every response: message.usage.input_tokens for the tokens you sent and message.usage.output_tokens for the tokens generated. Log both in development to calibrate your expectations before scaling.

The context window is the maximum number of tokens a model can process in a single call, including both input and output. Claude 3.5 Sonnet supports 200,000 tokens. This is generous, but an agent that accumulates long tool results over many steps can fill it. Strategies for managing context length, such as summarising older turns, are covered in later modules. For now, track token counts and set conservativemax_tokens limits while experimenting.

Temperature is a parameter between 0 and 1 that controls the randomness of the model's output. Temperature 0 makes the model near-deterministic, always favouring the most probable next token. Temperature 1 introduces significant variation. The same prompt at temperature 0 produces almost identical output on repeated calls; at temperature 1, outputs can differ substantially.

For agent tasks, including any scenario where the model must produce structured JSON for a tool call, use temperature 0. High temperature increases the chance of the model generating malformed JSON, deviating from an expected format, or including text before or after the JSON object that breaks parsing. The Avianca case involved a model generating confident, plausible-sounding fabrications: a property of statistical next-token prediction that temperature 0 does not eliminate but does reduce significantly.

Temperature 0.3 to 0.5 suits summarisation and analysis tasks where some variation in phrasing is acceptable but consistency matters. Temperature 0.7 to 1.0 is appropriate for creative writing and brainstorming where varied outputs are desirable. Set temperature explicitly in every API call rather than relying on the provider's default, which may differ between models.

The system prompt sets the context and persona for the model before any user message. It is the mechanism by which agents receive their instructions, tool descriptions, and behavioural constraints. In the Anthropic SDK, pass it as thesystem parameter to client.messages.create(). It is not part of the messages list; it occupies a separate, privileged position in the request.

An effective system prompt defines four things: scope (what the agent will and will not help with), tone (formal, concise, friendly), format (maximum response length, use of lists or prose), and fallback behaviour (what the agent says when a request is outside its scope). A weak system prompt such as "You are a helpful assistant" provides none of these constraints and produces inconsistent behaviour at scale.

System prompts are counted as input tokens on every call. A 500-token system prompt on an agent that makes 50 calls in a session contributes 25,000 input tokens before any user message or tool result is counted. This is expected and acceptable; just include it in your token budget calculations.

A conversation is represented as a list of messages with alternating userand assistant roles. The API is stateless: the model has no memory between calls. To continue a conversation, you must append both the model's previous response and the next user message to the history, then send the full list with every call. The model reads the entire history each time and generates the next assistant message.

This design choice is intentional. Stateless APIs are simpler to scale, easier to audit, and give the application full control over what context the model receives. The trade-off is that the application must manage conversation state. This responsibility sits in your code, not in the provider's infrastructure.

The practical implementation is a list called conversation_history that starts empty. Each turn appends a user message, calls the API with the full list, receives a response, and appends the assistant message. The function accepts the current history and a new user message, returns the assistant response and the updated history. Keeping the history management explicit and visible makes debugging straightforward: you can print the full history at any point to see exactly what the model received.