Module 03: Working with LLM APIs

Duration: 4 hours | Difficulty: Intermediate | Prerequisites: Modules 01–02

Learning Objectives

By the end of this module, you will be able to:

• Integrate OpenAI, Anthropic Claude, and Google Gemini APIs into Python applications
• Implement streaming responses for real-time user experiences
• Use function calling / tool use to extend LLM capabilities
• Handle errors, rate limits, and retries gracefully
• Estimate and optimize API costs across providers

1. The LLM API Landscape

Why Multiple Providers?

Nobody wins every benchmark forever. In production you might pick by latency, price, context, compliance, or modalities (vision, audio).

2. OpenAI API Deep Dive

Basic Chat Completion

Most tutorials start here: messages is an ordered transcript; temperature and max_tokens shape behavior.

Key Example: Non-streaming chat with usage fields you’ll log for cost dashboards.

[object Object], openai ,[object Object], OpenAI

client = OpenAI(api_key=,[object Object],)

response = client.chat.completions.create(
    model=,[object Object],,
    messages=[
        {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
        {,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],},
    ],
    temperature=,[object Object],,
    max_tokens=,[object Object],,
    top_p=,[object Object],,
)

,[object Object],(response.choices[,[object Object],].message.content)
,[object Object],(,[object Object],, response.usage.total_tokens)
,[object Object],(,[object Object],, response.choices[,[object Object],].finish_reason)

Streaming (Conceptual)

With stream=True, you iterate chunks and print partial text—chat UIs feel alive. You still concatenate chunks server-side if you need the final string for storage.

3. Anthropic Claude API

Claude’s Python SDK uses messages.create with system as a top-level string (not always a message with role system in the older style).

4. Google Gemini API

Gemini often starts with GenerativeModel(model_name) then generate_content or a start_chat session object for multi-turn state.

5. Universal LLM Client

Why teams build a thin wrapper

You don’t need abstraction on day one—add it when you have two providers or two environments.

6. Error Handling and Retry Logic

Typical production errors:

Key Example: Exponential backoff on rate limits—pattern is the same across SDKs; swap the client call for Anthropic/Gemini.

[object Object], time
,[object Object], openai ,[object Object], OpenAI, RateLimitError, APIConnectionError, APITimeoutError

client = OpenAI(api_key=,[object Object],)

,[object Object], ,[object Object],(,[object Object],):
    ,[object Object], attempt ,[object Object], ,[object Object],(max_retries):
        ,[object Object],:
            ,[object Object], client.chat.completions.create(
                model=model,
                messages=messages,
                timeout=,[object Object],,
            )
        ,[object Object], (RateLimitError, APIConnectionError, APITimeoutError):
            ,[object Object], attempt == max_retries - ,[object Object],:
                ,[object Object],
            time.sleep(base_delay * (,[object Object],**attempt))


resp = chat_with_retry(
    [{,[object Object],: ,[object Object],, ,[object Object],: ,[object Object],}]
)
,[object Object],(resp.choices[,[object Object],].message.content)

7. Cost Optimization Strategies

Treat tokens like cloud spend: log per route, alert on spikes, cap per user.

Practice Exercises

Mini-Project: LLM API Gateway

FastAPI service: /chat accepts {model, messages} → routes to provider, caches deterministic prompts, /stats returns spend estimates. Start with OpenAI only; add a second provider behind the same response schema.

Key Takeaways

Resources for Further Learning

• OpenAI API Reference
• Anthropic API Documentation
• Google Gemini API
• LiteLLM
• OpenRouter

← Previous: Prompt Engineering | Next: Prompt Chaining →