Skip to content
CodeKruPro

Call an LLM API with the OpenAI SDK

9/99

Module · Python and LLM APIs bridge

beginner

about 25 min

What this lesson teaches

The idea

Many AI providers share the same chat API, so one Python library, the OpenAI SDK, can talk to all of them, including this course's practice bot.

Why a tester cares

The library waits a long time and retries quietly by default. In a test or a support chat a slow reply is itself a failure, so you set limits.

Words you will meet

SDK:
a library that wraps an API so you call functions instead of writing HTTP requests.
Timeout:
how long to wait for a reply before giving up.
Retry:
sending the same request again after a failure.

What you will do

  1. 1Run the script
  2. 2Set timeout and retries
  3. 3Handle a timeout
  4. 4Submit
Jump to practice

Most LLM providers, and many company gateways, speak the same chat API, so the official openai Python SDK works against all of them. In this lesson the SDK points at the practice bot: the lesson runner sets OPENAI_BASE_URL and OPENAI_API_KEY, which the SDK reads on its own.

The script works, but its client uses the SDK's defaults, which wait up to ten minutes for a reply. A support chatbot that makes a customer wait that long has failed, and a test harness that hangs that long is useless.

Your task

  1. Create the client with a 2 second timeout and 3 retries.
  2. If the request still times out, print exactly {"error": "timeout"} as one line of JSON instead of crashing.
  3. Keep printing the content and total tokens as one line of JSON when the call succeeds.

On submit, the script runs against a normal bot, a bot that refuses the first request with 429 (rate limited), and a bot that takes four seconds to answer.

Sign in to practise

The lesson is free to read. A free account lets you talk to the practice bots, run your code and save your score. You will come back to this lesson.

Sign inCreate a free account
1

Your code

against the practice bot

How this is checked: When you submit, your script runs on 3 sets of data, 2 of them hidden, and the server compares what it prints with the right values. Run tries it on the data you can see.

Core checks earn 50 points and the harder checks the other 50. You can submit as many times as you like.

About this lesson

OpenAI Python SDK timeout and retries: stop an LLM test hanging for ten minutes

AI Testing for QA EngineersBeginnerPython and LLM APIs bridge

How it works

Many providers and company gateways speak the same chat API, so the official OpenAI Python SDK works against all of them. It reads its address and key from environment variables, which is how this course points it at the practice bot.

The SDK's defaults are generous: it waits a long time for a reply and retries quietly. That is reasonable for a background job and wrong for a test harness or a support chat, where a slow reply is itself a failure. The client accepts its own timeout and retry count, and raises a dedicated timeout exception when time runs out.

About the assignment

Configure the client with a short timeout and a small number of retries, print a fixed JSON error line when the request still times out, and keep printing the reply and token count when it succeeds.

What you will do

  1. Run the script as given to see it work against a normal bot.
  2. Pass the timeout and retry settings when you create the client.
  3. Catch the SDK's timeout exception around the call and print the error line instead.

Common mistakes

  • Catching the wrong exception

    A rate-limit error and a timeout are different classes. Catch the timeout specifically, or a 429 will be reported as a timeout.

  • Changing the success output

    Other code reads the JSON line. Keep its keys exactly as they were.

In a real test suite

Timeouts and retries are part of a harness's contract with the team: they decide how long CI waits and how a flaky provider shows up in results. Settle them once in a shared client instead of in every test.

Related lessons

Next lessonpytest parametrize, fixtures and markers for evalsAll lessons in the courseAI Testing for QA Engineers