Fix Connection and Timeout Errors with AI APIs

By the end of this guide your AI script will never hang silently again. You will set an explicit timeout on the client, understand the difference between waiting to reach the server and waiting for it to finish writing, add a bounded retry loop that backs off politely, stream long generations so the connection keeps producing tokens, and print a clear failure message when nothing works. Budget about twenty-five minutes, most of it spent watching your own retry loop print its attempts.

This guide belongs to Debugging Python AI Errors, the section that walks through the failures beginners hit most often when calling a hosted model from Python.

Prerequisites

You need Python 3.10 or newer, a working virtual environment, and the openai package already importable. If import openai raises an error, sort that out first with Fix ModuleNotFoundError: No Module Named openai, and if you have not made an isolated environment yet, Create a Python Virtual Environment for AI takes five minutes.

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install "openai>=1.40" "httpx>=0.27" "python-dotenv>=1.0"

Put your key in a .env file beside your script:

OPENAI_API_KEY=sk-your-key-here

Then add that file to .gitignore before you write another line, because a key in a public repository gets found and spent by strangers:

echo ".env" >> .gitignore

What the two errors actually mean

The openai library turns network trouble into two named exceptions, and they describe two genuinely different situations.

APIConnectionError means the conversation never happened. Your computer could not resolve the hostname, or a firewall dropped the packets, or a company proxy refused to forward them, or the encrypted handshake failed. No model ran and no tokens were billed. This is a plumbing problem, and the fix lives on your machine or your network, not in your prompt.

APITimeoutError means the opposite: you reached the server, it accepted the job, and then you walked away before it finished. The model may well have completed the answer a second after your script gave up. Nothing is broken — you simply asked for more work than your patience allowed. One important detail from the library's source: APITimeoutError is a subclass of APIConnectionError, so a single except APIConnectionError block catches both, and if you write separate blocks the timeout one must come first or it will never fire.

There is a third failure that beginners routinely file under "timeout" and should not: RateLimitError, the HTTP 429 response. That is the server answering you promptly and firmly, not failing to answer. The decision tree below shows which branch you are on and which move belongs to it.

Step 1 — Set an explicit timeout on the client

A timeout is a promise to yourself: after this many seconds of silence, treat the call as failed. Without one, a stalled socket can wait far longer than any human will, which is why so many beginner scripts appear to freeze. Set it once, on the client, and every request inherits it.

import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()  # reads .env; .env is already in .gitignore

client = OpenAI(
    api_key=os.getenv("OPENAI_API_KEY"),
    timeout=60.0,     # seconds, applied to every request this client makes
    max_retries=0,    # switch the SDK's own retries off while you experiment
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a 400-word product update."}],
)
print(response.choices[0].message.content)

Setting max_retries=0 for now is deliberate. The library retries some failures on your behalf, which is helpful in production but confusing while you are learning, because a single visible error may actually be the fourth attempt. Turn it off, see the raw behaviour, then turn it back on knowingly in Step 2.

One number covers two very different waits, though, and they deserve different values. The connect timeout covers finding the server and completing the encrypted handshake — work that either happens in a couple of seconds or is not going to happen. The read timeout covers waiting for the answer itself, which for a long generation can legitimately run to a minute or more. Give the first one a short leash and the second a long one:

import httpx
from openai import OpenAI

client = OpenAI(
    timeout=httpx.Timeout(90.0, connect=5.0),   # read 90s, connect 5s
    max_retries=0,
)

httpx is the HTTP library the openai package uses underneath, and httpx.Timeout lets you name each phase separately. The first positional value becomes the default for reading, writing and pooling; connect=5.0 overrides just the handshake. Now an unreachable network fails in five seconds instead of ninety, while a genuinely hard-working model still gets its minute and a half.

For a single unusually heavy call — a long document summary, say — you do not need a second client. Override the timeout for that one request:

long_response = client.with_options(timeout=240.0).chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarise this 40-page report..."}],
)

Step 2 — Add bounded retries with backoff and jitter

Retrying makes sense when the failure was random: a dropped packet, a brief blip, a server that was momentarily busy. It makes no sense to hammer a service that is struggling, and it makes no sense at all to keep going forever. So every retry loop needs three properties — a hard cap on attempts, a wait that grows between attempts (exponential backoff), and a small random extra (jitter) so that a thousand scripts failing at the same moment do not all return at the same moment.

import random
import time

from openai import APIConnectionError, APITimeoutError, OpenAI

client = OpenAI(timeout=60.0, max_retries=0)

MAX_ATTEMPTS = 4


def ask_with_retries(prompt: str) -> str:
    """Send one prompt, retrying transient network failures with backoff."""
    for attempt in range(1, MAX_ATTEMPTS + 1):
        try:
            response = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[{"role": "user", "content": prompt}],
            )
            return response.choices[0].message.content
        except (APITimeoutError, APIConnectionError) as err:
            if attempt == MAX_ATTEMPTS:
                raise RuntimeError(
                    f"Gave up after {MAX_ATTEMPTS} attempts: {type(err).__name__}"
                ) from err
            delay = min(2 ** (attempt - 1), 8) + random.uniform(0, 0.5)
            print(f"Attempt {attempt} failed ({type(err).__name__}); "
                  f"retrying in {delay:.1f}s")
            time.sleep(delay)
    raise RuntimeError("unreachable")


if __name__ == "__main__":
    print(ask_with_retries("Explain the Python with-statement in two sentences."))

Read the delay line closely, because it is where the thinking is. 2 ** (attempt - 1) gives 1, 2, 4, 8 seconds; min(..., 8) stops it growing past eight so a fourth failure does not leave you staring at the terminal; random.uniform(0, 0.5) adds the jitter. The raise ... from err on the last attempt keeps the original exception attached, so the traceback still shows what really happened. If tracebacks are new to you, Read a Python Traceback in Five Minutes makes them readable in one sitting.

Once you have written that loop by hand and watched it work, you can hand the job back to the library. The openai client has the same behaviour built in:

from openai import OpenAI

client = OpenAI(timeout=60.0, max_retries=3)   # SDK retries with its own backoff

With max_retries=3, the SDK quietly retries connection failures, timeouts and a handful of server-side statuses before raising anything to you. That is the right setting for a script you run unattended. Keep the hand-written loop for cases where you want to log each attempt, change the prompt between tries, or apply your own overall deadline — the last of which you will build in Step 4.

A word of caution on what retries cannot do: repeating a request that timed out because the answer was genuinely long will simply time out again, four times, at four times the token cost. Retries fix flaky networks. They do not fix impatience.

Step 3 — Stream the response so the connection keeps talking

Here is the insight that resolves most timeout complaints. In a normal call, your script sends a prompt and then receives absolutely nothing until the model has finished writing the entire answer. Ask for nine hundred words and the socket sits silent for the whole generation; the read timeout is racing against the total length of the reply.

Streaming changes the shape of that wait. With stream=True, the server sends each small piece of text the moment it is produced, so the connection is never idle for more than a fraction of a second. The read timeout now applies between chunks rather than to the whole answer, which means a five-minute generation can complete under a sixty-second read timeout without any trouble at all.

import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"), timeout=60.0, max_retries=2)

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a 900-word onboarding email sequence."}],
    stream=True,
)

pieces = []
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
        pieces.append(delta)

full_text = "".join(pieces)
print(f"\n\nDone: {len(full_text)} characters received.")

Each chunk carries a delta — the new text since the previous chunk — and that value is None on the opening and closing chunks, which is why the if delta: guard is there. Collecting the pieces into a list and joining once at the end is faster than repeatedly adding to a string, and it leaves you with the complete answer to save or post-process. The flush=True forces Python to print each fragment immediately rather than buffering it, so you actually see the text appear.

The side benefit is that your program becomes visibly alive. A reader watching words appear will wait a minute happily; the same reader staring at a blank terminal reaches for Ctrl-C after fifteen seconds. If you are wiring this into a chat interface, Stream Chatbot Responses with Python covers the presentation side.

Step 4 — Make the failure visible instead of silent

The worst outcome is not an error. It is a script that has been running for eleven minutes and you cannot tell whether it is working, waiting, or wedged. Give the whole operation a deadline, announce every retry, and end with a message a non-programmer could act on.

import random
import sys
import time

import httpx
from openai import APIConnectionError, APITimeoutError, OpenAI, RateLimitError

client = OpenAI(timeout=httpx.Timeout(60.0, connect=5.0), max_retries=0)

DEADLINE_SECONDS = 180


def generate(prompt: str) -> str:
    """Stream one completion, retrying until a hard overall deadline."""
    started = time.monotonic()
    attempt = 0
    while True:
        attempt += 1
        try:
            stream = client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[{"role": "user", "content": prompt}],
                stream=True,
            )
            return "".join(chunk.choices[0].delta.content or "" for chunk in stream)
        except RateLimitError:
            wait = 20.0        # 429 means slow down, not "wait harder"
        except (APITimeoutError, APIConnectionError):
            wait = min(2 ** (attempt - 1), 8) + random.uniform(0, 0.5)

        elapsed = time.monotonic() - started
        if elapsed + wait > DEADLINE_SECONDS:
            print(
                f"AI call failed after {attempt} attempts and {elapsed:.0f}s. "
                f"Check your internet connection, then run the script again.",
                file=sys.stderr,
            )
            sys.exit(1)
        print(f"Attempt {attempt} failed; waiting {wait:.1f}s", file=sys.stderr)
        time.sleep(wait)


if __name__ == "__main__":
    print(generate("List three ways to reduce customer churn."))

Three habits are worth copying from this. time.monotonic() is used instead of time.time() because it always moves forward, even if the machine's clock is adjusted mid-run. Progress messages go to sys.stderr, so you can redirect the actual answer to a file with python run.py > out.txt and still watch the retries on screen. And the deadline check adds the planned wait before sleeping, so the script never sleeps past its own budget. When you are ready to keep a permanent record of these events rather than just printing them, Log and Monitor AI API Calls in Production shows the next step.

Timeout or rate limit? Telling 429 apart

Confusing a 429 with a timeout leads people to raise their timeout, which changes nothing, or to retry immediately, which makes the rate limit worse. The two failures look similar from the outside — your script did not get its answer — but they come from opposite causes and take opposite fixes.

If your traceback says RateLimitError you are on the right-hand column, and Fix the 429 Rate-Limit Error in Python is the guide you want instead of this one.

Timeout settings quick reference

Troubleshooting

• openai.APIConnectionError: Connection error. — your machine never reached the API. Test the plumbing outside Python with curl -I https://api.openai.com/v1/models; if that hangs too, the problem is your network, VPN or proxy, not your code.
• openai.APITimeoutError: Request timed out. — the model was still writing when your clock expired. Raise the read timeout and add stream=True rather than retrying the identical call.
• httpx.ConnectTimeout — the handshake itself never completed, usually DNS or a blocked port. Raise connect slightly on a slow mobile connection, but treat a repeat as a network fault.
• ssl.SSLCertVerificationError: certificate verify failed — this surfaces wrapped inside APIConnectionError and is a certificate-store problem, not a timing one. Fix SSL: CERTIFICATE_VERIFY_FAILED in Python has the platform-specific cure.
• The script prints nothing and never exits — you have no timeout anywhere. Add timeout=60.0 to the client constructor and run it again; the hang becomes a normal, catchable exception.

When to use this vs. alternatives

• Raise the timeout and stream when the reply is legitimately long and the API is healthy. This is the common case, and it costs nothing extra.
• Retry with backoff when failures are intermittent and unrelated to the size of the answer — flaky wifi, a VPN reconnecting, an occasional server hiccup. Retrying a slow-but-successful generation just multiplies the bill.
• Shorten the work instead when even a generous timeout keeps expiring. Split the input into smaller requests, ask for a shorter answer, or move to a faster model such as gpt-4o-mini. If the request is enormous because the input is enormous, Fix the Context-Length-Exceeded Error in Python covers trimming it properly.
• Move the job off the critical path when a person is waiting on a web page. A request that needs four minutes belongs in a background task with a saved result, not in a page load.

Once your client has an explicit timeout, a capped retry loop, streaming on long jobs and an honest failure message, connection problems stop being mysterious. They become one of three things you can name in a second from the exception class alone, and each has a fix you have already written. If a failure ever surprises you again, start with the traceback and work down the branches of the first diagram. Back to Debugging Python AI Errors.

Related guides

• Read a Python Traceback in Five Minutes — identify which exception class you actually got before you fix anything.
• Fix the 429 Rate-Limit Error in Python — the right guide when the server answers with 429 instead of going quiet.
• Fix SSL: CERTIFICATE_VERIFY_FAILED in Python — for connection errors caused by certificate verification.
• Stream Chatbot Responses with Python — turn streaming chunks into a live typing effect for readers.
• Fix the 401 Unauthorized Error in OpenAI Python — when the connection works but the key is rejected.