Deploying Python AI Apps

Your script works. You run python summarise.py, it calls the model, it prints the answer, and everyone who watched over your shoulder was impressed. Then someone asks the obvious question: can it just do that every morning without you? That question is the whole subject of this section. The distance between a script that works when you press enter and a service that keeps working while you sleep is not talent or scale — it is four specific pieces of engineering, none of which take long once you know what they are.

This is the main guide for that jump, written for people who can already write Python that calls an AI model but have never handed a program to a machine that is not theirs. It sits under Building AI-Powered Business Applications, and it assumes you have something working locally already — a summariser, a classifier, a report generator, anything that reads input, calls a model, and produces output. If you are still building that thing, the guides on AI Document Processing with Python are a good place to get a first version running before you come back here.

The four pieces are: make the script deployable, pick a shape, put it somewhere it runs unattended, and prove it is alive. Skipping any one of them produces a specific, predictable failure. Skip the first and your app dies the moment it lands on a machine where your home directory does not exist. Skip the second and you build a web server for a job that should have been a nightly task. Skip the third and you either leak a key or you hand-restart a process every few days. Skip the fourth and your app fails silently for three weeks before a customer tells you.

Prerequisites

You need Python 3.10 or newer, a working virtual environment, and a script that already calls an AI model successfully from your machine. If the virtual environment part is new, read Create a Python Virtual Environment for AI first — everything below assumes your project has one.

Check your version and create the environment:

python --version
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

Install the packages this guide uses. FastAPI is a Python web framework, and uvicorn is the program that actually serves a FastAPI app over HTTP:

pip install "openai>=1.40" "httpx>=0.27" "fastapi>=0.110" "uvicorn[standard]>=0.29" "python-dotenv>=1.0"
pip freeze > requirements.txt

That pip freeze line matters more than it looks. It writes every installed package with its exact version into requirements.txt, which is the file your host will read to rebuild your environment. Without it the host installs whatever the newest version happens to be on deployment day, and a library that changed its behaviour last Tuesday becomes your outage.

Create a .env file in the project root for local development only:

OPENAI_API_KEY=sk-your-key-here
OPENAI_MODEL=gpt-4o-mini
LOG_LEVEL=INFO
APP_ENV=development

Then, before you make a single commit, tell Git to ignore it:

echo ".env" >> .gitignore

A key in a public repository gets found by automated scanners within minutes and spent by someone else. The .env file is a convenience for your laptop; in production the same values arrive as real environment variables set by the host, and your code will not know the difference.

Step 1 — Make the script deployable

A deployable script is one that carries no assumptions about the machine it runs on. There are four assumptions almost every first draft contains, and each has a mechanical fix.

Assumption one: the API key is somewhere in the file. Even if you moved it to a variable at the top, it is still in the source. Read it from the environment instead, and fail loudly at startup if it is missing. A program that crashes in the first second with a clear message is far cheaper to debug than one that crashes twenty minutes into a batch with AuthenticationError.

Assumption two: paths are absolute. /Users/you/Documents/invoices does not exist on a server. Every input folder, output folder, and cache location should come from a setting with a sensible relative default.

Assumption three: the dependency list is whatever you happened to install. Pin it.

Assumption four: there is no entry point. If running your project means "open the file and run the third cell", a host cannot start it. There must be exactly one command that starts the thing.

Here is a config.py that handles the first two and gives you a single place to look when a setting is wrong. Put it next to your script:

import os
from pathlib import Path

from dotenv import load_dotenv

load_dotenv()  # reads .env on your laptop; a no-op on a host that sets real env vars


def require(name: str) -> str:
    """Return an environment variable or stop the program with a clear message."""
    value = os.environ.get(name, "").strip()
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value


OPENAI_API_KEY = require("OPENAI_API_KEY")
OPENAI_MODEL = os.environ.get("OPENAI_MODEL", "gpt-4o-mini")
REQUEST_TIMEOUT = float(os.environ.get("REQUEST_TIMEOUT", "30"))
MAX_RETRIES = int(os.environ.get("MAX_RETRIES", "2"))
LOG_LEVEL = os.environ.get("LOG_LEVEL", "INFO").upper()
APP_ENV = os.environ.get("APP_ENV", "development")
PORT = int(os.environ.get("PORT", "8000"))

DATA_DIR = Path(os.environ.get("DATA_DIR", "./data")).resolve()
DATA_DIR.mkdir(parents=True, exist_ok=True)

Now from config import OPENAI_MODEL, DATA_DIR anywhere in your project, and changing the model in production is an environment-variable change rather than a code change. The diagram below shows the same four assumptions on the left and their replacements on the right.

The entry point is the last piece. Give your project a main.py whose whole job is to be startable, and put the real work in a function that returns data rather than printing it:

import json
import sys

from config import OPENAI_MODEL
from work import summarise_text


def run_once(text: str) -> dict:
    """Do one unit of work and return a plain dictionary."""
    return {"model": OPENAI_MODEL, "summary": summarise_text(text)}


if __name__ == "__main__":
    incoming = sys.stdin.read()
    print(json.dumps(run_once(incoming), ensure_ascii=False))

Returning a dictionary instead of printing is what lets the same function be called by a scheduler today and an HTTP endpoint tomorrow without a rewrite. It is a two-minute habit that saves a day later.

Step 2 — Choose a shape: job, API, or worker

Almost every deployed AI app is one of three shapes, and picking the wrong one is the most expensive mistake in this whole process, because you find out weeks in.

A scheduled job is a program that a clock starts. It wakes up, does all its work, writes its results somewhere, and exits. Nobody is waiting for it. Nightly report generation, a weekly digest, a hourly inbox sweep, a daily re-classification of new documents — all jobs. A job needs no web server, no port, and no always-on process, which makes it the cheapest and by far the most reliable shape. If your work fits here, stop reading the other two paragraphs.

A web API is a program that waits for requests and answers them. Something outside — a form, a browser, another service, a webhook from a vendor — sends data and blocks until you reply. This shape is right when the caller genuinely needs the answer now and the work finishes in a few seconds. It costs more to run, because the process must stay up permanently, and it introduces timeouts, concurrency and health checks as things you now have to think about.

A worker queue is the hybrid. A small API accepts the request, writes a task into a queue, and immediately replies with an identifier. A separate worker process reads tasks from the queue and does the slow part. Use this shape when the work takes longer than a caller will wait — summarising a 200-page PDF, processing a whole folder of scanned invoices, generating fifty images. The tell is simple: if a single request takes more than about thirty seconds, an ordinary HTTP client will give up before you finish, so you must hand back a receipt rather than a result.

Two questions decide it. What starts a run — a clock, or someone outside? And if it is someone outside, do they need the answer in this same request? The decision tree below is the version of that question I would draw on a whiteboard.

Shapes are not a life sentence, and the upgrade path is short if you kept run_once separate from whatever calls it. A job becomes an API by importing the same function into a FastAPI route. An API becomes a worker queue by replacing the body of the route with "write a task row, return its id" and running a second process that reads those rows. In each case the AI code itself does not change; only the thing that decides when it runs does. That is the payoff for the discipline in Step 1.

If the tree points you at a scheduled job, Schedule Python AI Jobs with GitHub Actions walks through the cheapest credible way to run one, with no server to maintain. If it points at a web API, Turn a Python AI Script into an API with FastAPI takes the run_once function above and wraps it in a real endpoint. Start with the job shape whenever you can defend it; a great many "we need an API" projects turn out to be a nightly task with a spreadsheet on the end.

Step 3 — Run it unattended and keep secrets out of the code

Unattended means two things that are easy to conflate. First, the process starts again by itself after a crash or a machine restart. Second, nobody has to log in and type a password for it to work. Both are properties of where you run it, not of your code.

For a scheduled job, the runner is a service that owns a clock — a hosted continuous-integration runner, a cloud scheduler, or a plain cron entry on a small virtual machine. For a web API, the runner is a process supervisor that keeps uvicorn alive: a container platform, an application host, or systemd on a virtual machine you manage yourself. The rule of thumb for a first deployment is to choose the option where you write the least infrastructure. If you can push a Git repository and have the host build and run it from requirements.txt and a start command, that is nearly always the right first choice.

The start command for a FastAPI app is worth memorising, because it appears in every host's configuration box:

uvicorn app:app --host 0.0.0.0 --port ${PORT:-8000}

app:app means "the object named app inside the file app.py". --host 0.0.0.0 tells the server to accept connections from outside the container rather than only from itself, which is the single most common reason a deployed app looks dead from the internet while its logs claim it started fine. ${PORT:-8000} uses the port the host assigns, falling back to 8000 locally.

If you do containerise, keep secrets out of the image. An image is a shareable artefact; anything copied into it travels with it, including layers you thought you deleted:

FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .

CMD ["sh", "-c", "uvicorn app:app --host 0.0.0.0 --port ${PORT:-8000}"]

Read that Dockerfile for what is missing rather than what is present: no .env, no key, no credential of any kind. The image describes how to build and start the program, and nothing about who it is allowed to be.

Two operational details are worth getting right the first time. Your process will be stopped and started far more often than you expect — on every deploy, on every host maintenance window, and whenever it exceeds its memory allowance — so treat a restart as normal rather than exceptional. That means no work is held only in a variable in memory, and anything half-finished can be safely repeated. It also means startup must be cheap: load configuration, build the client, and be ready to serve. If your program spends forty seconds warming up before it can answer, the host's own health check will kill it before it finishes and you will loop forever.

Add a .dockerignore file containing .env, .venv and .git so a stray copy of your key cannot be pulled in by that COPY . . line. The pattern to internalise is that secrets are always injected at run time by whatever starts your process: the host's secrets manager, the platform's environment settings, or your CI system's encrypted variables. Your code never reads a file that contains a key, and your repository never holds one. Manage API Keys Safely in Production covers key rotation, per-environment keys, and what to do the hour after you realise one has leaked.

One more production habit while you are here: put a spending ceiling on the account behind the key before the app goes live. A deployed app that retries in a loop can spend a surprising amount overnight, and Set a Monthly AI API Spending Limit explains where that control lives.

Step 4 — Prove it is alive

A deployed app that you cannot observe is a rumour. You need three signals, and they answer three genuinely different questions.

Structured logs answer "what happened on that particular request?". Structured means each log line is a JSON object with named fields rather than a sentence, so you can filter for model or duration_ms instead of grepping prose. Every hosting platform can search JSON logs; almost none can usefully search print() output. Log one line per unit of work, and include the fields you will want at 2am: the request identifier, the model name, token counts, how long it took, and whether it succeeded. Save this as logging_setup.py:

import json
import logging
import sys

from config import LOG_LEVEL


class JsonFormatter(logging.Formatter):
    def format(self, record: logging.LogRecord) -> str:
        payload = {
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        }
        payload.update(getattr(record, "fields", {}))
        return json.dumps(payload, ensure_ascii=False)


def configure_logging() -> logging.Logger:
    handler = logging.StreamHandler(sys.stdout)
    handler.setFormatter(JsonFormatter())
    root = logging.getLogger()
    root.handlers = [handler]
    root.setLevel(LOG_LEVEL)
    return logging.getLogger("app")

Call it once at startup, then log with logger.info("ai_call", extra={"fields": {"model": "gpt-4o-mini", "duration_ms": 812}}). Write to standard output, not to a file — every modern host collects standard output for you, and files inside a container disappear when it restarts.

Be deliberate about what you leave out. Never log the API key, obviously, but also think twice before logging the full prompt or the full completion: prompts often contain a customer's document, an email thread, or a name, and a log store is usually far less protected than your database. Record a length and a hash if you need to prove two requests were identical, and record the text itself only when you have decided that is acceptable.

A health endpoint answers "is the process up right now?". It is a URL that returns HTTP 200 and a tiny JSON body, does no AI work, calls nothing external, and costs nothing to serve. Keep it dumb on purpose: if /health calls the model, then an outage at your model provider makes your own app look dead and your host may restart a perfectly healthy process.

An error-rate alert answers "is it getting worse?". Count failed AI calls against total AI calls over a rolling window and alert when the share crosses a threshold you picked in advance. A single failure is normal — networks are networks. Fifteen per cent failures for ten minutes is an incident.

The full treatment of what to record on each model call, how to redact prompts before they reach your log store, and how to turn those records into a cost report lives in Log and Monitor AI API Calls in Production.

Parameter reference

These are the environment variables an AI service needs before it can run anywhere but your laptop. Set them all, even the ones with defaults, so that a new environment is described in one place.

Two defaults deserve a comment. REQUEST_TIMEOUT at 30 seconds is deliberately shorter than most default HTTP timeouts, because a model call that has not answered in half a minute is usually not going to. MAX_RETRIES at 2 is a ceiling, not a target: every retry multiplies your token spend for that request, so retry only on timeouts and rate limits, never on a bad-request error that will fail identically the second time.

Troubleshooting

1. RuntimeError: Missing required environment variable: OPENAI_API_KEY — your code is correct and the host is not passing the variable. Check the spelling in the host's settings panel and confirm you saved it to the same environment you deployed to. Restart the process afterwards; most hosts only read environment variables at start.
2. The deploy succeeds, the logs say Uvicorn running, but the URL times out. You bound to localhost. Start with --host 0.0.0.0 so the server accepts connections from outside its own container, and make sure you are listening on the port the host gave you in PORT.
3. ModuleNotFoundError: No module named 'openai' in production only. Your requirements.txt is stale or was never committed. Run pip freeze > requirements.txt inside the activated virtual environment, commit it, and redeploy. If the host builds from a lock file instead, regenerate that.
4. openai.RateLimitError appears the moment real traffic arrives. Your account limit is lower than your concurrency. Add a short exponential backoff, cap how many requests you make at once, and read Fix the 429 Rate-Limit Error in Python for the retry pattern. For per-user caps, see Rate-Limit AI API Calls in a SaaS with Python.
5. httpx.ConnectTimeout or APIConnectionError under load, never locally. The host's network is slower or more restricted than yours. Raise REQUEST_TIMEOUT a little, retry twice, and check whether outbound traffic needs to leave through a proxy. Fix Connection and Timeout Errors with AI APIs has the diagnosis steps.
6. The health check passes but every real request fails. That is the health endpoint doing its job correctly and your monitoring being incomplete. Add the error-rate alert from Step 4 so a broken dependency is visible even while the process itself is fine.

Worked example

Here is app.py: the run_once idea from Step 1, the JSON logging from Step 4, and a health endpoint, in one file you can run right now. It is the smallest thing that is honestly deployable.

import time
import uuid

from fastapi import FastAPI, HTTPException
from openai import OpenAI, OpenAIError
from pydantic import BaseModel

from config import APP_ENV, MAX_RETRIES, OPENAI_API_KEY, OPENAI_MODEL, REQUEST_TIMEOUT
from logging_setup import configure_logging

logger = configure_logging()
client = OpenAI(api_key=OPENAI_API_KEY, timeout=REQUEST_TIMEOUT, max_retries=MAX_RETRIES)
app = FastAPI(title="summariser")


class SummariseRequest(BaseModel):
    text: str


@app.get("/health")                      # dumb on purpose: no model call, no database
def health() -> dict:
    return {"status": "ok", "env": APP_ENV, "model": OPENAI_MODEL}


@app.post("/summarise")
def summarise(body: SummariseRequest) -> dict:
    request_id = str(uuid.uuid4())       # ties every log line to one request
    started = time.perf_counter()
    try:
        response = client.chat.completions.create(
            model=OPENAI_MODEL,
            messages=[
                {"role": "system", "content": "Summarise the user's text in two sentences."},
                {"role": "user", "content": body.text},
            ],
        )
    except OpenAIError as exc:           # log the failure, then fail the request cleanly
        logger.error("ai_call_failed", extra={"fields": {
            "request_id": request_id, "error": type(exc).__name__}})
        raise HTTPException(status_code=502, detail="upstream model error") from exc

    elapsed_ms = round((time.perf_counter() - started) * 1000)
    logger.info("ai_call", extra={"fields": {
        "request_id": request_id,
        "model": OPENAI_MODEL,
        "duration_ms": elapsed_ms,
        "total_tokens": response.usage.total_tokens,
    }})
    return {"request_id": request_id, "summary": response.choices[0].message.content}

Run it locally with the start command from Step 3, then check the two things that matter:

uvicorn app:app --host 0.0.0.0 --port 8000
curl http://localhost:8000/health
curl -X POST http://localhost:8000/summarise \
  -H "Content-Type: application/json" \
  -d '{"text": "Paste a few paragraphs here to summarise."}'

The first curl should answer instantly with {"status":"ok",...}. The second should return a summary and a request_id, and your terminal should show a single JSON log line containing that same identifier, the model name, the duration and the token count. Those two behaviours — a health check that never touches the model and a log line you can correlate with a user's complaint — are what make the difference between a deployed app and a deployed app you can operate.

Notice what the failure path does. It logs the exception type, not the exception text, so a key or a customer's prompt cannot leak into your log store, and it answers HTTP 502 rather than letting a stack trace reach the caller. If you want the response to arrive progressively instead of all at once, Stream Chatbot Responses with Python shows the streaming variant of this endpoint.

Where to go next

Take the four steps in order and each one has a guide that goes deeper than this page can. Start with the shape you chose in Step 2: Turn a Python AI Script into an API with FastAPI if a caller is waiting for the answer, or Schedule Python AI Jobs with GitHub Actions if a clock decides when the work runs.

Then close the two gaps that turn a working deployment into a safe one. Manage API Keys Safely in Production covers per-environment keys, rotation, and the checklist for a leaked credential. Log and Monitor AI API Calls in Production turns the single log line above into something you can search, chart, and alert on — including the token and cost fields that make Managing AI API Costs and Tokens tractable once real traffic arrives.

Back to Building AI-Powered Business Applications.

Related guides

• Turn a Python AI Script into an API with FastAPI — wrap your existing function in a real HTTP endpoint.
• Schedule Python AI Jobs with GitHub Actions — run a recurring AI task without maintaining a server.
• Manage API Keys Safely in Production — inject secrets at run time and rotate them without downtime.
• Log and Monitor AI API Calls in Production — searchable logs, latency, and an alert that reaches you.
• AI Document Processing with Python — the kind of workload that most often needs the worker-queue shape.