Managing AI API Costs and Tokens

An AI script does not send you a warning before it gets expensive. It runs a loop over a spreadsheet, retries a few failures, and by the time you check the dashboard the number has moved. Nothing dramatic happened — no bug, no abuse, just a few thousand calls you never counted. This section shows you how to make every call visible, predictable and capped, using nothing more than the Python you already run and a CSV file.

The plan has four moves, in order: measure what you are already spending, estimate what a call will cost before you send it, reduce the cost of each call, and enforce a limit your script physically cannot cross. Each move stands on its own, so you can stop after the first one and still be far better off than you are now. Together they take about an hour to put in place and then look after themselves.

This is one section of Python AI Fundamentals for Non-Developers, written for creators, marketers, founders and students who can run a Python file but have never had a production bill land on their card.

What you are actually paying for

Providers do not bill per request, per minute or per word. They bill per token — a chunk of text the model works in, usually about three-quarters of an English word, sometimes a single punctuation mark, sometimes a whole short word. "Unbelievable" might be three tokens; "cat" is one. You cannot see tokens in your text, which is exactly why costs feel unpredictable until you start counting them.

Two things surprise people the first time they look closely. First, input and output are priced separately, and output is normally the more expensive of the two. The long document you paste in is charged at the cheaper rate; the summary the model writes back is charged at the dearer one. Second, the whole input is re-sent and re-charged on every call. A chat script that keeps the conversation history is paying for the entire history again with each new message, so a twenty-turn conversation costs far more at turn twenty than it did at turn two. If you have watched a chatbot get slower and pricier as it runs, that is why.

A third detail matters for repeated work: many providers now bill cached input at a reduced rate. When you send the same long prefix — a fixed system instruction, a reference document — several times in a short window, the provider may recognise it and charge less for that portion. You do not control this directly, but you influence it by keeping the reusable part of your prompt at the front and unchanged. Whether it applies to your account and which rate it uses is on the provider's pricing page; never assume a number you read in a tutorial.

That is the entire billing model. Input tokens at one rate, output tokens at a higher rate, the whole input charged every time, and possibly a discount on repeated prefixes. Everything else in this section is arithmetic on top of those four facts.

Prerequisites

You need Python 3.10 or newer and a project folder with a virtual environment. If that phrase is new, Create a Python Virtual Environment for AI walks through it, and the wider Setting Up Python for AI section covers the install itself. Check your version:

python --version

Then create the environment and install the four packages this section uses. tiktoken is the tokeniser OpenAI publishes, which lets you count tokens on your own machine without sending anything:

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install "openai>=1.40" "tiktoken>=0.7" "httpx>=0.27" "python-dotenv>=1.0"
pip freeze > requirements.txt

Put your key in a .env file in the project folder:

OPENAI_API_KEY=sk-your-key-here

Add that file to .gitignore before you write another line:

echo ".env" >> .gitignore

This is a cost-control step as much as a security one. A key that leaks into a public repository gets scraped and used within minutes, and the calls it makes are billed to you. Every budget you build in this section is defeated by a key someone else is holding.

One habit to adopt now: keep a scratch file called cost_log.csv in the same folder and add it to .gitignore too. It will hold every call you make from here on, and you do not want a public commit history of your usage.

Step 1 — Measure: log every call to a CSV

You cannot manage a number you have never seen. Fortunately the provider hands it to you on every response: an object called usage carrying the input token count, the output token count, and their total. The habit to build is simple — never discard it. Write it to a file, always, even during experiments.

Here is the smallest useful version. It sends one request, reads the usage, and appends a row to cost_log.csv, creating the file with a header the first time:

import csv
import os
from datetime import datetime
from pathlib import Path

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()  # reads .env, which is listed in .gitignore
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

LOG = Path("cost_log.csv")

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Name three uses for a CSV file."}],
)

usage = response.usage
print(response.choices[0].message.content)
print(f"in={usage.prompt_tokens} out={usage.completion_tokens}")

new_file = not LOG.exists()
with LOG.open("a", newline="", encoding="utf-8") as handle:
    writer = csv.writer(handle)
    if new_file:
        writer.writerow(["timestamp", "model", "input_tokens", "output_tokens"])
    writer.writerow([
        datetime.now().isoformat(timespec="seconds"),
        "gpt-4o-mini",
        usage.prompt_tokens,
        usage.completion_tokens,
    ])

Run that a handful of times and open the CSV in any spreadsheet. Two columns of numbers per call is enough to answer questions you could not answer before: which prompt is the expensive one, whether output or input dominates, and how the totals grow when you loop over a file. The names differ by provider — OpenAI returns prompt_tokens and completion_tokens, Anthropic returns input_tokens and output_tokens — but the idea is identical, and Understanding LLM APIs covers the response shapes side by side.

The measuring loop is worth picturing as a circuit, because that is what it becomes once you add pricing and a budget check in the later steps.

Once the log exists, the guide on Count Tokens in Python Before You Send shows how to reconcile what you predicted against what you were actually charged, which is the fastest way to build a feel for how your prompts tokenise.

Step 2 — Estimate: price a call before you send it

Logging tells you what a call cost after the fact. Estimating tells you before, which is what you need when the script is about to loop over eight hundred rows. Two numbers give you an estimate: the token count of your prompt and the per-token rate.

The token count you can get locally, for free, in a few milliseconds. tiktoken implements the same tokeniser the model uses, so counting a string on your laptop gives you the number the provider will bill you for:

import tiktoken

encoding = tiktoken.encoding_for_model("gpt-4o-mini")

system_prompt = "You are a careful assistant. Answer in one short paragraph."
user_prompt = "Summarise the attached product review in one sentence."

input_tokens = len(encoding.encode(system_prompt)) + len(encoding.encode(user_prompt))
print(f"About {input_tokens} input tokens before the model replies.")

The rate you must read yourself. Prices change, they differ per model, and input and output have separate figures — so open your provider's pricing page, find the model you are using, and copy today's two numbers into your script. Do not take a number from any article, including this one:

import tiktoken

INPUT_RATE_PER_M = 0.0    # dollars per 1,000,000 input tokens — from the pricing page
OUTPUT_RATE_PER_M = 0.0   # dollars per 1,000,000 output tokens — from the pricing page

EXPECTED_OUTPUT_TOKENS = 120   # your max_tokens cap, or a realistic reply length
ROWS = 800                     # how many times the loop will run

encoding = tiktoken.encoding_for_model("gpt-4o-mini")
prompt = "You are a careful assistant. Summarise this review in one sentence."
input_tokens = len(encoding.encode(prompt))

per_call = (
    input_tokens * INPUT_RATE_PER_M
    + EXPECTED_OUTPUT_TOKENS * OUTPUT_RATE_PER_M
) / 1_000_000

print(f"Per call:  ${per_call:.6f}")
print(f"Whole job: ${per_call * ROWS:.2f}")

Until you paste real rates in, this prints zeros — that is deliberate, so a placeholder can never masquerade as a real forecast. With the rates filled in you get a figure to react to before any money moves. If the whole-job number makes you wince, that is the moment to change the plan: fewer rows, a smaller model, a tighter prompt.

Estimating output is the harder half, because you do not know how long the reply will be until it arrives. Two practical approaches: set max_tokens and use that as your worst case, which turns the estimate into an upper bound; or run the job on ten rows, look at the average completion_tokens in your log, and multiply. The second is more accurate, the first is safer. Estimate OpenAI API Costs with Python builds this into a reusable estimator that reads a rate table and prints a per-job forecast.

There is a second payoff to counting locally. The same number tells you whether a request will fit in the model's context window, the maximum tokens it can handle in one go. Catching an oversized prompt before you send it is far cheaper than a failed call, and if you have already hit that wall, Fix the Context-Length-Exceeded Error in Python explains the error and Split Long Text into Chunks for AI APIs shows how to break the input down.

Step 3 — Reduce: four levers that actually move the bill

With measurement and estimation in place you can tell which changes help instead of guessing. Four levers do nearly all the work, and they attack different parts of the bill.

Trim the system prompt. The system prompt is the standing instruction sent at the top of every single request, so every wasted word is paid for on every call. A 400-token system prompt across 5,000 calls is two million input tokens spent on wording you wrote once. Cut it to the rules the model actually needs and delete the polite preamble; models do not need warming up. Write System Prompts that Control Output Format shows how to get reliable structure out of a short instruction rather than a long one.

Cap max_tokens. This sets the maximum number of tokens the model may write back. Because output is the pricier side, a cap is the most direct ceiling you have on a single call's cost. Set it to a little more than the longest legitimate answer you expect. If replies come back cut off mid-sentence, the cap is too low, not broken.

Default to the small model. Small models such as gpt-4o-mini or claude-haiku-4-5 cost a fraction of their larger siblings per token and handle classification, extraction, tidying and short summaries perfectly well. Reach for gpt-4o or claude-sonnet-4-5 only when the small model's answer is genuinely not good enough — and let your code make that decision rather than your habits.

import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

SYSTEM = "Classify the review as positive, negative or mixed. Reply with one word."


def classify(review: str) -> str:
    """Try the cheap model first; escalate only if the answer is unusable."""
    small = client.chat.completions.create(
        model="gpt-4o-mini",
        max_tokens=5,                     # one word is all we want back
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": review},
        ],
    )
    answer = small.choices[0].message.content.strip().lower()
    if answer in {"positive", "negative", "mixed"}:
        return answer

    big = client.chat.completions.create(   # escalation, not the default path
        model="gpt-4o",
        max_tokens=5,
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": review},
        ],
    )
    return big.choices[0].message.content.strip().lower()


print(classify("Arrived late but the quality is genuinely excellent."))

Cache repeats. If your script can ever send the same prompt twice, storing the reply turns the second call into a dictionary lookup that costs nothing and returns instantly. Test loops, retried batch jobs and anything triggered by users hitting refresh are full of exact repeats. A cache is a handful of lines — hash the prompt, use it as a filename, read the file if it exists — and Cache AI Responses in Python to Cut Costs builds a durable version that survives restarts.

Deciding which lever applies is mechanical enough to draw as a flowchart. Walk a request down it and you land on the cheapest option that still does the job.

Notice what is not on the list. Retries are a hidden cost multiplier — a failed call you retry three times is charged four times if the failures happened after the model started writing, so build backoff rather than blind loops, as covered in Fix the 429 Rate-Limit Error in Python. And streaming does not change the price at all; it changes when the text arrives, not how much of it you pay for.

Step 4 — Enforce: a limit your script cannot cross

Measurement, estimation and reduction all depend on you paying attention. Enforcement does not, which is why it is the step that actually protects you. Use two independent limits.

The first lives in your code. Before every call, total this month's rows in cost_log.csv; if the total is at or above your budget, raise an exception instead of calling the API. Because the check reads a file on disk, it holds across separate runs of the script — a loop that crashed and restarted cannot reset its own budget.

import csv
from datetime import date
from pathlib import Path

LOG = Path("cost_log.csv")
MONTHLY_BUDGET_USD = 5.00


class BudgetExceeded(RuntimeError):
    """Raised instead of spending money the budget does not have."""


def spent_this_month() -> float:
    if not LOG.exists():
        return 0.0
    prefix = date.today().strftime("%Y-%m")     # e.g. "2026-07"
    total = 0.0
    with LOG.open(newline="", encoding="utf-8") as handle:
        for row in csv.DictReader(handle):
            if row["timestamp"].startswith(prefix):
                total += float(row["cost_usd"])
    return total


def assert_budget_remains() -> None:
    spent = spent_this_month()
    if spent >= MONTHLY_BUDGET_USD:
        raise BudgetExceeded(f"${spent:.2f} of ${MONTHLY_BUDGET_USD:.2f} already used")
    print(f"${MONTHLY_BUDGET_USD - spent:.2f} of budget left this month.")


assert_budget_remains()

The second lives in the provider's dashboard. Every major provider lets you set a hard usage limit and a warning threshold on the account itself. Set both. Your Python check can be bypassed by a script that forgets to import it, or by a key you pasted into a notebook months ago; the account limit cannot. Treat the dashboard cap as the fuse and your code check as the switch you actually operate. Set a Monthly AI API Spending Limit walks through wiring both together, including what to do when the budget runs out mid-job.

The four levers do not all pull on the same part of the bill, and the table below is the mental model worth keeping: some cut the input side, some cut the output side, and only caching removes the call entirely.

Where the money usually goes

When a beginner's bill is larger than expected, it is nearly always one of five things, and none of them is "the model is expensive".

Conversation history. A chat loop that appends every turn to messages resends the whole transcript each time. Turn 30 pays for turns 1 to 29 again. Fix it by keeping only the last few turns, or by summarising older turns into one short paragraph and dropping the originals.

Retries on failure. A rate-limit error, a timeout or a connection reset inside a for loop that retries three times turns one job into four. Some of those attempts are billed, because the model had already started producing tokens. Retry with a growing wait between attempts, cap the number of attempts, and log every attempt rather than only the successful one.

Embeddings on the whole dataset, twice. Embeddings — numeric representations of text used for search and similarity — are cheap per item and expensive in bulk if you regenerate them every run. Save them to disk after the first pass and reuse them.

Debug loops. Running the same script forty times while you fix a formatting bug costs forty times the tokens. Point the script at three rows while you debug, and cache responses so the runs after the first are free.

Images and audio. These are not billed per token at all — image generation is usually priced per image and per size, transcription per minute of audio. If your project mixes them in, your token log will look reassuring while the actual invoice does not. Track those calls in the same CSV with a units column so nothing sits outside your ledger.

The pattern behind all five is the same: cost scales with how many times something runs, not with how clever the prompt is. When you review your log at the end of a week, sort by count before you sort by size. The row you call two thousand times matters more than the one you call twice, even if the second one is ten times bigger.

One more habit worth building: reconcile your CSV against the provider dashboard once a week, early on. If the two numbers agree, your rate table and logging are correct and you can trust the budget check. If they diverge, you have found either a stale price, a call path that bypasses your wrapper, or a category of usage you are not logging — all three are worth knowing about before they compound.

Parameter reference

These are the settings that change what a call costs, plus one famous setting that does not.

Temperature deserves its own sentence because it is so often mistaken for a cost dial: it controls randomness, and randomness is free. What Is Temperature in an LLM API? explains what it does change.

Troubleshooting

1. AttributeError: 'NoneType' object has no attribute 'prompt_tokens' — you read response.usage on a streamed response. Streaming responses omit usage by default. Request it explicitly with the SDK's stream-options setting, or count tokens locally instead.
2. KeyError: 'gpt-4o' when pricing a call — your rate table has no entry for the model you used. Add the model with today's input and output prices, and make the lookup raise a clear message rather than a bare KeyError so the gap is obvious.
3. Your CSV total is far below the dashboard figure — you are logging only successful calls. Failed calls that got as far as generating text are still billed, and so is every retry. Log inside a finally block, or log the attempt before you inspect the result.
4. ValueError: could not convert string to float: '' — a row in cost_log.csv has an empty cost, usually written by an older version of the script before you added the column. Delete the stale rows or default missing values to zero when you read them.
5. The budget check never triggers — your timestamp format does not match the YYYY-MM prefix you compare against, so no rows count as "this month". Print spent_this_month() once by hand to confirm it returns a non-zero number.
6. Costs look right per call but the monthly total keeps climbing — something is calling the API outside your wrapper. Grep the project for client.chat.completions.create and make sure every occurrence goes through the tracker. The wider Debugging Python AI Errors section helps when the culprit is a stack trace rather than a stray call.

Worked example: a CostTracker that refuses to overspend

This ties all four steps into one file. CostTracker wraps the client, checks the budget before each call, prices the response from your rate table, appends a row to the CSV, and raises BudgetExceeded once the month is spent. Save it as cost_tracker.py and import it anywhere you would otherwise call the API directly.

"""cost_tracker.py — one wrapper that measures, prices, logs and caps every call."""
import csv
import os
from datetime import date, datetime
from pathlib import Path

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()  # .env holds OPENAI_API_KEY and is listed in .gitignore

RATES = {  # dollars per 1,000,000 tokens — copy today's numbers from the pricing page
    "gpt-4o-mini": {"input": 0.0, "output": 0.0},
    "gpt-4o": {"input": 0.0, "output": 0.0},
}


class BudgetExceeded(RuntimeError):
    """Raised instead of calling the API when the month's budget is gone."""


class CostTracker:
    def __init__(self, budget_usd: float = 5.00, log_path: str = "cost_log.csv"):
        self.client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
        self.budget = budget_usd
        self.log = Path(log_path)
        if not self.log.exists():                      # write the header once
            with self.log.open("w", newline="", encoding="utf-8") as handle:
                csv.writer(handle).writerow(
                    ["timestamp", "model", "input_tokens", "output_tokens", "cost_usd"]
                )

    def spent_this_month(self) -> float:
        prefix = date.today().strftime("%Y-%m")
        with self.log.open(newline="", encoding="utf-8") as handle:
            return sum(
                float(row["cost_usd"] or 0)
                for row in csv.DictReader(handle)
                if row["timestamp"].startswith(prefix)
            )

    def ask(self, prompt: str, model: str = "gpt-4o-mini", max_tokens: int = 300) -> str:
        spent = self.spent_this_month()
        if spent >= self.budget:                        # the enforcement step
            raise BudgetExceeded(f"${spent:.2f} of ${self.budget:.2f} used this month")

        response = self.client.chat.completions.create(
            model=model,
            max_tokens=max_tokens,                      # the output ceiling
            messages=[{"role": "user", "content": prompt}],
        )
        usage = response.usage
        rate = RATES[model]
        cost = (usage.prompt_tokens * rate["input"]
                + usage.completion_tokens * rate["output"]) / 1_000_000

        with self.log.open("a", newline="", encoding="utf-8") as handle:  # the measure step
            csv.writer(handle).writerow([
                datetime.now().isoformat(timespec="seconds"), model,
                usage.prompt_tokens, usage.completion_tokens, f"{cost:.8f}",
            ])
        return response.choices[0].message.content


if __name__ == "__main__":
    tracker = CostTracker(budget_usd=5.00)
    print(tracker.ask("Give me three uses for a CSV file, as a list."))
    print(f"Spent this month: ${tracker.spent_this_month():.4f}")

Run it with python cost_tracker.py. You get an answer, a new row in cost_log.csv, and a running total — and until you paste real rates into RATES, the cost column stays at zero, which is your reminder that the pricing page is the only source of truth. Once the rates are in, set budget_usd to a number you would genuinely be relaxed about losing, then swap every direct API call in your project for tracker.ask(...). When you later move a script to a server, the same log becomes your monitoring feed; Log and Monitor AI API Calls in Production picks up from there.

Where to go next

Take the four steps one at a time. Start with Count Tokens in Python Before You Send to get comfortable with what a token is and how your own prompts measure up, then use Estimate OpenAI API Costs with Python to turn those counts into a per-job forecast you can check before a big run.

When the numbers are visible, go after them. Cache AI Responses in Python to Cut Costs is the highest-leverage change for anything that repeats a prompt, and Set a Monthly AI API Spending Limit closes the loop with a cap in your code and a matching one in the provider dashboard. If a call is failing rather than merely costing, the Debugging Python AI Errors section is the faster route.

Back to Python AI Fundamentals for Non-Developers.

Related guides

• Count Tokens in Python Before You Send — measure a prompt locally with tiktoken before it costs anything.
• Estimate OpenAI API Costs with Python — forecast a whole batch job from a rate table and a sample.
• Cache AI Responses in Python to Cut Costs — make a repeated prompt free instead of paying twice.
• Set a Monthly AI API Spending Limit — a cap in your script plus a cap in the dashboard.
• Understanding LLM APIs — how the request, the response and the usage object fit together.