Write System Prompts that Control Output Format

This guide shows you how to force an AI model to return data in a fixed, reliable shape, such as JSON with named fields, in under fifteen minutes. If you have ever asked a model for "a JSON object" and got back a chatty paragraph wrapped around it, this is the fix. You will combine three layers that reinforce each other: a strict system prompt, the API's JSON mode, and a pydantic check that catches anything the model gets wrong before it reaches the rest of your program.

The payoff is code you can trust. Once the output shape is guaranteed, you can feed the model's answers straight into a spreadsheet, a database, or another script without writing brittle text-cleaning hacks. Everything downstream gets easier, too: a validated object is what turns a job like Extract Invoice Data from PDFs with Python and AI into a routine batch run rather than a pile of string surgery. This is the difference between a one-off experiment and an automation you can leave running.

Prerequisites

You need Python 3.10 or newer and an OpenAI API key. If you have not set up Python yet, work through Setting Up Python for AI first, and if the API itself is new to you, Understanding LLM APIs covers keys and requests from scratch.

Install the two libraries this guide uses. The openai SDK talks to the API, and pydantic (a library that checks data against a shape you define) validates the results.

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install openai pydantic python-dotenv

Store your key in a .env file so it never lands in your code:

OPENAI_API_KEY=sk-your-key-here

Add .env to your .gitignore so your key is never committed to version control.

How the three layers fit together

Before the steps, it helps to see why one layer is not enough. The system prompt is your most authoritative instruction, but on its own the model can still drift. JSON mode guarantees the text parses as JSON, but not that it has the right keys. Pydantic validation confirms the keys and types are exactly what you expected, but only after the call returns. Each layer covers the previous one's blind spot.

Step 1: Pin the output shape in the system prompt

The system prompt is the single most important lever you have. Models give system-role instructions more weight than regular user messages, so this is where the format contract belongs. Be specific, list the exact keys, and include one short example of the output you want. An example does more work than a paragraph of description because the model can copy its structure directly.

import os
from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

SYSTEM_PROMPT = """You extract contact details from messy text.
Return ONLY a JSON object with exactly these keys:
  "name": string, the person's full name
  "email": string, their email address, or "" if none is present
  "company": string, their employer, or "" if unknown
Do not add commentary, markdown, or extra keys.
Example: {"name": "Ada Lovelace", "email": "ada@analytical.io", "company": "Analytical Engines"}"""

user_text = "Hi, I'm Grace Hopper from Cobol Corp. Reach me at grace@cobolcorp.com."

Notice the prompt names every key, gives a fallback for missing values (an empty string rather than a guess), and forbids extras. That last sentence matters: without it, models like to add a friendly "note" field you never asked for.

Read that block again and you will see it is really four separate jobs stacked into one string, each doing something the others cannot. Knowing which line does which job tells you where to aim when a reply comes back wrong, instead of rewriting the whole prompt and hoping.

The example line at the bottom does the heaviest lifting of the four. When one sample is not enough — a nested object, a date format the model keeps mangling, a field it insists on translating — showing two or three worked samples instead is the technique covered in Few-Shot Prompting in Python with Examples.

Step 2: Turn on JSON mode

JSON mode is the API setting response_format={"type": "json_object"}. With it on, the model is constrained to emit text that parses as valid JSON, so json.loads will not choke on a stray sentence. One rule comes with it: your prompt must mention JSON somewhere, which the system prompt above already does. Pair it with temperature=0 so the model picks the most predictable wording every time — if you are unsure what that dial actually controls, What Is Temperature in an LLM API? explains it in one sitting.

import json

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_text},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

raw = response.choices[0].message.content
data = json.loads(raw)
print(data)  # {'name': 'Grace Hopper', 'email': 'grace@cobolcorp.com', 'company': 'Cobol Corp'}

The switch is one line of code, but it changes what actually arrives on the wire. Comparing a typical reply with the setting off against the same request with it on makes clear which text-cleaning chores disappear the moment you turn it on.

JSON mode guarantees the text parses, but it does not guarantee the contents. The model could still return {"full_name": "Grace Hopper"} with the wrong key, or drop the company field entirely. That gap is exactly what the next step closes. If you do hit a parsing failure despite JSON mode, the dedicated walkthrough Fix JSONDecodeError with AI API Responses in Python covers the usual culprits.

Step 3: Validate the result with pydantic

Pydantic lets you declare the shape you expect as a small class, then check any parsed data against it in one line. If a field is missing, misnamed, or the wrong type, pydantic raises a ValidationError with a clear message instead of letting the broken data slip into the rest of your program.

from pydantic import BaseModel, EmailStr, ValidationError


class Contact(BaseModel):
    name: str
    email: str  # use EmailStr to also reject malformed addresses
    company: str


try:
    contact = Contact.model_validate(data)
    print(contact.name, contact.email)
except ValidationError as exc:
    print("The model returned an unexpected shape:")
    print(exc)

The Contact class is your single source of truth for the format. Add EmailStr (install with pip install "pydantic[email]") when you want the email checked too, or mark a field optional with a default like company: str = "". Because the class doubles as documentation, anyone reading your code can see the exact contract at a glance.

Step 4: Handle when the model strays

Even with all three layers, a model occasionally returns something that fails validation. The robust pattern is a short retry loop that feeds the validation error back to the model so it can correct itself, then gives up after a fixed number of attempts rather than looping forever.

def get_validated_contact(text: str, max_attempts: int = 3) -> Contact:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": text},
    ]
    for attempt in range(max_attempts):
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            response_format={"type": "json_object"},
            temperature=0,
        )
        raw = response.choices[0].message.content
        try:
            return Contact.model_validate_json(raw)
        except ValidationError as exc:
            messages.append({"role": "assistant", "content": raw})
            messages.append({
                "role": "user",
                "content": f"That failed validation:\n{exc}\nReturn corrected JSON only.",
            })
    raise RuntimeError(f"No valid output after {max_attempts} attempts")

Contact.model_validate_json(raw) parses and validates in a single call, so you skip the separate json.loads. Appending the assistant's bad reply and the error gives the model the context it needs to fix its own mistake, which usually succeeds on the second try.

When retries keep failing on the same field, the prompt is usually asking for too much at once. Splitting the work across two calls — one that does the thinking in plain language, a second that only reformats that answer into your keys — is steadier than one heroic prompt, and Chain Prompts Together in Python shows how to wire the two calls together.

Parameter quick-reference

Troubleshooting

1. json.loads raises but JSON mode is on. You almost certainly forgot to pass response_format={"type": "json_object"} on this specific call, or the request hit an error before the format applied. Confirm the parameter is present and that the response is not an error object.
2. BadRequestError: 'messages' must contain the word 'json'. JSON mode requires the word "json" somewhere in your messages. Add it to the system prompt, as the examples here do.
3. Pydantic ValidationError on a key the model renamed. The model used full_name instead of name. Tighten the system prompt with the exact key spelling and an example, and keep the retry loop from Step 4, which feeds the error back for a correction.
4. The model wraps JSON in ```json fences. This happens when JSON mode is off. Turn JSON mode on, which strips the fences, rather than peeling them with string slicing.

When to use this vs. alternatives

The choice between these three routes is settled by one question: who or what reads the reply. Answer that first and the rest follows, because a human reader tolerates prose that would break a parser, and a parser needs a contract that a human would find tediously rigid.

• System prompt alone, no JSON mode: Fine for quick experiments or when your downstream code only reads the prose. As soon as another program parses the output, add JSON mode so you are not cleaning text by hand.
• JSON mode plus pydantic (this guide): The right default when you need one fixed object back and want a guarantee that the keys and types are correct. Simple, fast, and easy to reason about.
• Function or tool calling: Reach for this when the model must choose between several shaped actions or trigger real code, not just return one object. It carries more setup, so prefer JSON mode when a single schema covers your need.

Once you are comfortable enforcing a shape, the next move is reusing these patterns across real campaigns with Prompt Engineering Templates for Marketers, which applies the same JSON-and-validation discipline to ad copy and email sequences.

Back to Prompt Engineering Basics.

Related guides

• Prompt Engineering Basics — the main guide on instructing models reliably.
• Few-Shot Prompting in Python with Examples — when one example in the system prompt is not enough.
• Prompt Engineering Templates for Marketers — ready-to-run templates that use these same format controls.
• Fix JSONDecodeError with AI API Responses in Python — what to do when parsing still fails.
• Understanding LLM APIs — keys, requests, and responses from the ground up.