Auto-Reply to Comments with Python and AI

By the end of this guide you will have a Python script that pulls new comments from a social account, sorts each one into a category, drafts a reply only for the categories you chose to answer, and writes those drafts to a review queue that a human approves before anything is posted. Budget about an hour to build it and another hour to tune the voice. Nothing goes live without your say-so, which is the whole point of the design.

This is one guide in Automated Social Media Posting with Python and AI, which covers the publishing side of the same pipeline. Replying is a harder problem than posting, because a reply reacts to something a stranger wrote and you cannot preview every input in advance.

The shape of a safe auto-reply pipeline

Most people who try this build one loop: fetch a comment, ask a model for a reply, post the reply. It works for two days and then answers a refund demand with a cheerful "Great question!" The fix is to split the work into stages that each do one job, and to put a gate between drafting and publishing.

Six stages, in order. You fetch new comments and normalise them into plain Python dictionaries. You classify each one, because a complaint and a compliment need different handling. You decide, per category, whether you answer at all. You draft a reply for the categories you kept. You push that draft into a queue a person reads. Only text a person marked approved ever reaches the platform. Drafts that get rejected are edited or dropped, and they never leak into the posting step.

Each stage is a small function you can test on its own. That matters more than it sounds: when a reply comes out wrong, you want to know whether the classifier mislabelled the comment or the drafting prompt was too loose, and separate functions let you check each answer independently.

Prerequisites

You need Python 3.10 or newer in a virtual environment (an isolated folder of packages, so this project's libraries do not collide with another's). If that phrase is new, work through Create a Python Virtual Environment for AI first, then come back.

You also need a developer app and access token for the network whose comments you want to read, with permission to read comments and to write replies. Every platform grants those separately, and read-only tokens are common by default.

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install "openai>=1.40" "httpx>=0.27" "python-dotenv>=1.0"
pip freeze > requirements.txt

Put every secret in a .env file beside your script:

OPENAI_API_KEY=sk-your-openai-key-here
PLATFORM_TOKEN=your-social-platform-access-token
COMMENTS_URL=https://api.example-platform.test/v1/comments
REPLY_URL=https://api.example-platform.test/v1/replies
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/your/webhook/path

Add .env to .gitignore before your first commit. A leaked social token lets a stranger post as your brand, which is worse than a leaked API key you can simply rotate:

echo ".env" >> .gitignore

The two URL values are placeholders. Replace them with the real endpoints from your platform's developer documentation, because no two networks agree on the path, the field names, or the pagination style. For a broader treatment of secret handling once this runs on a server, see Manage API Keys Safely in Production.

Step 1 — Pull new comments into a list of dicts

The first function has one job: talk to the platform and hand back a list of plain dictionaries with the same five keys every time. Keeping this boundary clean is what lets the rest of the script stay platform-agnostic. When you add a second network later, you write a second fetch function and change nothing else.

Two details matter. Track the newest comment id you have already seen and pass it back as since_id, or you will reprocess the same thread every run and pay for it twice. And set a timeout on the HTTP call, because a hanging request in a scheduled job is a job that never finishes.

import os
import httpx
from dotenv import load_dotenv

load_dotenv()  # reads .env; .env is in .gitignore

PLATFORM_TOKEN = os.getenv("PLATFORM_TOKEN")
COMMENTS_URL = os.getenv("COMMENTS_URL")


def fetch_new_comments(since_id: str | None = None) -> list[dict]:
    """Return new comments as dicts. Adapt the field names to your platform."""
    params: dict[str, str | int] = {"limit": 50}
    if since_id:
        params["since_id"] = since_id

    response = httpx.get(
        COMMENTS_URL,
        params=params,
        headers={"Authorization": f"Bearer {PLATFORM_TOKEN}"},
        timeout=30.0,
    )
    response.raise_for_status()
    payload = response.json()

    comments = []
    for item in payload.get("data", []):
        comments.append(
            {
                "id": str(item["id"]),
                "author": item.get("username", "unknown"),
                "text": (item.get("text") or "").strip(),
                "post_id": str(item.get("post_id", "")),
                "created_at": item.get("created_at", ""),
            }
        )
    return comments


if __name__ == "__main__":
    for comment in fetch_new_comments():
        print(comment["id"], comment["author"], comment["text"][:60])

Run it and read the output before you write another line. If the author names and comment text look right, the hard part of the platform integration is done. If payload.get("data", []) comes back empty, print payload itself and adjust the key — some APIs nest results under items, comments, or a results array inside a meta wrapper.

Step 2 — Classify every comment before you write anything

Now the interesting part. Ask the model what kind of comment this is, using a tiny prompt that returns structured data instead of prose. Four labels cover almost everything a brand account receives: question, praise, complaint, and spam. A question wants information. Praise wants at most a short acknowledgement. A complaint wants a person, not a bot. Spam wants hiding or reporting and never wants a reply, because replying to spam rewards it with reach.

Set temperature=0 so the same comment always gets the same label — classification wants consistency, not creativity. If the trade-off between the two is unfamiliar, What Is Temperature in an LLM API? explains it in a page. Ask for a confidence number too, and treat anything low as if it were a complaint: send it to a human rather than guessing.

import json
import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

LABELS = {"question", "praise", "complaint", "spam"}

CLASSIFY_SYSTEM = """You label public comments left on a brand's social posts.
Return JSON only, in this exact shape:
{"label": "question", "confidence": 0.0}

label must be exactly one of: question, praise, complaint, spam.
- question: the person is asking for information.
- praise: positive or friendly with nothing to answer.
- complaint: unhappy, disputing, or demanding something.
- spam: promotion, link-dropping, bot text, or abuse.

confidence is your certainty from 0 to 1. Never explain."""


def classify(text: str) -> dict:
    """Return {'label': str, 'confidence': float} for one comment."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        temperature=0,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": CLASSIFY_SYSTEM},
            {"role": "user", "content": text[:1000]},
        ],
    )
    result = json.loads(response.choices[0].message.content)
    label = str(result.get("label", "")).lower()
    confidence = float(result.get("confidence", 0))

    if label not in LABELS or confidence < 0.6:
        return {"label": "complaint", "confidence": confidence}  # unsure -> a human
    return {"label": label, "confidence": confidence}


if __name__ == "__main__":
    samples = [
        "Does this ship to Ireland?",
        "obsessed with this, wearing it every day",
        "Third email ignored. Where is my order?",
        "MAKE $$$ FAST >> link in bio",
    ]
    for sample in samples:
        print(classify(sample), sample)

Notice text[:1000]. Comments are short, but a hostile user can paste a wall of text specifically to blow through your token budget, and truncating removes that lever. The same instinct applies anywhere untrusted text enters a prompt; Count Tokens in Python Before You Send shows how to measure the cost before the call rather than after the invoice.

The label alone does not decide anything. You still choose the action, and writing that policy down as a picture makes the gaps obvious.

Two of the four paths never call the drafting model at all. That is not a limitation, it is the saving — you cut your generation spend roughly in half and remove the two categories most likely to embarrass you.

Step 3 — Draft a reply with your brand voice and hard rules

The drafting prompt does two things at once. It carries your voice, which is soft guidance the model interprets, and it carries hard rules, which are absolutes the model must not cross. Write the hard rules as short imperative sentences with no hedging, and give the model an explicit escape hatch: a single token it returns when a comment should not be answered at all. Checking for that token in Python is far more reliable than hoping the model refuses gracefully.

The rules below are a starting set. Adapt the voice, but keep every prohibition — each one exists because a brand somewhere learned it the expensive way. For a deeper treatment of shaping model output with instructions, read Write System Prompts that Control Output Format.

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

BRAND_SYSTEM = """You draft short public replies for Northwind Coffee's social accounts.

Voice: warm, plain, a little dry. Sound like one person, not a department.
Length: 40 words maximum, one paragraph, no hashtags, no emoji.

Hard rules you must never break:
1. Never promise a refund, discount, credit, replacement or delivery date.
2. Never state a price, a stock level or a shipping time.
3. Never argue, never correct the person publicly, never be sarcastic.
4. Never invent a fact about a product. If unsure, invite a direct message.
5. If the comment involves money owed, health, safety, legal action, or a
   child, reply with exactly this and nothing else: ESCALATE

Write only the reply text."""


def draft_reply(comment: dict, label: str) -> str | None:
    """Return draft reply text, or None when a human must handle it."""
    user_block = (
        f"Comment type: {label}\n"
        f"From: @{comment['author']}\n"
        f"Comment: {comment['text'][:1000]}"
    )
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        temperature=0.4,
        max_tokens=160,
        messages=[
            {"role": "system", "content": BRAND_SYSTEM},
            {"role": "user", "content": user_block},
        ],
    )
    draft = response.choices[0].message.content.strip()

    if "ESCALATE" in draft.upper() or len(draft) > 400:
        return None
    return draft


if __name__ == "__main__":
    sample = {"author": "mira_k", "text": "Is the dark roast still single-origin?"}
    print(draft_reply(sample, "question"))

The temperature=0.4 is deliberate: high enough that forty replies do not read like the same sentence with the nouns swapped, low enough that the model stays inside your rules. The length guard catches the case where the model ignores the word limit and writes an essay, which is a signal something in the prompt confused it.

One habit worth building now: keep the exact BRAND_SYSTEM text in your repository and change it in one place. When a reply goes wrong, the fix is almost always a new line in that prompt, and you want the history of those lines.

Step 4 — Route drafts to a review queue and post only approved text

A review queue can be as simple as a CSV file with a status column. Your script appends rows with status set to pending. You open the file, read each draft, change pending to approved or rejected, edit any wording you dislike, and save. A second function reads the file back and posts only the approved rows. That is a complete human gate in about thirty lines, and it works offline, on a plane, in a spreadsheet app your colleague already knows.

If you want the drafts to come to you rather than the other way round, send a Slack message per draft with an incoming webhook. The message is a notification, not the approval — the CSV stays the source of truth so there is exactly one place where the decision lives.

import csv
import os
import pathlib
import httpx
from dotenv import load_dotenv

load_dotenv()

QUEUE = pathlib.Path("review_queue.csv")
FIELDS = ["comment_id", "author", "comment", "label", "draft", "status"]
REPLY_URL = os.getenv("REPLY_URL")
PLATFORM_TOKEN = os.getenv("PLATFORM_TOKEN")
SLACK_WEBHOOK_URL = os.getenv("SLACK_WEBHOOK_URL")


def queue_draft(comment: dict, label: str, draft: str) -> None:
    """Append one pending draft to the review CSV and ping Slack."""
    is_new = not QUEUE.exists()
    with QUEUE.open("a", newline="", encoding="utf-8") as handle:
        writer = csv.DictWriter(handle, fieldnames=FIELDS)
        if is_new:
            writer.writeheader()
        writer.writerow(
            {
                "comment_id": comment["id"],
                "author": comment["author"],
                "comment": comment["text"],
                "label": label,
                "draft": draft,
                "status": "pending",
            }
        )

    if SLACK_WEBHOOK_URL:
        httpx.post(
            SLACK_WEBHOOK_URL,
            json={"text": f"*@{comment['author']}* ({label})\n> {comment['text']}\nDraft: {draft}"},
            timeout=15.0,
        )


def post_approved() -> int:
    """Post every approved row, mark it posted, and return how many went out."""
    if not QUEUE.exists():
        return 0

    rows = list(csv.DictReader(QUEUE.open(encoding="utf-8")))
    sent = 0
    for row in rows:
        if row["status"].strip().lower() != "approved":
            continue
        response = httpx.post(
            REPLY_URL,
            json={"comment_id": row["comment_id"], "text": row["draft"]},
            headers={"Authorization": f"Bearer {PLATFORM_TOKEN}"},
            timeout=30.0,
        )
        response.raise_for_status()
        row["status"] = "posted"
        sent += 1

    with QUEUE.open("w", newline="", encoding="utf-8") as handle:
        writer = csv.DictWriter(handle, fieldnames=FIELDS)
        writer.writeheader()
        writer.writerows(rows)
    return sent


if __name__ == "__main__":
    print(f"posted {post_approved()} approved replies")

Note that post_approved only ever looks at the status column. It has no access to the model, cannot generate text, and will refuse to send anything you did not explicitly mark. Keeping the posting function that dumb is the safety property of the whole design — there is no code path from the model straight to the platform.

Once this runs by hand, put it on a schedule. Schedule Python AI Jobs with GitHub Actions covers running the fetch-classify-draft half every hour, and you approve the queue whenever it suits you.

The risk of full automation, stated plainly

It is tempting to delete the queue and let replies fly. Before you do, be clear about what changes. An unreviewed reply is a public statement from your brand, written by a system that cannot see your inventory, your refund policy, or the fact that this particular customer has already been let down twice. Screenshots of AI replies going wrong travel further than any campaign you will run this quarter, and the apology costs more attention than the replies ever saved.

There is a second, quieter risk. Platform policies generally allow you to reply through the official API to comments on your own posts, and generally forbid mass identical replies, engagement farming, and bots that pretend to be people. Those policies change, and enforcement is automated too: a burst of near-identical replies at machine speed looks exactly like the pattern the abuse systems are tuned to catch. Read the current developer terms for each network before you scale up, keep your reply rate human-plausible, and let visible replies come from an account a real person stands behind.

Read the bottom row honestly. Full automation is not free effort — it is deferred effort, paid all at once on the day something goes wrong, plus whatever the cleanup costs.

Quick reference: label to action

Copy this table into your notes and change the last column to match your own policy. It is the only part of the system that encodes judgement rather than mechanics.

Troubleshooting

• json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) — the model wrapped its JSON in prose or a code fence, so json.loads choked on the first character. Add response_format={"type": "json_object"} to the call, as Step 2 does; Fix JSONDecodeError with AI API Responses in Python covers the stubborn cases.
• openai.RateLimitError: Error code: 429 — you looped over a backlog of comments with no pause and hit your account's request limit. Process in small batches with a short sleep between them; Fix the 429 Rate-Limit Error in Python shows a retry with backoff.
• httpx.HTTPStatusError: Client error '401 Unauthorized' for url ... — your platform token expired or lacks the reply permission. Regenerate it in the developer portal, confirm the write scope is granted, and update .env.
• KeyError: 'id' — the response shape does not match what fetch_new_comments expects. Print the raw payload once, find where the comment objects actually live, and adjust the two item.get lines rather than guessing at field names.

When to use this vs. alternatives

• Use this script when you get tens to low hundreds of comments a week, your replies need product knowledge a canned macro cannot supply, and you already have API access to the platform. The queue keeps quality high while the model removes the blank-page work.
• Use your platform's built-in tools when you only need keyword auto-replies or hidden-word filters. Most networks ship both, they cost nothing, and no code beats no code when the requirement is that simple.
• Use a human inbox with saved replies when volume is low or comments are emotionally loaded — early-stage brands, health, finance, anything where a wrong sentence is expensive. Add drafting later, once you can describe your own voice precisely enough to put it in a prompt.

Build the fetch and classify functions first and run them for a week without generating anything, just to see the mix of labels your account actually receives. That week of data will tell you which categories are worth automating and which never should be, and it costs almost nothing. Then add drafting, keep the queue, and only revisit the gate when you have hundreds of approvals behind you and a clear pattern of what the model gets right. Back to Automated Social Media Posting with Python and AI.

Related guides

• Bulk-Schedule Social Posts with Python — the publishing side of the same queue pattern.
• Generate Twitter Threads with Python and AI — turn one idea into a thread before the comments arrive.
• Classify Incoming Documents with Python and AI — the same label-first technique applied to files.
• Python Script to Automate Email Sorting — triage another inbox with the same classify-then-route idea.
• Log and Monitor AI API Calls in Production — keep a record of every draft your script produced.