Requests, status codes and headers
By the end: You can read an API's documentation and call it correctly the first time.
- Methods and what each implies
- Status codes worth knowing by heart
- Headers that matter
- Reading an error response properly
New to http & apis? When your program uses a service on another machine, it sends an HTTP request across the network and waits for a response. The request says what you want and carries your data; the response carries the result and a status code, a three-digit number reporting whether it worked. Calling a language model is exactly this and nothing more exotic. Why it matters
Model providers are ordinary web APIs, so most of what goes wrong with them is ordinary HTTP going wrong. This section is the vocabulary you need to read a provider's documentation and get a call right first time: what actually goes in a request, what the status code coming back is telling you, and which headers carry the parts that matter. It is also how you work out from a failure whether the fault is your credentials, your payload or their servers.
The shape of a request#
A request is a method, a URL, some headers and usually a body. A response is a status code, some headers and a body. Everything else is detail.
import httpx
r = httpx.post(
"https://api.example.com/v1/messages",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
json={"model": "small", "messages": [...]},
timeout=60,
)
r.status_code # 200
r.json() # parsed body
r.headers # rate limit info lives hereMethods, and what each one promises#
The method is a promise to the caller about what happens if the request is repeated, and that promise is what every proxy, retry layer and browser in the path relies on. Safe means the call does not change anything. Idempotent means sending it twice leaves the world in the same state as sending it once, which is not the same as getting the same response back.
method safe idempotent body typical use
GET yes yes no fetch a thing. Never use it to change one.
HEAD yes yes no the headers only: size, type, existence
POST no NO yes create, or "do this". Retrying may duplicate.
PUT no yes yes replace the whole resource at this URL
PATCH no no yes partial update. Idempotent only if you design it so.
DELETE no yes no remove it. Second call returns 404, state is the same.
OPTIONS yes yes no what this endpoint allows. CORS preflight uses it.Two consequences follow and both bite in production. A GET that changes state will eventually be repeated by something you do not control, such as a link prefetcher or a retrying proxy. And because POST is not idempotent, a retry after a timeout can charge you twice for the same generation. The fix is an idempotency key: a value you generate per logical operation and send in a header, which the server uses to recognise a repeat and return the original result instead of doing the work again.
Status codes to know cold#
code meaning whose fault what to do
200 fine - use the body
201 created something - read the Location header
204 done, no body - do not call .json() on it
206 partial content - you sent a Range header
301 moved permanently - update your URL, it is not coming back
304 not modified - your cached copy is still good
400 malformed request yours read the body, it names the field
401 not authenticated yours the key is missing, wrong or expired
403 authenticated, denied yours the key is real and lacks the permission
404 no such thing probably a wrong path far more often than missing data
405 wrong method yours you sent POST to a GET endpoint
409 conflict yours something changed under you; re-read and retry
413 payload too large yours your prompt or file exceeded their limit
415 unsupported media type yours you forgot Content-Type: application/json
422 understood, invalid yours valid JSON, impossible request
429 rate limited shared back off, read Retry-After
500 server error theirs retry with backoff
502 bad gateway theirs transient, retry
503 unavailable theirs overloaded or deploying, retry
504 gateway timeout theirs their upstream was slow. May have run anyway.4xx and 5xx need different handling
A 4xx means retrying the identical request will fail identically, so retrying is wasted time and money. A 5xx or a timeout is often transient and worth retrying. Code that retries everything is as wrong as code that retries nothing. The one exception is 429, which is a 4xx that you must retry, because the thing that needs to change is the time rather than the request.
A provider that misbehaves on demand#
Reading about status codes is not the same as having handled them, and you cannot make somebody else's API fail on request. Run one of your own instead. The server below is about forty lines of standard library, returns each of the failures you care about on demand, and gives you something to point a client at while you are still writing it.
import json, threading, urllib.error, urllib.request
from http.server import BaseHTTPRequestHandler, HTTPServer
class FakeProvider(BaseHTTPRequestHandler):
"""Behaves like a model provider on a bad day, so the client can be tested."""
def _send(self, status, payload, extra=None):
body = json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("request-id", "req_0001")
for k, v in (extra or {}).items():
self.send_header(k, v)
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self):
n = int(self.headers.get("Content-Length", 0))
body = json.loads(self.rfile.read(n) or b"{}")
auth = self.headers.get("Authorization", "")
token = auth[7:] if auth.startswith("Bearer ") else ""
if self.path != "/v1/messages":
return self._send(404, {"error": {"type": "not_found", "message": "unknown path"}})
if not token:
return self._send(401, {"error": {"type": "authentication_error",
"message": "missing bearer token"}})
if token == "readonly":
return self._send(403, {"error": {"type": "permission_error",
"message": "key cannot write"}})
if "messages" not in body:
return self._send(400, {"error": {"type": "invalid_request_error",
"message": "messages: field required"}})
if body.get("model") == "busy":
return self._send(429, {"error": {"type": "rate_limit_error",
"message": "slow down"}},
{"Retry-After": "2",
"x-ratelimit-remaining-requests": "0",
"x-ratelimit-reset-requests": "2s"})
return self._send(200, {"id": "msg_01", "content": "hello"},
{"x-ratelimit-remaining-requests": "59"})
def log_message(self, *args):
pass # keep the test output clean
server = HTTPServer(("127.0.0.1", 0), FakeProvider)
threading.Thread(target=server.serve_forever, daemon=True).start()
BASE = "http://127.0.0.1:%d" % server.server_address[1]
def call(path="/v1/messages", token="sk-test", payload=None, timeout=5):
req = urllib.request.Request(
BASE + path,
data=json.dumps(payload if payload is not None else
{"model": "small", "messages": []}).encode(),
headers={"Authorization": "Bearer " + token,
"Content-Type": "application/json"},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
return r.status, dict(r.headers), json.loads(r.read())
except urllib.error.HTTPError as e:
return e.code, dict(e.headers), json.loads(e.read())
for label, kwargs in (
("ok", {}),
("no token", {"token": ""}),
("read-only key", {"token": "readonly"}),
("wrong path", {"path": "/v1/message"}),
("bad payload", {"payload": {"model": "small"}}),
("rate limited", {"payload": {"model": "busy", "messages": []}}),
):
status, headers, body = call(**kwargs)
detail = body.get("error", {}).get("message", body.get("content", ""))
print("%-14s %3d %-28s retry-after=%s"
% (label, status, detail, headers.get("Retry-After")))
# ok 200 hello retry-after=None
# no token 401 missing bearer token retry-after=None
# read-only key 403 key cannot write retry-after=None
# wrong path 404 unknown path retry-after=None
# bad payload 400 messages: field required retry-after=None
# rate limited 429 slow down retry-after=2One detail in that handler is worth copying rather than skimming. The 401 check tests the token, not the header: an empty Authorization header still starts with the word Bearer, so a naive prefix check accepts a request with no credentials in it. That is a real class of authentication bug, and writing the server is how you meet it.
Turning a response into a decision#
Callers do not want a status code, they want to know what to do next, and there are only four answers. Writing that mapping down once, in one function, is the difference between a client that degrades and one that produces a stack trace.
def triage(status, headers, body):
"""Turn a response into one of four actions. Everything a caller needs to
decide is in the status; everything a human needs to fix it is in the body."""
err = body.get("error", {})
where = err.get("message", "")
rid = headers.get("request-id", "-")
if status < 400:
return "use it", ""
if status in (408, 429) or status >= 500:
wait = headers.get("Retry-After")
return "retry", "after %s s" % wait if wait else "with backoff"
if status in (401, 403):
return "fix the credential", where
return "fix the request (id %s)" % rid, where
for label, kwargs in (
("ok", {}),
("no token", {"token": ""}),
("bad payload", {"payload": {"model": "small"}}),
("rate limited", {"payload": {"model": "busy", "messages": []}}),
):
action, detail = triage(*call(**kwargs))
print(("%-14s -> %-26s %s" % (label, action, detail)).rstrip())
# ok -> use it
# no token -> fix the credential missing bearer token
# bad payload -> fix the request (id req_0001) messages: field required
# rate limited -> retry after 2 sHeaders that matter#
on the way out
Authorization Bearer <key>. Some providers use their own header instead.
Content-Type application/json, or the upload fails with 415
Accept application/json, or text/event-stream for streaming
Idempotency-Key your id for this operation, so a retry cannot duplicate it
User-Agent name your service. It is what appears in their logs.
on the way back
request-id the one thing support will ask you for. Log it always.
Retry-After seconds, or a date. Obey it rather than guessing.
x-ratelimit-remaining-requests how many calls are left in this window
x-ratelimit-remaining-tokens the limit that surprises people
Content-Type check it before parsing. An HTML error page is not JSON.The last line of that list catches a failure that looks baffling. When something in front of the API fails, a load balancer or a proxy, you get an HTML error page with a 502 status, and code that calls json on every response raises a parse error instead of reporting the 502. Check the status first, then the content type, then parse.
Read the error body#
Most APIs put a useful message in the body of an error response, and most code throws it away and logs only the status. That habit turns a five-second fix into an afternoon.
if r.status_code >= 400:
log.error("api_error", extra={
"status": r.status_code,
"body": r.text[:500], # the part that says what went wrong
"request_id": r.headers.get("request-id"),
})Truncate rather than logging the whole body, because an error body can be large and a failing endpoint produces a lot of them. Five hundred characters reaches the message on every provider worth using. And never log the request body alongside it without thinking, since that is where the user's prompt lives.
Where people go wrong
Not setting a timeout. The default in many clients is to wait indefinitely, so one stalled connection hangs a process until someone notices.
If this comes up in an interview
What is the difference between 401 and 403?
401 means the request was not authenticated, so the credentials are missing or invalid. 403 means authentication succeeded but the caller is not permitted to do this. One is a credentials problem, the other a permissions problem.
Which HTTP errors should you retry?
429 after a backoff, and 5xx and network timeouts, which are often transient. Not 4xx generally, because an identical request will fail identically, so retrying wastes time and money.
What is idempotency and why does it matter for API calls?
An idempotent request has the same effect whether it runs once or several times. It matters because a timeout does not tell you whether the server acted, so a retry can duplicate the action. An idempotency key lets the server recognise the repeat and return the original result.
Build this: Call an API and deliberately trigger a 400, a 401 and a 429. Log the full error body each time and note how much more useful it is than the status code alone.
Watch it explained
This is one section of HTTP & APIs, 8 hours across 2 sections. Read it in full to get the whole topic in order.
Stop 2 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.