Errors, files and modules
By the end: You can handle failures properly, read and write files, and split code across modules.
- try and except, and catching the right thing
- Raising your own errors
- Reading and writing files with with
- Working with JSON
- Imports and modules
New to python? A programming language is the set of written instructions a computer will follow, and Python is the one this field settled on. It reads close to English: a line that filters a list of records looks roughly like a sentence describing that filter. Every model provider ships a Python library, and nearly every machine learning tool is written in it. Why it matters
Three things a program needs the moment it leaves your own machine. What to do when something fails instead of stopping dead. How to read and write files, including JSON, which is the format almost every API speaks. And how to split code across several files and use one from another. Handling failure properly is what decides whether a bad response costs you one request or the whole run.
Handling errors#
When something goes wrong Python raises an exception and stops. try and except let you catch it and decide what to do.
try:
age = int(user_input)
except ValueError:
print("That is not a number")The common built-in errors are worth recognising on sight.
- ValueError: right type, wrong value. int("abc") raises this.
- TypeError: wrong type entirely. "2" + 2 raises this.
- KeyError: that key is not in the dictionary.
- IndexError: that position is past the end of the list.
- FileNotFoundError: no file at that path.
- AttributeError: that object has no such method or attribute. Usually means the value is None.
Always name the exception you are catching. A bare except catches everything, including your own typos and the Ctrl-C that stops the program.
# Bad: hides real bugs
try:
result = process(data)
except:
result = None
# Good: catches one thing, on purpose
try:
result = json.loads(raw)
except json.JSONDecodeError:
result = Noneelse runs if nothing went wrong. finally always runs, whether or not it did.
try:
data = fetch()
except TimeoutError:
log("timed out")
else:
save(data) # only if fetch succeeded
finally:
close_connection() # alwaysRaising your own#
Raise an error when your function is given something it cannot work with. Failing loudly and early beats returning something wrong.
def set_temperature(value):
if not 0 <= value <= 2:
raise ValueError(f"temperature must be 0 to 2, got {value}")
...
# Your own error type, for when callers need to tell yours apart
class ModelOutputError(Exception):
pass
raise ModelOutputError("model returned invalid JSON")Files#
Always open files with a with block. It closes the file for you, including when something raises partway through.
# Reading
with open("notes.txt") as f:
content = f.read() # whole file as one string
with open("notes.txt") as f:
for line in f: # one line at a time, memory friendly
print(line.strip())
# Writing
with open("out.txt", "w") as f: # "w" overwrites
f.write("hello\n")
with open("out.txt", "a") as f: # "a" appends
f.write("another line\n")w destroys the file
Opening with mode "w" empties the file immediately, before you write anything. If you meant to add to it, use "a".
JSON#
JSON is how data moves between systems, including every API you will call. Python converts between JSON text and dictionaries with two pairs of functions.
import json
# Text to Python
data = json.loads('{"name": "Jack"}') # from a string
with open("config.json") as f:
data = json.load(f) # from a file
# Python to text
text = json.dumps(data) # to a string
text = json.dumps(data, indent=2) # readable
with open("out.json", "w") as f:
json.dump(data, f, indent=2) # to a fileModules#
Any Python file is a module. Importing one makes its functions available. This is how you split a program into files instead of one enormous script.
import json # whole module
from pathlib import Path # one thing from a module
from collections import Counter, defaultdict
import numpy as np # with a shorter name
# Your own file, helpers.py, in the same folder
from helpers import clean_textThree standard library tools are worth knowing now, because they save you writing code by hand.
from collections import Counter, defaultdict
from pathlib import Path
Counter(["a", "b", "a"]).most_common(1) # [('a', 2)]
groups = defaultdict(list)
groups["fruit"].append("apple") # no need to create the list first
Path("data/out.json").parent.mkdir(parents=True, exist_ok=True)Where people go wrong
Catching an exception and then continuing with partial data. Half-processed data flowing onward is worse than a clear failure, because the damage appears somewhere unrelated and much later.
If this comes up in an interview
Why should you avoid a bare except?
It catches everything, including KeyboardInterrupt and SystemExit, so the program cannot be stopped normally. It also hides real bugs like typos, turning a crash you would have noticed into silently wrong output.
What is the difference between json.load and json.loads?
load reads from a file object. loads reads from a string. The same distinction applies to dump and dumps.
Why use with when opening a file?
It closes the file automatically when the block ends, even if an exception is raised inside it. Without it, a failure partway through can leave the file open.
When would you create a custom exception class?
When callers need to handle your failure differently from built-in ones. A ModelOutputError can be caught and retried, whereas catching ValueError would also catch unrelated problems.
Build this: Write a program that reads a JSON file, handles the file being missing and the JSON being malformed with different messages, and writes a cleaned version to a new file.
Watch it explained
This is one section of Python, 33 hours across 9 sections. Read it in full to get the whole topic in order.
Stop 1 of 50 on the AI Engineer route
0 of 50 topics done · 0h behind you, 603h to go
What you have marked done is stored in this browser only. It does not reach a server, and clearing site data will clear it.