Working with CSV Data

By the end of this lesson you can read a CSV document from any file object, survive quoted cells and ragged rows, and write CSV back out so it matches byte for byte. In Python Fundamentals you used split(",") on tidy lines; here you see where that breaks.

Why split(",") is not enough

CSV (comma-separated values) is plain text, one record per line. The catch is that a cell may itself contain a comma, a quote or a newline, and then the file wraps that cell in double quotes:

['7', '"Dune', ' 2nd ed."', 'Ana']
['7', 'Dune, 2nd ed.', 'Ana']

split returns four pieces with stray quotes; csv.reader returns the three real cells. Its rules:

• A cell containing the delimiter, a quote or a newline is wrapped in "...".
• A quote inside a quoted cell is written twice: "The ""Hobbit""" is the cell The "Hobbit".
• A quoted cell may span lines, so a record is not always one line.

File objects and csv.reader

csv.reader takes any file object: anything you can loop over to get lines of text. That includes sys.stdin, the result of open(path, newline=""), and io.StringIO(text), which wraps a string so it behaves like a file. While a quote is open the reader pulls in more lines, so hand it the file, never lines you split yourself:

['book', 'borrower', 'days']
['The "Hobbit"', 'Ana', '14']
['Notes\nvol 2', 'Ben', '7']
['Short', 'Cy']

Every cell is a str; convert at the boundary and reject a bad cell with the try/except you know. The last row has two cells, not three: the reader returns a short list without complaint, so checking len(row) is your job. For a real file you would write with open("loans.csv", newline="") as f:; the with block closes it for you, and newline="" lets the reader see line endings itself. The grader has no files, so exercises use sys.stdin and sys.stdout.

csv.DictReader uses the first row as keys, and shows ragged rows differently:

{'book': 'The "Hobbit"', 'borrower': 'Ana', 'days': '14'}
{'book': 'Notes\nvol 2', 'borrower': 'Ben', 'days': '7'}
{'book': 'Short', 'borrower': 'Cy', 'days': None}

A missing cell becomes None, so int(row["days"]) raises TypeError, not ValueError. Extra cells are collected in a list under the key None. Pass delimiter=";" for other separators.

Reading input of unknown length

Instead of an EOFError loop, for line in sys.stdin yields every line with its trailing "\n", and sys.stdin.read() returns everything as one string. For CSV, skip both and give the reader the stream: csv.reader(sys.stdin).

Writing CSV

csv.writer(file_object) has writerow(list). It quotes cells on its own, converts numbers with str, and ends each row with \r\n by default:

'"Dune, 2nd ed.",14\r\n'
'"The ""Hobbit""",3.5\n'

The \r is invisible on screen but an exact comparison sees it, so pass lineterminator="\n" when the output is checked. Because numbers go through str, 3.5 is written as 3.5; for 3.50 format it yourself first with "%.2f" % value.

Common mistakes

• Splitting lines yourself. Smith, John becomes two cells, every later column shifts by one, and a total comes out wrong or a row is wrongly rejected. Hand the stream to csv.reader.
• Leaving the default line terminator. Every line ends in \r\n, so a report that looks right fails the exact comparison.
• Trusting DictReader on ragged rows. A short row yields None and crashes int(None) with a TypeError; a long row looks fine because the surplus hides under the None key and is never counted.
• Writing a float without formatting. 3.5 instead of 3.50, or 0.30000000000000004.

Your exercise

Build a sales report from a CSV document on standard input: a total per customer, written as CSV, plus a count of unusable rows. A negative quantity is still valid.