Regular Expressions

You will write patterns that describe the shape of a piece of text, then use them to validate a whole string, pull fields out of it, and rewrite parts of it. split, startswith and in find text you can spell out; a pattern says "two digits, a colon, two digits", which they cannot. Write patterns as raw strings (r"\d+"): without the r, Python reads "\b" as a backspace character before re sees it, the backslash rule from your first lesson.

Picking the function

search, match and fullmatch return a match object or None. finditer yields one match object per hit (with .start(), .group()), where findall gives bare strings, or tuples when the pattern has groups. Validating means fullmatch; extracting means search or finditer.

The pattern language

\d digit, \w letter/digit/underscore, \s whitespace, [a-z.-] one of those characters, [^"] anything but a quote, \. a literal dot. * zero or more, + one or more, ? optional, {2,4} two to four. ^ and $ anchor the ends, a|b is either, and \b is a word boundary: \bcat\b finds cat in "cat concat" once, plain cat finds it twice.

Greedy versus lazy. Quantifiers take as much as they can, so .* runs to the last possible spot:

. is any character, so example.com also accepts exampleXcom. * accepts nothing at all: re.findall(r"\d*", "a12") returns ['', '12', '']. Use + when something must be there.

Groups

Parentheses capture. Name them with (?P<name>...); m.group("name"), m.group(2) and m.groupdict() all read them. A group that did not take part is None, and (?:...) groups without capturing:

re.compile builds the pattern once and gives you its methods. Flags go in as a second argument: re.I ignores case (re.fullmatch(r"v\d+", "V7", re.I) matches), and re.X lets you lay a pattern out over lines with comments and spaces ignored (write a literal space as [ ]).

Rewriting with sub

The replacement can refer back to groups with \1 or \g<name>, or be a function that receives the match object and returns the new text:

When not to use one

line.split(","), s.startswith("v") and "ERROR" in line are shorter and obvious. Reach for re when the text has optional parts or repeated structure.

Common mistakes

• re.match instead of re.fullmatch on a line: msg="ok" trailing still matches because nothing checks the end, so a damaged line counts as good.
• re.I "to be safe": error then matches where only ERROR is allowed, so a lowercase level is no longer malformed.
• A greedy .* for the field between brackets: \[.*\] stretches to the last ] that still lets the rest match, so a message containing ] msg=" pulls part of itself into the component.
• Substituting without \b: a pattern for ip= also rewrites the tail of skip=.

Your exercise

Parse log lines with one compiled pattern of named groups, count what matched, and mask part of an address with re.sub. The starter reads the input and prints the summary; you build the pattern and fill in the loop. The likeliest error is a pattern that accepts a line it should reject.