[Python-Dev] should sre.Scanner be exposed through re and documented? (original) (raw)

Gary Herron gherron@islandtraining.com
Fri, 25 Apr 2003 12:49:33 -0700

On Friday 25 April 2003 12:38 pm, Guido van Rossum wrote:

> While moving tests from testsre to testre I stumbled upon a simple test > for sre.Scanner. This looks fairly cool. Should it be exposed through > re and documented?

What's Scanner?

You create a Scanner instance with a list of re's and associated functions, then you use it to scan a string, returning a list of parts which match the given re's. (Actually the matches are run through the associated functions, and their output is what forms the returned list.)

Here's the single test case Skip refereed to:

def s_ident(scanner, token): return token def s_operator(scanner, token): return "op%s" % token def s_float(scanner, token): return float(token) def s_int(scanner, token): return int(token)

scanner = sre.Scanner([ (r"[a-zA-Z_]\w*", s_ident), (r"\d+.\d*", s_float), (r"\d+", s_int), (r"=|+|-|*|/", s_operator), (r"\s+", None), ])

sanity check

test('scanner.scan("sum = 3foo + 312.50 + bar")', (['sum', 'op=', 3, 'op', 'foo', 'op+', 312.5, 'op+', 'bar'], ''))

Gary Herron