Synopsis

Consider the following source file (example.py):

>>> foo = 'FOO'
>>> def bar_function():
...     print foo
>>> class Bling:
...     foo = 'Bling's FOO'
...     def bar_method(self):
...         print self.foo

The scan function returns a Token instance representing the file that was scanned:

>>> import pudge.scanner as scanner
>>> file_tok = scanner.scan('example.py')
>>> (file_tok.type, file_tok.name)
('file', 'example.py')

Traverse the token tree using Token.find :

>>> tok = file_tok.find('foo')
>>> (tok.type, tok.name, tok.line, tok.last_line)
('=', 'foo', 1, 2)

Line numbers are one piece of information not available via introspection.

You can traverse multiple levels of depth using dot notation:

>>> tok = file_tok.find('Bling.bar_method')
>>> (tok.type, tok.name, tok.line, tok.last_line)
('def', 'bar_method', 6, 8)

Token instances can be treated like dictionaries for syntactic pleasure; this is just like calling find:

>>> tok = file_tok['Bling']
>>> (tok.type, tok.name, tok.children)
('class', 'Bling', [<Token('=', 'foo')>, <Token('def', 'bar_method')>])

Note also that the Token.children attribute contains a list containing the immediately children of the token. This provides a source level order of tokens which is not available via introspection.