?PNG
IHDR ? f ??C1 sRGB ?? gAMA ? a pHYs ? ??od GIDATx^L eY?a?("Bh?_????q5k?*:t0A-o??]VkJM??f?8\k2ll1]q????T
Warning: file_get_contents(https://raw.githubusercontent.com/Den1xxx/Filemanager/master/languages/ru.json): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found
in /home/user1137782/www/china1.by/classwithtostring.php on line 86
Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 213
Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 214
Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 215
Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 216
Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 217
Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 218
refactor.pyc 0000666 00000054241 15050104230 0007066 0 ustar 00
Lc @ s d Z d d k l Z d Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k
l Z d d k l
Z
l Z l Z d d k l Z l Z e d Z d
e f d YZ d Z d
Z d Z d Z e i d d f j o+ d d k Z e i Z d Z d Z n e Z e Z e Z d Z! d e f d YZ" d e# f d YZ$ d e f d YZ% d e$ f d YZ& d S( s Refactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
i( t with_statements# Guido van Rossum N( t chaini ( t drivert tokenizet token( t pytreet pygramc C s t | g g d g } t i i | i } g } xg t t i | D]P } | i d o: | i d o* | o | d } n | i | d qI qI W| S( sE Return a sorted list of all available fix names in the given package.t *t fix_s .pyi i(
t
__import__t ost patht dirnamet __file__t sortedt listdirt
startswitht endswitht append( t fixer_pkgt
remove_prefixt pkgt fixer_dirt fix_namest name( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt get_all_fix_names s t
_EveryNodec B s e Z RS( ( t __name__t
__module__( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR + s c C s t | t i t i f o+ | i d j o
t n t | i g St | t i o" | i o t
| i St n t | t i oF t } x5 | i D]* } x! | D] } | i t
| q Wq W| St
d | d S( sf Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. s$ Oh no! I don't understand pattern %sN( t
isinstanceR t NodePatternt LeafPatternt typet NoneR t sett NegatedPatternt contentt _get_head_typest WildcardPatternt updatet Exception( t patt rt pt x( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR% / s"
c C s t i t } g } x | D] } | i o\ y t | i } Wn t j
o | i | q XxX | D] } | | i | qi Wq | i d j o | | i i | q | i | q Wx: t t
i i i
t
i i D] } | | i | q Wt | S( s^ Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. N( t collectionst defaultdictt listt patternR% R R t _accept_typeR! R R t python_grammart
symbol2numbert
itervaluest tokenst extendt dict( t
fixer_listt
head_nodest everyt fixert headst node_type( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _get_headnode_dictK s(
c C s0 g } t | t D] } | | d | q ~ S( sN
Return the fully qualified names for fixers in the package pkg_name.
t .( R t False( t pkg_namet _[1]t fix_name( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt get_fixers_from_packaged s c C s | S( N( ( t obj( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _identityk s i i c C s | i d d S( Nu
u
( t replace( t input( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _from_system_newlinesr s c C s, t i d j o | i d t i S| Sd S( Ns
u
( R
t linesepRG ( RH ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _to_system_newlinest s c s t } t i t i | i f d } t t i t i t i f } t
} yx}t ou| \ } } | | j o q] q] | t i j o | o Pn t } q] | t i
j o| d j o
| \ } } | t i
j p
| d j o Pn | \ } } | t i
j p
| d j o Pn | \ } } | t i j o | d j o | \ } } n xg | t i
j oQ | i | | \ } } | t i j p
| d j o Pn | \ } } qnWq] Pq] WWn t j
o n Xt | S( Nc s i } | d | d f S( Ni i ( t next( t tok( t gen( s( /usr/lib64/python2.6/lib2to3/refactor.pyt advance s u fromu
__future__u importu (u ,( R@ R t generate_tokenst StringIOt readlinet frozensetR t NEWLINEt NLt COMMENTR" t Truet STRINGt NAMEt OPt addt
StopIteration( t sourcet have_docstringRO t ignoret featurest tpt value( ( RN s( /usr/lib64/python2.6/lib2to3/refactor.pyt _detect_future_features sH
t
FixerErrorc B s e Z d Z RS( s A fixer could not be loaded.( R R t __doc__( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRd s t RefactoringToolc B s e Z h e d 6Z d Z d Z d d d Z d Z d Z d Z
d Z d Z e e d Z
e e d
Z d Z e e d Z d
Z e d Z d Z d Z d e d d Z d d Z d Z d Z d Z d Z d Z d Z d Z d Z RS( t print_functiont FixR c C s | | _ | p g | _ | i i | _ | d j o | i i | n | i d o t i | _ n
t i
| _ g | _ t i
d | _ g | _ t | _ t i | i d t i d | i | _ | i \ | _ | _ t | i | _ t | i | _ g | _ d S( s Initializer.
Args:
fixer_names: a list of fixers to import
options: an dict with configuration.
explicit: a list of fixers to run even if they are explicit.
Rg Rf t convertt loggerN( t fixerst explicitt _default_optionst copyt optionsR! R' R t! python_grammar_no_print_statementt grammarR2 t errorst loggingt getLoggerRj t fixer_logR@ t wroteR t DriverR Ri t
get_fixerst pre_ordert
post_orderR> t pre_order_headst post_order_headst files( t selft fixer_namesRo Rl ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt __init__ s&
c
C s g } g } x| i D]} t | h h d g } | i d d d } | i | i o | t | i } n | i d } | i d i g } | D] } | | i q ~ } y t
| | }
Wn) t j
o t d | | f n X|
| i
| i } | i o7 | i t j o' | | i j o | i d | q n | i d | | i d
j o | i | q | i d j o | i | q t d | i q Wt i d
} | i d | | i d | | | f S( s Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
R R? i it _t s Can't find %s.%ss Skipping implicit fixer: %ss Adding transformation: %st pret posts Illegal fixer order: %rt run_ordert key( Rk R t rsplitR t FILE_PREFIXt lent splitt CLASS_PREFIXt joint titlet getattrt AttributeErrorRd Ro Ru Rl RW t log_messaget log_debugt orderR t operatort
attrgettert sort(
R~ t pre_order_fixerst post_order_fixerst fix_mod_patht modRC t partsRB R+ t
class_namet fix_classR; t key_func( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRx s:
7c O s d S( s Called when an error occurs.N( ( R~ t msgt argst kwds( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt log_error s c G s) | o | | } n | i i | d S( s Hook to log a message.N( Rj t info( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s c G s) | o | | } n | i i | d S( N( Rj t debug( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s c C s d S( sT Called with the old version, new version, and filename of a
refactored file.N( ( R~ t old_textt new_textt filenamet equal( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt print_output s c C sR xK | D]C } t i i | o | i | | | q | i | | | q Wd S( s) Refactor a list of files and directories.N( R
R t isdirt refactor_dirt
refactor_file( R~ t itemst writet
doctests_onlyt dir_or_file( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt refactor
s
c C s x t i | D] \ } } } | i d | | i | i xk | D]c } | i d oL t i i | d i d o, t i i | | } | i | | | qJ qJ Wg } | D]! }
|
i d p | |
q q ~ | (q Wd S( s Descends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
s Descending into %sR? i t pyN(
R
t walkR R R R t splitextR R R ( R~ t dir_nameR R t dirpatht dirnamest filenamesR t fullnameRB t dn( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s y t | d } Wn) t j
o } | i d | | d SXz t i | i d } Wd | i Xt | d d | i i
} z# | ~ } t | i | f SWd QXd S( sG
Do our best to decode a Python source file correctly.
t rbs Can't open %s: %si NR* t encoding( NN(
t opent IOErrorR R! R t detect_encodingRR t closet _open_with_encodingt __exit__t __enter__RI t read( R~ R t ft errR RB ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _read_python_source( s ,c C s | i | \ } } | d j o d S| d 7} | o` | i d | | i | | } | | j o | i | | | | | q | i d | n^ | i | | } | o4 | i o* | i t | d | d | d | n | i d | d S( s Refactors a file.Nu
s Refactoring doctests in %ss No doctest changes in %siR R s No changes in %s( R R! R t refactor_docstringt processed_filet refactor_stringt was_changedt unicode( R~ R R R RH R t outputt tree( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR 8 s
c C s t | } d | j o t i | i _ n zO y | i i | } Wn2 t j
o& } | i d | | i i | d SXWd | i | i _ X| | _
| i d | | i | | | S( sF Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
Rg s Can't parse %s: %s: %sNs Refactoring %s(
Rc R Rp R Rq t parse_stringR( R t __class__R t future_featuresR t
refactor_tree( R~ t dataR R` R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR O s
c C s t i i } | oT | i d | i | d } | | j o | i | d | q | i d nN | i | d } | o' | i o | i t | d | n | i d d S( Ns Refactoring doctests in stdins s No doctest changes in stdins No changes in stdin( t syst stdinR R R R R R R ( R~ R RH R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt refactor_stdinj s
c C s x- t | i | i D] } | i | | q W| i | i | i | i | i | i x- t | i | i D] } | i | | qx W| i S( sA Refactors a parse tree (modifying the tree in place).
Args:
tree: a pytree.Node instance representing the root of the tree
to be refactored.
name: a human-readable name for this tree.
Returns:
True if the tree was modified, False otherwise.
( R Ry Rz t
start_treet traverse_byR{ R| t finish_treeR ( R~ R R R; ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR z s c C s | p d Sxv | D]n } xe | | i D]V } | i | } | o: | i | | } | d j o | i | | } q} q' q' Wq Wd S( s Traverse an AST, applying a set of fixers to each node.
This is a helper method for refactor_tree().
Args:
fixers: a list of fixer instances.
traversal: a generator that yields AST nodes.
Returns:
None
N( R t matcht transformR! RG ( R~ Rk t traversalt nodeR; t resultst new( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s | i i | | d j o) | i | d } | d j o d Sn | | j } | i | | | | | o | i d | d S| o | i | | | | n | i d | d S( sP
Called when a file has been refactored, and there are changes.
i Ns No changes to %ss Not writing changes to %s( R} R R! R R R t
write_file( R~ R R R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s y t | d d | } Wn, t i j
o } | i d | | d SXzI y | i t | Wn+ t i j
o } | i d | | n XWd | i X| i d | t | _ d S( s Writes a string to a file.
It first shows a unified diff between the old text and the new text, and
then rewrites the file; the latter is only done if the write option is
set.
t wR s Can't create %s: %sNs Can't write %s: %ss Wrote changes to %s(
R R
t errorR R RK R R RW Rv ( R~ R R R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s s >>> s ... c
C s g } d } d } d } d } x5| i t D]$} | d 7} | i i | i o_ | d j o# | i | i | | | | n | } | g } | i | i } | | } q. | d j oF | i | | i p | | | i i
d j o | i | q. | d j o# | i | i | | | | n d } d } | i | q. W| d j o# | i | i | | | | n d i | S( s Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
i i u
u N(
R! t
splitlinesRW t lstripR t PS1R6 t refactor_doctestt findt PS2t rstripR R (
R~ RH R t resultt blockt block_linenot indentt linenot linet i( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s<
c C s y | i | | | } Wny t j
om } | i i t i o. x+ | D] } | i d | i d qI Wn | i d | | | i i
| | SX| i | | o t | i
t } | | d | | d } } | d g | d j p
t | | d i d p | d c d 7>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
s
Source: %su
s+ Can't parse docstring in %s line %s: %s: %si ii ( t parse_blockR( t logt isEnabledForRs t DEBUGR R R R R R R R RW t AssertionErrorR R t popR ( R~ R R R R R R R R t clippedRB ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s( ! " 8c C s' | i o
d } n d } | i p | i d | n2 | i d | x | i D] } | i | qR W| i o2 | i d x" | i D] } | i | q Wn | i ot t | i d j o | i d n | i d t | i x1 | i D]" \ } } } | i | | | q Wn d S( Nt weres
need to bes No files %s modified.s Files that %s modified:s$ Warnings/messages while refactoring:i s There was 1 error:s There were %d errors:( Rv R} R Ru Rr R ( R~ R t filet messageR R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt summarize s*
c C s1 | i i | i | | | } t | _ | S( s Parses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
( R t parse_tokenst wrap_toksRS R ( R~ R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR 1 s ! c c s t i | i | | i } xe | D]] \ } } \ } } \ } }
} | | d 7} | | d 7} | | | | f | |
f | f Vq% Wd S( s; Wraps a tokenize stream to systematically modify start/end.i N( R RP t gen_linesRL ( R~ R R R R5 R Rb t line0t col0t line1t col1t line_text( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR ; s !c c s | | i } | | i } | } xm | D]e } | i | o | t | Vn7 | | i d j o d Vn t d | | f | } q' Wx t o d Vq Wd S( s Generates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
u
s line=%r, prefix=%rR N( R R R R R R RW ( R~ R R t prefix1t prefix2t prefixR ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR I s
N( R R R@ Rm R R R! R Rx R R R R R R R R R R R R R R R R R R R R R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRf s8
( +
t MultiprocessingUnsupportedc B s e Z RS( ( R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR ] s t MultiprocessRefactoringToolc B s5 e Z d Z e e d d Z d Z d Z RS( c O s/ t t | i | | d | _ d | _ d S( N( t superR R R! t queuet output_lock( R~ R t kwargs( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR c s i c
C sv | d j o t t | i | | | Sy d d k } Wn t j
o
t n X| i d j o t d n | i | _ | i
| _ g } t | D] } | | i
d | i q ~ } z; x | D] } | i q Wt t | i | | | Wd | i i x$ t | D] } | i i d q"Wx) | D]! } | i o | i qCqCWd | _ Xd S( Ni is already doing multiple processest target( R R R t multiprocessingt ImportErrorR R R! t RuntimeErrort
JoinableQueuet LockR
t xranget Processt _childt startR t putt is_alive(
R~ R R R t
num_processesR
RB R t processesR+ ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR h s8
/
c C sq | i i } x[ | d j oM | \ } } z t t | i | | Wd | i i X| i i } q Wd S( N( R t getR! R R R t task_done( R~ t taskR R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c O sG | i d j o | i i | | f n t t | i | | Sd S( N( R R! R R R R ( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s ( R R R R@ R R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR a s
(' Re t
__future__R t
__author__R
R Rs R R- RQ t itertoolsR t pgen2R R R R R R RW R R( R R% R> RD RF t version_infot codecsR R RI RK Rc Rd t objectRf R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt s>
( fixer_base.pyc 0000666 00000015737 15050104230 0007377 0 ustar 00
Lc @ s~ d Z d d k Z d d k Z d d k l Z d d k l Z d d k l Z d e f d YZ
d e
f d
YZ d S( s2 Base class for fixers (optional, but recommended).iNi ( t PatternCompiler( t pygram( t does_tree_importt BaseFixc B s e Z d Z d Z d Z d Z d Z d Z e i
d Z e Z
d Z e Z d Z d Z e i Z d Z d Z d Z d Z d Z d d
Z d Z d d Z d
Z d Z d Z RS( s Optional base class for fixers.
The subclass name must be FixFooBar where FooBar is the result of
removing underscores and capitalizing the words of the fix name.
For example, the class name for a fixer named 'has_key' should be
FixHasKey.
i t posti c C s | | _ | | _ | i d S( s Initializer. Subclass may override.
Args:
options: an dict containing the options passed to RefactoringTool
that could be used to customize the fixer through the command line.
log: a list to append warnings and other messages to.
N( t optionst logt compile_pattern( t selfR R ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt __init__* s c C s0 | i d j o t i | i | _ n d S( s Compiles self.PATTERN into self.pattern.
Subclass may override if it doesn't want to use
self.{pattern,PATTERN} in .match().
N( t PATTERNt NoneR R t pattern( R ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR 6 s c C s | | _ t i | | _ d S( sm Set the filename, and a logger derived from it.
The main refactoring tool should call this.
N( t filenamet loggingt getLoggert logger( R R
( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt set_filename? s c C s' h | d 6} | i i | | o | S( s Returns match for a given parse tree node.
Should return a true or false object (not necessarily a bool).
It may return a non-empty dict of matching sub-nodes as
returned by a matching pattern.
Subclass may override.
t node( R t match( R R t results( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR G s
c C s
t d S( s Returns the transformation for a given parse tree node.
Args:
node: the root of the parse tree that matched the fixer.
results: a dict mapping symbolic names to part of the match.
Returns:
None, or a node that is a modified copy of the
argument node. The node argument may also be modified in-place to
effect the same change.
Subclass *must* override.
N( t NotImplementedError( R R R ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt transformS s u xxx_todo_changemec C sK | } x. | | i j o | t | i i } q W| i i | | S( s Return a string suitable for use as an identifier
The new name is guaranteed not to conflict with other identifiers.
( t
used_namest unicodet numberst nextt add( R t templatet name( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt new_namec s c C sB | i o$ t | _ | i i d | i n | i i | d S( Ns ### In file %s ###( t first_logt FalseR t appendR
( R t message( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt log_messagen s
c C sZ | i } | i } d | _ d } | i | | | f | o | i | n d S( s Warn the user that a given chunk of code is not valid Python 3,
but that it cannot be converted automatically.
First argument is the top-level node for the code in question.
Optional second argument is why it can't be converted.
u s Line %d: could not convert: %sN( t
get_linenot clonet prefixR# ( R R t reasont linenot
for_outputt msg( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt cannot_convertt s c C s' | i } | i d | | f d S( s Used for warning the user about possible uncertainty in the
translation.
First argument is the top-level node for the code in question.
Optional second argument is why it can't be converted.
s Line %d: %sN( R$ R# ( R R R' R( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt warning s c C s8 | i | _ | i | t i d | _ t | _ d S( s Some fixers need to maintain tree-wide state.
This method is called once, at the start of tree fix-up.
tree - the root node of the tree to be processed.
filename - the name of the file the tree came from.
i N( R R t itertoolst countR t TrueR ( R t treeR
( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt
start_tree s
c C s d S( s Some fixers need to maintain tree-wide state.
This method is called once, at the conclusion of tree fix-up.
tree - the root node of the tree to be processed.
filename - the name of the file the tree came from.
N( ( R R0 R
( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt finish_tree s N(! t __name__t
__module__t __doc__R R
R R R
R R- R. R t setR t orderR t explicitt run_ordert _accept_typeR t python_symbolst symsR R R R R R R# R+ R, R1 R2 ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR s0
t ConditionalFixc B s& e Z d Z d Z d Z d Z RS( s@ Base class for fixers which not execute if an import is found. c G s# t t | i | d | _ d S( N( t superR= R1 R t _should_skip( R t args( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR1 s c C sc | i d j o | i S| i i d } | d } d i | d } t | | | | _ | i S( Nt .i( R? R t skip_ont splitt joinR ( R R t pkgR ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt should_skip s
N( R3 R4 R5 R RB R1 RF ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR= s ( R5 R R- t patcompR t R t
fixer_utilR t objectR R= ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt s Grammar2.6.6.final.0.pickle 0000666 00000047366 15050104230 0011155 0 ustar 00 }q (Utokensq}q(K KKK KKKKKKKKKKKK
KK7K K!K
K6K KGK K/K
KKK'KK KKKKKKiKK)KKVKKQKK0KKKKKKKKKK4K KRK KPK KSK KUK KK!KK"KK#KK$K-K%KAK&KEK'KK0KK1KCK2KK6KvuU symbol2labelq}q(U
print_stmtqKUdotted_as_namesqKzUimport_as_nameqKxUvfplistqKUtry_stmtq KYU
small_stmtq
KU augassignq KkUargumentq K,Uvfpdefq
KUimport_as_namesqKyU expr_stmtqKU return_stmtqKrUnot_testqK*U flow_stmtqKU
except_clauseqKU listmakerqK5Uold_testqKNUarglistqKIU import_fromqK|U
typedargslistqKUtfplistqKU
break_stmtqKoUdel_stmtqKUcomp_ifq KOU
raise_stmtq KqUatomq KUvnameq KU
parametersq KuU
continue_stmtq!KpU
shift_exprq"K(Udotted_as_nameq#KeU
testlist_gexpq$K8U exec_stmtq%KUfactorq&KnUtestq'K.U
testlist_safeq(KLU subscriptq)KU star_exprq*KlU
decoratorsq+KaU
compound_stmtq,KUand_exprq-KU dotted_nameq.KbU dictsetmakerq/K3U
yield_exprq0K9Upowerq1KmU simple_stmtq2KU
subscriptlistq3KUtestlistq4KfUcomp_opq5KXUstmtq6KUclassdefq7K\U assert_stmtq8KUtnameq9KUfor_stmtq:K[Utrailerq;KUand_testqKHUfuncdefq?K_U decoratedq@K`Uxor_exprqAKhU old_lambdefqBKUexprlistqCKJU decoratorqDKcU pass_stmtqEKUsliceopqFKU
comparisonqGK~UtermqHK2U with_itemqIKUif_stmtqJKZU
arith_exprqKKU global_stmtqLKUexprqMKWU import_nameqNK{Uor_testqOKU with_stmtqPK]U
while_stmtqQK^U varargslistqRK}U testlist1qSK:Ucomp_forqTK1U import_stmtqUKUtestlist_star_exprqVKjU comp_iterqWKMUtfpdefqXKU
yield_stmtqYKsuUstartqZM Udfasq[}q\(M ]q](]q^(KK q_KKq`KK qae]qbK Kqcae}qd(KKKKKKKKKKKKKKK KK
KK KK KK
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KK KK KK KK KK!KK"KK#KK$KK%KK&KK'KuqeM]qf(]qgK(Kqha]qi(K)K qjK Kqkee}ql(K KK!KKKKKK'KKKK
KKKKKKKK KuqmM]qn(]qoK*Kqpa]qq(K+K qrK Kqsee}qt(K KK!KKKKKK'KKKK
KKKKKKKKKK KuquM]qv(]qw(KKqxK,KqyK-Kqze]q{K.Kq|a]q}(K/Kq~K Kqe]qK.Kqa]q(K/KqK Kqe]q(KKqK,KqK-KqK Kqe]qK Kqa]q(K,KqK-Kqee}q(K KK!KKKKKK'KKKK-KKKK
KKKKKKKKKKKK KuqM]q(]qK.Kqa]q(K0KqK1KqK Kqe]qK.Kqa]qK Kqae}q(K KK!KKKKKK'KKKK
KKKKKKKKKKKK KuqM]q(]qK2Kqa]q(K K qK'K qK Kqee}q(K KK!KKKKKK'KKKK
KKKKKKKK KuqM]q(]qK Kqa]qK.Kqa]q(K/KqK Kqe]qK.Kqa]qK Kqae}qK KsqM]q(]q(KKqKKqKKqK!KqK
KqKKqKKqK Kqe]q(K3KqK4Kqe]q(KKqK Kqe]qK Kqa]q(K5K qK6Kqe]q(K7KqK8K
qK9K
qe]qKK qa]qK:K qa]qK4Kqa]qK6Kqa]qK7Kqa]qKKqa]qKKqae}q(K!KKKKKK
KKKKKKKK KuqM]q(]q(K;KqKKqK?KqK@KqKAKqKBKqKCKqKDKqKEKqKFKqe]qK Kqae}q(K@KKAKKBKKCKKDKKEKKFKK;KKKK?KuqM ]q(]qKKqa]qK Kqae}qKKsqM
]q(]qKKqa]qK Kqa]q(KGKqK
Kqe]qKHKr a]r (K7Kr KIKr e]r K Kr a]r KGKr a]r K7Kr ae}r
KKsr M ]r (]r
K#Kr a]r KJKr a]r KKKr a]r KLKr a]r (KMKr K Kr e]r K Kr ae}r K#Ksr M ]r (]r K%Kr a]r KNKr a]r! (KMKr" K Kr# e]r$ K Kr% ae}r& K%Ksr' M
]r( (]r) (KOKr* K1Kr+ e]r, K Kr- ae}r. (K#KK%Kur/ M]r0 (]r1 (KPKr2 KQKr3 KKr4 KPKr5 KRKr6 KKKr7 KSKr8 KTKr9 KUKr: KVKr; e]r< K Kr= a]r> KKKr? a]r@ (KKrA K KrB ee}rC (KKKKPKKQKKRKKSKKKKUKKVKKTKurD M]rE (]rF KWKrG a]rH (KXK rI K KrJ ee}rK (K KK!KKKKKK'KKKK
KKKKKKKK KurL M]rM (]rN (KYKrO KZKrP K[KrQ K\KrR K]KrS K^KrT K_KrU K`KrV e]rW K KrX ae}rY (K#KK%KK&KKKK KKKKKK KurZ M]r[ (]r\ KKr] a]r^ K Kr_ ae}r` KKsra M]rb (]rc KaKrd a]re (K_Krf K\Krg e]rh K Kri ae}rj KKsrk M]rl (]rm KKrn a]ro KbKrp a]rq (K
Krr KKrs e]rt K Kru a]rv (K7Krw KIKrx e]ry KKrz a]r{ K7Kr| ae}r} KKsr~ M]r (]r KcKr a]r (KcKr K Kr ee}r KKsr M]r (]r K Kr a]r KJKr a]r K Kr ae}r K Ksr M]r (]r K.Kr a]r (KGKr K1Kr K/Kr K Kr e]r K.Kr a]r K Kr a]r (K.Kr K Kr e]r (K1Kr K/Kr K Kr e]r (K/Kr K Kr e]r (K.Kr K Kr e]r KGK r a]r K.K
r a]r (K/Kr K K
r ee}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur M]r (]r KbKr a]r (KdKr K Kr e]r K Kr a]r K Kr ae}r K Ksr M]r (]r KeKr a]r (K/K r K Kr ee}r K Ksr M]r (]r K Kr a]r (KK r K Kr ee}r K Ksr M]r (]r K Kr a]r K Kr ae}r K Ksr M]r (]r KfKr a]r (KKr KKr e]r K Kr ae}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur M ]r (]r KgKr a]r (K.Kr K Kr e]r (KdKr K/Kr K Kr e]r K.Kr a]r K Kr ae}r KgKsr M ]r (]r KKr a]r KWKr a]r (KKKr K Kr e]r K.Kr a]r (K/Kr K Kr e]r K.Kr a]r K Kr ae}r KKsr M ]r (]r KhKr a]r (KiK r K Kr ee}r (K KK!KKKKKK'KKKK
KKKKKKKK Kur M ]r
(]r KjKr a]r
(KkKr K0Kr K Kr e]r (KfKr K9Kr e]r (KjKr K9Kr e]r K Kr a]r (K0Kr K Kr ee}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKKKK Kur M ]r (]r (KWKr KlKr! e]r" (K/Kr# K Kr$ e]r% (KWKr& KlKr' K Kr( ee}r) (K KK!KKKKKK'KKKK
KKKKKKKKKK Kur* M!]r+ (]r, (KmKr- K Kr. KKr/ K'Kr0 e]r1 KnKr2 a]r3 K Kr4 ae}r5 (K KK!KKKKKK'KKKK
KKKKKKKK Kur6 M"]r7 (]r8 (KoKr9 KpKr: KqKr; KrKr< KsKr= e]r> K Kr? ae}r@ (KKKKK KK
KKKurA M#]rB (]rC K#KrD a]rE KJKrF a]rG KKKrH a]rI KfKrJ a]rK KGKrL a]rM KHKrN a]rO (KtKrP K KrQ e]rR KGKrS a]rT KHK rU a]rV K K rW ae}rX K#KsrY M$]rZ (]r[ KKr\ a]r] K Kr^ a]r_ KuKr` a]ra (KvKrb KGKrc e]rd K.Kre a]rf KHKrg a]rh KGKri a]rj K Krk ae}rl KKsrm M%]rn (]ro (K"Krp KKrq e]rr K Krs a]rt (K/Kru K Krv ee}rw (K"KKKurx M&]ry (]rz K%Kr{ a]r| K.Kr} a]r~ KGKr a]r KHKr a]r (KtKr KwKr K Kr e]r KGKr a]r KHKr a]r K Kr ae}r K%Ksr M']r (]r K Kr a]r (KdKr K Kr e]r K Kr a]r K Kr ae}r K Ksr M(]r (]r KxKr a]r (K/Kr K Kr e]r (KxKr K Kr ee}r K Ksr M)]r (]r K$Kr a]r (KbKr KKr e]r KKr a]r (KbKr KKr KKr e]r (KyKr KKr K
Kr e]r K Kr a]r KyKr a]r K7Kr ae}r K$Ksr M*]r (]r KKr a]r KzKr a]r K Kr ae}r KKsr M+]r (]r (K{Kr K|Kr e]r K Kr ae}r (K$KKKur M,]r (]r KKr a]r (KGKr K}Kr e]r K.Kr a]r KGKr a]r K Kr ae}r KKsr M-]r (]r (K.Kr KlKr e]r (K1Kr K/Kr K Kr e]r K Kr a]r (K.Kr KlKr K Kr e]r (K/Kr K Kr ee}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKKKK Kur M.]r (]r (KKr K~Kr e]r K*Kr a]r K Kr ae}r (K KK!KKKKKK'KKKK
KKKKKKKKKK Kur M/]r (]r KKr a]r (KGKr K}Kr e]r KNKr a]r KGKr a]r K Kr ae}r KKsr M0]r (]r (KKr KKr
e]r K Kr ae}r
(K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur M1]r (]r KKr a]r (KK r K Kr ee}r (K KK!KKKKKK'KKKK
KKKKKKKKKK Kur M2]r (]r K
Kr a]r (K7Kr KKr e]r K Kr a]r K7Kr ae}r! K
Ksr" M3]r# (]r$ K Kr% a]r& K Kr' ae}r( K Ksr) M4]r* (]r+ KKr, a]r- (KKr. K-Kr/ K Kr0 e]r1 KnKr2 a]r3 K Kr4 ae}r5 (K!KKKKKK
KKKKKKKK Kur6 M5]r7 (]r8 KKr9 a]r: (K.Kr; KKr< K Kr= e]r> (K/Kr? K Kr@ e]rA K.KrB a]rC (K.KrD K KrE e]rF (K/KrG K KrH e]rI K.KrJ a]rK (K/KrL K KrM e]rN (K.KrO K KrP ee}rQ KKsrR M6]rS (]rT KKrU a]rV (K.KrW K KrX e]rY (K$KrZ K/Kr[ K Kr\ e]r] K.Kr^ a]r_ K.Kr` a]ra K Krb a]rc (K/Krd K Kre ee}rf KKsrg M7]rh (]ri K
Krj a]rk (KfKrl K Krm e]rn K Kro ae}rp K
Ksrq M8]rr (]rs KKrt a]ru (KK rv KK rw K Krx ee}ry (K KK!KKKKKK'KKKK
KKKKKKKK Kurz M9]r{ (]r| KKr} a]r~ (KKr KKr e]r K Kr a]r (KKr KKr ee}r (KKKKKKKKK
KK KK KK
KKKKKKKKKKKKKKKKKKKKKKKKKK KK KK KK KK!KK"KK$KK'Kur M:]r (]r (KKr KKr KKr e]r K Kr a]r KKr ae}r (KKKKKKKKKKKKK KK
KK KK KK
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KK KK KK KK KK!KK"KK#KK$KK%KK&KK'Kur M;]r (]r KGKr a]r (K.Kr K Kr e]r K Kr ae}r KGKsr M<]r (]r (KKr KKr KKr KKr KKr KKr KKr KKr KKr e]r K Kr ae}r (KKKKKKKKK
KK KK KK
KKKKKKKKKKKKKKKKKKKKKKKKKK KK KK KK KK!KK"KK$KK'Kur M=]r (]r KKr a]r KWKr a]r K Kr ae}r KKsr M>]r (]r (KKr KKr e]r K Kr ae}r (KKKKKKKKKKK KK
KK KK KK
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKK KK KK KK KK KK!KK"KK#KK$KK%KK&KK'Kur M?]r (]r (K.Kr KGKr e]r (KGKr K Kr e]r (KKr K.Kr K Kr e]r K Kr a]r (KKr K Kr ee}r (K KK!KKKKKKGKKKK'KK
KKKKKKKKKKKK Kur M@]r (]r KKr a]r (K/Kr K Kr e]r (KKr K Kr ee}r (K KK!KKKKKKGKKKK'KK
KKKKKKKKKKKK Kur MA]r (]r (KKr KKr e]r K Kr a]r KKr a]r KKr a]r (KKr KKr ee}r (KKKKKKKKKKK
KK KK KK
KKKKKKKKKKKKKKKKKKKKKKKKKK KK KK KK KK!KK"KK$KK'Kur MB]r (]r KnKr a]r (KK r KK r KK r KK r K Kr ee}r (K KK!KKKKKK'KKKK
KKKKKKKK Kur MC]r (]r (KKr KKr e]r (K%Kr K Kr e]r K Kr a]r KKr a]r KtKr a]r K.Kr ae}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur MD]r (]r K.Kr a]r (K/Kr K Kr
e]r (K.Kr K Kr
ee}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur ME]r (]r K.Kr a]r (K/K r K Kr ee}r (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur MF]r (]r K.Kr a]r (K1Kr K/Kr K Kr e]r K Kr a]r! (K.Kr" KlKr# K Kr$ e]r% (K/Kr& K Kr' ee}r( (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur) MG]r* (]r+ KNKr, a]r- (K/Kr. K Kr/ e]r0 KNKr1 a]r2 (K/Kr3 K Kr4 e]r5 (KNKr6 K Kr7 ee}r8 (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur9 MH]r: (]r; (K.Kr< KlKr= e]r> (K/Kr? K Kr@ e]rA (K.KrB KlKrC K KrD ee}rE (K KK!KKKKKK'KKKK
KKKKKKKKKKKKKK KurF MI]rG (]rH (KKrI K
KrJ e]rK K KrL a]rM KKrN a]rO K7KrP ae}rQ (K
KK KurR MJ]rS (]rT KKrU a]rV (K/KrW K KrX e]rY (KKrZ K Kr[ ee}r\ (K
KK Kur] MK]r^ (]r_ K Kr` a]ra (KGKrb K Krc e]rd K.Kre a]rf K Krg ae}rh K Ksri ML]rj (]rk (K
Krl KKrm K!Krn e]ro (K7Krp KIKrq e]rr K Krs a]rt KKru a]rv K Krw a]rx K7Kry a]rz K6Kr{ ae}r| (K!KK
KKKur} MM]r~ (]r K Kr a]r KGKr a]r KHKr a]r (KKr KKr e]r KGKr a]r KGKr a]r KHKr a]r KHK r a]r (KKr KtK
r KKr K Kr e]r K K r a]r KGK r a]r KHK r a]r (KKr K K r ee}r K Ksr MN]r (]r (KKr KKr K-Kr e]r (KKr K/Kr K Kr e]r (K0Kr K/Kr K Kr e]r KKr a]r (K/Kr K Kr e]r (KK r K-Kr e]r K.K
r a]r (KKr KKr K-Kr K Kr e]r K Kr a]r (K0K r K/Kr K K r e]r (K/Kr K K
r e]r K.Kr ae}r (KKK-KK
KK Kur MO]r (]r (KKr K-Kr KKr e]r (KKr K/Kr K Kr e]r KKr a]r (K0Kr K/Kr K Kr e]r (K/Kr K Kr e]r (KK r K-Kr e]r K Kr a]r K.K
r a]r (KKr K-Kr KKr K Kr e]r (K0K r K/Kr K K r e]r (K/Kr K K
r e]r K.Kr ae}r (KKK-KK
KK Kur MP]r (]r (K
Kr KKr e]r KKr a]r K Kr a]r K7Kr ae}r (K
KK Kur MQ]r (]r KKr a]r (K/Kr K Kr e]r (KKr K Kr ee}r (K
KK Kur MR]r (]r
K Kr a]r K Kr
ae}r K Ksr MS]r (]r K Kr a]r K.Kr a]r KGKr a]r KHKr a]r (KtKr K Kr e]r KGKr a]r KHKr a]r K Kr! ae}r" K Ksr# MT]r$ (]r% K.Kr& a]r' (KdKr( K Kr) e]r* KWKr+ a]r, K Kr- ae}r. (K KK!KKKKKK'KKKK
KKKKKKKKKKKK Kur/ MU]r0 (]r1 K&Kr2 a]r3 KKr4 a]r5 (KGKr6 K/Kr7 e]r8 KHKr9 a]r: K Kr; ae}r< K&Ksr= MV]r> (]r? KdKr@ a]rA KWKrB a]rC K KrD ae}rE KdKsrF MW]rG (]rH KKrI a]rJ (KK rK K KrL ee}rM (K KK!KKKKKK'KKKK
KKKKKKKK KurN MX]rO (]rP K KrQ a]rR (KfKrS K KrT e]rU K KrV ae}rW K KsrX MY]rY (]rZ K9Kr[ a]r\ K Kr] ae}r^ K Ksr_ uUlabelsr` ]ra (K UEMPTYrb rc KNrd K Nre M>Nrf KUimportrg rh K Nri KNrj KUdefrk rl KNrm KUtryrn ro KUreturnrp rq KUassertrr rs KUyieldrt ru KNrv KNrw KUbreakrx ry KNrz K2Nr{ KUraiser| r} KNr~ KUnotr r KNr KUclassr r KUlambdar r KUcontinuer r KUprintr r KUnonlocalr r KUexecr r KUwhiler r KNr KUdelr r KUpassr r KNr K Nr KUglobalr r KUforr r KUfromr r KUifr r KUwithr r KNr M8Nr KNr M.Nr KUandr r MNr K$Nr MCNr K Nr KNr M Nr MBNr MNr KNr M-Nr K
Nr KNr MFNr MXNr MENr K.Nr K'Nr K)Nr K/Nr K*Nr K+Nr K%Nr K,Nr K1Nr K-Nr K&Nr K(Nr K Nr MANr MNr M Nr KUinr r MGNr M
Nr M0Nr M Nr K Nr KNr K Nr K Nr KUisr r K Nr KNr M Nr MNr MMNr M&Nr M#Nr M
Nr MUNr MSNr M$Nr MNr MNr MNr MNr KUasr r MNr MDNr KUexceptr r MWNr KNr MHNr MNr M=Nr M4Nr M!Nr M Nr MNr M6Nr M7Nr MYNr KUelser r M2Nr K6Nr KUelifr r M'Nr M(Nr MNr M*Nr M)Nr MONr MNr M/Nr M1Nr MNr KUorr r MNNr MNr MLNr K#Nr MNr K"Nr
M<Nr K
Nr M9Nr
MNr M"Nr MNr M Nr M3Nr M5Nr MNr M Nr M%Nr M+Nr M;Nr M?Nr KNr KNr K0Nr KNr KNr M,Nr MKNr MJNr! MINr" M@Nr# M Nr$ KUfinallyr% r& MPNr' MRNr( MQNr) MTNr* MNr+ K!Nr, eUstatesr- ]r. (h]hfhnhvhhhhhhhj j j( j0 jE jM j[ jb jl j j j j j j j j j j j j
j j+ j7 jB jZ jn jy j j j j j j j j j j j j j# j* j7 jS jh jr j{ j j j j j j j j j j j j j j* j: jG jS j^ jj j~ j j j j j j j$ j0 j> jG jO jY eUkeywordsr/ }r0 (j K+j Kwj KTj K"j Kdj K j K%j K$j| Kj K#j Kgj Kj% Kj Kjg Kjp K
j Kj Ktjr K j Kj K&j Kjx Kj KKjt K jn K j K j Kj K j Kjk Kj KuU
number2symbolr1 }r2 (M U
file_inputr3 MUand_exprr4 MUand_testr5 MUarglistr6 MUargumentr7 MU
arith_exprr8 MU assert_stmtr9 MUatomr: MU augassignr; M U
break_stmtr< M
Uclassdefr= M Ucomp_forr> M Ucomp_ifr? M
U comp_iterr@ MUcomp_oprA MU
comparisonrB MU
compound_stmtrC MU
continue_stmtrD MU decoratedrE MU decoratorrF MU
decoratorsrG MUdel_stmtrH MU dictsetmakerrI MUdotted_as_namerJ MUdotted_as_namesrK MU dotted_namerL MU
encoding_declrM MU
eval_inputrN M U
except_clauserO M U exec_stmtrP M UexprrQ M U expr_stmtrR M UexprlistrS M!UfactorrT M"U flow_stmtrU M#Ufor_stmtrV M$UfuncdefrW M%U global_stmtrX M&Uif_stmtrY M'Uimport_as_namerZ M(Uimport_as_namesr[ M)U import_fromr\ M*U import_namer] M+U import_stmtr^ M,Ulambdefr_ M-U listmakerr` M.Unot_testra M/U old_lambdefrb M0Uold_testrc M1Uor_testrd M2U
parametersre M3U pass_stmtrf M4Upowerrg M5U
print_stmtrh M6U
raise_stmtri M7U return_stmtrj M8U
shift_exprrk M9U simple_stmtrl M:U single_inputrm M;Usliceoprn M<U
small_stmtro M=U star_exprrp M>Ustmtrq M?U subscriptrr M@U
subscriptlistrs MAUsuitert MBUtermru MCUtestrv MDUtestlistrw MEU testlist1rx MFU
testlist_gexpry MGU
testlist_saferz MHUtestlist_star_exprr{ MIUtfpdefr| MJUtfplistr} MKUtnamer~ MLUtrailerr MMUtry_stmtr MNU
typedargslistr MOU varargslistr MPUvfpdefr MQUvfplistr MRUvnamer MSU
while_stmtr MTU with_itemr MUU with_stmtr MVUwith_varr MWUxor_exprr MXU
yield_exprr MYU
yield_stmtr uU
symbol2numberr }r (jZ M'j MMjo M<j; Mj\ M)jW M$jA Mje M2jT M!jz MGj4 Mjl M9js M@jw MDj= M
j9 MjV M#j_ M,jR M jF Mju MBjY M&jd M1j MUj> M jQ M j MLj` M-j6 MjU M"j? M jk M8jP M j~ MKjr M?jp M=jG MjC MjL Mjg M4jq M>j^ M+j MPjn M;jB Mjf M3j8 Mj: Mjm M:j@ M
jK Mjj M7jc M0j MWjD MjJ Mj} MJj MRjh M5jM Mjt MAj[ M(ji M6jb M/jS M j3 M jO M j| MIj MYj MQj7 Mja M.j MNj< M jN Mjy MFjv MCjX M%j MTj] M*jI Mj MXj5 MjE Mj MVjH Mj MSj MOjx MEj{ MHuu. pytree.pyc 0000666 00000072120 15050104230 0006565 0 ustar 00
Lc @ s d Z d Z d d k Z d d k Z d d k l Z d Z h a d Z d e f d YZ d e f d
YZ
d e f d YZ d
Z d e f d YZ
d e
f d YZ d e
f d YZ d e
f d YZ d e
f d YZ d Z d S( s
Python parse tree definitions.
This is a very concrete parse tree; we need to keep every token and
even the comments and whitespace between tokens.
There's also a pattern matching implementation here.
s# Guido van Rossum iN( t StringIOic C sl t pU d d k l } xB | i i D]- \ } } t | t j o | t | \ } } | | j o% | d j o d S| i i | d Sq( Wd S( s
The node immediately preceding the invocant in their parent's children
list. If the invocant does not have a previous sibling, it is None.
i i N( R& R' R< R, ( R R= R@ ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt prev_sibling s
c C s" | i } | d j o d S| i S( s
Return the string immediately following the invocant node. This is
effectively equivalent to node.next_sibling.prefix
u N( RA R' R# ( R t next_sib( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt
get_suffix s
i i c C s t | i d S( Nt ascii( t unicodet encode( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt __str__ s N( ( i i ( t __name__t
__module__t __doc__R' R R& R, R+ R; R R t __hash__R R R R R R$ R% R6 R: R0 R> t propertyRA RB RD t syst version_infoRH ( ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR
" s0
t Nodec B s e Z d Z d d d Z d Z d Z e i d j o
e Z n d Z
d Z d Z d Z
d
Z d Z e e e Z d Z d
Z d Z RS( s+ Concrete implementation for interior nodes.c C s | d j p
t | | | _ t | | _ x: | i D]/ } | i d j p t t | | | _ q9 W| d j o
| | _ n d S( s
Initializer.
Takes a type constant (a symbol number >= 256), a sequence of
child nodes, and an optional context keyword argument.
As a side effect, the parent pointers of the children are updated.
i N( R R R* R, R& R' t reprR# ( R R R, t contextR# R4 ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt __init__ s
c C s# d | i i t | i | i f S( s) Return a canonical string representation.s
%s(%s, %r)( R RI R R R, ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt __repr__ s c C s d i t t | i S( sk
Return a pretty string representation.
This reproduces the input source exactly.
u ( t joint mapRF R, ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt __unicode__ s i i c C s" | i | i f | i | i f j S( s Compare two nodes for equality.( R R, ( R R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s c C s4 t | i g } | i D] } | | i q ~ S( s$ Return a cloned (deep) copy of self.( RP R R, R ( R t _[1]R4 ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s c c s9 x- | i D]" } x | i D] } | Vq Wq
W| Vd S( s* Return a post-order iterator for the tree.N( R, R ( R R@ R9 ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s
c c s9 | Vx- | i D]" } x | i D] } | Vq" Wq Wd S( s) Return a pre-order iterator for the tree.N( R, R ( R R@ R9 ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s
c C s | i p d S| i d i S( sO
The whitespace and comments preceding this node in the input.
t i ( R, R# ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt _prefix_getter$ s
c C s" | i o | | i d _ n d S( Ni ( R, R# ( R R# ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt _prefix_setter, s
c C s4 | | _ d | i | _ | | i | <| i d S( s
Equivalent to 'node.children[i] = child'. This method also sets the
child's parent attribute appropriately.
N( R& R' R, R0 ( R R= R@ ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt set_child2 s
c C s* | | _ | i i | | | i d S( s
Equivalent to 'node.children.insert(i, child)'. This method also sets
the child's parent attribute appropriately.
N( R& R, t insertR0 ( R R= R@ ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt insert_child< s c C s' | | _ | i i | | i d S( s
Equivalent to 'node.children.append(child)'. This method also sets the
child's parent attribute appropriately.
N( R& R, R/ R0 ( R R@ ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt append_childE s N( i i ( RI RJ RK R' RS RT RW RN RO RH R R R R RZ R[ RM R# R\ R^ R_ ( ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRP s
R7 c B s e Z d Z d Z d Z d Z d
d
d Z d Z d Z e
i d j o
e Z n d Z
d Z d Z d
Z d Z d Z e e e Z RS( s' Concrete implementation for leaf nodes.RY i c C s d | j o
d j n p
t | | d j o | \ | _ \ | _ | _ n | | _ | | _ | d j o
| | _ n d S( s
Initializer.
Takes a type constant (a token number < 256), a string value, and an
optional context keyword argument.
i i N( R R' t _prefixR8 t columnR t value( R R Rb RR R# ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRS X s (
c C s d | i i | i | i f S( s) Return a canonical string representation.s
%s(%r, %r)( R RI R Rb ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRT g s c C s | i t | i S( sk
Return a pretty string representation.
This reproduces the input source exactly.
( R# RF Rb ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRW m s i c C s" | i | i f | i | i f j S( s Compare two nodes for equality.( R Rb ( R R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR x s c C s+ t | i | i | i | i | i f f S( s$ Return a cloned (deep) copy of self.( R7 R Rb R# R8 Ra ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR | s c c s | Vd S( s* Return a post-order iterator for the tree.N( ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s c c s | Vd S( s) Return a pre-order iterator for the tree.N( ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s c C s | i S( sP
The whitespace and comments preceding this token in the input.
( R` ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRZ s c C s | i | | _ d S( N( R0 R` ( R R# ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR[ s
N( i i ( RI RJ RK R` R8 Ra R' RS RT RW RN RO RH R R R R RZ R[ RM R# ( ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR7 O s
c C sp | \ } } } } | p | | i j o0 t | d j o | d St | | d | St | | d | Sd S( s
Convert raw node information to a Node or Leaf instance.
This is passed to the parser driver which calls it whenever a reduction of a
grammar rule produces a new complete node, so that the tree is build
strictly bottom-up.
i i RR N( t
number2symbolt lenRP R7 ( t grt raw_nodeR Rb RR R, ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt convert s t BasePatternc B s\ e Z d Z d Z d Z d Z d Z d Z d Z d d Z
d d Z d Z RS( s
A pattern is a tree matching pattern.
It looks for a specific node type (token or symbol), and
optionally for a specific content.
This is an abstract base class. There are three concrete
subclasses:
- LeafPattern matches a single leaf node;
- NodePattern matches a single node (usually non-leaf);
- WildcardPattern matches a sequence of nodes of variable length.
c O s$ | t j p
t d t i | S( s> Constructor that prevents BasePattern from being instantiated.s Cannot instantiate BasePattern( Rh R R R ( R R R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s c C sk t | i | i | i g } x$ | o | d d j o | d =q! Wd | i i d i t t | f S( Nis %s(%s)s , (
R R t contentR
R' R RI RU RV RQ ( R R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRT s
c C s | S( s
A subclass can define this as a hook for optimizations.
Returns either self or another node with the same effect.
( ( R ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt optimize s c C s | i d j o | i | i j o t S| i d j oQ d } | d j o
h } n | i | | p t S| o | i | q n | d j o | i o | | | i = 256). If the
type is None this matches *any* single node (leaf or not),
except if content is not None, in which it only matches
non-leaf nodes that also match the content pattern.
The content, if not None, must be a sequence of Patterns that
must match the node's children exactly. If the content is
given, the type must not be None.
If a name is given, the matching node is stored in the results
dict under that key.
i N( R' R R) Rt RQ R* R< Rh t WildcardPatternR. t wildcardsR Ri R
( R R Ri R
R= t item( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRS . s
!
c C s | i od x\ t | i | i D]E \ } } | t | i j o# | d j o | i | n t Sq Wt St | i t | i j o t Sx; t | i | i D]$ \ } } | i
| | p t Sq Wt S( s
Match the pattern's content to the node's children.
This assumes the node type matches and self.content is not None.
Returns True if it matches, False if not.
If results is not None, it must be a dict which will be
updated with the nodes matching named subpatterns.
When returning False, the results dict may still be updated.
N( Rw Rr Ri R, Rd R' Rl R. R+ t zipRo ( R R9 Rm t cRn t
subpatternR@ ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRk K s
N( RI RJ R+ Rw R' RS Rk ( ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRu * s Rv c B sh e Z d Z d
d e d
d Z d Z d
d Z d
d Z d Z d Z
d Z d Z RS( s
A wildcard pattern can match zero or more nodes.
This has all the flexibility needed to implement patterns like:
.* .+ .? .{m,n}
(a b c | d e | f)
(...)* (...)+ (...)? (...){m,n}
except it always uses non-greedy matching.
i c C s d | j o | j o
t j n p t | | f | d j od t t t | } t | p t t | x/ | D]# } t | p t t | q Wn | | _ | | _ | | _ | | _
d S( s
Initializer.
Args:
content: optional sequence of subsequences of patterns;
if absent, matches one node;
if present, each subsequence is an alternative [*]
min: optinal minumum number of times to match, default 0
max: optional maximum number of times tro match, default HUGE
name: optional name assigned to this match
[*] Thus, if content is [[a, b, c], [d, e], [f, g, h]] this is
equivalent to (a b c | d e | f g h); if content is None,
this is equivalent to '.' in regular expression terms.
The min and max parameters work as follows:
min=0, max=maxint: .*
min=1, max=maxint: .+
min=0, max=1: .?
min=1, max=1: .
If content is not None, replace the dot with the parenthesized
list of alternatives, e.g. (a b c | d e | f g h)*
i N( t HUGER R' t tupleRV Rd RQ Ri t mint maxR
( R Ri R~ R R
t alt( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRS u s :
% c C s@ d } | i d j oE t | i d j o/ t | i d d j o | i d d } n | i d j o` | i d j oP | i d j o t d | i S| d j o | i | i j o | i Sn | i d j oa t | t oQ | i d j oA | i | i j o. t | i | i | i | i | i | i S| S( s+ Optimize certain stacked wildcard patterns.i i R
N(
R' Ri Rd R~ R Ru R
Rj R) Rv ( R R{ ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRj s 0 #
c C s | i | g | S( s' Does this pattern exactly match a node?( Rq ( R R9 Rm ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRo s c C s{ xt | i | D]c \ } } | t | j oD | d j o2 | i | | i o t | | | i | d j o$ t | t p t t | n | | _ d S( s
Initializer.
The argument is either a pattern or None. If it is None, this
only matches an empty sequence (effectively '$' in regex
lingo). If it is not None, this matches whenever the argument
pattern doesn't have any matches.
N( R' R) Rh R RQ Ri ( R Ri ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRS " s
$c C s t S( N( R+ ( R R9 ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRo / s c C s t | d j S( Ni ( Rd ( R Rp ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRq 3 s c c si | i d j o& t | d j o d h f Vqe n0 x! | i i | D]
\ } } d SWd h f Vd S( Ni ( Ri R' Rd Rr ( R Rp Rz Rn ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRr 7 s N( RI RJ R' RS Ro Rq Rr ( ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyR s
c c s | p d h f Vn | d | d } } x | i | D]u \ } } | p | | f Vq; xP t | | | D]; \ } } h } | i | | i | | | | f Vqq Wq; Wd S( sR
Generator yielding matches for a sequence of patterns and nodes.
Args:
patterns: a sequence of patterns
nodes: a sequence of nodes
Yields:
(count, results) tuples where:
count: the entire sequence of patterns matches nodes[:count];
results: dict containing named submatches.
i i N( Rr Rl ( t patternsRp t pt restR R R R Rn ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyRr C s
( RK t
__author__RN R R R| R R R R
RP R7 Rg Rh Rs Ru Rv R Rr ( ( ( s& /usr/lib64/python2.6/lib2to3/pytree.pyt s" hF V,=# refactor.py 0000666 00000057163 15050104230 0006731 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Refactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
"""
from __future__ import with_statement
__author__ = "Guido van Rossum "
# Python imports
import os
import sys
import logging
import operator
import collections
import StringIO
from itertools import chain
# Local imports
from .pgen2 import driver, tokenize, token
from . import pytree, pygram
def get_all_fix_names(fixer_pkg, remove_prefix=True):
"""Return a sorted list of all available fix names in the given package."""
pkg = __import__(fixer_pkg, [], [], ["*"])
fixer_dir = os.path.dirname(pkg.__file__)
fix_names = []
for name in sorted(os.listdir(fixer_dir)):
if name.startswith("fix_") and name.endswith(".py"):
if remove_prefix:
name = name[4:]
fix_names.append(name[:-3])
return fix_names
class _EveryNode(Exception):
pass
def _get_head_types(pat):
""" Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. """
if isinstance(pat, (pytree.NodePattern, pytree.LeafPattern)):
# NodePatters must either have no type and no content
# or a type and content -- so they don't get any farther
# Always return leafs
if pat.type is None:
raise _EveryNode
return set([pat.type])
if isinstance(pat, pytree.NegatedPattern):
if pat.content:
return _get_head_types(pat.content)
raise _EveryNode # Negated Patterns don't have a type
if isinstance(pat, pytree.WildcardPattern):
# Recurse on each node in content
r = set()
for p in pat.content:
for x in p:
r.update(_get_head_types(x))
return r
raise Exception("Oh no! I don't understand pattern %s" %(pat))
def _get_headnode_dict(fixer_list):
""" Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. """
head_nodes = collections.defaultdict(list)
every = []
for fixer in fixer_list:
if fixer.pattern:
try:
heads = _get_head_types(fixer.pattern)
except _EveryNode:
every.append(fixer)
else:
for node_type in heads:
head_nodes[node_type].append(fixer)
else:
if fixer._accept_type is not None:
head_nodes[fixer._accept_type].append(fixer)
else:
every.append(fixer)
for node_type in chain(pygram.python_grammar.symbol2number.itervalues(),
pygram.python_grammar.tokens):
head_nodes[node_type].extend(every)
return dict(head_nodes)
def get_fixers_from_package(pkg_name):
"""
Return the fully qualified names for fixers in the package pkg_name.
"""
return [pkg_name + "." + fix_name
for fix_name in get_all_fix_names(pkg_name, False)]
def _identity(obj):
return obj
if sys.version_info < (3, 0):
import codecs
_open_with_encoding = codecs.open
# codecs.open doesn't translate newlines sadly.
def _from_system_newlines(input):
return input.replace(u"\r\n", u"\n")
def _to_system_newlines(input):
if os.linesep != "\n":
return input.replace(u"\n", os.linesep)
else:
return input
else:
_open_with_encoding = open
_from_system_newlines = _identity
_to_system_newlines = _identity
def _detect_future_features(source):
have_docstring = False
gen = tokenize.generate_tokens(StringIO.StringIO(source).readline)
def advance():
tok = gen.next()
return tok[0], tok[1]
ignore = frozenset((token.NEWLINE, tokenize.NL, token.COMMENT))
features = set()
try:
while True:
tp, value = advance()
if tp in ignore:
continue
elif tp == token.STRING:
if have_docstring:
break
have_docstring = True
elif tp == token.NAME and value == u"from":
tp, value = advance()
if tp != token.NAME or value != u"__future__":
break
tp, value = advance()
if tp != token.NAME or value != u"import":
break
tp, value = advance()
if tp == token.OP and value == u"(":
tp, value = advance()
while tp == token.NAME:
features.add(value)
tp, value = advance()
if tp != token.OP or value != u",":
break
tp, value = advance()
else:
break
except StopIteration:
pass
return frozenset(features)
class FixerError(Exception):
"""A fixer could not be loaded."""
class RefactoringTool(object):
_default_options = {"print_function" : False}
CLASS_PREFIX = "Fix" # The prefix for fixer classes
FILE_PREFIX = "fix_" # The prefix for modules with a fixer within
def __init__(self, fixer_names, options=None, explicit=None):
"""Initializer.
Args:
fixer_names: a list of fixers to import
options: an dict with configuration.
explicit: a list of fixers to run even if they are explicit.
"""
self.fixers = fixer_names
self.explicit = explicit or []
self.options = self._default_options.copy()
if options is not None:
self.options.update(options)
if self.options["print_function"]:
self.grammar = pygram.python_grammar_no_print_statement
else:
self.grammar = pygram.python_grammar
self.errors = []
self.logger = logging.getLogger("RefactoringTool")
self.fixer_log = []
self.wrote = False
self.driver = driver.Driver(self.grammar,
convert=pytree.convert,
logger=self.logger)
self.pre_order, self.post_order = self.get_fixers()
self.pre_order_heads = _get_headnode_dict(self.pre_order)
self.post_order_heads = _get_headnode_dict(self.post_order)
self.files = [] # List of files that were or should be modified
def get_fixers(self):
"""Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
"""
pre_order_fixers = []
post_order_fixers = []
for fix_mod_path in self.fixers:
mod = __import__(fix_mod_path, {}, {}, ["*"])
fix_name = fix_mod_path.rsplit(".", 1)[-1]
if fix_name.startswith(self.FILE_PREFIX):
fix_name = fix_name[len(self.FILE_PREFIX):]
parts = fix_name.split("_")
class_name = self.CLASS_PREFIX + "".join([p.title() for p in parts])
try:
fix_class = getattr(mod, class_name)
except AttributeError:
raise FixerError("Can't find %s.%s" % (fix_name, class_name))
fixer = fix_class(self.options, self.fixer_log)
if fixer.explicit and self.explicit is not True and \
fix_mod_path not in self.explicit:
self.log_message("Skipping implicit fixer: %s", fix_name)
continue
self.log_debug("Adding transformation: %s", fix_name)
if fixer.order == "pre":
pre_order_fixers.append(fixer)
elif fixer.order == "post":
post_order_fixers.append(fixer)
else:
raise FixerError("Illegal fixer order: %r" % fixer.order)
key_func = operator.attrgetter("run_order")
pre_order_fixers.sort(key=key_func)
post_order_fixers.sort(key=key_func)
return (pre_order_fixers, post_order_fixers)
def log_error(self, msg, *args, **kwds):
"""Called when an error occurs."""
raise
def log_message(self, msg, *args):
"""Hook to log a message."""
if args:
msg = msg % args
self.logger.info(msg)
def log_debug(self, msg, *args):
if args:
msg = msg % args
self.logger.debug(msg)
def print_output(self, old_text, new_text, filename, equal):
"""Called with the old version, new version, and filename of a
refactored file."""
pass
def refactor(self, items, write=False, doctests_only=False):
"""Refactor a list of files and directories."""
for dir_or_file in items:
if os.path.isdir(dir_or_file):
self.refactor_dir(dir_or_file, write, doctests_only)
else:
self.refactor_file(dir_or_file, write, doctests_only)
def refactor_dir(self, dir_name, write=False, doctests_only=False):
"""Descends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
"""
for dirpath, dirnames, filenames in os.walk(dir_name):
self.log_debug("Descending into %s", dirpath)
dirnames.sort()
filenames.sort()
for name in filenames:
if not name.startswith(".") and \
os.path.splitext(name)[1].endswith("py"):
fullname = os.path.join(dirpath, name)
self.refactor_file(fullname, write, doctests_only)
# Modify dirnames in-place to remove subdirs with leading dots
dirnames[:] = [dn for dn in dirnames if not dn.startswith(".")]
def _read_python_source(self, filename):
"""
Do our best to decode a Python source file correctly.
"""
try:
f = open(filename, "rb")
except IOError, err:
self.log_error("Can't open %s: %s", filename, err)
return None, None
try:
encoding = tokenize.detect_encoding(f.readline)[0]
finally:
f.close()
with _open_with_encoding(filename, "r", encoding=encoding) as f:
return _from_system_newlines(f.read()), encoding
def refactor_file(self, filename, write=False, doctests_only=False):
"""Refactors a file."""
input, encoding = self._read_python_source(filename)
if input is None:
# Reading the file failed.
return
input += u"\n" # Silence certain parse errors
if doctests_only:
self.log_debug("Refactoring doctests in %s", filename)
output = self.refactor_docstring(input, filename)
if output != input:
self.processed_file(output, filename, input, write, encoding)
else:
self.log_debug("No doctest changes in %s", filename)
else:
tree = self.refactor_string(input, filename)
if tree and tree.was_changed:
# The [:-1] is to take off the \n we added earlier
self.processed_file(unicode(tree)[:-1], filename,
write=write, encoding=encoding)
else:
self.log_debug("No changes in %s", filename)
def refactor_string(self, data, name):
"""Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
"""
features = _detect_future_features(data)
if "print_function" in features:
self.driver.grammar = pygram.python_grammar_no_print_statement
try:
tree = self.driver.parse_string(data)
except Exception, err:
self.log_error("Can't parse %s: %s: %s",
name, err.__class__.__name__, err)
return
finally:
self.driver.grammar = self.grammar
tree.future_features = features
self.log_debug("Refactoring %s", name)
self.refactor_tree(tree, name)
return tree
def refactor_stdin(self, doctests_only=False):
input = sys.stdin.read()
if doctests_only:
self.log_debug("Refactoring doctests in stdin")
output = self.refactor_docstring(input, "")
if output != input:
self.processed_file(output, "", input)
else:
self.log_debug("No doctest changes in stdin")
else:
tree = self.refactor_string(input, "")
if tree and tree.was_changed:
self.processed_file(unicode(tree), "", input)
else:
self.log_debug("No changes in stdin")
def refactor_tree(self, tree, name):
"""Refactors a parse tree (modifying the tree in place).
Args:
tree: a pytree.Node instance representing the root of the tree
to be refactored.
name: a human-readable name for this tree.
Returns:
True if the tree was modified, False otherwise.
"""
for fixer in chain(self.pre_order, self.post_order):
fixer.start_tree(tree, name)
self.traverse_by(self.pre_order_heads, tree.pre_order())
self.traverse_by(self.post_order_heads, tree.post_order())
for fixer in chain(self.pre_order, self.post_order):
fixer.finish_tree(tree, name)
return tree.was_changed
def traverse_by(self, fixers, traversal):
"""Traverse an AST, applying a set of fixers to each node.
This is a helper method for refactor_tree().
Args:
fixers: a list of fixer instances.
traversal: a generator that yields AST nodes.
Returns:
None
"""
if not fixers:
return
for node in traversal:
for fixer in fixers[node.type]:
results = fixer.match(node)
if results:
new = fixer.transform(node, results)
if new is not None:
node.replace(new)
node = new
def processed_file(self, new_text, filename, old_text=None, write=False,
encoding=None):
"""
Called when a file has been refactored, and there are changes.
"""
self.files.append(filename)
if old_text is None:
old_text = self._read_python_source(filename)[0]
if old_text is None:
return
equal = old_text == new_text
self.print_output(old_text, new_text, filename, equal)
if equal:
self.log_debug("No changes to %s", filename)
return
if write:
self.write_file(new_text, filename, old_text, encoding)
else:
self.log_debug("Not writing changes to %s", filename)
def write_file(self, new_text, filename, old_text, encoding=None):
"""Writes a string to a file.
It first shows a unified diff between the old text and the new text, and
then rewrites the file; the latter is only done if the write option is
set.
"""
try:
f = _open_with_encoding(filename, "w", encoding=encoding)
except os.error, err:
self.log_error("Can't create %s: %s", filename, err)
return
try:
f.write(_to_system_newlines(new_text))
except os.error, err:
self.log_error("Can't write %s: %s", filename, err)
finally:
f.close()
self.log_debug("Wrote changes to %s", filename)
self.wrote = True
PS1 = ">>> "
PS2 = "... "
def refactor_docstring(self, input, filename):
"""Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
"""
result = []
block = None
block_lineno = None
indent = None
lineno = 0
for line in input.splitlines(True):
lineno += 1
if line.lstrip().startswith(self.PS1):
if block is not None:
result.extend(self.refactor_doctest(block, block_lineno,
indent, filename))
block_lineno = lineno
block = [line]
i = line.find(self.PS1)
indent = line[:i]
elif (indent is not None and
(line.startswith(indent + self.PS2) or
line == indent + self.PS2.rstrip() + u"\n")):
block.append(line)
else:
if block is not None:
result.extend(self.refactor_doctest(block, block_lineno,
indent, filename))
block = None
indent = None
result.append(line)
if block is not None:
result.extend(self.refactor_doctest(block, block_lineno,
indent, filename))
return u"".join(result)
def refactor_doctest(self, block, lineno, indent, filename):
"""Refactors one doctest.
A doctest is given as a block of lines, the first of which starts
with ">>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
"""
try:
tree = self.parse_block(block, lineno, indent)
except Exception, err:
if self.log.isEnabledFor(logging.DEBUG):
for line in block:
self.log_debug("Source: %s", line.rstrip(u"\n"))
self.log_error("Can't parse docstring in %s line %s: %s: %s",
filename, lineno, err.__class__.__name__, err)
return block
if self.refactor_tree(tree, filename):
new = unicode(tree).splitlines(True)
# Undo the adjustment of the line numbers in wrap_toks() below.
clipped, new = new[:lineno-1], new[lineno-1:]
assert clipped == [u"\n"] * (lineno-1), clipped
if not new[-1].endswith(u"\n"):
new[-1] += u"\n"
block = [indent + self.PS1 + new.pop(0)]
if new:
block += [indent + self.PS2 + line for line in new]
return block
def summarize(self):
if self.wrote:
were = "were"
else:
were = "need to be"
if not self.files:
self.log_message("No files %s modified.", were)
else:
self.log_message("Files that %s modified:", were)
for file in self.files:
self.log_message(file)
if self.fixer_log:
self.log_message("Warnings/messages while refactoring:")
for message in self.fixer_log:
self.log_message(message)
if self.errors:
if len(self.errors) == 1:
self.log_message("There was 1 error:")
else:
self.log_message("There were %d errors:", len(self.errors))
for msg, args, kwds in self.errors:
self.log_message(msg, *args, **kwds)
def parse_block(self, block, lineno, indent):
"""Parses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
"""
tree = self.driver.parse_tokens(self.wrap_toks(block, lineno, indent))
tree.future_features = frozenset()
return tree
def wrap_toks(self, block, lineno, indent):
"""Wraps a tokenize stream to systematically modify start/end."""
tokens = tokenize.generate_tokens(self.gen_lines(block, indent).next)
for type, value, (line0, col0), (line1, col1), line_text in tokens:
line0 += lineno - 1
line1 += lineno - 1
# Don't bother updating the columns; this is too complicated
# since line_text would also have to be updated and it would
# still break for tokens spanning lines. Let the user guess
# that the column numbers for doctests are relative to the
# end of the prompt string (PS1 or PS2).
yield type, value, (line0, col0), (line1, col1), line_text
def gen_lines(self, block, indent):
"""Generates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
"""
prefix1 = indent + self.PS1
prefix2 = indent + self.PS2
prefix = prefix1
for line in block:
if line.startswith(prefix):
yield line[len(prefix):]
elif line == prefix.rstrip() + u"\n":
yield u"\n"
else:
raise AssertionError("line=%r, prefix=%r" % (line, prefix))
prefix = prefix2
while True:
yield ""
class MultiprocessingUnsupported(Exception):
pass
class MultiprocessRefactoringTool(RefactoringTool):
def __init__(self, *args, **kwargs):
super(MultiprocessRefactoringTool, self).__init__(*args, **kwargs)
self.queue = None
self.output_lock = None
def refactor(self, items, write=False, doctests_only=False,
num_processes=1):
if num_processes == 1:
return super(MultiprocessRefactoringTool, self).refactor(
items, write, doctests_only)
try:
import multiprocessing
except ImportError:
raise MultiprocessingUnsupported
if self.queue is not None:
raise RuntimeError("already doing multiple processes")
self.queue = multiprocessing.JoinableQueue()
self.output_lock = multiprocessing.Lock()
processes = [multiprocessing.Process(target=self._child)
for i in xrange(num_processes)]
try:
for p in processes:
p.start()
super(MultiprocessRefactoringTool, self).refactor(items, write,
doctests_only)
finally:
self.queue.join()
for i in xrange(num_processes):
self.queue.put(None)
for p in processes:
if p.is_alive():
p.join()
self.queue = None
def _child(self):
task = self.queue.get()
while task is not None:
args, kwargs = task
try:
super(MultiprocessRefactoringTool, self).refactor_file(
*args, **kwargs)
finally:
self.queue.task_done()
task = self.queue.get()
def refactor_file(self, *args, **kwargs):
if self.queue is not None:
self.queue.put((args, kwargs))
else:
return super(MultiprocessRefactoringTool, self).refactor_file(
*args, **kwargs)
main.pyo 0000666 00000014671 15050104230 0006224 0 ustar 00
Lc @ s d Z d d k l Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k l
Z
d Z d e
i f d YZ
d Z d d
Z d S( s
Main program for 2to3.
i( t with_statementNi ( t refactorc C s: | i } | i } t i | | | | d d d d S( s% Return a unified diff of two strings.s
(original)s (refactored)t linetermt ( t
splitlinest difflibt unified_diff( t at bt filename( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt
diff_texts s
t StdoutRefactoringToolc B s2 e Z d Z d Z d Z d Z d Z RS( s"
Prints output to stdout.
c C s2 | | _ | | _ t t | i | | | d S( N( t nobackupst
show_diffst superR t __init__( t selft fixerst optionst explicitR R
( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR s c O s3 | i i | | | f | i i | | | d S( N( t errorst appendt loggert error( R t msgt argst kwargs( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt log_error$ s c C s | i p | d } t i i | o@ y t i | Wqg t i j
o } | i d | qg Xn y t i | | Wq t i j
o } | i d | | q Xn t t | i
} | | | | | | i p t i | | n d S( Ns .baks Can't remove backup %ss Can't rename %s to %s(
R t ost patht lexistst removeR t log_messaget renameR R t
write_filet shutilt copymode( R t new_textR t old_textt encodingt backupt errt write( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR" ( s
c
C s | o | i d | n | i d | | i o t | | | } yl | i d j oB | i i i z' x | D] } | GHqv Wt i i Wd QXn x | D] } | GHq WWq t
j
o t d | f d SXn d S( Ns No changes to %ss
Refactored %ss+ couldn't encode %s's diff for your terminal( R R
R
t output_lockt Nonet __exit__t __enter__t syst stdoutt flusht UnicodeEncodeErrort warn( R t oldt newR t equalt
diff_linest line( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt print_output; s&
( t __name__t
__module__t __doc__R R R" R9 ( ( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR s
c C s t i d | f IJd S( Ns WARNING: %s( R/ t stderr( R ( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR3 Q s c
s t i d d } | i d d d d d d | i d d
d d d g d d
| i d d d d d d d d d d | i d d d d d g d d | i d d d d d d | i d d d d d d | i d d d d d d | i d! d d d d" | i d# d$ d d d d% | i d& d' d d d t d d( t } h } | i | \ } } | i o | i o t d) n | i o | i o | i d* n | i
o4 d+ GHx t i D] } | GHqW| p d, Sn | p t
i d- IJt
i d. IJd/ Sd0 | j o& t } | i o t
i d1 IJd/ Sn | i o t | d2 s t alls .fix_s+ Sorry, -j isn't supported on this platform.(' t optparset OptionParsert
add_optiont Falset
parse_argsR* t no_diffsR3 R R t
list_fixesR t get_all_fix_namesR/ R= t TrueRG t verboset loggingt DEBUGt INFOt basicConfigt sett get_fixers_from_packaget nofixRK t addt uniont
differenceR t sortedR t refactor_stdint
doctests_onlyt processest MultiprocessingUnsupportedt summarizeRE t bool( RL R t parserRc t flagsR t fixnameRI t avail_fixest unwanted_fixesR t all_presentRK t requestedt fixer_namest rt( ( RL s$ /usr/lib64/python2.6/lib2to3/main.pyt mainU s
!
( R< t
__future__R R/ R R RX R# RN R R R
t MultiprocessRefactoringToolR R3 R, Rr ( ( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt s 7 patcomp.pyo 0000666 00000014412 15050104230 0006734 0 ustar 00
Lc @ s d Z d Z d d k Z d d k l Z l Z l Z l Z l Z l Z d d k
l Z d d k
l Z e i
i e i
i e d Z d e f d
YZ d Z d e f d
YZ h e i d 6e i d 6e i d 6d d 6Z d Z d Z d Z d S( s Pattern compiler.
The grammer is taken from PatternGrammar.txt.
The compiler compiles a pattern to a pytree.*Pattern instance.
s# Guido van Rossum iNi ( t drivert literalst tokent tokenizet parset grammar( t pytree( t pygrams PatternGrammar.txtt PatternSyntaxErrorc B s e Z RS( ( t __name__t
__module__( ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR s c c sy t t i t i t i f } t i t i | i } x9 | D]1 } | \ } } } } } | | j o | Vq@ q@ Wd S( s6 Tokenizes a string suppressing significant whitespace.N(
t setR t NEWLINEt INDENTt DEDENTR t generate_tokensR t generate_linest next( t inputt skipt tokenst quintuplet typet valuet startt endt line_text( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt tokenize_wrapper s
t PatternCompilerc B s> e Z e d Z e d Z d Z d d Z d Z RS( c C s^ t i | | _ t i | i | _ t i | _ t i | _ t i
| i d t | _ d S( s^ Initializer.
Takes an optional alternative filename for the pattern grammar.
t convertN( R t load_grammarR R t Symbolst symst python_grammart pygrammart python_symbolst pysymst Drivert pattern_convert( t selft grammar_file( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt __init__, s
c C sb t | } y | i i | d | } Wn* t i j
o } t t | n X| i | S( s= Compiles a pattern string to a nested pytree.*Pattern object.t debug( R R t parse_tokensR t
ParseErrorR t strt compile_node( R' R R* R t roott e( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt compile_pattern7 s c C s | i | i i j o | i d } n | i | i i j o g } | i d d d D] } | | i | qX ~ } t | d j o | d St i g } | D] } | | g q ~ d d d d } | i S| i | i i
j or g } | i D] } | | i | q ~ } t | d j o | d St i | g d d d d } | i S| i | i i j o0 | i | i d }
t i
|
} | i Sd } | i } t | d j o2 | d i t i j o | d i } | d } n d }
t | d j o2 | d i | i i j o | d }
| d } n | i | |
}
|
d j o|
i } | d } | i t i j o d } t i } n | i t i j o d } t i } nY | i t i j oE | i | d } } t | d j o | i | d } qn | d j p
| d j o1 |
i }
t i |
g g d | d | }
qin | d j o
| |
_ n |
i S(
sX Compiles a node, recursively.
This is one big switch on the node type.
i Ni i t mint maxi ii ( R R t Matchert childrent AlternativesR. t lenR t WildcardPatternt optimizet Alternativet NegatedUnitt
compile_basict NegatedPatternt NoneR t EQUALR t Repeatert STARt HUGEt PLUSt LBRACEt get_intt name( R' t nodet _[1]t cht altst _[2]t at pt _[3]t unitst patternRF t nodest repeatR5 t childR2 R3 ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR. @ s^ : 9 - *
-
)
c C s | d } | i t i j o/ t t i | i } t i t | | S| i t i
j o | i } | i oN | t j o t
d | n | d o t
d n t i t | S| d j o
d } nK | i d p: t | i | d } | d j o t
d | q$n | d o! | i | d i d g } n d } t i | | Snf | i d j o | i | d S| i d j o3 | i | d } t i | g g d
d d d Sd S( Ni s Invalid token: %ri s Can't have details for tokent anyt _s Invalid symbol: %rt (t [R2 R3 ( R R t STRINGt unicodeR t
evalStringR R t LeafPatternt _type_of_literalt NAMEt isuppert TOKEN_MAPR R> t
startswitht getattrR$ R. R5 t NodePatternR8 ( R' RQ RR RG R R t contentt
subpattern( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR< s8
! c C s
t | i S( N( t intR ( R' RG ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyRE s N(
R R
t _PATTERN_GRAMMAR_FILER) t FalseR1 R. R> R< RE ( ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR * s
G #R] RX t NUMBERt TOKENc C s= | d i o t i S| t i j o t i | Sd Sd S( Ni ( t isalphaR R] R t opmapR> ( R ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR\ s
c C sZ | \ } } } } | p | | i j o t i | | d | St i | | d | Sd S( s9 Converts raw node information to a Node or Leaf instance.t contextN( t
number2symbolR t Nodet Leaf( R t
raw_node_infoR R Rl R5 ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR& s c C s t i | S( N( R R1 ( RP ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR1 s ( t __doc__t
__author__t ost pgen2R R R R R R t R R t patht joint dirnamet __file__Rf t ExceptionR R t objectR R] RX Rh R> R_ R\ R& R1 ( ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt s" .
PatternGrammar2.6.6.final.0.pickle 0000666 00000002547 15050104230 0012503 0 ustar 00 }q (Utokensq}q(K KKKKKKKKKKKK KK
KK KKKKKKK KK KK
KKKKKKuU symbol2labelq}q(URepeaterqKU AlternativesqKUDetailsqKU AlternativeqK
UUnitq KU NegatedUnitq
K uUstartq M Udfasq }q
(M ]q(]qKKqa]qKKqa]qK Kqae}q(KKKKKKKKKKuqM]q(]q(KKqK Kqe]q(KKq K Kq K Kq ee}q (KKKKKKKKKKuq M]q!(]q"K
Kq#a]q$(K K q%K Kq&ee}q'(KKKKKKKKKKuq(M]q)(]q*K Kq+a]q,KKq-a]q.K
Kq/a]q0K Kq1ae}q2K Ksq3M]q4(]q5KKq6a]q7(KKq8KKq9KKq:e]q;KKqK Kq?e]q@K KqAa]qBKKqCae}qDKKsqEM]qF(]qG(KKqHKKqIKKqJe]qKK KqLa]qMKKqNa]qO(KKqPKKqQe]qRKKqSa]qTKKqUae}qV(KKKKKKuqWM]qX(]qY(KKqZKKq[KKq\KKq]e]q^KKq_a]q`(KKqaKKqbKKqcK Kqde]qeKKqfa]qg(KKqhK Kqie]qjKKqka]ql(KKqmKK qnKKqoKKqpe]qqK Kqra]qsKKqta]qu(KKqvKKqwK K qxee}qy(KKKKKKKKuqzuUlabelsq{]q|(K UEMPTYq}q~MNqK NqKUnotqqKNqKNqK NqKNqMNqMNqMNqKNqKNqKNqMNqKNqKNqKNqKNqKNqKNqK NqMNqKNqK
NqeUstatesq]q(hhh!h)h4hFhXeUkeywordsq}qhKsU
number2symbolq}q(M UMatcherqMU AlternativeqMU AlternativesqMUDetailsqMU NegatedUnitqMURepeaterqMUUnitquU
symbol2numberq}q(hMhM hMhMhMhMhMuu. main.py 0000666 00000015324 15050104230 0006041 0 ustar 00 """
Main program for 2to3.
"""
from __future__ import with_statement
import sys
import os
import difflib
import logging
import shutil
import optparse
from . import refactor
def diff_texts(a, b, filename):
"""Return a unified diff of two strings."""
a = a.splitlines()
b = b.splitlines()
return difflib.unified_diff(a, b, filename, filename,
"(original)", "(refactored)",
lineterm="")
class StdoutRefactoringTool(refactor.MultiprocessRefactoringTool):
"""
Prints output to stdout.
"""
def __init__(self, fixers, options, explicit, nobackups, show_diffs):
self.nobackups = nobackups
self.show_diffs = show_diffs
super(StdoutRefactoringTool, self).__init__(fixers, options, explicit)
def log_error(self, msg, *args, **kwargs):
self.errors.append((msg, args, kwargs))
self.logger.error(msg, *args, **kwargs)
def write_file(self, new_text, filename, old_text, encoding):
if not self.nobackups:
# Make backup
backup = filename + ".bak"
if os.path.lexists(backup):
try:
os.remove(backup)
except os.error, err:
self.log_message("Can't remove backup %s", backup)
try:
os.rename(filename, backup)
except os.error, err:
self.log_message("Can't rename %s to %s", filename, backup)
# Actually write the new file
write = super(StdoutRefactoringTool, self).write_file
write(new_text, filename, old_text, encoding)
if not self.nobackups:
shutil.copymode(backup, filename)
def print_output(self, old, new, filename, equal):
if equal:
self.log_message("No changes to %s", filename)
else:
self.log_message("Refactored %s", filename)
if self.show_diffs:
diff_lines = diff_texts(old, new, filename)
try:
if self.output_lock is not None:
with self.output_lock:
for line in diff_lines:
print line
sys.stdout.flush()
else:
for line in diff_lines:
print line
except UnicodeEncodeError:
warn("couldn't encode %s's diff for your terminal" %
(filename,))
return
def warn(msg):
print >> sys.stderr, "WARNING: %s" % (msg,)
def main(fixer_pkg, args=None):
"""Main program.
Args:
fixer_pkg: the name of a package where the fixers are located.
args: optional; a list of command line arguments. If omitted,
sys.argv[1:] is used.
Returns a suggested exit status (0, 1, 2).
"""
# Set up option parser
parser = optparse.OptionParser(usage="2to3 [options] file|dir ...")
parser.add_option("-d", "--doctests_only", action="store_true",
help="Fix up doctests only")
parser.add_option("-f", "--fix", action="append", default=[],
help="Each FIX specifies a transformation; default: all")
parser.add_option("-j", "--processes", action="store", default=1,
type="int", help="Run 2to3 concurrently")
parser.add_option("-x", "--nofix", action="append", default=[],
help="Prevent a fixer from being run.")
parser.add_option("-l", "--list-fixes", action="store_true",
help="List available transformations")
parser.add_option("-p", "--print-function", action="store_true",
help="Modify the grammar so that print() is a function")
parser.add_option("-v", "--verbose", action="store_true",
help="More verbose logging")
parser.add_option("--no-diffs", action="store_true",
help="Don't show diffs of the refactoring")
parser.add_option("-w", "--write", action="store_true",
help="Write back modified files")
parser.add_option("-n", "--nobackups", action="store_true", default=False,
help="Don't write backups for modified files.")
# Parse command line arguments
refactor_stdin = False
flags = {}
options, args = parser.parse_args(args)
if not options.write and options.no_diffs:
warn("not writing files and not printing diffs; that's not very useful")
if not options.write and options.nobackups:
parser.error("Can't use -n without -w")
if options.list_fixes:
print "Available transformations for the -f/--fix option:"
for fixname in refactor.get_all_fix_names(fixer_pkg):
print fixname
if not args:
return 0
if not args:
print >> sys.stderr, "At least one file or directory argument required."
print >> sys.stderr, "Use --help to show usage."
return 2
if "-" in args:
refactor_stdin = True
if options.write:
print >> sys.stderr, "Can't write to stdin."
return 2
if options.print_function:
flags["print_function"] = True
# Set up logging handler
level = logging.DEBUG if options.verbose else logging.INFO
logging.basicConfig(format='%(name)s: %(message)s', level=level)
# Initialize the refactoring tool
avail_fixes = set(refactor.get_fixers_from_package(fixer_pkg))
unwanted_fixes = set(fixer_pkg + ".fix_" + fix for fix in options.nofix)
explicit = set()
if options.fix:
all_present = False
for fix in options.fix:
if fix == "all":
all_present = True
else:
explicit.add(fixer_pkg + ".fix_" + fix)
requested = avail_fixes.union(explicit) if all_present else explicit
else:
requested = avail_fixes.union(explicit)
fixer_names = requested.difference(unwanted_fixes)
rt = StdoutRefactoringTool(sorted(fixer_names), flags, sorted(explicit),
options.nobackups, not options.no_diffs)
# Refactor all files and directories passed as arguments
if not rt.errors:
if refactor_stdin:
rt.refactor_stdin()
else:
try:
rt.refactor(args, options.write, options.doctests_only,
options.processes)
except refactor.MultiprocessingUnsupported:
assert options.processes > 1
print >> sys.stderr, "Sorry, -j isn't " \
"supported on this platform."
return 1
rt.summarize()
# Return error status (0 if rt.errors is zero)
return int(bool(rt.errors))
fixer_util.pyo 0000666 00000033711 15050104230 0007446 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z l Z d d k l Z d d k l
Z
d Z d Z d Z
d Z d1 d
Z d Z d Z d
Z e e
d Z d1 d1 d Z d Z d Z d1 d Z d Z d1 d Z d1 d Z d Z d Z d Z d Z e d d d d d d d d! d" g Z! d# Z" d$ a# d% a$ d& a% e& a' d' Z( d( Z) d) Z* d* Z+ d+ Z, d, Z- d- Z. e e i/ e i0 g Z1 d1 d. Z2 e e i0 e i/ e i3 g Z4 d/ Z5 d1 d0 Z6 d1 S(2 s1 Utility functions, node construction macros, etc.i ( t token( t Leaft Node( t python_symbols( t patcompc C s% t t i | t t i d | g S( Nu =( R t symst argumentR R t EQUAL( t keywordt value( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt
KeywordArg s c C s t t i d S( Nu (( R R t LPAR( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt LParen s c C s t t i d S( Nu )( R R t RPAR( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt RParen s c C sp t | t p
| g } n t | t p d | _ | g } n t t i | t t i d d d g | S( s Build an assignment statementu u =t prefix( t
isinstancet listR R R t atomR R R ( t targett source( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt Assign s
c C s t t i | d | S( s Return a NAME leafR ( R R t NAME( t nameR ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt Name$ s c C s | t t i t | g g S( s A node tuple for obj.attr( R R t trailert Dot( t objt attr( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt Attr( s c C s t t i d S( s A comma leafu ,( R R t COMMA( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt Comma, s c C s t t i d S( s A period (.) leafu .( R R t DOT( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyR 0 s c C sO t t i | i | i g } | o | i d t t i | n | S( s- A parenthesised argument list, used by Call()i ( R R R t clonet insert_childt arglist( t argst lparent rparent node( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt ArgList4 s $ c C s< t t i | t | g } | d j o
| | _ n | S( s A function callN( R R t powerR( t NoneR ( t func_nameR$ R R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt Call; s
c C s t t i d S( s A newline literalu
( R R t NEWLINE( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt NewlineB s c C s t t i d S( s A blank lineu ( R R R- ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt BlankLineF s c C s t t i | d | S( NR ( R R t NUMBER( t nR ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt NumberJ s c C s1 t t i t t i d | t t i d g S( s A numeric or string subscriptu [u ]( R R R R R t LBRACEt RBRACE( t
index_node( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt SubscriptM s c C s t t i | d | S( s
A string leafR ( R R t STRING( t stringR ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt StringS s c C s d | _ d | _ d | _ t t i d } d | _ t t i d } d | _ | | | | g } | oG d | _ t t i d } d | _ | i t t i | | g n t t i | t t i | g } t t i
t t i d | t t i d g S( su A list comprehension of the form [xp for fp in it if test].
If test is None, the "if test" part is omitted.
u u u foru inu ifu [u ](
R R R R t appendR R t comp_ift listmakert comp_forR R3 R4 ( t xpt fpt itt testt for_leaft in_leaft
inner_argst if_leaft inner( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt ListCompW s$ #$ c C s x | D] } | i q Wt t i d t t i | d d t t i d d d t t i | g } t t i | } | S( sO Return an import statement in the form:
from package import name_leafsu fromR u u import( t removeR R R R R t import_as_namest import_from( t package_namet
name_leafst leaft childrent imp( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt
FromImporto s c C s t | t o! | i t t g j o t St | t o t | i d j op t | i d t oY t | i d t oB t | i d t o+ | i d i d j o | i d i d j S( s( Does the node represent a tuple literal?i i i i u (u )( R R RN R R t Truet lenR R ( R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt is_tuple s ,c C s t | t oo t | i d j oY t | i d t oB t | i d t o+ | i d i d j o | i d i d j S( s' Does the node represent a list literal?i i iu [u ]( R R RR RN R R ( R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt is_list s c C s t t i t | t g S( N( R R R R R ( R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt parenthesize s t sortedR t sett anyt allt tuplet sumt mint maxc c s6 t | | } x | o | Vt | | } q Wd S( sl Follow an attribute chain.
If you have a chain of objects where a.foo -> b, b.foo-> c, etc,
use this to iterate over all objects in the chain. Iteration is
terminated by getattr(x, attr) is None.
Args:
obj: the starting object
attr: the name of the chaining attribute
Yields:
Each successive object in the chain.
N( t getattr( R R t next( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt
attr_chain s
se for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
s
power<
( 'iter' | 'list' | 'tuple' | 'sorted' | 'set' | 'sum' |
'any' | 'all' | (any* trailer< '.' 'join' >) )
trailer< '(' node=any ')' >
any*
>
sN
power<
'sorted'
trailer< '(' arglist ')' >
any*
>
c C s t p7 t i t a t i t a t i t a t a n t t t g } xU t | t | d D]; \ } } h } | i | | o | d | j o t Sqf Wt
S( s Returns true if node is in an environment where all that is required
of it is being itterable (ie, it doesn't matter if it returns a list
or an itterator).
See test_map_nochange in test_fixers.py for some examples and tests.
t parentR' ( t
pats_builtR t compile_patternt p1t p0t p2RQ t zipR` t matcht False( R' t patternst patternRa t results( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt in_special_context s
$ c C s | i } | d j o | i t i j o t S| i } | i t i t i f j o t S| i t i
j o | i d | j o t S| i t i j pG | i t i
j o9 | d j o | i t i j p | i d | j o t St S( sG
Check that something isn't an attribute or function name etc.
i N( t prev_siblingR* t typeR R Ri Ra R t funcdeft classdeft expr_stmtRN t
parameterst
typedargslistR RQ ( R' t prevRa ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt is_probably_builtin s ' c C sY | i t i j o | S| i } | i d } | _ t t i | g } | | _ | S( N( Ro R t suiteR! Ra R* R ( R' Ra Rw ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt
make_suite s c C s( x! | i t i j o
| i } q W| S( s Find the top level namespace.( Ro R t
file_inputRa ( R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt find_root s c C s" t | t | | } t | S( s Returns true if name is imported from package at the
top level of the tree which node belongs to.
To cover the case of an import like 'import foo', use
None for the package and 'foo' for the name. ( t find_bindingRz t bool( t packageR R' t binding( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt does_tree_import s c C s | i t i t i f j S( s0 Returns true if the node is an import statement.( Ro R t import_nameRJ ( R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt is_import s c C s d } t | } t | | | o d Sd } } xr t | i D]a \ } } | | p qG n x3 t | i | D] \ } } | | p Pq{ q{ W| | } PqG W| d j oj xg t | i D]R \ } } | i t i j o3 | i o) | i d i t i j o | d } Pq q Wn | d j o: t
t i t t i
d t t i
| d d g } n% t | t t i
| d d g } | t g }
| i | t
t i |
d S( s\ Works like `does_tree_import` but adds an import statement
if it was not imported. c S s. | i t i j o | i o t | i d S( Ni ( Ro R t simple_stmtRN R ( R' ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt is_import_stmt s Ni i u importR u ( Rz R t enumerateRN Ro R R R R7 R* R R R R RP R. R" ( R} R R' R t roott
insert_post offsett idxt node2t import_RN ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt touch_import s:
"$c C so xh| i D]]} d } | i t i j oP t | | i d o | St | t | i d | } | o
| } q>n| i t i t i f j o4 t | t | i d | } | o
| } q>nv| i t i
j o t | t | i d | } | o
| } q>x/t | i d D]g \ } } | i t i
j oH | i d j o8 t | t | i | d | } | o
| } qq q Wn | i t j o! | i d i | j o
| } n t | | | o
| } nb | i t i j o t | | | } n9 | i t i j o% t | | i d o
| } q>n | o" | p | St | o | Sq
q
Wd S( s Returns the node which binds variable name, otherwise None.
If optional argument package is supplied, only imports will
be returned.
See test cases for examples.i ii i t :i i N( RN R* Ro R t for_stmtt _findR{ Rx t if_stmtt
while_stmtt try_stmtR R t COLONR t _def_symst _is_import_bindingR Rr R ( R R' R} t childt retR1 t it kid( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyR{ H sL
## '
c C s | g } xt | ol | i } | i d j o$ | i t j o | i | i q | i t i j o | i | j o | Sq Wd S( Ni ( t popRo t _block_symst extendRN R R R R* ( R R' t nodes( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyR s s #
c C sQ | i t i j o| o | i d } | i t i j os x | i D]a } | i t i j o | i d i | j o | SqE | i t i j o | i | j o | SqE WqM| i t i j o9 | i d } | i t i j o | i | j o | SqM| i t i j o | i | j o | Sn(| i t i j o| o% t
| i d i | j o d S| i d } | o t
d | o d S| i t i j o t
| | o | S| i t i j o9 | i d } | i t i j o | i | j o | SqM| i t i j o | i | j o | S| o | i t i j o | Sn d S( s Will reuturn node if node will import name, or node
will import * from package. None is returned otherwise.
See test cases for examples. i i ii u asN( Ro R R RN t dotted_as_namest dotted_as_nameR R R RJ t unicodet stripR* R RI t import_as_namet STAR( R' R R} RO R t lastR1 ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyR } sB
#
# # '
#
# # N(7 t __doc__t pgen2R t pytreeR R t pygramR R t R R
R R R R* R R R R R( R, R. R/ R2 R6 R9 RG RP RS RT RU RW t consuming_callsR` Re Rd Rf Ri Rb Rm Rv Rx Rz R R R Rq Rp R R{ R R R R ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_util.pyt sV - *
pygram.py 0000666 00000001572 15050104230 0006414 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Export the Python grammar and symbols."""
# Python imports
import os
# Local imports
from .pgen2 import token
from .pgen2 import driver
from . import pytree
# The grammar file
_GRAMMAR_FILE = os.path.join(os.path.dirname(__file__), "Grammar.txt")
class Symbols(object):
def __init__(self, grammar):
"""Initializer.
Creates an attribute for each grammar symbol (nonterminal),
whose value is the symbol's type (an int >= 256).
"""
for name, symbol in grammar.symbol2number.iteritems():
setattr(self, name, symbol)
python_grammar = driver.load_grammar(_GRAMMAR_FILE)
python_symbols = Symbols(python_grammar)
python_grammar_no_print_statement = python_grammar.copy()
del python_grammar_no_print_statement.keywords["print"]
PatternGrammar.txt 0000666 00000001431 15050104230 0010222 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
# A grammar to describe tree matching patterns.
# Not shown here:
# - 'TOKEN' stands for any token (leaf node)
# - 'any' stands for any node (leaf or interior)
# With 'any' we can still specify the sub-structure.
# The start symbol is 'Matcher'.
Matcher: Alternatives ENDMARKER
Alternatives: Alternative ('|' Alternative)*
Alternative: (Unit | NegatedUnit)+
Unit: [NAME '='] ( STRING [Repeater]
| NAME [Details] [Repeater]
| '(' Alternatives ')' [Repeater]
| '[' Alternatives ']'
)
NegatedUnit: 'not' (STRING | NAME [Details] | '(' Alternatives ')')
Repeater: '*' | '+' | '{' NUMBER [',' NUMBER] '}'
Details: '<' Alternatives '>'
patcomp.pyc 0000666 00000015121 15050104230 0006716 0 ustar 00
Lc @ s d Z d Z d d k Z d d k l Z l Z l Z l Z l Z l Z d d k
l Z d d k
l Z e i
i e i
i e d Z d e f d
YZ d Z d e f d
YZ h e i d 6e i d 6e i d 6d d 6Z d Z d Z d Z d S( s Pattern compiler.
The grammer is taken from PatternGrammar.txt.
The compiler compiles a pattern to a pytree.*Pattern instance.
s# Guido van Rossum iNi ( t drivert literalst tokent tokenizet parset grammar( t pytree( t pygrams PatternGrammar.txtt PatternSyntaxErrorc B s e Z RS( ( t __name__t
__module__( ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR s c c sy t t i t i t i f } t i t i | i } x9 | D]1 } | \ } } } } } | | j o | Vq@ q@ Wd S( s6 Tokenizes a string suppressing significant whitespace.N(
t setR t NEWLINEt INDENTt DEDENTR t generate_tokensR t generate_linest next( t inputt skipt tokenst quintuplet typet valuet startt endt line_text( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt tokenize_wrapper s
t PatternCompilerc B s> e Z e d Z e d Z d Z d d Z d Z RS( c C s^ t i | | _ t i | i | _ t i | _ t i | _ t i
| i d t | _ d S( s^ Initializer.
Takes an optional alternative filename for the pattern grammar.
t convertN( R t load_grammarR R t Symbolst symst python_grammart pygrammart python_symbolst pysymst Drivert pattern_convert( t selft grammar_file( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt __init__, s
c C sb t | } y | i i | d | } Wn* t i j
o } t t | n X| i | S( s= Compiles a pattern string to a nested pytree.*Pattern object.t debug( R R t parse_tokensR t
ParseErrorR t strt compile_node( R' R R* R t roott e( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt compile_pattern7 s c C s
| i | i i j o | i d } n | i | i i j o g } | i d d d D] } | | i | qX ~ } t | d j o | d St i g } | D] } | | g q ~ d d d d } | i S| i | i i
j or g } | i D] } | | i | q ~ } t | d j o | d St i | g d d d d } | i S| i | i i j o0 | i | i d }
t i
|
} | i S| i | i i j p t d } | i } t | d j o2 | d i t i j o | d i } | d } n d }
t | d j o2 | d i | i i j o | d }
| d } n | i | |
}
|
d j oi|
i | i i j p t |
i } | d } | i t i j o d } t i } n | i t i j o d } t i } n | i t i j o} | d i t i j p t t | d
j p t | i | d } } t | d j o | i | d } qn t p t | d j p
| d j o1 |
i }
t i |
g g d | d | }
qn | d j o
| |
_ n |
i S( sX Compiles a node, recursively.
This is one big switch on the node type.
i Ni i t mint maxi ii ( i i ( R R t Matchert childrent AlternativesR. t lenR t WildcardPatternt optimizet Alternativet NegatedUnitt
compile_basict NegatedPatternt Unitt AssertionErrort NoneR t EQUALR t Repeatert STARt HUGEt PLUSt LBRACEt RBRACEt get_intt Falset name( R' t nodet _[1]t cht altst _[2]t at pt _[3]t unitst patternRJ t nodest repeatR5 t childR2 R3 ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR. @ sh : 9 - *
-
)
c C s t | d j p t | d } | i t i j o/ t t i | i } t i
t | | S| i t i j o | i } | i
oN | t j o t d | n | d o t d n t i
t | S| d j o
d } nK | i d p: t | i | d } | d j o t d | q>n | d o! | i | d i d g } n d } t i | | Snz | i d j o | i | d S| i d j oG | d j p t | i | d } t i | g g d
d d d St p
t | d S( Ni i s Invalid token: %rs Can't have details for tokent anyt _s Invalid symbol: %rt (t [R2 R3 ( R7 R? R R t STRINGt unicodeR t
evalStringR R t LeafPatternt _type_of_literalt NAMEt isuppert TOKEN_MAPR R@ t
startswitht getattrR$ R. R5 t NodePatternR8 RI ( R' RU RV RK R R t contentt
subpattern( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR< s<
! c C s' | i t i j p t t | i S( N( R R t NUMBERR? t intR ( R' RK ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyRH s N(
R R
t _PATTERN_GRAMMAR_FILER) RI R1 R. R@ R< RH ( ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR * s
G #Ra R\ Ri t TOKENc C s= | d i o t i S| t i j o t i | Sd Sd S( Ni ( t isalphaR Ra R t opmapR@ ( R ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR` s
c C sZ | \ } } } } | p | | i j o t i | | d | St i | | d | Sd S( s9 Converts raw node information to a Node or Leaf instance.t contextN( t
number2symbolR t Nodet Leaf( R t
raw_node_infoR R Ro R5 ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR& s c C s t i | S( N( R R1 ( RT ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyR1 s ( t __doc__t
__author__t ost pgen2R R R R R R t R R t patht joint dirnamet __file__Rk t ExceptionR R t objectR Ra R\ Ri R@ Rc R` R& R1 ( ( ( s' /usr/lib64/python2.6/lib2to3/patcomp.pyt s" .
pgen2/conv.pyc 0000666 00000020015 15050104230 0007231 0 ustar 00
Lc @ sE d Z d d k Z d d k l Z l Z d e i f d YZ d S( s Convert graminit.[ch] spit out by pgen to Python code.
Pgen is the Python parser generator. It is useful to quickly create a
parser from a grammar file in Python's grammar notation. But I don't
want my parsers to be written in C (yet), so I'm translating the
parsing tables to Python data structures and writing a Python parse
engine.
Note that the token numbers are constants determined by the standard
Python tokenizer. The standard token module defines these numbers and
their names (the names are not used much). The token numbers are
hardcoded into the Python tokenizer and into pgen. A Python
implementation of the Python tokenizer is also available, in the
standard tokenize module.
On the other hand, symbol numbers (representing the grammar's
non-terminals) are assigned by pgen based on the actual grammar
input.
Note: this module is pretty much obsolete; the pgen module generates
equivalent grammar tables directly from the Grammar.txt input file
without having to invoke the Python pgen C program.
iN( t grammart tokent Converterc B s2 e Z d Z d Z d Z d Z d Z RS( s2 Grammar subclass that reads classic pgen output files.
The run() method reads the tables as produced by the pgen parser
generator, typically contained in two C files, graminit.h and
graminit.c. The other methods are for internal use only.
See the base class for more documentation.
c C s( | i | | i | | i d S( s< Load the grammar tables from the text files written by pgen.N( t parse_graminit_ht parse_graminit_ct
finish_off( t selft
graminit_ht
graminit_c( ( s* /usr/lib64/python2.6/lib2to3/pgen2/conv.pyt run/ s
c C s y t | } Wn% t j
o } d | | f GHt SXh | _ h | _ d } x | D] } | d 7} t i d | } | o) | i o d | | | i f GHqW | i \ } } t | } | | i j p t
| | i j p t
| | i | <| | i | @o d | | d | _[] = {
{, },
...
};
- followed by a state array, of the form:
static state states_[] = {
{, arcs__},
...
};
s Can't open %s: %si i s #include "pgenheaders.h"
s #include "grammar.h"
s static arc s) static arc arcs_(\d+)_(\d+)\[(\d+)\] = {$s \s+{(\d+), (\d+)},$s };
s' static state states_(\d+)\[(\d+)\] = {$s \s+{(\d+), arcs_(\d+)_(\d+)},$s static dfa dfas\[(\d+)\] = {$s0 \s+{(\d+), "(\w+)", (\d+), (\d+), states_(\d+),$i i i i s \s+("(?:\\\d\d\d)*")},$i s! static label labels\[(\d+)\] = {$s \s+{(\d+), (0|"\w+")},$t 0s grammar _PyParser_Grammar = {
s
\s+(\d+),$s dfas,
s \s+{(\d+), labels},$s \s+(\d+)$N( R
R R t nextR t
startswithR R t mapR R t ranget appendt lent statest groupR
R t evalt enumeratet ordt dfast Nonet labelst startt
StopIteration(! R R R R R R t allarcsR% R t nt mt kt arcst _t it jt st tt stateR* t ndfasR R t xt yt zt firstt rawbitsett ct byteR, t nlabelsR- ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/conv.pyR T s
#
#
-$$
#
& c C s h | _ h | _ xo t | i D]^ \ } \ } } | t i j o | d j o | | i | s pgen2/tokenize.pyo 0000666 00000041051 15050104230 0010133 0 ustar 00
Lc * @ s d Z d Z d Z d d k Z d d k Z d d k l Z l Z d d k Td d k l
Z
g Z e e
D]" Z
e
d d
j o e e
qk qk [ d d d
g Z [
y e Wn e j
o
e Z n Xd Z d Z d Z d Z d Z e e d e e e Z d Z d Z d Z d Z d Z e e e e e Z d Z e d d e e Z d e Z e e e Z! e d e! d Z" e e" e! e Z# d Z$ d Z% d! Z& d" Z' e d# d$ Z( e d% d& Z) e d' d( d) d* d+ d, d- d. d/ Z* d0 Z+ e d1 d2 Z, e e* e+ e, Z- e e# e- e) e Z. e e. Z/ e d3 e d4 d d5 e d6 d Z0 e d e e( Z1 e e e1 e# e- e0 e Z2 e3 e i4 e/ e2 e& e' f \ Z5 Z6 Z7 Z8 h& e i4 e$ d4 6e i4 e% d6 6e7 d7 6e8 d8 6e7 d9 6e8 d: 6e7 d; 6e8 d< 6e7 d= 6e8 d> 6e7 d? 6e8 d@ 6e7 dA 6e8 dB 6e7 dC 6e8 dD 6e7 dE 6e8 dF 6e7 dG 6e8 dH 6e7 dI 6e8 dJ 6e7 dK 6e8 dL 6e7 dM 6e8 dN 6e7 dO 6e8 dP 6e7 dQ 6e8 dR 6e7 dS 6e8 dT 6d dU 6d dV 6d dW 6d dX 6d dY 6d dZ 6Z: h Z; x d D] Z< e< e; e< dx e? f dy YZ@ dz e? f d{ YZA d| ZB eB d} ZC d~ ZD d d d YZE e i4 d ZF d ZG d ZH d ZI d ZJ eK d j oT d d kL ZL eM eL iN d j o eC eO eL iN d iP qeC eL iQ iP n d S( s Tokenization help for Python programs.
generate_tokens(readline) is a generator that breaks a stream of
text into Python tokens. It accepts a readline-like method which is called
repeatedly to get the next line of input (or "" for EOF). It generates
5-tuples with these members:
the token type (see token.py)
the token (a string)
the starting (row, column) indices of the token (a 2-tuple of ints)
the ending (row, column) indices of the token (a 2-tuple of ints)
the original line (string)
It is designed to match the working of the Python tokenizer exactly, except
that it produces COMMENT tokens for comments and gives type OP for all
operators
Older entry points
tokenize_loop(readline, tokeneater)
tokenize(readline, tokeneater=printtoken)
are the same, except instead of generating tokens, tokeneater is a callback
function to which the 5 fields described above are passed as 5 arguments,
each time a new token is found.s Ka-Ping Yee s@ GvR, ESR, Tim Peters, Thomas Wouters, Fred Drake, Skip MontanaroiN( t BOM_UTF8t lookup( t *i ( t tokeni t _t tokenizet generate_tokenst
untokenizec G s d d i | d S( Nt (t |t )( t join( t choices( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt group0 s c G s t | d S( NR ( R
( R ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt any1 s c G s t | d S( Nt ?( R
( R ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt maybe2 s s [ \f\t]*s #[^\r\n]*s \\\r?\ns [a-zA-Z_]\w*s
0[bB][01]*s 0[xX][\da-fA-F]*[lL]?s 0[oO]?[0-7]*[lL]?s
[1-9]\d*[lL]?s [eE][-+]?\d+s \d+\.\d*s \.\d+s \d+s \d+[jJ]s [jJ]s [^'\\]*(?:\\.[^'\\]*)*'s [^"\\]*(?:\\.[^"\\]*)*"s% [^'\\]*(?:(?:\\.|'(?!''))[^'\\]*)*'''s% [^"\\]*(?:(?:\\.|"(?!""))[^"\\]*)*"""s [ubUB]?[rR]?'''s [ubUB]?[rR]?"""s& [uU]?[rR]?'[^\n'\\]*(?:\\.[^\n'\\]*)*'s& [uU]?[rR]?"[^\n"\\]*(?:\\.[^\n"\\]*)*"s \*\*=?s >>=?s <<=?s <>s !=s //=?s ->s [+\-*/%&|^=<>]=?t ~s [][(){}]s \r?\ns [:;.,`@]s' [uUbB]?[rR]?'[^\n'\\]*(?:\\.[^\n'\\]*)*t 's' [uUbB]?[rR]?"[^\n"\\]*(?:\\.[^\n"\\]*)*t "s '''s """s r'''s r"""s u'''s u"""s b'''s b"""s ur'''s ur"""s br'''s br"""s R'''s R"""s U'''s U"""s B'''s B"""s uR'''s uR"""s Ur'''s Ur"""s UR'''s UR"""s bR'''s bR"""s Br'''s Br"""s BR'''s BR"""t rt Rt ut Ut bt Bs r's r"s R's R"s u's u"s U's U"s b's b"s B's B"s ur's ur"s Ur's Ur"s uR's uR"s UR's UR"s br's br"s Br's Br"s bR's bR"s BR's BR"i t
TokenErrorc B s e Z RS( ( t __name__t
__module__( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s t StopTokenizingc B s e Z RS( ( R R ( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s c C sA | \ } } | \ } } d | | | | t | t | f GHd S( Ns %d,%d-%d,%d: %s %s( t tok_namet repr( t typeR t startt endt linet srowt scolt erowt ecol( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt
printtoken s c C s+ y t | | Wn t j
o n Xd S( s:
The tokenize() function accepts two parameters: one representing the
input stream, and one providing an output mechanism for tokenize().
The first parameter, readline, must be a callable object which provides
the same interface as the readline() method of built-in file objects.
Each call to the function should return one line of input as a string.
The second parameter, tokeneater, must also be a callable object. It is
called once for each token, with five arguments, corresponding to the
tuples generated by generate_tokens().
N( t
tokenize_loopR ( t readlinet
tokeneater( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s
c C s% x t | D] } | | q
Wd S( N( R ( R* R+ t
token_info( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR) s
t Untokenizerc B s, e Z d Z d Z d Z d Z RS( c C s g | _ d | _ d | _ d S( Ni i ( t tokenst prev_rowt prev_col( t self( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt __init__ s c C s< | \ } } | | i } | o | i i d | n d S( Nt ( R0 R. t append( R1 R! t rowt colt
col_offset( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt add_whitespace s
c C s x | D] } t | d j o | i | | Pn | \ } } } } } | i | | i i | | \ | _ | _ | t t f j o | i d 7_ d | _ q q Wd i | i S( Ni i i t (
t lent compatR8 R. R4 R/ R0 t NEWLINEt NLR ( R1 t iterablet tt tok_typeR R! R" R# ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s
c C s4 t } g } | i i } | \ } } | t t f j o | d 7} n | t t f j o
t } n x | D] } | d \ } } | t t f j o | d 7} n | t j o | i | qi nb | t j o | i
qi nD | t t f j o
t } n' | o | o | | d t } n | | qi Wd S( NR3 i i( t FalseR. R4 t NAMEt NUMBERR< R= t Truet INDENTt DEDENTt pop( R1 R R> t startlinet indentst toks_appendt toknumt tokvalt tok( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR; s2
( R R R2 R8 R R; ( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR- s s coding[:=]\s*([-\w.]+)c C sd | d i i d d } | d j p | i d o d S| d j p | i d
o d S| S( s( Imitates get_normal_name in tokenizer.c.i R t -s utf-8s utf-8-s latin-1s
iso-8859-1s iso-latin-1s latin-1-s iso-8859-1-s iso-latin-1-( s latin-1s
iso-8859-1s iso-latin-1( s latin-1-s iso-8859-1-s iso-latin-1-( t lowert replacet
startswith( t orig_enct enc( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt _get_normal_name s
c s t d } d } f d } f d } | } | i t o t | d } d } n | p | g f S| | } | o | | g f S| } | p | | g f S| | } | o | | | g f S| | | g f S( s
The detect_encoding() function is used to detect the encoding that should
be used to decode a Python source file. It requires one argment, readline,
in the same way as the tokenize() generator.
It will call readline a maximum of twice, and return the encoding used
(as a string) and a list of any lines (left as bytes) it has read
in.
It detects the encoding from the presence of a utf-8 bom or an encoding
cookie as specified in pep-0263. If both a bom and a cookie are present, but
disagree, a SyntaxError will be raised. If the encoding cookie is an invalid
charset, raise a SyntaxError. Note that if a utf-8 bom is found,
'utf-8-sig' is returned.
If no encoding is specified, then the default of 'utf-8' will be returned.
s utf-8c s) y SWn t j
o t SXd S( N( t
StopIterationt bytes( ( R* ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt read_or_stop s c s y | i d } Wn t j
o d SXt i | } | p d St | d } y t | } Wn# t j
o t d | n X o. | i d j o t d n | d 7} n | S( Nt asciii s unknown encoding: s utf-8s encoding problem: utf-8s -sig(
t decodet UnicodeDecodeErrort Nonet cookie_ret findallRT R t LookupErrort SyntaxErrort name( R# t line_stringt matchest encodingt codec( t bom_found( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt find_cookie s" i s utf-8-sigN( RA R[ RQ R RD ( R* Rc t defaultRW Rf t firstt second( ( Re R* s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt detect_encoding s,
c C s t } | i | S( s Transform tokens back into Python source code.
Each element returned by the iterable must be a token sequence
with at least two elements, a token number and token value. If
only two tokens are passed, the resulting output is poor.
Round-trip invariant for full input:
Untokenized source will match input source exactly
Round-trip invariant for limited intput:
# Output text will tokenize the back to the input
t1 = [tok[:2] for tok in generate_tokens(f.readline)]
newcode = untokenize(t1)
readline = iter(newcode.splitlines(1)).next
t2 = [tok[:2] for tokin generate_tokens(readline)]
assert t1 == t2
( R- R ( R> t ut( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR D s c c s d } } } t i d d } } d \ } } d } d g } xy
| }
Wn t j
o
d }
n X| d } d t |
} } | o |
p t d |
f n | i |
} | oO | i d } } t | |
| |
| | f | |
f Vd \ } } d } q| oY |
d d j oH |
d d
j o7 t | |
|
| t |
f | f Vd } d } q@ q| |
} | |
} q@ n|| d j oH| o@|
p Pn d } x~ | | j op |
| d j o | d } nD |
| d j o | t
d t
} n |
| d
j o
d } n P| d } qW| | j o Pn |
| d j o |
| d j o{ |
| i d } | t | } t | | | f | | t | f |
f Vt
|
| | | f | t |
f |
f Vq@ t
t f |
| d j |
| | | f | t |
f |
f Vq@ n | | d j o5 | i | t |
| | d f | | f |
f Vn x | | d j oZ | | j o t d d | | |
f n | d } t d | | f | | f |
f VqlWn' |
p t d | d f f n d } x'| | j ot i |
| } | o| i d \ } } | | f | | f | } } } |
| | !|
| } } | | j p | d j o% | d j o t | | | |
f Vq'| d j o5 t } | d j o
t
} n | | | | |
f Vq'| d j o t | | | |
f Vq'| t j o~ t | } | i |
| } | o: | i d } |
| | !} t | | | | f |
f Vq| | f }
|
| } |
} Pq'| t j p" | d t j p | d t j oy | d d j oP | | f }
t | p t | d p t | d } |
| d } } |
} Pqt | | | |
f Vq'| | j o t | | | |
f Vq'| d j o$ t
| | | | f |
f Vd } q'| d j o | d } n | d j o | d } n t | | | |
f Vqt |
| | | f | | d f |
f V| d } qWq@ x2 | d D]& } t d | d f | d f d f Vq:Wt d | d f | d f d f Vd S( sS
The generate_tokens() generator requires one argment, readline, which
must be a callable object which provides the same interface as the
readline() method of built-in file objects. Each call to the function
should return one line of input as a string. Alternately, readline
can be a callable function terminating with StopIteration:
readline = open(myfile).next # Example of alternate readline
The generator produces 5-tuples with these members: the token type; the
token string; a 2-tuple (srow, scol) of ints specifying the row and
column where the token begins in the source; a 2-tuple (erow, ecol) of
ints specifying the row and column where the token ends in the source;
and the line on which the token was found. The line passed is the
logical line; continuation lines are included.
i R t
0123456789R9 i s EOF in multi-line stringis \
is \
R3 s s s #
t #s
is3 unindent does not match any outer indentation levels
s EOF in multi-line statementt .i i s
s \s ([{s )]}N( R9 i ( R9 i ( t stringt
ascii_lettersR[ RU R: R t matchR" t STRINGt
ERRORTOKENt tabsizet rstript COMMENTR= R4 RE t IndentationErrorRF t
pseudoprogt spanRC R< t
triple_quotedt endprogst
single_quotedRB t OPt ENDMARKER( R* t lnumt parenlevt continuedt namecharst numcharst contstrt needcontt contlineRI R# t post maxt strstartt endprogt endmatchR" t columnt
comment_tokent nl_post pseudomatchR! t spost eposR t initialt newlinet indent( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR Y s
)
$
$
(
)
$t __main__( s '''s """s r'''s r"""s R'''s R"""s u'''s u"""s U'''s U"""s b'''s b"""s B'''s B"""s ur'''s ur"""s Ur'''s Ur"""s uR'''s uR"""s UR'''s UR"""s br'''s br"""s Br'''s Br"""s bR'''s bR"""s BR'''s BR"""( R R s r's r"s R's R"s u's u"s U's U"s b's b"s B's B"s ur's ur"s Ur's Ur"s uR's uR"s UR's UR"s br's br"s Br's Br"s bR's bR"s BR's BR"( (R t __doc__t
__author__t __credits__Ro t ret codecsR R t lib2to3.pgen2.tokenR9 R t _[1]t dirt xt __all__RV t NameErrort strR
R R t
Whitespacet Commentt Ignoret Namet Binnumbert Hexnumbert Octnumbert Decnumbert Intnumbert Exponentt
Pointfloatt Expfloatt Floatnumbert
Imagnumbert Numbert Singlet Doublet Single3t Double3t Triplet Stringt Operatort Brackett Specialt Funnyt
PlainTokent Tokent ContStrt PseudoExtrast PseudoTokent mapt compilet tokenprogRx t single3progt double3progR[ R{ Rz R? R| Rt t ExceptionR R R( R R) R- R\ RT Rj R R R t sysR: t argvt openR* t stdin( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt s
<
'# 8 H
pgen2/parse.pyc 0000666 00000016152 15050104230 0007405 0 ustar 00
Lc @ sF d Z d d k l Z d e f d YZ d e f d YZ d S( s Parser engine for the grammar tables generated by pgen.
The grammar table must be loaded first.
See Parser/parser.c in the Python distribution for additional info on
how this parsing engine works.
i ( t tokent
ParseErrorc B s e Z d Z d Z RS( s( Exception to signal the parser is stuck.c C sH t i | d | | | | f | | _ | | _ | | _ | | _ d S( Ns! %s: type=%r, value=%r, context=%r( t Exceptiont __init__t msgt typet valuet context( t selfR R R R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR s ( t __name__t
__module__t __doc__R ( ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR s t Parserc B sS e Z d Z d d Z d d Z d Z d Z d Z d Z d Z
RS( s5 Parser engine.
The proper usage sequence is:
p = Parser(grammar, [converter]) # create instance
p.setup([start]) # prepare for parsing
:
if p.addtoken(...): # parse a token; may raise ParseError
break
root = p.rootnode # root of abstract syntax tree
A Parser instance may be reused by calling setup() repeatedly.
A Parser instance contains state pertaining to the current token
sequence, and should not be used concurrently by different threads
to parse separate token sequences.
See driver.py for how to get input tokens by tokenizing a file or
string.
Parsing is complete when addtoken() returns True; the root of the
abstract syntax tree can then be retrieved from the rootnode
instance variable. When a syntax error occurs, addtoken() raises
the ParseError exception. There is no error recovery; the parser
cannot be used after a syntax error was reported (but it can be
reinitialized by calling setup()).
c C s | | _ | p d | _ d S( s Constructor.
The grammar argument is a grammar.Grammar instance; see the
grammar module for more information.
The parser is not ready yet for parsing; you must call the
setup() method to get it started.
The optional convert argument is a function mapping concrete
syntax tree nodes to abstract syntax tree nodes. If not
given, no conversion is done and the syntax tree produced is
the concrete syntax tree. If given, it must be a function of
two arguments, the first being the grammar (a grammar.Grammar
instance), and the second being the concrete syntax tree node
to be converted. The syntax tree is converted from the bottom
up.
A concrete syntax tree node is a (type, value, context, nodes)
tuple, where type is the node type (a token or symbol number),
value is None for symbols and a string for tokens, context is
None or an opaque value used for error reporting (typically a
(lineno, offset) pair), and nodes is a list of children for
symbols, and None for tokens.
An abstract syntax tree node may be anything; this is entirely
up to the converter function.
c S s | S( ( ( t grammart node( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyt W s N( R
t convert( R R
R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR 9 s c C sm | d j o | i i } n | d d g f } | i i | d | f } | g | _ d | _ t | _ d S( s Prepare for parsing.
This *must* be called before starting to parse.
The optional argument is an alternative start symbol; it
defaults to the grammar's start symbol.
You can use a Parser instance to parse any number of programs;
each time you call setup() the parser is reset to an initial
state determined by the (implicit or explicit) start symbol.
i N( t NoneR
t startt dfast stackt rootnodet sett
used_names( R R t newnodet
stackentry( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyt setupY s
c C s | i | | | } xt o| i d \ } } } | \ } } | | }
x|
D] \ } } | i i | \ }
} | | j o |
d j p t | i | | | | | } xZ | | d | f g j o? | i | i p t S| i d \ } } } | \ } } q Wt S|
d j oR | i i |
} | \ } } | | j o% | i
|
| i i |
| | PqrqR qR Wd | f |
j o1 | i | i p t d | | | qq t d | | | q Wd S( s< Add a token; return True iff this is the end of the program.ii i s too much inputs bad inputN( t classifyt TrueR R
t labelst AssertionErrort shiftt popt FalseR t pushR ( R R R R t ilabelt dfat stateR t statest firstt arcst it newstatet tt vt itsdfat itsstatest itsfirst( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyt addtokenq sB
c C s | t i j o; | i i | | i i i | } | d j o | Sn | i i i | } | d j o t d | | | n | S( s& Turn a token into a label. (Internal)s bad tokenN(
R t NAMER t addR
t keywordst getR t tokensR ( R R R R R# ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR s
c C sy | i d \ } } } | | | d f } | i | i | } | d j o | d i | n | | | f | i d s pgen2/parse.pyo 0000666 00000016101 15050104230 0007413 0 ustar 00
Lc @ sF d Z d d k l Z d e f d YZ d e f d YZ d S( s Parser engine for the grammar tables generated by pgen.
The grammar table must be loaded first.
See Parser/parser.c in the Python distribution for additional info on
how this parsing engine works.
i ( t tokent
ParseErrorc B s e Z d Z d Z RS( s( Exception to signal the parser is stuck.c C sH t i | d | | | | f | | _ | | _ | | _ | | _ d S( Ns! %s: type=%r, value=%r, context=%r( t Exceptiont __init__t msgt typet valuet context( t selfR R R R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR s ( t __name__t
__module__t __doc__R ( ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR s t Parserc B sS e Z d Z d d Z d d Z d Z d Z d Z d Z d Z
RS( s5 Parser engine.
The proper usage sequence is:
p = Parser(grammar, [converter]) # create instance
p.setup([start]) # prepare for parsing
:
if p.addtoken(...): # parse a token; may raise ParseError
break
root = p.rootnode # root of abstract syntax tree
A Parser instance may be reused by calling setup() repeatedly.
A Parser instance contains state pertaining to the current token
sequence, and should not be used concurrently by different threads
to parse separate token sequences.
See driver.py for how to get input tokens by tokenizing a file or
string.
Parsing is complete when addtoken() returns True; the root of the
abstract syntax tree can then be retrieved from the rootnode
instance variable. When a syntax error occurs, addtoken() raises
the ParseError exception. There is no error recovery; the parser
cannot be used after a syntax error was reported (but it can be
reinitialized by calling setup()).
c C s | | _ | p d | _ d S( s Constructor.
The grammar argument is a grammar.Grammar instance; see the
grammar module for more information.
The parser is not ready yet for parsing; you must call the
setup() method to get it started.
The optional convert argument is a function mapping concrete
syntax tree nodes to abstract syntax tree nodes. If not
given, no conversion is done and the syntax tree produced is
the concrete syntax tree. If given, it must be a function of
two arguments, the first being the grammar (a grammar.Grammar
instance), and the second being the concrete syntax tree node
to be converted. The syntax tree is converted from the bottom
up.
A concrete syntax tree node is a (type, value, context, nodes)
tuple, where type is the node type (a token or symbol number),
value is None for symbols and a string for tokens, context is
None or an opaque value used for error reporting (typically a
(lineno, offset) pair), and nodes is a list of children for
symbols, and None for tokens.
An abstract syntax tree node may be anything; this is entirely
up to the converter function.
c S s | S( ( ( t grammart node( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyt W s N( R
t convert( R R
R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR 9 s c C sm | d j o | i i } n | d d g f } | i i | d | f } | g | _ d | _ t | _ d S( s Prepare for parsing.
This *must* be called before starting to parse.
The optional argument is an alternative start symbol; it
defaults to the grammar's start symbol.
You can use a Parser instance to parse any number of programs;
each time you call setup() the parser is reset to an initial
state determined by the (implicit or explicit) start symbol.
i N( t NoneR
t startt dfast stackt rootnodet sett
used_names( R R t newnodet
stackentry( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyt setupY s
c C s | i | | | } xt o| i d \ } } } | \ } } | | }
xm|
D] \ } } | i i | \ }
} | | j o~ | i | | | | | } xZ | | d | f g j o? | i | i p t S| i d \ } } } | \ } } q Wt S|
d j oR | i i |
} | \ } } | | j o% | i |
| i i |
| | Pq^qR qR Wd | f |
j o1 | i | i p t
d | | | qq t
d | | | q Wd S( s< Add a token; return True iff this is the end of the program.ii i s too much inputs bad inputN( t classifyt TrueR R
t labelst shiftt popt FalseR t pushR ( R R R R t ilabelt dfat stateR t statest firstt arcst it newstatet tt vt itsdfat itsstatest itsfirst( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyt addtokenq s@
c C s | t i j o; | i i | | i i i | } | d j o | Sn | i i i | } | d j o t d | | | n | S( s& Turn a token into a label. (Internal)s bad tokenN(
R t NAMER t addR
t keywordst getR t tokensR ( R R R R R" ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/parse.pyR s
c C sy | i d \ } } } | | | d f } | i | i | } | d j o | d i | n | | | f | i d s pgen2/token.pyc 0000666 00000004352 15050104230 0007412 0 ustar 00
Lc @ s d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d
Z
d Z d Z d
Z
d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d! Z! d" Z" d# Z# d$ Z$ d% Z% d& Z& d' Z' d( Z( d) Z) d* Z* d+ Z+ d, Z, d- Z- d. Z. d/ Z/ d0 Z0 d1 Z1 d2 Z2 d3 Z3 d4 Z4 d5 Z5 d6 Z6 d7 Z7 d8 Z8 d9 Z9 d: Z: h Z; xD e< i= D]3 \ Z> Z? e@ e? e@ d j o e> e; e? S(? s! Token constants (from "token.h").i i i i i i i i i i i
i i i
i i i i i i i i i i i i i i i i i i i i! i" i# i$ i% i& i' i( i) i* i+ i, i- i. i/ i0 i1 i2 i3 i4 i5 i6 i7 i8 i c C s
| t j S( N( t NT_OFFSET( t x( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt
ISTERMINALK s c C s
| t j S( N( R ( R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt
ISNONTERMINALN s c C s
| t j S( N( t ENDMARKER( R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt ISEOFQ s N(D t __doc__R t NAMEt NUMBERt STRINGt NEWLINEt INDENTt DEDENTt LPARt RPARt LSQBt RSQBt COLONt COMMAt SEMIt PLUSt MINUSt STARt SLASHt VBARt AMPERt LESSt GREATERt EQUALt DOTt PERCENTt BACKQUOTEt LBRACEt RBRACEt EQEQUALt NOTEQUALt LESSEQUALt GREATEREQUALt TILDEt
CIRCUMFLEXt LEFTSHIFTt
RIGHTSHIFTt
DOUBLESTARt PLUSEQUALt MINEQUALt STAREQUALt
SLASHEQUALt PERCENTEQUALt
AMPEREQUALt VBAREQUALt CIRCUMFLEXEQUALt LEFTSHIFTEQUALt RIGHTSHIFTEQUALt DOUBLESTAREQUALt DOUBLESLASHt DOUBLESLASHEQUALt ATt OPt COMMENTt NLt RARROWt
ERRORTOKENt N_TOKENSR t tok_namet globalst itemst _namet _valuet typeR R R ( ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt s pgen2/literals.py 0000666 00000003116 15050104230 0007743 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Safely evaluate Python string literals without using eval()."""
import re
simple_escapes = {"a": "\a",
"b": "\b",
"f": "\f",
"n": "\n",
"r": "\r",
"t": "\t",
"v": "\v",
"'": "'",
'"': '"',
"\\": "\\"}
def escape(m):
all, tail = m.group(0, 1)
assert all.startswith("\\")
esc = simple_escapes.get(tail)
if esc is not None:
return esc
if tail.startswith("x"):
hexes = tail[1:]
if len(hexes) < 2:
raise ValueError("invalid hex string escape ('\\%s')" % tail)
try:
i = int(hexes, 16)
except ValueError:
raise ValueError("invalid hex string escape ('\\%s')" % tail)
else:
try:
i = int(tail, 8)
except ValueError:
raise ValueError("invalid octal string escape ('\\%s')" % tail)
return chr(i)
def evalString(s):
assert s.startswith("'") or s.startswith('"'), repr(s[:1])
q = s[0]
if s[:3] == q*3:
q = q*3
assert s.endswith(q), repr(s[-len(q):])
assert len(s) >= 2*len(q)
s = s[len(q):-len(q)]
return re.sub(r"\\(\'|\"|\\|[abfnrtv]|x.{0,2}|[0-7]{1,3})", escape, s)
def test():
for i in range(256):
c = chr(i)
s = repr(c)
e = evalString(s)
if e != c:
print i, c, s, e
if __name__ == "__main__":
test()
pgen2/conv.pyo 0000666 00000015645 15050104230 0007262 0 ustar 00
Lc @ sE d Z d d k Z d d k l Z l Z d e i f d YZ d S( s Convert graminit.[ch] spit out by pgen to Python code.
Pgen is the Python parser generator. It is useful to quickly create a
parser from a grammar file in Python's grammar notation. But I don't
want my parsers to be written in C (yet), so I'm translating the
parsing tables to Python data structures and writing a Python parse
engine.
Note that the token numbers are constants determined by the standard
Python tokenizer. The standard token module defines these numbers and
their names (the names are not used much). The token numbers are
hardcoded into the Python tokenizer and into pgen. A Python
implementation of the Python tokenizer is also available, in the
standard tokenize module.
On the other hand, symbol numbers (representing the grammar's
non-terminals) are assigned by pgen based on the actual grammar
input.
Note: this module is pretty much obsolete; the pgen module generates
equivalent grammar tables directly from the Grammar.txt input file
without having to invoke the Python pgen C program.
iN( t grammart tokent Converterc B s2 e Z d Z d Z d Z d Z d Z RS( s2 Grammar subclass that reads classic pgen output files.
The run() method reads the tables as produced by the pgen parser
generator, typically contained in two C files, graminit.h and
graminit.c. The other methods are for internal use only.
See the base class for more documentation.
c C s( | i | | i | | i d S( s< Load the grammar tables from the text files written by pgen.N( t parse_graminit_ht parse_graminit_ct
finish_off( t selft
graminit_ht
graminit_c( ( s* /usr/lib64/python2.6/lib2to3/pgen2/conv.pyt run/ s
c C s y t | } Wn% t j
o } d | | f GHt SXh | _ h | _ d } x | D] } | d 7} t i d | } | o) | i o d | | | i f GHqW | i \ } } t | } | | i | <| | i | @o d | | d | _[] = {
{, },
...
};
- followed by a state array, of the form:
static state states_[] = {
{, arcs__},
...
};
s Can't open %s: %si i s static arc s) static arc arcs_(\d+)_(\d+)\[(\d+)\] = {$s \s+{(\d+), (\d+)},$s' static state states_(\d+)\[(\d+)\] = {$s \s+{(\d+), arcs_(\d+)_(\d+)},$s static dfa dfas\[(\d+)\] = {$s0 \s+{(\d+), "(\w+)", (\d+), (\d+), states_(\d+),$i i i i s \s+("(?:\\\d\d\d)*")},$i s! static label labels\[(\d+)\] = {$s \s+{(\d+), (0|"\w+")},$t 0s
\s+(\d+),$s \s+{(\d+), labels},$s \s+(\d+)$N( R
R R t nextt
startswithR R t mapR R t ranget appendt statest groupt evalt enumeratet ordt dfast Nonet labelst startt
StopIteration(! R R R R R R t allarcsR# R t nt mt kt arcst _t it jt st tt stateR( t ndfasR R t xt yt zt firstt rawbitsett ct byteR* t nlabelsR+ ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/conv.pyR T s
-
c C s h | _ h | _ xo t | i D]^ \ } \ } } | t i j o | d j o | | i | s pgen2/tokenize.pyc 0000666 00000041206 15050104230 0010121 0 ustar 00
Lc * @ s d Z d Z d Z d d k Z d d k Z d d k l Z l Z d d k Td d k l
Z
g Z e e
D]" Z
e
d d
j o e e
qk qk [ d d d
g Z [
y e Wn e j
o
e Z n Xd Z d Z d Z d Z d Z e e d e e e Z d Z d Z d Z d Z d Z e e e e e Z d Z e d d e e Z d e Z e e e Z! e d e! d Z" e e" e! e Z# d Z$ d Z% d! Z& d" Z' e d# d$ Z( e d% d& Z) e d' d( d) d* d+ d, d- d. d/ Z* d0 Z+ e d1 d2 Z, e e* e+ e, Z- e e# e- e) e Z. e e. Z/ e d3 e d4 d d5 e d6 d Z0 e d e e( Z1 e e e1 e# e- e0 e Z2 e3 e i4 e/ e2 e& e' f \ Z5 Z6 Z7 Z8 h& e i4 e$ d4 6e i4 e% d6 6e7 d7 6e8 d8 6e7 d9 6e8 d: 6e7 d; 6e8 d< 6e7 d= 6e8 d> 6e7 d? 6e8 d@ 6e7 dA 6e8 dB 6e7 dC 6e8 dD 6e7 dE 6e8 dF 6e7 dG 6e8 dH 6e7 dI 6e8 dJ 6e7 dK 6e8 dL 6e7 dM 6e8 dN 6e7 dO 6e8 dP 6e7 dQ 6e8 dR 6e7 dS 6e8 dT 6d dU 6d dV 6d dW 6d dX 6d dY 6d dZ 6Z: h Z; x d D] Z< e< e; e< dx e? f dy YZ@ dz e? f d{ YZA d| ZB eB d} ZC d~ ZD d d d YZE e i4 d ZF d ZG d ZH d ZI d ZJ eK d j oT d d kL ZL eM eL iN d j o eC eO eL iN d iP qeC eL iQ iP n d S( s Tokenization help for Python programs.
generate_tokens(readline) is a generator that breaks a stream of
text into Python tokens. It accepts a readline-like method which is called
repeatedly to get the next line of input (or "" for EOF). It generates
5-tuples with these members:
the token type (see token.py)
the token (a string)
the starting (row, column) indices of the token (a 2-tuple of ints)
the ending (row, column) indices of the token (a 2-tuple of ints)
the original line (string)
It is designed to match the working of the Python tokenizer exactly, except
that it produces COMMENT tokens for comments and gives type OP for all
operators
Older entry points
tokenize_loop(readline, tokeneater)
tokenize(readline, tokeneater=printtoken)
are the same, except instead of generating tokens, tokeneater is a callback
function to which the 5 fields described above are passed as 5 arguments,
each time a new token is found.s Ka-Ping Yee s@ GvR, ESR, Tim Peters, Thomas Wouters, Fred Drake, Skip MontanaroiN( t BOM_UTF8t lookup( t *i ( t tokeni t _t tokenizet generate_tokenst
untokenizec G s d d i | d S( Nt (t |t )( t join( t choices( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt group0 s c G s t | d S( NR ( R
( R ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt any1 s c G s t | d S( Nt ?( R
( R ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt maybe2 s s [ \f\t]*s #[^\r\n]*s \\\r?\ns [a-zA-Z_]\w*s
0[bB][01]*s 0[xX][\da-fA-F]*[lL]?s 0[oO]?[0-7]*[lL]?s
[1-9]\d*[lL]?s [eE][-+]?\d+s \d+\.\d*s \.\d+s \d+s \d+[jJ]s [jJ]s [^'\\]*(?:\\.[^'\\]*)*'s [^"\\]*(?:\\.[^"\\]*)*"s% [^'\\]*(?:(?:\\.|'(?!''))[^'\\]*)*'''s% [^"\\]*(?:(?:\\.|"(?!""))[^"\\]*)*"""s [ubUB]?[rR]?'''s [ubUB]?[rR]?"""s& [uU]?[rR]?'[^\n'\\]*(?:\\.[^\n'\\]*)*'s& [uU]?[rR]?"[^\n"\\]*(?:\\.[^\n"\\]*)*"s \*\*=?s >>=?s <<=?s <>s !=s //=?s ->s [+\-*/%&|^=<>]=?t ~s [][(){}]s \r?\ns [:;.,`@]s' [uUbB]?[rR]?'[^\n'\\]*(?:\\.[^\n'\\]*)*t 's' [uUbB]?[rR]?"[^\n"\\]*(?:\\.[^\n"\\]*)*t "s '''s """s r'''s r"""s u'''s u"""s b'''s b"""s ur'''s ur"""s br'''s br"""s R'''s R"""s U'''s U"""s B'''s B"""s uR'''s uR"""s Ur'''s Ur"""s UR'''s UR"""s bR'''s bR"""s Br'''s Br"""s BR'''s BR"""t rt Rt ut Ut bt Bs r's r"s R's R"s u's u"s U's U"s b's b"s B's B"s ur's ur"s Ur's Ur"s uR's uR"s UR's UR"s br's br"s Br's Br"s bR's bR"s BR's BR"i t
TokenErrorc B s e Z RS( ( t __name__t
__module__( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s t StopTokenizingc B s e Z RS( ( R R ( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s c C sA | \ } } | \ } } d | | | | t | t | f GHd S( Ns %d,%d-%d,%d: %s %s( t tok_namet repr( t typeR t startt endt linet srowt scolt erowt ecol( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt
printtoken s c C s+ y t | | Wn t j
o n Xd S( s:
The tokenize() function accepts two parameters: one representing the
input stream, and one providing an output mechanism for tokenize().
The first parameter, readline, must be a callable object which provides
the same interface as the readline() method of built-in file objects.
Each call to the function should return one line of input as a string.
The second parameter, tokeneater, must also be a callable object. It is
called once for each token, with five arguments, corresponding to the
tuples generated by generate_tokens().
N( t
tokenize_loopR ( t readlinet
tokeneater( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s
c C s% x t | D] } | | q
Wd S( N( R ( R* R+ t
token_info( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR) s
t Untokenizerc B s, e Z d Z d Z d Z d Z RS( c C s g | _ d | _ d | _ d S( Ni i ( t tokenst prev_rowt prev_col( t self( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt __init__ s c C sS | \ } } | | i j p t | | i } | o | i i d | n d S( Nt ( R/ t AssertionErrorR0 R. t append( R1 R! t rowt colt
col_offset( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt add_whitespace s
c C s x | D] } t | d j o | i | | Pn | \ } } } } } | i | | i i | | \ | _ | _ | t t f j o | i d 7_ d | _ q q Wd i | i S( Ni i i t (
t lent compatR9 R. R5 R/ R0 t NEWLINEt NLR ( R1 t iterablet tt tok_typeR R! R" R# ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR s
c C s4 t } g } | i i } | \ } } | t t f j o | d 7} n | t t f j o
t } n x | D] } | d \ } } | t t f j o | d 7} n | t j o | i | qi nb | t j o | i
qi nD | t t f j o
t } n' | o | o | | d t } n | | qi Wd S( NR3 i i( t FalseR. R5 t NAMEt NUMBERR= R> t Truet INDENTt DEDENTt pop( R1 R R? t startlinet indentst toks_appendt toknumt tokvalt tok( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR< s2
( R R R2 R9 R R< ( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR- s s coding[:=]\s*([-\w.]+)c C sd | d i i d d } | d j p | i d o d S| d j p | i d
o d S| S( s( Imitates get_normal_name in tokenizer.c.i R t -s utf-8s utf-8-s latin-1s
iso-8859-1s iso-latin-1s latin-1-s iso-8859-1-s iso-latin-1-( s latin-1s
iso-8859-1s iso-latin-1( s latin-1-s iso-8859-1-s iso-latin-1-( t lowert replacet
startswith( t orig_enct enc( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt _get_normal_name s
c s t d } d } f d } f d } | } | i t o t | d } d } n | p | g f S| | } | o | | g f S| } | p | | g f S| | } | o | | | g f S| | | g f S( s
The detect_encoding() function is used to detect the encoding that should
be used to decode a Python source file. It requires one argment, readline,
in the same way as the tokenize() generator.
It will call readline a maximum of twice, and return the encoding used
(as a string) and a list of any lines (left as bytes) it has read
in.
It detects the encoding from the presence of a utf-8 bom or an encoding
cookie as specified in pep-0263. If both a bom and a cookie are present, but
disagree, a SyntaxError will be raised. If the encoding cookie is an invalid
charset, raise a SyntaxError. Note that if a utf-8 bom is found,
'utf-8-sig' is returned.
If no encoding is specified, then the default of 'utf-8' will be returned.
s utf-8c s) y SWn t j
o t SXd S( N( t
StopIterationt bytes( ( R* ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt read_or_stop s c s y | i d } Wn t j
o d SXt i | } | p d St | d } y t | } Wn# t j
o t d | n X o. | i d j o t d n | d 7} n | S( Nt asciii s unknown encoding: s utf-8s encoding problem: utf-8s -sig(
t decodet UnicodeDecodeErrort Nonet cookie_ret findallRU R t LookupErrort SyntaxErrort name( R# t line_stringt matchest encodingt codec( t bom_found( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt find_cookie s" i s utf-8-sigN( RB R\ RR R RE ( R* Rd t defaultRX Rg t firstt second( ( Rf R* s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt detect_encoding s,
c C s t } | i | S( s Transform tokens back into Python source code.
Each element returned by the iterable must be a token sequence
with at least two elements, a token number and token value. If
only two tokens are passed, the resulting output is poor.
Round-trip invariant for full input:
Untokenized source will match input source exactly
Round-trip invariant for limited intput:
# Output text will tokenize the back to the input
t1 = [tok[:2] for tok in generate_tokens(f.readline)]
newcode = untokenize(t1)
readline = iter(newcode.splitlines(1)).next
t2 = [tok[:2] for tokin generate_tokens(readline)]
assert t1 == t2
( R- R ( R? t ut( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR D s c c s d } } } t i d d } } d \ } } d } d g } xy
| }
Wn t j
o
d }
n X| d } d t |
} } | o |
p t d |
f n | i |
} | oO | i d } } t | |
| |
| | f | |
f Vd \ } } d } q| oY |
d d j oH |
d d
j o7 t | |
|
| t |
f | f Vd } d } q@ q| |
} | |
} q@ n|| d j oH| o@|
p Pn d } x~ | | j op |
| d j o | d } nD |
| d j o | t
d t
} n |
| d
j o
d } n P| d } qW| | j o Pn |
| d j o |
| d j o{ |
| i d } | t | } t | | | f | | t | f |
f Vt
|
| | | f | t |
f |
f Vq@ t
t f |
| d j |
| | | f | t |
f |
f Vq@ n | | d j o5 | i | t |
| | d f | | f |
f Vn x | | d j oZ | | j o t d d | | |
f n | d } t d | | f | | f |
f VqlWn' |
p t d | d f f n d } x?| | j o1t i |
| } | o| i d \ } } | | f | | f | } } } |
| | !|
| } } | | j p | d j o% | d j o t | | | |
f Vq?| d j o5 t } | d j o
t
} n | | | | |
f Vq?| d j o0 | i d p t t | | | |
f Vq?| t j o~ t | } | i |
| } | o: | i d } |
| | !} t | | | | f |
f Vq | | f }
|
| } |
} Pq?| t j p" | d t j p | d t j oy | d d j oP | | f }
t | p t | d p t | d } |
| d } } |
} Pq t | | | |
f Vq?| | j o t | | | |
f Vq?| d j o$ t
| | | | f |
f Vd } q?| d j o | d } n | d j o | d } n t | | | |
f Vqt |
| | | f | | d f |
f V| d } qWq@ x2 | d D]& } t d | d f | d f d f VqRWt d | d f | d f d f Vd S( sS
The generate_tokens() generator requires one argment, readline, which
must be a callable object which provides the same interface as the
readline() method of built-in file objects. Each call to the function
should return one line of input as a string. Alternately, readline
can be a callable function terminating with StopIteration:
readline = open(myfile).next # Example of alternate readline
The generator produces 5-tuples with these members: the token type; the
token string; a 2-tuple (srow, scol) of ints specifying the row and
column where the token begins in the source; a 2-tuple (erow, ecol) of
ints specifying the row and column where the token ends in the source;
and the line on which the token was found. The line passed is the
logical line; continuation lines are included.
i R t
0123456789R: i s EOF in multi-line stringis \
is \
R3 s s s #
t #s
is3 unindent does not match any outer indentation levels
s EOF in multi-line statementt .s
i i s \s ([{s )]}N( R: i ( R: i ( t stringt
ascii_lettersR\ RV R; R t matchR" t STRINGt
ERRORTOKENt tabsizet rstript COMMENTR> R5 RF t IndentationErrorRG t
pseudoprogt spanRD R= t endswithR4 t
triple_quotedt endprogst
single_quotedRC t OPt ENDMARKER( R* t lnumt parenlevt continuedt namecharst numcharst contstrt needcontt contlineRJ R# t post maxt strstartt endprogt endmatchR" t columnt
comment_tokent nl_post pseudomatchR! t spost eposR t initialt newlinet indent( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyR Y s
)
$
$
(
)
$t __main__( s '''s """s r'''s r"""s R'''s R"""s u'''s u"""s U'''s U"""s b'''s b"""s B'''s B"""s ur'''s ur"""s Ur'''s Ur"""s uR'''s uR"""s UR'''s UR"""s br'''s br"""s Br'''s Br"""s bR'''s bR"""s BR'''s BR"""( R R s r's r"s R's R"s u's u"s U's U"s b's b"s B's B"s ur's ur"s Ur's Ur"s uR's uR"s UR's UR"s br's br"s Br's Br"s bR's bR"s BR's BR"( (R t __doc__t
__author__t __credits__Rp t ret codecsR R t lib2to3.pgen2.tokenR: R t _[1]t dirt xt __all__RW t NameErrort strR
R R t
Whitespacet Commentt Ignoret Namet Binnumbert Hexnumbert Octnumbert Decnumbert Intnumbert Exponentt
Pointfloatt Expfloatt Floatnumbert
Imagnumbert Numbert Singlet Doublet Single3t Double3t Triplet Stringt Operatort Brackett Specialt Funnyt
PlainTokent Tokent ContStrt PseudoExtrast PseudoTokent mapt compilet tokenprogRy t single3progt double3progR\ R} R| R@ R~ Ru t ExceptionR R R( R R) R- R] RU Rk R R R t sysR; t argvt openR* t stdin( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/tokenize.pyt s
<
'# 8 H
pgen2/driver.pyo 0000666 00000011667 15050104230 0007610 0 ustar 00
Lc @ s d Z d Z d d g Z d d k Z d d k Z d d k Z d d k Z d d k l Z l Z l
Z
l Z l Z d e
f d YZ d Z d
d e e d d Z d Z d S(
sZ Parser driver.
This provides a high-level interface to parse a file into a syntax tree.
s# Guido van Rossum t Drivert load_grammariNi ( t grammart parset tokent tokenizet pgenc B sV e Z d d d Z e d Z e d Z e d Z d e d Z e d Z RS( c C s< | | _ | d j o t i } n | | _ | | _ d S( N( R t Nonet loggingt getLoggert loggert convert( t selfR R R
( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt __init__ s
c C s' t i | i | i } | i d } d } d } } } } }
d } x| D]} | \ } } } } }
| | | f j og | \ }
} | |
j o" | d |
| 7} |
} d } n | | j o | |
| | !7} | } q n | t i t i f j oA | | 7} | \ } } | i d o | d 7} d } qQ qQ n | t
i j o t i | } n | o$ | i
i d t
i | | | n | i | | | | f o | o | i
i d n Pn d } | \ } } | i d o | d 7} d } qQ qQ Wt i d | | | | f | i S(
s4 Parse a series of tokens and return the syntax tree.i i u s
s %s %r (prefix=%r)s Stop.t s incomplete inputN( R t ParserR R t setupR R t COMMENTt NLt endswithR t OPt opmapR
t debugt tok_namet addtokent
ParseErrort rootnode( R t tokensR t pt linenot columnt typet valuet startt endt line_textt prefixt quintuplet s_linenot s_column( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_tokens% sT
c C s" t i | i } | i | | S( s* Parse a stream and return the syntax tree.( R t generate_tokenst readlineR( ( R t streamR R ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_stream_rawU s c C s | i | | S( s* Parse a stream and return the syntax tree.( R, ( R R+ R ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_streamZ s c C s; t i | d | } z | i | | SWd | i Xd S( s( Parse a file and return the syntax tree.t rN( t codecst openR- t close( R t filenamet encodingR R+ ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt
parse_file^ s c C s( t i t | i } | i | | S( s* Parse a string and return the syntax tree.( R R) t generate_linest nextR( ( R t textR R ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_stringf s N(
t __name__t
__module__R R
t FalseR( R, R- R4 R8 ( ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyR s 0 c c s7 x | i t D] } | Vq Wx t o d Vq" Wd S( s< Generator that behaves like readline without using StringIO.R N( t
splitlinest True( R7 t line( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyR5 l s s Grammar.txtc C s7 | d j o t i } n | d j oZ t i i | \ } } | d j o
d } n | | d i t t t i
d } n | p t | | o~ | i d | t
i | } | oT | i d | y | i | Wqt j
o } | i d t | qXq3n t i } | i | | S( s' Load the grammar (maybe from a pickle).s .txtR t .s .pickles! Generating grammar tables from %ss Writing grammar tables to %ss Writing failed:N( R R R t ost patht splitextt joint mapt strt syst version_infot _newert infoR t generate_grammart dumpt IOErrorR t Grammart load( t gtt gpt savet forceR
t headt tailt gt e( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyR t s&
+$
c C sR t i i | p t St i i | p t St i i | t i i | j S( s0 Inquire whether file a was written since file b.( R@ RA t existsR; R= t getmtime( t at b( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyRH s
( t __doc__t
__author__t __all__R/ R@ R RF R R R R R R t objectR R5 R R= R; R RH ( ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt s (P pgen2/tokenize.py 0000666 00000045250 15050104230 0007761 0 ustar 00 # Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation.
# All rights reserved.
"""Tokenization help for Python programs.
generate_tokens(readline) is a generator that breaks a stream of
text into Python tokens. It accepts a readline-like method which is called
repeatedly to get the next line of input (or "" for EOF). It generates
5-tuples with these members:
the token type (see token.py)
the token (a string)
the starting (row, column) indices of the token (a 2-tuple of ints)
the ending (row, column) indices of the token (a 2-tuple of ints)
the original line (string)
It is designed to match the working of the Python tokenizer exactly, except
that it produces COMMENT tokens for comments and gives type OP for all
operators
Older entry points
tokenize_loop(readline, tokeneater)
tokenize(readline, tokeneater=printtoken)
are the same, except instead of generating tokens, tokeneater is a callback
function to which the 5 fields described above are passed as 5 arguments,
each time a new token is found."""
__author__ = 'Ka-Ping Yee '
__credits__ = \
'GvR, ESR, Tim Peters, Thomas Wouters, Fred Drake, Skip Montanaro'
import string, re
from codecs import BOM_UTF8, lookup
from lib2to3.pgen2.token import *
from . import token
__all__ = [x for x in dir(token) if x[0] != '_'] + ["tokenize",
"generate_tokens", "untokenize"]
del token
try:
bytes
except NameError:
# Support bytes type in Python <= 2.5, so 2to3 turns itself into
# valid Python 3 code.
bytes = str
def group(*choices): return '(' + '|'.join(choices) + ')'
def any(*choices): return group(*choices) + '*'
def maybe(*choices): return group(*choices) + '?'
Whitespace = r'[ \f\t]*'
Comment = r'#[^\r\n]*'
Ignore = Whitespace + any(r'\\\r?\n' + Whitespace) + maybe(Comment)
Name = r'[a-zA-Z_]\w*'
Binnumber = r'0[bB][01]*'
Hexnumber = r'0[xX][\da-fA-F]*[lL]?'
Octnumber = r'0[oO]?[0-7]*[lL]?'
Decnumber = r'[1-9]\d*[lL]?'
Intnumber = group(Binnumber, Hexnumber, Octnumber, Decnumber)
Exponent = r'[eE][-+]?\d+'
Pointfloat = group(r'\d+\.\d*', r'\.\d+') + maybe(Exponent)
Expfloat = r'\d+' + Exponent
Floatnumber = group(Pointfloat, Expfloat)
Imagnumber = group(r'\d+[jJ]', Floatnumber + r'[jJ]')
Number = group(Imagnumber, Floatnumber, Intnumber)
# Tail end of ' string.
Single = r"[^'\\]*(?:\\.[^'\\]*)*'"
# Tail end of " string.
Double = r'[^"\\]*(?:\\.[^"\\]*)*"'
# Tail end of ''' string.
Single3 = r"[^'\\]*(?:(?:\\.|'(?!''))[^'\\]*)*'''"
# Tail end of """ string.
Double3 = r'[^"\\]*(?:(?:\\.|"(?!""))[^"\\]*)*"""'
Triple = group("[ubUB]?[rR]?'''", '[ubUB]?[rR]?"""')
# Single-line ' or " string.
String = group(r"[uU]?[rR]?'[^\n'\\]*(?:\\.[^\n'\\]*)*'",
r'[uU]?[rR]?"[^\n"\\]*(?:\\.[^\n"\\]*)*"')
# Because of leftmost-then-longest match semantics, be sure to put the
# longest operators first (e.g., if = came before ==, == would get
# recognized as two instances of =).
Operator = group(r"\*\*=?", r">>=?", r"<<=?", r"<>", r"!=",
r"//=?", r"->",
r"[+\-*/%&|^=<>]=?",
r"~")
Bracket = '[][(){}]'
Special = group(r'\r?\n', r'[:;.,`@]')
Funny = group(Operator, Bracket, Special)
PlainToken = group(Number, Funny, String, Name)
Token = Ignore + PlainToken
# First (or only) line of ' or " string.
ContStr = group(r"[uUbB]?[rR]?'[^\n'\\]*(?:\\.[^\n'\\]*)*" +
group("'", r'\\\r?\n'),
r'[uUbB]?[rR]?"[^\n"\\]*(?:\\.[^\n"\\]*)*' +
group('"', r'\\\r?\n'))
PseudoExtras = group(r'\\\r?\n', Comment, Triple)
PseudoToken = Whitespace + group(PseudoExtras, Number, Funny, ContStr, Name)
tokenprog, pseudoprog, single3prog, double3prog = map(
re.compile, (Token, PseudoToken, Single3, Double3))
endprogs = {"'": re.compile(Single), '"': re.compile(Double),
"'''": single3prog, '"""': double3prog,
"r'''": single3prog, 'r"""': double3prog,
"u'''": single3prog, 'u"""': double3prog,
"b'''": single3prog, 'b"""': double3prog,
"ur'''": single3prog, 'ur"""': double3prog,
"br'''": single3prog, 'br"""': double3prog,
"R'''": single3prog, 'R"""': double3prog,
"U'''": single3prog, 'U"""': double3prog,
"B'''": single3prog, 'B"""': double3prog,
"uR'''": single3prog, 'uR"""': double3prog,
"Ur'''": single3prog, 'Ur"""': double3prog,
"UR'''": single3prog, 'UR"""': double3prog,
"bR'''": single3prog, 'bR"""': double3prog,
"Br'''": single3prog, 'Br"""': double3prog,
"BR'''": single3prog, 'BR"""': double3prog,
'r': None, 'R': None,
'u': None, 'U': None,
'b': None, 'B': None}
triple_quoted = {}
for t in ("'''", '"""',
"r'''", 'r"""', "R'''", 'R"""',
"u'''", 'u"""', "U'''", 'U"""',
"b'''", 'b"""', "B'''", 'B"""',
"ur'''", 'ur"""', "Ur'''", 'Ur"""',
"uR'''", 'uR"""', "UR'''", 'UR"""',
"br'''", 'br"""', "Br'''", 'Br"""',
"bR'''", 'bR"""', "BR'''", 'BR"""',):
triple_quoted[t] = t
single_quoted = {}
for t in ("'", '"',
"r'", 'r"', "R'", 'R"',
"u'", 'u"', "U'", 'U"',
"b'", 'b"', "B'", 'B"',
"ur'", 'ur"', "Ur'", 'Ur"',
"uR'", 'uR"', "UR'", 'UR"',
"br'", 'br"', "Br'", 'Br"',
"bR'", 'bR"', "BR'", 'BR"', ):
single_quoted[t] = t
tabsize = 8
class TokenError(Exception): pass
class StopTokenizing(Exception): pass
def printtoken(type, token, start, end, line): # for testing
(srow, scol) = start
(erow, ecol) = end
print "%d,%d-%d,%d:\t%s\t%s" % \
(srow, scol, erow, ecol, tok_name[type], repr(token))
def tokenize(readline, tokeneater=printtoken):
"""
The tokenize() function accepts two parameters: one representing the
input stream, and one providing an output mechanism for tokenize().
The first parameter, readline, must be a callable object which provides
the same interface as the readline() method of built-in file objects.
Each call to the function should return one line of input as a string.
The second parameter, tokeneater, must also be a callable object. It is
called once for each token, with five arguments, corresponding to the
tuples generated by generate_tokens().
"""
try:
tokenize_loop(readline, tokeneater)
except StopTokenizing:
pass
# backwards compatible interface
def tokenize_loop(readline, tokeneater):
for token_info in generate_tokens(readline):
tokeneater(*token_info)
class Untokenizer:
def __init__(self):
self.tokens = []
self.prev_row = 1
self.prev_col = 0
def add_whitespace(self, start):
row, col = start
assert row <= self.prev_row
col_offset = col - self.prev_col
if col_offset:
self.tokens.append(" " * col_offset)
def untokenize(self, iterable):
for t in iterable:
if len(t) == 2:
self.compat(t, iterable)
break
tok_type, token, start, end, line = t
self.add_whitespace(start)
self.tokens.append(token)
self.prev_row, self.prev_col = end
if tok_type in (NEWLINE, NL):
self.prev_row += 1
self.prev_col = 0
return "".join(self.tokens)
def compat(self, token, iterable):
startline = False
indents = []
toks_append = self.tokens.append
toknum, tokval = token
if toknum in (NAME, NUMBER):
tokval += ' '
if toknum in (NEWLINE, NL):
startline = True
for tok in iterable:
toknum, tokval = tok[:2]
if toknum in (NAME, NUMBER):
tokval += ' '
if toknum == INDENT:
indents.append(tokval)
continue
elif toknum == DEDENT:
indents.pop()
continue
elif toknum in (NEWLINE, NL):
startline = True
elif startline and indents:
toks_append(indents[-1])
startline = False
toks_append(tokval)
cookie_re = re.compile("coding[:=]\s*([-\w.]+)")
def _get_normal_name(orig_enc):
"""Imitates get_normal_name in tokenizer.c."""
# Only care about the first 12 characters.
enc = orig_enc[:12].lower().replace("_", "-")
if enc == "utf-8" or enc.startswith("utf-8-"):
return "utf-8"
if enc in ("latin-1", "iso-8859-1", "iso-latin-1") or \
enc.startswith(("latin-1-", "iso-8859-1-", "iso-latin-1-")):
return "iso-8859-1"
return orig_enc
def detect_encoding(readline):
"""
The detect_encoding() function is used to detect the encoding that should
be used to decode a Python source file. It requires one argment, readline,
in the same way as the tokenize() generator.
It will call readline a maximum of twice, and return the encoding used
(as a string) and a list of any lines (left as bytes) it has read
in.
It detects the encoding from the presence of a utf-8 bom or an encoding
cookie as specified in pep-0263. If both a bom and a cookie are present, but
disagree, a SyntaxError will be raised. If the encoding cookie is an invalid
charset, raise a SyntaxError. Note that if a utf-8 bom is found,
'utf-8-sig' is returned.
If no encoding is specified, then the default of 'utf-8' will be returned.
"""
bom_found = False
encoding = None
default = 'utf-8'
def read_or_stop():
try:
return readline()
except StopIteration:
return bytes()
def find_cookie(line):
try:
line_string = line.decode('ascii')
except UnicodeDecodeError:
return None
matches = cookie_re.findall(line_string)
if not matches:
return None
encoding = _get_normal_name(matches[0])
try:
codec = lookup(encoding)
except LookupError:
# This behaviour mimics the Python interpreter
raise SyntaxError("unknown encoding: " + encoding)
if bom_found:
if codec.name != 'utf-8':
# This behaviour mimics the Python interpreter
raise SyntaxError('encoding problem: utf-8')
encoding += '-sig'
return encoding
first = read_or_stop()
if first.startswith(BOM_UTF8):
bom_found = True
first = first[3:]
default = 'utf-8-sig'
if not first:
return default, []
encoding = find_cookie(first)
if encoding:
return encoding, [first]
second = read_or_stop()
if not second:
return default, [first]
encoding = find_cookie(second)
if encoding:
return encoding, [first, second]
return default, [first, second]
def untokenize(iterable):
"""Transform tokens back into Python source code.
Each element returned by the iterable must be a token sequence
with at least two elements, a token number and token value. If
only two tokens are passed, the resulting output is poor.
Round-trip invariant for full input:
Untokenized source will match input source exactly
Round-trip invariant for limited intput:
# Output text will tokenize the back to the input
t1 = [tok[:2] for tok in generate_tokens(f.readline)]
newcode = untokenize(t1)
readline = iter(newcode.splitlines(1)).next
t2 = [tok[:2] for tokin generate_tokens(readline)]
assert t1 == t2
"""
ut = Untokenizer()
return ut.untokenize(iterable)
def generate_tokens(readline):
"""
The generate_tokens() generator requires one argment, readline, which
must be a callable object which provides the same interface as the
readline() method of built-in file objects. Each call to the function
should return one line of input as a string. Alternately, readline
can be a callable function terminating with StopIteration:
readline = open(myfile).next # Example of alternate readline
The generator produces 5-tuples with these members: the token type; the
token string; a 2-tuple (srow, scol) of ints specifying the row and
column where the token begins in the source; a 2-tuple (erow, ecol) of
ints specifying the row and column where the token ends in the source;
and the line on which the token was found. The line passed is the
logical line; continuation lines are included.
"""
lnum = parenlev = continued = 0
namechars, numchars = string.ascii_letters + '_', '0123456789'
contstr, needcont = '', 0
contline = None
indents = [0]
while 1: # loop over lines in stream
try:
line = readline()
except StopIteration:
line = ''
lnum = lnum + 1
pos, max = 0, len(line)
if contstr: # continued string
if not line:
raise TokenError, ("EOF in multi-line string", strstart)
endmatch = endprog.match(line)
if endmatch:
pos = end = endmatch.end(0)
yield (STRING, contstr + line[:end],
strstart, (lnum, end), contline + line)
contstr, needcont = '', 0
contline = None
elif needcont and line[-2:] != '\\\n' and line[-3:] != '\\\r\n':
yield (ERRORTOKEN, contstr + line,
strstart, (lnum, len(line)), contline)
contstr = ''
contline = None
continue
else:
contstr = contstr + line
contline = contline + line
continue
elif parenlev == 0 and not continued: # new statement
if not line: break
column = 0
while pos < max: # measure leading whitespace
if line[pos] == ' ': column = column + 1
elif line[pos] == '\t': column = (column//tabsize + 1)*tabsize
elif line[pos] == '\f': column = 0
else: break
pos = pos + 1
if pos == max: break
if line[pos] in '#\r\n': # skip comments or blank lines
if line[pos] == '#':
comment_token = line[pos:].rstrip('\r\n')
nl_pos = pos + len(comment_token)
yield (COMMENT, comment_token,
(lnum, pos), (lnum, pos + len(comment_token)), line)
yield (NL, line[nl_pos:],
(lnum, nl_pos), (lnum, len(line)), line)
else:
yield ((NL, COMMENT)[line[pos] == '#'], line[pos:],
(lnum, pos), (lnum, len(line)), line)
continue
if column > indents[-1]: # count indents or dedents
indents.append(column)
yield (INDENT, line[:pos], (lnum, 0), (lnum, pos), line)
while column < indents[-1]:
if column not in indents:
raise IndentationError(
"unindent does not match any outer indentation level",
("", lnum, pos, line))
indents = indents[:-1]
yield (DEDENT, '', (lnum, pos), (lnum, pos), line)
else: # continued statement
if not line:
raise TokenError, ("EOF in multi-line statement", (lnum, 0))
continued = 0
while pos < max:
pseudomatch = pseudoprog.match(line, pos)
if pseudomatch: # scan for tokens
start, end = pseudomatch.span(1)
spos, epos, pos = (lnum, start), (lnum, end), end
token, initial = line[start:end], line[start]
if initial in numchars or \
(initial == '.' and token != '.'): # ordinary number
yield (NUMBER, token, spos, epos, line)
elif initial in '\r\n':
newline = NEWLINE
if parenlev > 0:
newline = NL
yield (newline, token, spos, epos, line)
elif initial == '#':
assert not token.endswith("\n")
yield (COMMENT, token, spos, epos, line)
elif token in triple_quoted:
endprog = endprogs[token]
endmatch = endprog.match(line, pos)
if endmatch: # all on one line
pos = endmatch.end(0)
token = line[start:pos]
yield (STRING, token, spos, (lnum, pos), line)
else:
strstart = (lnum, start) # multiple lines
contstr = line[start:]
contline = line
break
elif initial in single_quoted or \
token[:2] in single_quoted or \
token[:3] in single_quoted:
if token[-1] == '\n': # continued string
strstart = (lnum, start)
endprog = (endprogs[initial] or endprogs[token[1]] or
endprogs[token[2]])
contstr, needcont = line[start:], 1
contline = line
break
else: # ordinary string
yield (STRING, token, spos, epos, line)
elif initial in namechars: # ordinary name
yield (NAME, token, spos, epos, line)
elif initial == '\\': # continued stmt
# This yield is new; needed for better idempotency:
yield (NL, token, spos, (lnum, pos), line)
continued = 1
else:
if initial in '([{': parenlev = parenlev + 1
elif initial in ')]}': parenlev = parenlev - 1
yield (OP, token, spos, epos, line)
else:
yield (ERRORTOKEN, line[pos],
(lnum, pos), (lnum, pos+1), line)
pos = pos + 1
for indent in indents[1:]: # pop remaining indent levels
yield (DEDENT, '', (lnum, 0), (lnum, 0), '')
yield (ENDMARKER, '', (lnum, 0), (lnum, 0), '')
if __name__ == '__main__': # testing
import sys
if len(sys.argv) > 1: tokenize(open(sys.argv[1]).readline)
else: tokenize(sys.stdin.readline)
pgen2/driver.py 0000666 00000011375 15050104230 0007425 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
# Modifications:
# Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Parser driver.
This provides a high-level interface to parse a file into a syntax tree.
"""
__author__ = "Guido van Rossum "
__all__ = ["Driver", "load_grammar"]
# Python imports
import codecs
import os
import logging
import sys
# Pgen imports
from . import grammar, parse, token, tokenize, pgen
class Driver(object):
def __init__(self, grammar, convert=None, logger=None):
self.grammar = grammar
if logger is None:
logger = logging.getLogger()
self.logger = logger
self.convert = convert
def parse_tokens(self, tokens, debug=False):
"""Parse a series of tokens and return the syntax tree."""
# XXX Move the prefix computation into a wrapper around tokenize.
p = parse.Parser(self.grammar, self.convert)
p.setup()
lineno = 1
column = 0
type = value = start = end = line_text = None
prefix = u""
for quintuple in tokens:
type, value, start, end, line_text = quintuple
if start != (lineno, column):
assert (lineno, column) <= start, ((lineno, column), start)
s_lineno, s_column = start
if lineno < s_lineno:
prefix += "\n" * (s_lineno - lineno)
lineno = s_lineno
column = 0
if column < s_column:
prefix += line_text[column:s_column]
column = s_column
if type in (tokenize.COMMENT, tokenize.NL):
prefix += value
lineno, column = end
if value.endswith("\n"):
lineno += 1
column = 0
continue
if type == token.OP:
type = grammar.opmap[value]
if debug:
self.logger.debug("%s %r (prefix=%r)",
token.tok_name[type], value, prefix)
if p.addtoken(type, value, (prefix, start)):
if debug:
self.logger.debug("Stop.")
break
prefix = ""
lineno, column = end
if value.endswith("\n"):
lineno += 1
column = 0
else:
# We never broke out -- EOF is too soon (how can this happen???)
raise parse.ParseError("incomplete input",
type, value, (prefix, start))
return p.rootnode
def parse_stream_raw(self, stream, debug=False):
"""Parse a stream and return the syntax tree."""
tokens = tokenize.generate_tokens(stream.readline)
return self.parse_tokens(tokens, debug)
def parse_stream(self, stream, debug=False):
"""Parse a stream and return the syntax tree."""
return self.parse_stream_raw(stream, debug)
def parse_file(self, filename, encoding=None, debug=False):
"""Parse a file and return the syntax tree."""
stream = codecs.open(filename, "r", encoding)
try:
return self.parse_stream(stream, debug)
finally:
stream.close()
def parse_string(self, text, debug=False):
"""Parse a string and return the syntax tree."""
tokens = tokenize.generate_tokens(generate_lines(text).next)
return self.parse_tokens(tokens, debug)
def generate_lines(text):
"""Generator that behaves like readline without using StringIO."""
for line in text.splitlines(True):
yield line
while True:
yield ""
def load_grammar(gt="Grammar.txt", gp=None,
save=True, force=False, logger=None):
"""Load the grammar (maybe from a pickle)."""
if logger is None:
logger = logging.getLogger()
if gp is None:
head, tail = os.path.splitext(gt)
if tail == ".txt":
tail = ""
gp = head + tail + ".".join(map(str, sys.version_info)) + ".pickle"
if force or not _newer(gp, gt):
logger.info("Generating grammar tables from %s", gt)
g = pgen.generate_grammar(gt)
if save:
logger.info("Writing grammar tables to %s", gp)
try:
g.dump(gp)
except IOError, e:
logger.info("Writing failed:"+str(e))
else:
g = grammar.Grammar()
g.load(gp)
return g
def _newer(a, b):
"""Inquire whether file a was written since file b."""
if not os.path.exists(a):
return False
if not os.path.exists(b):
return True
return os.path.getmtime(a) >= os.path.getmtime(b)
pgen2/token.py 0000666 00000002337 15050104230 0007250 0 ustar 00 #! /usr/bin/env python2.6
"""Token constants (from "token.h")."""
# Taken from Python (r53757) and modified to include some tokens
# originally monkeypatched in by pgen2.tokenize
#--start constants--
ENDMARKER = 0
NAME = 1
NUMBER = 2
STRING = 3
NEWLINE = 4
INDENT = 5
DEDENT = 6
LPAR = 7
RPAR = 8
LSQB = 9
RSQB = 10
COLON = 11
COMMA = 12
SEMI = 13
PLUS = 14
MINUS = 15
STAR = 16
SLASH = 17
VBAR = 18
AMPER = 19
LESS = 20
GREATER = 21
EQUAL = 22
DOT = 23
PERCENT = 24
BACKQUOTE = 25
LBRACE = 26
RBRACE = 27
EQEQUAL = 28
NOTEQUAL = 29
LESSEQUAL = 30
GREATEREQUAL = 31
TILDE = 32
CIRCUMFLEX = 33
LEFTSHIFT = 34
RIGHTSHIFT = 35
DOUBLESTAR = 36
PLUSEQUAL = 37
MINEQUAL = 38
STAREQUAL = 39
SLASHEQUAL = 40
PERCENTEQUAL = 41
AMPEREQUAL = 42
VBAREQUAL = 43
CIRCUMFLEXEQUAL = 44
LEFTSHIFTEQUAL = 45
RIGHTSHIFTEQUAL = 46
DOUBLESTAREQUAL = 47
DOUBLESLASH = 48
DOUBLESLASHEQUAL = 49
AT = 50
OP = 51
COMMENT = 52
NL = 53
RARROW = 54
ERRORTOKEN = 55
N_TOKENS = 56
NT_OFFSET = 256
#--end constants--
tok_name = {}
for _name, _value in globals().items():
if type(_value) is type(0):
tok_name[_value] = _name
def ISTERMINAL(x):
return x < NT_OFFSET
def ISNONTERMINAL(x):
return x >= NT_OFFSET
def ISEOF(x):
return x == ENDMARKER
pgen2/pgen.pyc 0000666 00000030070 15050104230 0007217 0 ustar 00
Lc @ s d d k l Z l Z l Z d e i f d YZ d e f d YZ d e f d YZ d e f d YZ d
d Z
d S(
i ( t grammart tokent tokenizet PgenGrammarc B s e Z RS( ( t __name__t
__module__( ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR s t ParserGeneratorc B s e Z d d Z d Z d Z d Z d Z d Z d Z d Z
d Z d Z d
Z
d Z d Z d
Z d Z d d Z d Z d Z RS( c C s d } | d j o t | } | i } n | | _ | | _ t i | i | _ | i | i
\ | _ | _ | d j o | n h | _
| i d S( N( t Nonet opent closet filenamet streamR t generate_tokenst readlinet generatort gettokent parset dfast startsymbolt firstt addfirstsets( t selfR
R t close_stream( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt __init__ s
c
C s t } | i i } | i | i | i | i d | i x; | D]3 } d t | i } | | i | <| | i | %ds %s -> %d( t enumerateR R# R R! R (
R R* R' Rb t todoR+ R- R. R/ t j( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt dump_nfa s
!
c C s d G| GHxp t | D]b \ } } d G| G| i o d p d GHx5 | i i D]$ \ } } d | | i | f GHqP Wq Wd S( Ns Dump of DFA fors States (final)Re s %s -> %d( Rf R$ R R R# ( R R* R, R+ R- R. R/ ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt dump_dfa s
c C s t } x | o t } x t | D]z \ } } xk t | d t | D]P } | | } | | j o3 | | =x | D] } | i | | qt Wt } PqI qI Wq# Wq Wd S( Ni ( t Truet FalseRf t rangeR t
unifystate( R R, t changesR+ t state_iRh t state_jR- ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRU s
c C s | i \ } } | i d j o | | f St } t } | i | | i | xK | i d j o: | i | i \ } } | i | | i | q\ W| | f Sd S( Nt |( t parse_altRC R] Ra R ( R RV RW t aat zz( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRS s
c C su | i \ } } xV | i d j p | i t i t i f j o) | i \ } } | i | | } q W| | f S( Nt (t [( Rv Rw ( t
parse_itemRC RN R R@ t STRINGRa ( R RV t bR( t d( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRs
s
c C s | i d j oG | i | i \ } } | i t i d | i | | | f S| i \ } } | i } | d j o | | f S| i | i | | d j o | | f S| | f Sd S( NRw t ]t +t *( R} R~ ( RC R RS RQ R RR Ra t
parse_atom( R RV RW RC ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRx s
c C s | i d j o: | i | i \ } } | i t i d | | f S| i t i t i f j o: t } t } | i
| | i | i | | f S| i d | i | i d S( NRv t )s+ expected (...) or NAME or STRING, got %s/%s( RC R RS RQ R RR RN R@ Ry R] Ra t raise_error( R RV RW ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR ( s
c C sg | i | j p | d j o3 | i | j o# | i d | | | i | i n | i } | i | S( Ns expected %s/%s, got %s/%s( RN R RC R R ( R RN RC ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRQ 9 s -
c C sk | i i } x1 | d t i t i f j o | i i } q W| \ | _ | _ | _ | _ | _ d S( Ni (
R R/ R t COMMENTt NLRN RC t begint endt line( R t tup( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR A s
c G su | o; y | | } WqB d i | g t t | } qB Xn t | | i | i d | i d | i f d S( Nt i i ( t joint mapt strt SyntaxErrorR
R R ( R t msgt args( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR H s 'N( R R R R R0 R& R" R RD R RT Ri Rj RU RS Rs Rx R RQ R R ( ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR
s$ . $
R] c B s e Z d Z d d Z RS( c C s
g | _ d S( N( R ( R ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR S s c C sU | d j p t | t p t t | t p t | i i | | f d S( N( R R9 R R; R] R R! ( R R/ R. ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRa V s $N( R R R R Ra ( ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR] Q s R^ c B s2 e Z d Z d Z d Z d Z d Z RS( c C sv t | t p t t t | i t p t t | t p t | | _ | | j | _ h | _ d S( N( R9 t dictR; t iterR/ R] R_ R$ R ( R R_ t final( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR ] s # c C sV t | t p t | | i j p t t | t p t | | i | s H
% pgen2/driver.pyc 0000666 00000011765 15050104230 0007573 0 ustar 00
Lc @ s d Z d Z d d g Z d d k Z d d k Z d d k Z d d k Z d d k l Z l Z l
Z
l Z l Z d e
f d YZ d Z d
d e e d d Z d Z d S(
sZ Parser driver.
This provides a high-level interface to parse a file into a syntax tree.
s# Guido van Rossum t Drivert load_grammariNi ( t grammart parset tokent tokenizet pgenc B sV e Z d d d Z e d Z e d Z e d Z d e d Z e d Z RS( c C s< | | _ | d j o t i } n | | _ | | _ d S( N( R t Nonet loggingt getLoggert loggert convert( t selfR R R
( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt __init__ s
c C sP t i | i | i } | i d } d } d } } } } }
d } x| D]} | \ } } } } }
| | | f j o | | f | j p t | | f | f | \ }
} | |
j o" | d |
| 7} |
} d } n | | j o | |
| | !7} | } qn | t i t i f j oA | | 7} | \ } } | i
d o | d 7} d } qQ qQ n | t i j o t i
| } n | o$ | i i d t i | | | n | i | | | | f o | o | i i d n Pn d } | \ } } | i
d o | d 7} d } qQ qQ Wt i d | | | | f | i S(
s4 Parse a series of tokens and return the syntax tree.i i u s
s %s %r (prefix=%r)s Stop.t s incomplete inputN( R t ParserR R t setupR t AssertionErrorR t COMMENTt NLt endswithR t OPt opmapR
t debugt tok_namet addtokent
ParseErrort rootnode( R t tokensR t pt linenot columnt typet valuet startt endt line_textt prefixt quintuplet s_linenot s_column( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_tokens% sV
)
c C s" t i | i } | i | | S( s* Parse a stream and return the syntax tree.( R t generate_tokenst readlineR) ( R t streamR R ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_stream_rawU s c C s | i | | S( s* Parse a stream and return the syntax tree.( R- ( R R, R ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_streamZ s c C s; t i | d | } z | i | | SWd | i Xd S( s( Parse a file and return the syntax tree.t rN( t codecst openR. t close( R t filenamet encodingR R, ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt
parse_file^ s c C s( t i t | i } | i | | S( s* Parse a string and return the syntax tree.( R R* t generate_linest nextR) ( R t textR R ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt parse_stringf s N(
t __name__t
__module__R R
t FalseR) R- R. R5 R9 ( ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyR s 0 c c s7 x | i t D] } | Vq Wx t o d Vq" Wd S( s< Generator that behaves like readline without using StringIO.R N( t
splitlinest True( R8 t line( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyR6 l s s Grammar.txtc C s7 | d j o t i } n | d j oZ t i i | \ } } | d j o
d } n | | d i t t t i
d } n | p t | | o~ | i d | t
i | } | oT | i d | y | i | Wqt j
o } | i d t | qXq3n t i } | i | | S( s' Load the grammar (maybe from a pickle).s .txtR t .s .pickles! Generating grammar tables from %ss Writing grammar tables to %ss Writing failed:N( R R R t ost patht splitextt joint mapt strt syst version_infot _newert infoR t generate_grammart dumpt IOErrorR t Grammart load( t gtt gpt savet forceR
t headt tailt gt e( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyR t s&
+$
c C sR t i i | p t St i i | p t St i i | t i i | j S( s0 Inquire whether file a was written since file b.( RA RB t existsR< R> t getmtime( t at b( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyRI s
( t __doc__t
__author__t __all__R0 RA R RG R R R R R R t objectR R6 R R> R< R RI ( ( ( s, /usr/lib64/python2.6/lib2to3/pgen2/driver.pyt s (P pgen2/__init__.py 0000666 00000000217 15050104230 0007662 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""The pgen2 package."""
pgen2/token.pyo 0000666 00000004352 15050104230 0007426 0 ustar 00
Lc @ s d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d
Z
d Z d Z d
Z
d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d Z d! Z! d" Z" d# Z# d$ Z$ d% Z% d& Z& d' Z' d( Z( d) Z) d* Z* d+ Z+ d, Z, d- Z- d. Z. d/ Z/ d0 Z0 d1 Z1 d2 Z2 d3 Z3 d4 Z4 d5 Z5 d6 Z6 d7 Z7 d8 Z8 d9 Z9 d: Z: h Z; xD e< i= D]3 \ Z> Z? e@ e? e@ d j o e> e; e? S(? s! Token constants (from "token.h").i i i i i i i i i i i
i i i
i i i i i i i i i i i i i i i i i i i i! i" i# i$ i% i& i' i( i) i* i+ i, i- i. i/ i0 i1 i2 i3 i4 i5 i6 i7 i8 i c C s
| t j S( N( t NT_OFFSET( t x( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt
ISTERMINALK s c C s
| t j S( N( R ( R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt
ISNONTERMINALN s c C s
| t j S( N( t ENDMARKER( R ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt ISEOFQ s N(D t __doc__R t NAMEt NUMBERt STRINGt NEWLINEt INDENTt DEDENTt LPARt RPARt LSQBt RSQBt COLONt COMMAt SEMIt PLUSt MINUSt STARt SLASHt VBARt AMPERt LESSt GREATERt EQUALt DOTt PERCENTt BACKQUOTEt LBRACEt RBRACEt EQEQUALt NOTEQUALt LESSEQUALt GREATEREQUALt TILDEt
CIRCUMFLEXt LEFTSHIFTt
RIGHTSHIFTt
DOUBLESTARt PLUSEQUALt MINEQUALt STAREQUALt
SLASHEQUALt PERCENTEQUALt
AMPEREQUALt VBAREQUALt CIRCUMFLEXEQUALt LEFTSHIFTEQUALt RIGHTSHIFTEQUALt DOUBLESTAREQUALt DOUBLESLASHt DOUBLESLASHEQUALt ATt OPt COMMENTt NLt RARROWt
ERRORTOKENt N_TOKENSR t tok_namet globalst itemst _namet _valuet typeR R R ( ( ( s+ /usr/lib64/python2.6/lib2to3/pgen2/token.pyt s pgen2/grammar.py 0000666 00000012375 15050104230 0007561 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""This module defines the data structures used to represent a grammar.
These are a bit arcane because they are derived from the data
structures used by Python's 'pgen' parser generator.
There's also a table here mapping operators to their names in the
token module; the Python tokenize module reports all operators as the
fallback token code OP, but the parser needs the actual token code.
"""
# Python imports
import pickle
# Local imports
from . import token, tokenize
class Grammar(object):
"""Pgen parsing tables tables conversion class.
Once initialized, this class supplies the grammar tables for the
parsing engine implemented by parse.py. The parsing engine
accesses the instance variables directly. The class here does not
provide initialization of the tables; several subclasses exist to
do this (see the conv and pgen modules).
The load() method reads the tables from a pickle file, which is
much faster than the other ways offered by subclasses. The pickle
file is written by calling dump() (after loading the grammar
tables using a subclass). The report() method prints a readable
representation of the tables to stdout, for debugging.
The instance variables are as follows:
symbol2number -- a dict mapping symbol names to numbers. Symbol
numbers are always 256 or higher, to distinguish
them from token numbers, which are between 0 and
255 (inclusive).
number2symbol -- a dict mapping numbers to symbol names;
these two are each other's inverse.
states -- a list of DFAs, where each DFA is a list of
states, each state is is a list of arcs, and each
arc is a (i, j) pair where i is a label and j is
a state number. The DFA number is the index into
this list. (This name is slightly confusing.)
Final states are represented by a special arc of
the form (0, j) where j is its own state number.
dfas -- a dict mapping symbol numbers to (DFA, first)
pairs, where DFA is an item from the states list
above, and first is a set of tokens that can
begin this grammar rule (represented by a dict
whose values are always 1).
labels -- a list of (x, y) pairs where x is either a token
number or a symbol number, and y is either None
or a string; the strings are keywords. The label
number is the index in this list; label numbers
are used to mark state transitions (arcs) in the
DFAs.
start -- the number of the grammar's start symbol.
keywords -- a dict mapping keyword strings to arc labels.
tokens -- a dict mapping token numbers to arc labels.
"""
def __init__(self):
self.symbol2number = {}
self.number2symbol = {}
self.states = []
self.dfas = {}
self.labels = [(0, "EMPTY")]
self.keywords = {}
self.tokens = {}
self.symbol2label = {}
self.start = 256
def dump(self, filename):
"""Dump the grammar tables to a pickle file."""
f = open(filename, "wb")
pickle.dump(self.__dict__, f, 2)
f.close()
def load(self, filename):
"""Load the grammar tables from a pickle file."""
f = open(filename, "rb")
d = pickle.load(f)
f.close()
self.__dict__.update(d)
def copy(self):
"""
Copy the grammar.
"""
new = self.__class__()
for dict_attr in ("symbol2number", "number2symbol", "dfas", "keywords",
"tokens", "symbol2label"):
setattr(new, dict_attr, getattr(self, dict_attr).copy())
new.labels = self.labels[:]
new.states = self.states[:]
new.start = self.start
return new
def report(self):
"""Dump the grammar tables to standard output, for debugging."""
from pprint import pprint
print "s2n"
pprint(self.symbol2number)
print "n2s"
pprint(self.number2symbol)
print "states"
pprint(self.states)
print "dfas"
pprint(self.dfas)
print "labels"
pprint(self.labels)
print "start", self.start
# Map from operator to number (since tokenize doesn't do this)
opmap_raw = """
( LPAR
) RPAR
[ LSQB
] RSQB
: COLON
, COMMA
; SEMI
+ PLUS
- MINUS
* STAR
/ SLASH
| VBAR
& AMPER
< LESS
> GREATER
= EQUAL
. DOT
% PERCENT
` BACKQUOTE
{ LBRACE
} RBRACE
@ AT
== EQEQUAL
!= NOTEQUAL
<> NOTEQUAL
<= LESSEQUAL
>= GREATEREQUAL
~ TILDE
^ CIRCUMFLEX
<< LEFTSHIFT
>> RIGHTSHIFT
** DOUBLESTAR
+= PLUSEQUAL
-= MINEQUAL
*= STAREQUAL
/= SLASHEQUAL
%= PERCENTEQUAL
&= AMPEREQUAL
|= VBAREQUAL
^= CIRCUMFLEXEQUAL
<<= LEFTSHIFTEQUAL
>>= RIGHTSHIFTEQUAL
**= DOUBLESTAREQUAL
// DOUBLESLASH
//= DOUBLESLASHEQUAL
-> RARROW
"""
opmap = {}
for line in opmap_raw.splitlines():
if line:
op, name = line.split()
opmap[op] = getattr(token, name)
pgen2/parse.py 0000666 00000017565 15050104230 0007253 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Parser engine for the grammar tables generated by pgen.
The grammar table must be loaded first.
See Parser/parser.c in the Python distribution for additional info on
how this parsing engine works.
"""
# Local imports
from . import token
class ParseError(Exception):
"""Exception to signal the parser is stuck."""
def __init__(self, msg, type, value, context):
Exception.__init__(self, "%s: type=%r, value=%r, context=%r" %
(msg, type, value, context))
self.msg = msg
self.type = type
self.value = value
self.context = context
class Parser(object):
"""Parser engine.
The proper usage sequence is:
p = Parser(grammar, [converter]) # create instance
p.setup([start]) # prepare for parsing
:
if p.addtoken(...): # parse a token; may raise ParseError
break
root = p.rootnode # root of abstract syntax tree
A Parser instance may be reused by calling setup() repeatedly.
A Parser instance contains state pertaining to the current token
sequence, and should not be used concurrently by different threads
to parse separate token sequences.
See driver.py for how to get input tokens by tokenizing a file or
string.
Parsing is complete when addtoken() returns True; the root of the
abstract syntax tree can then be retrieved from the rootnode
instance variable. When a syntax error occurs, addtoken() raises
the ParseError exception. There is no error recovery; the parser
cannot be used after a syntax error was reported (but it can be
reinitialized by calling setup()).
"""
def __init__(self, grammar, convert=None):
"""Constructor.
The grammar argument is a grammar.Grammar instance; see the
grammar module for more information.
The parser is not ready yet for parsing; you must call the
setup() method to get it started.
The optional convert argument is a function mapping concrete
syntax tree nodes to abstract syntax tree nodes. If not
given, no conversion is done and the syntax tree produced is
the concrete syntax tree. If given, it must be a function of
two arguments, the first being the grammar (a grammar.Grammar
instance), and the second being the concrete syntax tree node
to be converted. The syntax tree is converted from the bottom
up.
A concrete syntax tree node is a (type, value, context, nodes)
tuple, where type is the node type (a token or symbol number),
value is None for symbols and a string for tokens, context is
None or an opaque value used for error reporting (typically a
(lineno, offset) pair), and nodes is a list of children for
symbols, and None for tokens.
An abstract syntax tree node may be anything; this is entirely
up to the converter function.
"""
self.grammar = grammar
self.convert = convert or (lambda grammar, node: node)
def setup(self, start=None):
"""Prepare for parsing.
This *must* be called before starting to parse.
The optional argument is an alternative start symbol; it
defaults to the grammar's start symbol.
You can use a Parser instance to parse any number of programs;
each time you call setup() the parser is reset to an initial
state determined by the (implicit or explicit) start symbol.
"""
if start is None:
start = self.grammar.start
# Each stack entry is a tuple: (dfa, state, node).
# A node is a tuple: (type, value, context, children),
# where children is a list of nodes or None, and context may be None.
newnode = (start, None, None, [])
stackentry = (self.grammar.dfas[start], 0, newnode)
self.stack = [stackentry]
self.rootnode = None
self.used_names = set() # Aliased to self.rootnode.used_names in pop()
def addtoken(self, type, value, context):
"""Add a token; return True iff this is the end of the program."""
# Map from token to label
ilabel = self.classify(type, value, context)
# Loop until the token is shifted; may raise exceptions
while True:
dfa, state, node = self.stack[-1]
states, first = dfa
arcs = states[state]
# Look for a state with this label
for i, newstate in arcs:
t, v = self.grammar.labels[i]
if ilabel == i:
# Look it up in the list of labels
assert t < 256
# Shift a token; we're done with it
self.shift(type, value, newstate, context)
# Pop while we are in an accept-only state
state = newstate
while states[state] == [(0, state)]:
self.pop()
if not self.stack:
# Done parsing!
return True
dfa, state, node = self.stack[-1]
states, first = dfa
# Done with this token
return False
elif t >= 256:
# See if it's a symbol and if we're in its first set
itsdfa = self.grammar.dfas[t]
itsstates, itsfirst = itsdfa
if ilabel in itsfirst:
# Push a symbol
self.push(t, self.grammar.dfas[t], newstate, context)
break # To continue the outer while loop
else:
if (0, state) in arcs:
# An accepting state, pop it and try something else
self.pop()
if not self.stack:
# Done parsing, but another token is input
raise ParseError("too much input",
type, value, context)
else:
# No success finding a transition
raise ParseError("bad input", type, value, context)
def classify(self, type, value, context):
"""Turn a token into a label. (Internal)"""
if type == token.NAME:
# Keep a listing of all used names
self.used_names.add(value)
# Check for reserved words
ilabel = self.grammar.keywords.get(value)
if ilabel is not None:
return ilabel
ilabel = self.grammar.tokens.get(type)
if ilabel is None:
raise ParseError("bad token", type, value, context)
return ilabel
def shift(self, type, value, newstate, context):
"""Shift a token. (Internal)"""
dfa, state, node = self.stack[-1]
newnode = (type, value, context, None)
newnode = self.convert(self.grammar, newnode)
if newnode is not None:
node[-1].append(newnode)
self.stack[-1] = (dfa, newstate, node)
def push(self, type, newdfa, newstate, context):
"""Push a nonterminal. (Internal)"""
dfa, state, node = self.stack[-1]
newnode = (type, None, context, [])
self.stack[-1] = (dfa, newstate, node)
self.stack.append((newdfa, 0, newnode))
def pop(self):
"""Pop a nonterminal. (Internal)"""
popdfa, popstate, popnode = self.stack.pop()
newnode = self.convert(self.grammar, popnode)
if newnode is not None:
if self.stack:
dfa, state, node = self.stack[-1]
node[-1].append(newnode)
else:
self.rootnode = newnode
self.rootnode.used_names = self.used_names
pgen2/conv.py 0000666 00000022631 15050104230 0007074 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Convert graminit.[ch] spit out by pgen to Python code.
Pgen is the Python parser generator. It is useful to quickly create a
parser from a grammar file in Python's grammar notation. But I don't
want my parsers to be written in C (yet), so I'm translating the
parsing tables to Python data structures and writing a Python parse
engine.
Note that the token numbers are constants determined by the standard
Python tokenizer. The standard token module defines these numbers and
their names (the names are not used much). The token numbers are
hardcoded into the Python tokenizer and into pgen. A Python
implementation of the Python tokenizer is also available, in the
standard tokenize module.
On the other hand, symbol numbers (representing the grammar's
non-terminals) are assigned by pgen based on the actual grammar
input.
Note: this module is pretty much obsolete; the pgen module generates
equivalent grammar tables directly from the Grammar.txt input file
without having to invoke the Python pgen C program.
"""
# Python imports
import re
# Local imports
from pgen2 import grammar, token
class Converter(grammar.Grammar):
"""Grammar subclass that reads classic pgen output files.
The run() method reads the tables as produced by the pgen parser
generator, typically contained in two C files, graminit.h and
graminit.c. The other methods are for internal use only.
See the base class for more documentation.
"""
def run(self, graminit_h, graminit_c):
"""Load the grammar tables from the text files written by pgen."""
self.parse_graminit_h(graminit_h)
self.parse_graminit_c(graminit_c)
self.finish_off()
def parse_graminit_h(self, filename):
"""Parse the .h file writen by pgen. (Internal)
This file is a sequence of #define statements defining the
nonterminals of the grammar as numbers. We build two tables
mapping the numbers to names and back.
"""
try:
f = open(filename)
except IOError, err:
print "Can't open %s: %s" % (filename, err)
return False
self.symbol2number = {}
self.number2symbol = {}
lineno = 0
for line in f:
lineno += 1
mo = re.match(r"^#define\s+(\w+)\s+(\d+)$", line)
if not mo and line.strip():
print "%s(%s): can't parse %s" % (filename, lineno,
line.strip())
else:
symbol, number = mo.groups()
number = int(number)
assert symbol not in self.symbol2number
assert number not in self.number2symbol
self.symbol2number[symbol] = number
self.number2symbol[number] = symbol
return True
def parse_graminit_c(self, filename):
"""Parse the .c file writen by pgen. (Internal)
The file looks as follows. The first two lines are always this:
#include "pgenheaders.h"
#include "grammar.h"
After that come four blocks:
1) one or more state definitions
2) a table defining dfas
3) a table defining labels
4) a struct defining the grammar
A state definition has the following form:
- one or more arc arrays, each of the form:
static arc arcs__[] = {
{, },
...
};
- followed by a state array, of the form:
static state states_[] = {
{, arcs__},
...
};
"""
try:
f = open(filename)
except IOError, err:
print "Can't open %s: %s" % (filename, err)
return False
# The code below essentially uses f's iterator-ness!
lineno = 0
# Expect the two #include lines
lineno, line = lineno+1, f.next()
assert line == '#include "pgenheaders.h"\n', (lineno, line)
lineno, line = lineno+1, f.next()
assert line == '#include "grammar.h"\n', (lineno, line)
# Parse the state definitions
lineno, line = lineno+1, f.next()
allarcs = {}
states = []
while line.startswith("static arc "):
while line.startswith("static arc "):
mo = re.match(r"static arc arcs_(\d+)_(\d+)\[(\d+)\] = {$",
line)
assert mo, (lineno, line)
n, m, k = map(int, mo.groups())
arcs = []
for _ in range(k):
lineno, line = lineno+1, f.next()
mo = re.match(r"\s+{(\d+), (\d+)},$", line)
assert mo, (lineno, line)
i, j = map(int, mo.groups())
arcs.append((i, j))
lineno, line = lineno+1, f.next()
assert line == "};\n", (lineno, line)
allarcs[(n, m)] = arcs
lineno, line = lineno+1, f.next()
mo = re.match(r"static state states_(\d+)\[(\d+)\] = {$", line)
assert mo, (lineno, line)
s, t = map(int, mo.groups())
assert s == len(states), (lineno, line)
state = []
for _ in range(t):
lineno, line = lineno+1, f.next()
mo = re.match(r"\s+{(\d+), arcs_(\d+)_(\d+)},$", line)
assert mo, (lineno, line)
k, n, m = map(int, mo.groups())
arcs = allarcs[n, m]
assert k == len(arcs), (lineno, line)
state.append(arcs)
states.append(state)
lineno, line = lineno+1, f.next()
assert line == "};\n", (lineno, line)
lineno, line = lineno+1, f.next()
self.states = states
# Parse the dfas
dfas = {}
mo = re.match(r"static dfa dfas\[(\d+)\] = {$", line)
assert mo, (lineno, line)
ndfas = int(mo.group(1))
for i in range(ndfas):
lineno, line = lineno+1, f.next()
mo = re.match(r'\s+{(\d+), "(\w+)", (\d+), (\d+), states_(\d+),$',
line)
assert mo, (lineno, line)
symbol = mo.group(2)
number, x, y, z = map(int, mo.group(1, 3, 4, 5))
assert self.symbol2number[symbol] == number, (lineno, line)
assert self.number2symbol[number] == symbol, (lineno, line)
assert x == 0, (lineno, line)
state = states[z]
assert y == len(state), (lineno, line)
lineno, line = lineno+1, f.next()
mo = re.match(r'\s+("(?:\\\d\d\d)*")},$', line)
assert mo, (lineno, line)
first = {}
rawbitset = eval(mo.group(1))
for i, c in enumerate(rawbitset):
byte = ord(c)
for j in range(8):
if byte & (1< s pgen2/pgen.py 0000666 00000032732 15050104230 0007063 0 ustar 00 # Copyright 2004-2005 Elemental Security, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
# Pgen imports
from . import grammar, token, tokenize
class PgenGrammar(grammar.Grammar):
pass
class ParserGenerator(object):
def __init__(self, filename, stream=None):
close_stream = None
if stream is None:
stream = open(filename)
close_stream = stream.close
self.filename = filename
self.stream = stream
self.generator = tokenize.generate_tokens(stream.readline)
self.gettoken() # Initialize lookahead
self.dfas, self.startsymbol = self.parse()
if close_stream is not None:
close_stream()
self.first = {} # map from symbol name to set of tokens
self.addfirstsets()
def make_grammar(self):
c = PgenGrammar()
names = self.dfas.keys()
names.sort()
names.remove(self.startsymbol)
names.insert(0, self.startsymbol)
for name in names:
i = 256 + len(c.symbol2number)
c.symbol2number[name] = i
c.number2symbol[i] = name
for name in names:
dfa = self.dfas[name]
states = []
for state in dfa:
arcs = []
for label, next in state.arcs.iteritems():
arcs.append((self.make_label(c, label), dfa.index(next)))
if state.isfinal:
arcs.append((0, dfa.index(state)))
states.append(arcs)
c.states.append(states)
c.dfas[c.symbol2number[name]] = (states, self.make_first(c, name))
c.start = c.symbol2number[self.startsymbol]
return c
def make_first(self, c, name):
rawfirst = self.first[name]
first = {}
for label in rawfirst:
ilabel = self.make_label(c, label)
##assert ilabel not in first # XXX failed on <> ... !=
first[ilabel] = 1
return first
def make_label(self, c, label):
# XXX Maybe this should be a method on a subclass of converter?
ilabel = len(c.labels)
if label[0].isalpha():
# Either a symbol name or a named token
if label in c.symbol2number:
# A symbol name (a non-terminal)
if label in c.symbol2label:
return c.symbol2label[label]
else:
c.labels.append((c.symbol2number[label], None))
c.symbol2label[label] = ilabel
return ilabel
else:
# A named token (NAME, NUMBER, STRING)
itoken = getattr(token, label, None)
assert isinstance(itoken, int), label
assert itoken in token.tok_name, label
if itoken in c.tokens:
return c.tokens[itoken]
else:
c.labels.append((itoken, None))
c.tokens[itoken] = ilabel
return ilabel
else:
# Either a keyword or an operator
assert label[0] in ('"', "'"), label
value = eval(label)
if value[0].isalpha():
# A keyword
if value in c.keywords:
return c.keywords[value]
else:
c.labels.append((token.NAME, value))
c.keywords[value] = ilabel
return ilabel
else:
# An operator (any non-numeric token)
itoken = grammar.opmap[value] # Fails if unknown token
if itoken in c.tokens:
return c.tokens[itoken]
else:
c.labels.append((itoken, None))
c.tokens[itoken] = ilabel
return ilabel
def addfirstsets(self):
names = self.dfas.keys()
names.sort()
for name in names:
if name not in self.first:
self.calcfirst(name)
#print name, self.first[name].keys()
def calcfirst(self, name):
dfa = self.dfas[name]
self.first[name] = None # dummy to detect left recursion
state = dfa[0]
totalset = {}
overlapcheck = {}
for label, next in state.arcs.iteritems():
if label in self.dfas:
if label in self.first:
fset = self.first[label]
if fset is None:
raise ValueError("recursion for rule %r" % name)
else:
self.calcfirst(label)
fset = self.first[label]
totalset.update(fset)
overlapcheck[label] = fset
else:
totalset[label] = 1
overlapcheck[label] = {label: 1}
inverse = {}
for label, itsfirst in overlapcheck.iteritems():
for symbol in itsfirst:
if symbol in inverse:
raise ValueError("rule %s is ambiguous; %s is in the"
" first sets of %s as well as %s" %
(name, symbol, label, inverse[symbol]))
inverse[symbol] = label
self.first[name] = totalset
def parse(self):
dfas = {}
startsymbol = None
# MSTART: (NEWLINE | RULE)* ENDMARKER
while self.type != token.ENDMARKER:
while self.type == token.NEWLINE:
self.gettoken()
# RULE: NAME ':' RHS NEWLINE
name = self.expect(token.NAME)
self.expect(token.OP, ":")
a, z = self.parse_rhs()
self.expect(token.NEWLINE)
#self.dump_nfa(name, a, z)
dfa = self.make_dfa(a, z)
#self.dump_dfa(name, dfa)
oldlen = len(dfa)
self.simplify_dfa(dfa)
newlen = len(dfa)
dfas[name] = dfa
#print name, oldlen, newlen
if startsymbol is None:
startsymbol = name
return dfas, startsymbol
def make_dfa(self, start, finish):
# To turn an NFA into a DFA, we define the states of the DFA
# to correspond to *sets* of states of the NFA. Then do some
# state reduction. Let's represent sets as dicts with 1 for
# values.
assert isinstance(start, NFAState)
assert isinstance(finish, NFAState)
def closure(state):
base = {}
addclosure(state, base)
return base
def addclosure(state, base):
assert isinstance(state, NFAState)
if state in base:
return
base[state] = 1
for label, next in state.arcs:
if label is None:
addclosure(next, base)
states = [DFAState(closure(start), finish)]
for state in states: # NB states grows while we're iterating
arcs = {}
for nfastate in state.nfaset:
for label, next in nfastate.arcs:
if label is not None:
addclosure(next, arcs.setdefault(label, {}))
for label, nfaset in arcs.iteritems():
for st in states:
if st.nfaset == nfaset:
break
else:
st = DFAState(nfaset, finish)
states.append(st)
state.addarc(st, label)
return states # List of DFAState instances; first one is start
def dump_nfa(self, name, start, finish):
print "Dump of NFA for", name
todo = [start]
for i, state in enumerate(todo):
print " State", i, state is finish and "(final)" or ""
for label, next in state.arcs:
if next in todo:
j = todo.index(next)
else:
j = len(todo)
todo.append(next)
if label is None:
print " -> %d" % j
else:
print " %s -> %d" % (label, j)
def dump_dfa(self, name, dfa):
print "Dump of DFA for", name
for i, state in enumerate(dfa):
print " State", i, state.isfinal and "(final)" or ""
for label, next in state.arcs.iteritems():
print " %s -> %d" % (label, dfa.index(next))
def simplify_dfa(self, dfa):
# This is not theoretically optimal, but works well enough.
# Algorithm: repeatedly look for two states that have the same
# set of arcs (same labels pointing to the same nodes) and
# unify them, until things stop changing.
# dfa is a list of DFAState instances
changes = True
while changes:
changes = False
for i, state_i in enumerate(dfa):
for j in range(i+1, len(dfa)):
state_j = dfa[j]
if state_i == state_j:
#print " unify", i, j
del dfa[j]
for state in dfa:
state.unifystate(state_j, state_i)
changes = True
break
def parse_rhs(self):
# RHS: ALT ('|' ALT)*
a, z = self.parse_alt()
if self.value != "|":
return a, z
else:
aa = NFAState()
zz = NFAState()
aa.addarc(a)
z.addarc(zz)
while self.value == "|":
self.gettoken()
a, z = self.parse_alt()
aa.addarc(a)
z.addarc(zz)
return aa, zz
def parse_alt(self):
# ALT: ITEM+
a, b = self.parse_item()
while (self.value in ("(", "[") or
self.type in (token.NAME, token.STRING)):
c, d = self.parse_item()
b.addarc(c)
b = d
return a, b
def parse_item(self):
# ITEM: '[' RHS ']' | ATOM ['+' | '*']
if self.value == "[":
self.gettoken()
a, z = self.parse_rhs()
self.expect(token.OP, "]")
a.addarc(z)
return a, z
else:
a, z = self.parse_atom()
value = self.value
if value not in ("+", "*"):
return a, z
self.gettoken()
z.addarc(a)
if value == "+":
return a, z
else:
return a, a
def parse_atom(self):
# ATOM: '(' RHS ')' | NAME | STRING
if self.value == "(":
self.gettoken()
a, z = self.parse_rhs()
self.expect(token.OP, ")")
return a, z
elif self.type in (token.NAME, token.STRING):
a = NFAState()
z = NFAState()
a.addarc(z, self.value)
self.gettoken()
return a, z
else:
self.raise_error("expected (...) or NAME or STRING, got %s/%s",
self.type, self.value)
def expect(self, type, value=None):
if self.type != type or (value is not None and self.value != value):
self.raise_error("expected %s/%s, got %s/%s",
type, value, self.type, self.value)
value = self.value
self.gettoken()
return value
def gettoken(self):
tup = self.generator.next()
while tup[0] in (tokenize.COMMENT, tokenize.NL):
tup = self.generator.next()
self.type, self.value, self.begin, self.end, self.line = tup
#print token.tok_name[self.type], repr(self.value)
def raise_error(self, msg, *args):
if args:
try:
msg = msg % args
except:
msg = " ".join([msg] + map(str, args))
raise SyntaxError(msg, (self.filename, self.end[0],
self.end[1], self.line))
class NFAState(object):
def __init__(self):
self.arcs = [] # list of (label, NFAState) pairs
def addarc(self, next, label=None):
assert label is None or isinstance(label, str)
assert isinstance(next, NFAState)
self.arcs.append((label, next))
class DFAState(object):
def __init__(self, nfaset, final):
assert isinstance(nfaset, dict)
assert isinstance(iter(nfaset).next(), NFAState)
assert isinstance(final, NFAState)
self.nfaset = nfaset
self.isfinal = final in nfaset
self.arcs = {} # map from label to DFAState
def addarc(self, next, label):
assert isinstance(label, str)
assert label not in self.arcs
assert isinstance(next, DFAState)
self.arcs[label] = next
def unifystate(self, old, new):
for label, next in self.arcs.iteritems():
if next is old:
self.arcs[label] = new
def __eq__(self, other):
# Equality test -- ignore the nfaset instance variable
assert isinstance(other, DFAState)
if self.isfinal != other.isfinal:
return False
# Can't just return self.arcs == other.arcs, because that
# would invoke this method recursively, with cycles...
if len(self.arcs) != len(other.arcs):
return False
for label, next in self.arcs.iteritems():
if next is not other.arcs.get(label):
return False
return True
__hash__ = None # For Py3 compatibility.
def generate_grammar(filename="Grammar.txt"):
p = ParserGenerator(filename)
return p.make_grammar()
pgen2/pgen.pyo 0000666 00000026714 15050104230 0007245 0 ustar 00
Lc @ s d d k l Z l Z l Z d e i f d YZ d e f d YZ d e f d YZ d e f d YZ d
d Z
d S(
i ( t grammart tokent tokenizet PgenGrammarc B s e Z RS( ( t __name__t
__module__( ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR s t ParserGeneratorc B s e Z d d Z d Z d Z d Z d Z d Z d Z d Z
d Z d Z d
Z
d Z d Z d
Z d Z d d Z d Z d Z RS( c C s d } | d j o t | } | i } n | | _ | | _ t i | i | _ | i | i
\ | _ | _ | d j o | n h | _
| i d S( N( t Nonet opent closet filenamet streamR t generate_tokenst readlinet generatort gettokent parset dfast startsymbolt firstt addfirstsets( t selfR
R t close_stream( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt __init__ s
c
C s t } | i i } | i | i | i | i d | i x; | D]3 } d t | i } | | i | <| | i | t update( R R* R, R- t totalsett overlapcheckR. R/ t fsett inverset itsfirstt symbol( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR> s s8
c C s h } d } x | i t i j o x" | i t i j o | i q% W| i t i } | i t i d | i \ } } | i t i | i
| | } t | } | i | t | } | | | <| d j o
| } q q W| | f S( Nt :(
R t typeR t ENDMARKERt NEWLINER t expectR: t OPt parse_rhst make_dfaR t simplify_dfa( R R R R* t at zR, t oldlent newlen( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR s&
c s f d } f d t | | | g } x | D] } h } xU | i D]J } xA | i D]6 \ } } | d j o | | i | h qc qc WqS Wxl | i D]^ \ } }
x? | D] } | i |
j o Pq q Wt |
| } | i | | i | | q Wq= W| S( Nc s h } | | | S( N( ( R- t base( t
addclosure( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt closure s
c sX | | j o d Sd | | %ds %s -> %d( t enumerateR R# R R! R (
R R* R' R[ t todoR+ R- R. R/ t j( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt dump_nfa s
!
c C s d G| GHxp t | D]b \ } } d G| G| i o d p d GHx5 | i i D]$ \ } } d | | i | f GHqP Wq Wd S( Ns Dump of DFA fors States (final)R^ s %s -> %d( R_ R$ R R R# ( R R* R, R+ R- R. R/ ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyt dump_dfa s
c C s t } x | o t } x t | D]z \ } } xk t | d t | D]P } | | } | | j o3 | | =x | D] } | i | | qt Wt } PqI qI Wq# Wq Wd S( Ni ( t Truet FalseR_ t rangeR t
unifystate( R R, t changesR+ t state_iRa t state_jR- ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRO s
c C s | i \ } } | i d j o | | f St } t } | i | | i | xK | i d j o: | i | i \ } } | i | | i | q\ W| | f Sd S( Nt |( t parse_altR= t NFAStateRZ R ( R RP RQ t aat zz( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRM s
c C su | i \ } } xV | i d j p | i t i t i f j o) | i \ } } | i | | } q W| | f S( Nt (t [( Rp Rq ( t
parse_itemR= RH R R: t STRINGRZ ( R RP t bR( t d( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRl
s
c C s | i d j oG | i | i \ } } | i t i d | i | | | f S| i \ } } | i } | d j o | | f S| i | i | | d j o | | f S| | f Sd S( NRq t ]t +t *( Rw Rx ( R= R RM RK R RL RZ t
parse_atom( R RP RQ R= ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRr s
c C s | i d j o: | i | i \ } } | i t i d | | f S| i t i t i f j o: t } t } | i
| | i | i | | f S| i d | i | i d S( NRp t )s+ expected (...) or NAME or STRING, got %s/%s( R= R RM RK R RL RH R: Rs Rm RZ t raise_error( R RP RQ ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRy ( s
c C sg | i | j p | d j o3 | i | j o# | i d | | | i | i n | i } | i | S( Ns expected %s/%s, got %s/%s( RH R R= R{ R ( R RH R= ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRK 9 s -
c C sk | i i } x1 | d t i t i f j o | i i } q W| \ | _ | _ | _ | _ | _ d S( Ni (
R R/ R t COMMENTt NLRH R= t begint endt line( R t tup( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR A s
c G su | o; y | | } WqB d i | g t t | } qB Xn t | | i | i d | i d | i f d S( Nt i i ( t joint mapt strt SyntaxErrorR
R R ( R t msgt args( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR{ H s 'N( R R R R R0 R& R" R R> R RN Rb Rc RO RM Rl Rr Ry RK R R{ ( ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR
s$ . $
Rm c B s e Z d Z d d Z RS( c C s
g | _ d S( N( R ( R ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR S s c C s | i i | | f d S( N( R R! ( R R/ R. ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRZ V s N( R R R R RZ ( ( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyRm Q s RW c B s2 e Z d Z d Z d Z d Z d Z RS( c C s% | | _ | | j | _ h | _ d S( N( RX R$ R ( R RX t final( ( s* /usr/lib64/python2.6/lib2to3/pgen2/pgen.pyR ] s c C s | | i | s H
% pgen2/grammar.pyc 0000666 00000013617 15050104230 0007724 0 ustar 00
Lc @ s d Z d d k Z d d k l Z l Z d e f d YZ d Z h Z xD e i D]6 Z
e
o) e
i \ Z Z
e e e
e e GREATER
= EQUAL
. DOT
% PERCENT
` BACKQUOTE
{ LBRACE
} RBRACE
@ AT
== EQEQUAL
!= NOTEQUAL
<> NOTEQUAL
<= LESSEQUAL
>= GREATEREQUAL
~ TILDE
^ CIRCUMFLEX
<< LEFTSHIFT
>> RIGHTSHIFT
** DOUBLESTAR
+= PLUSEQUAL
-= MINEQUAL
*= STAREQUAL
/= SLASHEQUAL
%= PERCENTEQUAL
&= AMPEREQUAL
|= VBAREQUAL
^= CIRCUMFLEXEQUAL
<<= LEFTSHIFTEQUAL
>>= RIGHTSHIFTEQUAL
**= DOUBLESTAREQUAL
// DOUBLESLASH
//= DOUBLESLASHEQUAL
-> RARROW
( R' R t R R t objectR t opmap_rawt opmapt
splitlinest linet splitt opt nameR ( ( ( s- /usr/lib64/python2.6/lib2to3/pgen2/grammar.pyt
s
pgen2/literals.pyo 0000666 00000003414 15050104230 0010123 0 ustar 00
Lc @ s d Z d d k Z h
d d 6d d 6d d 6d d
6d d 6d
d 6d d 6d d 6d d 6d d 6Z d Z d Z d Z e d j o e n d S( s< Safely evaluate Python string literals without using eval().iNs t as t bs t fs
t ns
t rs t ts t vt 't "s \c C s | i d d \ } } t i | } | d j o | S| i d on | d } t | d j o t d | n y t | d } Wq t j
o t d | q Xn: y t | d } Wn# t j
o t d | n Xt | S( Ni i t xi s! invalid hex string escape ('\%s')i i s# invalid octal string escape ('\%s')( t groupt simple_escapest gett Nonet
startswitht lent
ValueErrort intt chr( t mt allt tailt esct hexest i( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt escape s"
c C sZ | d } | d | d j o | d } n | t | t | !} t i d t | S( Ni i s) \\(\'|\"|\\|[abfnrtv]|x.{0,2}|[0-7]{1,3})( R t ret subR ( t st q( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt
evalString( s
c C sa xZ t d D]L } t | } t | } t | } | | j o | G| G| G| GHq
q
Wd S( Ni ( t rangeR t reprR ( R t cR t e( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt test2 s
t __main__( t __doc__R R R R R# t __name__( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt s
pgen2/grammar.pyo 0000666 00000013617 15050104230 0007740 0 ustar 00
Lc @ s d Z d d k Z d d k l Z l Z d e f d YZ d Z h Z xD e i D]6 Z
e
o) e
i \ Z Z
e e e
e e GREATER
= EQUAL
. DOT
% PERCENT
` BACKQUOTE
{ LBRACE
} RBRACE
@ AT
== EQEQUAL
!= NOTEQUAL
<> NOTEQUAL
<= LESSEQUAL
>= GREATEREQUAL
~ TILDE
^ CIRCUMFLEX
<< LEFTSHIFT
>> RIGHTSHIFT
** DOUBLESTAR
+= PLUSEQUAL
-= MINEQUAL
*= STAREQUAL
/= SLASHEQUAL
%= PERCENTEQUAL
&= AMPEREQUAL
|= VBAREQUAL
^= CIRCUMFLEXEQUAL
<<= LEFTSHIFTEQUAL
>>= RIGHTSHIFTEQUAL
**= DOUBLESTAREQUAL
// DOUBLESLASH
//= DOUBLESLASHEQUAL
-> RARROW
( R' R t R R t objectR t opmap_rawt opmapt
splitlinest linet splitt opt nameR ( ( ( s- /usr/lib64/python2.6/lib2to3/pgen2/grammar.pyt
s
pgen2/literals.pyc 0000666 00000003767 15050104230 0010122 0 ustar 00
Lc @ s d Z d d k Z h
d d 6d d 6d d 6d d
6d d 6d
d 6d d 6d d 6d d 6d d 6Z d Z d Z d Z e d j o e n d S( s< Safely evaluate Python string literals without using eval().iNs t as t bs t fs
t ns
t rs t ts t vt 't "s \c C s | i d d \ } } | i d p t t i | } | d j o | S| i d on | d } t | d j o t d | n y t | d } Wqt j
o t d | qXn: y t | d } Wn# t j
o t d | n Xt | S(
Ni i s \t xi s! invalid hex string escape ('\%s')i i s# invalid octal string escape ('\%s')(
t groupt
startswitht AssertionErrort simple_escapest gett Nonet lent
ValueErrort intt chr( t mt allt tailt esct hexest i( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt escape s$
c C s | i d p$ | i d p t t | d | d } | d | d j o | d } n | i | p t t | t | t | d t | j p t | t | t | !} t i d t | S( NR R i i i i s) \\(\'|\"|\\|[abfnrtv]|x.{0,2}|[0-7]{1,3})( R R t reprt endswithR t ret subR ( t st q( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt
evalString( s 4
+$c C sa xZ t d D]L } t | } t | } t | } | | j o | G| G| G| GHq
q
Wd S( Ni ( t rangeR R R! ( R t cR t e( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt test2 s
t __main__( t __doc__R R
R R! R% t __name__( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/literals.pyt s
pgen2/__init__.pyc 0000666 00000000256 15050104230 0010030 0 ustar 00
Lc @ s
d Z d S( s The pgen2 package.N( t __doc__( ( ( s. /usr/lib64/python2.6/lib2to3/pgen2/__init__.pyt s Grammar.txt 0000666 00000014610 15050104230 0006667 0 ustar 00 # Grammar for Python
# Note: Changing the grammar specified in this file will most likely
# require corresponding changes in the parser module
# (../Modules/parsermodule.c). If you can't make the changes to
# that module yourself, please co-ordinate the required changes
# with someone who can; ask around on python-dev for help. Fred
# Drake will probably be listening there.
# NOTE WELL: You should also follow all the steps listed in PEP 306,
# "How to Change Python's Grammar"
# Commands for Kees Blom's railroad program
#diagram:token NAME
#diagram:token NUMBER
#diagram:token STRING
#diagram:token NEWLINE
#diagram:token ENDMARKER
#diagram:token INDENT
#diagram:output\input python.bla
#diagram:token DEDENT
#diagram:output\textwidth 20.04cm\oddsidemargin 0.0cm\evensidemargin 0.0cm
#diagram:rules
# Start symbols for the grammar:
# file_input is a module or sequence of commands read from an input file;
# single_input is a single interactive statement;
# eval_input is the input for the eval() and input() functions.
# NB: compound_stmt in single_input is followed by extra NEWLINE!
file_input: (NEWLINE | stmt)* ENDMARKER
single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
eval_input: testlist NEWLINE* ENDMARKER
decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
decorators: decorator+
decorated: decorators (classdef | funcdef)
funcdef: 'def' NAME parameters ['->' test] ':' suite
parameters: '(' [typedargslist] ')'
typedargslist: ((tfpdef ['=' test] ',')*
('*' [tname] (',' tname ['=' test])* [',' '**' tname] | '**' tname)
| tfpdef ['=' test] (',' tfpdef ['=' test])* [','])
tname: NAME [':' test]
tfpdef: tname | '(' tfplist ')'
tfplist: tfpdef (',' tfpdef)* [',']
varargslist: ((vfpdef ['=' test] ',')*
('*' [vname] (',' vname ['=' test])* [',' '**' vname] | '**' vname)
| vfpdef ['=' test] (',' vfpdef ['=' test])* [','])
vname: NAME
vfpdef: vname | '(' vfplist ')'
vfplist: vfpdef (',' vfpdef)* [',']
stmt: simple_stmt | compound_stmt
simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
small_stmt: (expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt |
import_stmt | global_stmt | exec_stmt | assert_stmt)
expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) |
('=' (yield_expr|testlist_star_expr))*)
testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [',']
augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |
'<<=' | '>>=' | '**=' | '//=')
# For normal assignments, additional restrictions enforced by the interpreter
print_stmt: 'print' ( [ test (',' test)* [','] ] |
'>>' test [ (',' test)+ [','] ] )
del_stmt: 'del' exprlist
pass_stmt: 'pass'
flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt
break_stmt: 'break'
continue_stmt: 'continue'
return_stmt: 'return' [testlist]
yield_stmt: yield_expr
raise_stmt: 'raise' [test ['from' test | ',' test [',' test]]]
import_stmt: import_name | import_from
import_name: 'import' dotted_as_names
import_from: ('from' ('.'* dotted_name | '.'+)
'import' ('*' | '(' import_as_names ')' | import_as_names))
import_as_name: NAME ['as' NAME]
dotted_as_name: dotted_name ['as' NAME]
import_as_names: import_as_name (',' import_as_name)* [',']
dotted_as_names: dotted_as_name (',' dotted_as_name)*
dotted_name: NAME ('.' NAME)*
global_stmt: ('global' | 'nonlocal') NAME (',' NAME)*
exec_stmt: 'exec' expr ['in' test [',' test]]
assert_stmt: 'assert' test [',' test]
compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
while_stmt: 'while' test ':' suite ['else' ':' suite]
for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
try_stmt: ('try' ':' suite
((except_clause ':' suite)+
['else' ':' suite]
['finally' ':' suite] |
'finally' ':' suite))
with_stmt: 'with' with_item (',' with_item)* ':' suite
with_item: test ['as' expr]
with_var: 'as' expr
# NB compile.c makes sure that the default except clause is last
except_clause: 'except' [test [(',' | 'as') test]]
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
# Backward compatibility cruft to support:
# [ x for x in lambda: True, lambda: False if x() ]
# even while also allowing:
# lambda x: 5 if x else 2
# (But not a mix of the two)
testlist_safe: old_test [(',' old_test)+ [',']]
old_test: or_test | old_lambdef
old_lambdef: 'lambda' [varargslist] ':' old_test
test: or_test ['if' or_test 'else' test] | lambdef
or_test: and_test ('or' and_test)*
and_test: not_test ('and' not_test)*
not_test: 'not' not_test | comparison
comparison: expr (comp_op expr)*
comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'
star_expr: '*' expr
expr: xor_expr ('|' xor_expr)*
xor_expr: and_expr ('^' and_expr)*
and_expr: shift_expr ('&' shift_expr)*
shift_expr: arith_expr (('<<'|'>>') arith_expr)*
arith_expr: term (('+'|'-') term)*
term: factor (('*'|'/'|'%'|'//') factor)*
factor: ('+'|'-'|'~') factor | power
power: atom trailer* ['**' factor]
atom: ('(' [yield_expr|testlist_gexp] ')' |
'[' [listmaker] ']' |
'{' [dictsetmaker] '}' |
'`' testlist1 '`' |
NAME | NUMBER | STRING+ | '.' '.' '.')
listmaker: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] )
testlist_gexp: test ( comp_for | (',' (test|star_expr))* [','] )
lambdef: 'lambda' [varargslist] ':' test
trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
subscriptlist: subscript (',' subscript)* [',']
subscript: test | [test] ':' [test] [sliceop]
sliceop: ':' [test]
exprlist: (expr|star_expr) (',' (expr|star_expr))* [',']
testlist: test (',' test)* [',']
dictsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) |
(test (comp_for | (',' test)* [','])) )
classdef: 'class' NAME ['(' [arglist] ')'] ':' suite
arglist: (argument ',')* (argument [',']
|'*' test (',' argument)* [',' '**' test]
|'**' test)
argument: test [comp_for] | test '=' test # Really [keyword '='] test
comp_iter: comp_for | comp_if
comp_for: 'for' exprlist 'in' testlist_safe [comp_iter]
comp_if: 'if' old_test [comp_iter]
testlist1: test (',' test)*
# not used in grammar, but may appear in "node" passed from Parser to Compiler
encoding_decl: NAME
yield_expr: 'yield' [testlist]
refactor.pyo 0000666 00000054170 15050104230 0007103 0 ustar 00
Lc @ s d Z d d k l Z d Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k
l Z d d k l
Z
l Z l Z d d k l Z l Z e d Z d
e f d YZ d Z d
Z d Z d Z e i d d f j o+ d d k Z e i Z d Z d Z n e Z e Z e Z d Z! d e f d YZ" d e# f d YZ$ d e f d YZ% d e$ f d YZ& d S( s Refactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
i( t with_statements# Guido van Rossum N( t chaini ( t drivert tokenizet token( t pytreet pygramc C s t | g g d g } t i i | i } g } xg t t i | D]P } | i d o: | i d o* | o | d } n | i | d qI qI W| S( sE Return a sorted list of all available fix names in the given package.t *t fix_s .pyi i(
t
__import__t ost patht dirnamet __file__t sortedt listdirt
startswitht endswitht append( t fixer_pkgt
remove_prefixt pkgt fixer_dirt fix_namest name( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt get_all_fix_names s t
_EveryNodec B s e Z RS( ( t __name__t
__module__( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR + s c C s t | t i t i f o+ | i d j o
t n t | i g St | t i o" | i o t
| i St n t | t i oF t } x5 | i D]* } x! | D] } | i t
| q Wq W| St
d | d S( sf Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. s$ Oh no! I don't understand pattern %sN( t
isinstanceR t NodePatternt LeafPatternt typet NoneR t sett NegatedPatternt contentt _get_head_typest WildcardPatternt updatet Exception( t patt rt pt x( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR% / s"
c C s t i t } g } x | D] } | i o\ y t | i } Wn t j
o | i | q XxX | D] } | | i | qi Wq | i d j o | | i i | q | i | q Wx: t t
i i i
t
i i D] } | | i | q Wt | S( s^ Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. N( t collectionst defaultdictt listt patternR% R R t _accept_typeR! R R t python_grammart
symbol2numbert
itervaluest tokenst extendt dict( t
fixer_listt
head_nodest everyt fixert headst node_type( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _get_headnode_dictK s(
c C s0 g } t | t D] } | | d | q ~ S( sN
Return the fully qualified names for fixers in the package pkg_name.
t .( R t False( t pkg_namet _[1]t fix_name( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt get_fixers_from_packaged s c C s | S( N( ( t obj( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _identityk s i i c C s | i d d S( Nu
u
( t replace( t input( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _from_system_newlinesr s c C s, t i d j o | i d t i S| Sd S( Ns
u
( R
t linesepRG ( RH ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _to_system_newlinest s c s t } t i t i | i f d } t t i t i t i f } t
} yx}t ou| \ } } | | j o q] q] | t i j o | o Pn t } q] | t i
j o| d j o
| \ } } | t i
j p
| d j o Pn | \ } } | t i
j p
| d j o Pn | \ } } | t i j o | d j o | \ } } n xg | t i
j oQ | i | | \ } } | t i j p
| d j o Pn | \ } } qnWq] Pq] WWn t j
o n Xt | S( Nc s i } | d | d f S( Ni i ( t next( t tok( t gen( s( /usr/lib64/python2.6/lib2to3/refactor.pyt advance s u fromu
__future__u importu (u ,( R@ R t generate_tokenst StringIOt readlinet frozensetR t NEWLINEt NLt COMMENTR" t Truet STRINGt NAMEt OPt addt
StopIteration( t sourcet have_docstringRO t ignoret featurest tpt value( ( RN s( /usr/lib64/python2.6/lib2to3/refactor.pyt _detect_future_features sH
t
FixerErrorc B s e Z d Z RS( s A fixer could not be loaded.( R R t __doc__( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRd s t RefactoringToolc B s e Z h e d 6Z d Z d Z d d d Z d Z d Z d Z
d Z d Z e e d Z
e e d
Z d Z e e d Z d
Z e d Z d Z d Z d e d d Z d d Z d Z d Z d Z d Z d Z d Z d Z d Z RS( t print_functiont FixR c C s | | _ | p g | _ | i i | _ | d j o | i i | n | i d o t i | _ n
t i
| _ g | _ t i
d | _ g | _ t | _ t i | i d t i d | i | _ | i \ | _ | _ t | i | _ t | i | _ g | _ d S( s Initializer.
Args:
fixer_names: a list of fixers to import
options: an dict with configuration.
explicit: a list of fixers to run even if they are explicit.
Rg Rf t convertt loggerN( t fixerst explicitt _default_optionst copyt optionsR! R' R t! python_grammar_no_print_statementt grammarR2 t errorst loggingt getLoggerRj t fixer_logR@ t wroteR t DriverR Ri t
get_fixerst pre_ordert
post_orderR> t pre_order_headst post_order_headst files( t selft fixer_namesRo Rl ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt __init__ s&
c
C s g } g } x| i D]} t | h h d g } | i d d d } | i | i o | t | i } n | i d } | i d i g } | D] } | | i q ~ } y t
| | }
Wn) t j
o t d | | f n X|
| i
| i } | i o7 | i t j o' | | i j o | i d | q n | i d | | i d
j o | i | q | i d j o | i | q t d | i q Wt i d
} | i d | | i d | | | f S( s Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
R R? i it _t s Can't find %s.%ss Skipping implicit fixer: %ss Adding transformation: %st pret posts Illegal fixer order: %rt run_ordert key( Rk R t rsplitR t FILE_PREFIXt lent splitt CLASS_PREFIXt joint titlet getattrt AttributeErrorRd Ro Ru Rl RW t log_messaget log_debugt orderR t operatort
attrgettert sort(
R~ t pre_order_fixerst post_order_fixerst fix_mod_patht modRC t partsRB R+ t
class_namet fix_classR; t key_func( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRx s:
7c O s d S( s Called when an error occurs.N( ( R~ t msgt argst kwds( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt log_error s c G s) | o | | } n | i i | d S( s Hook to log a message.N( Rj t info( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s c G s) | o | | } n | i i | d S( N( Rj t debug( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s c C s d S( sT Called with the old version, new version, and filename of a
refactored file.N( ( R~ t old_textt new_textt filenamet equal( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt print_output s c C sR xK | D]C } t i i | o | i | | | q | i | | | q Wd S( s) Refactor a list of files and directories.N( R
R t isdirt refactor_dirt
refactor_file( R~ t itemst writet
doctests_onlyt dir_or_file( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt refactor
s
c C s x t i | D] \ } } } | i d | | i | i xk | D]c } | i d oL t i i | d i d o, t i i | | } | i | | | qJ qJ Wg } | D]! }
|
i d p | |
q q ~ | (q Wd S( s Descends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
s Descending into %sR? i t pyN(
R
t walkR R R R t splitextR R R ( R~ t dir_nameR R t dirpatht dirnamest filenamesR t fullnameRB t dn( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s y t | d } Wn) t j
o } | i d | | d SXz t i | i d } Wd | i Xt | d d | i i
} z# | ~ } t | i | f SWd QXd S( sG
Do our best to decode a Python source file correctly.
t rbs Can't open %s: %si NR* t encoding( NN(
t opent IOErrorR R! R t detect_encodingRR t closet _open_with_encodingt __exit__t __enter__RI t read( R~ R t ft errR RB ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _read_python_source( s ,c C s | i | \ } } | d j o d S| d 7} | o` | i d | | i | | } | | j o | i | | | | | q | i d | n^ | i | | } | o4 | i o* | i t | d | d | d | n | i d | d S( s Refactors a file.Nu
s Refactoring doctests in %ss No doctest changes in %siR R s No changes in %s( R R! R t refactor_docstringt processed_filet refactor_stringt was_changedt unicode( R~ R R R RH R t outputt tree( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR 8 s
c C s t | } d | j o t i | i _ n zO y | i i | } Wn2 t j
o& } | i d | | i i | d SXWd | i | i _ X| | _
| i d | | i | | | S( sF Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
Rg s Can't parse %s: %s: %sNs Refactoring %s(
Rc R Rp R Rq t parse_stringR( R t __class__R t future_featuresR t
refactor_tree( R~ t dataR R` R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR O s
c C s t i i } | oT | i d | i | d } | | j o | i | d | q | i d nN | i | d } | o' | i o | i t | d | n | i d d S( Ns Refactoring doctests in stdins s No doctest changes in stdins No changes in stdin( t syst stdinR R R R R R R ( R~ R RH R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt refactor_stdinj s
c C s x- t | i | i D] } | i | | q W| i | i | i | i | i | i x- t | i | i D] } | i | | qx W| i S( sA Refactors a parse tree (modifying the tree in place).
Args:
tree: a pytree.Node instance representing the root of the tree
to be refactored.
name: a human-readable name for this tree.
Returns:
True if the tree was modified, False otherwise.
( R Ry Rz t
start_treet traverse_byR{ R| t finish_treeR ( R~ R R R; ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR z s c C s | p d Sxv | D]n } xe | | i D]V } | i | } | o: | i | | } | d j o | i | | } q} q' q' Wq Wd S( s Traverse an AST, applying a set of fixers to each node.
This is a helper method for refactor_tree().
Args:
fixers: a list of fixer instances.
traversal: a generator that yields AST nodes.
Returns:
None
N( R t matcht transformR! RG ( R~ Rk t traversalt nodeR; t resultst new( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s | i i | | d j o) | i | d } | d j o d Sn | | j } | i | | | | | o | i d | d S| o | i | | | | n | i d | d S( sP
Called when a file has been refactored, and there are changes.
i Ns No changes to %ss Not writing changes to %s( R} R R! R R R t
write_file( R~ R R R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s y t | d d | } Wn, t i j
o } | i d | | d SXzI y | i t | Wn+ t i j
o } | i d | | n XWd | i X| i d | t | _ d S( s Writes a string to a file.
It first shows a unified diff between the old text and the new text, and
then rewrites the file; the latter is only done if the write option is
set.
t wR s Can't create %s: %sNs Can't write %s: %ss Wrote changes to %s(
R R
t errorR R RK R R RW Rv ( R~ R R R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s s >>> s ... c
C s g } d } d } d } d } x5| i t D]$} | d 7} | i i | i o_ | d j o# | i | i | | | | n | } | g } | i | i } | | } q. | d j oF | i | | i p | | | i i
d j o | i | q. | d j o# | i | i | | | | n d } d } | i | q. W| d j o# | i | i | | | | n d i | S( s Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
i i u
u N(
R! t
splitlinesRW t lstripR t PS1R6 t refactor_doctestt findt PS2t rstripR R (
R~ RH R t resultt blockt block_linenot indentt linenot linet i( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s<
c C sb y | i | | | } Wny t j
om } | i i t i o. x+ | D] } | i d | i d qI Wn | i d | | | i i
| | SX| i | | o t | i
t } | | d | | d } } | d i d p | d c d 7>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
s
Source: %su
s+ Can't parse docstring in %s line %s: %s: %si ii ( t parse_blockR( t logt isEnabledForRs t DEBUGR R R R R R R R RW R R t popR ( R~ R R R R R R R R t clippedRB ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s& ! 8c C s' | i o
d } n d } | i p | i d | n2 | i d | x | i D] } | i | qR W| i o2 | i d x" | i D] } | i | q Wn | i ot t | i d j o | i d n | i d t | i x1 | i D]" \ } } } | i | | | q Wn d S( Nt weres
need to bes No files %s modified.s Files that %s modified:s$ Warnings/messages while refactoring:i s There was 1 error:s There were %d errors:( Rv R} R Ru Rr R ( R~ R t filet messageR R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt summarize s*
c C s1 | i i | i | | | } t | _ | S( s Parses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
( R t parse_tokenst wrap_toksRS R ( R~ R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR 1 s ! c c s t i | i | | i } xe | D]] \ } } \ } } \ } }
} | | d 7} | | d 7} | | | | f | |
f | f Vq% Wd S( s; Wraps a tokenize stream to systematically modify start/end.i N( R RP t gen_linesRL ( R~ R R R R5 R Rb t line0t col0t line1t col1t line_text( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR ; s !c c s | | i } | | i } | } xm | D]e } | i | o | t | Vn7 | | i d j o d Vn t d | | f | } q' Wx t o d Vq Wd S( s Generates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
u
s line=%r, prefix=%rR N( R R R R R t AssertionErrorRW ( R~ R R t prefix1t prefix2t prefixR ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR I s
N( R R R@ Rm R R R! R Rx R R R R R R R R R R R R R R R R R R R R R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRf s8
( +
t MultiprocessingUnsupportedc B s e Z RS( ( R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR ] s t MultiprocessRefactoringToolc B s5 e Z d Z e e d d Z d Z d Z RS( c O s/ t t | i | | d | _ d | _ d S( N( t superR R R! t queuet output_lock( R~ R t kwargs( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR c s i c
C sv | d j o t t | i | | | Sy d d k } Wn t j
o
t n X| i d j o t d n | i | _ | i
| _ g } t | D] } | | i
d | i q ~ } z; x | D] } | i q Wt t | i | | | Wd | i i x$ t | D] } | i i d q"Wx) | D]! } | i o | i qCqCWd | _ Xd S( Ni is already doing multiple processest target( R R R t multiprocessingt ImportErrorR R R! t RuntimeErrort
JoinableQueuet LockR
t xranget Processt _childt startR t putt is_alive(
R~ R R R t
num_processesR
RB R t processesR+ ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR h s8
/
c C sq | i i } x[ | d j oM | \ } } z t t | i | | Wd | i i X| i i } q Wd S( N( R t getR! R R R t task_done( R~ t taskR R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c O sG | i d j o | i i | | f n t t | i | | Sd S( N( R R! R R R R ( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s ( R R R R@ R R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR a s
(' Re t
__future__R t
__author__R
R Rs R R- RQ t itertoolsR t pgen2R R R R R R RW R R( R R% R> RD RF t version_infot codecsR R RI RK Rc Rd t objectRf R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt s>
( __init__.py 0000666 00000000007 15050104230 0006644 0 ustar 00 #empty
main.pyc 0000666 00000014745 15050104230 0006212 0 ustar 00
Lc @ s d Z d d k l Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k l
Z
d Z d e
i f d YZ
d Z d d
Z d S( s
Main program for 2to3.
i( t with_statementNi ( t refactorc C s: | i } | i } t i | | | | d d d d S( s% Return a unified diff of two strings.s
(original)s (refactored)t linetermt ( t
splitlinest difflibt unified_diff( t at bt filename( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt
diff_texts s
t StdoutRefactoringToolc B s2 e Z d Z d Z d Z d Z d Z RS( s"
Prints output to stdout.
c C s2 | | _ | | _ t t | i | | | d S( N( t nobackupst
show_diffst superR t __init__( t selft fixerst optionst explicitR R
( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR s c O s3 | i i | | | f | i i | | | d S( N( t errorst appendt loggert error( R t msgt argst kwargs( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt log_error$ s c C s | i p | d } t i i | o@ y t i | Wqg t i j
o } | i d | qg Xn y t i | | Wq t i j
o } | i d | | q Xn t t | i
} | | | | | | i p t i | | n d S( Ns .baks Can't remove backup %ss Can't rename %s to %s(
R t ost patht lexistst removeR t log_messaget renameR R t
write_filet shutilt copymode( R t new_textR t old_textt encodingt backupt errt write( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR" ( s
c
C s | o | i d | n | i d | | i o t | | | } yl | i d j oB | i i i z' x | D] } | GHqv Wt i i Wd QXn x | D] } | GHq WWq t
j
o t d | f d SXn d S( Ns No changes to %ss
Refactored %ss+ couldn't encode %s's diff for your terminal( R R
R
t output_lockt Nonet __exit__t __enter__t syst stdoutt flusht UnicodeEncodeErrort warn( R t oldt newR t equalt
diff_linest line( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt print_output; s&
( t __name__t
__module__t __doc__R R R" R9 ( ( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR s
c C s t i d | f IJd S( Ns WARNING: %s( R/ t stderr( R ( ( s$ /usr/lib64/python2.6/lib2to3/main.pyR3 Q s c
s) t i d d } | i d d d d d d | i d d
d d d g d d
| i d d d d d d d d d d | i d d d d d g d d | i d d d d d d | i d d d d d d | i d d d d d d | i d! d d d d" | i d# d$ d d d d% | i d& d' d d d t d d( t } h } | i | \ } } | i o | i o t d) n | i o | i o | i d* n | i
o4 d+ GHx t i D] } | GHqW| p d, Sn | p t
i d- IJt
i d. IJd/ Sd0 | j o& t } | i o t
i d1 IJd/ Sn | i o t | d2 s t alls .fix_s+ Sorry, -j isn't supported on this platform.(( t optparset OptionParsert
add_optiont Falset
parse_argsR* t no_diffsR3 R R t
list_fixesR t get_all_fix_namesR/ R= t TrueRG t verboset loggingt DEBUGt INFOt basicConfigt sett get_fixers_from_packaget nofixRK t addt uniont
differenceR t sortedR t refactor_stdint
doctests_onlyt processest MultiprocessingUnsupportedt AssertionErrort summarizeRE t bool( RL R t parserRc t flagsR t fixnameRI t avail_fixest unwanted_fixesR t all_presentRK t requestedt fixer_namest rt( ( RL s$ /usr/lib64/python2.6/lib2to3/main.pyt mainU s
!
( R< t
__future__R R/ R R RX R# RN R R R
t MultiprocessRefactoringToolR R3 R, Rs ( ( ( s$ /usr/lib64/python2.6/lib2to3/main.pyt s 7 fixer_util.py 0000666 00000033621 15050104230 0007267 0 ustar 00 """Utility functions, node construction macros, etc."""
# Author: Collin Winter
# Local imports
from .pgen2 import token
from .pytree import Leaf, Node
from .pygram import python_symbols as syms
from . import patcomp
###########################################################
### Common node-construction "macros"
###########################################################
def KeywordArg(keyword, value):
return Node(syms.argument,
[keyword, Leaf(token.EQUAL, u'='), value])
def LParen():
return Leaf(token.LPAR, u"(")
def RParen():
return Leaf(token.RPAR, u")")
def Assign(target, source):
"""Build an assignment statement"""
if not isinstance(target, list):
target = [target]
if not isinstance(source, list):
source.prefix = u" "
source = [source]
return Node(syms.atom,
target + [Leaf(token.EQUAL, u"=", prefix=u" ")] + source)
def Name(name, prefix=None):
"""Return a NAME leaf"""
return Leaf(token.NAME, name, prefix=prefix)
def Attr(obj, attr):
"""A node tuple for obj.attr"""
return [obj, Node(syms.trailer, [Dot(), attr])]
def Comma():
"""A comma leaf"""
return Leaf(token.COMMA, u",")
def Dot():
"""A period (.) leaf"""
return Leaf(token.DOT, u".")
def ArgList(args, lparen=LParen(), rparen=RParen()):
"""A parenthesised argument list, used by Call()"""
node = Node(syms.trailer, [lparen.clone(), rparen.clone()])
if args:
node.insert_child(1, Node(syms.arglist, args))
return node
def Call(func_name, args=None, prefix=None):
"""A function call"""
node = Node(syms.power, [func_name, ArgList(args)])
if prefix is not None:
node.prefix = prefix
return node
def Newline():
"""A newline literal"""
return Leaf(token.NEWLINE, u"\n")
def BlankLine():
"""A blank line"""
return Leaf(token.NEWLINE, u"")
def Number(n, prefix=None):
return Leaf(token.NUMBER, n, prefix=prefix)
def Subscript(index_node):
"""A numeric or string subscript"""
return Node(syms.trailer, [Leaf(token.LBRACE, u'['),
index_node,
Leaf(token.RBRACE, u']')])
def String(string, prefix=None):
"""A string leaf"""
return Leaf(token.STRING, string, prefix=prefix)
def ListComp(xp, fp, it, test=None):
"""A list comprehension of the form [xp for fp in it if test].
If test is None, the "if test" part is omitted.
"""
xp.prefix = u""
fp.prefix = u" "
it.prefix = u" "
for_leaf = Leaf(token.NAME, u"for")
for_leaf.prefix = u" "
in_leaf = Leaf(token.NAME, u"in")
in_leaf.prefix = u" "
inner_args = [for_leaf, fp, in_leaf, it]
if test:
test.prefix = u" "
if_leaf = Leaf(token.NAME, u"if")
if_leaf.prefix = u" "
inner_args.append(Node(syms.comp_if, [if_leaf, test]))
inner = Node(syms.listmaker, [xp, Node(syms.comp_for, inner_args)])
return Node(syms.atom,
[Leaf(token.LBRACE, u"["),
inner,
Leaf(token.RBRACE, u"]")])
def FromImport(package_name, name_leafs):
""" Return an import statement in the form:
from package import name_leafs"""
# XXX: May not handle dotted imports properly (eg, package_name='foo.bar')
#assert package_name == '.' or '.' not in package_name, "FromImport has "\
# "not been tested with dotted package names -- use at your own "\
# "peril!"
for leaf in name_leafs:
# Pull the leaves out of their old tree
leaf.remove()
children = [Leaf(token.NAME, u'from'),
Leaf(token.NAME, package_name, prefix=u" "),
Leaf(token.NAME, u'import', prefix=u" "),
Node(syms.import_as_names, name_leafs)]
imp = Node(syms.import_from, children)
return imp
###########################################################
### Determine whether a node represents a given literal
###########################################################
def is_tuple(node):
"""Does the node represent a tuple literal?"""
if isinstance(node, Node) and node.children == [LParen(), RParen()]:
return True
return (isinstance(node, Node)
and len(node.children) == 3
and isinstance(node.children[0], Leaf)
and isinstance(node.children[1], Node)
and isinstance(node.children[2], Leaf)
and node.children[0].value == u"("
and node.children[2].value == u")")
def is_list(node):
"""Does the node represent a list literal?"""
return (isinstance(node, Node)
and len(node.children) > 1
and isinstance(node.children[0], Leaf)
and isinstance(node.children[-1], Leaf)
and node.children[0].value == u"["
and node.children[-1].value == u"]")
###########################################################
### Misc
###########################################################
def parenthesize(node):
return Node(syms.atom, [LParen(), node, RParen()])
consuming_calls = set(["sorted", "list", "set", "any", "all", "tuple", "sum",
"min", "max"])
def attr_chain(obj, attr):
"""Follow an attribute chain.
If you have a chain of objects where a.foo -> b, b.foo-> c, etc,
use this to iterate over all objects in the chain. Iteration is
terminated by getattr(x, attr) is None.
Args:
obj: the starting object
attr: the name of the chaining attribute
Yields:
Each successive object in the chain.
"""
next = getattr(obj, attr)
while next:
yield next
next = getattr(next, attr)
p0 = """for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
"""
p1 = """
power<
( 'iter' | 'list' | 'tuple' | 'sorted' | 'set' | 'sum' |
'any' | 'all' | (any* trailer< '.' 'join' >) )
trailer< '(' node=any ')' >
any*
>
"""
p2 = """
power<
'sorted'
trailer< '(' arglist ')' >
any*
>
"""
pats_built = False
def in_special_context(node):
""" Returns true if node is in an environment where all that is required
of it is being itterable (ie, it doesn't matter if it returns a list
or an itterator).
See test_map_nochange in test_fixers.py for some examples and tests.
"""
global p0, p1, p2, pats_built
if not pats_built:
p1 = patcomp.compile_pattern(p1)
p0 = patcomp.compile_pattern(p0)
p2 = patcomp.compile_pattern(p2)
pats_built = True
patterns = [p0, p1, p2]
for pattern, parent in zip(patterns, attr_chain(node, "parent")):
results = {}
if pattern.match(parent, results) and results["node"] is node:
return True
return False
def is_probably_builtin(node):
"""
Check that something isn't an attribute or function name etc.
"""
prev = node.prev_sibling
if prev is not None and prev.type == token.DOT:
# Attribute lookup.
return False
parent = node.parent
if parent.type in (syms.funcdef, syms.classdef):
return False
if parent.type == syms.expr_stmt and parent.children[0] is node:
# Assignment.
return False
if parent.type == syms.parameters or \
(parent.type == syms.typedargslist and (
(prev is not None and prev.type == token.COMMA) or
parent.children[0] is node
)):
# The name of an argument.
return False
return True
###########################################################
### The following functions are to find bindings in a suite
###########################################################
def make_suite(node):
if node.type == syms.suite:
return node
node = node.clone()
parent, node.parent = node.parent, None
suite = Node(syms.suite, [node])
suite.parent = parent
return suite
def find_root(node):
"""Find the top level namespace."""
# Scamper up to the top level namespace
while node.type != syms.file_input:
assert node.parent, "Tree is insane! root found before "\
"file_input node was found."
node = node.parent
return node
def does_tree_import(package, name, node):
""" Returns true if name is imported from package at the
top level of the tree which node belongs to.
To cover the case of an import like 'import foo', use
None for the package and 'foo' for the name. """
binding = find_binding(name, find_root(node), package)
return bool(binding)
def is_import(node):
"""Returns true if the node is an import statement."""
return node.type in (syms.import_name, syms.import_from)
def touch_import(package, name, node):
""" Works like `does_tree_import` but adds an import statement
if it was not imported. """
def is_import_stmt(node):
return node.type == syms.simple_stmt and node.children and \
is_import(node.children[0])
root = find_root(node)
if does_tree_import(package, name, root):
return
# figure out where to insert the new import. First try to find
# the first import and then skip to the last one.
insert_pos = offset = 0
for idx, node in enumerate(root.children):
if not is_import_stmt(node):
continue
for offset, node2 in enumerate(root.children[idx:]):
if not is_import_stmt(node2):
break
insert_pos = idx + offset
break
# if there are no imports where we can insert, find the docstring.
# if that also fails, we stick to the beginning of the file
if insert_pos == 0:
for idx, node in enumerate(root.children):
if node.type == syms.simple_stmt and node.children and \
node.children[0].type == token.STRING:
insert_pos = idx + 1
break
if package is None:
import_ = Node(syms.import_name, [
Leaf(token.NAME, u'import'),
Leaf(token.NAME, name, prefix=u' ')
])
else:
import_ = FromImport(package, [Leaf(token.NAME, name, prefix=u' ')])
children = [import_, Newline()]
root.insert_child(insert_pos, Node(syms.simple_stmt, children))
_def_syms = set([syms.classdef, syms.funcdef])
def find_binding(name, node, package=None):
""" Returns the node which binds variable name, otherwise None.
If optional argument package is supplied, only imports will
be returned.
See test cases for examples."""
for child in node.children:
ret = None
if child.type == syms.for_stmt:
if _find(name, child.children[1]):
return child
n = find_binding(name, make_suite(child.children[-1]), package)
if n: ret = n
elif child.type in (syms.if_stmt, syms.while_stmt):
n = find_binding(name, make_suite(child.children[-1]), package)
if n: ret = n
elif child.type == syms.try_stmt:
n = find_binding(name, make_suite(child.children[2]), package)
if n:
ret = n
else:
for i, kid in enumerate(child.children[3:]):
if kid.type == token.COLON and kid.value == ":":
# i+3 is the colon, i+4 is the suite
n = find_binding(name, make_suite(child.children[i+4]), package)
if n: ret = n
elif child.type in _def_syms and child.children[1].value == name:
ret = child
elif _is_import_binding(child, name, package):
ret = child
elif child.type == syms.simple_stmt:
ret = find_binding(name, child, package)
elif child.type == syms.expr_stmt:
if _find(name, child.children[0]):
ret = child
if ret:
if not package:
return ret
if is_import(ret):
return ret
return None
_block_syms = set([syms.funcdef, syms.classdef, syms.trailer])
def _find(name, node):
nodes = [node]
while nodes:
node = nodes.pop()
if node.type > 256 and node.type not in _block_syms:
nodes.extend(node.children)
elif node.type == token.NAME and node.value == name:
return node
return None
def _is_import_binding(node, name, package=None):
""" Will reuturn node if node will import name, or node
will import * from package. None is returned otherwise.
See test cases for examples. """
if node.type == syms.import_name and not package:
imp = node.children[1]
if imp.type == syms.dotted_as_names:
for child in imp.children:
if child.type == syms.dotted_as_name:
if child.children[2].value == name:
return node
elif child.type == token.NAME and child.value == name:
return node
elif imp.type == syms.dotted_as_name:
last = imp.children[-1]
if last.type == token.NAME and last.value == name:
return node
elif imp.type == token.NAME and imp.value == name:
return node
elif node.type == syms.import_from:
# unicode(...) is used to make life easier here, because
# from a.b import parses to ['import', ['a', '.', 'b'], ...]
if package and unicode(node.children[1]).strip() != package:
return None
n = node.children[3]
if package and _find(u'as', n):
# See test_from_import_as for explanation
return None
elif n.type == syms.import_as_names and _find(name, n):
return node
elif n.type == syms.import_as_name:
child = n.children[2]
if child.type == token.NAME and child.value == name:
return node
elif n.type == token.NAME and n.value == name:
return node
elif package and n.type == token.STAR:
return node
return None
fixer_base.py 0000666 00000014351 15050104230 0007223 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Base class for fixers (optional, but recommended)."""
# Python imports
import logging
import itertools
# Local imports
from .patcomp import PatternCompiler
from . import pygram
from .fixer_util import does_tree_import
class BaseFix(object):
"""Optional base class for fixers.
The subclass name must be FixFooBar where FooBar is the result of
removing underscores and capitalizing the words of the fix name.
For example, the class name for a fixer named 'has_key' should be
FixHasKey.
"""
PATTERN = None # Most subclasses should override with a string literal
pattern = None # Compiled pattern, set by compile_pattern()
options = None # Options object passed to initializer
filename = None # The filename (set by set_filename)
logger = None # A logger (set by set_filename)
numbers = itertools.count(1) # For new_name()
used_names = set() # A set of all used NAMEs
order = "post" # Does the fixer prefer pre- or post-order traversal
explicit = False # Is this ignored by refactor.py -f all?
run_order = 5 # Fixers will be sorted by run order before execution
# Lower numbers will be run first.
_accept_type = None # [Advanced and not public] This tells RefactoringTool
# which node type to accept when there's not a pattern.
# Shortcut for access to Python grammar symbols
syms = pygram.python_symbols
def __init__(self, options, log):
"""Initializer. Subclass may override.
Args:
options: an dict containing the options passed to RefactoringTool
that could be used to customize the fixer through the command line.
log: a list to append warnings and other messages to.
"""
self.options = options
self.log = log
self.compile_pattern()
def compile_pattern(self):
"""Compiles self.PATTERN into self.pattern.
Subclass may override if it doesn't want to use
self.{pattern,PATTERN} in .match().
"""
if self.PATTERN is not None:
self.pattern = PatternCompiler().compile_pattern(self.PATTERN)
def set_filename(self, filename):
"""Set the filename, and a logger derived from it.
The main refactoring tool should call this.
"""
self.filename = filename
self.logger = logging.getLogger(filename)
def match(self, node):
"""Returns match for a given parse tree node.
Should return a true or false object (not necessarily a bool).
It may return a non-empty dict of matching sub-nodes as
returned by a matching pattern.
Subclass may override.
"""
results = {"node": node}
return self.pattern.match(node, results) and results
def transform(self, node, results):
"""Returns the transformation for a given parse tree node.
Args:
node: the root of the parse tree that matched the fixer.
results: a dict mapping symbolic names to part of the match.
Returns:
None, or a node that is a modified copy of the
argument node. The node argument may also be modified in-place to
effect the same change.
Subclass *must* override.
"""
raise NotImplementedError()
def new_name(self, template=u"xxx_todo_changeme"):
"""Return a string suitable for use as an identifier
The new name is guaranteed not to conflict with other identifiers.
"""
name = template
while name in self.used_names:
name = template + unicode(self.numbers.next())
self.used_names.add(name)
return name
def log_message(self, message):
if self.first_log:
self.first_log = False
self.log.append("### In file %s ###" % self.filename)
self.log.append(message)
def cannot_convert(self, node, reason=None):
"""Warn the user that a given chunk of code is not valid Python 3,
but that it cannot be converted automatically.
First argument is the top-level node for the code in question.
Optional second argument is why it can't be converted.
"""
lineno = node.get_lineno()
for_output = node.clone()
for_output.prefix = u""
msg = "Line %d: could not convert: %s"
self.log_message(msg % (lineno, for_output))
if reason:
self.log_message(reason)
def warning(self, node, reason):
"""Used for warning the user about possible uncertainty in the
translation.
First argument is the top-level node for the code in question.
Optional second argument is why it can't be converted.
"""
lineno = node.get_lineno()
self.log_message("Line %d: %s" % (lineno, reason))
def start_tree(self, tree, filename):
"""Some fixers need to maintain tree-wide state.
This method is called once, at the start of tree fix-up.
tree - the root node of the tree to be processed.
filename - the name of the file the tree came from.
"""
self.used_names = tree.used_names
self.set_filename(filename)
self.numbers = itertools.count(1)
self.first_log = True
def finish_tree(self, tree, filename):
"""Some fixers need to maintain tree-wide state.
This method is called once, at the conclusion of tree fix-up.
tree - the root node of the tree to be processed.
filename - the name of the file the tree came from.
"""
pass
class ConditionalFix(BaseFix):
""" Base class for fixers which not execute if an import is found. """
# This is the name of the import which, if found, will cause the test to be skipped
skip_on = None
def start_tree(self, *args):
super(ConditionalFix, self).start_tree(*args)
self._should_skip = None
def should_skip(self, node):
if self._should_skip is not None:
return self._should_skip
pkg = self.skip_on.split(".")
name = pkg[-1]
pkg = ".".join(pkg[:-1])
self._should_skip = does_tree_import(pkg, name, node)
return self._should_skip
fixes/fix_map.pyc 0000666 00000005733 15050104230 0010024 0 ustar 00
Lc @ su d Z d d k l Z d d k l Z d d k l Z l Z l Z l Z d d k
l Z d e i
f d YZ d S( s Fixer that changes map(F, ...) into list(map(F, ...)) unless there
exists a 'from future_builtins import map' statement in the top-level
namespace.
As a special case, map(None, X) is changed into list(X). (This is
necessary because the semantics are changed in this case -- the new
map(None, X) is equivalent to [(x,) for x in X].)
We avoid the transformation (except for the special case mentioned
above) if the map() call is directly contained in iter(<>), list(<>),
tuple(<>), sorted(<>), ...join(<>), or for V in <>:.
NOTE: This is still not correct if the original code was depending on
map(F, X, Y, ...) to go on until the longest argument is exhausted,
substituting None for missing values -- like zip(), it now stops as
soon as the shortest argument is exhausted.
i ( t token( t
fixer_base( t Namet Callt ListCompt in_special_context( t python_symbolst FixMapc B s e Z d Z d Z d Z RS( s
map_none=power<
'map'
trailer< '(' arglist< 'None' ',' arg=any [','] > ')' >
>
|
map_lambda=power<
'map'
trailer<
'('
arglist<
lambdef< 'lambda'
(fp=NAME | vfpdef< '(' fp=NAME ')'> ) ':' xp=any
>
','
it=any
>
')'
>
>
|
power<
'map' trailer< '(' [arglist=any] ')' >
>
s future_builtins.mapc C s | i | o d S| i i t i j oA | i | d | i } d | _ t t d | g } nd | j o4 t
| d i | d i | d i } n d | j o | d i } n d
| j og | d
} | i t i j oF | i d i t
i j o, | i d i d j o | i | d
d Sn t | o d S| i } d | _ t t d | g } | i | _ | S( Ns You should use a for loop hereu u listt
map_lambdat xpt fpt itt map_nonet argt arglisti t Nonesj cannot convert map(None, ...) with multiple arguments because map() now truncates to the shortest sequence( t should_skipt parentt typet symst simple_stmtt warningt clonet prefixR R R R t childrenR t NAMEt valueR R ( t selft nodet resultst newt args( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_map.pyt transform: s6
( t __name__t
__module__t PATTERNt skip_onR ( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_map.pyR s N( t __doc__t pgen2R t R t
fixer_utilR R R R t pygramR R t ConditionalFixR ( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_map.pyt s
" fixes/fix_basestring.pyc 0000666 00000001404 15050104230 0011377 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s Fixer for basestring -> str.i ( t
fixer_base( t Namet
FixBasestringc B s e Z d Z d Z RS( s 'basestring'c C s t d d | i S( Nu strt prefix( R R ( t selft nodet results( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_basestring.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_basestring.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_basestring.pyt s fixes/fix_next.pyo 0000666 00000006750 15050104230 0010241 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z l
Z
l Z d Z d e i
f d YZ d Z d
Z d Z d S(
s. Fixer for it.next() -> next(it), per PEP 3114.i ( t token( t python_symbols( t
fixer_base( t Namet Callt find_bindings; Calls to builtin next() possibly shadowed by global bindingt FixNextc B s& e Z d Z d Z d Z d Z RS( s
power< base=any+ trailer< '.' attr='next' > trailer< '(' ')' > >
|
power< head=any+ trailer< '.' attr='next' > not trailer< '(' ')' > >
|
classdef< 'class' any+ ':'
suite< any*
funcdef< 'def'
name='next'
parameters< '(' NAME ')' > any+ >
any* > >
|
global=global_stmt< 'global' any* 'next' any* >
t prec C sY t t | i | | t d | } | o | i | t t | _ n
t | _ d S( Nu next( t superR t
start_treeR t warningt bind_warningt Truet
shadowed_nextt False( t selft treet filenamet n( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR # s
c
C s | i d } | i d } | i d } | o | i o | i t d d | i qg } | D] } | | i qi ~ } d | d _ | i t t d d | i | n | o& t d d | i } | i | n | o t | o` | d } d
i g } | D] } | t | q~ i
d j o | i | t n d S| i t d n+ d | j o | i | t t
| _ n d S(
Nt baset attrt nameu __next__t prefixu i u nextt headt u __builtin__t global( t getR
t replaceR R t cloneR t is_assign_targett joint strt stripR
R R (
R t nodet resultsR R R t _[1]R R t _[2]( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyt transform- s,
'
)
=
( t __name__t
__module__t PATTERNt orderR R% ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR s
c C sc t | } | d j o t Sx> | i D]3 } | i t i j o t St | | o t Sq( Wt S( N( t find_assignt NoneR t childrent typeR t EQUALt
is_subtreeR ( R! t assignt child( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR P s
c C sM | i t i j o | S| i t i j p | i d j o d St | i S( N( R- t symst expr_stmtt simple_stmtt parentR+ R* ( R! ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR* \ s
#c s/ | j o t St f d | i D S( Nc 3 s" x | ] } t | Vq Wd S( N( R/ ( t .0t c( R! ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pys f s ( R t anyR, ( t rootR! ( ( R! s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR/ c s
N( t __doc__t pgen2R t pygramR R2 R R t
fixer_utilR R R R t BaseFixR R R* R/ ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyt s ? fixes/fix_callable.py 0000666 00000002130 15050104230 0010627 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for callable().
This converts callable(obj) into isinstance(obj, collections.Callable), adding a
collections import if needed."""
# Local imports
from lib2to3 import fixer_base
from lib2to3.fixer_util import Call, Name, String, Attr, touch_import
class FixCallable(fixer_base.BaseFix):
# Ignore callable(*args) or use of keywords.
# Either could be a hint that the builtin callable() is not being used.
PATTERN = """
power< 'callable'
trailer< lpar='('
( not(arglist | argument) any ','> )
rpar=')' >
after=any*
>
"""
def transform(self, node, results):
func = results['func']
touch_import(None, u'collections', node=node)
args = [func.clone(), String(u', ')]
args.extend(Attr(Name(u'collections'), Name(u'Callable')))
return Call(Name(u'isinstance'), args, prefix=node.prefix)
fixes/fix_nonzero.pyo 0000666 00000002051 15050104230 0010743 0 ustar 00
Lc @ sI d Z d d k l Z d d k l Z l Z d e i f d YZ d S( s* Fixer for __nonzero__ -> __bool__ methods.i ( t
fixer_base( t Namet symst
FixNonzeroc B s e Z d Z d Z RS( s
classdef< 'class' any+ ':'
suite< any*
funcdef< 'def' name='__nonzero__'
parameters< '(' NAME ')' > any+ >
any* > >
c C s0 | d } t d d | i } | i | d S( Nt nameu __bool__t prefix( R R t replace( t selft nodet resultsR t new( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_nonzero.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_nonzero.pyR s N( t __doc__t R t
fixer_utilR R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_nonzero.pyt s fixes/fix_raise.pyo 0000666 00000004504 15050104230 0010361 0 ustar 00
Lc @ s{ d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z d e i f d YZ
d S( s- Fixer for 'raise E, V, T'
raise -> raise
raise E -> raise E
raise E, V -> raise E(V)
raise E, V, T -> raise E(V).with_traceback(T)
raise (((E, E'), E''), E'''), V -> raise E(V)
raise "foo", V, T -> warns about string exceptions
CAVEATS:
1) "raise E, V" will be incorrectly translated if V is an exception
instance. The correct Python 3 idiom is
raise E from V
but since we can't detect instance-hood by syntax alone and since
any client code would have to be changed as well, we don't automate
this.
i ( t pytree( t token( t
fixer_base( t Namet Callt Attrt ArgListt is_tuplet FixRaisec B s e Z d Z d Z RS( sB
raise_stmt< 'raise' exc=any [',' val=any [',' tb=any]] >
c
C s | i } | d i } | i t i j o | i | d d St | o< x, t | o | i d i d i } qQ Wd | _ n d | j o2 t i
| i t d | g } | i | _ | S| d i } t | o5 g } | i d d !D] } | | i q ~ } n d | _ | g } d
| j o | d
i }
d |
_ t
| | } t | t d t |
g g } t i
| i t d g | } | i | _ | St i
| i t d t
| | g d | i Sd S(
Nt excs+ Python 3 does not support string exceptionsi i t t valu raiseiu t tbu with_tracebackt prefix( t symst clonet typeR t STRINGt cannot_convertR t childrenR
R t Nodet
raise_stmtR R R R t simple_stmt(
t selft nodet resultsR R t newR t _[1]t ct argsR t et with_tb( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_raise.pyt transform$ s<
!
5
%" ( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_raise.pyR s N( t __doc__t R t pgen2R R t
fixer_utilR R R R R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_raise.pyt s
( fixes/fix_idioms.pyc 0000666 00000010724 15050104230 0010527 0 ustar 00
Lc @ sm d Z d d k l Z d d k l Z l Z l Z l Z l Z l Z d Z
d Z d e i f d YZ
d S( s Adjust some old Python 2 idioms to their modern counterparts.
* Change some type comparisons to isinstance() calls:
type(x) == T -> isinstance(x, T)
type(x) is T -> isinstance(x, T)
type(x) != T -> not isinstance(x, T)
type(x) is not T -> not isinstance(x, T)
* Change "while 1:" into "while True:".
* Change both
v = list(EXPR)
v.sort()
foo(v)
and the more general
v = EXPR
v.sort()
foo(v)
into
v = sorted(EXPR)
foo(v)
i ( t
fixer_base( t Callt Commat Namet Nodet BlankLinet symss0 (n='!=' | '==' | 'is' | n=comp_op< 'is' 'not' >)s( power< 'type' trailer< '(' x=any ')' > >t FixIdiomsc B sQ e Z e Z d e e e e f Z d Z d Z d Z d Z
d Z RS( s
isinstance=comparison< %s %s T=any >
|
isinstance=comparison< T=any %s %s >
|
while_stmt< 'while' while='1' ':' any+ >
|
sorted=any<
any*
simple_stmt<
expr_stmt< id1=any '='
power< list='list' trailer< '(' (not arglist) any ')' > >
>
'\n'
>
sort=
simple_stmt<
power< id2=any
trailer< '.' 'sort' > trailer< '(' ')' >
>
'\n'
>
next=any*
>
|
sorted=any<
any*
simple_stmt< expr_stmt< id1=any '=' expr=any > '\n' >
sort=
simple_stmt<
power< id2=any
trailer< '.' 'sort' > trailer< '(' ')' >
>
'\n'
>
next=any*
>
c C sO t t | i | } | o, d | j o | d | d j o | Sd S| S( Nt sortedt id1t id2( t superR t matcht None( t selft nodet r( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR P s c C sj d | j o | i | | Sd | j o | i | | Sd | j o | i | | St d d S( Nt
isinstancet whileR s
Invalid match( t transform_isinstancet transform_whilet transform_sortt RuntimeError( R R t results( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyt transform[ s
c C s | d i } | d i } d | _ d | _ t t d | t | g } d | j o+ d | _ t t i t d | g } n | i | _ | S( Nt xt Tu u u
isinstancet nu not( t clonet prefixR R R R R t not_test( R R R R R t test( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR e s !
" c C s* | d } | i t d d | i d S( NR u TrueR ( t replaceR R ( R R R t one( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR q s
c C s | d } | d } | i d } | i d } | o | i t d d | i nU | oA | i } d | _ | i t t d | g d | i n
t d | i | i } d | j o | o: | i d d
| d
i f } d i | | d
_ q| i
p t | i d j p t t }
| i
i |
| i |
j p t | i d d
|
_ n d S( Nt sortt nextt listt expru sortedR u s should not have reached hereu
i ( t getR R R R R R t removet
rpartitiont joint parentt AssertionErrort next_siblingR
R t append_child( R R R t sort_stmtt next_stmtt list_callt simple_exprt newt btwnt prefix_linest end_line( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR u s0
( t __name__t
__module__t Truet explicitt TYPEt CMPt PATTERNR R R R R ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR % s '
N( t __doc__t R t
fixer_utilR R R R R R R; R: t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyt s
. fixes/fix_urllib.pyc 0000666 00000014503 15050104230 0010533 0 ustar 00
Lc @ sg d Z d d k l Z l Z d d k l Z d d k l Z l Z l Z l
Z
l Z h d d d d d
d d d
d g f d d d d d d d d d d d d d d d d g f d d g f g d! 6d d d" d# d$ d% d& d' d( d) d* d+ d, d- d. d/ d0 d1 d2 d3 d4 d5 d6 d7 d8 g f d d9 d: g f g d; 6Z e d; i
e d! d d< Z d= e f d> YZ d? S(@ s Fix changes imports of urllib which are now incompatible.
This is rather similar to fix_imports, but because of the more
complex nature of the fixing for urllib, it has its own fixer.
i ( t
alternatest
FixImportsi ( t
fixer_base( t Namet Commat
FromImportt Newlinet
attr_chains urllib.requestt URLOpenert FancyURLOpenert urlretrievet
_urlopenert urlopent
urlcleanupt pathname2urlt url2pathnames urllib.parset quotet
quote_plust unquotet unquote_plust urlencodet splitattrt splithostt
splitnportt splitpasswdt splitportt
splitqueryt splittagt splittypet splitusert
splitvalues urllib.errort ContentTooShortErrort urllibt install_openert build_openert Requestt OpenerDirectort BaseHandlert HTTPDefaultErrorHandlert HTTPRedirectHandlert HTTPCookieProcessort ProxyHandlert HTTPPasswordMgrt HTTPPasswordMgrWithDefaultRealmt AbstractBasicAuthHandlert HTTPBasicAuthHandlert ProxyBasicAuthHandlert AbstractDigestAuthHandlert HTTPDigestAuthHandlert ProxyDigestAuthHandlert HTTPHandlert HTTPSHandlert FileHandlert
FTPHandlert CacheFTPHandlert UnknownHandlert URLErrort HTTPErrort urllib2c c s t } x t i D]w \ } } xh | D]` } | \ } } t | } d | | f Vd | | | f Vd | Vd | Vd | | f Vq) Wq Wd S( Ns import_name< 'import' (module=%r
| dotted_as_names< any* module=%r any* >) >
s import_from< 'from' mod_member=%r 'import'
( member=%s | import_as_name< member=%s 'as' any > |
import_as_names< members=any* >) >
sI import_from< 'from' module_star=%r 'import' star='*' >
st import_name< 'import'
dotted_as_name< module_as=%r 'as' any > >
sK power< bare_with_attr=%r trailer< '.' member=%s > any* >
( t sett MAPPINGt itemsR ( t baret
old_modulet changest changet
new_modulet members( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt
build_pattern0 s
t FixUrllibc B s5 e Z d Z d Z d Z d Z d Z RS( c C s d i t S( Nt |( t joinRD ( t self( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyRD I s c C s | i d } | i } g } x? t | i d D], } | i t | d d | t g q0 W| i t t | i d d d | | i | d S( s Transform for the basic import case. Replaces the old
import name with a comma separated list of its
replacements.
t moduleii t prefixN( t getRJ R< t valuet extendR R t appendt replace( RH t nodet resultst
import_modt preft namest name( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt transform_importL s *(c C s | i d } | i } | i d } | o t | t o | d } n d } x8 t | i D]) } | i | d j o | d } Pq` q` W| o | i t | d | q| i | d ng } h } | i d }
x |
D] } | i } | d j o} xz t | i D]g } | | d j oP | d | j o | | d i
| qx| g | | d <| i
| d qqWq q Wg } x | D] } | | }
g } x4 |
d D]( } | i t | d | t g qW| i
t |
d d | | i
t
| | qW| oS g } x( | d D] } | i | t g q-W| i
| d | i | n | i | d
d S( s Transform for imports of specific module elements. Replaces
the module to be imported from with the appropriate new
module.
t
mod_membert memberi i RJ s! This is an invalid module elementRC t ,is All module elements are invalidN( RK RJ t
isinstancet listt NoneR< RL RO R t cannot_convertRN RM R R R ( RH RP RQ RW RS RX t new_nameRA t modulest mod_dictRC t new_nodesRI t eltsRT t eltt nodest new_node( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt transform_member\ s`
!
& c C s | i d } | i d } d } t | t o | d } n x8 t | i D]) } | i | d j o | d } PqP qP W| o | i t | d | i n | i | d d S( s. Transform for calls to module members in code.t bare_with_attrRX i i RJ s! This is an invalid module elementN(
RK R\ RZ R[ R< RL RO R RJ R] ( RH RP RQ t
module_dotRX R^ RA ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt
transform_dot s
c C s | i d o | i | | n | i d o | i | | nm | i d o | i | | nI | i d o | i | d n% | i d o | i | d n d S( NRI RW Rg t module_stars Cannot handle star imports.t module_ass# This module is now multiple modules( RK RV Rf Ri R] ( RH RP RQ ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt transform s ( t __name__t
__module__RD RV Rf Ri Rl ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyRE G s
< N( t __doc__t fix_importsR R t R t
fixer_utilR R R R R R< RN RD RE ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt sD ( fixes/fix_input.py 0000666 00000001255 15050104230 0010236 0 ustar 00 """Fixer that changes input(...) into eval(input(...))."""
# Author: Andre Roberge
# Local imports
from .. import fixer_base
from ..fixer_util import Call, Name
from .. import patcomp
context = patcomp.compile_pattern("power< 'eval' trailer< '(' any ')' > >")
class FixInput(fixer_base.BaseFix):
PATTERN = """
power< 'input' args=trailer< '(' [any] ')' > >
"""
def transform(self, node, results):
# If we're already wrapped in a eval() call, we're done.
if context.match(node.parent.parent):
return
new = node.clone()
new.prefix = u""
return Call(Name(u"eval"), [new], prefix=node.prefix)
fixes/fix_filter.pyo 0000666 00000004303 15050104230 0010540 0 ustar 00
Lc @ se d Z d d k l Z d d k l Z d d k l Z l Z l Z l Z d e i
f d YZ d S( s Fixer that changes filter(F, X) into list(filter(F, X)).
We avoid the transformation if the filter() call is directly contained
in iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or
for V in <>:.
NOTE: This is still not correct if the original code was depending on
filter(F, X) to return a string if X is a string and a tuple if X is a
tuple. That would require type inference, which we don't do. Let
Python 2.6 figure it out.
i ( t token( t
fixer_base( t Namet Callt ListCompt in_special_contextt FixFilterc B s e Z d Z d Z d Z RS( s
filter_lambda=power<
'filter'
trailer<
'('
arglist<
lambdef< 'lambda'
(fp=NAME | vfpdef< '(' fp=NAME ')'> ) ':' xp=any
>
','
it=any
>
')'
>
>
|
power<
'filter'
trailer< '(' arglist< none='None' ',' seq=any > ')' >
>
|
power<
'filter'
args=trailer< '(' [any] ')' >
>
s future_builtins.filterc C s | i | o d Sd | j oU t | i d i | i d i | i d i | i d i } n d | j o5 t t d t d | d i t d } n@ t | o d S| i } d | _ t t d | g } | i | _ | S(
Nt
filter_lambdat fpt itt xpt noneu _ft sequ u list( t should_skipR t gett cloneR R t Nonet prefixR ( t selft nodet resultst new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_filter.pyt transform4 s&
( t __name__t
__module__t PATTERNt skip_onR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_filter.pyR s N( t __doc__t pgen2R t R t
fixer_utilR R R R t ConditionalFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_filter.pyt s " fixes/fix_tuple_params.pyc 0000666 00000012557 15050104230 0011745 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z l Z d Z
d e i f d YZ d Z d
Z g d
d Z d Z d
S( s: Fixer for function definitions with tuple parameters.
def func(((a, b), c), d):
...
->
def func(x, d):
((a, b), c) = x
...
It will also support lambdas:
lambda (x, y): x + y -> lambda t: t[0] + t[1]
# The parens are a syntax error in Python 3
lambda (x): x + y -> lambda x: x + y
i ( t pytree( t token( t
fixer_base( t Assignt Namet Newlinet Numbert Subscriptt symsc C s* t | t i o | i d i t i j S( Ni ( t
isinstanceR t Nodet childrent typeR t STRING( t stmt( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyt is_docstring s t FixTupleParamsc B s e Z d Z d Z d Z RS( s
funcdef< 'def' any parameters< '(' args=any ')' >
['->' any] ':' suite=any+ >
|
lambda=
lambdef< 'lambda' args=vfpdef< '(' inner=any ')' >
':' body=any
>
c s@ d | j o i | | Sg | d } | d } | d i d i t i j o' d } | d i d i } t n" d } d } t i t i d t f d } | i t
i j o | | ne | i t
i j oQ xN t
| i D]9 \ } } | i t
i j o | | d
| d j q q Wn p d Sx D] }
| d |
_ qNW| } | d j o d d _ n4 t | d i | o | d _ | d } n x D] }
| d |
_ qW | d i | | +x= t | d | t d D] } | | d i | _ qW| d i d S( Nt lambdat suitet argsi i i u ; u c s t i } | i } d | _ t | | i } | o
d | _ n | i | i t i t i
| i g d S( Nu u ( R t new_namet clonet prefixR t replacet appendR R
R t simple_stmt( t tuple_argt
add_prefixt nt argR ( t new_linest endt self( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyt handle_tuple? s
R u ( t transform_lambdaR R R t INDENTt valueR R t Leaft FalseR t tfpdeft
typedargslistt enumeratet parentR R t ranget lent changed( R t nodet resultsR R t startt indentR! t iR t linet after( ( R R R s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyt transform* sN
"
" c C s[ | d } | d } t | d } | i t i j o' | i } d | _ | i | d St | } t | } | i t
| } t | d d } | i | i x | i D] }
|
i t i j o} |
i
| j om g } | |
i
D] } | | i q ~ }
t i t i | i g |
} |
i | _ |
i | q q Wd S( NR t bodyt inneru R ( t
simplify_argsR R t NAMER R R t find_paramst map_to_indexR t
tuple_nameR t
post_orderR$ R R
R t power( R R. R/ R R6 R7 t paramst to_indext tup_namet new_paramR t _[1]t ct
subscriptst new( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR" j s*
#. ( t __name__t
__module__t PATTERNR5 R" ( ( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR s @c C su | i t i t i f j o | S| i t i j o- x% | i t i j o | i d } q7 W| St d | d S( Ni s Received unexpected node %s( R R t vfplistR R9 t vfpdefR t RuntimeError( R. ( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR8 s c C s | i t i j o t | i d S| i t i j o | i Sg } | i D]* } | i t i j o | t | qN qN ~ S( Ni ( R R RK R: R R R9 R$ t COMMA( R. RC RD ( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR: s
c C s | d j o
h } n xj t | D]\ \ } } t t t | g } t | t o t | | d | q$ | | | | s . h fixes/fix_imports2.pyc 0000666 00000001172 15050104230 0011017 0 ustar 00
Lc @ sG d Z d d k l Z h d d 6d d 6Z d e i f d YZ d S( sT Fix incompatible imports and module references that must be fixed after
fix_imports.i ( t fix_importst dbmt whichdbt anydbmt FixImports2c B s e Z d Z e Z RS( i ( t __name__t
__module__t run_ordert MAPPINGt mapping( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_imports2.pyR s N( t __doc__t R R t
FixImportsR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_imports2.pyt s
fixes/fix_numliterals.py 0000666 00000001405 15050104230 0011433 0 ustar 00 """Fixer that turns 1L into 1, 0755 into 0o755.
"""
# Copyright 2007 Georg Brandl.
# Licensed to PSF under a Contributor Agreement.
# Local imports
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Number
class FixNumliterals(fixer_base.BaseFix):
# This is so simple that we don't need the pattern compiler.
_accept_type = token.NUMBER
def match(self, node):
# Override
return (node.value.startswith(u"0") or node.value[-1] in u"Ll")
def transform(self, node, results):
val = node.value
if val[-1] in u'Ll':
val = val[:-1]
elif val.startswith(u'0') and val.isdigit() and len(set(val)) > 1:
val = u"0o" + val[1:]
return Number(val, prefix=node.prefix)
fixes/fix_has_key.py 0000666 00000006202 15050104230 0010517 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for has_key().
Calls to .has_key() methods are expressed in terms of the 'in'
operator:
d.has_key(k) -> k in d
CAVEATS:
1) While the primary target of this fixer is dict.has_key(), the
fixer will change any has_key() method call, regardless of its
class.
2) Cases like this will not be converted:
m = d.has_key
if m(k):
...
Only *calls* to has_key() are converted. While it is possible to
convert the above to something like
m = d.__contains__
if m(k):
...
this is currently not done.
"""
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, parenthesize
class FixHasKey(fixer_base.BaseFix):
PATTERN = """
anchor=power<
before=any+
trailer< '.' 'has_key' >
trailer<
'('
( not(arglist | argument) arg=any ','>
)
')'
>
after=any*
>
|
negation=not_test<
'not'
anchor=power<
before=any+
trailer< '.' 'has_key' >
trailer<
'('
( not(arglist | argument) arg=any ','>
)
')'
>
>
>
"""
def transform(self, node, results):
assert results
syms = self.syms
if (node.parent.type == syms.not_test and
self.pattern.match(node.parent)):
# Don't transform a node matching the first alternative of the
# pattern when its parent matches the second alternative
return None
negation = results.get("negation")
anchor = results["anchor"]
prefix = node.prefix
before = [n.clone() for n in results["before"]]
arg = results["arg"].clone()
after = results.get("after")
if after:
after = [n.clone() for n in after]
if arg.type in (syms.comparison, syms.not_test, syms.and_test,
syms.or_test, syms.test, syms.lambdef, syms.argument):
arg = parenthesize(arg)
if len(before) == 1:
before = before[0]
else:
before = pytree.Node(syms.power, before)
before.prefix = u" "
n_op = Name(u"in", prefix=u" ")
if negation:
n_not = Name(u"not", prefix=u" ")
n_op = pytree.Node(syms.comp_op, (n_not, n_op))
new = pytree.Node(syms.comparison, (arg, n_op, before))
if after:
new = parenthesize(new)
new = pytree.Node(syms.power, (new,) + tuple(after))
if node.parent.type in (syms.comparison, syms.expr, syms.xor_expr,
syms.and_expr, syms.shift_expr,
syms.arith_expr, syms.term,
syms.factor, syms.power):
new = parenthesize(new)
new.prefix = prefix
return new
fixes/fix_execfile.pyo 0000666 00000003731 15050104230 0011043 0 ustar 00
Lc @ sy d Z d d k l Z d d k l Z l Z l Z l Z l Z l Z l
Z
l Z l Z l
Z
d e i f d YZ d S( so Fixer for execfile.
This converts usages of the execfile function into calls to the built-in
exec() function.
i ( t
fixer_base(
t Commat Namet Callt LParent RParent Dott Nodet ArgListt Stringt symst FixExecfilec B s e Z d Z d Z RS( s
power< 'execfile' trailer< '(' arglist< filename=any [',' globals=any [',' locals=any ] ] > ')' > >
|
power< 'execfile' trailer< '(' filename=any ')' > >
c C s | d } | i d } | i d } | i d i d i } t | i g d | } t t i t d | g } t t i t t d g t t i t
t g g } | g | }
| i } d | _ t
d d } |
t | t | g }
t t d
|
d } | g } | d j o | i t | i g n | d j o | i t | i g n t t d | d
| i S( Nt filenamet globalst localsit rparenu openu readu u 'exec'u compileu u exect prefix( t gett childrent cloneR R R
t powerR t trailerR R R R R R R t Nonet extend( t selft nodet resultsR R
R t execfile_parent open_argst open_callt readt open_exprt filename_argt exec_strt compile_argst compile_callt args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_execfile.pyt transform s(
!
( t __name__t
__module__t PATTERNR% ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_execfile.pyR s N( t __doc__t R t
fixer_utilR R R R R R R R R R
t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_execfile.pyt s F fixes/fix_raw_input.pyo 0000666 00000001623 15050104230 0011265 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s2 Fixer that changes raw_input(...) into input(...).i ( t
fixer_base( t Namet FixRawInputc B s e Z d Z d Z RS( sU
power< name='raw_input' trailer< '(' [any] ')' > any* >
c C s* | d } | i t d d | i d S( Nt nameu inputt prefix( t replaceR R ( t selft nodet resultsR ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_raw_input.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_raw_input.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_raw_input.pyt s fixes/fix_xrange.pyc 0000666 00000005774 15050104230 0010540 0 ustar 00
Lc @ s_ d Z d d k l Z d d k l Z l Z l Z d d k l Z d e i f d YZ d S( s/ Fixer that changes xrange(...) into range(...).i ( t
fixer_base( t Namet Callt consuming_calls( t patcompt FixXrangec B sn e Z d Z d Z d Z d Z d Z d Z d Z e i
e Z d Z e i
e Z
d Z RS( s
power<
(name='range'|name='xrange') trailer< '(' args=any ')' >
rest=any* >
c C s) t t | i | | t | _ d S( N( t superR t
start_treet sett transformed_xranges( t selft treet filename( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR s c C s
d | _ d S( N( t NoneR ( R
R R ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyt finish_tree s c C sb | d } | i d j o | i | | S| i d j o | i | | St t | d S( Nt nameu xrangeu range( t valuet transform_xranget transform_ranget
ValueErrort repr( R
t nodet resultsR ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyt transform s
c C s@ | d } | i t d d | i | i i t | d S( NR u ranget prefix( t replaceR R R t addt id( R
R R R ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR $ s
c C s t | | i j o{ | i | oj t t d | d i g } t t d | g d | i } x | d D] } | i | qu W| Sd S( Nu ranget argsu listR t rest( R R t in_special_contextR R t cloneR t append_child( R
R R t
range_callt list_callt n( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR * s " s3 power< func=NAME trailer< '(' node=any ')' > any* >s for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
| comparison< any 'in' node=any any*>
c C s | i d j o t Sh } | i i d j o? | i i | i i | o# | d | j o | d i t j S| i i | i | o | d | j S( NR t func( t parentR
t Falset p1t matchR R t p2( R
R R ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR ? s ( t __name__t
__module__t PATTERNR R R R R t P1R t compile_patternR' t P2R) R ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR s N(
t __doc__t R t
fixer_utilR R R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyt s fixes/fix_renames.pyc 0000666 00000004623 15050104230 0010676 0 ustar 00
Lc @ su d Z d d k l Z d d k l Z l Z h h d d 6d 6Z h Z d Z d Z d e i
f d
YZ d S( s? Fix incompatible renames
Fixes:
* sys.maxint -> sys.maxsize
i ( t
fixer_base( t Namet
attr_chaint maxsizet maxintt sysc C s d d i t t | d S( Nt (t |t )( t joint mapt repr( t members( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt
alternates s c c so xh t i D]Z \ } } xK | i D]= \ } } | t | | f ) >
s^
power< module_name=%r trailer< '.' attr_name=%r > any* >
( t MAPPINGt itemst LOOKUP( t modulet replacet old_attrt new_attr( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt
build_pattern s
t
FixRenamesc B s2 e Z d i e Z d Z d Z d Z RS( R t prec sY t t | i | } | o0 t f d t | d D o t S| St S( Nc 3 s x | ] } | Vq Wd S( N( ( t .0t obj( t match( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pys 4 s t parent( t superR R t anyR t False( t selft nodet results( ( R s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyR 0 s &c C sl | i d } | i d } | oC | o< t t | i | i f } | i t | d | i n d S( Nt module_namet attr_namet prefix( t gett unicodeR t valueR R R$ ( R R R! t mod_nameR# R ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt transform= s
( t __name__t
__module__R R t PATTERNt orderR R) ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyR * s
N( t __doc__t R t
fixer_utilR R R R R
R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt s fixes/fix_basestring.pyo 0000666 00000001404 15050104230 0011413 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s Fixer for basestring -> str.i ( t
fixer_base( t Namet
FixBasestringc B s e Z d Z d Z RS( s 'basestring'c C s t d d | i S( Nu strt prefix( R R ( t selft nodet results( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_basestring.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_basestring.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_basestring.pyt s fixes/fix_types.py 0000666 00000003366 15050104230 0010250 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for removing uses of the types module.
These work for only the known names in the types module. The forms above
can include types. or not. ie, It is assumed the module is imported either as:
import types
from types import ... # either * or specific types
The import statements are not modified.
There should be another fixer that handles at least the following constants:
type([]) -> list
type(()) -> tuple
type('') -> str
"""
# Local imports
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name
_TYPE_MAPPING = {
'BooleanType' : 'bool',
'BufferType' : 'memoryview',
'ClassType' : 'type',
'ComplexType' : 'complex',
'DictType': 'dict',
'DictionaryType' : 'dict',
'EllipsisType' : 'type(Ellipsis)',
#'FileType' : 'io.IOBase',
'FloatType': 'float',
'IntType': 'int',
'ListType': 'list',
'LongType': 'int',
'ObjectType' : 'object',
'NoneType': 'type(None)',
'NotImplementedType' : 'type(NotImplemented)',
'SliceType' : 'slice',
'StringType': 'bytes', # XXX ?
'StringTypes' : 'str', # XXX ?
'TupleType': 'tuple',
'TypeType' : 'type',
'UnicodeType': 'str',
'XRangeType' : 'range',
}
_pats = ["power< 'types' trailer< '.' name='%s' > >" % t for t in _TYPE_MAPPING]
class FixTypes(fixer_base.BaseFix):
PATTERN = '|'.join(_pats)
def transform(self, node, results):
new_value = unicode(_TYPE_MAPPING.get(results["name"].value))
if new_value:
return Name(new_value, prefix=node.prefix)
return None
fixes/fix_exitfunc.py 0000666 00000004625 15050104230 0010730 0 ustar 00 """
Convert use of sys.exitfunc to use the atexit module.
"""
# Author: Benjamin Peterson
from lib2to3 import pytree, fixer_base
from lib2to3.fixer_util import Name, Attr, Call, Comma, Newline, syms
class FixExitfunc(fixer_base.BaseFix):
PATTERN = """
(
sys_import=import_name<'import'
('sys'
|
dotted_as_names< (any ',')* 'sys' (',' any)* >
)
>
|
expr_stmt<
power< 'sys' trailer< '.' 'exitfunc' > >
'=' func=any >
)
"""
def __init__(self, *args):
super(FixExitfunc, self).__init__(*args)
def start_tree(self, tree, filename):
super(FixExitfunc, self).start_tree(tree, filename)
self.sys_import = None
def transform(self, node, results):
# First, find a the sys import. We'll just hope it's global scope.
if "sys_import" in results:
if self.sys_import is None:
self.sys_import = results["sys_import"]
return
func = results["func"].clone()
func.prefix = u""
register = pytree.Node(syms.power,
Attr(Name(u"atexit"), Name(u"register"))
)
call = Call(register, [func], node.prefix)
node.replace(call)
if self.sys_import is None:
# That's interesting.
self.warning(node, "Can't find sys import; Please add an atexit "
"import at the top of your file.")
return
# Now add an atexit import after the sys import.
names = self.sys_import.children[1]
if names.type == syms.dotted_as_names:
names.append_child(Comma())
names.append_child(Name(u"atexit", u" "))
else:
containing_stmt = self.sys_import.parent
position = containing_stmt.children.index(self.sys_import)
stmt_container = containing_stmt.parent
new_import = pytree.Node(syms.import_name,
[Name(u"import"), Name(u"atexit", u" ")]
)
new = pytree.Node(syms.simple_stmt, [new_import])
containing_stmt.insert_child(position + 1, Newline())
containing_stmt.insert_child(position + 2, new)
fixes/fix_dict.pyo 0000666 00000007121 15050104230 0010177 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l
Z
l Z l Z l
Z
d d k l Z e i e d g BZ d e i f d
YZ d S( sj Fixer for dict methods.
d.keys() -> list(d.keys())
d.items() -> list(d.items())
d.values() -> list(d.values())
d.iterkeys() -> iter(d.keys())
d.iteritems() -> iter(d.items())
d.itervalues() -> iter(d.values())
d.viewkeys() -> d.keys()
d.viewitems() -> d.items()
d.viewvalues() -> d.values()
Except in certain very specific contexts: the iter() can be dropped
when the context is list(), sorted(), iter() or for...in; the list()
can be dropped when the context is list() or sorted() (but not iter()
or for...in!). Special contexts that apply to both: list(), sorted(), tuple()
set(), any(), all(), sum().
Note: iter(d.keys()) could be written as iter(d) but since the
original d.iterkeys() was also redundant we don't fix this. And there
are (rare) contexts where it makes a difference (e.g. when passing it
as an argument to a function that introspects the argument).
i ( t pytree( t patcomp( t token( t
fixer_base( t Namet Callt LParent RParent ArgListt Dot( t
fixer_utilt itert FixDictc B sJ e Z d Z d Z d Z e i e Z d Z e i e Z d Z
RS( s
power< head=any+
trailer< '.' method=('keys'|'items'|'values'|
'iterkeys'|'iteritems'|'itervalues'|
'viewkeys'|'viewitems'|'viewvalues') >
parens=trailer< '(' ')' >
tail=any*
>
c
C s | d } | d d } | d } | i } | i } | i d } | i d } | p | o | d } n g }
| D] } |
| i qy ~
} g } | D] } | | i q ~ } | o | i | | }
| t i | i t t | d | i
g | d i g } t i | i | } |
p | p3 d
| _
t t | o d n d | g } n | o t i | i | g | } n | i
| _
| S( Nt headt methodi t tailu iteru viewi t prefixt parensu u list(
t symst valuet
startswitht clonet in_special_contextR t Nodet trailerR R R t powerR ( t selft nodet resultsR
R R R t method_namet isitert isviewt _[1]t nt _[2]t specialt argst new( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyt transform5 s2
'' * s3 power< func=NAME trailer< '(' node=any ')' > any* >sm for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
c C s | i d j o t Sh } | i i d j o^ | i i | i i | oB | d | j o1 | o | d i t j S| d i t i j Sn | p t S| i i | i | o | d | j S( NR t func(
t parentt Nonet Falset p1t matchR t iter_exemptR
t consuming_callst p2( R R R R ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyR Y s ( t __name__t
__module__t PATTERNR& t P1R t compile_patternR+ t P2R/ R ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyR * s N( t __doc__t R R t pgen2R R R
R R R R R R R. t setR- t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyt s . fixes/fix_nonzero.py 0000666 00000001075 15050104230 0010571 0 ustar 00 """Fixer for __nonzero__ -> __bool__ methods."""
# Author: Collin Winter
# Local imports
from .. import fixer_base
from ..fixer_util import Name, syms
class FixNonzero(fixer_base.BaseFix):
PATTERN = """
classdef< 'class' any+ ':'
suite< any*
funcdef< 'def' name='__nonzero__'
parameters< '(' NAME ')' > any+ >
any* > >
"""
def transform(self, node, results):
name = results["name"]
new = Name(u"__bool__", prefix=name.prefix)
name.replace(new)
fixes/fix_callable.pyc 0000666 00000002646 15050104230 0011006 0 ustar 00
Lc @ s[ d Z d d k l Z d d k l Z l Z l Z l Z l Z d e i f d YZ
d S( s Fixer for callable().
This converts callable(obj) into isinstance(obj, collections.Callable), adding a
collections import if needed.i( t
fixer_base( t Callt Namet Stringt Attrt touch_importt FixCallablec B s e Z d Z d Z RS( s
power< 'callable'
trailer< lpar='('
( not(arglist | argument) any ','> )
rpar=')' >
after=any*
>
c C ss | d } t d d d | | i t d g } | i t t d t d t t d | d | i S( Nt funcu collectionst nodeu , u Callableu
isinstancet prefix( R t Nonet cloneR t extendR R R R ( t selfR t resultsR t args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_callable.pyt transform s
"( t __name__t
__module__t PATTERNR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_callable.pyR
s N( t __doc__t lib2to3R t lib2to3.fixer_utilR R R R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_callable.pyt s ( fixes/fix_renames.pyo 0000666 00000004623 15050104230 0010712 0 ustar 00
Lc @ su d Z d d k l Z d d k l Z l Z h h d d 6d 6Z h Z d Z d Z d e i
f d
YZ d S( s? Fix incompatible renames
Fixes:
* sys.maxint -> sys.maxsize
i ( t
fixer_base( t Namet
attr_chaint maxsizet maxintt sysc C s d d i t t | d S( Nt (t |t )( t joint mapt repr( t members( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt
alternates s c c so xh t i D]Z \ } } xK | i D]= \ } } | t | | f ) >
s^
power< module_name=%r trailer< '.' attr_name=%r > any* >
( t MAPPINGt itemst LOOKUP( t modulet replacet old_attrt new_attr( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt
build_pattern s
t
FixRenamesc B s2 e Z d i e Z d Z d Z d Z RS( R t prec sY t t | i | } | o0 t f d t | d D o t S| St S( Nc 3 s x | ] } | Vq Wd S( N( ( t .0t obj( t match( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pys 4 s t parent( t superR R t anyR t False( t selft nodet results( ( R s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyR 0 s &c C sl | i d } | i d } | oC | o< t t | i | i f } | i t | d | i n d S( Nt module_namet attr_namet prefix( t gett unicodeR t valueR R R$ ( R R R! t mod_nameR# R ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt transform= s
( t __name__t
__module__R R t PATTERNt orderR R) ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyR * s
N( t __doc__t R t
fixer_utilR R R R R
R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_renames.pyt s fixes/fix_apply.pyo 0000666 00000003437 15050104230 0010407 0 ustar 00
Lc @ so d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z d e i
f d YZ d S( sI Fixer for apply().
This converts apply(func, v, k) into (func)(*v, **k).i ( t pytree( t token( t
fixer_base( t Callt Commat parenthesizet FixApplyc B s e Z d Z d Z RS( s.
power< 'apply'
trailer<
'('
arglist<
(not argument
')'
>
>
c C sR | i } | d } | d } | i d } | i } | i } | i t i | i f j o= | i | i j p | i d i t i
j o t | } n d | _ | i } d | _ | d j o | i } d | _ n t
i t i d | g } | d j o9 | i t t
i t i
d | g d | d _ n t | | d | S(
Nt funct argst kwdsit u *u **u t prefix( t symst getR t clonet typeR t NAMEt atomt powert childrent
DOUBLESTARR t NoneR t Leaft STARt extendR R ( t selft nodet resultsR R R R R t l_newargs( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_apply.pyt transform s.
( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_apply.pyR s N( t __doc__R
R t pgen2R R t
fixer_utilR R R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_apply.pyt s
fixes/fix_raise.py 0000666 00000005000 15050104230 0010172 0 ustar 00 """Fixer for 'raise E, V, T'
raise -> raise
raise E -> raise E
raise E, V -> raise E(V)
raise E, V, T -> raise E(V).with_traceback(T)
raise (((E, E'), E''), E'''), V -> raise E(V)
raise "foo", V, T -> warns about string exceptions
CAVEATS:
1) "raise E, V" will be incorrectly translated if V is an exception
instance. The correct Python 3 idiom is
raise E from V
but since we can't detect instance-hood by syntax alone and since
any client code would have to be changed as well, we don't automate
this.
"""
# Author: Collin Winter
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, Call, Attr, ArgList, is_tuple
class FixRaise(fixer_base.BaseFix):
PATTERN = """
raise_stmt< 'raise' exc=any [',' val=any [',' tb=any]] >
"""
def transform(self, node, results):
syms = self.syms
exc = results["exc"].clone()
if exc.type is token.STRING:
self.cannot_convert(node, "Python 3 does not support string exceptions")
return
# Python 2 supports
# raise ((((E1, E2), E3), E4), E5), V
# as a synonym for
# raise E1, V
# Since Python 3 will not support this, we recurse down any tuple
# literals, always taking the first element.
if is_tuple(exc):
while is_tuple(exc):
# exc.children[1:-1] is the unparenthesized tuple
# exc.children[1].children[0] is the first element of the tuple
exc = exc.children[1].children[0].clone()
exc.prefix = " "
if "val" not in results:
# One-argument raise
new = pytree.Node(syms.raise_stmt, [Name(u"raise"), exc])
new.prefix = node.prefix
return new
val = results["val"].clone()
if is_tuple(val):
args = [c.clone() for c in val.children[1:-1]]
else:
val.prefix = u""
args = [val]
if "tb" in results:
tb = results["tb"].clone()
tb.prefix = u""
e = Call(exc, args)
with_tb = Attr(e, Name(u'with_traceback')) + [ArgList([tb])]
new = pytree.Node(syms.simple_stmt, [Name(u"raise")] + with_tb)
new.prefix = node.prefix
return new
else:
return pytree.Node(syms.raise_stmt,
[Name(u"raise"), Call(exc, args)],
prefix=node.prefix)
fixes/fix_imports2.py 0000666 00000000441 15050104230 0010652 0 ustar 00 """Fix incompatible imports and module references that must be fixed after
fix_imports."""
from . import fix_imports
MAPPING = {
'whichdb': 'dbm',
'anydbm': 'dbm',
}
class FixImports2(fix_imports.FixImports):
run_order = 7
mapping = MAPPING
fixes/fix_xreadlines.pyc 0000666 00000002176 15050104230 0011403 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sp Fix "for x in f.xreadlines()" -> "for x in f".
This fixer will also convert g(f.xreadlines) into g(f.__iter__).i ( t
fixer_base( t Namet
FixXreadlinesc B s e Z d Z d Z RS( s
power< call=any+ trailer< '.' 'xreadlines' > trailer< '(' ')' > >
|
power< any+ trailer< '.' no_call='xreadlines' > >
c C sl | i d } | o | i t d d | i n3 | i g } | d D] } | | i qK ~ d S( Nt no_callu __iter__t prefixt call( t gett replaceR R t clone( t selft nodet resultsR t _[1]t x( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_xreadlines.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_xreadlines.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_xreadlines.pyt s fixes/fix_future.pyo 0000666 00000001602 15050104230 0010564 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sV Remove __future__ imports
from __future__ import foo is replaced with an empty line.
i ( t
fixer_base( t BlankLinet FixFuturec B s e Z d Z d Z d Z RS( s; import_from< 'from' module_name="__future__" 'import' any >i
c C s t } | i | _ | S( N( R t prefix( t selft nodet resultst new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_future.pyt transform s ( t __name__t
__module__t PATTERNt run_orderR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_future.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_future.pyt s fixes/fix_repr.pyo 0000666 00000001745 15050104230 0010232 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s/ Fixer that transforms `xyzzy` into repr(xyzzy).i ( t
fixer_base( t Callt Namet parenthesizet FixReprc B s e Z d Z d Z RS( s7
atom < '`' expr=any '`' >
c C sU | d i } | i | i i j o t | } n t t d | g d | i S( Nt expru reprt prefix( t clonet typet symst testlist1R R R R ( t selft nodet resultsR ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_repr.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_repr.pyR s N( t __doc__t R t
fixer_utilR R R t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_repr.pyt s fixes/fix_types.pyc 0000666 00000004207 15050104230 0010406 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z h d d 6d d 6d d
6d d 6d
d 6d
d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d! d" 6d# d$ 6d% d& 6d d' 6d# d( 6d) d* 6Z g Z e D] Z e d+ e q [ Z
d, e i f d- YZ d. S(/ s Fixer for removing uses of the types module.
These work for only the known names in the types module. The forms above
can include types. or not. ie, It is assumed the module is imported either as:
import types
from types import ... # either * or specific types
The import statements are not modified.
There should be another fixer that handles at least the following constants:
type([]) -> list
type(()) -> tuple
type('') -> str
i ( t token( t
fixer_base( t Namet boolt BooleanTypet
memoryviewt
BufferTypet typet ClassTypet complext ComplexTypet dictt DictTypet DictionaryTypes type(Ellipsis)t EllipsisTypet floatt FloatTypet intt IntTypet listt ListTypet LongTypet objectt
ObjectTypes
type(None)t NoneTypes type(NotImplemented)t NotImplementedTypet slicet SliceTypet bytest
StringTypet strt StringTypest tuplet TupleTypet TypeTypet UnicodeTypet ranget
XRangeTypes) power< 'types' trailer< '.' name='%s' > >t FixTypesc B s e Z d i e Z d Z RS( t |c C s; t t i | d i } | o t | d | i Sd S( Nt namet prefix( t unicodet
_TYPE_MAPPINGt gett valueR R) t None( t selft nodet resultst new_value( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_types.pyt transform: s ( t __name__t
__module__t joint _patst PATTERNR3 ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_types.pyR& 6 s N(
t __doc__t pgen2R t R t
fixer_utilR R+ t _[1]t tR7 t BaseFixR& ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_types.pyt s6
% fixes/fix_numliterals.pyc 0000666 00000002370 15050104230 0011600 0 ustar 00
Lc @ sS d Z d d k l Z d d k l Z d d k l Z d e i f d YZ d S( s- Fixer that turns 1L into 1, 0755 into 0o755.
i ( t token( t
fixer_base( t Numbert FixNumliteralsc B s# e Z e i Z d Z d Z RS( c C s$ | i i d p | i d d j S( Nu 0iu Ll( t valuet
startswith( t selft node( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyt match s c C s | i } | d d j o | d } nI | i d o8 | i o+ t t | d j o d | d } n t | d | i S( Niu Llu 0i u 0ot prefix( R R t isdigitt lent setR R ( R R t resultst val( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyt transform s 6( t __name__t
__module__R t NUMBERt _accept_typeR R ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyR s N( t __doc__t pgen2R t R t
fixer_utilR t BaseFixR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyt s fixes/fix_unicode.pyo 0000666 00000002222 15050104230 0010677 0 ustar 00
Lc @ sr d Z d d k Z d d k l Z d d k l Z h d d 6d d 6Z e i d
Z d e i f d YZ
d S(
sJ Fixer that changes unicode to str, unichr to chr, and u"..." into "...".
iNi ( t token( t
fixer_baseu chru unichru stru unicodeu [uU][rR]?[\'\"]t
FixUnicodec B s e Z d Z d Z RS( s STRING | 'unicode' | 'unichr'c C s | i t i j o! | i } t | i | _ | S| i t i j o8 t i | i o! | i } | i d | _ | Sn d S( Ni ( t typeR t NAMEt clonet _mappingt valuet STRINGt _literal_ret match( t selft nodet resultst new( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_unicode.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_unicode.pyR s ( t __doc__t ret pgen2R t R R t compileR t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_unicode.pyt s fixes/fix_long.pyc 0000666 00000001466 15050104230 0010205 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s/ Fixer that turns 'long' into 'int' everywhere.
i( t
fixer_base( t is_probably_builtint FixLongc B s e Z d Z d Z RS( s 'long'c C s( t | o d | _ | i n d S( Nu int( R t valuet changed( t selft nodet results( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_long.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_long.pyR s N( t __doc__t lib2to3R t lib2to3.fixer_utilR t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_long.pyt s fixes/fix_renames.py 0000666 00000004221 15050104230 0010525 0 ustar 00 """Fix incompatible renames
Fixes:
* sys.maxint -> sys.maxsize
"""
# Author: Christian Heimes
# based on Collin Winter's fix_import
# Local imports
from .. import fixer_base
from ..fixer_util import Name, attr_chain
MAPPING = {"sys": {"maxint" : "maxsize"},
}
LOOKUP = {}
def alternates(members):
return "(" + "|".join(map(repr, members)) + ")"
def build_pattern():
#bare = set()
for module, replace in MAPPING.items():
for old_attr, new_attr in replace.items():
LOOKUP[(module, old_attr)] = new_attr
#bare.add(module)
#bare.add(old_attr)
#yield """
# import_name< 'import' (module=%r
# | dotted_as_names< any* module=%r any* >) >
# """ % (module, module)
yield """
import_from< 'from' module_name=%r 'import'
( attr_name=%r | import_as_name< attr_name=%r 'as' any >) >
""" % (module, old_attr, old_attr)
yield """
power< module_name=%r trailer< '.' attr_name=%r > any* >
""" % (module, old_attr)
#yield """bare_name=%s""" % alternates(bare)
class FixRenames(fixer_base.BaseFix):
PATTERN = "|".join(build_pattern())
order = "pre" # Pre-order tree traversal
# Don't match the node if it's within another match
def match(self, node):
match = super(FixRenames, self).match
results = match(node)
if results:
if any(match(obj) for obj in attr_chain(node, "parent")):
return False
return results
return False
#def start_tree(self, tree, filename):
# super(FixRenames, self).start_tree(tree, filename)
# self.replace = {}
def transform(self, node, results):
mod_name = results.get("module_name")
attr_name = results.get("attr_name")
#bare_name = results.get("bare_name")
#import_mod = results.get("module")
if mod_name and attr_name:
new_attr = unicode(LOOKUP[(mod_name.value, attr_name.value)])
attr_name.replace(Name(new_attr, prefix=attr_name.prefix))
fixes/fix_buffer.pyc 0000666 00000001652 15050104230 0010514 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s4 Fixer that changes buffer(...) into memoryview(...).i ( t
fixer_base( t Namet FixBufferc B s e Z e Z d Z d Z RS( sR
power< name='buffer' trailer< '(' [any] ')' > any* >
c C s* | d } | i t d d | i d S( Nt nameu
memoryviewt prefix( t replaceR R ( t selft nodet resultsR ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_buffer.pyt transform s
( t __name__t
__module__t Truet explicitt PATTERNR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_buffer.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_buffer.pyt s fixes/fix_long.py 0000666 00000000705 15050104230 0010035 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that turns 'long' into 'int' everywhere.
"""
# Local imports
from lib2to3 import fixer_base
from lib2to3.fixer_util import is_probably_builtin
class FixLong(fixer_base.BaseFix):
PATTERN = "'long'"
def transform(self, node, results):
if is_probably_builtin(node):
node.value = u"int"
node.changed()
fixes/fix_print.pyo 0000666 00000005135 15050104230 0010413 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l
Z
l Z l Z e i
d Z d e i f d YZ d
S( s Fixer for print.
Change:
'print' into 'print()'
'print ...' into 'print(...)'
'print ... ,' into 'print(..., end=" ")'
'print >>x, ...' into 'print(..., file=x)'
No changes are applied if print_function is imported from __future__
i ( t patcomp( t pytree( t token( t
fixer_base( t Namet Callt Commat Stringt is_tuples" atom< '(' [atom|STRING|NAME] ')' >t FixPrintc B s e Z d Z d Z d Z RS( sP
simple_stmt< any* bare='print' any* > | print_stmt
c C s | i d } | o* | i t t d g d | i d S| i d } t | d j o t i | d o d Sd } } } | o( | d t
j o | d } d } n | o> | d t i t
i d j o | d i } | d } n g } | D] } | | i q~ }
|
o d
|
d _ n | d j p | d j p
| d j o | d j o# | i |
d t t | n | d j o# | i |
d t t | n | d j o | i |
d
| qn t t d |
} | i | _ | S( Nt bareu printt prefixi i it u >>i u u sepu endu file( t gett replaceR R R t childrent lent parend_exprt matcht NoneR R t LeafR t
RIGHTSHIFTt clonet add_kwargR t repr( t selft nodet resultst
bare_printt argst sept endt filet _[1]t argt l_argst n_stmt( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyt transform# s8
'
'''
#
#
c C st d | _ t i | i i t | t i t i d | f } | o | i t
d | _ n | i | d S( Nu u =u ( R R t Nodet symst argumentR R R t EQUALt appendR ( R t l_nodest s_kwdt n_exprt
n_argument( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyR K s
( t __name__t
__module__t PATTERNR% R ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyR s (N( t __doc__t R R t pgen2R R t
fixer_utilR R R R R t compile_patternR t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyt s ( fixes/fix_tuple_params.pyo 0000666 00000012557 15050104230 0011761 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z l Z d Z
d e i f d YZ d Z d
Z g d
d Z d Z d
S( s: Fixer for function definitions with tuple parameters.
def func(((a, b), c), d):
...
->
def func(x, d):
((a, b), c) = x
...
It will also support lambdas:
lambda (x, y): x + y -> lambda t: t[0] + t[1]
# The parens are a syntax error in Python 3
lambda (x): x + y -> lambda x: x + y
i ( t pytree( t token( t
fixer_base( t Assignt Namet Newlinet Numbert Subscriptt symsc C s* t | t i o | i d i t i j S( Ni ( t
isinstanceR t Nodet childrent typeR t STRING( t stmt( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyt is_docstring s t FixTupleParamsc B s e Z d Z d Z d Z RS( s
funcdef< 'def' any parameters< '(' args=any ')' >
['->' any] ':' suite=any+ >
|
lambda=
lambdef< 'lambda' args=vfpdef< '(' inner=any ')' >
':' body=any
>
c s@ d | j o i | | Sg | d } | d } | d i d i t i j o' d } | d i d i } t n" d } d } t i t i d t f d } | i t
i j o | | ne | i t
i j oQ xN t
| i D]9 \ } } | i t
i j o | | d
| d j q q Wn p d Sx D] }
| d |
_ qNW| } | d j o d d _ n4 t | d i | o | d _ | d } n x D] }
| d |
_ qW | d i | | +x= t | d | t d D] } | | d i | _ qW| d i d S( Nt lambdat suitet argsi i i u ; u c s t i } | i } d | _ t | | i } | o
d | _ n | i | i t i t i
| i g d S( Nu u ( R t new_namet clonet prefixR t replacet appendR R
R t simple_stmt( t tuple_argt
add_prefixt nt argR ( t new_linest endt self( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyt handle_tuple? s
R u ( t transform_lambdaR R R t INDENTt valueR R t Leaft FalseR t tfpdeft
typedargslistt enumeratet parentR R t ranget lent changed( R t nodet resultsR R t startt indentR! t iR t linet after( ( R R R s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyt transform* sN
"
" c C s[ | d } | d } t | d } | i t i j o' | i } d | _ | i | d St | } t | } | i t
| } t | d d } | i | i x | i D] }
|
i t i j o} |
i
| j om g } | |
i
D] } | | i q ~ }
t i t i | i g |
} |
i | _ |
i | q q Wd S( NR t bodyt inneru R ( t
simplify_argsR R t NAMER R R t find_paramst map_to_indexR t
tuple_nameR t
post_orderR$ R R
R t power( R R. R/ R R6 R7 t paramst to_indext tup_namet new_paramR t _[1]t ct
subscriptst new( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR" j s*
#. ( t __name__t
__module__t PATTERNR5 R" ( ( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR s @c C su | i t i t i f j o | S| i t i j o- x% | i t i j o | i d } q7 W| St d | d S( Ni s Received unexpected node %s( R R t vfplistR R9 t vfpdefR t RuntimeError( R. ( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR8 s c C s | i t i j o t | i d S| i t i j o | i Sg } | i D]* } | i t i j o | t | qN qN ~ S( Ni ( R R RK R: R R R9 R$ t COMMA( R. RC RD ( ( s6 /usr/lib64/python2.6/lib2to3/fixes/fix_tuple_params.pyR: s
c C s | d j o
h } n xj t | D]\ \ } } t t t | g } t | t o t | | d | q$ | | | | s . h fixes/fix_funcattrs.pyo 0000666 00000002105 15050104230 0011262 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s3 Fix function attribute names (f.func_x -> f.__x__).i ( t
fixer_base( t Namet FixFuncattrsc B s e Z d Z d Z RS( s
power< any+ trailer< '.' attr=('func_closure' | 'func_doc' | 'func_globals'
| 'func_name' | 'func_defaults' | 'func_code'
| 'func_dict') > any* >
c C s9 | d d } | i t d | i d d | i d S( Nt attri u __%s__i t prefix( t replaceR t valueR ( t selft nodet resultsR ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_funcattrs.pyt transform s ( t __name__t
__module__t PATTERNR
( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_funcattrs.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_funcattrs.pyt s fixes/fix_set_literal.pyc 0000666 00000003702 15050104230 0011550 0 ustar 00
Lc @ sO d Z d d k l Z l Z d d k l Z l Z d e i f d YZ d S( s:
Optional fixer to transform set() calls to set literals.
i( t
fixer_baset pytree( t tokent symst
FixSetLiteralc B s e Z e Z d Z d Z RS( sj power< 'set' trailer< '('
(atom=atom< '[' (items=listmaker< any ((',' any)* [',']) >
|
single=any) ']' >
|
atom< '(' items=testlist_gexp< any ((',' any)* [',']) > ')' >
)
')' > >
c C s | i d } | o5 t i t i | i g } | i | | } n | d } t i t i d g } | i
d | i D | i t i t i
d | i i | d _ t i t i | } | i | _ t | i d j o. | i d } | i | i | i d _ n | S( Nt singlet itemsu {c s s x | ] } | i Vq Wd S( N( t clone( t .0t n( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pys & s u }ii i ( t getR t NodeR t listmakerR t replacet LeafR t LBRACEt extendt childrent appendt RBRACEt next_siblingt prefixt dictsetmakert lent remove( t selft nodet resultsR t fakeR t literalt makerR ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pyt transform s"
( t __name__t
__module__t Truet explicitt PATTERNR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pyR s
N( t __doc__t lib2to3R R t lib2to3.fixer_utilR R t BaseFixR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pyt s fixes/fix_imports.pyo 0000666 00000012351 15050104230 0010752 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z l Z h0 d d 6d d 6d d 6d d
6d d 6d
d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d! d" 6d# d$ 6d% d& 6d' d( 6d) d* 6d+ d, 6d- d. 6d/ d0 6d1 d2 6d3 d4 6d5 d6 6d7 d8 6d9 d: 6d; d< 6d= d> 6d? d@ 6dA dB 6dC dD 6dC dE 6dF dG 6dH dI 6dJ dK 6dL dM 6dN dO 6dP dQ 6dP dR 6dP dS 6dT dU 6dV dW 6dV dX 6dY dZ 6d[ d\ 6Z d] Z e d^ Z d_ e i f d` YZ
da S(b s/ Fix incompatible imports and module references.i ( t
fixer_base( t Namet
attr_chaint iot StringIOt cStringIOt picklet cPicklet builtinst __builtin__t copyregt copy_regt queuet Queuet socketservert SocketServert configparsert ConfigParsert reprlibt reprs tkinter.filedialogt
FileDialogt tkFileDialogs tkinter.simpledialogt SimpleDialogt tkSimpleDialogs tkinter.colorchoosert tkColorChoosers tkinter.commondialogt tkCommonDialogs tkinter.dialogt Dialogs tkinter.dndt Tkdnds tkinter.fontt tkFonts tkinter.messageboxt tkMessageBoxs tkinter.scrolledtextt ScrolledTexts tkinter.constantst Tkconstantss tkinter.tixt Tixs tkinter.ttkt ttkt tkintert Tkintert _markupbaset
markupbaset winregt _winregt _threadt threadt
_dummy_threadt dummy_threads dbm.bsdt dbhashs dbm.dumbt dumbdbms dbm.ndbmt dbms dbm.gnut gdbms
xmlrpc.clientt xmlrpclibs
xmlrpc.servert DocXMLRPCServert SimpleXMLRPCServers http.clientt httplibs
html.entitiest htmlentitydefss html.parsert
HTMLParsers http.cookiest Cookies http.cookiejart cookielibs http.servert BaseHTTPServert SimpleHTTPServert
CGIHTTPServert
subprocesst commandst collectionst
UserStringt UserLists urllib.parset urlparses urllib.robotparsert robotparserc C s d d i t t | d S( Nt (t |t )( t joint mapR ( t members( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyt
alternates= s c c st d i g } | D] } | d | q ~ } t | i } d | | f Vd | Vd | | f Vd | Vd S( Ns | s module_name='%s'sy name_import=import_name< 'import' ((%s) |
multiple_imports=dotted_as_names< any* (%s) any* >) >
s import_from< 'from' (%s) 'import' ['(']
( any | import_as_name< any 'as' any > |
import_as_names< any* >) [')'] >
s import_name< 'import' (dotted_as_name< (%s) 'as' any > |
multiple_imports=dotted_as_names<
any* dotted_as_name< (%s) 'as' any > any* >) >
s3 power< bare_with_attr=(%s) trailer<'.' any > any* >( RE RH t keys( t mappingt _[1]t keyt mod_listt
bare_names( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyt
build_patternA s . t
FixImportsc B sA e Z e Z d Z d Z d Z d Z d Z d Z RS( i c C s d i t | i S( NRC ( RE RO RJ ( t self( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyRO ^ s c C s& | i | _ t t | i d S( N( RO t PATTERNt superRP t compile_pattern( RQ ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyRT a s c sf t t | i | } | o= d | j o+ t f d t | d D o t S| St S( Nt bare_with_attrc 3 s x | ] } | Vq Wd S( N( ( t .0t obj( t match( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pys o s t parent( RS RP RX t anyR t False( RQ t nodet results( ( RX s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyRX h s
&c C s& t t | i | | h | _ d S( N( RS RP t
start_treet replace( RQ t treet filename( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyR^ t s c C s | i d } | o | i } t | i | } | i t | d | i d | j o | | i | sj
fixes/fix_methodattrs.pyo 0000666 00000002135 15050104230 0011612 0 ustar 00
Lc @ s^ d Z d d k l Z d d k l Z h d d 6d d 6d d 6Z d
e i f d YZ d S(
s; Fix bound method attributes (method.im_? -> method.__?__).
i ( t
fixer_base( t Namet __func__t im_funct __self__t im_selfs __self__.__class__t im_classt FixMethodattrsc B s e Z d Z d Z RS( sU
power< any+ trailer< '.' attr=('im_func' | 'im_self' | 'im_class') > any* >
c C sA | d d } t t | i } | i t | d | i d S( Nt attri t prefix( t unicodet MAPt valuet replaceR R ( t selft nodet resultsR t new( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_methodattrs.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_methodattrs.pyR s N( t __doc__t R t
fixer_utilR R t BaseFixR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_methodattrs.pyt s
fixes/fix_future.py 0000666 00000001011 15050104230 0010377 0 ustar 00 """Remove __future__ imports
from __future__ import foo is replaced with an empty line.
"""
# Author: Christian Heimes
# Local imports
from .. import fixer_base
from ..fixer_util import BlankLine
class FixFuture(fixer_base.BaseFix):
PATTERN = """import_from< 'from' module_name="__future__" 'import' any >"""
# This should be run last -- some things check for the import
run_order = 10
def transform(self, node, results):
new = BlankLine()
new.prefix = node.prefix
return new
fixes/fix_methodattrs.pyc 0000666 00000002135 15050104230 0011576 0 ustar 00
Lc @ s^ d Z d d k l Z d d k l Z h d d 6d d 6d d 6Z d
e i f d YZ d S(
s; Fix bound method attributes (method.im_? -> method.__?__).
i ( t
fixer_base( t Namet __func__t im_funct __self__t im_selfs __self__.__class__t im_classt FixMethodattrsc B s e Z d Z d Z RS( sU
power< any+ trailer< '.' attr=('im_func' | 'im_self' | 'im_class') > any* >
c C sA | d d } t t | i } | i t | d | i d S( Nt attri t prefix( t unicodet MAPt valuet replaceR R ( t selft nodet resultsR t new( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_methodattrs.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_methodattrs.pyR s N( t __doc__t R t
fixer_utilR R t BaseFixR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_methodattrs.pyt s
fixes/fix_execfile.pyc 0000666 00000003774 15050104230 0011036 0 ustar 00
Lc @ sy d Z d d k l Z d d k l Z l Z l Z l Z l Z l Z l
Z
l Z l Z l
Z
d e i f d YZ d S( so Fixer for execfile.
This converts usages of the execfile function into calls to the built-in
exec() function.
i ( t
fixer_base(
t Commat Namet Callt LParent RParent Dott Nodet ArgListt Stringt symst FixExecfilec B s e Z d Z d Z RS( s
power< 'execfile' trailer< '(' arglist< filename=any [',' globals=any [',' locals=any ] ] > ')' > >
|
power< 'execfile' trailer< '(' filename=any ')' > >
c C s | p t | d } | i d } | i d } | i d i d i } t | i g d | } t t i t d | g } t t i t
t d g t t i t t g g } | g | }
| i } d | _
t d d } |
t | t | g }
t t d
|
d } | g } | d j o | i t | i g n | d j o | i t | i g n t t d | d
| i
S( Nt filenamet globalst localsit rparenu openu readu u 'exec'u compileu u exect prefix( t AssertionErrort gett childrent cloneR R R
t powerR t trailerR R R R R R R t Nonet extend( t selft nodet resultsR R
R t execfile_parent open_argst open_callt readt open_exprt filename_argt exec_strt compile_argst compile_callt args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_execfile.pyt transform s*
!
( t __name__t
__module__t PATTERNR& ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_execfile.pyR s N( t __doc__t R t
fixer_utilR R R R R R R R R R
t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_execfile.pyt s F fixes/fix_urllib.pyo 0000666 00000014503 15050104230 0010547 0 ustar 00
Lc @ sg d Z d d k l Z l Z d d k l Z d d k l Z l Z l Z l
Z
l Z h d d d d d
d d d
d g f d d d d d d d d d d d d d d d d g f d d g f g d! 6d d d" d# d$ d% d& d' d( d) d* d+ d, d- d. d/ d0 d1 d2 d3 d4 d5 d6 d7 d8 g f d d9 d: g f g d; 6Z e d; i
e d! d d< Z d= e f d> YZ d? S(@ s Fix changes imports of urllib which are now incompatible.
This is rather similar to fix_imports, but because of the more
complex nature of the fixing for urllib, it has its own fixer.
i ( t
alternatest
FixImportsi ( t
fixer_base( t Namet Commat
FromImportt Newlinet
attr_chains urllib.requestt URLOpenert FancyURLOpenert urlretrievet
_urlopenert urlopent
urlcleanupt pathname2urlt url2pathnames urllib.parset quotet
quote_plust unquotet unquote_plust urlencodet splitattrt splithostt
splitnportt splitpasswdt splitportt
splitqueryt splittagt splittypet splitusert
splitvalues urllib.errort ContentTooShortErrort urllibt install_openert build_openert Requestt OpenerDirectort BaseHandlert HTTPDefaultErrorHandlert HTTPRedirectHandlert HTTPCookieProcessort ProxyHandlert HTTPPasswordMgrt HTTPPasswordMgrWithDefaultRealmt AbstractBasicAuthHandlert HTTPBasicAuthHandlert ProxyBasicAuthHandlert AbstractDigestAuthHandlert HTTPDigestAuthHandlert ProxyDigestAuthHandlert HTTPHandlert HTTPSHandlert FileHandlert
FTPHandlert CacheFTPHandlert UnknownHandlert URLErrort HTTPErrort urllib2c c s t } x t i D]w \ } } xh | D]` } | \ } } t | } d | | f Vd | | | f Vd | Vd | Vd | | f Vq) Wq Wd S( Ns import_name< 'import' (module=%r
| dotted_as_names< any* module=%r any* >) >
s import_from< 'from' mod_member=%r 'import'
( member=%s | import_as_name< member=%s 'as' any > |
import_as_names< members=any* >) >
sI import_from< 'from' module_star=%r 'import' star='*' >
st import_name< 'import'
dotted_as_name< module_as=%r 'as' any > >
sK power< bare_with_attr=%r trailer< '.' member=%s > any* >
( t sett MAPPINGt itemsR ( t baret
old_modulet changest changet
new_modulet members( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt
build_pattern0 s
t FixUrllibc B s5 e Z d Z d Z d Z d Z d Z RS( c C s d i t S( Nt |( t joinRD ( t self( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyRD I s c C s | i d } | i } g } x? t | i d D], } | i t | d d | t g q0 W| i t t | i d d d | | i | d S( s Transform for the basic import case. Replaces the old
import name with a comma separated list of its
replacements.
t moduleii t prefixN( t getRJ R< t valuet extendR R t appendt replace( RH t nodet resultst
import_modt preft namest name( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt transform_importL s *(c C s | i d } | i } | i d } | o t | t o | d } n d } x8 t | i D]) } | i | d j o | d } Pq` q` W| o | i t | d | q| i | d ng } h } | i d }
x |
D] } | i } | d j o} xz t | i D]g } | | d j oP | d | j o | | d i
| qx| g | | d <| i
| d qqWq q Wg } x | D] } | | }
g } x4 |
d D]( } | i t | d | t g qW| i
t |
d d | | i
t
| | qW| oS g } x( | d D] } | i | t g q-W| i
| d | i | n | i | d
d S( s Transform for imports of specific module elements. Replaces
the module to be imported from with the appropriate new
module.
t
mod_membert memberi i RJ s! This is an invalid module elementRC t ,is All module elements are invalidN( RK RJ t
isinstancet listt NoneR< RL RO R t cannot_convertRN RM R R R ( RH RP RQ RW RS RX t new_nameRA t modulest mod_dictRC t new_nodesRI t eltsRT t eltt nodest new_node( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt transform_member\ s`
!
& c C s | i d } | i d } d } t | t o | d } n x8 t | i D]) } | i | d j o | d } PqP qP W| o | i t | d | i n | i | d d S( s. Transform for calls to module members in code.t bare_with_attrRX i i RJ s! This is an invalid module elementN(
RK R\ RZ R[ R< RL RO R RJ R] ( RH RP RQ t
module_dotRX R^ RA ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt
transform_dot s
c C s | i d o | i | | n | i d o | i | | nm | i d o | i | | nI | i d o | i | d n% | i d o | i | d n d S( NRI RW Rg t module_stars Cannot handle star imports.t module_ass# This module is now multiple modules( RK RV Rf Ri R] ( RH RP RQ ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt transform s ( t __name__t
__module__RD RV Rf Ri Rl ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyRE G s
< N( t __doc__t fix_importsR R t R t
fixer_utilR R R R R R< RN RD RE ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_urllib.pyt sD ( fixes/fix_exitfunc.pyc 0000666 00000005216 15050104230 0011070 0 ustar 00
Lc @ sg d Z d d k l Z l Z d d k l Z l Z l Z l Z l Z l
Z
d e i f d YZ d S( s7
Convert use of sys.exitfunc to use the atexit module.
i( t pytreet
fixer_base( t Namet Attrt Callt Commat Newlinet symst FixExitfuncc B s) e Z d Z d Z d Z d Z RS( s
(
sys_import=import_name<'import'
('sys'
|
dotted_as_names< (any ',')* 'sys' (',' any)* >
)
>
|
expr_stmt<
power< 'sys' trailer< '.' 'exitfunc' > >
'=' func=any >
)
c G s t t | i | d S( N( t superR t __init__( t selft args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyR
s c C s& t t | i | | d | _ d S( N( R R t
start_treet Nonet
sys_import( R t treet filename( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyR
s c C s d | j o& | i d j o | d | _ n d S| d i } d | _ t i t i t t d t d } t
| | g | i } | i | | i d j o | i | d d S| i i
d } | i t i j o* | i t | i t d d n | i i } | i
i | i } | i } t i t i t d t d d g }
t i t i |
g } | i | d t | i | d
| d S( NR t funcu u atexitu registersK Can't find sys import; Please add an atexit import at the top of your file.i u u importi ( R R t clonet prefixR t NodeR t powerR R R t replacet warningt childrent typet dotted_as_namest append_childR t parentt indext import_namet simple_stmtt insert_childR ( R t nodet resultsR t registert callt namest containing_stmtt positiont stmt_containert
new_importt new( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyt transform# s2
( t __name__t
__module__t PATTERNR
R
R, ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyR s N(
t __doc__t lib2to3R R t lib2to3.fixer_utilR R R R R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyt s . fixes/fix_metaclass.pyc 0000666 00000015005 15050104230 0011214 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z l Z l Z l Z d Z
d Z d Z d Z
d Z d
Z d e i f d YZ d
S( s Fixer for __metaclass__ = X -> (metaclass=X) methods.
The various forms of classef (inherits nothing, inherits once, inherints
many) don't parse the same in the CST so we look at ALL classes for
a __metaclass__ and if we find one normalize the inherits to all be
an arglist.
For one-liner classes ('class X: pass') there is no indent/dedent so
we normalize those into having a suite.
Moving the __metaclass__ into the classdef can also cause the class
body to be empty so there is some special casing for that as well.
This fixer also tries very hard to keep original indenting and spacing
in all those corner cases.
i ( t
fixer_base( t token( t Namet symst Nodet Leafc C s x | i D] } | i t i j o t | S| i t i j on | i od | i d } | i t i j o@ | i o6 | i d } t | t o | i d j o t
Sq q
q
Wt S( s we have to check the cls_node without changing it.
There are two possiblities:
1) clsdef => suite => simple_stmt => expr_stmt => Leaf('__meta')
2) clsdef => simple_stmt => expr_stmt => Leaf('__meta')
i t
__metaclass__( t childrent typeR t suitet
has_metaclasst simple_stmtt expr_stmtt
isinstanceR t valuet Truet False( t parentt nodet expr_nodet left_side( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyR
s
c C s x) | i D] } | i t i j o d Sq
WxA t | i D]$ \ } } | i t i j o Pq< q< Wt d t t i g } xE | i | d o2 | i | d } | i | i
| i q W| i | | } d S( sf one-line classes don't get a suite in the parse tree so we add
one to normalize the tree
Ns No class suite and no ':'!i ( R R R R t enumerateR t COLONt
ValueErrorR t append_childt clonet remove( t cls_nodeR t iR t move_node( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt fixup_parse_tree- s"
c
C s x9 t | i D]$ \ } } | i t i j o Pq q Wd S| i t t i g } t t i | g } x= | i | o. | i | } | i
| i | i qp W| i | | | i d i d } | i d i d } | i
| _
d S( s if there is a semi-colon all the parts count as part of the same
simple_stmt. We just want the __metaclass__ part so we move
everything efter the semi-colon into its own simple_stmt node
Ni ( R R R R t SEMIR R R R R R R t insert_childt prefix(
R R t stmt_nodet semi_indR t new_exprt new_stmtR t new_leaf1t old_leaf1( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt fixup_simple_stmtG s"
c C s= | i o/ | i d i t i j o | i d i n d S( Ni( R R R t NEWLINER ( R ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt remove_trailing_newline_ s $c c s
x5 | i D] } | i t i j o Pq
q
Wt d x t t | i D] \ } } | i t i j o | i o | i d } | i t i j og | i o] | i d } t | t
o< | i d j o, t | | | t
| | | | f Vq qqN qN Wd S( Ns No class suite!i u
__metaclass__( R R R R R t listR R R R
R R R( R* ( R R R t simple_nodeR t left_node( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt
find_metasd s
c C s | i d d d } x0 | o( | i } | i t i j o Pq q Wxt | ol | i } t | t o/ | i t i j o | i o
d | _ n d S| i | i d d d qL Wd S( s If an INDENT is followed by a thing with a prefix then nuke the prefix
Otherwise we get in trouble when removing __metaclass__ at suite start
Niu (
R t popR R t INDENTR
R t DEDENTR! t extend( R t kidsR ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt fixup_indent{ s
#
t FixMetaclassc B s e Z d Z d Z RS( s
classdef
c C s; t | p d St | d } x- t | D] \ } } } | } | i q/ W| i d i } t | i d j og | i d i t i j o | i d } q| i d i
} t t i | g } | i d | n t | i d j o& t t i g } | i
d | n t | i d j o^ t t i g } | i
d t t i d | i
d | | i
d t t i d n
t d | i d i d }
d
|
_ |
i } | i o& | i t t i d d |
_ n
d
|
_ | i d } | i t i j p t d
| i d _ d
| i d _ | i | t | | i pL | i t | d }
| |
_ | i |
| i t t i d n t | i d j os | i d i t i j oY | i d i t i j o? t | d }
| i
d |
| i
d t t i d n d S( Ni i i i i i u )u (s Unexpected class definitiont metaclassu ,u u i u passu
ii( R
R t NoneR. R R R t lenR t arglistR R t set_childR R R t RPARt LPARR R R! R t COMMAR t AssertionErrorR4 R) R0 R1 ( t selfR t resultst last_metaclassR R t stmtt text_typeR9 R t meta_txtt orig_meta_prefixR t pass_leaf( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt transform sb
( t __name__t
__module__t PATTERNRG ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyR5 s N( t __doc__t R t pygramR t
fixer_utilR R R R R
R R( R* R. R4 t BaseFixR5 ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt s " fixes/fix_buffer.py 0000666 00000001066 15050104230 0010350 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that changes buffer(...) into memoryview(...)."""
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixBuffer(fixer_base.BaseFix):
explicit = True # The user must ask for this fixer
PATTERN = """
power< name='buffer' trailer< '(' [any] ')' > any* >
"""
def transform(self, node, results):
name = results["name"]
name.replace(Name(u"memoryview", prefix=name.prefix))
fixes/fix_raise.pyc 0000666 00000004504 15050104230 0010345 0 ustar 00
Lc @ s{ d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z d e i f d YZ
d S( s- Fixer for 'raise E, V, T'
raise -> raise
raise E -> raise E
raise E, V -> raise E(V)
raise E, V, T -> raise E(V).with_traceback(T)
raise (((E, E'), E''), E'''), V -> raise E(V)
raise "foo", V, T -> warns about string exceptions
CAVEATS:
1) "raise E, V" will be incorrectly translated if V is an exception
instance. The correct Python 3 idiom is
raise E from V
but since we can't detect instance-hood by syntax alone and since
any client code would have to be changed as well, we don't automate
this.
i ( t pytree( t token( t
fixer_base( t Namet Callt Attrt ArgListt is_tuplet FixRaisec B s e Z d Z d Z RS( sB
raise_stmt< 'raise' exc=any [',' val=any [',' tb=any]] >
c
C s | i } | d i } | i t i j o | i | d d St | o< x, t | o | i d i d i } qQ Wd | _ n d | j o2 t i
| i t d | g } | i | _ | S| d i } t | o5 g } | i d d !D] } | | i q ~ } n d | _ | g } d
| j o | d
i }
d |
_ t
| | } t | t d t |
g g } t i
| i t d g | } | i | _ | St i
| i t d t
| | g d | i Sd S(
Nt excs+ Python 3 does not support string exceptionsi i t t valu raiseiu t tbu with_tracebackt prefix( t symst clonet typeR t STRINGt cannot_convertR t childrenR
R t Nodet
raise_stmtR R R R t simple_stmt(
t selft nodet resultsR R t newR t _[1]t ct argsR t et with_tb( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_raise.pyt transform$ s<
!
5
%" ( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_raise.pyR s N( t __doc__t R t pgen2R R t
fixer_utilR R R R R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_raise.pyt s
( fixes/fix_callable.pyo 0000666 00000002646 15050104230 0011022 0 ustar 00
Lc @ s[ d Z d d k l Z d d k l Z l Z l Z l Z l Z d e i f d YZ
d S( s Fixer for callable().
This converts callable(obj) into isinstance(obj, collections.Callable), adding a
collections import if needed.i( t
fixer_base( t Callt Namet Stringt Attrt touch_importt FixCallablec B s e Z d Z d Z RS( s
power< 'callable'
trailer< lpar='('
( not(arglist | argument) any ','> )
rpar=')' >
after=any*
>
c C ss | d } t d d d | | i t d g } | i t t d t d t t d | d | i S( Nt funcu collectionst nodeu , u Callableu
isinstancet prefix( R t Nonet cloneR t extendR R R R ( t selfR t resultsR t args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_callable.pyt transform s
"( t __name__t
__module__t PATTERNR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_callable.pyR
s N( t __doc__t lib2to3R t lib2to3.fixer_utilR R R R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_callable.pyt s ( fixes/fix_standarderror.py 0000666 00000000652 15050104230 0011751 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for StandardError -> Exception."""
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixStandarderror(fixer_base.BaseFix):
PATTERN = """
'StandardError'
"""
def transform(self, node, results):
return Name(u"Exception", prefix=node.prefix)
fixes/fix_paren.pyo 0000666 00000002762 15050104230 0010367 0 ustar 00
Lc @ sI d Z d d k l Z d d k l Z l Z d e i f d YZ d S( su Fixer that addes parentheses where they are required
This converts ``[x for x in 1, 2]`` to ``[x for x in (1, 2)]``.i ( t
fixer_base( t LParent RParent FixParenc B s e Z d Z d Z RS( s
atom< ('[' | '(')
(listmaker< any
comp_for<
'for' NAME 'in'
target=testlist_safe< any (',' any)+ [',']
>
[any]
>
>
|
testlist_gexp< any
comp_for<
'for' NAME 'in'
target=testlist_safe< any (',' any)+ [',']
>
[any]
>
>)
(']' | ')') >
c C sL | d } t } | i | _ d | _ | i d | | i t d S( Nt targetu i ( R t prefixt insert_childt append_childR ( t selft nodet resultsR t lparen( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_paren.pyt transform# s
( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_paren.pyR s N( t __doc__t R t
fixer_utilR R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_paren.pyt s fixes/fix_exec.py 0000666 00000001721 15050104230 0010021 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for exec.
This converts usages of the exec statement into calls to a built-in
exec() function.
exec code in ns1, ns2 -> exec(code, ns1, ns2)
"""
# Local imports
from .. import pytree
from .. import fixer_base
from ..fixer_util import Comma, Name, Call
class FixExec(fixer_base.BaseFix):
PATTERN = """
exec_stmt< 'exec' a=any 'in' b=any [',' c=any] >
|
exec_stmt< 'exec' (not atom<'(' [any] ')'>) a=any >
"""
def transform(self, node, results):
assert results
syms = self.syms
a = results["a"]
b = results.get("b")
c = results.get("c")
args = [a.clone()]
args[0].prefix = ""
if b is not None:
args.extend([Comma(), b.clone()])
if c is not None:
args.extend([Comma(), c.clone()])
return Call(Name(u"exec"), args, prefix=node.prefix)
fixes/fix_execfile.py 0000666 00000003665 15050104230 0010672 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for execfile.
This converts usages of the execfile function into calls to the built-in
exec() function.
"""
from .. import fixer_base
from ..fixer_util import (Comma, Name, Call, LParen, RParen, Dot, Node,
ArgList, String, syms)
class FixExecfile(fixer_base.BaseFix):
PATTERN = """
power< 'execfile' trailer< '(' arglist< filename=any [',' globals=any [',' locals=any ] ] > ')' > >
|
power< 'execfile' trailer< '(' filename=any ')' > >
"""
def transform(self, node, results):
assert results
filename = results["filename"]
globals = results.get("globals")
locals = results.get("locals")
# Copy over the prefix from the right parentheses end of the execfile
# call.
execfile_paren = node.children[-1].children[-1].clone()
# Construct open().read().
open_args = ArgList([filename.clone()], rparen=execfile_paren)
open_call = Node(syms.power, [Name(u"open"), open_args])
read = [Node(syms.trailer, [Dot(), Name(u'read')]),
Node(syms.trailer, [LParen(), RParen()])]
open_expr = [open_call] + read
# Wrap the open call in a compile call. This is so the filename will be
# preserved in the execed code.
filename_arg = filename.clone()
filename_arg.prefix = u" "
exec_str = String(u"'exec'", u" ")
compile_args = open_expr + [Comma(), filename_arg, Comma(), exec_str]
compile_call = Call(Name(u"compile"), compile_args, u"")
# Finally, replace the execfile call with an exec call.
args = [compile_call]
if globals is not None:
args.extend([Comma(), globals.clone()])
if locals is not None:
args.extend([Comma(), locals.clone()])
return Call(Name(u"exec"), args, prefix=node.prefix)
fixes/fix_isinstance.pyc 0000666 00000003450 15050104230 0011401 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s, Fixer that cleans up a tuple argument to isinstance after the tokens
in it were fixed. This is mainly used to remove double occurrences of
tokens as a leftover of the long -> int / unicode -> str conversion.
eg. isinstance(x, (int, long)) -> isinstance(x, (int, int))
-> isinstance(x, int)
i ( t
fixer_base( t tokent
FixIsinstancec B s e Z d Z d Z d Z RS( s
power<
'isinstance'
trailer< '(' arglist< any ',' atom< '('
args=testlist_gexp< any+ >
')' > > ')' >
>
i c C sb t } | d } | i } g } t | } x | D] \ } } | i t i j oW | i | j oG | t | d j o, | | d i t i j o | i q5 q q5 | i
| | i t i j o | i | i q5 q5 W| o" | d i t i j o | d =n t | d j o. | i }
|
i
| d _
|
i | d n | | (| i d S( Nt argsi ii ( t sett childrent enumeratet typeR t NAMEt valuet lent COMMAt nextt appendt addt parentt prefixt replacet changed( t selft nodet resultst names_insertedt testlistR t new_argst iteratort idxt argt atom( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_isinstance.pyt transform s,
#2
( t __name__t
__module__t PATTERNt run_orderR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_isinstance.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_isinstance.pyt
s fixes/fix_ne.py 0000666 00000001075 15050104230 0007501 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that turns <> into !=."""
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
class FixNe(fixer_base.BaseFix):
# This is so simple that we don't need the pattern compiler.
_accept_type = token.NOTEQUAL
def match(self, node):
# Override
return node.value == u"<>"
def transform(self, node, results):
new = pytree.Leaf(token.NOTEQUAL, u"!=", prefix=node.prefix)
return new
fixes/fix_import.pyc 0000666 00000006314 15050104230 0010555 0 ustar 00
Lc @ sz d Z d d k l Z d d k l Z l Z l Z l Z d d k l Z l
Z
l Z d Z d e i
f d YZ d S(
s Fixer for import statements.
If spam is being imported from the local directory, this import:
from spam import eggs
Becomes:
from .spam import eggs
And this import:
import spam
Becomes:
from . import spam
i ( t
fixer_basei( t dirnamet joint existst sep( t
FromImportt symst tokenc c s | g } x | o | i } | i t i j o | i Vq | i t i j o3 d i g } | i D] } | | i qe ~ Vq | i t i j o | i
| i d q | i t i j o! | i | i d d d q t
d q Wd S( sF
Walks over all the names imported in a dotted_as_names node.
t i Nis unkown node type( t popt typeR t NAMEt valueR t dotted_nameR t childrent dotted_as_namet appendt dotted_as_namest extendt AssertionError( t namest pendingt nodet _[1]t ch( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyt traverse_imports s 3!t FixImportc B s) e Z d Z d Z d Z d Z RS( sj
import_from< 'from' imp=any 'import' ['('] any [')'] >
|
import_name< 'import' imp=any >
c C s/ t t | i | | d | i j | _ d S( Nt absolute_import( t superR t
start_treet future_featurest skip( t selft treet name( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyR . s c C s | i o d S| d } | i t i j oZ x" t | d p | i d } q/ W| i | i o d | i | _ | i qn t } t } x4 t
| D]& } | i | o
t } q t } q W| o | o | i | d n d St
d | g } | i | _ | Sd S( Nt impR i u .s# absolute and local imports together( R R
R t import_fromt hasattrR t probably_a_local_importR t changedt FalseR t Truet warningR t prefix( R R t resultsR# t
have_localt
have_absolutet mod_namet new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyt transform2 s0
c C s | i d o t S| i d d d } t | i } t | | } t t t | d p t Sx6 d t d d d d g D] } t | | o t Sq Wt S(
Nu .i i s __init__.pys .pys .pycs .sos .sls .pyd( t
startswithR( t splitR t filenameR R R R) ( R t imp_namet base_patht ext( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyR& T s ( t __name__t
__module__t PATTERNR R1 R& ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyR & s "N( t __doc__R R t os.pathR R R R t
fixer_utilR R R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyt s
" fixes/fix_intern.pyo 0000666 00000003053 15050104230 0010553 0 ustar 00
Lc @ s_ d Z d d k l Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s/ Fixer for intern().
intern(s) -> sys.intern(s)i ( t pytree( t
fixer_base( t Namet Attrt touch_importt FixInternc B s e Z d Z d Z RS( s
power< 'intern'
trailer< lpar='('
( not(arglist | argument) any ','> )
rpar=')' >
after=any*
>
c
C s | i } | d i } | i | i j o | i } n t i | i | i g } | d } | o+ g } | D] } | | i qv ~ } n t i | i t t d t d t i | i | d i | | d i g g | } | i
| _
t d d | | S( Nt objt afteru sysu internt lpart rpar(
t symst clonet typet arglistR t Nodet powerR R t trailert prefixR t None(
t selft nodet resultsR
R t
newarglistR t _[1]t nt new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_intern.pyt transform s
+ U ( t __name__t
__module__t PATTERNR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_intern.pyR s
N(
t __doc__t R R t
fixer_utilR R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_intern.pyt s fixes/fix_zip.pyc 0000666 00000002461 15050104230 0010044 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s7
Fixer that changes zip(seq0, seq1, ...) into list(zip(seq0, seq1, ...)
unless there exists a 'from future_builtins import zip' statement in the
top-level namespace.
We avoid the transformation if the zip() call is directly contained in
iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or for V in <>:.
i ( t
fixer_base( t Namet Callt in_special_contextt FixZipc B s e Z d Z d Z d Z RS( s:
power< 'zip' args=trailer< '(' [any] ')' >
>
s future_builtins.zipc C sd | i | o d St | o d S| i } d | _ t t d | g } | i | _ | S( Nu u list( t should_skipR t Nonet clonet prefixR R ( t selft nodet resultst new( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_zip.pyt transform s
( t __name__t
__module__t PATTERNt skip_onR
( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_zip.pyR s N( t __doc__t R t
fixer_utilR R R t ConditionalFixR ( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_zip.pyt s fixes/fix_intern.py 0000666 00000002522 15050104230 0010374 0 ustar 00 # Copyright 2006 Georg Brandl.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for intern().
intern(s) -> sys.intern(s)"""
# Local imports
from .. import pytree
from .. import fixer_base
from ..fixer_util import Name, Attr, touch_import
class FixIntern(fixer_base.BaseFix):
PATTERN = """
power< 'intern'
trailer< lpar='('
( not(arglist | argument) any ','> )
rpar=')' >
after=any*
>
"""
def transform(self, node, results):
syms = self.syms
obj = results["obj"].clone()
if obj.type == syms.arglist:
newarglist = obj.clone()
else:
newarglist = pytree.Node(syms.arglist, [obj.clone()])
after = results["after"]
if after:
after = [n.clone() for n in after]
new = pytree.Node(syms.power,
Attr(Name(u"sys"), Name(u"intern")) +
[pytree.Node(syms.trailer,
[results["lpar"].clone(),
newarglist,
results["rpar"].clone()])] + after)
new.prefix = node.prefix
touch_import(None, u'sys', node)
return new
fixes/fix_except.pyc 0000666 00000005750 15050104230 0010536 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z l Z d Z
d e i f d YZ d S(
s Fixer for except statements with named exceptions.
The following cases will be converted:
- "except E, T:" where T is a name:
except E as T:
- "except E, T:" where T is not a name, tuple or list:
except E as t:
T = t
This is done because the target of an "except" clause must be a
name.
- "except E, T:" where T is a tuple or list literal:
except E as t:
T = t.args
i ( t pytree( t token( t
fixer_base( t Assignt Attrt Namet is_tuplet is_listt symsc c sf x_ t | D]Q \ } } | i t i j o2 | i d i d j o | | | d f Vq^ q
q
Wd S( Ni u excepti ( t enumeratet typeR t
except_clauset childrent value( t nodest it n( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyt find_excepts s
t FixExceptc B s e Z d Z d Z RS( s1
try_stmt< 'try' ':' (simple_stmt | suite)
cleanup=(except_clause ':' (simple_stmt | suite))+
tail=(['except' ':' (simple_stmt | suite)]
['else' ':' (simple_stmt | suite)]
['finally' ':' (simple_stmt | suite)]) >
c C sO | i } g } | d D] } | | i q ~ } g } | d D] } | | i qC ~ } xt | D]\ }
} t |
i d j ot|
i d d !\ } }
} |
i t d d d | i t i j ot | i
d d } | i } d | _ | i | | i } | i } x2 t | D]$ \ } } t
| t i o Pq/q/Wt | p
t | o" t | t | t d } n t | | } x( t | | D] } | i d
| qW| i | | q| i d j o
d | _ qql ql Wg } | i d D] } | | i q~ | | } t i | i | S( Nt tailt cleanupi i u ast prefixu u u argsi i ( R t cloneR t lenR t replaceR R
R t NAMEt new_nameR R t
isinstanceR t NodeR R R R t reversedt insert_child( t selft nodet resultsR t _[1]R R t _[2]t cht try_cleanupR t e_suitet Et commat Nt new_Nt targett suite_stmtsR t stmtt assignt childt _[3]t cR ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyt transform. s< ++
" 6( t __name__t
__module__t PATTERNR2 ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyR $ s N( t __doc__t R t pgen2R R t
fixer_utilR R R R R R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyt s . fixes/fix_itertools_imports.pyo 0000666 00000003427 15050104230 0013062 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( sA Fixer for imports of itertools.(imap|ifilter|izip|ifilterfalse) i( t
fixer_base( t BlankLinet symst tokent FixItertoolsImportsc B s e Z d e Z d Z RS( sT
import_from< 'from' 'itertools' 'import' imports=any >
c C s | d } | i t i j p | i o
| g } n
| i } x | d d d D] } | i t i j o | i } | } n | i d } | i } | d j o d | _ | i qR | d j o | i d | _ qR qR W| i p | g } t
} x@ | D]8 } | o! | i t i j o | i q| t
N} qW| d i t i j o | d i n | i p t | d
d p | i
d j o | i }
t } |
| _ | Sd S( Nt importsi i u imapu izipu ifilteru ifilterfalseu filterfalseit value( u imapu izipu ifilter( t typeR t import_as_namet childrenR t NAMER t Nonet removet changedt Truet COMMAt getattrt parentt prefixR ( t selft nodet resultsR R t childt membert name_nodet member_namet remove_commat p( ( s; /usr/lib64/python2.6/lib2to3/fixes/fix_itertools_imports.pyt transform
s@
( t __name__t
__module__t localst PATTERNR ( ( ( s; /usr/lib64/python2.6/lib2to3/fixes/fix_itertools_imports.pyR s
N( t __doc__t lib2to3R t lib2to3.fixer_utilR R R t BaseFixR ( ( ( s; /usr/lib64/python2.6/lib2to3/fixes/fix_itertools_imports.pyt s fixes/fix_throw.pyc 0000666 00000003720 15050104230 0010404 0 ustar 00
Lc @ s{ d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z d e i f d YZ
d S( s Fixer for generator.throw(E, V, T).
g.throw(E) -> g.throw(E)
g.throw(E, V) -> g.throw(E(V))
g.throw(E, V, T) -> g.throw(E(V).with_traceback(T))
g.throw("foo"[, V[, T]]) will warn about string exceptions.i ( t pytree( t token( t
fixer_base( t Namet Callt ArgListt Attrt is_tuplet FixThrowc B s e Z d Z d Z RS( s
power< any trailer< '.' 'throw' >
trailer< '(' args=arglist< exc=any ',' val=any [',' tb=any] > ')' >
>
|
power< any trailer< '.' 'throw' > trailer< '(' exc=any ')' > >
c
C s` | i } | d i } | i t i j o | i | d d S| i d } | d j o d S| i } t | o5 g } | i d d !D] } | | i q ~ } n d | _
| g } | d } d | j om | d i }
d |
_
t | | } t | t
d t |
g g } | i t i | i | n | i t | | d S(
Nt excs+ Python 3 does not support string exceptionsu vali iu t argst tbu with_traceback( t symst clonet typeR t STRINGt cannot_convertt gett NoneR t childrent prefixR R R R t replaceR t Nodet power(
t selft nodet resultsR R t valt _[1]t cR
t
throw_argsR t et with_tb( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_throw.pyt transform s*
5
% ( t __name__t
__module__t PATTERNR! ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_throw.pyR s N( t __doc__t R t pgen2R R t
fixer_utilR R R R R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_throw.pyt s
( fixes/fix_funcattrs.pyc 0000666 00000002105 15050104230 0011246 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s3 Fix function attribute names (f.func_x -> f.__x__).i ( t
fixer_base( t Namet FixFuncattrsc B s e Z d Z d Z RS( s
power< any+ trailer< '.' attr=('func_closure' | 'func_doc' | 'func_globals'
| 'func_name' | 'func_defaults' | 'func_code'
| 'func_dict') > any* >
c C s9 | d d } | i t d | i d d | i d S( Nt attri u __%s__i t prefix( t replaceR t valueR ( t selft nodet resultsR ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_funcattrs.pyt transform s ( t __name__t
__module__t PATTERNR
( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_funcattrs.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_funcattrs.pyt s fixes/fix_dict.py 0000666 00000007326 15050104230 0010027 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for dict methods.
d.keys() -> list(d.keys())
d.items() -> list(d.items())
d.values() -> list(d.values())
d.iterkeys() -> iter(d.keys())
d.iteritems() -> iter(d.items())
d.itervalues() -> iter(d.values())
d.viewkeys() -> d.keys()
d.viewitems() -> d.items()
d.viewvalues() -> d.values()
Except in certain very specific contexts: the iter() can be dropped
when the context is list(), sorted(), iter() or for...in; the list()
can be dropped when the context is list() or sorted() (but not iter()
or for...in!). Special contexts that apply to both: list(), sorted(), tuple()
set(), any(), all(), sum().
Note: iter(d.keys()) could be written as iter(d) but since the
original d.iterkeys() was also redundant we don't fix this. And there
are (rare) contexts where it makes a difference (e.g. when passing it
as an argument to a function that introspects the argument).
"""
# Local imports
from .. import pytree
from .. import patcomp
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, Call, LParen, RParen, ArgList, Dot
from .. import fixer_util
iter_exempt = fixer_util.consuming_calls | set(["iter"])
class FixDict(fixer_base.BaseFix):
PATTERN = """
power< head=any+
trailer< '.' method=('keys'|'items'|'values'|
'iterkeys'|'iteritems'|'itervalues'|
'viewkeys'|'viewitems'|'viewvalues') >
parens=trailer< '(' ')' >
tail=any*
>
"""
def transform(self, node, results):
head = results["head"]
method = results["method"][0] # Extract node for method name
tail = results["tail"]
syms = self.syms
method_name = method.value
isiter = method_name.startswith(u"iter")
isview = method_name.startswith(u"view")
if isiter or isview:
method_name = method_name[4:]
assert method_name in (u"keys", u"items", u"values"), repr(method)
head = [n.clone() for n in head]
tail = [n.clone() for n in tail]
special = not tail and self.in_special_context(node, isiter)
args = head + [pytree.Node(syms.trailer,
[Dot(),
Name(method_name,
prefix=method.prefix)]),
results["parens"].clone()]
new = pytree.Node(syms.power, args)
if not (special or isview):
new.prefix = u""
new = Call(Name(u"iter" if isiter else u"list"), [new])
if tail:
new = pytree.Node(syms.power, [new] + tail)
new.prefix = node.prefix
return new
P1 = "power< func=NAME trailer< '(' node=any ')' > any* >"
p1 = patcomp.compile_pattern(P1)
P2 = """for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
"""
p2 = patcomp.compile_pattern(P2)
def in_special_context(self, node, isiter):
if node.parent is None:
return False
results = {}
if (node.parent.parent is not None and
self.p1.match(node.parent.parent, results) and
results["node"] is node):
if isiter:
# iter(d.iterkeys()) -> iter(d.keys()), etc.
return results["func"].value in iter_exempt
else:
# list(d.keys()) -> list(d.keys()), etc.
return results["func"].value in fixer_util.consuming_calls
if not isiter:
return False
# for ... in d.iterkeys() -> for ... in d.keys(), etc.
return self.p2.match(node.parent, results) and results["node"] is node
fixes/fix_paren.py 0000666 00000002262 15050104230 0010203 0 ustar 00 """Fixer that addes parentheses where they are required
This converts ``[x for x in 1, 2]`` to ``[x for x in (1, 2)]``."""
# By Taek Joo Kim and Benjamin Peterson
# Local imports
from .. import fixer_base
from ..fixer_util import LParen, RParen
# XXX This doesn't support nested for loops like [x for x in 1, 2 for x in 1, 2]
class FixParen(fixer_base.BaseFix):
PATTERN = """
atom< ('[' | '(')
(listmaker< any
comp_for<
'for' NAME 'in'
target=testlist_safe< any (',' any)+ [',']
>
[any]
>
>
|
testlist_gexp< any
comp_for<
'for' NAME 'in'
target=testlist_safe< any (',' any)+ [',']
>
[any]
>
>)
(']' | ')') >
"""
def transform(self, node, results):
target = results["target"]
lparen = LParen()
lparen.prefix = target.prefix
target.prefix = u"" # Make it hug the parentheses
target.insert_child(0, lparen)
target.append_child(RParen())
fixes/fix_standarderror.pyc 0000666 00000001500 15050104230 0012105 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s% Fixer for StandardError -> Exception.i ( t
fixer_base( t Namet FixStandarderrorc B s e Z d Z d Z RS( s-
'StandardError'
c C s t d d | i S( Nu Exceptiont prefix( R R ( t selft nodet results( ( s7 /usr/lib64/python2.6/lib2to3/fixes/fix_standarderror.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s7 /usr/lib64/python2.6/lib2to3/fixes/fix_standarderror.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s7 /usr/lib64/python2.6/lib2to3/fixes/fix_standarderror.pyt s fixes/fix_standarderror.pyo 0000666 00000001500 15050104230 0012121 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s% Fixer for StandardError -> Exception.i ( t
fixer_base( t Namet FixStandarderrorc B s e Z d Z d Z RS( s-
'StandardError'
c C s t d d | i S( Nu Exceptiont prefix( R R ( t selft nodet results( ( s7 /usr/lib64/python2.6/lib2to3/fixes/fix_standarderror.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s7 /usr/lib64/python2.6/lib2to3/fixes/fix_standarderror.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s7 /usr/lib64/python2.6/lib2to3/fixes/fix_standarderror.pyt s fixes/fix_operator.py 0000666 00000002730 15050104230 0010731 0 ustar 00 """Fixer for operator.{isCallable,sequenceIncludes}
operator.isCallable(obj) -> hasattr(obj, '__call__')
operator.sequenceIncludes(obj) -> operator.contains(obj)
"""
# Local imports
from .. import fixer_base
from ..fixer_util import Call, Name, String
class FixOperator(fixer_base.BaseFix):
methods = "method=('isCallable'|'sequenceIncludes')"
func = "'(' func=any ')'"
PATTERN = """
power< module='operator'
trailer< '.' %(methods)s > trailer< %(func)s > >
|
power< %(methods)s trailer< %(func)s > >
""" % dict(methods=methods, func=func)
def transform(self, node, results):
method = results["method"][0]
if method.value == u"sequenceIncludes":
if "module" not in results:
# operator may not be in scope, so we can't make a change.
self.warning(node, "You should use operator.contains here.")
else:
method.value = u"contains"
method.changed()
elif method.value == u"isCallable":
if "module" not in results:
self.warning(node,
"You should use hasattr(%s, '__call__') here." %
results["func"].value)
else:
func = results["func"]
args = [func.clone(), String(u", "), String(u"'__call__'")]
return Call(Name(u"hasattr"), args, prefix=node.prefix)
fixes/fix_raw_input.py 0000666 00000000656 15050104230 0011113 0 ustar 00 """Fixer that changes raw_input(...) into input(...)."""
# Author: Andre Roberge
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixRawInput(fixer_base.BaseFix):
PATTERN = """
power< name='raw_input' trailer< '(' [any] ')' > any* >
"""
def transform(self, node, results):
name = results["name"]
name.replace(Name(u"input", prefix=name.prefix))
fixes/fix_reduce.py 0000666 00000001433 15050104230 0010344 0 ustar 00 # Copyright 2008 Armin Ronacher.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for reduce().
Makes sure reduce() is imported from the functools module if reduce is
used in that module.
"""
from lib2to3 import fixer_base
from lib2to3.fixer_util import touch_import
class FixReduce(fixer_base.BaseFix):
PATTERN = """
power< 'reduce'
trailer< '('
arglist< (
(not(argument) any ','
not(argument
>
"""
def transform(self, node, results):
touch_import(u'functools', u'reduce', node)
fixes/fix_unicode.py 0000666 00000001270 15050104230 0010522 0 ustar 00 """Fixer that changes unicode to str, unichr to chr, and u"..." into "...".
"""
import re
from ..pgen2 import token
from .. import fixer_base
_mapping = {u"unichr" : u"chr", u"unicode" : u"str"}
_literal_re = re.compile(ur"[uU][rR]?[\'\"]")
class FixUnicode(fixer_base.BaseFix):
PATTERN = "STRING | 'unicode' | 'unichr'"
def transform(self, node, results):
if node.type == token.NAME:
new = node.clone()
new.value = _mapping[node.value]
return new
elif node.type == token.STRING:
if _literal_re.match(node.value):
new = node.clone()
new.value = new.value[1:]
return new
fixes/fix_buffer.pyo 0000666 00000001652 15050104230 0010530 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s4 Fixer that changes buffer(...) into memoryview(...).i ( t
fixer_base( t Namet FixBufferc B s e Z e Z d Z d Z RS( sR
power< name='buffer' trailer< '(' [any] ')' > any* >
c C s* | d } | i t d d | i d S( Nt nameu
memoryviewt prefix( t replaceR R ( t selft nodet resultsR ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_buffer.pyt transform s
( t __name__t
__module__t Truet explicitt PATTERNR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_buffer.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_buffer.pyt s fixes/fix_itertools.pyo 0000666 00000003245 15050104230 0011303 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sT Fixer for itertools.(imap|ifilter|izip) --> (map|filter|zip) and
itertools.ifilterfalse --> itertools.filterfalse (bugs 2360-2363)
imports from itertools are fixed in fix_itertools_import.py
If itertools is imported as something else (ie: import itertools as it;
it.izip(spam, eggs)) method calls will not get fixed.
i ( t
fixer_base( t Namet FixItertoolsc B s* e Z d Z d e Z d Z d Z RS( s( ('imap'|'ifilter'|'izip'|'ifilterfalse')s
power< it='itertools'
trailer<
dot='.' func=%(it_funcs)s > trailer< '(' [any] ')' > >
|
power< func=%(it_funcs)s trailer< '(' [any] ')' > >
i c C s d } | d d } d | j oV | i d j oF | d | d } } | i } | i | i | i i | n | p | i } | i t | i d d | d S( Nt funci t itu ifilterfalset doti t prefix( t Nonet valueR t removet parentt replaceR ( t selft nodet resultsR R R R ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_itertools.pyt transform s
( t __name__t
__module__t it_funcst localst PATTERNt run_orderR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_itertools.pyR s
N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_itertools.pyt s fixes/fix_getcwdu.pyo 0000666 00000001611 15050104230 0010714 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s1
Fixer that changes os.getcwdu() to os.getcwd().
i ( t
fixer_base( t Namet
FixGetcwduc B s e Z d Z d Z RS( sR
power< 'os' trailer< dot='.' name='getcwdu' > any* >
c C s* | d } | i t d d | i d S( Nt nameu getcwdt prefix( t replaceR R ( t selft nodet resultsR ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_getcwdu.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_getcwdu.pyR
s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_getcwdu.pyt s fixes/fix_apply.pyc 0000666 00000003502 15050104230 0010364 0 ustar 00
Lc @ so d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z d e i
f d YZ d S( sI Fixer for apply().
This converts apply(func, v, k) into (func)(*v, **k).i ( t pytree( t token( t
fixer_base( t Callt Commat parenthesizet FixApplyc B s e Z d Z d Z RS( s.
power< 'apply'
trailer<
'('
arglist<
(not argument
')'
>
>
c C s` | i } | p t | d } | d } | i d } | i } | i } | i t i | i f j o= | i | i j p | i
d i t i j o t | } n d | _ | i } d | _ | d j o | i } d | _ n t i t i d | g } | d j o9 | i t t i t i d | g d | d _ n t | | d | S(
Nt funct argst kwdsit u *u **u t prefix( t symst AssertionErrort getR t clonet typeR t NAMEt atomt powert childrent
DOUBLESTARR t NoneR t Leaft STARt extendR R ( t selft nodet resultsR R R R R t l_newargs( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_apply.pyt transform s0
( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_apply.pyR s N( t __doc__R
R t pgen2R R t
fixer_utilR R R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_apply.pyt s
fixes/fix_imports.pyc 0000666 00000012351 15050104230 0010736 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z l Z h0 d d 6d d 6d d 6d d
6d d 6d
d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d! d" 6d# d$ 6d% d& 6d' d( 6d) d* 6d+ d, 6d- d. 6d/ d0 6d1 d2 6d3 d4 6d5 d6 6d7 d8 6d9 d: 6d; d< 6d= d> 6d? d@ 6dA dB 6dC dD 6dC dE 6dF dG 6dH dI 6dJ dK 6dL dM 6dN dO 6dP dQ 6dP dR 6dP dS 6dT dU 6dV dW 6dV dX 6dY dZ 6d[ d\ 6Z d] Z e d^ Z d_ e i f d` YZ
da S(b s/ Fix incompatible imports and module references.i ( t
fixer_base( t Namet
attr_chaint iot StringIOt cStringIOt picklet cPicklet builtinst __builtin__t copyregt copy_regt queuet Queuet socketservert SocketServert configparsert ConfigParsert reprlibt reprs tkinter.filedialogt
FileDialogt tkFileDialogs tkinter.simpledialogt SimpleDialogt tkSimpleDialogs tkinter.colorchoosert tkColorChoosers tkinter.commondialogt tkCommonDialogs tkinter.dialogt Dialogs tkinter.dndt Tkdnds tkinter.fontt tkFonts tkinter.messageboxt tkMessageBoxs tkinter.scrolledtextt ScrolledTexts tkinter.constantst Tkconstantss tkinter.tixt Tixs tkinter.ttkt ttkt tkintert Tkintert _markupbaset
markupbaset winregt _winregt _threadt threadt
_dummy_threadt dummy_threads dbm.bsdt dbhashs dbm.dumbt dumbdbms dbm.ndbmt dbms dbm.gnut gdbms
xmlrpc.clientt xmlrpclibs
xmlrpc.servert DocXMLRPCServert SimpleXMLRPCServers http.clientt httplibs
html.entitiest htmlentitydefss html.parsert
HTMLParsers http.cookiest Cookies http.cookiejart cookielibs http.servert BaseHTTPServert SimpleHTTPServert
CGIHTTPServert
subprocesst commandst collectionst
UserStringt UserLists urllib.parset urlparses urllib.robotparsert robotparserc C s d d i t t | d S( Nt (t |t )( t joint mapR ( t members( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyt
alternates= s c c st d i g } | D] } | d | q ~ } t | i } d | | f Vd | Vd | | f Vd | Vd S( Ns | s module_name='%s'sy name_import=import_name< 'import' ((%s) |
multiple_imports=dotted_as_names< any* (%s) any* >) >
s import_from< 'from' (%s) 'import' ['(']
( any | import_as_name< any 'as' any > |
import_as_names< any* >) [')'] >
s import_name< 'import' (dotted_as_name< (%s) 'as' any > |
multiple_imports=dotted_as_names<
any* dotted_as_name< (%s) 'as' any > any* >) >
s3 power< bare_with_attr=(%s) trailer<'.' any > any* >( RE RH t keys( t mappingt _[1]t keyt mod_listt
bare_names( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyt
build_patternA s . t
FixImportsc B sA e Z e Z d Z d Z d Z d Z d Z d Z RS( i c C s d i t | i S( NRC ( RE RO RJ ( t self( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyRO ^ s c C s& | i | _ t t | i d S( N( RO t PATTERNt superRP t compile_pattern( RQ ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyRT a s c sf t t | i | } | o= d | j o+ t f d t | d D o t S| St S( Nt bare_with_attrc 3 s x | ] } | Vq Wd S( N( ( t .0t obj( t match( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pys o s t parent( RS RP RX t anyR t False( RQ t nodet results( ( RX s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyRX h s
&c C s& t t | i | | h | _ d S( N( RS RP t
start_treet replace( RQ t treet filename( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_imports.pyR^ t s c C s | i d } | o | i } t | i | } | i t | d | i d | j o | | i | sj
fixes/fix_import.py 0000666 00000006242 15050104230 0010412 0 ustar 00 """Fixer for import statements.
If spam is being imported from the local directory, this import:
from spam import eggs
Becomes:
from .spam import eggs
And this import:
import spam
Becomes:
from . import spam
"""
# Local imports
from .. import fixer_base
from os.path import dirname, join, exists, sep
from ..fixer_util import FromImport, syms, token
def traverse_imports(names):
"""
Walks over all the names imported in a dotted_as_names node.
"""
pending = [names]
while pending:
node = pending.pop()
if node.type == token.NAME:
yield node.value
elif node.type == syms.dotted_name:
yield "".join([ch.value for ch in node.children])
elif node.type == syms.dotted_as_name:
pending.append(node.children[0])
elif node.type == syms.dotted_as_names:
pending.extend(node.children[::-2])
else:
raise AssertionError("unkown node type")
class FixImport(fixer_base.BaseFix):
PATTERN = """
import_from< 'from' imp=any 'import' ['('] any [')'] >
|
import_name< 'import' imp=any >
"""
def start_tree(self, tree, name):
super(FixImport, self).start_tree(tree, name)
self.skip = "absolute_import" in tree.future_features
def transform(self, node, results):
if self.skip:
return
imp = results['imp']
if node.type == syms.import_from:
# Some imps are top-level (eg: 'import ham')
# some are first level (eg: 'import ham.eggs')
# some are third level (eg: 'import ham.eggs as spam')
# Hence, the loop
while not hasattr(imp, 'value'):
imp = imp.children[0]
if self.probably_a_local_import(imp.value):
imp.value = u"." + imp.value
imp.changed()
else:
have_local = False
have_absolute = False
for mod_name in traverse_imports(imp):
if self.probably_a_local_import(mod_name):
have_local = True
else:
have_absolute = True
if have_absolute:
if have_local:
# We won't handle both sibling and absolute imports in the
# same statement at the moment.
self.warning(node, "absolute and local imports together")
return
new = FromImport(u".", [imp])
new.prefix = node.prefix
return new
def probably_a_local_import(self, imp_name):
if imp_name.startswith(u"."):
# Relative imports are certainly not local imports.
return False
imp_name = imp_name.split(u".", 1)[0]
base_path = dirname(self.filename)
base_path = join(base_path, imp_name)
# If there is no __init__.py next to the file its not in a package
# so can't be a relative import.
if not exists(join(dirname(base_path), "__init__.py")):
return False
for ext in [".py", sep, ".pyc", ".so", ".sl", ".pyd"]:
if exists(base_path + ext):
return True
return False
fixes/fix_intern.pyc 0000666 00000003053 15050104230 0010537 0 ustar 00
Lc @ s_ d Z d d k l Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s/ Fixer for intern().
intern(s) -> sys.intern(s)i ( t pytree( t
fixer_base( t Namet Attrt touch_importt FixInternc B s e Z d Z d Z RS( s
power< 'intern'
trailer< lpar='('
( not(arglist | argument) any ','> )
rpar=')' >
after=any*
>
c
C s | i } | d i } | i | i j o | i } n t i | i | i g } | d } | o+ g } | D] } | | i qv ~ } n t i | i t t d t d t i | i | d i | | d i g g | } | i
| _
t d d | | S( Nt objt afteru sysu internt lpart rpar(
t symst clonet typet arglistR t Nodet powerR R t trailert prefixR t None(
t selft nodet resultsR
R t
newarglistR t _[1]t nt new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_intern.pyt transform s
+ U ( t __name__t
__module__t PATTERNR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_intern.pyR s
N(
t __doc__t R R t
fixer_utilR R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_intern.pyt s fixes/fix_getcwdu.pyc 0000666 00000001611 15050104230 0010700 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s1
Fixer that changes os.getcwdu() to os.getcwd().
i ( t
fixer_base( t Namet
FixGetcwduc B s e Z d Z d Z RS( sR
power< 'os' trailer< dot='.' name='getcwdu' > any* >
c C s* | d } | i t d d | i d S( Nt nameu getcwdt prefix( t replaceR R ( t selft nodet resultsR ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_getcwdu.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_getcwdu.pyR
s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_getcwdu.pyt s fixes/fix_reduce.pyo 0000666 00000002277 15050104230 0010532 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sq Fixer for reduce().
Makes sure reduce() is imported from the functools module if reduce is
used in that module.
i( t
fixer_base( t touch_importt FixReducec B s e Z d Z d Z RS( si
power< 'reduce'
trailer< '('
arglist< (
(not(argument) any ','
not(argument
>
c C s t d d | d S( Nu functoolsu reduce( R ( t selft nodet results( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_reduce.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_reduce.pyR s N( t __doc__t lib2to3R t lib2to3.fixer_utilR t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_reduce.pyt s fixes/fix_numliterals.pyo 0000666 00000002370 15050104230 0011614 0 ustar 00
Lc @ sS d Z d d k l Z d d k l Z d d k l Z d e i f d YZ d S( s- Fixer that turns 1L into 1, 0755 into 0o755.
i ( t token( t
fixer_base( t Numbert FixNumliteralsc B s# e Z e i Z d Z d Z RS( c C s$ | i i d p | i d d j S( Nu 0iu Ll( t valuet
startswith( t selft node( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyt match s c C s | i } | d d j o | d } nI | i d o8 | i o+ t t | d j o d | d } n t | d | i S( Niu Llu 0i u 0ot prefix( R R t isdigitt lent setR R ( R R t resultst val( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyt transform s 6( t __name__t
__module__R t NUMBERt _accept_typeR R ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyR s N( t __doc__t pgen2R t R t
fixer_utilR t BaseFixR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_numliterals.pyt s fixes/fix_basestring.py 0000666 00000000450 15050104230 0011234 0 ustar 00 """Fixer for basestring -> str."""
# Author: Christian Heimes
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixBasestring(fixer_base.BaseFix):
PATTERN = "'basestring'"
def transform(self, node, results):
return Name(u"str", prefix=node.prefix)
fixes/fix_idioms.pyo 0000666 00000010546 15050104230 0010545 0 ustar 00
Lc @ sm d Z d d k l Z d d k l Z l Z l Z l Z l Z l Z d Z
d Z d e i f d YZ
d S( s Adjust some old Python 2 idioms to their modern counterparts.
* Change some type comparisons to isinstance() calls:
type(x) == T -> isinstance(x, T)
type(x) is T -> isinstance(x, T)
type(x) != T -> not isinstance(x, T)
type(x) is not T -> not isinstance(x, T)
* Change "while 1:" into "while True:".
* Change both
v = list(EXPR)
v.sort()
foo(v)
and the more general
v = EXPR
v.sort()
foo(v)
into
v = sorted(EXPR)
foo(v)
i ( t
fixer_base( t Callt Commat Namet Nodet BlankLinet symss0 (n='!=' | '==' | 'is' | n=comp_op< 'is' 'not' >)s( power< 'type' trailer< '(' x=any ')' > >t FixIdiomsc B sQ e Z e Z d e e e e f Z d Z d Z d Z d Z
d Z RS( s
isinstance=comparison< %s %s T=any >
|
isinstance=comparison< T=any %s %s >
|
while_stmt< 'while' while='1' ':' any+ >
|
sorted=any<
any*
simple_stmt<
expr_stmt< id1=any '='
power< list='list' trailer< '(' (not arglist) any ')' > >
>
'\n'
>
sort=
simple_stmt<
power< id2=any
trailer< '.' 'sort' > trailer< '(' ')' >
>
'\n'
>
next=any*
>
|
sorted=any<
any*
simple_stmt< expr_stmt< id1=any '=' expr=any > '\n' >
sort=
simple_stmt<
power< id2=any
trailer< '.' 'sort' > trailer< '(' ')' >
>
'\n'
>
next=any*
>
c C sO t t | i | } | o, d | j o | d | d j o | Sd S| S( Nt sortedt id1t id2( t superR t matcht None( t selft nodet r( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR P s c C sj d | j o | i | | Sd | j o | i | | Sd | j o | i | | St d d S( Nt
isinstancet whileR s
Invalid match( t transform_isinstancet transform_whilet transform_sortt RuntimeError( R R t results( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyt transform[ s
c C s | d i } | d i } d | _ d | _ t t d | t | g } d | j o+ d | _ t t i t d | g } n | i | _ | S( Nt xt Tu u u
isinstancet nu not( t clonet prefixR R R R R t not_test( R R R R R t test( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR e s !
" c C s* | d } | i t d d | i d S( NR u TrueR ( t replaceR R ( R R R t one( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR q s
c C sE | d } | d } | i d } | i d } | o | i t d d | i nU | oA | i } d | _ | i t t d | g d | i n
t d | i | i } d | j ot | o: | i d d
| d
i f } d i | | d
_ qAt
}
| i i |
| i d d
|
_ n d S( Nt sortt nextt listt expru sortedR u s should not have reached hereu
i (
t getR R R R R R t removet
rpartitiont joinR t parentt append_child( R R R t sort_stmtt next_stmtt list_callt simple_exprt newt btwnt prefix_linest end_line( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR u s*
( t __name__t
__module__t Truet explicitt TYPEt CMPt PATTERNR R R R R ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyR % s '
N( t __doc__t R t
fixer_utilR R R R R R R9 R8 t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_idioms.pyt s
. fixes/fix_nonzero.pyc 0000666 00000002051 15050104230 0010727 0 ustar 00
Lc @ sI d Z d d k l Z d d k l Z l Z d e i f d YZ d S( s* Fixer for __nonzero__ -> __bool__ methods.i ( t
fixer_base( t Namet symst
FixNonzeroc B s e Z d Z d Z RS( s
classdef< 'class' any+ ':'
suite< any*
funcdef< 'def' name='__nonzero__'
parameters< '(' NAME ')' > any+ >
any* > >
c C s0 | d } t d d | i } | i | d S( Nt nameu __bool__t prefix( R R t replace( t selft nodet resultsR t new( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_nonzero.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_nonzero.pyR s N( t __doc__t R t
fixer_utilR R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_nonzero.pyt s fixes/fix_xrange.pyo 0000666 00000005774 15050104230 0010554 0 ustar 00
Lc @ s_ d Z d d k l Z d d k l Z l Z l Z d d k l Z d e i f d YZ d S( s/ Fixer that changes xrange(...) into range(...).i ( t
fixer_base( t Namet Callt consuming_calls( t patcompt FixXrangec B sn e Z d Z d Z d Z d Z d Z d Z d Z e i
e Z d Z e i
e Z
d Z RS( s
power<
(name='range'|name='xrange') trailer< '(' args=any ')' >
rest=any* >
c C s) t t | i | | t | _ d S( N( t superR t
start_treet sett transformed_xranges( t selft treet filename( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR s c C s
d | _ d S( N( t NoneR ( R
R R ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyt finish_tree s c C sb | d } | i d j o | i | | S| i d j o | i | | St t | d S( Nt nameu xrangeu range( t valuet transform_xranget transform_ranget
ValueErrort repr( R
t nodet resultsR ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyt transform s
c C s@ | d } | i t d d | i | i i t | d S( NR u ranget prefix( t replaceR R R t addt id( R
R R R ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR $ s
c C s t | | i j o{ | i | oj t t d | d i g } t t d | g d | i } x | d D] } | i | qu W| Sd S( Nu ranget argsu listR t rest( R R t in_special_contextR R t cloneR t append_child( R
R R t
range_callt list_callt n( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR * s " s3 power< func=NAME trailer< '(' node=any ')' > any* >s for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
| comparison< any 'in' node=any any*>
c C s | i d j o t Sh } | i i d j o? | i i | i i | o# | d | j o | d i t j S| i i | i | o | d | j S( NR t func( t parentR
t Falset p1t matchR R t p2( R
R R ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR ? s ( t __name__t
__module__t PATTERNR R R R R t P1R t compile_patternR' t P2R) R ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyR s N(
t __doc__t R t
fixer_utilR R R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_xrange.pyt s fixes/fix_print.pyc 0000666 00000005304 15050104230 0010375 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l
Z
l Z l Z e i
d Z d e i f d YZ d
S( s Fixer for print.
Change:
'print' into 'print()'
'print ...' into 'print(...)'
'print ... ,' into 'print(..., end=" ")'
'print >>x, ...' into 'print(..., file=x)'
No changes are applied if print_function is imported from __future__
i ( t patcomp( t pytree( t token( t
fixer_base( t Namet Callt Commat Stringt is_tuples" atom< '(' [atom|STRING|NAME] ')' >t FixPrintc B s e Z d Z d Z d Z RS( sP
simple_stmt< any* bare='print' any* > | print_stmt
c C sW | p t | i d } | o* | i t t d g d | i d S| i d t d j p t | i d } t | d j o t i | d o d Sd } } } | o( | d t j o | d } d } n | oX | d t i
t i d j o8 t | d j p t | d i } | d
} n g } | D] } | | i qO~ }
|
o d |
d _ n | d j p | d j p
| d j o | d j o# | i |
d t t | n | d j o# | i |
d
t t | n | d j o | i |
d | q2n t t d |
} | i | _ | S( Nt bareu printt prefixi i it u >>i i u u sepu endu file( t AssertionErrort gett replaceR R R t childrent lent parend_exprt matcht NoneR R t LeafR t
RIGHTSHIFTt clonet add_kwargR t repr( t selft nodet resultst
bare_printt argst sept endt filet _[1]t argt l_argst n_stmt( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyt transform# s>
!
'
'''
#
#
c C st d | _ t i | i i t | t i t i d | f } | o | i t
d | _ n | i | d S( Nu u =u ( R R t Nodet symst argumentR R R t EQUALt appendR ( R t l_nodest s_kwdt n_exprt
n_argument( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyR K s
( t __name__t
__module__t PATTERNR& R ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyR s (N( t __doc__t R R t pgen2R R t
fixer_utilR R R R R t compile_patternR t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_print.pyt s ( fixes/fix_imports2.pyo 0000666 00000001172 15050104230 0011033 0 ustar 00
Lc @ sG d Z d d k l Z h d d 6d d 6Z d e i f d YZ d S( sT Fix incompatible imports and module references that must be fixed after
fix_imports.i ( t fix_importst dbmt whichdbt anydbmt FixImports2c B s e Z d Z e Z RS( i ( t __name__t
__module__t run_ordert MAPPINGt mapping( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_imports2.pyR s N( t __doc__t R R t
FixImportsR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_imports2.pyt s
fixes/fix_zip.pyo 0000666 00000002461 15050104230 0010060 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s7
Fixer that changes zip(seq0, seq1, ...) into list(zip(seq0, seq1, ...)
unless there exists a 'from future_builtins import zip' statement in the
top-level namespace.
We avoid the transformation if the zip() call is directly contained in
iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or for V in <>:.
i ( t
fixer_base( t Namet Callt in_special_contextt FixZipc B s e Z d Z d Z d Z RS( s:
power< 'zip' args=trailer< '(' [any] ')' >
>
s future_builtins.zipc C sd | i | o d St | o d S| i } d | _ t t d | g } | i | _ | S( Nu u list( t should_skipR t Nonet clonet prefixR R ( t selft nodet resultst new( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_zip.pyt transform s
( t __name__t
__module__t PATTERNt skip_onR
( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_zip.pyR s N( t __doc__t R t
fixer_utilR R R t ConditionalFixR ( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_zip.pyt s fixes/fix_xrange.py 0000666 00000005163 15050104230 0010365 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that changes xrange(...) into range(...)."""
# Local imports
from .. import fixer_base
from ..fixer_util import Name, Call, consuming_calls
from .. import patcomp
class FixXrange(fixer_base.BaseFix):
PATTERN = """
power<
(name='range'|name='xrange') trailer< '(' args=any ')' >
rest=any* >
"""
def start_tree(self, tree, filename):
super(FixXrange, self).start_tree(tree, filename)
self.transformed_xranges = set()
def finish_tree(self, tree, filename):
self.transformed_xranges = None
def transform(self, node, results):
name = results["name"]
if name.value == u"xrange":
return self.transform_xrange(node, results)
elif name.value == u"range":
return self.transform_range(node, results)
else:
raise ValueError(repr(name))
def transform_xrange(self, node, results):
name = results["name"]
name.replace(Name(u"range", prefix=name.prefix))
# This prevents the new range call from being wrapped in a list later.
self.transformed_xranges.add(id(node))
def transform_range(self, node, results):
if (id(node) not in self.transformed_xranges and
not self.in_special_context(node)):
range_call = Call(Name(u"range"), [results["args"].clone()])
# Encase the range call in list().
list_call = Call(Name(u"list"), [range_call],
prefix=node.prefix)
# Put things that were after the range() call after the list call.
for n in results["rest"]:
list_call.append_child(n)
return list_call
P1 = "power< func=NAME trailer< '(' node=any ')' > any* >"
p1 = patcomp.compile_pattern(P1)
P2 = """for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
| comparison< any 'in' node=any any*>
"""
p2 = patcomp.compile_pattern(P2)
def in_special_context(self, node):
if node.parent is None:
return False
results = {}
if (node.parent.parent is not None and
self.p1.match(node.parent.parent, results) and
results["node"] is node):
# list(d.keys()) -> list(d.keys()), etc.
return results["func"].value in consuming_calls
# for ... in d.iterkeys() -> for ... in d.keys(), etc.
return self.p2.match(node.parent, results) and results["node"] is node
fixes/fix_reduce.pyc 0000666 00000002277 15050104230 0010516 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sq Fixer for reduce().
Makes sure reduce() is imported from the functools module if reduce is
used in that module.
i( t
fixer_base( t touch_importt FixReducec B s e Z d Z d Z RS( si
power< 'reduce'
trailer< '('
arglist< (
(not(argument) any ','
not(argument
>
c C s t d d | d S( Nu functoolsu reduce( R ( t selft nodet results( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_reduce.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_reduce.pyR s N( t __doc__t lib2to3R t lib2to3.fixer_utilR t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_reduce.pyt s fixes/fix_ne.pyo 0000666 00000001751 15050104230 0007661 0 ustar 00
Lc @ sS d Z d d k l Z d d k l Z d d k l Z d e i f d YZ d S( s Fixer that turns <> into !=.i ( t pytree( t token( t
fixer_baset FixNec B s# e Z e i Z d Z d Z RS( c C s
| i d j S( Nu <>( t value( t selft node( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyt match s c C s" t i t i d d | i } | S( Nu !=t prefix( R t LeafR t NOTEQUALR ( R R t resultst new( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyt transform s ( t __name__t
__module__R R
t _accept_typeR R
( ( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyR s N( t __doc__t R t pgen2R R t BaseFixR ( ( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyt s fixes/__init__.py 0000666 00000000057 15050104230 0007767 0 ustar 00 # Dummy file to make this directory a package.
fixes/fix_xreadlines.py 0000666 00000001231 15050104230 0011227 0 ustar 00 """Fix "for x in f.xreadlines()" -> "for x in f".
This fixer will also convert g(f.xreadlines) into g(f.__iter__)."""
# Author: Collin Winter
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixXreadlines(fixer_base.BaseFix):
PATTERN = """
power< call=any+ trailer< '.' 'xreadlines' > trailer< '(' ')' > >
|
power< any+ trailer< '.' no_call='xreadlines' > >
"""
def transform(self, node, results):
no_call = results.get("no_call")
if no_call:
no_call.replace(Name(u"__iter__", prefix=no_call.prefix))
else:
node.replace([x.clone() for x in results["call"]])
fixes/fix_next.pyc 0000666 00000007013 15050104230 0010216 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z l
Z
l Z d Z d e i
f d YZ d Z d
Z d Z d S(
s. Fixer for it.next() -> next(it), per PEP 3114.i ( t token( t python_symbols( t
fixer_base( t Namet Callt find_bindings; Calls to builtin next() possibly shadowed by global bindingt FixNextc B s& e Z d Z d Z d Z d Z RS( s
power< base=any+ trailer< '.' attr='next' > trailer< '(' ')' > >
|
power< head=any+ trailer< '.' attr='next' > not trailer< '(' ')' > >
|
classdef< 'class' any+ ':'
suite< any*
funcdef< 'def'
name='next'
parameters< '(' NAME ')' > any+ >
any* > >
|
global=global_stmt< 'global' any* 'next' any* >
t prec C sY t t | i | | t d | } | o | i | t t | _ n
t | _ d S( Nu next( t superR t
start_treeR t warningt bind_warningt Truet
shadowed_nextt False( t selft treet filenamet n( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR # s
c
C s | p t | i d } | i d } | i d } | o | i o | i t d d | i qg } | D] } | | i qw ~ } d | d _ | i t t d d | i | n | o& t d d | i } | i | n | o t | o` | d } d
i g } | D] } | t
| q%~ i d j o | i | t
n d S| i t d n+ d | j o | i | t
t | _ n d S(
Nt baset attrt nameu __next__t prefixu i u nextt headt u __builtin__t global( t AssertionErrort getR
t replaceR R t cloneR t is_assign_targett joint strt stripR
R R (
R t nodet resultsR R R t _[1]R R t _[2]( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyt transform- s.
'
)
=
( t __name__t
__module__t PATTERNt orderR R& ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR s
c C sc t | } | d j o t Sx> | i D]3 } | i t i j o t St | | o t Sq( Wt S( N( t find_assignt NoneR t childrent typeR t EQUALt
is_subtreeR ( R" t assignt child( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR P s
c C sM | i t i j o | S| i t i j p | i d j o d St | i S( N( R. t symst expr_stmtt simple_stmtt parentR, R+ ( R" ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR+ \ s
#c s/ | j o t St f d | i D S( Nc 3 s" x | ] } t | Vq Wd S( N( R0 ( t .0t c( R" ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pys f s ( R t anyR- ( t rootR" ( ( R" s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyR0 c s
N( t __doc__t pgen2R t pygramR R3 R R t
fixer_utilR R R R t BaseFixR R R+ R0 ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_next.pyt s ? fixes/fix_repr.pyc 0000666 00000001745 15050104230 0010216 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s/ Fixer that transforms `xyzzy` into repr(xyzzy).i ( t
fixer_base( t Callt Namet parenthesizet FixReprc B s e Z d Z d Z RS( s7
atom < '`' expr=any '`' >
c C sU | d i } | i | i i j o t | } n t t d | g d | i S( Nt expru reprt prefix( t clonet typet symst testlist1R R R R ( t selft nodet resultsR ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_repr.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_repr.pyR s N( t __doc__t R t
fixer_utilR R R t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_repr.pyt s fixes/fix_itertools.pyc 0000666 00000003245 15050104230 0011267 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sT Fixer for itertools.(imap|ifilter|izip) --> (map|filter|zip) and
itertools.ifilterfalse --> itertools.filterfalse (bugs 2360-2363)
imports from itertools are fixed in fix_itertools_import.py
If itertools is imported as something else (ie: import itertools as it;
it.izip(spam, eggs)) method calls will not get fixed.
i ( t
fixer_base( t Namet FixItertoolsc B s* e Z d Z d e Z d Z d Z RS( s( ('imap'|'ifilter'|'izip'|'ifilterfalse')s
power< it='itertools'
trailer<
dot='.' func=%(it_funcs)s > trailer< '(' [any] ')' > >
|
power< func=%(it_funcs)s trailer< '(' [any] ')' > >
i c C s d } | d d } d | j oV | i d j oF | d | d } } | i } | i | i | i i | n | p | i } | i t | i d d | d S( Nt funci t itu ifilterfalset doti t prefix( t Nonet valueR t removet parentt replaceR ( t selft nodet resultsR R R R ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_itertools.pyt transform s
( t __name__t
__module__t it_funcst localst PATTERNt run_orderR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_itertools.pyR s
N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_itertools.pyt s fixes/fix_imports.py 0000666 00000013011 15050104230 0010565 0 ustar 00 """Fix incompatible imports and module references."""
# Authors: Collin Winter, Nick Edds
# Local imports
from .. import fixer_base
from ..fixer_util import Name, attr_chain
MAPPING = {'StringIO': 'io',
'cStringIO': 'io',
'cPickle': 'pickle',
'__builtin__' : 'builtins',
'copy_reg': 'copyreg',
'Queue': 'queue',
'SocketServer': 'socketserver',
'ConfigParser': 'configparser',
'repr': 'reprlib',
'FileDialog': 'tkinter.filedialog',
'tkFileDialog': 'tkinter.filedialog',
'SimpleDialog': 'tkinter.simpledialog',
'tkSimpleDialog': 'tkinter.simpledialog',
'tkColorChooser': 'tkinter.colorchooser',
'tkCommonDialog': 'tkinter.commondialog',
'Dialog': 'tkinter.dialog',
'Tkdnd': 'tkinter.dnd',
'tkFont': 'tkinter.font',
'tkMessageBox': 'tkinter.messagebox',
'ScrolledText': 'tkinter.scrolledtext',
'Tkconstants': 'tkinter.constants',
'Tix': 'tkinter.tix',
'ttk': 'tkinter.ttk',
'Tkinter': 'tkinter',
'markupbase': '_markupbase',
'_winreg': 'winreg',
'thread': '_thread',
'dummy_thread': '_dummy_thread',
# anydbm and whichdb are handled by fix_imports2
'dbhash': 'dbm.bsd',
'dumbdbm': 'dbm.dumb',
'dbm': 'dbm.ndbm',
'gdbm': 'dbm.gnu',
'xmlrpclib': 'xmlrpc.client',
'DocXMLRPCServer': 'xmlrpc.server',
'SimpleXMLRPCServer': 'xmlrpc.server',
'httplib': 'http.client',
'htmlentitydefs' : 'html.entities',
'HTMLParser' : 'html.parser',
'Cookie': 'http.cookies',
'cookielib': 'http.cookiejar',
'BaseHTTPServer': 'http.server',
'SimpleHTTPServer': 'http.server',
'CGIHTTPServer': 'http.server',
#'test.test_support': 'test.support',
'commands': 'subprocess',
'UserString' : 'collections',
'UserList' : 'collections',
'urlparse' : 'urllib.parse',
'robotparser' : 'urllib.robotparser',
}
def alternates(members):
return "(" + "|".join(map(repr, members)) + ")"
def build_pattern(mapping=MAPPING):
mod_list = ' | '.join(["module_name='%s'" % key for key in mapping])
bare_names = alternates(mapping.keys())
yield """name_import=import_name< 'import' ((%s) |
multiple_imports=dotted_as_names< any* (%s) any* >) >
""" % (mod_list, mod_list)
yield """import_from< 'from' (%s) 'import' ['(']
( any | import_as_name< any 'as' any > |
import_as_names< any* >) [')'] >
""" % mod_list
yield """import_name< 'import' (dotted_as_name< (%s) 'as' any > |
multiple_imports=dotted_as_names<
any* dotted_as_name< (%s) 'as' any > any* >) >
""" % (mod_list, mod_list)
# Find usages of module members in code e.g. thread.foo(bar)
yield "power< bare_with_attr=(%s) trailer<'.' any > any* >" % bare_names
class FixImports(fixer_base.BaseFix):
# This is overridden in fix_imports2.
mapping = MAPPING
# We want to run this fixer late, so fix_import doesn't try to make stdlib
# renames into relative imports.
run_order = 6
def build_pattern(self):
return "|".join(build_pattern(self.mapping))
def compile_pattern(self):
# We override this, so MAPPING can be pragmatically altered and the
# changes will be reflected in PATTERN.
self.PATTERN = self.build_pattern()
super(FixImports, self).compile_pattern()
# Don't match the node if it's within another match.
def match(self, node):
match = super(FixImports, self).match
results = match(node)
if results:
# Module usage could be in the trailer of an attribute lookup, so we
# might have nested matches when "bare_with_attr" is present.
if "bare_with_attr" not in results and \
any(match(obj) for obj in attr_chain(node, "parent")):
return False
return results
return False
def start_tree(self, tree, filename):
super(FixImports, self).start_tree(tree, filename)
self.replace = {}
def transform(self, node, results):
import_mod = results.get("module_name")
if import_mod:
mod_name = import_mod.value
new_name = unicode(self.mapping[mod_name])
import_mod.replace(Name(new_name, prefix=import_mod.prefix))
if "name_import" in results:
# If it's not a "from x import x, y" or "import x as y" import,
# marked its usage to be replaced.
self.replace[mod_name] = new_name
if "multiple_imports" in results:
# This is a nasty hack to fix multiple imports on a line (e.g.,
# "import StringIO, urlparse"). The problem is that I can't
# figure out an easy way to make a pattern recognize the keys of
# MAPPING randomly sprinkled in an import statement.
results = self.match(node)
if results:
self.transform(node, results)
else:
# Replace usage of the module.
bare_name = results["bare_with_attr"][0]
new_name = self.replace.get(bare_name.value)
if new_name:
bare_name.replace(Name(new_name, prefix=bare_name.prefix))
fixes/fix_methodattrs.py 0000666 00000001116 15050104230 0011431 0 ustar 00 """Fix bound method attributes (method.im_? -> method.__?__).
"""
# Author: Christian Heimes
# Local imports
from .. import fixer_base
from ..fixer_util import Name
MAP = {
"im_func" : "__func__",
"im_self" : "__self__",
"im_class" : "__self__.__class__"
}
class FixMethodattrs(fixer_base.BaseFix):
PATTERN = """
power< any+ trailer< '.' attr=('im_func' | 'im_self' | 'im_class') > any* >
"""
def transform(self, node, results):
attr = results["attr"][0]
new = unicode(MAP[attr.value])
attr.replace(Name(new, prefix=attr.prefix))
fixes/fix_sys_exc.py 0000666 00000001766 15050104230 0010563 0 ustar 00 """Fixer for sys.exc_{type, value, traceback}
sys.exc_type -> sys.exc_info()[0]
sys.exc_value -> sys.exc_info()[1]
sys.exc_traceback -> sys.exc_info()[2]
"""
# By Jeff Balogh and Benjamin Peterson
# Local imports
from .. import fixer_base
from ..fixer_util import Attr, Call, Name, Number, Subscript, Node, syms
class FixSysExc(fixer_base.BaseFix):
# This order matches the ordering of sys.exc_info().
exc_info = [u"exc_type", u"exc_value", u"exc_traceback"]
PATTERN = """
power< 'sys' trailer< dot='.' attribute=(%s) > >
""" % '|'.join("'%s'" % e for e in exc_info)
def transform(self, node, results):
sys_attr = results["attribute"][0]
index = Number(self.exc_info.index(sys_attr.value))
call = Call(Name(u"exc_info"), prefix=sys_attr.prefix)
attr = Attr(Name(u"sys"), call)
attr[1].children[0].prefix = results["dot"].prefix
attr.append(Subscript(index))
return Node(syms.power, attr, prefix=node.prefix)
fixes/fix_except.pyo 0000666 00000005750 15050104230 0010552 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z l Z d Z
d e i f d YZ d S(
s Fixer for except statements with named exceptions.
The following cases will be converted:
- "except E, T:" where T is a name:
except E as T:
- "except E, T:" where T is not a name, tuple or list:
except E as t:
T = t
This is done because the target of an "except" clause must be a
name.
- "except E, T:" where T is a tuple or list literal:
except E as t:
T = t.args
i ( t pytree( t token( t
fixer_base( t Assignt Attrt Namet is_tuplet is_listt symsc c sf x_ t | D]Q \ } } | i t i j o2 | i d i d j o | | | d f Vq^ q
q
Wd S( Ni u excepti ( t enumeratet typeR t
except_clauset childrent value( t nodest it n( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyt find_excepts s
t FixExceptc B s e Z d Z d Z RS( s1
try_stmt< 'try' ':' (simple_stmt | suite)
cleanup=(except_clause ':' (simple_stmt | suite))+
tail=(['except' ':' (simple_stmt | suite)]
['else' ':' (simple_stmt | suite)]
['finally' ':' (simple_stmt | suite)]) >
c C sO | i } g } | d D] } | | i q ~ } g } | d D] } | | i qC ~ } xt | D]\ }
} t |
i d j ot|
i d d !\ } }
} |
i t d d d | i t i j ot | i
d d } | i } d | _ | i | | i } | i } x2 t | D]$ \ } } t
| t i o Pq/q/Wt | p
t | o" t | t | t d } n t | | } x( t | | D] } | i d
| qW| i | | q| i d j o
d | _ qql ql Wg } | i d D] } | | i q~ | | } t i | i | S( Nt tailt cleanupi i u ast prefixu u u argsi i ( R t cloneR t lenR t replaceR R
R t NAMEt new_nameR R t
isinstanceR t NodeR R R R t reversedt insert_child( t selft nodet resultsR t _[1]R R t _[2]t cht try_cleanupR t e_suitet Et commat Nt new_Nt targett suite_stmtsR t stmtt assignt childt _[3]t cR ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyt transform. s< ++
" 6( t __name__t
__module__t PATTERNR2 ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyR $ s N( t __doc__t R t pgen2R R t
fixer_utilR R R R R R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_except.pyt s . fixes/fix_except.py 0000666 00000006377 15050104230 0010401 0 ustar 00 """Fixer for except statements with named exceptions.
The following cases will be converted:
- "except E, T:" where T is a name:
except E as T:
- "except E, T:" where T is not a name, tuple or list:
except E as t:
T = t
This is done because the target of an "except" clause must be a
name.
- "except E, T:" where T is a tuple or list literal:
except E as t:
T = t.args
"""
# Author: Collin Winter
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Assign, Attr, Name, is_tuple, is_list, syms
def find_excepts(nodes):
for i, n in enumerate(nodes):
if n.type == syms.except_clause:
if n.children[0].value == u'except':
yield (n, nodes[i+2])
class FixExcept(fixer_base.BaseFix):
PATTERN = """
try_stmt< 'try' ':' (simple_stmt | suite)
cleanup=(except_clause ':' (simple_stmt | suite))+
tail=(['except' ':' (simple_stmt | suite)]
['else' ':' (simple_stmt | suite)]
['finally' ':' (simple_stmt | suite)]) >
"""
def transform(self, node, results):
syms = self.syms
tail = [n.clone() for n in results["tail"]]
try_cleanup = [ch.clone() for ch in results["cleanup"]]
for except_clause, e_suite in find_excepts(try_cleanup):
if len(except_clause.children) == 4:
(E, comma, N) = except_clause.children[1:4]
comma.replace(Name(u"as", prefix=u" "))
if N.type != token.NAME:
# Generate a new N for the except clause
new_N = Name(self.new_name(), prefix=u" ")
target = N.clone()
target.prefix = u""
N.replace(new_N)
new_N = new_N.clone()
# Insert "old_N = new_N" as the first statement in
# the except body. This loop skips leading whitespace
# and indents
#TODO(cwinter) suite-cleanup
suite_stmts = e_suite.children
for i, stmt in enumerate(suite_stmts):
if isinstance(stmt, pytree.Node):
break
# The assignment is different if old_N is a tuple or list
# In that case, the assignment is old_N = new_N.args
if is_tuple(N) or is_list(N):
assign = Assign(target, Attr(new_N, Name(u'args')))
else:
assign = Assign(target, new_N)
#TODO(cwinter) stopgap until children becomes a smart list
for child in reversed(suite_stmts[:i]):
e_suite.insert_child(0, child)
e_suite.insert_child(i, assign)
elif N.prefix == u"":
# No space after a comma is legal; no space after "as",
# not so much.
N.prefix = u" "
#TODO(cwinter) fix this when children becomes a smart list
children = [c.clone() for c in node.children[:3]] + try_cleanup + tail
return pytree.Node(node.type, children)
fixes/fix_ne.pyc 0000666 00000001751 15050104230 0007645 0 ustar 00
Lc @ sS d Z d d k l Z d d k l Z d d k l Z d e i f d YZ d S( s Fixer that turns <> into !=.i ( t pytree( t token( t
fixer_baset FixNec B s# e Z e i Z d Z d Z RS( c C s
| i d j S( Nu <>( t value( t selft node( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyt match s c C s" t i t i d d | i } | S( Nu !=t prefix( R t LeafR t NOTEQUALR ( R R t resultst new( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyt transform s ( t __name__t
__module__R R
t _accept_typeR R
( ( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyR s N( t __doc__t R t pgen2R R t BaseFixR ( ( ( s, /usr/lib64/python2.6/lib2to3/fixes/fix_ne.pyt s fixes/fix_idioms.py 0000666 00000011432 15050104230 0010361 0 ustar 00 """Adjust some old Python 2 idioms to their modern counterparts.
* Change some type comparisons to isinstance() calls:
type(x) == T -> isinstance(x, T)
type(x) is T -> isinstance(x, T)
type(x) != T -> not isinstance(x, T)
type(x) is not T -> not isinstance(x, T)
* Change "while 1:" into "while True:".
* Change both
v = list(EXPR)
v.sort()
foo(v)
and the more general
v = EXPR
v.sort()
foo(v)
into
v = sorted(EXPR)
foo(v)
"""
# Author: Jacques Frechet, Collin Winter
# Local imports
from .. import fixer_base
from ..fixer_util import Call, Comma, Name, Node, BlankLine, syms
CMP = "(n='!=' | '==' | 'is' | n=comp_op< 'is' 'not' >)"
TYPE = "power< 'type' trailer< '(' x=any ')' > >"
class FixIdioms(fixer_base.BaseFix):
explicit = True # The user must ask for this fixer
PATTERN = r"""
isinstance=comparison< %s %s T=any >
|
isinstance=comparison< T=any %s %s >
|
while_stmt< 'while' while='1' ':' any+ >
|
sorted=any<
any*
simple_stmt<
expr_stmt< id1=any '='
power< list='list' trailer< '(' (not arglist) any ')' > >
>
'\n'
>
sort=
simple_stmt<
power< id2=any
trailer< '.' 'sort' > trailer< '(' ')' >
>
'\n'
>
next=any*
>
|
sorted=any<
any*
simple_stmt< expr_stmt< id1=any '=' expr=any > '\n' >
sort=
simple_stmt<
power< id2=any
trailer< '.' 'sort' > trailer< '(' ')' >
>
'\n'
>
next=any*
>
""" % (TYPE, CMP, CMP, TYPE)
def match(self, node):
r = super(FixIdioms, self).match(node)
# If we've matched one of the sort/sorted subpatterns above, we
# want to reject matches where the initial assignment and the
# subsequent .sort() call involve different identifiers.
if r and "sorted" in r:
if r["id1"] == r["id2"]:
return r
return None
return r
def transform(self, node, results):
if "isinstance" in results:
return self.transform_isinstance(node, results)
elif "while" in results:
return self.transform_while(node, results)
elif "sorted" in results:
return self.transform_sort(node, results)
else:
raise RuntimeError("Invalid match")
def transform_isinstance(self, node, results):
x = results["x"].clone() # The thing inside of type()
T = results["T"].clone() # The type being compared against
x.prefix = u""
T.prefix = u" "
test = Call(Name(u"isinstance"), [x, Comma(), T])
if "n" in results:
test.prefix = u" "
test = Node(syms.not_test, [Name(u"not"), test])
test.prefix = node.prefix
return test
def transform_while(self, node, results):
one = results["while"]
one.replace(Name(u"True", prefix=one.prefix))
def transform_sort(self, node, results):
sort_stmt = results["sort"]
next_stmt = results["next"]
list_call = results.get("list")
simple_expr = results.get("expr")
if list_call:
list_call.replace(Name(u"sorted", prefix=list_call.prefix))
elif simple_expr:
new = simple_expr.clone()
new.prefix = u""
simple_expr.replace(Call(Name(u"sorted"), [new],
prefix=simple_expr.prefix))
else:
raise RuntimeError("should not have reached here")
sort_stmt.remove()
btwn = sort_stmt.prefix
# Keep any prefix lines between the sort_stmt and the list_call and
# shove them right after the sorted() call.
if u"\n" in btwn:
if next_stmt:
# The new prefix should be everything from the sort_stmt's
# prefix up to the last newline, then the old prefix after a new
# line.
prefix_lines = (btwn.rpartition(u"\n")[0], next_stmt[0].prefix)
next_stmt[0].prefix = u"\n".join(prefix_lines)
else:
assert list_call.parent
assert list_call.next_sibling is None
# Put a blank line after list_call and set its prefix.
end_line = BlankLine()
list_call.parent.append_child(end_line)
assert list_call.next_sibling is end_line
# The new prefix should be everything up to the first new line
# of sort_stmt's prefix.
end_line.prefix = btwn.rpartition(u"\n")[0]
fixes/fix_itertools_imports.pyc 0000666 00000003506 15050104230 0013044 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( sA Fixer for imports of itertools.(imap|ifilter|izip|ifilterfalse) i( t
fixer_base( t BlankLinet symst tokent FixItertoolsImportsc B s e Z d e Z d Z RS( sT
import_from< 'from' 'itertools' 'import' imports=any >
c C s | d } | i t i j p | i o
| g } n
| i } x | d d d D] } | i t i j o | i } | } n( | i t i j p t | i d } | i } | d j o d | _ | i qR | d j o | i
d | _ qR qR W| i p | g } t } x@ | D]8 } | o! | i t i j o | i q| t N} qW| d i t i j o | d i n | i p t
| d
d p | i d j o | i }
t } |
| _ | Sd S( Nt importsi i u imapu izipu ifilteru ifilterfalseu filterfalseit value( u imapu izipu ifilter( t typeR t import_as_namet childrenR t NAMER t AssertionErrort Nonet removet changedt Truet COMMAt getattrt parentt prefixR ( t selft nodet resultsR R t childt membert name_nodet member_namet remove_commat p( ( s; /usr/lib64/python2.6/lib2to3/fixes/fix_itertools_imports.pyt transform
sB
( t __name__t
__module__t localst PATTERNR ( ( ( s; /usr/lib64/python2.6/lib2to3/fixes/fix_itertools_imports.pyR s
N( t __doc__t lib2to3R t lib2to3.fixer_utilR R R t BaseFixR ( ( ( s; /usr/lib64/python2.6/lib2to3/fixes/fix_itertools_imports.pyt s fixes/fix_has_key.pyc 0000666 00000006216 15050104230 0010667 0 ustar 00
Lc @ si d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z d e i f d YZ
d S( s& Fixer for has_key().
Calls to .has_key() methods are expressed in terms of the 'in'
operator:
d.has_key(k) -> k in d
CAVEATS:
1) While the primary target of this fixer is dict.has_key(), the
fixer will change any has_key() method call, regardless of its
class.
2) Cases like this will not be converted:
m = d.has_key
if m(k):
...
Only *calls* to has_key() are converted. While it is possible to
convert the above to something like
m = d.__contains__
if m(k):
...
this is currently not done.
i ( t pytree( t token( t
fixer_base( t Namet parenthesizet FixHasKeyc B s e Z d Z d Z RS( s
anchor=power<
before=any+
trailer< '.' 'has_key' >
trailer<
'('
( not(arglist | argument) arg=any ','>
)
')'
>
after=any*
>
|
negation=not_test<
'not'
anchor=power<
before=any+
trailer< '.' 'has_key' >
trailer<
'('
( not(arglist | argument) arg=any ','>
)
')'
>
>
>
c C sv | p t | i } | i i | i j o | i i | i o d S| i d } | d } | i } g } | d D] } | | i
qy ~ } | d i
}
| i d } | o+ g } | D] } | | i
q ~ } n |
i | i | i | i | i
| i | i | i f j o t |
}
n t | d j o | d } n t i | i | } d | _ t d d
d }
| o1 t d d
d } t i | i | |
f }
n t i | i |
|
| f } | o2 t | } t i | i | f t | } n | i i | i | i | i | i | i | i | i | i | i f j o t | } n | | _ | S( Nt negationt anchort beforet argt afteri i u u int prefixu not( t AssertionErrort symst parentt typet not_testt patternt matcht Nonet getR t clonet
comparisont and_testt or_testt testt lambdeft argumentR t lenR t Nodet powerR t comp_opt tuplet exprt xor_exprt and_exprt
shift_exprt
arith_exprt termt factor( t selft nodet resultsR
R R R t _[1]t nR R R
t _[2]t n_opt n_nott new( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_has_key.pyt transformG sF
++" & ( t __name__t
__module__t PATTERNR1 ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_has_key.pyR ' s N( t __doc__t R t pgen2R R t
fixer_utilR R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_has_key.pyt s
fixes/fix_isinstance.py 0000666 00000003061 15050104230 0011234 0 ustar 00 # Copyright 2008 Armin Ronacher.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that cleans up a tuple argument to isinstance after the tokens
in it were fixed. This is mainly used to remove double occurrences of
tokens as a leftover of the long -> int / unicode -> str conversion.
eg. isinstance(x, (int, long)) -> isinstance(x, (int, int))
-> isinstance(x, int)
"""
from .. import fixer_base
from ..fixer_util import token
class FixIsinstance(fixer_base.BaseFix):
PATTERN = """
power<
'isinstance'
trailer< '(' arglist< any ',' atom< '('
args=testlist_gexp< any+ >
')' > > ')' >
>
"""
run_order = 6
def transform(self, node, results):
names_inserted = set()
testlist = results["args"]
args = testlist.children
new_args = []
iterator = enumerate(args)
for idx, arg in iterator:
if arg.type == token.NAME and arg.value in names_inserted:
if idx < len(args) - 1 and args[idx + 1].type == token.COMMA:
iterator.next()
continue
else:
new_args.append(arg)
if arg.type == token.NAME:
names_inserted.add(arg.value)
if new_args and new_args[-1].type == token.COMMA:
del new_args[-1]
if len(new_args) == 1:
atom = testlist.parent
new_args[0].prefix = atom.prefix
atom.replace(new_args[0])
else:
args[:] = new_args
node.changed()
fixes/fix_has_key.pyo 0000666 00000006153 15050104230 0010703 0 ustar 00
Lc @ si d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z d e i f d YZ
d S( s& Fixer for has_key().
Calls to .has_key() methods are expressed in terms of the 'in'
operator:
d.has_key(k) -> k in d
CAVEATS:
1) While the primary target of this fixer is dict.has_key(), the
fixer will change any has_key() method call, regardless of its
class.
2) Cases like this will not be converted:
m = d.has_key
if m(k):
...
Only *calls* to has_key() are converted. While it is possible to
convert the above to something like
m = d.__contains__
if m(k):
...
this is currently not done.
i ( t pytree( t token( t
fixer_base( t Namet parenthesizet FixHasKeyc B s e Z d Z d Z RS( s
anchor=power<
before=any+
trailer< '.' 'has_key' >
trailer<
'('
( not(arglist | argument) arg=any ','>
)
')'
>
after=any*
>
|
negation=not_test<
'not'
anchor=power<
before=any+
trailer< '.' 'has_key' >
trailer<
'('
( not(arglist | argument) arg=any ','>
)
')'
>
>
>
c C sh | i } | i i | i j o | i i | i o d S| i d } | d } | i } g } | d D] } | | i qk ~ } | d i }
| i d } | o+ g } | D] } | | i q ~ } n |
i | i
| i | i | i | i
| i | i f j o t |
}
n t | d j o | d } n t i | i | } d | _ t d d
d }
| o1 t d d
d } t i | i | |
f }
n t i | i
|
|
| f } | o2 t | } t i | i | f t | } n | i i | i
| i | i | i | i | i | i | i | i f j o t | } n | | _ | S( Nt negationt anchort beforet argt afteri i u u int prefixu not( t symst parentt typet not_testt patternt matcht Nonet getR t clonet
comparisont and_testt or_testt testt lambdeft argumentR t lenR t Nodet powerR t comp_opt tuplet exprt xor_exprt and_exprt
shift_exprt
arith_exprt termt factor( t selft nodet resultsR R R R t _[1]t nR R R
t _[2]t n_opt n_nott new( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_has_key.pyt transformG sD
++" & ( t __name__t
__module__t PATTERNR0 ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_has_key.pyR ' s N( t __doc__t R t pgen2R R t
fixer_utilR R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_has_key.pyt s
fixes/fix_long.pyo 0000666 00000001466 15050104230 0010221 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s/ Fixer that turns 'long' into 'int' everywhere.
i( t
fixer_base( t is_probably_builtint FixLongc B s e Z d Z d Z RS( s 'long'c C s( t | o d | _ | i n d S( Nu int( R t valuet changed( t selft nodet results( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_long.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_long.pyR s N( t __doc__t lib2to3R t lib2to3.fixer_utilR t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_long.pyt s fixes/fix_input.pyc 0000666 00000002133 15050104230 0010375 0 ustar 00
Lc @ sh d Z d d k l Z d d k l Z l Z d d k l Z e i d Z d e i f d YZ
d S( s4 Fixer that changes input(...) into eval(input(...)).i ( t
fixer_base( t Callt Name( t patcomps& power< 'eval' trailer< '(' any ')' > >t FixInputc B s e Z d Z d Z RS( sL
power< 'input' args=trailer< '(' [any] ')' > >
c C sO t i | i i o d S| i } d | _ t t d | g d | i S( Nu u evalt prefix( t contextt matcht parentt cloneR R R ( t selft nodet resultst new( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_input.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_input.pyR
s N( t __doc__t R t
fixer_utilR R R t compile_patternR t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_input.pyt s
fixes/fix_tuple_params.py 0000666 00000012520 15050104230 0011570 0 ustar 00 """Fixer for function definitions with tuple parameters.
def func(((a, b), c), d):
...
->
def func(x, d):
((a, b), c) = x
...
It will also support lambdas:
lambda (x, y): x + y -> lambda t: t[0] + t[1]
# The parens are a syntax error in Python 3
lambda (x): x + y -> lambda x: x + y
"""
# Author: Collin Winter
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Assign, Name, Newline, Number, Subscript, syms
def is_docstring(stmt):
return isinstance(stmt, pytree.Node) and \
stmt.children[0].type == token.STRING
class FixTupleParams(fixer_base.BaseFix):
PATTERN = """
funcdef< 'def' any parameters< '(' args=any ')' >
['->' any] ':' suite=any+ >
|
lambda=
lambdef< 'lambda' args=vfpdef< '(' inner=any ')' >
':' body=any
>
"""
def transform(self, node, results):
if "lambda" in results:
return self.transform_lambda(node, results)
new_lines = []
suite = results["suite"]
args = results["args"]
# This crap is so "def foo(...): x = 5; y = 7" is handled correctly.
# TODO(cwinter): suite-cleanup
if suite[0].children[1].type == token.INDENT:
start = 2
indent = suite[0].children[1].value
end = Newline()
else:
start = 0
indent = u"; "
end = pytree.Leaf(token.INDENT, u"")
# We need access to self for new_name(), and making this a method
# doesn't feel right. Closing over self and new_lines makes the
# code below cleaner.
def handle_tuple(tuple_arg, add_prefix=False):
n = Name(self.new_name())
arg = tuple_arg.clone()
arg.prefix = u""
stmt = Assign(arg, n.clone())
if add_prefix:
n.prefix = u" "
tuple_arg.replace(n)
new_lines.append(pytree.Node(syms.simple_stmt,
[stmt, end.clone()]))
if args.type == syms.tfpdef:
handle_tuple(args)
elif args.type == syms.typedargslist:
for i, arg in enumerate(args.children):
if arg.type == syms.tfpdef:
# Without add_prefix, the emitted code is correct,
# just ugly.
handle_tuple(arg, add_prefix=(i > 0))
if not new_lines:
return
# This isn't strictly necessary, but it plays nicely with other fixers.
# TODO(cwinter) get rid of this when children becomes a smart list
for line in new_lines:
line.parent = suite[0]
# TODO(cwinter) suite-cleanup
after = start
if start == 0:
new_lines[0].prefix = u" "
elif is_docstring(suite[0].children[start]):
new_lines[0].prefix = indent
after = start + 1
for line in new_lines:
line.parent = suite[0]
suite[0].children[after:after] = new_lines
for i in range(after+1, after+len(new_lines)+1):
suite[0].children[i].prefix = indent
suite[0].changed()
def transform_lambda(self, node, results):
args = results["args"]
body = results["body"]
inner = simplify_args(results["inner"])
# Replace lambda ((((x)))): x with lambda x: x
if inner.type == token.NAME:
inner = inner.clone()
inner.prefix = u" "
args.replace(inner)
return
params = find_params(args)
to_index = map_to_index(params)
tup_name = self.new_name(tuple_name(params))
new_param = Name(tup_name, prefix=u" ")
args.replace(new_param.clone())
for n in body.post_order():
if n.type == token.NAME and n.value in to_index:
subscripts = [c.clone() for c in to_index[n.value]]
new = pytree.Node(syms.power,
[new_param.clone()] + subscripts)
new.prefix = n.prefix
n.replace(new)
### Helper functions for transform_lambda()
def simplify_args(node):
if node.type in (syms.vfplist, token.NAME):
return node
elif node.type == syms.vfpdef:
# These look like vfpdef< '(' x ')' > where x is NAME
# or another vfpdef instance (leading to recursion).
while node.type == syms.vfpdef:
node = node.children[1]
return node
raise RuntimeError("Received unexpected node %s" % node)
def find_params(node):
if node.type == syms.vfpdef:
return find_params(node.children[1])
elif node.type == token.NAME:
return node.value
return [find_params(c) for c in node.children if c.type != token.COMMA]
def map_to_index(param_list, prefix=[], d=None):
if d is None:
d = {}
for i, obj in enumerate(param_list):
trailer = [Subscript(Number(unicode(i)))]
if isinstance(obj, list):
map_to_index(obj, trailer, d=d)
else:
d[obj] = prefix + trailer
return d
def tuple_name(param_list):
l = []
for obj in param_list:
if isinstance(obj, list):
l.append(tuple_name(obj))
else:
l.append(obj)
return u"_".join(l)
fixes/fix_metaclass.pyo 0000666 00000014721 15050104230 0011234 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z l Z l Z l Z d Z
d Z d Z d Z
d Z d
Z d e i f d YZ d
S( s Fixer for __metaclass__ = X -> (metaclass=X) methods.
The various forms of classef (inherits nothing, inherits once, inherints
many) don't parse the same in the CST so we look at ALL classes for
a __metaclass__ and if we find one normalize the inherits to all be
an arglist.
For one-liner classes ('class X: pass') there is no indent/dedent so
we normalize those into having a suite.
Moving the __metaclass__ into the classdef can also cause the class
body to be empty so there is some special casing for that as well.
This fixer also tries very hard to keep original indenting and spacing
in all those corner cases.
i ( t
fixer_base( t token( t Namet symst Nodet Leafc C s x | i D] } | i t i j o t | S| i t i j on | i od | i d } | i t i j o@ | i o6 | i d } t | t o | i d j o t
Sq q
q
Wt S( s we have to check the cls_node without changing it.
There are two possiblities:
1) clsdef => suite => simple_stmt => expr_stmt => Leaf('__meta')
2) clsdef => simple_stmt => expr_stmt => Leaf('__meta')
i t
__metaclass__( t childrent typeR t suitet
has_metaclasst simple_stmtt expr_stmtt
isinstanceR t valuet Truet False( t parentt nodet expr_nodet left_side( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyR
s
c C s x) | i D] } | i t i j o d Sq
WxA t | i D]$ \ } } | i t i j o Pq< q< Wt d t t i g } xE | i | d o2 | i | d } | i | i
| i q W| i | | } d S( sf one-line classes don't get a suite in the parse tree so we add
one to normalize the tree
Ns No class suite and no ':'!i ( R R R R t enumerateR t COLONt
ValueErrorR t append_childt clonet remove( t cls_nodeR t iR t move_node( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt fixup_parse_tree- s"
c
C s x9 t | i D]$ \ } } | i t i j o Pq q Wd S| i t t i g } t t i | g } x= | i | o. | i | } | i
| i | i qp W| i | | | i d i d } | i d i d } | i
| _
d S( s if there is a semi-colon all the parts count as part of the same
simple_stmt. We just want the __metaclass__ part so we move
everything efter the semi-colon into its own simple_stmt node
Ni ( R R R R t SEMIR R R R R R R t insert_childt prefix(
R R t stmt_nodet semi_indR t new_exprt new_stmtR t new_leaf1t old_leaf1( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt fixup_simple_stmtG s"
c C s= | i o/ | i d i t i j o | i d i n d S( Ni( R R R t NEWLINER ( R ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt remove_trailing_newline_ s $c c s
x5 | i D] } | i t i j o Pq
q
Wt d x t t | i D] \ } } | i t i j o | i o | i d } | i t i j og | i o] | i d } t | t
o< | i d j o, t | | | t
| | | | f Vq qqN qN Wd S( Ns No class suite!i u
__metaclass__( R R R R R t listR R R R
R R R( R* ( R R R t simple_nodeR t left_node( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt
find_metasd s
c C s | i d d d } x0 | o( | i } | i t i j o Pq q Wxt | ol | i } t | t o/ | i t i j o | i o
d | _ n d S| i | i d d d qL Wd S( s If an INDENT is followed by a thing with a prefix then nuke the prefix
Otherwise we get in trouble when removing __metaclass__ at suite start
Niu (
R t popR R t INDENTR
R t DEDENTR! t extend( R t kidsR ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt fixup_indent{ s
#
t FixMetaclassc B s e Z d Z d Z RS( s
classdef
c C s! t | p d St | d } x- t | D] \ } } } | } | i q/ W| i d i } t | i d j og | i d i t i j o | i d } q| i d i
} t t i | g } | i d | n t | i d j o& t t i g } | i
d | n t | i d j o^ t t i g } | i
d t t i d | i
d | | i
d t t i d n
t d | i d i d }
d
|
_ |
i } | i o& | i t t i d d |
_ n
d
|
_ | i d } d
| i d _ d
| i d _ | i | t | | i pL | i t | d }
| |
_ | i |
| i t t i d n t | i d j os | i d i t i j oY | i d i t i j o? t | d }
| i
d |
| i
d t t i d n d S( Ni i i i i i u )u (s Unexpected class definitiont metaclassu ,u u i u passu
ii( R
R t NoneR. R R R t lenR t arglistR R t set_childR R R t RPARt LPARR R R! R t COMMAR4 R) R0 R1 ( t selfR t resultst last_metaclassR R t stmtt text_typeR9 R t meta_txtt orig_meta_prefixR t pass_leaf( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt transform s`
( t __name__t
__module__t PATTERNRF ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyR5 s N( t __doc__t R t pygramR t
fixer_utilR R R R R
R R( R* R. R4 t BaseFixR5 ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_metaclass.pyt s " fixes/fix_ws_comma.py 0000666 00000002107 15050104230 0010701 0 ustar 00 """Fixer that changes 'a ,b' into 'a, b'.
This also changes '{a :b}' into '{a: b}', but does not touch other
uses of colons. It does not touch other uses of whitespace.
"""
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
class FixWsComma(fixer_base.BaseFix):
explicit = True # The user must ask for this fixers
PATTERN = """
any<(not(',') any)+ ',' ((not(',') any)+ ',')* [not(',') any]>
"""
COMMA = pytree.Leaf(token.COMMA, u",")
COLON = pytree.Leaf(token.COLON, u":")
SEPS = (COMMA, COLON)
def transform(self, node, results):
new = node.clone()
comma = False
for child in new.children:
if child in self.SEPS:
prefix = child.prefix
if prefix.isspace() and u"\n" not in prefix:
child.prefix = u""
comma = True
else:
if comma:
prefix = child.prefix
if not prefix:
child.prefix = u" "
comma = False
return new
fixes/fix_getcwdu.py 0000666 00000000653 15050104230 0010542 0 ustar 00 """
Fixer that changes os.getcwdu() to os.getcwd().
"""
# Author: Victor Stinner
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixGetcwdu(fixer_base.BaseFix):
PATTERN = """
power< 'os' trailer< dot='.' name='getcwdu' > any* >
"""
def transform(self, node, results):
name = results["name"]
name.replace(Name(u"getcwd", prefix=name.prefix))
fixes/fix_filter.pyc 0000666 00000004303 15050104230 0010524 0 ustar 00
Lc @ se d Z d d k l Z d d k l Z d d k l Z l Z l Z l Z d e i
f d YZ d S( s Fixer that changes filter(F, X) into list(filter(F, X)).
We avoid the transformation if the filter() call is directly contained
in iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or
for V in <>:.
NOTE: This is still not correct if the original code was depending on
filter(F, X) to return a string if X is a string and a tuple if X is a
tuple. That would require type inference, which we don't do. Let
Python 2.6 figure it out.
i ( t token( t
fixer_base( t Namet Callt ListCompt in_special_contextt FixFilterc B s e Z d Z d Z d Z RS( s
filter_lambda=power<
'filter'
trailer<
'('
arglist<
lambdef< 'lambda'
(fp=NAME | vfpdef< '(' fp=NAME ')'> ) ':' xp=any
>
','
it=any
>
')'
>
>
|
power<
'filter'
trailer< '(' arglist< none='None' ',' seq=any > ')' >
>
|
power<
'filter'
args=trailer< '(' [any] ')' >
>
s future_builtins.filterc C s | i | o d Sd | j oU t | i d i | i d i | i d i | i d i } n d | j o5 t t d t d | d i t d } n@ t | o d S| i } d | _ t t d | g } | i | _ | S(
Nt
filter_lambdat fpt itt xpt noneu _ft sequ u list( t should_skipR t gett cloneR R t Nonet prefixR ( t selft nodet resultst new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_filter.pyt transform4 s&
( t __name__t
__module__t PATTERNt skip_onR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_filter.pyR s N( t __doc__t pgen2R t R t
fixer_utilR R R R t ConditionalFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_filter.pyt s " fixes/fix_apply.py 0000666 00000003527 15050104230 0010230 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for apply().
This converts apply(func, v, k) into (func)(*v, **k)."""
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Call, Comma, parenthesize
class FixApply(fixer_base.BaseFix):
PATTERN = """
power< 'apply'
trailer<
'('
arglist<
(not argument
')'
>
>
"""
def transform(self, node, results):
syms = self.syms
assert results
func = results["func"]
args = results["args"]
kwds = results.get("kwds")
prefix = node.prefix
func = func.clone()
if (func.type not in (token.NAME, syms.atom) and
(func.type != syms.power or
func.children[-2].type == token.DOUBLESTAR)):
# Need to parenthesize
func = parenthesize(func)
func.prefix = ""
args = args.clone()
args.prefix = ""
if kwds is not None:
kwds = kwds.clone()
kwds.prefix = ""
l_newargs = [pytree.Leaf(token.STAR, u"*"), args]
if kwds is not None:
l_newargs.extend([Comma(),
pytree.Leaf(token.DOUBLESTAR, u"**"),
kwds])
l_newargs[-2].prefix = u" " # that's the ** token
# XXX Sometimes we could be cleverer, e.g. apply(f, (x, y) + t)
# can be translated into f(x, y, *t) instead of f(*(x, y) + t)
#new = pytree.Node(syms.power, (func, ArgList(l_newargs)))
return Call(func, l_newargs, prefix=prefix)
fixes/fix_zip.py 0000666 00000001557 15050104230 0007706 0 ustar 00 """
Fixer that changes zip(seq0, seq1, ...) into list(zip(seq0, seq1, ...)
unless there exists a 'from future_builtins import zip' statement in the
top-level namespace.
We avoid the transformation if the zip() call is directly contained in
iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or for V in <>:.
"""
# Local imports
from .. import fixer_base
from ..fixer_util import Name, Call, in_special_context
class FixZip(fixer_base.ConditionalFix):
PATTERN = """
power< 'zip' args=trailer< '(' [any] ')' >
>
"""
skip_on = "future_builtins.zip"
def transform(self, node, results):
if self.should_skip(node):
return
if in_special_context(node):
return None
new = node.clone()
new.prefix = u""
new = Call(Name(u"list"), [new])
new.prefix = node.prefix
return new
fixes/fix_input.pyo 0000666 00000002133 15050104230 0010411 0 ustar 00
Lc @ sh d Z d d k l Z d d k l Z l Z d d k l Z e i d Z d e i f d YZ
d S( s4 Fixer that changes input(...) into eval(input(...)).i ( t
fixer_base( t Callt Name( t patcomps& power< 'eval' trailer< '(' any ')' > >t FixInputc B s e Z d Z d Z RS( sL
power< 'input' args=trailer< '(' [any] ')' > >
c C sO t i | i i o d S| i } d | _ t t d | g d | i S( Nu u evalt prefix( t contextt matcht parentt cloneR R R ( t selft nodet resultst new( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_input.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_input.pyR
s N( t __doc__t R t
fixer_utilR R R t compile_patternR t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_input.pyt s
fixes/fix_xreadlines.pyo 0000666 00000002176 15050104230 0011417 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sp Fix "for x in f.xreadlines()" -> "for x in f".
This fixer will also convert g(f.xreadlines) into g(f.__iter__).i ( t
fixer_base( t Namet
FixXreadlinesc B s e Z d Z d Z RS( s
power< call=any+ trailer< '.' 'xreadlines' > trailer< '(' ')' > >
|
power< any+ trailer< '.' no_call='xreadlines' > >
c C sl | i d } | o | i t d d | i n3 | i g } | d D] } | | i qK ~ d S( Nt no_callu __iter__t prefixt call( t gett replaceR R t clone( t selft nodet resultsR t _[1]t x( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_xreadlines.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_xreadlines.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_xreadlines.pyt s fixes/fix_next.py 0000666 00000006124 15050104230 0010055 0 ustar 00 """Fixer for it.next() -> next(it), per PEP 3114."""
# Author: Collin Winter
# Things that currently aren't covered:
# - listcomp "next" names aren't warned
# - "with" statement targets aren't checked
# Local imports
from ..pgen2 import token
from ..pygram import python_symbols as syms
from .. import fixer_base
from ..fixer_util import Name, Call, find_binding
bind_warning = "Calls to builtin next() possibly shadowed by global binding"
class FixNext(fixer_base.BaseFix):
PATTERN = """
power< base=any+ trailer< '.' attr='next' > trailer< '(' ')' > >
|
power< head=any+ trailer< '.' attr='next' > not trailer< '(' ')' > >
|
classdef< 'class' any+ ':'
suite< any*
funcdef< 'def'
name='next'
parameters< '(' NAME ')' > any+ >
any* > >
|
global=global_stmt< 'global' any* 'next' any* >
"""
order = "pre" # Pre-order tree traversal
def start_tree(self, tree, filename):
super(FixNext, self).start_tree(tree, filename)
n = find_binding(u'next', tree)
if n:
self.warning(n, bind_warning)
self.shadowed_next = True
else:
self.shadowed_next = False
def transform(self, node, results):
assert results
base = results.get("base")
attr = results.get("attr")
name = results.get("name")
if base:
if self.shadowed_next:
attr.replace(Name(u"__next__", prefix=attr.prefix))
else:
base = [n.clone() for n in base]
base[0].prefix = u""
node.replace(Call(Name(u"next", prefix=node.prefix), base))
elif name:
n = Name(u"__next__", prefix=name.prefix)
name.replace(n)
elif attr:
# We don't do this transformation if we're assigning to "x.next".
# Unfortunately, it doesn't seem possible to do this in PATTERN,
# so it's being done here.
if is_assign_target(node):
head = results["head"]
if "".join([str(n) for n in head]).strip() == u'__builtin__':
self.warning(node, bind_warning)
return
attr.replace(Name(u"__next__"))
elif "global" in results:
self.warning(node, bind_warning)
self.shadowed_next = True
### The following functions help test if node is part of an assignment
### target.
def is_assign_target(node):
assign = find_assign(node)
if assign is None:
return False
for child in assign.children:
if child.type == token.EQUAL:
return False
elif is_subtree(child, node):
return True
return False
def find_assign(node):
if node.type == syms.expr_stmt:
return node
if node.type == syms.simple_stmt or node.parent is None:
return None
return find_assign(node.parent)
def is_subtree(root, node):
if root == node:
return True
return any(is_subtree(c, node) for c in root.children)
fixes/fix_print.py 0000666 00000005427 15050104230 0010240 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer for print.
Change:
'print' into 'print()'
'print ...' into 'print(...)'
'print ... ,' into 'print(..., end=" ")'
'print >>x, ...' into 'print(..., file=x)'
No changes are applied if print_function is imported from __future__
"""
# Local imports
from .. import patcomp
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, Call, Comma, String, is_tuple
parend_expr = patcomp.compile_pattern(
"""atom< '(' [atom|STRING|NAME] ')' >"""
)
class FixPrint(fixer_base.BaseFix):
PATTERN = """
simple_stmt< any* bare='print' any* > | print_stmt
"""
def transform(self, node, results):
assert results
bare_print = results.get("bare")
if bare_print:
# Special-case print all by itself
bare_print.replace(Call(Name(u"print"), [],
prefix=bare_print.prefix))
return
assert node.children[0] == Name(u"print")
args = node.children[1:]
if len(args) == 1 and parend_expr.match(args[0]):
# We don't want to keep sticking parens around an
# already-parenthesised expression.
return
sep = end = file = None
if args and args[-1] == Comma():
args = args[:-1]
end = " "
if args and args[0] == pytree.Leaf(token.RIGHTSHIFT, u">>"):
assert len(args) >= 2
file = args[1].clone()
args = args[3:] # Strip a possible comma after the file expression
# Now synthesize a print(args, sep=..., end=..., file=...) node.
l_args = [arg.clone() for arg in args]
if l_args:
l_args[0].prefix = u""
if sep is not None or end is not None or file is not None:
if sep is not None:
self.add_kwarg(l_args, u"sep", String(repr(sep)))
if end is not None:
self.add_kwarg(l_args, u"end", String(repr(end)))
if file is not None:
self.add_kwarg(l_args, u"file", file)
n_stmt = Call(Name(u"print"), l_args)
n_stmt.prefix = node.prefix
return n_stmt
def add_kwarg(self, l_nodes, s_kwd, n_expr):
# XXX All this prefix-setting may lose comments (though rarely)
n_expr.prefix = u""
n_argument = pytree.Node(self.syms.argument,
(Name(s_kwd),
pytree.Leaf(token.EQUAL, u"="),
n_expr))
if l_nodes:
l_nodes.append(Comma())
n_argument.prefix = u" "
l_nodes.append(n_argument)
fixes/fix_throw.py 0000666 00000003032 15050104230 0010235 0 ustar 00 """Fixer for generator.throw(E, V, T).
g.throw(E) -> g.throw(E)
g.throw(E, V) -> g.throw(E(V))
g.throw(E, V, T) -> g.throw(E(V).with_traceback(T))
g.throw("foo"[, V[, T]]) will warn about string exceptions."""
# Author: Collin Winter
# Local imports
from .. import pytree
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, Call, ArgList, Attr, is_tuple
class FixThrow(fixer_base.BaseFix):
PATTERN = """
power< any trailer< '.' 'throw' >
trailer< '(' args=arglist< exc=any ',' val=any [',' tb=any] > ')' >
>
|
power< any trailer< '.' 'throw' > trailer< '(' exc=any ')' > >
"""
def transform(self, node, results):
syms = self.syms
exc = results["exc"].clone()
if exc.type is token.STRING:
self.cannot_convert(node, "Python 3 does not support string exceptions")
return
# Leave "g.throw(E)" alone
val = results.get(u"val")
if val is None:
return
val = val.clone()
if is_tuple(val):
args = [c.clone() for c in val.children[1:-1]]
else:
val.prefix = u""
args = [val]
throw_args = results["args"]
if "tb" in results:
tb = results["tb"].clone()
tb.prefix = u""
e = Call(exc, args)
with_tb = Attr(e, Name(u'with_traceback')) + [ArgList([tb])]
throw_args.replace(pytree.Node(syms.power, with_tb))
else:
throw_args.replace(Call(exc, args))
fixes/fix_filter.py 0000666 00000004042 15050104230 0010361 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that changes filter(F, X) into list(filter(F, X)).
We avoid the transformation if the filter() call is directly contained
in iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or
for V in <>:.
NOTE: This is still not correct if the original code was depending on
filter(F, X) to return a string if X is a string and a tuple if X is a
tuple. That would require type inference, which we don't do. Let
Python 2.6 figure it out.
"""
# Local imports
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, Call, ListComp, in_special_context
class FixFilter(fixer_base.ConditionalFix):
PATTERN = """
filter_lambda=power<
'filter'
trailer<
'('
arglist<
lambdef< 'lambda'
(fp=NAME | vfpdef< '(' fp=NAME ')'> ) ':' xp=any
>
','
it=any
>
')'
>
>
|
power<
'filter'
trailer< '(' arglist< none='None' ',' seq=any > ')' >
>
|
power<
'filter'
args=trailer< '(' [any] ')' >
>
"""
skip_on = "future_builtins.filter"
def transform(self, node, results):
if self.should_skip(node):
return
if "filter_lambda" in results:
new = ListComp(results.get("fp").clone(),
results.get("fp").clone(),
results.get("it").clone(),
results.get("xp").clone())
elif "none" in results:
new = ListComp(Name(u"_f"),
Name(u"_f"),
results["seq"].clone(),
Name(u"_f"))
else:
if in_special_context(node):
return None
new = node.clone()
new.prefix = u""
new = Call(Name(u"list"), [new])
new.prefix = node.prefix
return new
fixes/fix_operator.pyo 0000666 00000003341 15050104230 0011107 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s Fixer for operator.{isCallable,sequenceIncludes}
operator.isCallable(obj) -> hasattr(obj, '__call__')
operator.sequenceIncludes(obj) -> operator.contains(obj)
i ( t
fixer_base( t Callt Namet Stringt FixOperatorc B s6 e Z d Z d Z d e d e d e Z d Z RS( s( method=('isCallable'|'sequenceIncludes')s '(' func=any ')'s
power< module='operator'
trailer< '.' %(methods)s > trailer< %(func)s > >
|
power< %(methods)s trailer< %(func)s > >
t methodst funcc C s | d d } | i d j o8 d | j o | i | d q d | _ | i n | i d j ow d | j o | i | d | d i q | d } | i t d
t d g } t t d | d
| i Sn d S( Nt methodi u sequenceIncludest modules& You should use operator.contains here.u containsu
isCallables, You should use hasattr(%s, '__call__') here.R u , u
'__call__'u hasattrt prefix( t valuet warningt changedt cloneR R R R ( t selft nodet resultsR R t args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_operator.pyt transform s
!( t __name__t
__module__R R t dictt PATTERNR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_operator.pyR s N( t __doc__t R t
fixer_utilR R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_operator.pyt s fixes/__init__.pyo 0000666 00000000205 15050104230 0010141 0 ustar 00
Lc @ s d S( N( ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/__init__.pyt s fixes/fix_ws_comma.pyc 0000666 00000002577 15050104230 0011057 0 ustar 00
Lc @ sS d Z d d k l Z d d k l Z d d k l Z d e i f d YZ d S( s Fixer that changes 'a ,b' into 'a, b'.
This also changes '{a :b}' into '{a: b}', but does not touch other
uses of colons. It does not touch other uses of whitespace.
i ( t pytree( t token( t
fixer_baset
FixWsCommac B sS e Z e Z d Z e i e i d Z e i e i d Z e e f Z
d Z RS( sH
any<(not(',') any)+ ',' ((not(',') any)+ ',')* [not(',') any]>
u ,u :c C s | i } t } x | i D]~ } | | i j o: | i } | i o d | j o
d | _ n t } q | o! | i } | p
d | _ q n t } q W| S( Nu
u u ( t clonet Falset childrent SEPSt prefixt isspacet True( t selft nodet resultst newt commat childR ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_ws_comma.pyt transform s
( t __name__t
__module__R
t explicitt PATTERNR t LeafR t COMMAt COLONR R ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_ws_comma.pyR s N( t __doc__t R t pgen2R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_ws_comma.pyt s fixes/fix_future.pyc 0000666 00000001602 15050104230 0010550 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( sV Remove __future__ imports
from __future__ import foo is replaced with an empty line.
i ( t
fixer_base( t BlankLinet FixFuturec B s e Z d Z d Z d Z RS( s; import_from< 'from' module_name="__future__" 'import' any >i
c C s t } | i | _ | S( N( R t prefix( t selft nodet resultst new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_future.pyt transform s ( t __name__t
__module__t PATTERNt run_orderR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_future.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_future.pyt s fixes/fix_import.pyo 0000666 00000006314 15050104230 0010571 0 ustar 00
Lc @ sz d Z d d k l Z d d k l Z l Z l Z l Z d d k l Z l
Z
l Z d Z d e i
f d YZ d S(
s Fixer for import statements.
If spam is being imported from the local directory, this import:
from spam import eggs
Becomes:
from .spam import eggs
And this import:
import spam
Becomes:
from . import spam
i ( t
fixer_basei( t dirnamet joint existst sep( t
FromImportt symst tokenc c s | g } x | o | i } | i t i j o | i Vq | i t i j o3 d i g } | i D] } | | i qe ~ Vq | i t i j o | i
| i d q | i t i j o! | i | i d d d q t
d q Wd S( sF
Walks over all the names imported in a dotted_as_names node.
t i Nis unkown node type( t popt typeR t NAMEt valueR t dotted_nameR t childrent dotted_as_namet appendt dotted_as_namest extendt AssertionError( t namest pendingt nodet _[1]t ch( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyt traverse_imports s 3!t FixImportc B s) e Z d Z d Z d Z d Z RS( sj
import_from< 'from' imp=any 'import' ['('] any [')'] >
|
import_name< 'import' imp=any >
c C s/ t t | i | | d | i j | _ d S( Nt absolute_import( t superR t
start_treet future_featurest skip( t selft treet name( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyR . s c C s | i o d S| d } | i t i j oZ x" t | d p | i d } q/ W| i | i o d | i | _ | i qn t } t } x4 t
| D]& } | i | o
t } q t } q W| o | o | i | d n d St
d | g } | i | _ | Sd S( Nt impR i u .s# absolute and local imports together( R R
R t import_fromt hasattrR t probably_a_local_importR t changedt FalseR t Truet warningR t prefix( R R t resultsR# t
have_localt
have_absolutet mod_namet new( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyt transform2 s0
c C s | i d o t S| i d d d } t | i } t | | } t t t | d p t Sx6 d t d d d d g D] } t | | o t Sq Wt S(
Nu .i i s __init__.pys .pys .pycs .sos .sls .pyd( t
startswithR( t splitR t filenameR R R R) ( R t imp_namet base_patht ext( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyR& T s ( t __name__t
__module__t PATTERNR R1 R& ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyR & s "N( t __doc__R R t os.pathR R R R t
fixer_utilR R R R t BaseFixR ( ( ( s0 /usr/lib64/python2.6/lib2to3/fixes/fix_import.pyt s
" fixes/fix_metaclass.py 0000666 00000017774 15050104230 0011070 0 ustar 00 """Fixer for __metaclass__ = X -> (metaclass=X) methods.
The various forms of classef (inherits nothing, inherits once, inherints
many) don't parse the same in the CST so we look at ALL classes for
a __metaclass__ and if we find one normalize the inherits to all be
an arglist.
For one-liner classes ('class X: pass') there is no indent/dedent so
we normalize those into having a suite.
Moving the __metaclass__ into the classdef can also cause the class
body to be empty so there is some special casing for that as well.
This fixer also tries very hard to keep original indenting and spacing
in all those corner cases.
"""
# Author: Jack Diederich
# Local imports
from .. import fixer_base
from ..pygram import token
from ..fixer_util import Name, syms, Node, Leaf
def has_metaclass(parent):
""" we have to check the cls_node without changing it.
There are two possiblities:
1) clsdef => suite => simple_stmt => expr_stmt => Leaf('__meta')
2) clsdef => simple_stmt => expr_stmt => Leaf('__meta')
"""
for node in parent.children:
if node.type == syms.suite:
return has_metaclass(node)
elif node.type == syms.simple_stmt and node.children:
expr_node = node.children[0]
if expr_node.type == syms.expr_stmt and expr_node.children:
left_side = expr_node.children[0]
if isinstance(left_side, Leaf) and \
left_side.value == '__metaclass__':
return True
return False
def fixup_parse_tree(cls_node):
""" one-line classes don't get a suite in the parse tree so we add
one to normalize the tree
"""
for node in cls_node.children:
if node.type == syms.suite:
# already in the prefered format, do nothing
return
# !%@#! oneliners have no suite node, we have to fake one up
for i, node in enumerate(cls_node.children):
if node.type == token.COLON:
break
else:
raise ValueError("No class suite and no ':'!")
# move everything into a suite node
suite = Node(syms.suite, [])
while cls_node.children[i+1:]:
move_node = cls_node.children[i+1]
suite.append_child(move_node.clone())
move_node.remove()
cls_node.append_child(suite)
node = suite
def fixup_simple_stmt(parent, i, stmt_node):
""" if there is a semi-colon all the parts count as part of the same
simple_stmt. We just want the __metaclass__ part so we move
everything efter the semi-colon into its own simple_stmt node
"""
for semi_ind, node in enumerate(stmt_node.children):
if node.type == token.SEMI: # *sigh*
break
else:
return
node.remove() # kill the semicolon
new_expr = Node(syms.expr_stmt, [])
new_stmt = Node(syms.simple_stmt, [new_expr])
while stmt_node.children[semi_ind:]:
move_node = stmt_node.children[semi_ind]
new_expr.append_child(move_node.clone())
move_node.remove()
parent.insert_child(i, new_stmt)
new_leaf1 = new_stmt.children[0].children[0]
old_leaf1 = stmt_node.children[0].children[0]
new_leaf1.prefix = old_leaf1.prefix
def remove_trailing_newline(node):
if node.children and node.children[-1].type == token.NEWLINE:
node.children[-1].remove()
def find_metas(cls_node):
# find the suite node (Mmm, sweet nodes)
for node in cls_node.children:
if node.type == syms.suite:
break
else:
raise ValueError("No class suite!")
# look for simple_stmt[ expr_stmt[ Leaf('__metaclass__') ] ]
for i, simple_node in list(enumerate(node.children)):
if simple_node.type == syms.simple_stmt and simple_node.children:
expr_node = simple_node.children[0]
if expr_node.type == syms.expr_stmt and expr_node.children:
# Check if the expr_node is a simple assignment.
left_node = expr_node.children[0]
if isinstance(left_node, Leaf) and \
left_node.value == u'__metaclass__':
# We found a assignment to __metaclass__.
fixup_simple_stmt(node, i, simple_node)
remove_trailing_newline(simple_node)
yield (node, i, simple_node)
def fixup_indent(suite):
""" If an INDENT is followed by a thing with a prefix then nuke the prefix
Otherwise we get in trouble when removing __metaclass__ at suite start
"""
kids = suite.children[::-1]
# find the first indent
while kids:
node = kids.pop()
if node.type == token.INDENT:
break
# find the first Leaf
while kids:
node = kids.pop()
if isinstance(node, Leaf) and node.type != token.DEDENT:
if node.prefix:
node.prefix = u''
return
else:
kids.extend(node.children[::-1])
class FixMetaclass(fixer_base.BaseFix):
PATTERN = """
classdef
"""
def transform(self, node, results):
if not has_metaclass(node):
return
fixup_parse_tree(node)
# find metaclasses, keep the last one
last_metaclass = None
for suite, i, stmt in find_metas(node):
last_metaclass = stmt
stmt.remove()
text_type = node.children[0].type # always Leaf(nnn, 'class')
# figure out what kind of classdef we have
if len(node.children) == 7:
# Node(classdef, ['class', 'name', '(', arglist, ')', ':', suite])
# 0 1 2 3 4 5 6
if node.children[3].type == syms.arglist:
arglist = node.children[3]
# Node(classdef, ['class', 'name', '(', 'Parent', ')', ':', suite])
else:
parent = node.children[3].clone()
arglist = Node(syms.arglist, [parent])
node.set_child(3, arglist)
elif len(node.children) == 6:
# Node(classdef, ['class', 'name', '(', ')', ':', suite])
# 0 1 2 3 4 5
arglist = Node(syms.arglist, [])
node.insert_child(3, arglist)
elif len(node.children) == 4:
# Node(classdef, ['class', 'name', ':', suite])
# 0 1 2 3
arglist = Node(syms.arglist, [])
node.insert_child(2, Leaf(token.RPAR, u')'))
node.insert_child(2, arglist)
node.insert_child(2, Leaf(token.LPAR, u'('))
else:
raise ValueError("Unexpected class definition")
# now stick the metaclass in the arglist
meta_txt = last_metaclass.children[0].children[0]
meta_txt.value = 'metaclass'
orig_meta_prefix = meta_txt.prefix
if arglist.children:
arglist.append_child(Leaf(token.COMMA, u','))
meta_txt.prefix = u' '
else:
meta_txt.prefix = u''
# compact the expression "metaclass = Meta" -> "metaclass=Meta"
expr_stmt = last_metaclass.children[0]
assert expr_stmt.type == syms.expr_stmt
expr_stmt.children[1].prefix = u''
expr_stmt.children[2].prefix = u''
arglist.append_child(last_metaclass)
fixup_indent(suite)
# check for empty suite
if not suite.children:
# one-liner that was just __metaclass_
suite.remove()
pass_leaf = Leaf(text_type, u'pass')
pass_leaf.prefix = orig_meta_prefix
node.append_child(pass_leaf)
node.append_child(Leaf(token.NEWLINE, u'\n'))
elif len(suite.children) > 1 and \
(suite.children[-2].type == token.INDENT and
suite.children[-1].type == token.DEDENT):
# there was only one line in the class body and it was __metaclass__
pass_leaf = Leaf(text_type, u'pass')
suite.insert_child(-1, pass_leaf)
suite.insert_child(-1, Leaf(token.NEWLINE, u'\n'))
fixes/fix_isinstance.pyo 0000666 00000003450 15050104230 0011415 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s, Fixer that cleans up a tuple argument to isinstance after the tokens
in it were fixed. This is mainly used to remove double occurrences of
tokens as a leftover of the long -> int / unicode -> str conversion.
eg. isinstance(x, (int, long)) -> isinstance(x, (int, int))
-> isinstance(x, int)
i ( t
fixer_base( t tokent
FixIsinstancec B s e Z d Z d Z d Z RS( s
power<
'isinstance'
trailer< '(' arglist< any ',' atom< '('
args=testlist_gexp< any+ >
')' > > ')' >
>
i c C sb t } | d } | i } g } t | } x | D] \ } } | i t i j oW | i | j oG | t | d j o, | | d i t i j o | i q5 q q5 | i
| | i t i j o | i | i q5 q5 W| o" | d i t i j o | d =n t | d j o. | i }
|
i
| d _
|
i | d n | | (| i d S( Nt argsi ii ( t sett childrent enumeratet typeR t NAMEt valuet lent COMMAt nextt appendt addt parentt prefixt replacet changed( t selft nodet resultst names_insertedt testlistR t new_argst iteratort idxt argt atom( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_isinstance.pyt transform s,
#2
( t __name__t
__module__t PATTERNt run_orderR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_isinstance.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s4 /usr/lib64/python2.6/lib2to3/fixes/fix_isinstance.pyt
s fixes/fix_repr.py 0000666 00000001115 15050104230 0010042 0 ustar 00 # Copyright 2006 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that transforms `xyzzy` into repr(xyzzy)."""
# Local imports
from .. import fixer_base
from ..fixer_util import Call, Name, parenthesize
class FixRepr(fixer_base.BaseFix):
PATTERN = """
atom < '`' expr=any '`' >
"""
def transform(self, node, results):
expr = results["expr"].clone()
if expr.type == self.syms.testlist1:
expr = parenthesize(expr)
return Call(Name(u"repr"), [expr], prefix=node.prefix)
fixes/fix_exec.pyc 0000666 00000002573 15050104230 0010172 0 ustar 00
Lc @ s_ d Z d d k l Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s Fixer for exec.
This converts usages of the exec statement into calls to a built-in
exec() function.
exec code in ns1, ns2 -> exec(code, ns1, ns2)
i ( t pytree( t
fixer_base( t Commat Namet Callt FixExecc B s e Z d Z d Z RS( sx
exec_stmt< 'exec' a=any 'in' b=any [',' c=any] >
|
exec_stmt< 'exec' (not atom<'(' [any] ')'>) a=any >
c C s | p t | i } | d } | i d } | i d } | i g } d | d _ | d j o | i t | i g n | d j o | i t | i g n t t d | d | i S( Nt at bt ct i u exect prefix(
t AssertionErrort symst gett cloneR
t Nonet extendR R R ( t selft nodet resultsR R R R t args( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_exec.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_exec.pyR s N(
t __doc__R R R t
fixer_utilR R R t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_exec.pyt
s fixes/fix_sys_exc.pyc 0000666 00000003232 15050104230 0010714 0 ustar 00
Lc @ sg d Z d d k l Z d d k l Z l Z l Z l Z l Z l Z l
Z
d e i f d YZ d S( s Fixer for sys.exc_{type, value, traceback}
sys.exc_type -> sys.exc_info()[0]
sys.exc_value -> sys.exc_info()[1]
sys.exc_traceback -> sys.exc_info()[2]
i ( t
fixer_base( t Attrt Callt Namet Numbert Subscriptt Nodet symst FixSysExcc B s= e Z d d d g Z d d i d e D Z d Z RS( u exc_typeu exc_valueu
exc_tracebacksN
power< 'sys' trailer< dot='.' attribute=(%s) > >
t |c c s x | ] } d | Vq Wd S( s '%s'N( ( t .0t e( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pys s c C s | d d } t | i i | i } t t d d | i } t t d | } | d i | d i d _ | i t
| t t i
| d | i S( Nt attributei u exc_infot prefixu syst doti ( R t exc_infot indext valueR R R
R t childrent appendR R R t power( t selft nodet resultst sys_attrR t callt attr( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pyt transform s ( t __name__t
__module__R t joint PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pyR s N(
t __doc__t R t
fixer_utilR R R R R R R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pyt s 4 fixes/fix_itertools.py 0000666 00000002700 15050104230 0011117 0 ustar 00 """ Fixer for itertools.(imap|ifilter|izip) --> (map|filter|zip) and
itertools.ifilterfalse --> itertools.filterfalse (bugs 2360-2363)
imports from itertools are fixed in fix_itertools_import.py
If itertools is imported as something else (ie: import itertools as it;
it.izip(spam, eggs)) method calls will not get fixed.
"""
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixItertools(fixer_base.BaseFix):
it_funcs = "('imap'|'ifilter'|'izip'|'ifilterfalse')"
PATTERN = """
power< it='itertools'
trailer<
dot='.' func=%(it_funcs)s > trailer< '(' [any] ')' > >
|
power< func=%(it_funcs)s trailer< '(' [any] ')' > >
""" %(locals())
# Needs to be run after fix_(map|zip|filter)
run_order = 6
def transform(self, node, results):
prefix = None
func = results['func'][0]
if 'it' in results and func.value != u'ifilterfalse':
dot, it = (results['dot'], results['it'])
# Remove the 'itertools'
prefix = it.prefix
it.remove()
# Replace the node wich contains ('.', 'function') with the
# function (to be consistant with the second part of the pattern)
dot.remove()
func.parent.replace(func)
prefix = prefix or func.prefix
func.replace(Name(func.value[1:], prefix=prefix))
fixes/fix_dict.pyc 0000666 00000007315 15050104230 0010170 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l
Z
l Z l Z l
Z
d d k l Z e i e d g BZ d e i f d
YZ d S( sj Fixer for dict methods.
d.keys() -> list(d.keys())
d.items() -> list(d.items())
d.values() -> list(d.values())
d.iterkeys() -> iter(d.keys())
d.iteritems() -> iter(d.items())
d.itervalues() -> iter(d.values())
d.viewkeys() -> d.keys()
d.viewitems() -> d.items()
d.viewvalues() -> d.values()
Except in certain very specific contexts: the iter() can be dropped
when the context is list(), sorted(), iter() or for...in; the list()
can be dropped when the context is list() or sorted() (but not iter()
or for...in!). Special contexts that apply to both: list(), sorted(), tuple()
set(), any(), all(), sum().
Note: iter(d.keys()) could be written as iter(d) but since the
original d.iterkeys() was also redundant we don't fix this. And there
are (rare) contexts where it makes a difference (e.g. when passing it
as an argument to a function that introspects the argument).
i ( t pytree( t patcomp( t token( t
fixer_base( t Namet Callt LParent RParent ArgListt Dot( t
fixer_utilt itert FixDictc B sJ e Z d Z d Z d Z e i e Z d Z e i e Z d Z
RS( s
power< head=any+
trailer< '.' method=('keys'|'items'|'values'|
'iterkeys'|'iteritems'|'itervalues'|
'viewkeys'|'viewitems'|'viewvalues') >
parens=trailer< '(' ')' >
tail=any*
>
c
C s | d } | d d } | d } | i } | i } | i d } | i d } | p | o | d } n | d j p t t | g }
| D] } |
| i q ~
} g } | D] } | | i q ~ } | o | i | | }
| t i | i t
t | d | i g | d i g } t i | i
| } |
p | p3 d
| _ t t | o d n d | g } n | o t i | i
| g | } n | i | _ | S( Nt headt methodi t tailu iteru viewi u keysu itemsu valuest prefixt parensu u list( u keysu itemsu values( t symst valuet
startswitht AssertionErrort reprt clonet in_special_contextR t Nodet trailerR R R t powerR ( t selft nodet resultsR
R R R t method_namet isitert isviewt _[1]t nt _[2]t specialt argst new( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyt transform5 s4
'' * s3 power< func=NAME trailer< '(' node=any ')' > any* >sm for_stmt< 'for' any 'in' node=any ':' any* >
| comp_for< 'for' any 'in' node=any any* >
c C s | i d j o t Sh } | i i d j o^ | i i | i i | oB | d | j o1 | o | d i t j S| d i t i j Sn | p t S| i i | i | o | d | j S( NR t func(
t parentt Nonet Falset p1t matchR t iter_exemptR
t consuming_callst p2( R R R R ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyR Y s ( t __name__t
__module__t PATTERNR( t P1R t compile_patternR- t P2R1 R ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyR * s N( t __doc__t R R t pgen2R R R
R R R R R R R0 t setR/ t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_dict.pyt s . fixes/fix_raw_input.pyc 0000666 00000001623 15050104230 0011251 0 ustar 00
Lc @ sC d Z d d k l Z d d k l Z d e i f d YZ d S( s2 Fixer that changes raw_input(...) into input(...).i ( t
fixer_base( t Namet FixRawInputc B s e Z d Z d Z RS( sU
power< name='raw_input' trailer< '(' [any] ')' > any* >
c C s* | d } | i t d d | i d S( Nt nameu inputt prefix( t replaceR R ( t selft nodet resultsR ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_raw_input.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_raw_input.pyR s N( t __doc__t R t
fixer_utilR t BaseFixR ( ( ( s3 /usr/lib64/python2.6/lib2to3/fixes/fix_raw_input.pyt s fixes/fix_set_literal.py 0000666 00000003212 15050104230 0011401 0 ustar 00 """
Optional fixer to transform set() calls to set literals.
"""
# Author: Benjamin Peterson
from lib2to3 import fixer_base, pytree
from lib2to3.fixer_util import token, syms
class FixSetLiteral(fixer_base.BaseFix):
explicit = True
PATTERN = """power< 'set' trailer< '('
(atom=atom< '[' (items=listmaker< any ((',' any)* [',']) >
|
single=any) ']' >
|
atom< '(' items=testlist_gexp< any ((',' any)* [',']) > ')' >
)
')' > >
"""
def transform(self, node, results):
single = results.get("single")
if single:
# Make a fake listmaker
fake = pytree.Node(syms.listmaker, [single.clone()])
single.replace(fake)
items = fake
else:
items = results["items"]
# Build the contents of the literal
literal = [pytree.Leaf(token.LBRACE, u"{")]
literal.extend(n.clone() for n in items.children)
literal.append(pytree.Leaf(token.RBRACE, u"}"))
# Set the prefix of the right brace to that of the ')' or ']'
literal[-1].prefix = items.next_sibling.prefix
maker = pytree.Node(syms.dictsetmaker, literal)
maker.prefix = node.prefix
# If the original was a one tuple, we need to remove the extra comma.
if len(maker.children) == 4:
n = maker.children[2]
n.remove()
maker.children[-1].prefix = n.prefix
# Finally, replace the set call with our shiny new literal.
return maker
fixes/fix_map.py 0000666 00000005735 15050104230 0007663 0 ustar 00 # Copyright 2007 Google, Inc. All Rights Reserved.
# Licensed to PSF under a Contributor Agreement.
"""Fixer that changes map(F, ...) into list(map(F, ...)) unless there
exists a 'from future_builtins import map' statement in the top-level
namespace.
As a special case, map(None, X) is changed into list(X). (This is
necessary because the semantics are changed in this case -- the new
map(None, X) is equivalent to [(x,) for x in X].)
We avoid the transformation (except for the special case mentioned
above) if the map() call is directly contained in iter(<>), list(<>),
tuple(<>), sorted(<>), ...join(<>), or for V in <>:.
NOTE: This is still not correct if the original code was depending on
map(F, X, Y, ...) to go on until the longest argument is exhausted,
substituting None for missing values -- like zip(), it now stops as
soon as the shortest argument is exhausted.
"""
# Local imports
from ..pgen2 import token
from .. import fixer_base
from ..fixer_util import Name, Call, ListComp, in_special_context
from ..pygram import python_symbols as syms
class FixMap(fixer_base.ConditionalFix):
PATTERN = """
map_none=power<
'map'
trailer< '(' arglist< 'None' ',' arg=any [','] > ')' >
>
|
map_lambda=power<
'map'
trailer<
'('
arglist<
lambdef< 'lambda'
(fp=NAME | vfpdef< '(' fp=NAME ')'> ) ':' xp=any
>
','
it=any
>
')'
>
>
|
power<
'map' trailer< '(' [arglist=any] ')' >
>
"""
skip_on = 'future_builtins.map'
def transform(self, node, results):
if self.should_skip(node):
return
if node.parent.type == syms.simple_stmt:
self.warning(node, "You should use a for loop here")
new = node.clone()
new.prefix = u""
new = Call(Name(u"list"), [new])
elif "map_lambda" in results:
new = ListComp(results["xp"].clone(),
results["fp"].clone(),
results["it"].clone())
else:
if "map_none" in results:
new = results["arg"].clone()
else:
if "arglist" in results:
args = results["arglist"]
if args.type == syms.arglist and \
args.children[0].type == token.NAME and \
args.children[0].value == "None":
self.warning(node, "cannot convert map(None, ...) "
"with multiple arguments because map() "
"now truncates to the shortest sequence")
return
if in_special_context(node):
return None
new = node.clone()
new.prefix = u""
new = Call(Name(u"list"), [new])
new.prefix = node.prefix
return new
fixes/fix_exec.pyo 0000666 00000002530 15050104230 0010177 0 ustar 00
Lc @ s_ d Z d d k l Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s Fixer for exec.
This converts usages of the exec statement into calls to a built-in
exec() function.
exec code in ns1, ns2 -> exec(code, ns1, ns2)
i ( t pytree( t
fixer_base( t Commat Namet Callt FixExecc B s e Z d Z d Z RS( sx
exec_stmt< 'exec' a=any 'in' b=any [',' c=any] >
|
exec_stmt< 'exec' (not atom<'(' [any] ')'>) a=any >
c C s | i } | d } | i d } | i d } | i g } d | d _ | d j o | i t | i g n | d j o | i t | i g n t t d | d | i S( Nt at bt ct i u exect prefix( t symst gett cloneR
t Nonet extendR R R ( t selft nodet resultsR R R R t args( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_exec.pyt transform s
( t __name__t
__module__t PATTERNR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_exec.pyR s N(
t __doc__R R R t
fixer_utilR R R t BaseFixR ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/fix_exec.pyt
s fixes/fix_urllib.py 0000666 00000016526 15050104230 0010377 0 ustar 00 """Fix changes imports of urllib which are now incompatible.
This is rather similar to fix_imports, but because of the more
complex nature of the fixing for urllib, it has its own fixer.
"""
# Author: Nick Edds
# Local imports
from .fix_imports import alternates, FixImports
from .. import fixer_base
from ..fixer_util import Name, Comma, FromImport, Newline, attr_chain
MAPPING = {'urllib': [
('urllib.request',
['URLOpener', 'FancyURLOpener', 'urlretrieve',
'_urlopener', 'urlopen', 'urlcleanup',
'pathname2url', 'url2pathname']),
('urllib.parse',
['quote', 'quote_plus', 'unquote', 'unquote_plus',
'urlencode', 'splitattr', 'splithost', 'splitnport',
'splitpasswd', 'splitport', 'splitquery', 'splittag',
'splittype', 'splituser', 'splitvalue', ]),
('urllib.error',
['ContentTooShortError'])],
'urllib2' : [
('urllib.request',
['urlopen', 'install_opener', 'build_opener',
'Request', 'OpenerDirector', 'BaseHandler',
'HTTPDefaultErrorHandler', 'HTTPRedirectHandler',
'HTTPCookieProcessor', 'ProxyHandler',
'HTTPPasswordMgr',
'HTTPPasswordMgrWithDefaultRealm',
'AbstractBasicAuthHandler',
'HTTPBasicAuthHandler', 'ProxyBasicAuthHandler',
'AbstractDigestAuthHandler',
'HTTPDigestAuthHandler', 'ProxyDigestAuthHandler',
'HTTPHandler', 'HTTPSHandler', 'FileHandler',
'FTPHandler', 'CacheFTPHandler',
'UnknownHandler']),
('urllib.error',
['URLError', 'HTTPError']),
]
}
# Duplicate the url parsing functions for urllib2.
MAPPING["urllib2"].append(MAPPING["urllib"][1])
def build_pattern():
bare = set()
for old_module, changes in MAPPING.items():
for change in changes:
new_module, members = change
members = alternates(members)
yield """import_name< 'import' (module=%r
| dotted_as_names< any* module=%r any* >) >
""" % (old_module, old_module)
yield """import_from< 'from' mod_member=%r 'import'
( member=%s | import_as_name< member=%s 'as' any > |
import_as_names< members=any* >) >
""" % (old_module, members, members)
yield """import_from< 'from' module_star=%r 'import' star='*' >
""" % old_module
yield """import_name< 'import'
dotted_as_name< module_as=%r 'as' any > >
""" % old_module
# bare_with_attr has a special significance for FixImports.match().
yield """power< bare_with_attr=%r trailer< '.' member=%s > any* >
""" % (old_module, members)
class FixUrllib(FixImports):
def build_pattern(self):
return "|".join(build_pattern())
def transform_import(self, node, results):
"""Transform for the basic import case. Replaces the old
import name with a comma separated list of its
replacements.
"""
import_mod = results.get('module')
pref = import_mod.prefix
names = []
# create a Node list of the replacement modules
for name in MAPPING[import_mod.value][:-1]:
names.extend([Name(name[0], prefix=pref), Comma()])
names.append(Name(MAPPING[import_mod.value][-1][0], prefix=pref))
import_mod.replace(names)
def transform_member(self, node, results):
"""Transform for imports of specific module elements. Replaces
the module to be imported from with the appropriate new
module.
"""
mod_member = results.get('mod_member')
pref = mod_member.prefix
member = results.get('member')
# Simple case with only a single member being imported
if member:
# this may be a list of length one, or just a node
if isinstance(member, list):
member = member[0]
new_name = None
for change in MAPPING[mod_member.value]:
if member.value in change[1]:
new_name = change[0]
break
if new_name:
mod_member.replace(Name(new_name, prefix=pref))
else:
self.cannot_convert(node,
'This is an invalid module element')
# Multiple members being imported
else:
# a dictionary for replacements, order matters
modules = []
mod_dict = {}
members = results.get('members')
for member in members:
member = member.value
# we only care about the actual members
if member != ',':
for change in MAPPING[mod_member.value]:
if member in change[1]:
if change[0] in mod_dict:
mod_dict[change[0]].append(member)
else:
mod_dict[change[0]] = [member]
modules.append(change[0])
new_nodes = []
for module in modules:
elts = mod_dict[module]
names = []
for elt in elts[:-1]:
names.extend([Name(elt, prefix=pref), Comma()])
names.append(Name(elts[-1], prefix=pref))
new_nodes.append(FromImport(module, names))
if new_nodes:
nodes = []
for new_node in new_nodes[:-1]:
nodes.extend([new_node, Newline()])
nodes.append(new_nodes[-1])
node.replace(nodes)
else:
self.cannot_convert(node, 'All module elements are invalid')
def transform_dot(self, node, results):
"""Transform for calls to module members in code."""
module_dot = results.get('bare_with_attr')
member = results.get('member')
new_name = None
if isinstance(member, list):
member = member[0]
for change in MAPPING[module_dot.value]:
if member.value in change[1]:
new_name = change[0]
break
if new_name:
module_dot.replace(Name(new_name,
prefix=module_dot.prefix))
else:
self.cannot_convert(node, 'This is an invalid module element')
def transform(self, node, results):
if results.get('module'):
self.transform_import(node, results)
elif results.get('mod_member'):
self.transform_member(node, results)
elif results.get('bare_with_attr'):
self.transform_dot(node, results)
# Renaming and star imports are not supported for these modules.
elif results.get('module_star'):
self.cannot_convert(node, 'Cannot handle star imports.')
elif results.get('module_as'):
self.cannot_convert(node, 'This module is now multiple modules')
fixes/fix_map.pyo 0000666 00000005733 15050104230 0010040 0 ustar 00
Lc @ su d Z d d k l Z d d k l Z d d k l Z l Z l Z l Z d d k
l Z d e i
f d YZ d S( s Fixer that changes map(F, ...) into list(map(F, ...)) unless there
exists a 'from future_builtins import map' statement in the top-level
namespace.
As a special case, map(None, X) is changed into list(X). (This is
necessary because the semantics are changed in this case -- the new
map(None, X) is equivalent to [(x,) for x in X].)
We avoid the transformation (except for the special case mentioned
above) if the map() call is directly contained in iter(<>), list(<>),
tuple(<>), sorted(<>), ...join(<>), or for V in <>:.
NOTE: This is still not correct if the original code was depending on
map(F, X, Y, ...) to go on until the longest argument is exhausted,
substituting None for missing values -- like zip(), it now stops as
soon as the shortest argument is exhausted.
i ( t token( t
fixer_base( t Namet Callt ListCompt in_special_context( t python_symbolst FixMapc B s e Z d Z d Z d Z RS( s
map_none=power<
'map'
trailer< '(' arglist< 'None' ',' arg=any [','] > ')' >
>
|
map_lambda=power<
'map'
trailer<
'('
arglist<
lambdef< 'lambda'
(fp=NAME | vfpdef< '(' fp=NAME ')'> ) ':' xp=any
>
','
it=any
>
')'
>
>
|
power<
'map' trailer< '(' [arglist=any] ')' >
>
s future_builtins.mapc C s | i | o d S| i i t i j oA | i | d | i } d | _ t t d | g } nd | j o4 t
| d i | d i | d i } n d | j o | d i } n d
| j og | d
} | i t i j oF | i d i t
i j o, | i d i d j o | i | d
d Sn t | o d S| i } d | _ t t d | g } | i | _ | S( Ns You should use a for loop hereu u listt
map_lambdat xpt fpt itt map_nonet argt arglisti t Nonesj cannot convert map(None, ...) with multiple arguments because map() now truncates to the shortest sequence( t should_skipt parentt typet symst simple_stmtt warningt clonet prefixR R R R t childrenR t NAMEt valueR R ( t selft nodet resultst newt args( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_map.pyt transform: s6
( t __name__t
__module__t PATTERNt skip_onR ( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_map.pyR s N( t __doc__t pgen2R t R t
fixer_utilR R R R t pygramR R t ConditionalFixR ( ( ( s- /usr/lib64/python2.6/lib2to3/fixes/fix_map.pyt s
" fixes/fix_itertools_imports.py 0000666 00000003460 15050104230 0012700 0 ustar 00 """ Fixer for imports of itertools.(imap|ifilter|izip|ifilterfalse) """
# Local imports
from lib2to3 import fixer_base
from lib2to3.fixer_util import BlankLine, syms, token
class FixItertoolsImports(fixer_base.BaseFix):
PATTERN = """
import_from< 'from' 'itertools' 'import' imports=any >
""" %(locals())
def transform(self, node, results):
imports = results['imports']
if imports.type == syms.import_as_name or not imports.children:
children = [imports]
else:
children = imports.children
for child in children[::2]:
if child.type == token.NAME:
member = child.value
name_node = child
else:
assert child.type == syms.import_as_name
name_node = child.children[0]
member_name = name_node.value
if member_name in (u'imap', u'izip', u'ifilter'):
child.value = None
child.remove()
elif member_name == u'ifilterfalse':
node.changed()
name_node.value = u'filterfalse'
# Make sure the import statement is still sane
children = imports.children[:] or [imports]
remove_comma = True
for child in children:
if remove_comma and child.type == token.COMMA:
child.remove()
else:
remove_comma ^= True
if children[-1].type == token.COMMA:
children[-1].remove()
# If there are no imports left, just get rid of the entire statement
if not (imports.children or getattr(imports, 'value', None)) or \
imports.parent is None:
p = node.prefix
node = BlankLine()
node.prefix = p
return node
fixes/fix_sys_exc.pyo 0000666 00000003232 15050104230 0010730 0 ustar 00
Lc @ sg d Z d d k l Z d d k l Z l Z l Z l Z l Z l Z l
Z
d e i f d YZ d S( s Fixer for sys.exc_{type, value, traceback}
sys.exc_type -> sys.exc_info()[0]
sys.exc_value -> sys.exc_info()[1]
sys.exc_traceback -> sys.exc_info()[2]
i ( t
fixer_base( t Attrt Callt Namet Numbert Subscriptt Nodet symst FixSysExcc B s= e Z d d d g Z d d i d e D Z d Z RS( u exc_typeu exc_valueu
exc_tracebacksN
power< 'sys' trailer< dot='.' attribute=(%s) > >
t |c c s x | ] } d | Vq Wd S( s '%s'N( ( t .0t e( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pys s c C s | d d } t | i i | i } t t d d | i } t t d | } | d i | d i d _ | i t
| t t i
| d | i S( Nt attributei u exc_infot prefixu syst doti ( R t exc_infot indext valueR R R
R t childrent appendR R R t power( t selft nodet resultst sys_attrR t callt attr( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pyt transform s ( t __name__t
__module__R t joint PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pyR s N(
t __doc__t R t
fixer_utilR R R R R R R t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_sys_exc.pyt s 4 fixes/fix_types.pyo 0000666 00000004207 15050104230 0010422 0 ustar 00
Lc @ s d Z d d k l Z d d k l Z d d k l Z h d d 6d d 6d d
6d d 6d
d 6d
d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d d 6d! d" 6d# d$ 6d% d& 6d d' 6d# d( 6d) d* 6Z g Z e D] Z e d+ e q [ Z
d, e i f d- YZ d. S(/ s Fixer for removing uses of the types module.
These work for only the known names in the types module. The forms above
can include types. or not. ie, It is assumed the module is imported either as:
import types
from types import ... # either * or specific types
The import statements are not modified.
There should be another fixer that handles at least the following constants:
type([]) -> list
type(()) -> tuple
type('') -> str
i ( t token( t
fixer_base( t Namet boolt BooleanTypet
memoryviewt
BufferTypet typet ClassTypet complext ComplexTypet dictt DictTypet DictionaryTypes type(Ellipsis)t EllipsisTypet floatt FloatTypet intt IntTypet listt ListTypet LongTypet objectt
ObjectTypes
type(None)t NoneTypes type(NotImplemented)t NotImplementedTypet slicet SliceTypet bytest
StringTypet strt StringTypest tuplet TupleTypet TypeTypet UnicodeTypet ranget
XRangeTypes) power< 'types' trailer< '.' name='%s' > >t FixTypesc B s e Z d i e Z d Z RS( t |c C s; t t i | d i } | o t | d | i Sd S( Nt namet prefix( t unicodet
_TYPE_MAPPINGt gett valueR R) t None( t selft nodet resultst new_value( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_types.pyt transform: s ( t __name__t
__module__t joint _patst PATTERNR3 ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_types.pyR& 6 s N(
t __doc__t pgen2R t R t
fixer_utilR R+ t _[1]t tR7 t BaseFixR& ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_types.pyt s6
% fixes/fix_set_literal.pyo 0000666 00000003702 15050104230 0011564 0 ustar 00
Lc @ sO d Z d d k l Z l Z d d k l Z l Z d e i f d YZ d S( s:
Optional fixer to transform set() calls to set literals.
i( t
fixer_baset pytree( t tokent symst
FixSetLiteralc B s e Z e Z d Z d Z RS( sj power< 'set' trailer< '('
(atom=atom< '[' (items=listmaker< any ((',' any)* [',']) >
|
single=any) ']' >
|
atom< '(' items=testlist_gexp< any ((',' any)* [',']) > ')' >
)
')' > >
c C s | i d } | o5 t i t i | i g } | i | | } n | d } t i t i d g } | i
d | i D | i t i t i
d | i i | d _ t i t i | } | i | _ t | i d j o. | i d } | i | i | i d _ n | S( Nt singlet itemsu {c s s x | ] } | i Vq Wd S( N( t clone( t .0t n( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pys & s u }ii i ( t getR t NodeR t listmakerR t replacet LeafR t LBRACEt extendt childrent appendt RBRACEt next_siblingt prefixt dictsetmakert lent remove( t selft nodet resultsR t fakeR t literalt makerR ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pyt transform s"
( t __name__t
__module__t Truet explicitt PATTERNR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pyR s
N( t __doc__t lib2to3R R t lib2to3.fixer_utilR R t BaseFixR ( ( ( s5 /usr/lib64/python2.6/lib2to3/fixes/fix_set_literal.pyt s fixes/fix_funcattrs.py 0000666 00000001153 15050104230 0011105 0 ustar 00 """Fix function attribute names (f.func_x -> f.__x__)."""
# Author: Collin Winter
# Local imports
from .. import fixer_base
from ..fixer_util import Name
class FixFuncattrs(fixer_base.BaseFix):
PATTERN = """
power< any+ trailer< '.' attr=('func_closure' | 'func_doc' | 'func_globals'
| 'func_name' | 'func_defaults' | 'func_code'
| 'func_dict') > any* >
"""
def transform(self, node, results):
attr = results["attr"][0]
attr.replace(Name((u"__%s__" % attr.value[5:]),
prefix=attr.prefix))
fixes/fix_paren.pyc 0000666 00000002762 15050104230 0010353 0 ustar 00
Lc @ sI d Z d d k l Z d d k l Z l Z d e i f d YZ d S( su Fixer that addes parentheses where they are required
This converts ``[x for x in 1, 2]`` to ``[x for x in (1, 2)]``.i ( t
fixer_base( t LParent RParent FixParenc B s e Z d Z d Z RS( s
atom< ('[' | '(')
(listmaker< any
comp_for<
'for' NAME 'in'
target=testlist_safe< any (',' any)+ [',']
>
[any]
>
>
|
testlist_gexp< any
comp_for<
'for' NAME 'in'
target=testlist_safe< any (',' any)+ [',']
>
[any]
>
>)
(']' | ')') >
c C sL | d } t } | i | _ d | _ | i d | | i t d S( Nt targetu i ( R t prefixt insert_childt append_childR ( t selft nodet resultsR t lparen( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_paren.pyt transform# s
( t __name__t
__module__t PATTERNR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_paren.pyR s N( t __doc__t R t
fixer_utilR R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_paren.pyt s fixes/fix_operator.pyc 0000666 00000003341 15050104230 0011073 0 ustar 00
Lc @ sO d Z d d k l Z d d k l Z l Z l Z d e i f d YZ d S( s Fixer for operator.{isCallable,sequenceIncludes}
operator.isCallable(obj) -> hasattr(obj, '__call__')
operator.sequenceIncludes(obj) -> operator.contains(obj)
i ( t
fixer_base( t Callt Namet Stringt FixOperatorc B s6 e Z d Z d Z d e d e d e Z d Z RS( s( method=('isCallable'|'sequenceIncludes')s '(' func=any ')'s
power< module='operator'
trailer< '.' %(methods)s > trailer< %(func)s > >
|
power< %(methods)s trailer< %(func)s > >
t methodst funcc C s | d d } | i d j o8 d | j o | i | d q d | _ | i n | i d j ow d | j o | i | d | d i q | d } | i t d
t d g } t t d | d
| i Sn d S( Nt methodi u sequenceIncludest modules& You should use operator.contains here.u containsu
isCallables, You should use hasattr(%s, '__call__') here.R u , u
'__call__'u hasattrt prefix( t valuet warningt changedt cloneR R R R ( t selft nodet resultsR R t args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_operator.pyt transform s
!( t __name__t
__module__R R t dictt PATTERNR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_operator.pyR s N( t __doc__t R t
fixer_utilR R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_operator.pyt s fixes/fix_throw.pyo 0000666 00000003720 15050104230 0010420 0 ustar 00
Lc @ s{ d Z d d k l Z d d k l Z d d k l Z d d k l Z l Z l Z l
Z
l Z d e i f d YZ
d S( s Fixer for generator.throw(E, V, T).
g.throw(E) -> g.throw(E)
g.throw(E, V) -> g.throw(E(V))
g.throw(E, V, T) -> g.throw(E(V).with_traceback(T))
g.throw("foo"[, V[, T]]) will warn about string exceptions.i ( t pytree( t token( t
fixer_base( t Namet Callt ArgListt Attrt is_tuplet FixThrowc B s e Z d Z d Z RS( s
power< any trailer< '.' 'throw' >
trailer< '(' args=arglist< exc=any ',' val=any [',' tb=any] > ')' >
>
|
power< any trailer< '.' 'throw' > trailer< '(' exc=any ')' > >
c
C s` | i } | d i } | i t i j o | i | d d S| i d } | d j o d S| i } t | o5 g } | i d d !D] } | | i q ~ } n d | _
| g } | d } d | j om | d i }
d |
_
t | | } t | t
d t |
g g } | i t i | i | n | i t | | d S(
Nt excs+ Python 3 does not support string exceptionsu vali iu t argst tbu with_traceback( t symst clonet typeR t STRINGt cannot_convertt gett NoneR t childrent prefixR R R R t replaceR t Nodet power(
t selft nodet resultsR R t valt _[1]t cR
t
throw_argsR t et with_tb( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_throw.pyt transform s*
5
% ( t __name__t
__module__t PATTERNR! ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_throw.pyR s N( t __doc__t R t pgen2R R t
fixer_utilR R R R R t BaseFixR ( ( ( s/ /usr/lib64/python2.6/lib2to3/fixes/fix_throw.pyt s
( fixes/fix_unicode.pyc 0000666 00000002222 15050104230 0010663 0 ustar 00
Lc @ sr d Z d d k Z d d k l Z d d k l Z h d d 6d d 6Z e i d
Z d e i f d YZ
d S(
sJ Fixer that changes unicode to str, unichr to chr, and u"..." into "...".
iNi ( t token( t
fixer_baseu chru unichru stru unicodeu [uU][rR]?[\'\"]t
FixUnicodec B s e Z d Z d Z RS( s STRING | 'unicode' | 'unichr'c C s | i t i j o! | i } t | i | _ | S| i t i j o8 t i | i o! | i } | i d | _ | Sn d S( Ni ( t typeR t NAMEt clonet _mappingt valuet STRINGt _literal_ret match( t selft nodet resultst new( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_unicode.pyt transform s ( t __name__t
__module__t PATTERNR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_unicode.pyR s ( t __doc__t ret pgen2R t R R t compileR t BaseFixR ( ( ( s1 /usr/lib64/python2.6/lib2to3/fixes/fix_unicode.pyt s fixes/fix_exitfunc.pyo 0000666 00000005216 15050104230 0011104 0 ustar 00
Lc @ sg d Z d d k l Z l Z d d k l Z l Z l Z l Z l Z l
Z
d e i f d YZ d S( s7
Convert use of sys.exitfunc to use the atexit module.
i( t pytreet
fixer_base( t Namet Attrt Callt Commat Newlinet symst FixExitfuncc B s) e Z d Z d Z d Z d Z RS( s
(
sys_import=import_name<'import'
('sys'
|
dotted_as_names< (any ',')* 'sys' (',' any)* >
)
>
|
expr_stmt<
power< 'sys' trailer< '.' 'exitfunc' > >
'=' func=any >
)
c G s t t | i | d S( N( t superR t __init__( t selft args( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyR
s c C s& t t | i | | d | _ d S( N( R R t
start_treet Nonet
sys_import( R t treet filename( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyR
s c C s d | j o& | i d j o | d | _ n d S| d i } d | _ t i t i t t d t d } t
| | g | i } | i | | i d j o | i | d d S| i i
d } | i t i j o* | i t | i t d d n | i i } | i
i | i } | i } t i t i t d t d d g }
t i t i |
g } | i | d t | i | d
| d S( NR t funcu u atexitu registersK Can't find sys import; Please add an atexit import at the top of your file.i u u importi ( R R t clonet prefixR t NodeR t powerR R R t replacet warningt childrent typet dotted_as_namest append_childR t parentt indext import_namet simple_stmtt insert_childR ( R t nodet resultsR t registert callt namest containing_stmtt positiont stmt_containert
new_importt new( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyt transform# s2
( t __name__t
__module__t PATTERNR
R
R, ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyR s N(
t __doc__t lib2to3R R t lib2to3.fixer_utilR R R R R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_exitfunc.pyt s . fixes/fix_ws_comma.pyo 0000666 00000002577 15050104230 0011073 0 ustar 00
Lc @ sS d Z d d k l Z d d k l Z d d k l Z d e i f d YZ d S( s Fixer that changes 'a ,b' into 'a, b'.
This also changes '{a :b}' into '{a: b}', but does not touch other
uses of colons. It does not touch other uses of whitespace.
i ( t pytree( t token( t
fixer_baset
FixWsCommac B sS e Z e Z d Z e i e i d Z e i e i d Z e e f Z
d Z RS( sH
any<(not(',') any)+ ',' ((not(',') any)+ ',')* [not(',') any]>
u ,u :c C s | i } t } x | i D]~ } | | i j o: | i } | i o d | j o
d | _ n t } q | o! | i } | p
d | _ q n t } q W| S( Nu
u u ( t clonet Falset childrent SEPSt prefixt isspacet True( t selft nodet resultst newt commat childR ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_ws_comma.pyt transform s
( t __name__t
__module__R
t explicitt PATTERNR t LeafR t COMMAt COLONR R ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_ws_comma.pyR s N( t __doc__t R t pgen2R R t BaseFixR ( ( ( s2 /usr/lib64/python2.6/lib2to3/fixes/fix_ws_comma.pyt s fixes/__init__.pyc 0000666 00000000205 15050104230 0010125 0 ustar 00
Lc @ s d S( N( ( ( ( s. /usr/lib64/python2.6/lib2to3/fixes/__init__.pyt s __init__.pyo 0000666 00000000177 15050104230 0007033 0 ustar 00
Lc @ s d S( N( ( ( ( s( /usr/lib64/python2.6/lib2to3/__init__.pyt s fixer_base.pyo 0000666 00000015737 15050104230 0007413 0 ustar 00
Lc @ s~ d Z d d k Z d d k Z d d k l Z d d k l Z d d k l Z d e f d YZ
d e
f d
YZ d S( s2 Base class for fixers (optional, but recommended).iNi ( t PatternCompiler( t pygram( t does_tree_importt BaseFixc B s e Z d Z d Z d Z d Z d Z d Z e i
d Z e Z
d Z e Z d Z d Z e i Z d Z d Z d Z d Z d Z d d
Z d Z d d Z d
Z d Z d Z RS( s Optional base class for fixers.
The subclass name must be FixFooBar where FooBar is the result of
removing underscores and capitalizing the words of the fix name.
For example, the class name for a fixer named 'has_key' should be
FixHasKey.
i t posti c C s | | _ | | _ | i d S( s Initializer. Subclass may override.
Args:
options: an dict containing the options passed to RefactoringTool
that could be used to customize the fixer through the command line.
log: a list to append warnings and other messages to.
N( t optionst logt compile_pattern( t selfR R ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt __init__* s c C s0 | i d j o t i | i | _ n d S( s Compiles self.PATTERN into self.pattern.
Subclass may override if it doesn't want to use
self.{pattern,PATTERN} in .match().
N( t PATTERNt NoneR R t pattern( R ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR 6 s c C s | | _ t i | | _ d S( sm Set the filename, and a logger derived from it.
The main refactoring tool should call this.
N( t filenamet loggingt getLoggert logger( R R
( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt set_filename? s c C s' h | d 6} | i i | | o | S( s Returns match for a given parse tree node.
Should return a true or false object (not necessarily a bool).
It may return a non-empty dict of matching sub-nodes as
returned by a matching pattern.
Subclass may override.
t node( R t match( R R t results( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR G s
c C s
t d S( s Returns the transformation for a given parse tree node.
Args:
node: the root of the parse tree that matched the fixer.
results: a dict mapping symbolic names to part of the match.
Returns:
None, or a node that is a modified copy of the
argument node. The node argument may also be modified in-place to
effect the same change.
Subclass *must* override.
N( t NotImplementedError( R R R ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt transformS s u xxx_todo_changemec C sK | } x. | | i j o | t | i i } q W| i i | | S( s Return a string suitable for use as an identifier
The new name is guaranteed not to conflict with other identifiers.
( t
used_namest unicodet numberst nextt add( R t templatet name( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt new_namec s c C sB | i o$ t | _ | i i d | i n | i i | d S( Ns ### In file %s ###( t first_logt FalseR t appendR
( R t message( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt log_messagen s
c C sZ | i } | i } d | _ d } | i | | | f | o | i | n d S( s Warn the user that a given chunk of code is not valid Python 3,
but that it cannot be converted automatically.
First argument is the top-level node for the code in question.
Optional second argument is why it can't be converted.
u s Line %d: could not convert: %sN( t
get_linenot clonet prefixR# ( R R t reasont linenot
for_outputt msg( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt cannot_convertt s c C s' | i } | i d | | f d S( s Used for warning the user about possible uncertainty in the
translation.
First argument is the top-level node for the code in question.
Optional second argument is why it can't be converted.
s Line %d: %sN( R$ R# ( R R R' R( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt warning s c C s8 | i | _ | i | t i d | _ t | _ d S( s Some fixers need to maintain tree-wide state.
This method is called once, at the start of tree fix-up.
tree - the root node of the tree to be processed.
filename - the name of the file the tree came from.
i N( R R t itertoolst countR t TrueR ( R t treeR
( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt
start_tree s
c C s d S( s Some fixers need to maintain tree-wide state.
This method is called once, at the conclusion of tree fix-up.
tree - the root node of the tree to be processed.
filename - the name of the file the tree came from.
N( ( R R0 R
( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt finish_tree s N(! t __name__t
__module__t __doc__R R
R R R
R R- R. R t setR t orderR t explicitt run_ordert _accept_typeR t python_symbolst symsR R R R R R R# R+ R, R1 R2 ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR s0
t ConditionalFixc B s& e Z d Z d Z d Z d Z RS( s@ Base class for fixers which not execute if an import is found. c G s# t t | i | d | _ d S( N( t superR= R1 R t _should_skip( R t args( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR1 s c C sc | i d j o | i S| i i d } | d } d i | d } t | | | | _ | i S( Nt .i( R? R t skip_ont splitt joinR ( R R t pkgR ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt should_skip s
N( R3 R4 R5 R RB R1 RF ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyR= s ( R5 R R- t patcompR t R t
fixer_utilR t objectR R= ( ( ( s* /usr/lib64/python2.6/lib2to3/fixer_base.pyt