PNG  IHDR;IDATxܻn0K )(pA 7LeG{ §㻢|ذaÆ 6lذaÆ 6lذaÆ 6lom$^yذag5bÆ 6lذaÆ 6lذa{ 6lذaÆ `}HFkm,mӪôô! x|'ܢ˟;E:9&ᶒ}{v]n&6 h_tڠ͵-ҫZ;Z$.Pkž)!o>}leQfJTu іچ\X=8Rن4`Vwl>nG^is"ms$ui?wbs[m6K4O.4%/bC%t Mז -lG6mrz2s%9s@-k9=)kB5\+͂Zsٲ Rn~GRC wIcIn7jJhۛNCS|j08yiHKֶۛkɈ+;SzL/F*\Ԕ#"5m2[S=gnaPeғL lذaÆ 6l^ḵaÆ 6lذaÆ 6lذa; _ذaÆ 6lذaÆ 6lذaÆ RIENDB`  oYm@sdZddlmZdZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z mZmZddlmZdd lmZmZdd lmZdd lmZd d dZGdddeZddZddZddZddZej ddfkrpddl!Z!e!j"Z#ddZ$ddZ%ne"Z#eZ$eZ%ddZ&Gd d!d!eZ'Gd"d#d#e(Z)Gd$d%d%eZ*Gd&d'd'e)Z+dS)(zRefactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. )with_statementz#Guido van Rossum N)chain)drivertokenizetoken) find_root)pytreepygram) btm_utils) btm_matcherTcCst|ggdg}tjj|j}g}xhttj|D]Q}|jdrI|jdrI|r|dd}|j |ddqIW|S)zEReturn a sorted list of all available fix names in the given package.*fix_z.pyN) __import__ospathdirname__file__sortedlistdir startswithendswithappend)Z fixer_pkgZ remove_prefixZpkgZ fixer_dirZ fix_namesnamer'/opt/python35/lib/python3.5/refactor.pyget_all_fix_names!src@seZdZdS) _EveryNodeN)__name__ __module__ __qualname__rrrrr .s r cCst|tjtjfr:|jdkr0t|jhSt|tjrh|jrbt|jStt|tj rt }x5|jD]*}x!|D]}|j t|qWqW|St d|dS)zf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s) isinstancer Z NodePatternZ LeafPatterntyper ZNegatedPatternZcontent_get_head_typesZWildcardPatternsetupdate Exception)patrpxrrrr&2s     r&c Cstjt}g}x|D]}|jryt|j}Wntk r_|j|YqXxU|D]}||j|qgWq|jdk r||jj|q|j|qWx:tt j j j t j j D]}||j|qWt|S)z^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N) collections defaultdictlistpatternr&r rZ _accept_typerr python_grammarZ symbol2numbervaluestokensextenddict)Z fixer_listZ head_nodesZeveryfixerZheadsZ node_typerrr_get_headnode_dictNs"    r8cs fddtdDS)zN Return the fully qualified names for fixers in the package pkg_name. csg|]}d|qS).r).0fix_name)pkg_namerr ks z+get_fixers_from_package..F)r)r<r)r<rget_fixers_from_packagegsr>cCs|S)Nr)objrrr _identitynsr@rcCs|jddS)Nz  )replace)inputrrr_from_system_newlinesusrDcCs*tjdkr"|jdtjS|SdS)NrA)rlineseprB)rCrrr_to_system_newlineswsrFc sd}tjtj|jfdd}ttjtjtj h}t }yVxO|\}}||krq`q`|tj kr|rPd}q`|tj kr|dkr|\}}|tj ks|dkrP|\}}|tj ks|dkrP|\}}|tj krJ|dkrJ|\}}x^|tj kr|j||\}}|tj ks|d krP|\}}qMWq`Pq`WWntk rYnXt|S) NFcst}|d|dfS)Nrr)next)tok)genrradvances z(_detect_future_features..advanceTfrom __future__import(,)rgenerate_tokensioStringIOreadline frozensetrNEWLINENLCOMMENTr'STRINGNAMEOPadd StopIteration)sourceZhave_docstringrJignorefeaturestpvaluer)rIr_detect_future_featuressD      rbc@seZdZdZdS) FixerErrorzA fixer could not be loaded.N)r!r"r#__doc__rrrrrcs rcc@sieZdZddddiZdZdZddddZd d Zd d Zd dZ ddZ ddZ ddddZ ddddZ ddZddddZddZdddZdd Zd!d"Zdddd#d$Zdd%d&Zd'Zd(Zd)d*Zd+d,Zd-d.Zd/d0Zd1d2Zd3d4ZdS)5RefactoringToolprint_functionFwrite_unchanged_filesZFixrNcCs||_|pg|_|jj|_|dk rF|jj||jdrbtj|_n tj |_|jj d|_ g|_ t jd|_g|_d|_tj|jdtjd|j|_|j\|_|_g|_tj|_g|_g|_xzt|j|jD]c}|j rQ|jj!|q/||jkrs|jj"|q/||jkr/|jj"|q/Wt#|j|_$t#|j|_%dS)zInitializer. Args: fixer_names: a list of fixers to import options: a dict with configuration. explicit: a list of fixers to run even if they are explicit. NrfrgreFconvertlogger)&fixersexplicit_default_optionscopyoptionsr(r !python_grammar_no_print_statementgrammarr2getrgerrorslogging getLoggerri fixer_logwroterZDriverr rh get_fixers pre_order post_orderfilesbmZ BottomMatcherBMZ bmi_pre_orderZbmi_post_orderrZ BM_compatibleZ add_fixerrr8bmi_pre_order_headsbmi_post_order_heads)selfZ fixer_namesrnrkr7rrr__init__s<            zRefactoringTool.__init__c Csg}g}x|jD]}t|iidg}|jddd}|j|jru|t|jd}|jd}|jdjdd|D}yt ||}Wn(t k rt d ||fYnX||j |j } | jr4|jd k r4||jkr4|jd |q|jd || jd krc|j| q| jdkr|j| qt d| jqWtjd} |jd| |jd| ||fS)aInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. r r9rN_cSsg|]}|jqSr)title)r:r,rrrr=s z.RefactoringTool.get_fixers..zCan't find %s.%sTzSkipping optional fixer: %szAdding transformation: %spreZpostzIllegal fixer order: %rZ run_orderkey)rjrrsplitr FILE_PREFIXlensplit CLASS_PREFIXjoingetattrAttributeErrorrcrnrurk log_message log_debugorderroperator attrgettersort) rZpre_order_fixersZpost_order_fixersZ fix_mod_pathmodr;parts class_nameZ fix_classr7Zkey_funcrrrrws8# zRefactoringTool.get_fixerscOsdS)zCalled when an error occurs.Nr)rmsgargskwdsrrr log_errorszRefactoringTool.log_errorcGs$|r||}|jj|dS)zHook to log a message.N)riinfo)rrrrrrrs zRefactoringTool.log_messagecGs$|r||}|jj|dS)N)ridebug)rrrrrrrs zRefactoringTool.log_debugcCsdS)zTCalled with the old version, new version, and filename of a refactored file.Nr)rold_textnew_textfilenameequalrrr print_output!szRefactoringTool.print_outputcCsPxI|D]A}tjj|r5|j|||q|j|||qWdS)z)Refactor a list of files and directories.N)rrisdir refactor_dir refactor_file)ritemswrite doctests_onlyZ dir_or_filerrrrefactor&s zRefactoringTool.refactorc Cstjd}xtj|D]\}}}|jd||j|jxb|D]Z}|jd rWtjj|d|krWtjj||} |j | ||qWWdd|D|dd.N) rextsepwalkrrrrsplitextrr) rZdir_namerrZpy_extdirpathdirnames filenamesrfullnamerrrr/s    zRefactoringTool.refactor_dircCsyt|d}Wn<tk rQ}z|jd||dSWYdd}~XnXztj|jd}Wd|jXt|dd|}t|j |fSWdQRXdS)zG Do our best to decode a Python source file correctly. rbzCan't open %s: %sNrr+encoding)NN) openOSErrorrrdetect_encodingrSclose_open_with_encodingrDread)rrferrrrrr_read_python_sourceCs z#RefactoringTool._read_python_sourcecCs|j|\}}|dkr%dS|d7}|r|jd||j||}|jsl||kr|j|||||q|jd|ni|j||}|js|r|jr|jt|dd |d|d|n|jd|dS) zRefactors a file.NrAzRefactoring doctests in %szNo doctest changes in %srrrzNo changes in %sr)rrrefactor_docstringrgprocessed_filerefactor_string was_changedstr)rrrrrCroutputtreerrrrSs  zRefactoringTool.refactor_filecCst|}d|kr'tj|j_zby|jj|}WnEtk r}z%|jd||jj |dSWYdd}~XnXWd|j|j_X||_ |j d||j |||S)aFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. rfzCan't parse %s: %s: %sNzRefactoring %s) rbr rorrpZ parse_stringr)r __class__r!future_featuresr refactor_tree)rdatarr_rrrrrrjs    zRefactoringTool.refactor_stringcCstjj}|ro|jd|j|d}|jsI||kr_|j|d|q|jdnS|j|d}|js|r|jr|jt |d|n |jddS)NzRefactoring doctests in stdinzzNo doctest changes in stdinzNo changes in stdin) sysstdinrrrrgrrrr)rrrCrrrrrrefactor_stdins zRefactoringTool.refactor_stdinc Csx-t|j|jD]}|j||qW|j|j|j|j|j|j|jj|j }xt |j rLx|jj D]}||kr||r||j dtjjdd|jr||j dtjjxGt||D]5}|||kr3||j|yt|Wntk rXw YnX|jrt||jkrtq |j|}|r |j||}|dk r |j|x6|jD](}|jsg|_|jj|qW|jj|j }x9|D]1} | |kr(g|| <|| j|| q Wq WqWq}Wx-t|j|jD]}|j||qcW|jS)aRefactors a parse tree (modifying the tree in place). For compatible patterns the bottom matcher module is used. Otherwise the tree is traversed node-to-node for matches. Args: tree: a pytree.Node instance representing the root of the tree to be refactored. name: a human-readable name for this tree. Returns: True if the tree was modified, False otherwise. rreverseTN)rrxryZ start_tree traverse_byr}r~r|runZleavesanyr3rjrr ZBasedepthZkeep_line_orderZ get_linenor0remover ValueErrorZfixers_appliedmatch transformrBrr5Z finish_treer) rrrr7Z match_setnoderesultsnewZ new_matchesZfxrrrrrsJ          %zRefactoringTool.refactor_treecCs}|s dSxl|D]d}x[||jD]L}|j|}|r%|j||}|dk r%|j||}q%WqWdS)aTraverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N)r%rrrB)rrjZ traversalrr7rrrrrrs    zRefactoringTool.traverse_bycCs|jj||dkr?|j|d}|dkr?dS||k}|j|||||r|jd||jsdS|r|j||||n|jd|dS)zR Called when a file has been refactored and there may be changes. NrzNo changes to %szNot writing changes to %s)rzrrrrrg write_file)rrrrrrrrrrrs    zRefactoringTool.processed_filec%Csyt|dd|}Wn<tk rW}z|jd||dSWYdd}~XnXzVy|jt|Wn8tk r}z|jd||WYdd}~XnXWd|jX|jd|d|_dS)zWrites a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. wrzCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)rrrrrFrrrv)rrrrrrrrrrrs* zRefactoringTool.write_filez>>> z... c Csg}d}d}d}d}x.|jddD]}|d7}|jj|jr|dk r|j|j|||||}|g}|j|j} |d| }q1|dk r|j||js|||jjdkr|j |q1|dk r2|j|j||||d}d}|j |q1W|dk rz|j|j||||dj |S)aRefactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) NrkeependsTrrAr) splitlineslstriprPS1r5refactor_doctestfindPS2rstriprr) rrCrresultblockZ block_linenoindentlinenolineirrrr(s:          z"RefactoringTool.refactor_docstringc syj||}Wntk r}zgjjtjrmx'|D]}jd|jdqJWjd|||j j ||SWYdd}~XnXj ||r}t |j dd}|d|d||dd} }| dg|dkst| |d jds:|d d7<j|jdg}|r}|fd d |D7}|S) zRefactors one doctest. A doctest is given as a block of lines, the first of which starts with ">>>" (possibly indented), while the remaining lines start with "..." (identically indented). z Source: %srAz+Can't parse docstring in %s line %s: %s: %sNrTrrcs!g|]}j|qSr)r)r:r)rrrrr=ms z4RefactoringTool.refactor_doctest..rr) parse_blockr)ri isEnabledForrsDEBUGrrrrr!rrrAssertionErrorrrpop) rrrrrrrrrZclippedr)rrrrSs&  )# z RefactoringTool.refactor_doctestcCs|jrd}nd}|js4|jd|n1|jd|x|jD]}|j|qNW|jr|jdx|jD]}|j|qW|jrt|jdkr|jdn|jdt|jx-|jD]"\}}}|j|||qWdS) Nwerez need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:rzThere was 1 error:zThere were %d errors:)rvrzrrurrr)rrfilemessagerrrrrr summarizeps$      zRefactoringTool.summarizecCs1|jj|j|||}t|_|S)zParses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. )rZ parse_tokens wrap_toksrTr)rrrrrrrrrs! zRefactoringTool.parse_blockc cstj|j||j}xe|D]]\}}\}}\} } } ||d7}| |d7} ||||f| | f| fVq%WdS)z;Wraps a tokenize stream to systematically modify start/end.rN)rrP gen_lines__next__) rrrrr4r%raZline0Zcol0Zline1Zcol1Z line_textrrrrs (zRefactoringTool.wrap_toksccs||j}||j}|}xo|D]g}|j|rT|t|dVn4||jdkrrdVntd||f|}q'Wx dVqWdS)zGenerates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. NrAzline=%r, prefix=%rr)rrrrrr)rrrZprefix1Zprefix2prefixrrrrrs    zRefactoringTool.gen_lines)r!r"r#rlrrrrwrrrrrrrrrrrrrrrrrrrrrrrrrrres:  4 (        O  +   rec@seZdZdS)MultiprocessingUnsupportedN)r!r"r#rrrrrs rcsaeZdZfddZdddfddZfddZfd d ZS) MultiprocessRefactoringToolcs/tt|j||d|_d|_dS)N)superrrqueue output_lock)rrkwargs)rrrrs z$MultiprocessRefactoringTool.__init__FrcsU|dkr(ttj|||SyddlWntk rRtYnXjdk rntdj_j _ fddt |D}z;x|D]}|j qWttj|||Wdjj x$t |D]}jjdqWx$|D]}|jr'|j q'Wd_XdS)Nrrz already doing multiple processescs%g|]}jdjqS)target)Process_child)r:r)multiprocessingrrrr=s z8MultiprocessRefactoringTool.refactor..)rrrr ImportErrorrr RuntimeError JoinableQueueLockrrangestartrputis_alive)rrrrZ num_processes processesr,r)r)rrrrs2          z$MultiprocessRefactoringTool.refactorc so|jj}xY|dk rj|\}}ztt|j||Wd|jjX|jj}qWdS)N)rrqrrr task_done)rZtaskrr)rrrrs z"MultiprocessRefactoringTool._childcsE|jdk r(|jj||fntt|j||SdS)N)rrrrr)rrr)rrrrsz)MultiprocessRefactoringTool.refactor_file)r!r"r#rrrrrr)rrrs  r),rdrLr __author__rrrsrr.rQ itertoolsrZpgen2rrrZ fixer_utilrrr r r Zbur r{rr)r r&r8r>r@ version_infocodecsrrrDrFrbrcobjectrerrrrrr sF               (