Richard Shea avatar Richard Shea committed 0d732e9 Draft

Some pyLint driven tidy ups of the main code as well as some changes to pavement.py / pyLintRunner.bat to suppress the more annoying pyLint messages

Comments (0)

Files changed (11)

Add a comment to this file

dist/smtpErrorAnalysis-0.1.zip

Binary file modified.

Add a comment to this file

doc/_build/doctrees/environment.pickle

Binary file modified.

Add a comment to this file

doc/_build/doctrees/findBadAddresses.doctree

Binary file modified.

doc/_build/html/findBadAddresses.html

                         PATH to output csv file
   -v, --verbose         Show each file processed</pre>
 </div>
+<dl class="class">
+<dt id="findBadAddresses.EmailType">
+<em class="property">class </em><tt class="descclassname">findBadAddresses.</tt><tt class="descname">EmailType</tt><a class="headerlink" href="#findBadAddresses.EmailType" title="Permalink to this definition">¶</a></dt>
+<dd><p>An enumerated type to allow easy description
+of which type of email we&#8217;re processing</p>
+<dl class="attribute">
+<dt id="findBadAddresses.EmailType.STYLE1">
+<tt class="descname">STYLE1</tt><em class="property"> = 0</em><a class="headerlink" href="#findBadAddresses.EmailType.STYLE1" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="findBadAddresses.EmailType.STYLE2">
+<tt class="descname">STYLE2</tt><em class="property"> = 1</em><a class="headerlink" href="#findBadAddresses.EmailType.STYLE2" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="findBadAddresses.EmailType.UNKNOWN">
+<tt class="descname">UNKNOWN</tt><em class="property"> = 2</em><a class="headerlink" href="#findBadAddresses.EmailType.UNKNOWN" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
 <dl class="exception">
 <dt id="findBadAddresses.FindBadAddExcptn">
 <em class="property">exception </em><tt class="descclassname">findBadAddresses.</tt><tt class="descname">FindBadAddExcptn</tt><big>(</big><em>value</em><big>)</big><a class="reference internal" href="_modules/findBadAddresses.html#FindBadAddExcptn"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#findBadAddresses.FindBadAddExcptn" title="Permalink to this definition">¶</a></dt>
 <p>Base class for errors in this script.</p>
 </dd></dl>
 
+<dl class="class">
+<dt id="findBadAddresses.HeaderValue">
+<em class="property">class </em><tt class="descclassname">findBadAddresses.</tt><tt class="descname">HeaderValue</tt><big>(</big><em>lst</em><big>)</big><a class="headerlink" href="#findBadAddresses.HeaderValue" title="Permalink to this definition">¶</a></dt>
+<dd><p>Bases: <tt class="xref py py-class docutils literal"><span class="pre">object</span></tt></p>
+<p>Represents a &#8216;header/value&#8217; pair in an
+email etc</p>
+</dd></dl>
+
 <dl class="function">
 <dt id="findBadAddresses.build_ignore_list">
 <tt class="descclassname">findBadAddresses.</tt><tt class="descname">build_ignore_list</tt><big>(</big><big>)</big><a class="reference internal" href="_modules/findBadAddresses.html#build_ignore_list"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#findBadAddresses.build_ignore_list" title="Permalink to this definition">¶</a></dt>
 </dd></dl>
 
 <dl class="function">
+<dt id="findBadAddresses.convert_gen_to_list">
+<tt class="descclassname">findBadAddresses.</tt><tt class="descname">convert_gen_to_list</tt><big>(</big><em>gen</em><big>)</big><a class="headerlink" href="#findBadAddresses.convert_gen_to_list" title="Permalink to this definition">¶</a></dt>
+<dd><p>Converts a generator to a list</p>
+</dd></dl>
+
+<dl class="function">
 <dt id="findBadAddresses.find_email">
 <tt class="descclassname">findBadAddresses.</tt><tt class="descname">find_email</tt><big>(</big><em>instr</em><big>)</big><a class="reference internal" href="_modules/findBadAddresses.html#find_email"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#findBadAddresses.find_email" title="Permalink to this definition">¶</a></dt>
 <dd><p>Given a string searches for all email addresses contained
 </dd></dl>
 
 <dl class="function">
-<dt id="findBadAddresses.parse_email_for_del_stat_part">
-<tt class="descclassname">findBadAddresses.</tt><tt class="descname">parse_email_for_del_stat_part</tt><big>(</big><em>file_name</em>, <em>path_em_file</em>, <em>csv_dict_wrtr</em>, <em>options</em><big>)</big><a class="reference internal" href="_modules/findBadAddresses.html#parse_email_for_del_stat_part"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#findBadAddresses.parse_email_for_del_stat_part" title="Permalink to this definition">¶</a></dt>
+<dt id="findBadAddresses.parse_email_for_dlv_stat_info">
+<tt class="descclassname">findBadAddresses.</tt><tt class="descname">parse_email_for_dlv_stat_info</tt><big>(</big><em>file_name</em>, <em>path_em_file</em>, <em>csv_dict_wrtr</em>, <em>options</em><big>)</big><a class="headerlink" href="#findBadAddresses.parse_email_for_dlv_stat_info" title="Permalink to this definition">¶</a></dt>
 <dd><p>Given the text of a SMTP &#8216;bounce message&#8217; writes a CSV row 
 to match the headers in the global variable HDR_OUTPUT_COLS.</p>
-<p>It does this by finding the &#8216;message/delivery-status&#8217; part of 
+<p>There are two styles of &#8216;bounce message&#8217; it does this for.</p>
+<p>STYLE 1</p>
+<p>&#8216;STYLE1&#8217; is a multi-part mime message.</p>
+<p>Parsing is done by finding the &#8216;message/delivery-status&#8217; part of 
 the entire email and parsing the headers.</p>
 <p>An &#8216;message/delivery-status&#8217; part of a &#8216;bounce email&#8217; looks a 
 little like this</p>
 Remote-MTA: dns; smtp.e.web
 Diagnostic-Code: smtp; 550 &lt;john.smith@e.web&gt;, Recipient unknown</pre>
 </div>
+<p>STYLE 2</p>
+<p>&#8216;STYLE2&#8217; is not multi-part mime message. It consists of :</p>
+<blockquote>
+<div><ul class="simple">
+<li>Email headers</li>
+<li>Text detailing nature of the problem encountered</li>
+<li>A copy of the original email message as sent</li>
+</ul>
+</div></blockquote>
+<p>It&#8217;s not possible to obtain such rich information from &#8216;STYLE 2&#8217; emails
+as it is from &#8216;STYLE 1&#8217; as it&#8217;s a significantly less informative email
+nevertheless it is possible to extract a useful subset of the information
+derived from &#8216;STYLE 1&#8217; emails.</p>
+<p>The earlier part of such a message looks a little like this</p>
+<div class="highlight-python"><pre>Return-Path: &lt;&gt;
+X-Original-To: someone@c.d.web
+Delivered-To: someone@c.d.web
+Received: from alpha.g.h.web (beta.g.h.web [101.101.101.101])
+        by c.d.web (Postfix) with ESMTP id 010101010
+        for &lt;someone@c.d.web&gt;; Tue,  8 May 2012 21:00:38 -0700 (PDT)
+Received: from unknown (HELO alpha.i.j.web) ([102.102.102.102])
+  by gamma.i.j.web with ESMTP; 09 May 2012 16:00:24 +1200
+Received: from mailuser by alpha.i.j.web with LOCAL (Exim 4.69)
+        id 9ABc2f-111111-AA
+        for someone@c.d.web; Wed, 09 May 2012 16:00:24 +1200
+X-Failed-Recipients: John.Smith@e.web
+Auto-Submitted: auto-replied
+From: Mail Delivery System &lt;Mailer-Daemon@i.j.web&gt;
+To: someone@c.d.web
+Subject: Mail delivery failed: returning message to sender
+Message-Id: &lt;E1SRy4e-11111b-DW@alpha.i.j.web&gt;
+Date: Wed, 09 May 2012 16:00:24 +1200
+
+This message was created automatically by mail delivery software.
+
+A message that you sent could not be delivered to one or more of its
+recipients. This is a permanent error. The following address(es) failed:
+
+  save to inbox
+    generated by John.Smith@e.web
+    mailbox is full: retry timeout exceeded
+
+------ This is a copy of the message, including all the headers. ------</pre>
+</div>
 <p>NB: All sorts of assumptions are made about the structure of the 
 bounce message which seem to hold true for a large sample I have 
 used in testing but it seems likely that somewhere there are &#8216;bounce
 </dd></dl>
 
 <dl class="function">
+<dt id="findBadAddresses.populate_header_val_dict">
+<tt class="descclassname">findBadAddresses.</tt><tt class="descname">populate_header_val_dict</tt><big>(</big><em>em_style</em>, <em>lst_hv</em><big>)</big><a class="headerlink" href="#findBadAddresses.populate_header_val_dict" title="Permalink to this definition">¶</a></dt>
+<dd><p>Populate dictionary from a list of HeaderValue objects. Using 
+the &#8216;header_name&#8217; property of each instance as the dictionary
+key and the &#8216;value&#8217; property as the value.</p>
+<p>In the case of STYLE2 emails ignore some of <cite>lst_hv</cite> and modify
+the &#8216;header_name&#8217; property values for those we do process</p>
+<p>Throw exception if the input list would generate the same key
+in the output dictionary twice</p>
+</dd></dl>
+
+<dl class="function">
+<dt id="findBadAddresses.pre_process_headers_for_style2">
+<tt class="descclassname">findBadAddresses.</tt><tt class="descname">pre_process_headers_for_style2</tt><big>(</big><em>lst_hv</em><big>)</big><a class="headerlink" href="#findBadAddresses.pre_process_headers_for_style2" title="Permalink to this definition">¶</a></dt>
+<dd><p>Takes a list of HeaderValue objects, <cite>lst_hv</cite> 
+and creates another list of HeaderValue objects 
+with modified membership set and content.</p>
+<p>In summary we&#8217;re dropping some elements of the
+input list and for those we don&#8217;t drop we&#8217;re 
+modifying the &#8216;header_name&#8217; property of the resulting
+HeaderValue instance.</p>
+<p>Each element of the input, <cite>lst_hv</cite> is tested 
+to see if it &#8216;header_name&#8217; value exists as a key of 
+the constant dictionary <cite>STYLE2_MAPPINGS</cite> if it does then
+a modified form of that HeaderValue object is added to the 
+output list of HeaderValue objects.</p>
+<p>The output HeaderValue object has a &#8216;header_name&#8217; property 
+corresponding to the value of the found element in <cite>STYLE2_MAPPINGS</cite>
+and a &#8216;value&#8217; property corresponding to the &#8216;value&#8217; element
+of the input HeaderValue object.</p>
+</dd></dl>
+
+<dl class="function">
+<dt id="findBadAddresses.pre_process_key_values">
+<tt class="descclassname">findBadAddresses.</tt><tt class="descname">pre_process_key_values</tt><big>(</big><em>lst_kv</em><big>)</big><a class="headerlink" href="#findBadAddresses.pre_process_key_values" title="Permalink to this definition">¶</a></dt>
+<dd><p>Takes a list of tuples, <cite>lst_kv</cite>, and:</p>
+<ul class="simple">
+<li>Uppercases the first element of each tuple</li>
+<li>Strips LF/CR from second element of each tuple</li>
+</ul>
+</dd></dl>
+
+<dl class="function">
+<dt id="findBadAddresses.process_email_and_write_row">
+<tt class="descclassname">findBadAddresses.</tt><tt class="descname">process_email_and_write_row</tt><big>(</big><em>em_style</em>, <em>lst_hdrs_kv_pairs</em>, <em>file_name</em>, <em>csv_dict_wrtr</em><big>)</big><a class="headerlink" href="#findBadAddresses.process_email_and_write_row" title="Permalink to this definition">¶</a></dt>
+<dd><p>Takes a list of tuples and extracts elements of interest
+for writing as a row to a CSV file</p>
+</dd></dl>
+
+<dl class="function">
 <dt id="findBadAddresses.remove_rfc_notation">
 <tt class="descclassname">findBadAddresses.</tt><tt class="descname">remove_rfc_notation</tt><big>(</big><em>email_to_be_cleaned</em><big>)</big><a class="reference internal" href="_modules/findBadAddresses.html#remove_rfc_notation"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#findBadAddresses.remove_rfc_notation" title="Permalink to this definition">¶</a></dt>
 <dd><p>Given a string which contains an email address in oe of the two 

doc/_build/html/genindex.html

 
 <div class="genindex-jumpbox">
  <a href="#B"><strong>B</strong></a>
+ | <a href="#C"><strong>C</strong></a>
+ | <a href="#E"><strong>E</strong></a>
  | <a href="#F"><strong>F</strong></a>
+ | <a href="#H"><strong>H</strong></a>
  | <a href="#M"><strong>M</strong></a>
  | <a href="#P"><strong>P</strong></a>
  | <a href="#R"><strong>R</strong></a>
  | <a href="#S"><strong>S</strong></a>
+ | <a href="#U"><strong>U</strong></a>
  
 </div>
 <h2 id="B">B</h2>
   </dl></td>
 </tr></table>
 
+<h2 id="C">C</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%" valign="top"><dl>
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.convert_gen_to_list">convert_gen_to_list() (in module findBadAddresses)</a>
+  </dt>
+
+  </dl></td>
+</tr></table>
+
+<h2 id="E">E</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%" valign="top"><dl>
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.EmailType">EmailType (class in findBadAddresses)</a>
+  </dt>
+
+  </dl></td>
+</tr></table>
+
 <h2 id="F">F</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%" valign="top"><dl>
   </dl></td>
 </tr></table>
 
+<h2 id="H">H</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%" valign="top"><dl>
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.HeaderValue">HeaderValue (class in findBadAddresses)</a>
+  </dt>
+
+  </dl></td>
+</tr></table>
+
 <h2 id="M">M</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%" valign="top"><dl>
   <dt><a href="findBadAddresses.html#findBadAddresses.parse_args">parse_args() (in module findBadAddresses)</a>
   </dt>
 
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.parse_email_for_dlv_stat_info">parse_email_for_dlv_stat_info() (in module findBadAddresses)</a>
+  </dt>
+
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.populate_header_val_dict">populate_header_val_dict() (in module findBadAddresses)</a>
+  </dt>
+
   </dl></td>
   <td style="width: 33%" valign="top"><dl>
       
-  <dt><a href="findBadAddresses.html#findBadAddresses.parse_email_for_del_stat_part">parse_email_for_del_stat_part() (in module findBadAddresses)</a>
+  <dt><a href="findBadAddresses.html#findBadAddresses.pre_process_headers_for_style2">pre_process_headers_for_style2() (in module findBadAddresses)</a>
+  </dt>
+
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.pre_process_key_values">pre_process_key_values() (in module findBadAddresses)</a>
+  </dt>
+
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.process_email_and_write_row">process_email_and_write_row() (in module findBadAddresses)</a>
   </dt>
 
   </dl></td>
   <dt><a href="findBadAddresses.html#findBadAddresses.strip_line_feeds">strip_line_feeds() (in module findBadAddresses)</a>
   </dt>
 
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.EmailType.STYLE1">STYLE1 (findBadAddresses.EmailType attribute)</a>
+  </dt>
+
+  </dl></td>
+  <td style="width: 33%" valign="top"><dl>
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.EmailType.STYLE2">STYLE2 (findBadAddresses.EmailType attribute)</a>
+  </dt>
+
+  </dl></td>
+</tr></table>
+
+<h2 id="U">U</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%" valign="top"><dl>
+      
+  <dt><a href="findBadAddresses.html#findBadAddresses.EmailType.UNKNOWN">UNKNOWN (findBadAddresses.EmailType attribute)</a>
+  </dt>
+
   </dl></td>
 </tr></table>
 
Add a comment to this file

doc/_build/html/objects.inv

Binary file modified.

doc/_build/html/searchindex.js

-Search.setIndex({objects:{"":{findBadAddresses:[1,0,1,""],regexEmailTester:[2,0,1,""]},findBadAddresses:{strip_line_feeds:[1,1,1,""],FindBadAddExcptn:[1,2,1,""],parse_args:[1,1,1,""],build_ignore_list:[1,1,1,""],remove_rfc_notation:[1,1,1,""],find_email:[1,1,1,""],parse_email_for_del_stat_part:[1,1,1,""],main:[1,1,1,""]},regexEmailTester:{main:[2,1,1,""]}},terms:{recipi:1,all:1,code:1,help:1,show:1,text:1,global:1,find_email:1,follow:1,find:1,row:1,web:1,locat:1,"808f17f8080":1,smith:1,except:1,param:[],should:1,other:1,suspect:1,subsequ:1,analys:1,bounc:1,match:1,them:1,sourc:[1,2],"return":1,string:1,format:1,bounceback:1,ident:1,report:1,bar:1,path_em_fil:1,findbadaddexcptn:1,somewher:1,like:1,specif:[],list:1,chuck:2,server:1,rfc:1,regexemailtest:[0,2],remove_rfc_not:1,contain:1,output:1,where:1,page:0,set:1,outpath:1,hard:1,sampl:1,multipart:1,fail:1,variabl:1,index:0,statu:1,someth:1,content:[0,1],written:1,larg:1,parse_email_for_del_stat_part:1,current:1,foo:1,email:1,file_nam:1,assumpt:1,who:1,each:1,usag:1,given:1,were:1,base:1,crlf:1,dictionari:1,found:1,path:1,tue:1,valu:1,search:[0,1],sender:1,pdt:1,queue:1,entir:1,place:1,action:1,onto:2,imposs:1,origin:1,via:[],instr:1,appli:1,modul:[0,1,2],within:1,arriv:1,two:1,header:1,inbox:1,owner:1,assum:1,differ:1,convent:1,script:1,unknown:1,support:1,findbadaddress:[0,1],due:1,messag:1,name:1,verbos:1,john:1,type:1,"final":1,deliveri:1,main:[1,2],option:1,tupl:[],optionpars:1,strip_line_fe:1,about:1,specifi:1,argument:1,sort:1,part:1,pars:1,particular:1,line:1,hold:1,"true":1,than:1,those:1,account:1,made:1,input:1,csv:1,remot:1,might:1,remov:1,work:[],focu:1,structur:1,charact:1,postfix:1,"while":2,respres:[],sent:1,error:[0,1],problem:1,mta:1,document:0,address:1,"function":[1,2],have:1,csv_dict_wrtr:1,look:1,process:1,deliv:1,smtp:[0,1],build_ignore_list:1,indic:0,repres:1,diagnost:1,exit:1,file:[1,2],tabl:0,need:2,seem:1,date:1,welcom:0,want:2,detail:1,invalid:1,write:1,valid:1,which:[1,2],test:1,ignor:1,eventu:2,email_to_be_clean:1,analysi:0,hdr_output_col:1,allow:1,rfc822:1,object:1,hang:2,someon:1,rais:1,consid:1,mai:1,littl:1,"class":1,least:1,parse_arg:1,directori:1,descript:1,rule:1,doe:1,command:1,thi:[1,2],order:[],left:1},objtypes:{"0":"py:module","1":"py:function","2":"py:exception"},titles:["Welcome to &#8216;smtp-error-analysis&#8217;&#8217;s documentation!","findBadAddresses Module","regexEmailTester Module"],objnames:{"0":["py","module","Python module"],"1":["py","function","Python function"],"2":["py","exception","Python exception"]},filenames:["index","findBadAddresses","regexEmailTester"]})
+Search.setIndex({objects:{"":{regexEmailTester:[2,0,1,""],findBadAddresses:[1,0,1,""]},findBadAddresses:{strip_line_feeds:[1,1,1,""],FindBadAddExcptn:[1,4,1,""],pre_process_key_values:[1,1,1,""],HeaderValue:[1,2,1,""],EmailType:[1,2,1,""],parse_email_for_dlv_stat_info:[1,1,1,""],populate_header_val_dict:[1,1,1,""],process_email_and_write_row:[1,1,1,""],build_ignore_list:[1,1,1,""],pre_process_headers_for_style2:[1,1,1,""],convert_gen_to_list:[1,1,1,""],find_email:[1,1,1,""],remove_rfc_notation:[1,1,1,""],main:[1,1,1,""],parse_args:[1,1,1,""]},regexEmailTester:{main:[2,1,1,""]},"findBadAddresses.EmailType":{STYLE2:[1,3,1,""],STYLE1:[1,3,1,""],UNKNOWN:[1,3,1,""]}},terms:{recipi:1,style2:1,all:1,code:1,help:1,show:1,text:1,global:1,style1:1,least:1,detail:1,lst:1,optionpars:1,invalid:1,find_email:1,left:1,follow:1,helo:1,find:1,row:1,web:1,style:1,locat:1,easi:1,wed:1,"808f17f8080":1,smith:1,except:1,param:[],should:1,em_styl:1,lst_hv:1,rich:1,larg:1,subsequ:1,analys:1,bounc:1,match:1,them:1,sourc:[1,2],"return":1,string:1,format:1,auto:1,bounceback:1,account:1,obtain:1,ident:1,automat:1,base:1,report:1,bar:1,headervalu:1,gen:1,path_em_fil:1,postfix:1,daemon:1,findbadaddexcptn:1,somewher:1,like:1,specif:[],esmtp:1,who:1,drop:1,list:1,chuck:2,server:1,rfc:1,regexemailtest:[0,2],remove_rfc_not:1,timeout:1,each:1,found:1,email_to_be_clean:1,where:1,page:0,summari:1,subset:1,encount:1,exceed:1,set:1,outpath:1,natur:1,parse_email_for_dlv_stat_info:1,hard:1,some:1,pre_process_key_valu:1,second:1,sampl:1,multipart:1,fail:1,extract:1,subject:1,variabl:1,index:0,statu:1,twice:1,deliveri:1,repli:1,someth:1,enumer:1,exim:1,content:[0,1],written:1,retri:1,pre_process_headers_for_style2:1,parse_email_for_del_stat_part:[],foo:1,email:1,file_nam:1,assumpt:1,full:1,deriv:1,kei:1,gener:1,contain:1,usag:1,given:1,mailbox:1,particular:1,were:1,entir:1,mime:1,crlf:1,dictionari:1,output:1,might:1,path:1,modifi:1,tue:1,valu:1,popul:1,convert:1,header_nam:1,sender:1,valid:1,could:1,current:1,beta:1,queue:1,etc:1,instanc:1,place:1,perman:1,action:1,onto:2,imposs:1,gamma:1,emailtyp:1,origin:1,softwar:1,suspect:1,via:[],instr:1,appli:1,modul:[0,1,2],within:1,arriv:1,process_email_and_write_row:1,due:1,header:1,done:1,inbox:1,strip:1,owner:1,assum:1,"9abc2f":1,save:1,differ:1,doe:1,from:1,csv_dict_wrtr:1,convent:1,script:1,would:1,unknown:1,support:1,findbadaddress:[0,1],system:1,submit:1,name:1,verbos:1,csv:1,includ:1,interest:1,messag:1,lst_hdrs_kv_pair:1,john:1,type:1,"final":1,more:1,analysi:0,main:[1,2],mailer:1,option:1,form:1,tupl:1,copi:1,strip_line_fe:1,search:[0,1],specifi:1,argument:1,sort:1,part:1,pars:1,indic:0,mail:1,"throw":1,line:1,hold:1,"true":1,than:1,those:1,"case":1,multi:1,made:1,input:1,consist:1,possibl:1,remot:1,local:1,remov:1,properti:1,focu:1,structur:1,charact:1,directori:1,"while":2,mailus:1,sent:1,error:[0,1],about:1,problem:1,mta:1,document:0,address:1,significantli:1,"function":[1,2],pdt:1,constant:1,have:1,creat:1,look:1,process:1,deliv:1,smtp:[0,1],build_ignore_list:1,convert_gen_to_list:1,repres:1,diagnost:1,exit:1,exist:1,rule:1,file:[1,2],tabl:0,need:2,headernam:[],seem:1,work:[],result:1,"11111b":1,date:1,welcom:0,want:2,receiv:1,anoth:1,populate_header_val_dict:1,same:1,write:1,other:1,pair:1,take:1,which:[1,2],test:1,ignor:1,you:1,eventu:2,uppercas:1,respres:[],hdr_output_col:1,allow:1,see:1,rfc822:1,less:1,earlier:1,hang:2,someon:1,membership:1,rais:1,e1sry4:1,two:1,consid:1,mai:1,alpha:1,littl:1,"class":1,don:1,parse_arg:1,nevertheless:1,descript:1,correspond:1,object:1,order:[],inform:1,command:1,thi:[1,2],element:1,first:1,lst_kv:1,style2_map:1},objtypes:{"0":"py:module","1":"py:function","2":"py:class","3":"py:attribute","4":"py:exception"},titles:["Welcome to &#8216;smtp-error-analysis&#8217;&#8217;s documentation!","findBadAddresses Module","regexEmailTester Module"],objnames:{"0":["py","module","Python module"],"1":["py","function","Python function"],"2":["py","class","Python class"],"3":["py","attribute","Python attribute"],"4":["py","exception","Python exception"]},filenames:["index","findBadAddresses","regexEmailTester"]})
     lintArgs.append("--output-format=%s" % options.defaultOutputFormat)
     lintArgs.append("--reports=n")
     lintArgs.append("--good-names=%s" % forceTheseNamesToBeAcceptable)
+    lintArgs.append("--disable=R0903,W0142,W0105,W0212,W0613,E1101")
     lintArgs.append("smtpErrorAnalysis") 
     try:
         lint.Run(lintArgs)
-pylint.bat --output-format=html --reports=y --good-names=smtpErrorAnalysis,findBadAddresses,regexEmailTester smtpErrorAnalysis > smtpErrorAnalysis-pylintoutput.html
+pylint.bat --output-format=html --reports=y --good-names=smtpErrorAnalysis,findBadAddresses,regexEmailTester --disable=R0903,W0142,W0105,W0212,W0613,E1101 smtpErrorAnalysis > smtpErrorAnalysis-pylintoutput.html

smtpErrorAnalysis-pylintoutput.html

 <body>
 <div>
 <h1>Report</h1>
-<p>110 statements analysed.</p><div>
+<p>194 statements analysed.</p><div>
 <h2>Messages by category</h2>
 <table>
 <tr class="header">
 </tr>
 <tr class="odd">
 <td>refactor</td>
-<td>0</td>
-<td>0</td>
+<td>2</td>
+<td>2</td>
 <td>=</td>
 </tr>
 <tr class="even">
 <td>warning</td>
 <td>0</td>
-<td>0</td>
-<td>=</td>
+<td>2</td>
+<td>-2.00</td>
 </tr>
 <tr class="odd">
 <td>error</td>
 </table>
 </div>
 <div>
+<h2>Messages</h2>
+<table>
+<tr class="header">
+<th>message id</th>
+<th>occurrences</th>
+</tr>
+<tr class="even">
+<td>R0914</td>
+<td>1</td>
+</tr>
+<tr class="odd">
+<td>R0912</td>
+<td>1</td>
+</tr>
+</table>
+</div>
+<div>
 <h2>Global evaluation</h2>
-Your code has been rated at 10.00/10 (previous run: 10.00/10)</div>
+Your code has been rated at 9.90/10 (previous run: 9.79/10)</div>
 <div>
 <h2>Statistics by type</h2>
 <table>
 </tr>
 <tr class="odd">
 <td>class</td>
-<td>1</td>
-<td>1</td>
+<td>3</td>
+<td>3</td>
 <td>=</td>
 <td>100.00</td>
 <td>0.00</td>
 </tr>
 <tr class="even">
 <td>method</td>
-<td>2</td>
-<td>2</td>
+<td>4</td>
+<td>4</td>
 <td>=</td>
 <td>100.00</td>
 <td>0.00</td>
 </tr>
 <tr class="odd">
 <td>function</td>
-<td>7</td>
-<td>7</td>
+<td>13</td>
+<td>13</td>
 <td>=</td>
 <td>100.00</td>
 <td>0.00</td>
 <h2>External dependencies</h2>
 <pre>csv (smtpErrorAnalysis.findBadAddresses)
 email (smtpErrorAnalysis.findBadAddresses)
+optparse 
+  \-OptionParser (smtpErrorAnalysis.findBadAddresses)
 os (smtpErrorAnalysis.findBadAddresses)
 pprint (smtpErrorAnalysis.findBadAddresses,smtpErrorAnalysis.regexEmailTester)
 re (smtpErrorAnalysis.findBadAddresses,smtpErrorAnalysis.regexEmailTester)</pre></div>
 </tr>
 <tr class="even">
 <td>code</td>
-<td>213</td>
-<td>60.51</td>
-<td>213</td>
+<td>310</td>
+<td>49.68</td>
+<td>310</td>
 <td>=</td>
 </tr>
 <tr class="odd">
 <td>docstring</td>
-<td>100</td>
-<td>28.41</td>
-<td>100</td>
+<td>263</td>
+<td>42.15</td>
+<td>263</td>
 <td>=</td>
 </tr>
 <tr class="even">
 <td>comment</td>
-<td>9</td>
-<td>2.56</td>
-<td>9</td>
+<td>7</td>
+<td>1.12</td>
+<td>7</td>
 <td>=</td>
 </tr>
 <tr class="odd">
 <td>empty</td>
-<td>30</td>
-<td>8.52</td>
-<td>30</td>
+<td>44</td>
+<td>7.05</td>
+<td>44</td>
 <td>=</td>
 </tr>
 </table>
 </div>
+<div>
+<h2>Messages</h2>
+<table>
+<tr class="header">
+<th>type</th>
+<th>module</th>
+<th>object</th>
+<th>line</th>
+<th>message</th>
+</tr>
+<tr class="even">
+<td>I</td>
+<td>smtpErrorAnalysis.findBadAddresses</td>
+<td>&#160;</td>
+<td>244</td>
+<td>Locally disabling W0612</td>
+</tr>
+<tr class="odd">
+<td>R</td>
+<td>smtpErrorAnalysis.findBadAddresses</td>
+<td>parse_email_for_dlv_stat_info</td>
+<td>132</td>
+<td>Too many local variables (16/15)</td>
+</tr>
+<tr class="even">
+<td>R</td>
+<td>smtpErrorAnalysis.findBadAddresses</td>
+<td>parse_email_for_dlv_stat_info</td>
+<td>132</td>
+<td>Too many branches (19/12)</td>
+</tr>
+</table>
+</div>
 </div>
 </body>
 </html>

smtpErrorAnalysis/findBadAddresses.py

                     'LAST-ATTEMPT-DATE',
                     'WILL-RETRY-UNTIL'] 
 STYLE2_MAPPINGS = { 'TEMPKEY':'DIAGNOSTIC-CODE', 
-                    'X-Failed-Recipients':'FINAL-RECIPIENT'} 
+                    'X-FAILED-RECIPIENTS':'FINAL-RECIPIENT'} 
 class EmailType:
+    '''
+    An enumerated type to allow easy description
+    of which type of email we're processing
+    '''
     (STYLE1, STYLE2, UNKNOWN) = range(0, 3)
+
+    def __init__(self):
+        pass
 class HeaderValue(object):
     '''
     Represents a 'header/value' pair in an
     email etc
     '''
-    def __init__(self,lst):
-        self.headerName = lst[0]
+    def __init__(self, lst):
+        self.header_name = lst[0]
         self.value = lst[1]
+    def __repr__(self):
+        return "('%s','%s')" % (self.header_name, self.value)
 
 
 
     string = string.replace("\r","")
     string = string.replace("\n","")
     vng = string.strip()
-    return string
+    return vng 
 
 def find_email(instr):
     '''
     this function will return ``a@foo.bar``
     '''
     l_em_to_be_clnd = email_to_be_cleaned.split(';')
-    if len(l_em_to_be_clnd) == 0:
+    if len(l_em_to_be_clnd) == 1:
         return l_em_to_be_clnd[0]
     else:
         return l_em_to_be_clnd[1]
 
 def parse_email_for_dlv_stat_info(file_name, path_em_file, 
                                     csv_dict_wrtr, options):
+    # pylint: disable=R0912
+    # pylint: disable=R0914
     '''
     Given the text of a SMTP 'bounce message' writes a CSV row 
     to match the headers in the global variable HDR_OUTPUT_COLS.
     There are two styles of 'bounce message' it does this for.
 
     STYLE 1
-    =======
 
     'STYLE1' is a multi-part mime message.
 
         Diagnostic-Code: smtp; 550 <john.smith@e.web>, Recipient unknown
 
     STYLE 2
-    =======
 
     'STYLE2' is not multi-part mime message. It consists of :
       
     if options.verbose:
         print "About to process : %s" % file_name
 
-    if file_name == '''1336519817.V811I75c5d67M552469.diezel''':
+    if file_name == '''1306139441.V811I65cc4f2M766920.diezel''':
         print "Debug here"
 
     em_style = EmailType.UNKNOWN
     bail out
     '''
     email_parts = em_msg.get_payload()
-    bln_got_notification = False
+
+    bln_got_notification = False # pylint: disable=W0612
     bln_got_dlv_report = False
 
     if em_msg.is_multipart():
         for email_part in email_parts:
             if email_part.has_key('Content-Description'):
                 if email_part['Content-Description'] == 'Notification':
-                    email_part_notification = email_part
+                    email_part_notification = email_part # pylint: disable=W0612
                     bln_got_notification = True  
                 elif email_part['Content-Description'] == 'Delivery report':
                     email_part_dlv_report = email_part
 
             if bln_got_dlv_report == False:  
                 if email_part.has_key('Content-Type'):
-                    if email_part['Content-Type'].split(";")[0] == 'message/delivery-status':
+                    if email_part['Content-Type'].split(";")[0] \
+                            == 'message/delivery-status':
                         email_part_dlv_report = email_part
                         bln_got_dlv_report = True  
     elif em_msg.has_key('X-Failed-Recipients'):
         em_style = EmailType.STYLE2
         email_style2 = em_msg
-#        '''
-#        TODO:There is another class of SMTP bounceback message which is not
-#        multipart and which could be processed here but I'm not going to 
-#        try to do that at the moment
-#        '''
-#        print "1" * 50
-#        print "File %s is potentially a unsupported format [d]" % file_name
-#        for hdr_kv in em_msg.items():
-#            hdr_name = hdr_kv[0].upper()
-#            hdr_val = hdr_kv[1]
-#            hdr_val = hdr_val.replace("\r","")
-#            hdr_val = hdr_val.replace("\n","")
-#            print "%s : %s" % (hdr_name, hdr_val)
-#        dic_test = None
-#        '''
-#        for i in em_msg.items():
-#            print "%s -> %s" % (i[0],i[1])
-#        '''
-#        print "2" * 50
     else:
         print "File %s is not a supported format [a]" % file_name
         '''
     that's been found. At some point we might also pull some stuff out 
     'Notification'
     '''
-    if em_style == EmailType.STYLE1
+    if em_style == EmailType.STYLE1:
         if bln_got_dlv_report == True:  
             #Convert the generator of email.message.Message objects returned
             #by .walk() to a list of email.message.Message
             try:
-                lst_email_part_dlv_report = convert_gen_to_list(email_part_dlv_report.walk()) 
+                lst_email_part_dlv_report = \
+                        convert_gen_to_list(email_part_dlv_report.walk()) 
             except AttributeError:
                 print "File %s is not a supported format [b]" % file_name
             else:
                 n=2 email.message.Message and so the others are ignored
                 '''
                 lst_hdrs_kv_pairs = lst_email_part_dlv_report[2].items()
-                lst_hd_val = pre_process_key_values(lst_hdrs_kv_pairs)
-
-                #Populate the dic_header_vals dictionary using the header
-                #names as keys and the header values of element values
-                dic_header_vals = populate_header_val_dict(em_style, lst_hd_val)
-
-                #Add a couple of non-header derived values and write the row
-                common_final_process_and_write(em_style, lst_hdrs_kv_pairs, file_name, csv_dict_wrtr)
+                process_email_and_write_row(em_style, lst_hdrs_kv_pairs, \
+                                            file_name, csv_dict_wrtr)
         else:
             print "File %s is not a supported format [c]" % file_name
-    elif em_style == EmailType.STYLE2
+    elif em_style == EmailType.STYLE2:
         lst_hdrs_kv_pairs = email_style2.items()
-
-        #Add a couple of non-header derived values and write the row
-        common_final_process_and_write(dic_header_vals, file_name, csv_dict_wrtr)
+        process_email_and_write_row(em_style, lst_hdrs_kv_pairs, \
+                                    file_name, csv_dict_wrtr)
     else:
         print "File %s is not a supported format [d]" % file_name
 
-def common_final_process_and_write(em_style, lst_hdrs_kv_pairs, file_name, csv_dict_wrtr):
+def process_email_and_write_row(em_style, lst_hdrs_kv_pairs, \
+                                file_name, csv_dict_wrtr):
+    '''
+    Takes a list of tuples and extracts elements of interest
+    for writing as a row to a CSV file
+    '''
     lst_hd_val = pre_process_key_values(lst_hdrs_kv_pairs)
     #Populate the dic_header_vals dictionary using the header
     #names as keys and the header values of element values
         hdr_val = hdr_kv[1]
         hdr_val = hdr_val.replace("\r","")
         hdr_val = hdr_val.replace("\n","")
-        lst_kv_out.append(HeaderValue([hdr_name, hdr_val))
+        lst_kv_out.append(HeaderValue((hdr_name, hdr_val)))
     return lst_kv_out
 
 def pre_process_headers_for_style2(lst_hv):
 
     In summary we're dropping some elements of the
     input list and for those we don't drop we're 
-    modifying the 'headerName' property of the resulting
+    modifying the 'header_name' property of the resulting
     HeaderValue instance.
     
     Each element of the input, `lst_hv` is tested 
-    to see if it 'headerName' value exists as a key of 
+    to see if it 'header_name' value exists as a key of 
     the constant dictionary `STYLE2_MAPPINGS` if it does then
     a modified form of that HeaderValue object is added to the 
     output list of HeaderValue objects.
 
-    The output HeaderValue object has a 'headerName' property 
+    The output HeaderValue object has a 'header_name' property 
     corresponding to the value of the found element in `STYLE2_MAPPINGS`
     and a 'value' property corresponding to the 'value' element
     of the input HeaderValue object.
     
     '''
     lst_hv_out = []
-    for hv in lst_hv:
-        if hv.headerName in STYLE2_MAPPINGS:
-            lst_hv_out.append(STYLE2_MAPPINGS[hv.headerName], hv.value)
+    for hval in lst_hv:
+        if hval.header_name in STYLE2_MAPPINGS:
+            hv_inst = HeaderValue((STYLE2_MAPPINGS[hval.header_name], hval.value))
+            lst_hv_out.append(hv_inst)
     return lst_hv_out
 
 def populate_header_val_dict(em_style, lst_hv):
     '''
-    Populate dictionary with lst of tuples. Using upper-cased n=0 
-    elem of each tuple as key and n=1 elem (after stripping lined 
-    feeds) as value.
-
+    Populate dictionary from a list of HeaderValue objects. Using 
+    the 'header_name' property of each instance as the dictionary
+    key and the 'value' property as the value.
+    
+    In the case of STYLE2 emails ignore some of `lst_hv` and modify
+    the 'header_name' property values for those we do process
+    
     Throw exception if the input list would generate the same key
-    twice
+    in the output dictionary twice
     '''
     dic_header_vals = {}
 
     if em_style == EmailType.STYLE2:
         lst_hv = pre_process_headers_for_style2(lst_hv)
 
-    for hv in lst_hv:
-        if dic_header_vals.has_key(hv.headerName):
+    for hval in lst_hv:
+        if dic_header_vals.has_key(hval.header_name):
             raise FindBadAddExcptn(
                     ERR3 % pprint.pformat(lst_hv))
         else:
-            dic_header_vals[hv.headerName] = hv.value 
+            dic_header_vals[hval.header_name] = hval.value 
     return dic_header_vals
 
 def convert_gen_to_list(gen):
     parser.add_option(  "-v", "--verbose", action="store_true", 
                         dest="verbose", help="Show each file processed")
 
-    (options, args) = parser.parse_args()
+    (options, args) = parser.parse_args() # pylint: disable=W0612
+
 
     if (options.inbox is None) and (options.outpath is None):   
         parser.print_help()
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.