Thomas Figg committed 1573678

trailing newlines check

Comments (0)

Files changed (2)

             for record in fh:
                 if isinstance(record, WarcRecord):
-                    print >> sys.stderr, 'warcrecord found'
+                    print >> sys.stderr, '   WARC', record.url
                     warcs = [record]
+                    print >> sys.stderr, 'ARC    ', record.url
                     warcs = arc.convert(record)
                 for warcrecord in warcs:


     def parse(self, stream, offset=None):
         line = stream.readline()
-        if line.startswith('WARC'):
-            record, errors, offset = self.warc.parse(stream, offset, line=line)
-            errors = list(errors) + list(self.warc.trim(stream))
-        else:
-            record, errors, offset = self.arc.parse(stream, offset, line=line)
+        while line:
+            if line.startswith('WARC'):
+                return self.warc.parse(stream, offset, line=line)
+            elif line not in ('\n','\r\n','\r'):
+                return self.arc.parse(stream, offset, line=line)
-        return record, errors,offset
+            line = stream.readline()
+        return None, (), offset
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.