Commits

Tim Tomes committed 613dfe6

major update. see the readme for a detailed changelog.

Comments (0)

Files changed (4)

-# Usage
+### Usage
 
 1. Install pre-requisites.
 
-    - Captures require PyQt4 or Phantomjs.
-    - Phantomjs (recommended):
-
-        + Compile Phantomjs and place the binary in the same directory as the source files.
-        + Make sure the binary is called "phantomjs".
+    - cURL
+    - PhantomJS:
+        - Compile Phantomjs and place the binary in the same directory as the source files.
+        - Name the binary is called "phantomjs".
 
 2. Run the script.
 
- - python ./peepingtom.py -h
+    - `python ./peepingtom.py -h`
+
+### Notes
+
+- Input is completely controlled by PhantomJS. PhantomJS will take any string as input for the location of a resource. If a protocol is not given, PhantomJS assumes a file path on the local operating system. Ports apply as normal.
+- Pages using JavaScript to redirect the browser will show up as a blank screen shot.
 
-# Changelog
+### Changelog
 
-### v1.2 (11.26.12)
+7.3.13
 
- - cleaned up the code for release
+- removed PyQt4 support.
+- complete codebase reorganization and optimization.
+- input is now limited only to what PhantomJS allows (see Notes).
+- added the ability to screenshot local files.
+- added full header infromation for all redirects and the final destination.
+- added clickable images for full size viewing.
+- added dynamic resizing of the html report.
+- report now groups and sorts results based on hashes of the screenshot images.
 
-### v1.1 (7.15.12)
+11.26.12
 
- - no longer freezes on redirects to 401 authentication.
- - stores each run in a unique directory.
- - shows headers for final destination rather than redirect.
- - denotes redirect next to the status header.
+- cleaned up the code for release.
 
-# Notes
+7.15.12
 
- - Keep in mind that there is no good way to follow a JavaScript redirect in an automated fashion. Pages using JavaScript to redirect the browser will show up as a blank screen shot.
- - Increased verbosity will show a lot of errors from Phantomjs and PyQt4. Most of these are debugging errors and will not impact the fidelity of the report.
+- no longer freezes on redirects to 401 authentication.
+- stores each run in a unique directory.
+- shows headers for final destination rather than redirect.
+- denotes redirect next to the status header.
-var page = require('webpage').create(),
-  url, filename, size;
+var page = require('webpage').create();
 
-url = phantom.args[0];
-filename = phantom.args[1];
+var url = phantom.args[0];
+var filename = phantom.args[1];
 page.viewportSize = { width: 1024, height: 768 };
 page.clipRect = { top: 0, left: 0, width: 1024, height: 768 };
 
-page.open(url, function (status) {
-  if ( status !== 'success')
-  {
-    console.log('FAILED to load the address ' + url );
-    phantom.exit();
-  }
-  window.setTimeout(function ()
-  {
-    page.render(filename);
-    console.log('Rendered ' + filename ' from ' + url );
-    phantom.exit();
-  }, 2000);
+page.open(url, function(status) {
+    if (status !== 'success') {
+        console.log('Unable to load the address: ' + url);
+        phantom.exit()
+    }
+    window.setTimeout(function () {
+        page.render(filename);
+        console.log('Successfully rendered: ' + url);
+        phantom.exit();
+    }, 2000);
 });

capture.py

-# -*- coding: utf-8 -*-
-
-"""
-This tries to do more or less the same thing as CutyCapt, but as a
-python module.
-
-Modified by Tim Tomes (@LaNMaSteR53) July 2012 to support PeepingTom.
-
-This is a derived work from CutyCapt: http://cutycapt.sourceforge.net/
-
-////////////////////////////////////////////////////////////////////
-//
-// CutyCapt - A Qt WebKit Web Page Rendering Capture Utility
-//
-// Copyright (C) 2003-2010 Bjoern Hoehrmann <bjoern@hoehrmann.de>
-//
-// This program is free software; you can redistribute it and/or
-// modify it under the terms of the GNU General Public License
-// as published by the Free Software Foundation; either version 2
-// of the License, or (at your option) any later version.
-//
-// This program is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU General Public License for more details.
-//
-// $Id$
-//
-////////////////////////////////////////////////////////////////////
-
-"""
-
-import sys
-from PyQt4 import QtCore, QtGui, QtWebKit, QtNetwork
-
-
-class Capturer(object):
-    """A class to capture webpages as images"""
-
-    def __init__(self, url, filename):
-        self.url = url
-        self.filename = filename
-        self.saw_initial_layout = False
-        self.saw_document_complete = False
-
-    def loadFinishedSlot(self):
-        self.saw_document_complete = True
-        if self.saw_initial_layout and self.saw_document_complete:
-            self.doCapture()
-
-    def initialLayoutSlot(self):
-        self.saw_initial_layout = True
-        if self.saw_initial_layout and self.saw_document_complete:
-            self.doCapture()
-
-    def capture(self):
-        """Captures url as an image to the file specified"""
-        self.wb = QtWebKit.QWebPage()
-        self.network_manager = QtNetwork.QNetworkAccessManager()
-        self.network_manager.sslErrors.connect(self.on_ssl_errors)
-        self.wb.setNetworkAccessManager(self.network_manager)
-        self.wb.mainFrame().setScrollBarPolicy(
-            QtCore.Qt.Horizontal, QtCore.Qt.ScrollBarAlwaysOff)
-        self.wb.mainFrame().setScrollBarPolicy(
-            QtCore.Qt.Vertical, QtCore.Qt.ScrollBarAlwaysOff)
-
-        self.wb.loadFinished.connect(self.loadFinishedSlot)
-        self.wb.mainFrame().initialLayoutCompleted.connect(
-            self.initialLayoutSlot)
-
-        self.wb.mainFrame().load(QtCore.QUrl(self.url))
-
-    def doCapture(self):
-        # Set the size of the (virtual) browser window
-        size = self.wb.mainFrame().contentsSize()
-        size.setWidth(800)
-        size.setHeight(600)
-        self.wb.setViewportSize(size)
-        #self.wb.setViewportSize(self.wb.mainFrame().contentsSize())
-        img = QtGui.QImage(self.wb.viewportSize(), QtGui.QImage.Format_ARGB32)
-        #print self.wb.viewportSize()
-        painter = QtGui.QPainter(img)
-        self.wb.mainFrame().render(painter)
-        painter.end()
-        img.save(self.filename)
-        QtCore.QCoreApplication.instance().quit()
-
-    def on_ssl_errors(self, reply, errors):
-        url = unicode(reply.url().toString())
-        reply.ignoreSslErrors()
-        #print "SSL certificate error ignored: %s" % url
-
-if __name__ == "__main__":
-    """Run a simple capture"""
-    app = QtGui.QApplication(sys.argv)
-    c = Capturer(sys.argv[1], sys.argv[2])
-    c.capture()
-    app.exec_()
 import sys
-import socket
 import urllib2
 import subprocess
 import re
 import time
 import os
-from urlparse import urlparse
+import hashlib
 
 #=================================================
 # MAIN FUNCTION
 #=================================================
 
 def main():
+    # check for pre-requisites
+    if not os.path.exists('phantomjs'):
+        print '[!] PhantomJS required.'
+        return
+    if not os.path.exists('/usr/bin/curl'):
+        print '[!] cURL required.'
+        return
+
+    # continue with options parsing
     import optparse
-    usage = "%prog [options]\n\n%prog - Tim Tomes (@LaNMaSteR53) (www.lanmaster53.com)"
-    parser = optparse.OptionParser(usage=usage, version="%prog 1.2")
-    parser.add_option('-v', help='Enable verbose mode.', dest='verbose', default=False, action='store_true')
-    parser.add_option('-i', help='File input mode. Name of input file. [IP:PORT]', dest='infile', type='string', action='store')
+    usage = '%prog [options]\n\n%prog - Tim Tomes (@LaNMaSteR53) (www.lanmaster53.com)'
+    parser = optparse.OptionParser(usage=usage)
+    parser.add_option('-l', help='File input mode. Name of input file.', dest='infile', type='string', action='store')
     parser.add_option('-u', help='Single URL input mode. URL as a string.', dest='url', type='string', action='store')
-    parser.add_option('-q', help='PyQt4 capture mode. PyQt4 python modules required.', dest='pyqt', default=False, action='store_true')
-    parser.add_option('-p', help='Phantomjs capture mode. Phantomjs required.', dest='phantom', default=False, action='store_true')
     (opts, args) = parser.parse_args()
-
     if not opts.infile and not opts.url:
-        parser.error("[!] Must provide input. Mode option required.")
-    if not opts.pyqt and not opts.phantom:
-        capture = False
-        print '[!] WARNING: No capture mode provided. Retrieving header data only.'
-    else:
-        capture = True
+        print '[!] Input mode required.'
+        return
     if opts.infile:
-        targets = open(opts.infile).read().split()
+        try:
+            targets = open(opts.infile).read().split()
+        except IOError:
+            print '[!] Invalid path \'%s\'.' % opts.infile
+            return
     if opts.url:
-        targets = []
-        targets.append(opts.url)
+        targets = [opts.url]
 
-    dir = time.strftime('%y%m%d_%H%M%S', time.localtime())
-    print '[*] Storing data in \'%s/\'' % (dir)
-    os.mkdir(dir)
-    outfile = '%s/report.html' % (dir)
-    
-    socket.setdefaulttimeout(5)
+    # setup data storage location
+    directory = time.strftime('%y%m%d_%H%M%S', time.localtime())
+    print '[*] Storing data in \'%s/\'' % (directory)
+    os.mkdir(directory)
+    outfile = '%s/report.html' % (directory)
 
-    zombies = []
-    servers = {}
-    # logic for validating list of urls and building a new list which understands the redirected sites.
-    try:
-        for target in targets:
-            headers = None
-            prefix = ''
-            # best guess at protocol prefix
-            if not target.lower().startswith('http'):
-                if target.find(':') == -1: target += ':80'
-                prefix = 'http://'
-                if target.split(':')[1].find('443') != -1:
-                    prefix = 'https://'
-            # drop port suffix where not needed
-            if target.endswith(':80'): target = ':'.join(target.split(':')[:-1])
-            if target.endswith(':443'): target = ':'.join(target.split(':')[:-1])
-            # build legitimate target url
-            target = prefix + target
-            code, headers = getHeaderData(target)
-            if code == 'zombie':
-                zombies.append((target, headers))
-            else:
-                filename = '%s.png' % re.sub('\W','',target)
-                servers[target] = [code, filename, headers]
-                if capture: getCapture(code, target, '%s/%s' % (dir,filename), opts)
-    except KeyboardInterrupt:
-        print ''
+    # logic to gather screenshots and headers for the given targets
+    db = {'targets': {'live': [], 'dead': []}}
+    for target in targets:
+        filename = '%s.png' % re.sub('\W','',target)
+        filepath = '%s/%s' % (directory, filename)
+        print '[*] %s' % (target)
+        if getCapture(target, filepath):
+            target_dict = {}
+            target_dict['url'] = target
+            target_dict['path'] = filename
+            target_dict['hash'] = hashlib.md5(open(filepath).read()).hexdigest()
+            target_dict['headers'] = getHeaders(target)
+            db['targets']['live'].append(target_dict)
+        else:
+            db['targets']['dead'].append(target)
     
-    generatePage(servers, zombies, outfile)
+    # build the report and exit
+    buildReport(db, outfile)
     print 'Done.'
 
 #=================================================
 # SUPPORT FUNCTIONS
 #=================================================
 
-def getCapture(code, url, filename, opts):
-    if code != 401:
-        verbose = opts.verbose
-        try:
-            if opts.pyqt:      cmd = 'python ./capture.py %s %s' % (url, filename)
-            elif opts.phantom: cmd = './phantomjs --ignore-ssl-errors=yes ./capture.js %s %s' % (url, filename)
-            else: return
-            proc = subprocess.Popen([cmd], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
-            stdout, stderr = proc.communicate()
-            response = str(stdout) + str(stderr)
-            returncode = proc.returncode
-            if returncode != 0:
-                print '[!] %d: %s' % (returncode, response)
-            if response != 'None':
-                if verbose: print '[+] \'%s\' => %s' % (cmd, repr(response))
-        except KeyboardInterrupt:
-            pass
+def getCapture(url, filename):
+    cmd = './phantomjs --ignore-ssl-errors=yes ./capture.js %s %s' % (url, filename)
+    response = runCommand(cmd)
+    print '\t[PhantomJS] %s' % (response)
+    if not 'Successfully rendered' in response:
+        return False
+    return True
 
-def getHeaderData(target):
-    server = None
-    url = None
-    code = None
-    status = None
-    headers = None
-    header_str = None
-    server = urlparse(target)
-    # set up request for getting header information
-    opener = urllib2.build_opener(SmartRedirectHandler) # debug with urllib2.HTTPHandler(debuglevel=1)
-    urllib2.install_opener(opener)
-    req = urllib2.Request(server.geturl())
-    try:
-        res = urllib2.urlopen(req)#,'',3)
-        print '[*] %s %s. Good.' % (target, res.getcode())
-    except Exception as res:
-        try:
-            res.getcode()
-            print '[*] %s %s. Good.' % (target, res.getcode())
-        except:
-            error = res.__str__()
-            print '[*] %s %s. Visit manually from report.' % (target, error)
-            return 'zombie', error
+def getHeaders(url):
+    cmd = 'curl -sILk %s' % (url)
+    response = runCommand(cmd)
+    if response:
+        msg = 'Headers retrieved'
+    else:
+        msg = 'No headers found'
+    print '\t[cURL] %s: %s' % (msg, url)
+    return response
 
-    url = res.geturl()
-    code = res.code
-    status = res.msg
-    headers = res.info().headers       
-    header_str = '<br />%s %s<br />\n' % (code, status)
-    for header in headers:
-        header_str += '<span class="header">%s</span>: %s<br />\n' % (header.split(':')[0].strip(), header.split(':')[1].strip())
-    return code, header_str
+def runCommand(cmd):
+    proc = subprocess.Popen([cmd], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
+    stdout, stderr = proc.communicate()
+    response = ''
+    if stdout: response += str(stdout)
+    if stderr: response += str(stderr)
+    return response.strip()
 
-def generatePage(servers, zombies, outfile):
-    tmarkup = ''
-    zmarkup = ''
-    for server in servers.keys():
-        tmarkup += "<tr>\n<td class='img'><img src='%s' /></td>\n<td class='head'><a href='%s' target='_blank'>%s</a> %s</td>\n</tr>\n" % (servers[server][1],server,server,servers[server][2])
-    if len(zombies) > 0:
-      zmarkup = '<tr><td><h2>Failed Requests</h2></td><td>\n'
-      for server in zombies:
-          zmarkup +=  "<a href='%s' target='_blank'>%s</a> %s<br />\n" % (server[0],server[0],server[1])
-      zmarkup += '</td></tr>\n'
+def buildReport(db, outfile):
+    live_markup = ''
+    dead_markup = ''
+    for live in sorted(db['targets']['live'], key=lambda k: k['hash']):
+        live_markup += "<tr><td class='img'><a href='{0}'><img src='{0}' /></a></td><td class='head'><a href='{1}' target='_blank'>{1}</a><br /><pre>{2}</pre></td></tr>\n".format(live['path'],live['url'],live['headers'])
+    if len(db['targets']['dead']) > 0:
+        dead_markup = '<tr><td><h2>Invalid Targets</h2></td><td>\n'
+        for dead in db['targets']['dead']:
+          dead_markup +=  "<a href='{0}' target='_blank'>{0}</a><br />\n".format(dead)
+        dead_markup += '</td></tr>\n'
     file = open(outfile, 'w')
     file.write("""
 <!doctype html>
 <head>
 <style>
 table, td, th {border: 1px solid black;border-collapse: collapse;padding: 5px;font-size: .9em;font-family: tahoma;}
-table {table-layout:fixed;}
-td.img {width: 400px;white-space: nowrap;}
-td.head {vertical-align: top;word-wrap:break-word;}
-.header {font-weight: bold;}
-img {width: 400px;}
+table {width: 100%%;table-layout: fixed;min-width: 1000px;}
+td.img {width: 40%%;}
+img {width: 100%%;}
+td.head {vertical-align: top;word-wrap: break-word;}
 </style>
 </head>
 <body>
-<table width='100%%'>
+<table>
 %s%s
 </table>
 </body>
-</html>""" % (tmarkup, zmarkup))
+</html>""" % (live_markup, dead_markup))
     file.close()
 
 #=================================================
-# CUSTOM CLASS WRAPPERS
-#=================================================
-
-class SmartRedirectHandler(urllib2.HTTPRedirectHandler):
-
-    def http_error_301(self, req, fp, code, msg, headers):
-        result = urllib2.HTTPRedirectHandler.http_error_301(self, req, fp, code, msg, headers)
-        result.status = code
-        result.msg = msg + ' (Redirect)'
-        return result
-    http_error_302 = http_error_303 = http_error_307 = http_error_301
-
-#=================================================
 # START
 #=================================================