Author: Ian Hunter

Email: ianfhunter@gmail.com / hunterif@tcd.ie

License: MIT

Purpose: Final Year Project Proxy System

Webpage Reduction Proxy

This is my Final Year Project for my Computer Science course and is an alternative to Opera Turbo & Chrome's compression proxy servers, featuring additional options

#Technical Overview:

A normal web connection would be a client to server interaction (C -> S) Instead, we have two intermediary proxies for the connection to go through. (C -> Proxy1 -> Proxy2 -> S)

The user will have to run the given local proxy (L) on their machine and point their proxy settings towards it. This local proxy in turn points to our remote proxy (R), which is hosted on a known server. It is here that the connection finally makes its way out to the desired server to retrieve a webpage. (C -> L -> R -> S)

The route on the returning journey is simply the reverse. (S -> R -> L -> C)

#Why two rather than one?

Most of the work will be done on our remote server. It is responsible for taking the webpage's content and reducing it down as much as possible. This will be done using various techniques such as compression, image vectorization, etc.

This minimized content is then sent to the local proxy (It is this transfer that we are trying to minimize). The local proxy is responsible for converting any data from a format that a browser would not be able to read into one that it can. (eg. a browser wouldn't be able to read a tar.bz version of a css file)

Other Elements: The remote proxy will also host a simple webpage where we can gather information about the client, so that it can return a customized reduction for that individual user. The uniqueness of a client may be taken from its IP or a generated ID. The user can interact with this webpage through navigating directly to the site, or via a Chrome extension.

#Technologies:

The remote proxy is written in Python as plugin scripts for MITMProxy. MITMProxy is a powerful proxy that lets a developer intercept, modify, save and inspect HTTP/S web traffic The local proxy is written in Javascript in Node.js. This was chosen as it was easy enough to modify packets, but didn't need the power of MITMProxy, especially as a clientside program. The webserver for info gathering is written in Flask Flask consists of a Python backend, and Javascript & HTML for the frontend. No external Javascript libraries are used. The chrome extension is simply a frame around the webpage, with some additional items.

Elements may change as the project progresses.

##Dependencies

sudo apt-get install pip
sudo apt-get install npm
sudo apt-get install python-opencv

pip install PIL
pip install numpy
pip install slimit


Note: you will have to change the url in the Chrome Proxy Customization Extension and the local proxy to your remote proxy url

##Running

python run.py //Flask server runs on port 5000 by default
python mitmdump --host -p 8383 -s fyp-script.py -v //Remote Proxy runs on port 8383, using script 'fyp-script'
forever start --spinSleepTime 500 localproxy.js    

Issues Running?

PIL Extra Dependencies

Reduction Levels

0. • None
1. • Gzip
2. • Image Cmpress
3. • All 3
4. • SVG
5. • SVG & compress

#License MITMProxy is licensed under an MIT Licence.

As of such, this program will also go under the MIT license.

#!

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.