PYPI Testing Infrastructure (PyTI) : ==================================== This project have been done during the GSoC of 2010, the aim of the project was to create an infrastructure for testing all uploaded package in pypi. in the end of the GSoC this infrastrucuture was composed from 3 entities : - An EC2 machine where tests are run. - The continuous integration system pony-build that build packages following 4 rules: - Download package from pypi. - Install package using tox. - Run package test uning py.test. - Calculate cheesecake kwality factors of the package. - A web interface with a PuSH subscriber that get notification from pypi and display testing result. How to install PyTI ? ===================== First of all clone the repository in your local machine with the commands :: hg clone https://email@example.com/mouad/pypi-testing-infrastructure-pyti - Start by installing pony-build $ cd pony-build/ $ python setup.py install - Install than the GUI package $ cd GUI/ $ python setup.py install $ python manage.py syncdb N.B: Create also the super user of the project when you are asked for. - Run the devlopment server: cd GUI/; python run.py --port <port> -i 0.0.0.0 Now you can access to the web interface from your browser but first go to admin interface and add the name of the site in the table Sites Next add the hub and the topic in the table Subscriptions eg. hub = https://pubsubhubbub.appspot.com/ topic = http://pypi.python.org/pypi?:action=lasthour verify token = subscribe_pyti (must start with "subscribe") Now go to the hub and subscribe your self given the callback url and the topic eg. callback = http://lyorn.idyll.org:8084/push/ topic = http://pypi.python.org/pypi?:action=lasthour Verify type = Asynchronous Mode = subscribe verify token = subscribe_pyti (must start with "subscribe") For finishing you have to personalize the build client in ``pony-build/client/build-pypi-package.py`` if you want to use the same client first of all you have to change arguments of EC2Context by specifing a new EC2 machine and a new username and a new private key, and an other thing if you want to use the command 'tox' you have to upload into your EC2 machine an tox.ini file and to put it in the home directory of the username given to EC2Context eg. ~/tox.ini [tox] envlist = py26 [testenv] commands=py.test Project structure: ================== This project will contain 2 main sub project . pony-build : This is a clone of the continuous integration system pony-build in which i added some feature like EC2Context. GUI : This is the web interface it's based on Django and jquery, this sub project contain an app that work as a pubsubhubbub subscriber to get package from pypi so that they can be tested. What Next ? =========== - The client script "pony-build/client/build-pypi-package.py" that run builds commands can be ameliorate to allow build to be run in more than one EC2 machine like that we can have a EC2 machine for windows and another for linux. - The EC2Context in "pony-build/client/pony_client.py" is a first version of a context that enable pony-build to run build commands in an EC2 machine, its need more tests and work (i did have in the periode of working only one machine (big thanks to Jesse Noller.)) - To trigger the package build, PyTI use PuSH notification from pypi but it's not all, to extend the system maybe we can add other PuSH source to the system an example of the use of this feature will be that a devloper can create a post commit that he notify PyTI (using PuSH for example), and by adding the hub and PuSH url to PyTI, PyTI can run his package tests for him. - The idea of implementing an AOuth in pypi was discussed in one of the irc reunion to enable third-party application like PyTI to use the same authentication as pypi, this is very helpful in a way that every pypi user can have also his account in PyTI and he can use it to build his packages to run some schedule builds ... Big Thanks: ======= I want to thanks first my mentor Jesse Noller for being there all the periode of the project and for all his remarks and advices, also i want to thanks Titus Brown for his help and for pony-build and i want to thanks all the packagers student and their mentors for all their help and suggestion.
website_scraping began watching mouad/PYPI Testing Infrastructure :PyTI
Commits by mouad were pushed to mouad/PYPI Testing Infrastructure :PyTI
PuSH subscribtion don't work