Joe Heck avatar Joe Heck committed 8ada3eb

updating documentation

Comments (0)

Files changed (8)

docs/faq/index.rst

-.. _faq-index:
-
-========
-Eyes FAQ
-========
-   
-Why does this project exist?
-----------------------------
-
-How do I do X? Why doesn't Y work? Where can I go to get help?
---------------------------------------------------------------
-
-If this FAQ doesn't contain an answer to your question, you might want to
-try the `eyes-users mailing list`_. Feel free to ask any question related
-to installing, using, or debugging Eyes.
-
-If you prefer IRC, the `#eyes IRC channel`_ on the Freenode IRC network is an
-active community of helpful individuals who may be able to solve your problem.
-
-.. _`eyes-users mailing list`: http://groups.google.com/group/eyes-users
-.. _`#eyes IRC channel`: irc://irc.freenode.net/eyes
-
-How do I get started?
----------------------
-
-    #. `Download the code`_.
-    #. Install Django (read the :ref:`installation guide <intro-install>`).
-    #. Check out the rest of the :ref:`documentation <index>`, and ask questions if you
-       run into trouble.
-
-.. _`Download the code`: http://bitbucket.org/heckj/eyes/src/
-
-What are Eyes's prerequisites?
---------------------------------
-
-Eyes requires
-* Python_ 2.4 or later.
-* Django_ 1.2 or later
-
-For a development environment -- if you just want to experiment with Django --
-you don't need to have a separate Web server installed; Django comes with its
-own lightweight development server. For a production environment, we recommend
-`Apache 2`_ and mod_python_, although Django follows the WSGI_ spec, which
-means it can run on a variety of server platforms.
-
-Using Django requires a database. PostgreSQL_, MySQL_, `SQLite 3`_, and Oracle_ are supported.
-
-.. _Django: http://www.djangoproject.com/
-.. _Python: http://www.python.org/
-.. _Apache 2: http://httpd.apache.org/
-.. _mod_python: http://www.modpython.org/
-.. _WSGI: http://www.python.org/peps/pep-0333.html
-.. _PostgreSQL: http://www.postgresql.org/
-.. _MySQL: http://www.mysql.com/
-.. _`SQLite 3`: http://www.sqlite.org/
-.. _Oracle: http://www.oracle.com/
 .. toctree::
    :maxdepth: 2
 
-   intro/index
-   faq/index
+   intro/overview
+   intro/faq
+   intro/install
+   intro/hackerguide
 
-   intro/hackerguide
 
 Indices and tables
 ==================

docs/intro/faq.rst

+.. _faq-index:
+
+========
+Eyes FAQ
+========
+   
+Why does this project exist?
+----------------------------
+
+How do I do X? Why doesn't Y work? Where can I go to get help?
+--------------------------------------------------------------
+
+If this FAQ doesn't contain an answer to your question, you might want to
+try the `eyes-users mailing list`_. Feel free to ask any question related
+to installing, using, or debugging Eyes.
+
+If you prefer IRC, the `#eyes IRC channel`_ on the Freenode IRC network is an
+active community of helpful individuals who may be able to solve your problem.
+
+.. _`eyes-users mailing list`: http://groups.google.com/group/eyes-users
+.. _`#eyes IRC channel`: irc://irc.freenode.net/eyes
+
+How do I get started?
+---------------------
+
+    #. `Download the code`_.
+    #. Install Django (read the :ref:`installation guide <intro-install>`).
+    #. Check out the rest of the :ref:`documentation <index>`, and ask questions if you
+       run into trouble.
+
+.. _`Download the code`: http://bitbucket.org/heckj/eyes/src/
+
+What are Eyes's prerequisites?
+--------------------------------
+
+Eyes requires
+* Python_ 2.4 or later.
+* Django_ 1.2 or later
+
+For a development environment -- if you just want to experiment with Django --
+you don't need to have a separate Web server installed; Django comes with its
+own lightweight development server. For a production environment, we recommend
+`Apache 2`_ and mod_python_, although Django follows the WSGI_ spec, which
+means it can run on a variety of server platforms.
+
+Using Django requires a database. PostgreSQL_, MySQL_, `SQLite 3`_, and Oracle_ are supported.
+
+.. _Django: http://www.djangoproject.com/
+.. _Python: http://www.python.org/
+.. _Apache 2: http://httpd.apache.org/
+.. _mod_python: http://www.modpython.org/
+.. _WSGI: http://www.python.org/peps/pep-0333.html
+.. _PostgreSQL: http://www.postgresql.org/
+.. _MySQL: http://www.mysql.com/
+.. _`SQLite 3`: http://www.sqlite.org/
+.. _Oracle: http://www.oracle.com/

docs/intro/hackerguide.rst

-Hackers/Development Install Guide
+Hacker/Development Install Guide
 =================================
 
-Base System Installation Pieces
-===============================
+Base Installation Pieces
+===========================
 
 MacOS X Development setup:
 --------------------------
 
 * *get and install MacPorts*
-* sudo port install nagios-plugins rrdtool beanstalkd
-* easy_install pip
-* pip install virtualenv
+::
+
+	sudo port install nagios-plugins rrdtool beanstalkd
+	easy_install pip
+	pip install virtualenv
 
 Ubuntu (9.10) linux setup
 -------------------------
 
 *the follow should all be invoked as 'root'*
 
-* apt-get update
-* apt-get dist-upgrade
-* apt-get install nagios-plugins mercurial rrdtool python-dev python-pip
-* apt-get install avahi-daemon avahi-autoipd
-* apt-get install build-essential
-* apt-get install ubuntu-dev-tools
-* apt-get install git-core
-* apt-get install subversion
-* apt-get install openssh-server
-* apt-get install fping
-* apt-get install python-setuptools python-distutils-extra
-* apt-get install pylint pyflakes
-* apt-get install python-openssl
-* apt-get install python-memcache
-* apt-get install python-virtualenv
-* easy_install pip
+::
+
+	apt-get update
+	apt-get dist-upgrade
+	apt-get install nagios-plugins mercurial rrdtool python-dev python-pip
+	apt-get install avahi-daemon avahi-autoipd
+	apt-get install build-essential
+	apt-get install ubuntu-dev-tools
+	apt-get install git-core
+	apt-get install subversion
+	apt-get install openssh-server
+	apt-get install fping
+	apt-get install python-setuptools python-distutils-extra
+	apt-get install pylint pyflakes
+	apt-get install python-openssl
+	apt-get install python-memcache
+	apt-get install python-virtualenv
+	easy_install pip
 
 CentOS (5.4) linux setup
 -------------------------
 
-* yum -y update
-* yum install nagios-plugins mercurial rrdtool
-* yum install python-dev python-pip
+::
+
+	yum -y update
+	yum install nagios-plugins mercurial rrdtool
+	yum install python-dev python-pip
 
 Setting up for development
 ==========================
 Set up a VirtualEnv for working with the project
 ------------------------------------------------
 
-* cd ~/virtualenv
-* pip install -E mon -U django
+::
+
+	cd ~/virtualenv
+	pip install -E mon -U django
 
 Check out the code into the virtualenv
 --------------------------------------
 
-* cd ~/virtualenv/mon
-* hg clone http://bitbucket.org/heckj/eyes/
-* workon mon
+::
+
+	cd ~/virtualenv/mon
+	hg clone http://bitbucket.org/heckj/eyes/
+	source bin/activate
 
 Install the required libraries
 ------------------------------
 
-* pip install -r eyes/requirements.txt
+::
+
+	pip install -r eyes/requirements.txt
 
 Verify by running the tests
 ---------------------------
 
-* cd ~/virtualenv/mon/eyes
-* make clean
-* make test
+::
+
+	cd ~/virtualenv/mon/eyes
+	make clean
+	make test
 
 Working with the code:
 ----------------------
 
-* cd ~/virtualenv/mon/eyes/eyeswebapp
-* python manage.py validate # *validates django config & models*)
-* ./reset.bash && python util/simplepoller.py # *resets database to latest scheme, loads data, and runs poller*
-* python manage.py shell # *access django shell*
-* python manage.py runserver # *fire up the django server @ http://localhost:8000/*
+::
+
+	cd ~/virtualenv/mon/eyes/eyeswebapp
+	python manage.py validate # *validates django config & models*)
+	./reset.bash && python util/simplepoller.py # *resets database to latest scheme, loads data, and runs poller*
+	python manage.py shell # *access django shell*
+	python manage.py runserver # *fire up the django server @ http://localhost:8000/*
 
 Useful links:
 -------------
 Running the simplest setup:
 ---------------------------
 
-* cd eyeswebapp
-* ./reset.bash && python util/simplepoller.py
-** does a local ping (initial data model loaded from modeltestdata.json) and stores the results into RRD files
-** default location (now) for RRD files is under ./tmp_rrd/ directory. 
-** default location (now) for PNG files is under ./tmp_png/ directory. 
+::
+
+	cd eyeswebapp
+	./reset.bash && python util/simplepoller.py
+	does a local ping (initial data model loaded from modeltestdata.json) and stores the results into RRD files
+	default location (now) for RRD files is under ./tmp_rrd/ directory. 
+	default location (now) for PNG files is under ./tmp_png/ directory. 
 
 Testing in sections
 -------------------
 
-* cd eyeswebapp
-* python manage.py test util
-* python manage.py test core
-* python manage.py test api
+::
+
+	cd eyeswebapp
+	python manage.py test util
+	python manage.py test core
+	python manage.py test api
 
 Migrating database changes with South
 -------------------------------------
 
-* python manage.py migrate
+::
 
-Stashing migrations as you develop
-----------------------------------
+	python manage.py migrate
+
+Pulling updated code from the repository
+----------------------------------------
+
+::
+
+	cd ~/virtualenv/mon
+	source bin/activate
+	cd eyes
+	hg incoming
+
+*this command will show you any pending changes to be pulled down and merged*
+
+To retrieve those updates:
+
+::
+
+	hg fetch
+
+Pushing code into the repository
+--------------------------------
+
+* *establish base working from latest code...*::
+	cd ~/virtualenv/mon
+	source bin/activate
+	cd eyes
+	hg incoming
+
+* *do your coding, etc*
+
+* *if you've changed any of the models or database elements of the code*::
+
+	./manage.py startmigration <app_name> <migration_name> --auto_
+
+* *for example, if you edited the models in the application "asset", you might use*::
+
+	./manage.py startmigration asset done_something_asset --auto
+	./manage.py migrate
+
+* *if you've created a new application*::
+
+	./manage.py startmigration <app_name> <migration_name> --initial_
+
+* for example, if you created the application "about", you might use::
+
+	./manage.py startmigration about initial_about --initial`
+	hg st
+	hg add
+	./manage.py test util core api
+
+* *any other baseline tests that have been established*::
+
+	hg commit
+	hg push
+
+
+Working with migrations as you develop
+--------------------------------------
 
 The project includes South, which is a framework for dealing with schema transitions and migrations
 while a project is under active development. South will only track applications for schema migration
 			asset.save()
 
 * << edit the core/models.py file >>
-* python manage.py schemamigration core remove_hostip --auto
+::
+	python manage.py schemamigration core remove_hostip --auto
 * * if you've made a model change that can lead to inconsistencies, the result might be an error that will request that you make some decisions about the model and try again.
 * verify file created at core/migrations/*
 * if all looks good migrate:
-* * python manage.py migrate
+
+::
+	python manage.py migrate
 * add into source control
-* * hg add core/migrations/*
-* * hg commit -m "adding migration for ..."
+
+::
+	hg add core/migrations/*
+	hg commit -m "adding migration for ..."
 
 Setting up Continuous Integration (using Hudson)
 ================================================
 
-* http://www.rhonabwy.com/wp/2009/11/04/setting-up-a-python-ci-server-with-hudson/
+I'm using the write up on setting up Hudson for python projects at http://www.rhonabwy.com/wp/2009/11/04/setting-up-a-python-ci-server-with-hudson/
 
 * wget http://hudson-ci.org/latest/hudson.war
 * java -jar hudson.war

docs/intro/index.rst

-.. _intro-index:
-
-Getting started
-===============
-
-New to Eyes? Well, you came to the right place: read this material to quickly get up and running.
-
-.. toctree::
-   :maxdepth: 1
-    
-   overview
-   install
-   hackerguide

docs/intro/install.rst

 .. _intro-install:
 
-Quick install guide
+Install guide
 ===================
 
 Before you can use Eyes, you'll need to get it installed. This guide will guide you to a simple, minimal installation that'll work while you walk through the introduction.
 
 Start the local poller::
 
- nohup python util/simplepoller.py &
+ nohup python util/poster.py &
 
 Log in and go from there:
 -------------------------
 * navigate a browser to http://your_site.com/admin/
 * initial user is "admin", password "admin"
 
-* Go to http://192.168.239.130/admin/auth/user/1/ to update the admin user's email and password
-* Go to http://192.168.239.130/admin/sites/site/1/ and update the site name to your local site name
+* Go to http://your_site.com/admin/auth/user/1/ to update the admin user's email and password
+* Go to http://your_site.com/admin/sites/site/1/ and update the site name to your local site name
 
 Getting updates from source
 ---------------------------
 
  python manage.py migrate
 
-to create your own local branch for your deployment. You can always see the log of changes with::
+You can always see the log of changes with::
 
  hg log
 

docs/intro/overview.rst

 .. _intro-overview:
 
 ================
-Eyes at a glance
+Eyes Overview
 ================
 
 Eyes is a project to enable quick, simple, and API enabled monitoring and data collection. Eyes (in it's current form) takes advantage of nagios plugins and wraps them with a Django web framework instead of the Nagios framework - intended to make dynamically creating, updating, and processing of monitor points far easier than it exists today.
 
+First and foremost, Eyes has been created to enable systems and infrastructure with a very high rate of change. Fundamentally, this means being API driven from the base and going up - enabling the creation, destruction, and update of monitors and data sources at an API level so they can be programmatically manipulated.
+
+Eyes is also built to be a component in a larger system, processing information and doing its work asynchronously from other components in the system.
+
 The core of eyes is a poll-based system, but the REST API design allows for passive consumption of monitoring  data as well (i.e. agent based mechanisms).
 
 Project Philosophy:
 * Testing for verification built into the application function from the start
 * unit testing and functional testing structures built into the development process
 * Documentation for Eyes to be built alongside the code
-* * How to install and use the product
-* * APIs and example code for manipulating Eyes
+* - How to install and use the product
+* - APIs and example code for manipulating Eyes
+
+Overall Monitoring Goals:
+-------------------------
+
+#. API level access to all elements and components of a monitoring system
+#. Auditing - all actions and changes to the system audited and traceable
+#. Delegated Access
+#. Ability to have external verification that the system is functional
+#. The capability to monitor and report, but not alert
+#. de-duplication of alerts/message storms through dependency analysis
+#. templates for monitors and monitor sets
+#. grouping and reporting in multiple dimensions
+#. event correlation and alert suppression using a system model and additional logic
+
+Detailed Goals:
+---------------
+
+1. API level access to all elements of the monitors
+ * attributes (such as associated CI, priority/impact if monitor is triggered, thresholds if set centrally)
+ * api to deploy a given monitor across 1..N additional servers/CI’s (maybe services in some cases)
+ * api to create all elements of a monitor from scratch programmatically so that we can generate monitors when we deploy applications
+ * api to enable the removal of monitors on decommissioning of a server or service
+
+2. Audit ability
+ * be able to associate, list, and report on what monitors exist against which servers & services (by CI in a CMDB)
+ * be able to verify the content that a monitor triggers includes desired minimal information
+ * * CI data label/link/reference
+ * * Priority/Impact data label
+ * Any relevant Knowledge Base instructions or documents for people responding to a monitor alert
+ * Given a monitor template or collection of monitors, I want to know all the nodes it has been deployed to.
+ * I want to be able to run basic reporting against nodes, policies, alert types.
+
+3. Delegated access
+ * enable folks outside of the core monitoring administrators to create and set up monitors
+ * a end user should be able
+ * * to create a monitor
+ * * test a monitor
+ * * ask that the monitor be deployed against 1..N systems
+ * * update that monitor with new critical attributes (links to wiki articles, priority and/or impact of incidents on a failure, associations for the monitor to other systems)
+ * In order to do delegated access properly we need to provide a feedback loop to whomever will be using/creating/editing monitors.  The users need to be able to see their alert, see that it is correct or incorrect, and be able to get feedback without intervention.  ie.  They need to be able to debug and fix their own problems.
+ * They also need to be prevented from taking down the monitoring infrastructure.
+
+4. Ability to have external verification that system is functional
+ * external verification that the monitoring system is functioning
+ * agents on systems (if relevant), message passing, and generating ticketing
+ * integration flow verification from multiple remote data centers to any central consoles/servicedesk
+
+5. Operational but not alerting
+ * monitor, but not generate events during operator or engineering invoked "shut up, I'm doing maintenance" time
+ * API to toggle this per monitor or all monitors associated to a CI
+
+6. Dedupe/spamming
+ * rate limiting event creation to keep from spamming and shutting down queues in system
+ * internal monitoring and queue reporting to show efficiency and effectiveness of the system
+
+7. Templates:
+ * engineering (non core monitoring administrators) create templates for groups of monitors that can be applied to servers or services
+ * be able to assume system variables (IP / hostname) as I apply them to the next server
+ * For instance I would want to create a standard set of “DB SAN Server monitors” that would monitor HBAs, SQL queues, etc that would be added in addition to standard server monitors.
+ * Ideally a SQL server would get “Standard Pack” + “SAN pack” + “MSSQL Pack” of monitors
+
+8. Grouping and Reporting
+ * I want to be able to group and report on the monitors in multiple different dimensions
+ * I specifically *don't* want the reporting of monitors tied to a single hierarchy
+ * some examples:
+ * * All SQL monitors
+ * * All Windows server monitors
+ * * All SAN Disk monitors
+ * Be able to reporting on monitors disabled for X days/weeks/months
+
+9. Event correlation with CMDB using
+ * custom logic designed by system engineering
+ * delegated authority to implement/set/update these pieces of logic
+ * some API level mechanism to enable auditing/reporting of the logic components and what CI’s are associated with these monitors
 [build_sphinx]
 source-dir = docs/
-build-dir = docs/_build
+build-dir = docs/build
 all_files = 1
 
 [upload_sphinx]
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.