1. akoha
  2. django-lean

Commits

<adm...@akoha.com  committed 6c3ed69

  • Participants
  • Parent commits 162dd10
  • Branches default

Comments (0)

Files changed (1)

File Home.wiki

View file
  • Ignore whitespace
 * For conversion experiments, results and confidence are displayed per conversion goal type (and for 'any' goal). Confidence is calculated using the chi-square method.
 * For engagement experiments, confidence is calculated using the chi-square method.
 
+Experiment reports are prepared using the {{{update_experiment_reports}}} management command. It's advisable to execute this command from a nightly cron-job.
+
 === Bot Exclusion ===
 
 {{{django-lean}}} attempts to exclude non-human visitors from experiment reports by only recording data for visitors who have JavaScript enabled.
 
 === Experiment Management ===
 
-Experiments may be defined, enabled, disabled, or promoted via the {{{django-admin}}} interface. You may also define experiments in your source tree and have them automatically loaded into the database (see {{{experiments.loader.ExperimentLoader}}}).
+Experiments may be defined, enabled, disabled, or promoted via the {{{django-admin}}} interface. You may also define experiments in your source tree and have them automatically loaded into the database (see [[http://bitbucket.org/akoha/django-lean/src/tip/experiments/loader.py|{{{experiments.loader.ExperimentLoader}}}]]).
+
+Each experiment has a state, which affects whether visitors are enrolled in the experiment, and whether they see the control or test case of the experiment.
+
+* {{{disabled}}}: No visitors are enrolled in the experiment. All visitors see the //control// case of the experiment, even if they were previously enrolled in the test group.
+* {{{enabled}}}: All visitors who encounter the experiment are enrolled randomly in either the test or control group, and see the corresponding case.
+* {{{promoted}}}: No visitors are enrolled in the experiment. All visitors see the //test// case of the experiment, even if they were previously enrolled in the control group.
+
+New experiments start in the {{{disabled}}} state.
 
 === Experiment Implementation ===
 
 } else {
   // control case
 }
+}}}
+
+=== Conversion Tracking ===
+
+Conversion experiments track the rate of conversion for their test and control groups. It is up to you to define and record the achievement of one or more project specific conversion goals.
+
+==== Defining Conversion Goals ====
+
+Conversion goals are defined by placing rows in the {{{experiments_goaltypes}}} table. This table is not currently exposed via {{{django-admin}}} but probably should be (patches welcome!). Alternative ways to populate it include manually via SQL, manually via the Django shell, via your {{{initial_data}}} fixture, or by defining a data migration in your database management tool (we use [[http://south.aeracode.org/|{{{django-south}}}]]).
+
+Here is an example of defining a goal type using the Django shell:
+
+{{{
+#!pycon
+erik-wrights-macbook-pro:akoha erikwright$ ./manage.py shell --plain
+Python 2.6.2 (r262:71600, Jul 16 2009, 12:11:28) 
+[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
+Type "help", "copyright", "credits" or "license" for more information.
+(InteractiveConsole)
+>>> from experiments.models import GoalType
+>>> GoalType.objects.create(name="signup")
+<GoalType: <GoalType: signup>>
+>>> 
+}}}
+
+==== Recording Conversions ====
+
+Once you have defined a conversion goal type, you must record it when it is achieved place. This may be done either programatically, or using a tracking pixel.
+
+===== Recording Conversions Programatically =====
+
+{{{
+#!python
+from experiments.models import GoalRecord
+from experiments.util import WebUser
+
+...
+
+def my_signup_view_func(request, *args, **kwargs):
+    # ... process the signup request
+    GoalRecord.record("signup", WebUser(request))
+    # ...
+}}}
+
+==== Recording Conversions Using a Tracking Pixel ====
+
+Sometimes a conversion happens somewhere that you don't control (for example, an e-commerce platform). In this case, you can record the conversion by placing a transparent 1x1 pixel on the page that users see after the conversion occurs (i.e., the 'Thank You' page after purchase).
+
+{{{
+#!html
+<img src="http://example.com/experiments/goal/purchase" height="1" width="1" />
+}}}
+
+=== Engagement Tracking ===
+
+Engagement experiments track an arbitrary engagement value for each user in their test and control groups. It is up to you to define a function that calculates an appropriate engagement value for your users.
+
+Here is an example engagement calculator:
+
+{{{
+#!python
+class MyEngagementScoreCalculator(object):
+
+    def calculate_user_engagement_score(self, user, start_date, end_date):
+        """
+        Define engagement as 'dollars spent per day'
+        """
+
+        days_in_period = (end_date - start_date).days + 1
+
+        period_purchase_total = sum([p.subtotal for p in Purchase.objects.filter(
+                                                        user=user, date__gte=start_date, date__lte=end_date)])
+        engagement_score = ((float)(period_purchase_total) /
+                                                        days_in_period)
+        return engagement_score
+}}}
+
+Your engagement calculator must be registered in {{{settings.py}}} as follows:
+
+{{{
+#!python
+...
+LEAN_ENGAGEMENT_CALCULATOR = "mycompany.MyEngagementScoreCalculator"
+...
 }}}