Anonymous avatar Anonymous committed 2a1c28c

Documentation updates.

Comments (0)

Files changed (2)

docs/rst/Orange.evaluation.reliability.rst

 ********************************************************
 
 Reliability assessment aims to predict reliabilities of individual
-predictions. Most of implemented algorithms for regression described in
-[Bosnic2008]_ and in [Pevec2011]_ for classification.
+predictions. Most of the implemented algorithms for regression are described in
+[Bosnic2008]_; the algorithms for classification are described in [Pevec2011]_.
 
-We can use reliability estimation with any Orange learners. The following example:
+We can use reliability estimation with any prediction method. The following example:
 
  * Constructs reliability estimators (implemented in this module),
- * The :obj:`Learner` wrapper combines a regular learner, here a :obj:`~Orange.classification.knn.kNNLearner`, with reliability estimators.
- * Obtains prediction probabilities from the constructed classifier
-   (:obj:`Orange.classification.Classifier.GetBoth` option). The resulting
-   probabilities have an additional attribute, :obj:`reliability_estimate`,
+ * The :obj:`Learner` wrapper combines a a prediction method (learner), here a :obj:`~Orange.classification.knn.kNNLearner`, with reliability estimators.
+ * Obtains prediction probabilities, which have an additional attribute, 
+   :obj:`reliability_estimate`,
    that contains a list of :class:`Orange.evaluation.reliability.Estimate`.
 
 .. literalinclude:: code/reliability-basic.py
     :lines: 7-
 
-We could also evaluate more examples. The next example prints reliability estimates
+The next example prints reliability estimates
 for first 10 instances (with cross-validation):
 
 .. literalinclude:: code/reliability-run.py
 .. data:: METHOD_NAME
 
     A dictionary that that maps reliability estimation
-    method IDs (integerss) to method names (strings).
+    method IDs (integers) to method names (strings).
 
 .. autoclass:: Estimate
     :members:

orangecontrib/reliability/__init__.py

 
 def get_pearson_r(res):
     """
-    :param res: results of evaluation, done using learners,
-        wrapped into :class:`Orange.evaluation.reliability.Classifier`.
+    :param res: Evaluation results with :obj:`reliability_estimate`.
     :type res: :class:`Orange.evaluation.testing.ExperimentResults`
 
-    Return Pearson's coefficient between the prediction error and each of the
-    used reliability estimates. Also, return the p-value of each of
-    the coefficients.
+    Pearson's coefficients between the prediction error and 
+    reliability estimates with p-values.
     """
     prediction_error = get_prediction_error_list(res)
     results = []
 
 def get_spearman_r(res):
     """
-    :param res: results of evaluation, done using learners,
-        wrapped into :class:`Orange.evaluation.reliability.Classifier`.
+    :param res: Evaluation results with :obj:`reliability_estimate`.
     :type res: :class:`Orange.evaluation.testing.ExperimentResults`
 
-    Return Spearman's coefficient between the prediction error and each of the
-    used reliability estimates. Also, return the p-value of each of
-    the coefficients.
+    Spearman's coefficients between the prediction error and 
+    reliability estimates with p-values.
     """
     prediction_error = get_prediction_error_list(res)
     results = []
 
 def get_pearson_r_by_iterations(res):
     """
-    :param res: results of evaluation, done using learners,
-        wrapped into :class:`Orange.evaluation.reliability.Classifier`.
+    :param res: Evaluation results with :obj:`reliability_estimate`.
     :type res: :class:`Orange.evaluation.testing.ExperimentResults`
 
-    Return average Pearson's coefficient over all folds between prediction error
-    and each of the used estimates.
+    Pearson's coefficients between prediction error
+    and reliability estimates averaged over all folds.
     """
     results_by_fold = Orange.evaluation.scoring.split_by_iterations(res)
     number_of_estimates = len(res.results[0].probabilities[0].reliability_estimate)
     number_of_instances = len(res.results)
     number_of_folds = len(results_by_fold)
     results = [0 for _ in xrange(number_of_estimates)]
+    M
     sig = [0 for _ in xrange(number_of_estimates)]
     method_list = [0 for _ in xrange(number_of_estimates)]
 
 
 class Estimate:
     """
-    Reliability estimate. Contains attributes that describe the results of
-    reliability estimation.
+    Describes a reliability estimate.
 
     .. attribute:: estimate
 
-        A numerical reliability estimate.
+        Value of reliability.
 
     .. attribute:: signed_or_absolute
 
-        Determines whether the method used gives a signed or absolute result.
+        Determines whether the method returned a signed or absolute result.
         Has a value of either :obj:`SIGNED` or :obj:`ABSOLUTE`.
 
     .. attribute:: method
 
-        An integer ID of reliability estimation method used.
+        An integer ID of the reliability estimation method used.
 
     .. attribute:: method_name
 
-        Name (string) of reliability estimation method used.
+        Name (string) of the reliability estimation method used.
 
     """
     def __init__(self, estimate, signed_or_absolute, method):
     
     :rtype: :class:`Orange.evaluation.reliability.SensitivityAnalysisClassifier`
     
-    To estimate the reliability of prediction for a given instance,
-    the learning set is extended with that instance with the label changes to 
+    The learning set is extended with that instancem, where the label is changed to 
     :math:`K + \epsilon (l_{max} - l_{min})` (:math:`K` is  the initial prediction,
     :math:`\epsilon` a sensitivity parameter, and :math:`l_{min}` and
-    :math:`l_{max}` the lower and upper bounds of labels on training data)
+    :math:`l_{max}` the lower and upper bounds of labels on training data).
     Results for multiple values of :math:`\epsilon` are combined
-    into SAvar and SAbias. SAbias can be used either in a signed or absolute form.
+    into SAvar and SAbias. SAbias has a signed or absolute form.
 
     :math:`SAvar = \\frac{\sum_{\epsilon \in E}(K_{\epsilon} - K_{-\epsilon})}{|E|}`
     :math:`SAbias = \\frac{\sum_{\epsilon \in E} (K_{\epsilon} - K ) + (K_{-\epsilon} - K)}{2 |E|}`
 class BaggingVariance:
     """
     
-    :param m: Number of bagging models to be used with BAGV estimate
+    :param m: Number of bagged models. Default: 50.
     :type m: int
     
-    :param for instances:  Optional. If test instances
+    :param for_instances:  Optional. If test instances
       are given as a parameter, this class can compute their reliabilities
       on the fly, which saves memory. 
 
     
     :rtype: :class:`Orange.evaluation.reliability.BaggingVarianceClassifier`
     
-    
-    :math:`m` different bagging models are used to estimate
-    the value of dependent variable for a given instance. For regression,
-    the variance of predictions is a reliability
-    estimate:
+    For regression, BAGV is the variance of predictions:
 
     :math:`BAGV = \\frac{1}{m} \sum_{i=1}^{m} (K_i - K)^2`, where 
     :math:`K = \\frac{\sum_{i=1}^{m} K_i}{m}` and :math:`K_i` are
     predictions of individual models.
 
-    For classification, 1 minus the average Euclidean distance between class
-    probability distributions predicted by the model, and distributions
-    predicted by the individual bagged models, is the BAGV reliability
-    measure. For classification, a greater value implies a better
-    prediction.
-    
+    For classification, BAGV is 1 minus the average Euclidean
+    distance between class probability distributions predicted by the
+    model, and distributions predicted by the individual bagged model;
+    a greater value implies a better prediction.
+
     This reliability measure can run out of memory if individual classifiers themselves
     use a lot of memory; it needs :math:`m` times memory
     for a single classifier. 
         is Hellinger distance.
     :type distance: function
 
-    :param distance_weighted: For classification,
+    :param distance_weighted: Relevant only for classification;
         use an average distance between distributions, weighted by :math:`e^{-d}`,
         where :math:`d` is the distance between predicted instance and the
         neighbour.
     
     :rtype: :class:`Orange.evaluation.reliability.CNeighboursClassifier`
     
-    For regression, CNK is defined a difference
+    For regression, CNK is a difference
     between average label of its nearest neighbours and the prediction. CNK
     can be either signed or absolute. A greater value implies greater prediction error.
 
 
 class Learner:
     """
-    Adds reliability estimation to any learner: multiple reliability estimation 
-    algorithms can be used simultaneously.
-    This learner can be used as any other learner,
+    Adds reliability estimation to any prediction method.
+    This class can be used as any other Orange learner,
     but returns the classifier wrapped into an instance of
     :class:`Orange.evaluation.reliability.Classifier`.
-    
+
     :param box_learner: Learner to wrap into a reliability estimation
         classifier.
     :type box_learner: :obj:`~Orange.classification.Learner`
 
 
     def __call__(self, instances, weight=None, **kwds):
-        """Learn from the given table of data instances.
+        """Construct a classifier.
         
-        :param instances: Data to learn from.
+        :param instances: Learning data.
         :type instances: Orange.data.Table
         :param weight: Id of meta attribute with weights of instances
         :type weight: int
 
     def __call__(self, instance, result_type=Orange.core.GetValue):
         """
-        Classify and estimate reliability of estimation for a new instance.
+        Classify and estimate reliability for a new instance.
         When :obj:`result_type` is set to
         :obj:`Orange.classification.Classifier.GetBoth` or
         :obj:`Orange.classification.Classifier.GetProbabilities`,
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.