Source

org-drill / README.org

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
# -*- mode: org; coding: utf-8 -*-
#+STARTUP: showall
#+OPTIONS: num:nil
#+TITLE: Org-Drill
#+AUTHOR: Paul Sexton

* Synopsis


Org-Drill uses the spaced repetition algorithm in =org-learn= to conduct
interactive "drill sessions", using org files as sources of facts to be
memorised. The material to be remembered is presented to the student in random
order. The student rates his or her recall of each item, and this information
is fed back to =org-learn= to schedule the item for later revision.

Each drill session can be restricted to topics in the current buffer
(default), one or several files, all agenda files, or a subtree. A single
topic can also be drilled.

Different "topic types" can be defined, which present their information to the
student in different ways.

For more on the spaced repetition algorithm, and examples of other programs
that use it, see:
- [[http://supermemo.com/index.htm][SuperMemo]] (the SM5 algorithm is discussed [[http://www.supermemo.com/english/ol/sm5.htm][here]])
- [[http://ichi2.net/anki/][Anki]]
- [[http://mnemosyne-proj.org/index.php][Mnemosyne]]


* Installation and Customisation


Put the following in your =.emacs=. You will also need to make sure that Org's
"contrib/lisp" directory is in the emacs load-path.

#+BEGIN_EXAMPLE
(require 'org-drill)
#+END_EXAMPLE

I also recommend the following, so that items are always eventually retested,
even when you remember them well.

#+BEGIN_EXAMPLE
(setq org-learn-always-reschedule t)
#+END_EXAMPLE

If you want cloze-deleted text to show up in a special font within Org mode
buffers, also add:

#+BEGIN_EXAMPLE
(setq org-drill-use-visible-cloze-face-p t)
#+END_EXAMPLE

Org-Drill supports two different spaced repetition algorithms -- SM5 (the
default, implemented by =org-learn=) and SM2. SM2 is an earlier algorithm which
remains very popular -- Anki and Mnemosyne, two of the most popular spaced
repetition programs, use SM2.

If you want Org-Drill to use the SM2 algorithm, put the following in your
=.emacs=:

#+BEGIN_EXAMPLE
(setq org-drill-spaced-repetition-algorithm 'sm2)
#+END_EXAMPLE

The intervals generated by the SM2 and SM5 algorithms are pretty
deterministic. If you tend to add items in large, infrequent batches, the lack
of variation in interval scheduling can lead to the problem of "lumpiness" --
one day a large batch of items are due for review, the next there is almost
nothing, a few days later another big pile of items is due.

This problem can be ameliorated by adding some random "noise" to the interval
scheduling algorithm. The author of SuperMemo actually recommends this approach
for the SM5 algorithm, and Org-Drill's implementation uses [[http://www.supermemo.com/english/ol/sm5.htm][his code]].

To enable random "noise" for item intervals, set the variable
=org-drill-add-random-noise-to-intervals-p= to true by putting the following in
your =.emacs=:

#+BEGIN_EXAMPLE
(setq org-drill-add-random-noise-to-intervals-p t)
#+END_EXAMPLE


* Demonstration


Load the file [[file:spanish.org][spanish.org]]. Press =M-x= and run the function =org-drill=. Follow
the prompts at the bottom of the screen.

When the drill finishes, you can look at =spanish.org= to get some idea of how
drill topics are written.


* Writing the questions


Org-Drill uses org mode topics as 'drill items'. To be used as a drill item,
the topic must have a tag that matches =org-drill-question-tag=. This is
=:drill:= by default. Any other org topics will be ignored.

You don't need to schedule the topics initially.  However =org-drill= *will*
recognise items that have been scheduled previously with
=org-learn=. Unscheduled items are considered to be 'new' and ready for
memorisation.

How should 'drill topics' be structured? Any org topic is a legal drill topic
-- it will simply be shown with subheadings collapsed. After pressing a
key, any hidden subheadings will be revealed, and you will be asked to rate
your "recall" of the item.

This will be adequate for some items, but usually you will want to write items
where you have more control over what information is hidden from the user for
recall purposes.

** Simple topics

The simplest drill topic has no special structure. When such a topic is
presented during a drill session, any subheadings are "collapsed" with their
contents hidden. So, you could include the question as text beneath the main
heading, and the answer within a subheading. For example:

#+BEGIN_EXAMPLE
* Item                                   :drill:
What is the capital city of Estonia?

** The Answer
Tallinn.
#+END_EXAMPLE

When this item is presented for review, the text beneath the main heading will
be visible, but the contents of the subheading ("The Answer") will be hidden.


** Cloze deletion

Cloze deletion can be used in any drill topic regardless of whether it is
otherwise 'simple', or is one of the specialised topic types discussed
below. To use cloze deletion, one or more parts of the body of the topic is
marked as /cloze text/ by surrounding it with single square brackets, [like
so]. When the topic is presented for review, the text within square brackets
will be obscured. The text is then revealed after the user presses a key. For
example:

#+BEGIN_EXAMPLE
* Item                                   :drill:
The capital city of Estonia is [Tallinn].
#+END_EXAMPLE

During review, the user will see:

#+BEGIN_QUOTE
The capital city of Estonia is @<font style="background-color: blue;" color="cyan">
@<tt>[...]@</tt>@</font>.
#+END_QUOTE

When the user presses a key, the text "Tallinn" will become visible.

Clozed text can also contain a "hint" about the answer. If the text 
surrounded by single square brackets contains a `|' character (vertical bar),
all text after that character is treated as a hint, and will be visible when
the rest of the text is hidden.

Example:

#+BEGIN_EXAMPLE
Type 1 hypersensitivity reactions are mediated by [immunoglobulin E|molecule]
and [mast cells|cell type].
#+END_EXAMPLE

#+BEGIN_QUOTE
Type 1 hypersensitivity reactions are mediated by 
@<font style="background-color: blue;" color="cyan">
@<tt>[...molecule]@</tt>@</font>
and @<font style="background-color: blue;" color="cyan">
@<tt>[...cell type]@</tt>@</font>.
#+END_QUOTE


** Two-sided cards

The remaining topic types all use the topic property, =DRILL_CARD_TYPE=. This
property tells =org-drill= which function to use to present the topic during
review. If this property has the value =twosided= then the topic is treated as
a "two sided card". When a two sided card is reviewed, /one of the first two/
subheadings within the topic will be visible -- all other
subheadings will be hidden.

Two-sided cards are meant to emulate the type of flipcard where either side is
useful as test material (for example, a card with a word in a foreign language
on one side, and its translation on the other).

A two sided card can have more than 2 subheadings, but all subheadings after
the first two are considered as "notes" and will always be hidden during topic
review.

#+BEGIN_EXAMPLE
* Noun                                               :drill:
    :PROPERTIES:
    :DRILL_CARD_TYPE: twosided
    :END:

Translate this word.

** Spanish
la mujer

** English
the woman

** Example sentence
¿Quién fue esa mujer? 
Who was that woman?
#+END_EXAMPLE

In this example, the user will be shown the main text -- "Translate this word"
-- and either 'la mujer', /or/ 'the woman', at random. The section 'Example
sentence' will never be shown until after the user presses a key, because it is
not one of the first two 'sides' of the topic.


** Multi-sided cards

The =multisided= card type is similar to =twosided=, except that any
subheading has a chance of being presented during the topic review. One
subheading is always shown and all others are always hidden. 

#+BEGIN_EXAMPLE
* Noun                                               :drill:
    :PROPERTIES:
    :DRILL_CARD_TYPE: multisided
    :END:

Translate.

** Spanish
la mesa

** English
the table

** Picture
[[file:table.jpg][PICTURE]]
#+END_EXAMPLE

The user will be shown the main text and either 'la mujer', /or/ 'the woman',
/or/ a picture of a table.

** Multi-cloze cards

Often, you will wish to create cards out of sentences that express several
facts, such as the following:

#+BEGIN_QUOTE
The capital city of New Zealand is Wellington, which is located in the
South Island and has a population of about 400,000.
#+END_QUOTE

There is more than one fact in this statement -- you could create a single
'simple' card with all the facts marked as cloze text, like so:

#+BEGIN_QUOTE
The capital city of [New Zealand] is [Wellington], which is located in 
[the South Island] and has a population of about [400,000].
#+END_QUOTE

But this card will be difficult to remember. If you get just one of the 4
hidden facts wrong, you will fail the card. A card like this is likely to
become a [[leeches][leech]].

A better way to express all these facts using 'simple' cards is to create
several cards, with one fact per card. You might end up with something
like this:

#+BEGIN_EXAMPLE
* Fact
The capital city of [New Zealand] is Wellington, which has a population of
about 400,000.

* Fact
The capital city of New Zealand is [Wellington], which has a population of
about 400,000.

* Fact
The capital city of New Zealand is Wellington, which has a population of
about [400,000].

* Fact
The capital city of [New Zealand] is Wellington, which is located in the
the South Island.

* Fact
The capital city of New Zealand is [Wellington], which is located in 
the South Island.

* Fact
The capital city of New Zealand is Wellington, which is located in 
[the South Island].
#+END_EXAMPLE

However, this is really cumbersome. The 'multicloze' card type exists for this
situation. Multicloze cards behave like 'simple' cards, except that when there
is more than one area marked as cloze text, only one of the marked areas will
be hidden during review -- the others all remain visible. The hidden text area
is chosen randomly at each review.

So, for the above example, we can actually use the original 'bad' simple card,
but change its card type to 'multicloze'. Each time the card is presented for
review, one of 'New Zealand', 'Wellington', 'the South Island' or '400,000'
will be hidden.

#+BEGIN_EXAMPLE
* Fact
  :PROPERTIES:
  :DRILL_CARD_TYPE: multicloze
  :END:

The capital city of [New Zealand] is [Wellington], which is located in 
[the South Island] and has a population of about [400,000].
#+END_EXAMPLE

** User-defined topic types

Finally, you can write your own elisp functions to define new kinds of
topics. Any new topic type will need to be added to
=org-drill-card-type-alist=, and cards using that topic type will need to have
it as the value of their =DRILL_CARD_TYPE= property. For an example, see the
function =org-drill-present-spanish-verb=, which defines the new topic type
=spanish_verb=, used in 'spanish.org'.

See the file [[file:spanish.org][spanish.org]] for a full set of example material.


* Running the drill session


Start a drill session with =M-x org-drill=. By default, this includes all
non-hidden topics in the current buffer. =org-drill= takes an optional
argument, SCOPE, which allows it to take drill items from other
sources. Possible values for SCOPE are:

- tree :: The subtree starting with the entry at the cursor.
- file :: The current buffer, including both hidden and non-hidden items.
- file-with-archives :: The current buffer, and any archives associated with it.
- agenda :: All agenda files.
- agenda-with-archives :: All agenda files with any archive files associated
     with them.
- (file1 file2 ...) :: A list of filenames. All files in the list will be
     scanned.

During a drill session, you will be presented with each item, then asked to
rate your recall of it by pressing a key between 0 and 5. The meaning of these
numbers is (taken from =org-learn=):

| Quality | SuperMemo label | Meaning                                    |
|---------+-----------------+--------------------------------------------|
|       0 | NULL            | You have forgotten this card completely.   |
|       1 | BAD             | Wrong answer.                              |
|       2 | FAIL            | Barely correct, the interval was too long. |
|       3 | PASS            | Correct answer, but with much effort.      |
|       4 | GOOD            | Correct answer, with a little thought.     |
|       5 | BRIGHT          | Correct answer, effortless.                |

You can press '?'  at the prompt if you have trouble remembering what the
numbers 0--5 signify. At any time you can press 'q' to finish the drill early
(your progress will be saved), 's' to skip the current item without viewing the
answer, or 'e' to finish the drill and jump to the current topic for editing
(your progress up to that point will be saved).


* Leeches
# <<leeches>>

From the Anki website, http://ichi2.net/anki/wiki/Leeches:

#+BEGIN_QUOTE
Leeches are cards that you keep on forgetting. Because they require so many
reviews, they take up a lot more of your time than other cards.
#+END_QUOTE

Like Anki, Org-Drill defines leeches as cards that you have "failed" many
times. The number of times an item must be failed before it is considered a
leech is set by the variable =org-drill-leech-failure-threshold= (15 by
default). When you fail to remember an item more than this many times, the item
will be given the =:leech:= tag.

Leech items can be handled in one of three ways. You can choose how Org-Drill
handles leeches by setting the variable =org-drill-leech-method= to one of the
following values:
- nil :: Leech items are tagged with the =leech= tag, but otherwise treated the
         same as normal items.
- skip :: Leech items are not included in drill sessions.
- warn :: Leech items are still included in drill sessions, but a warning
  message is printed when each leech item is presented.

The best way to deal with a leech is either to delete it, or reformulate it so
that it is easier to remember, for example by splitting it into more than one
card. 

See [[http://www.supermemo.com/help/leech.htm][the SuperMemo website]] for more on leeches.


* Cram mode


There are some situations, such as before an exam, where you will want to
revise all of your cards regardless of when they are next due for review.

To do this, run a /cram session/ with the =org-drill-cram= command (=M-x
org-drill-cram RET=). This works the same as a normal drill session, except
that all items are considered due for review unless you reviewed them within
the last 12 hours (you can change the number of hours by customising the
variable =org-drill-cram-hours=).


* Incremental reading


An innovative feature of the program SuperMemo is so-called "incremental
reading". This refers to the ability to quickly and easily make drill items
from selected portions of text as you read an article (a web page for
example). See [[http://www.supermemo.com/help/read.htm][the SuperMemo website]] for more on incremental reading.

Much of the infrastructure for incremental reading is already provided by Org
Mode, with the help of some other emacs packages. You can provide yourself with
an incremental reading facility by using 'org-capture' alongside a package that
allows you to browse web pages either in emacs (w3 or [[http://www.emacswiki.org/emacs/emacs-w3m][emacs-w3m]]) or in the
external browser of your choice ([[http://orgmode.org/worg/org-contrib/org-protocol.php][org-protocol]]).

Another important component of incremental reading is the ability to save your
exact place in a document, so you can read it /incrementally/ rather than all
at once. There is a large variety of bookmarking packages for emacs which
provide advanced bookmarking functionality: see the [[http://www.emacswiki.org/emacs/BookMarks][Emacs Wiki]] for details.
Bookmarking exact webpage locations in an external browser seems to be a bit
more difficult. For Firefox, the [[http://www.wired-marker.org/][Wired Marker]] addon works well.

An example of using Org-Drill for incremental reading is given below. First,
and most importantly, we need to define a couple of =org-capture= templates for
captured facts. 

#+BEGIN_EXAMPLE
(setq org-capture-templates
       `(("u"
         "Task: Read this URL"
         entry
         (file+headline "tasks.org" "Articles To Read")
         ,(concat "* TODO Read article: '%:description'\nURL: %c\n\n")
         :empty-lines 1
         :immediate-finish t)

        ("w"
         "Capture web snippet"
         entry
         (file+headline "my-facts.org" "Inbox")
         ,(concat "* Fact: '%:description'        :"
                  (format "%s" org-drill-question-tag)
                  ":\n:PROPERTIES:\n:DATE_ADDED: %u\n:SOURCE_URL: %c\n:END:\n\n%i\n%?\n")
         :empty-lines 1
         :immediate-finish t)
        ;; ...other capture templates...
    ))
#+END_EXAMPLE

Using these templates and =org-protocol=, you can set up buttons in your web
browser to:
- Create a task telling you to read the URL of the currently viewed webpage
- Turn a region of selected text on a webpage, into a new fact which is saved
  to whichever file and heading you nominate in the template. The fact will
  contain a timestamp, and a hyperlink back to the webpage where you created
  it.

For example, suppose you are reading the Wikipedia entry on tuberculosis [[http://en.wikipedia.org/wiki/Tuberculosis][here]].

You read the following:

#+BEGIN_QUOTE
The classic symptoms of tuberculosis are a chronic cough with blood-tinged
sputum, fever, night sweats, and weight loss. Infection of other organs causes
a wide range of symptoms. Treatment is difficult and requires long courses of
multiple antibiotics. Antibiotic resistance is a growing problem in
(extensively) multi-drug-resistant tuberculosis. Prevention relies on screening
programs and vaccination, usually with Bacillus Calmette-Guérin vaccine.
#+END_QUOTE

You decide you want to remember that "Bacillus Calmette-Guérin vaccine" is the
name of the vaccine against tuberculosis. First, you select the `interesting'
portion of the text with the mouse:

#+BEGIN_QUOTE
The classic symptoms of tuberculosis are a chronic cough with blood-tinged
sputum, fever, night sweats, and weight loss. Infection of other organs causes
a wide range of symptoms. Treatment is difficult and requires long courses of
multiple antibiotics. Antibiotic resistance is a growing problem in
(extensively) multi-drug-resistant tuberculosis. 
@<font style="background-color: yellow;">Prevention relies
on screening programs and vaccination, usually with Bacillus Calmette-Guérin
vaccine.@</font>
#+END_QUOTE

Then you press the button you created when setting up =org-protocol=, which is
configured to activate the capture template "w: Capture web snippet". The
selected text will be sent to Emacs, turned into a new fact using the template,
and filed away for your later attention.

(Note that it might be more efficient to turn the entire paragraph into a drill
item -- since it contains several important facts -- then split it up into
multiple items when you edit it later in Emacs.)

Once you have had enough of reading the article, save your place, then go to
your "fact" file in Emacs. You should see that each piece of text you selected
has been turned into a drill item. Continuing the above example, you would see
something like:

#+BEGIN_EXAMPLE
** Fact: 'Tuberculosis - Wikipedia, the Free Encyclopedia'        :drill:

Prevention relies on screening programs and vaccination, usually with Bacillus
Calmette-Guérin vaccine.
#+END_EXAMPLE

You need to edit this fact so it makes sense independent of its context, as
that is how it will be presented to you in future. The easiest way to turn the
text into a 'question' is by cloze deletion. All you need to do is surround the
'hidden' parts of the text with square brackets.

: Prevention of tuberculosis relies on screening programs and vaccination,
: usually with [Bacillus Calmette-Guérin vaccine].


You can of course define browser buttons that use several different "fact"
templates, each of which might send its fact to a different file or subheading,
or give it different tags or properties, for example. 


* Still to do

- =org-drill-question-tag= should use a tag match string, rather than a
  single tag? Can use =org-make-tag-matcher=.
- perhaps take account of item priorities, showing high priority items first
- get tooltips to work for old/new/etc counts during review?
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.