Depending on how many prerequisites you already have installed, and
-what your Cabal configuration looks like, the build may take a few
-minutes: a few seconds for `criterion`, and the rest for its
+what your Cabal configuration looks like, the build will probably take
* "**OLS regression**" estimates the time needed for a single
execution of the activity being benchmarked, using an
[ordinary least-squares regression model](https://en.wikipedia.org/wiki/Ordinary_least_squares).
- This number should be similar to the "mean estimate" row beneath it.
- It is usually more accurate, as it more effectively eliminates
- measurement overhead and other constant factors.
+ This number should be similar to the "mean execution time" row a
+ couple of rows beneath. The OLS estimate is usually more accurate
+ than the mean, as it more effectively eliminates measurement
+ overhead and other constant factors.
* "**R² goodness-of-fit**" is a measure of how accurately the linear
regression model fits the observed measurements. If the measurements
within between the lower and upper bounds. When the main estimate is
of good quality, the lower and upper bounds will be close to its
+# How to write a benchmark
+A criterion benchmark suite consists of a series of
+ bgroup "fib" [ bench "1" $ whnf fib 1
+ , bench "5" $ whnf fib 5
+ , bench "9" $ whnf fib 9
+ , bench "11" $ whnf fib 11
+We group related benchmarks together using the
+function. Its first argument is a name for the group of benchmarks.
+bgroup :: String -> [Benchmark] -> Benchmark
+All the magic happens with the
+function. The first argument to `bench` is a name that describes the
+activity we're benchmarking.
+bench :: String -> Benchmarkable -> Benchmark
+type is a container for code that can be benchmarked.
+By default, criterion allows two kinds of code to be benchmarked.
+* Any `IO` action can be benchmarked directly.
+* With a little trickery, we can benchmark pure functions.
+## Benchmarking an `IO` action
+This function shows how we can benchmark an `IO` action.
+ bench "readFile" $ nfIO (readFile "GoodReadFile.hs")
+to specify that after we run the `IO` action, its result must be
+evaluated to **normal form**, i.e. so that all of its internal
+constructors are fully evaluated, and it contains no thunks.
+nfIO :: NFData a => IO a -> IO ()
+Rules of thumb for when to use `nfIO`:
+* Any time that lazy I/O is involved, use `nfIO` to avoid resource
+* If you're not sure how much evaluation will have been performed on
+ the result of an action, use `nfIO` to be certain that it's fully
+In addition to `nfIO`, criterion provides a
+function that evaluates the result of an action only to the point that
+the outermost constructor is known (using `seq`).
+This function is useful if your `IO` action returns a simple value
+like an `Int`, or something more complex like a
+where evaluating the outermost constructor will do "enough work".
+# Be careful with lazy I/O!
+Experienced Haskell programmers don't use lazy I/O very often, and
+here's an example of why: if you try to run the benchmark below, it
+ bench "whnfIO readFile" $ whnfIO (readFile "BadReadFile.hs")
+The reason for the crash is that `readFile` reads the contents of a
+file lazily: it can't close the file handle until whoever opened the
+file reads the whole thing. Since `whnfIO` only evaluates the very
+first constructor after the file is opened, the benchmarking loop
+causes a large number of open files to accumulate, until the
+benchmarking whnfIO readFile
+openFile: resource exhausted (Too many open files)
+GHC is an aggressive compiler. If you have an `IO` action that
+doesn't really interact with the outside world, *and* it has just the
+right structure, GHC may notice that a substantial amount of its
+computation can be memoised via "let-floating".
+[somewhat contrived example](https://github.com/bos/criterion/blob/master/examples/ConduitVsPipes.hs)
+of this problem, where the first two benchmarks run between 40 and
+40,000 times faster than they "should".
+As always, if you see numbers that look wildly out of whack, you
+shouldn't rejoice that you have magic fast performance---be skeptical
+For this particular misbehaving benchmark suite, GHC has an option
+that will turn off let-floating and restore the first two benchmarks
+to performing as expected.