Bryan O'Sullivan avatar Bryan O'Sullivan committed d108cea

A little internal documentation.

Comments (0)

Files changed (1)

Data/Text/Fusion.hs

 infixl 2 :!:
 data PairS a b = !a :!: !b
 
+-- | Allow a function over a stream to switch between two states.
 data Switch = S1 | S2
 
-data Stream a = forall s. Stream (s -> Step s a) !s {-# UNPACK #-}!Int
+data Stream a =
+    forall s. Stream
+    (s -> Step s a)             -- stepper function
+    !s                          -- current state
+    {-# UNPACK #-}!Int          -- length hint
+
+-- The length hint in a Stream has two roles.  If its value is zero,
+-- we trust it, and treat the stream as empty.  Otherwise, we treat it
+-- as a hint: it should usually be accurate, so we use it when
+-- unstreaming to decide what size array to allocate.  However, the
+-- unstreaming functions must be able to cope with the hint being too
+-- small or too large.
+--
+-- The size hint tries to track the UTF-16 code points in a stream,
+-- but often counts the number of characters instead.  It can easily
+-- undercount if, for instance, a transformed stream contains astral
+-- plane characters (those above 0x10000).
 
 data Step s a = Done
               | Skip !s
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.