Commits

Author Commit Message Labels Comments Date
Eric Rochester
Merged monad-stack.
Branches
develop
Eric Rochester
Added JSON output for the tokenizer.
Branches
monad-stack
Eric Rochester
Moved CSV output functions to Text.Bakers12.Csv.
Branches
monad-stack
Eric Rochester
Fixed import issues.
Branches
monad-stack
Eric Rochester
Added command-line option to specify the tokenizer filter.
Branches
monad-stack
Eric Rochester
Added a minimal token filter that removes WS and separator characters.
Branches
monad-stack
Eric Rochester
Fixed linting suggestions.
Branches
monad-stack
Eric Rochester
Fixed compiler warnings.
Branches
monad-stack
Eric Rochester
Added task to run hlint.
Branches
monad-stack
Eric Rochester
Updated the Python tokenizer to have output closer to the Penn TB.
Branches
monad-stack
Eric Rochester
escape returns a Builder.
Branches
monad-stack
Eric Rochester
Use a lazy text builder to construct the CSV rows.
Branches
monad-stack
Eric Rochester
Output directly as strings (better, could still be faster).
Branches
monad-stack
Eric Rochester
Added a profiling flag for the cabal script.
Branches
monad-stack
Eric Rochester
Hoisted some functions to the top level so I can profile them.
Branches
monad-stack
Eric Rochester
More profiling enabled.
Branches
monad-stack
Eric Rochester
By default use the Penn TB filter for tokenizing.
Branches
monad-stack
Eric Rochester
Commented out a debugging print statement.
Branches
monad-stack
Eric Rochester
Give a type for the empty Token.
Branches
monad-stack
Eric Rochester
Enabled profiling.
Branches
monad-stack
Eric Rochester
hgignore
Branches
monad-stack
Eric Rochester
Handle Penn TB run-ons. Last of Penn TB filter.
Branches
monad-stack
Eric Rochester
Handle Penn TB contractions.
Branches
monad-stack
Eric Rochester
Refactored penn function to make it slightly more clear and to leverage deconstruction better.
Branches
monad-stack
Eric Rochester
Handle Penn TB possessives.
Branches
monad-stack
Eric Rochester
Handle Penn TB curly braces.
Branches
monad-stack
Eric Rochester
Handle Penn TB square brackets.
Branches
monad-stack
Eric Rochester
Handle Penn TB parentheses.
Branches
monad-stack
Eric Rochester
Handle Penn TB dashes.
Branches
monad-stack
Eric Rochester
Handle Penn TB ellipses.
Branches
monad-stack
  1. Prev
  2. Next