Optimizing heap utilization
Vengadanathan
Branch: vengadan_amzn/vengadan-seunjeon:heap_optimize
Branch: eunjeon/seunjeon:heap_optimize
Merged
Merged pull request
Improving throughput by removing snappy compression on feature array
Improving throughput by removing snappy compression on feature array
Description:
seunjeon is occupying heap of around 560 MB Initial optimization done by us reduced the heap utilization of the plugin to ~ 174 MB (around 68% reduction) in heap usage.
Profiling and metrics:
Each query (when compared to unoptimized version of the plugin) will take additional .8-1ms (on an average) to execute because of decompression that we do when reading the feature array from memory.
seunjeon lexiconDict is occupying heap of around 174 MB instead of existing 560 MB
Testing: All existing UT's and newly added UT's passed