Clean up the library dependency tree.

[Robert Massaioli]

Currently the libraries are an absolute mess. They are included in a pretty much random fashion that was required to get the code to compile. I should be spending my time making sure that the compile correctly and that each library only target_link_library's to the ones that it actually needs and that it would not get transitively.