load_all/dump_all do not work well with streams.
I have a Python system that sends Yaml records around a networked system.
pyyaml has been working very well for this, but I had to work around a feature defect in load_all to do it.
Consider the following code snippet:
from __future__ import absolute_import, division, print_function, unicode_literals import time import yaml # Setting either or both of these to true makes no difference to the results. IS_SAFE = False USE_SLEEP = False load_all = yaml.safe_load_all if IS_SAFE else yaml.load_all dump_all = yaml.safe_dump_all if IS_SAFE else yaml.dump_all def send(): for i in range(3): print('send', i) yield i if USE_SLEEP: time.sleep(0.5) for i in load_all(dump_all(send())): print('receive', i) # Result: # send 0 # send 1 # send 2 # receive 0 # receive 1 # receive 2
For a network transaction, we obviously want the sends and receives to be interleaved.
In my code, I simply handle it by using safe_dump/safe_load, and synthesizing my own Yaml record separators. Pretty easy!
I did this a few months ago... I seem to remember that at the time I did get pyyaml to be a bit more sensible and not try to suck up its generators, but then there was a secondary issue where it waits to send the separator after a record until the next record arrives, which means of course that your last record in a series doesn't appear until you send more data, but I can't find my notes on this. :-(