Tests failing

[Alisson Patricio]

I'm running Python 3.4.0, Mongo 2.4.10 and testing using:

python3.4 -m unittest discover

I fixed a few failing tests here: https://bitbucket.org/alisson/asyncio-mongo/commits/b0b08744112b124241fdefc6dd0ea46dbddf1c0c

But I'm struggling with the problem that each time I run the all tests I get different errors. Without changing a code between tests.

We do I start to look so I can fix all tests?

On the first time I got:

./usr/src/asyncio-mongo/tests/test_collection.py:49: DeprecationWarning: Please use assertTrue instead.
  self.assert_(isinstance(self.db.test, Collection))
./usr/src/asyncio-mongo/tests/test_collection.py:84: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(ix, "hello_world")
.....F....{'lulz': 123, '_id': ObjectId('53488de0fa8c9101395b312f'), 'oh': 'hai'}
..................
======================================================================
FAIL: test_index_info (tests.test_collection.TestCollection)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/asyncio-mongo/tests/base.py", line 32, in inner
    run_now(func, *args, **kwargs)
  File "/usr/src/asyncio-mongo/tests/base.py", line 26, in run_now
    asyncio.Task(call_maybe_yield(func, *args, **kwargs))
  File "/usr/lib/python3.4/asyncio/base_events.py", line 208, in run_until_complete
    return future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 243, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 281, in _step
    result = coro.send(value)
  File "/usr/src/asyncio-mongo/tests/base.py", line 19, in call_maybe_yield
    rv = yield from rv
  File "/usr/src/asyncio-mongo/tests/test_collection.py", line 149, in test_index_info
    self.assertEqual([("hello", -1), ("world", 1)], list(ix_info["hello_-1_world_1"]))
AssertionError: Lists differ: [('hello', -1), ('world', 1)] != [('world', 1), ('hello', -1)]

First differing element 0:
('hello', -1)
('world', 1)

- [('hello', -1), ('world', 1)]
+ [('world', 1), ('hello', -1)]

----------------------------------------------------------------------
Ran 30 tests in 1.824s

FAILED (failures=1)
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf539a8>
/usr/lib/python3.4/importlib/_bootstrap.py:2127: ImportWarning: sys.meta_path is empty
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf53948>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf53b88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf53ca8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf85408>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf85468>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf854c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf85528>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf85588>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dacf855e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac478ee8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac478f48>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac478fa8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485048>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4850a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485108>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4851c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485228>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485288>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485348>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4853a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485408>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485468>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4854c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485528>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485588>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4855e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485648>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4856a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485708>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485768>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4857c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485828>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485888>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4858e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485948>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac4859a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485a08>
sys:1: ResourceWarning: unclosed <socket object at 0x7f9dac485a68>

On the second try:

./usr/src/asyncio-mongo/tests/test_collection.py:49: DeprecationWarning: Please use assertTrue instead.
  self.assert_(isinstance(self.db.test, Collection))
./usr/src/asyncio-mongo/tests/test_collection.py:84: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(ix, "hello_world")
..........{'lulz': 123, 'oh': 'hai', '_id': ObjectId('53488de8fa8c910140f857d6')}
..F...............
======================================================================
FAIL: test_MongoOperations (tests.test_objects.TestMongoObjects)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/asyncio-mongo/tests/base.py", line 32, in inner
    run_now(func, *args, **kwargs)
  File "/usr/src/asyncio-mongo/tests/base.py", line 26, in run_now
    asyncio.Task(call_maybe_yield(func, *args, **kwargs))
  File "/usr/lib/python3.4/asyncio/base_events.py", line 208, in run_until_complete
    return future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 243, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 281, in _step
    result = coro.send(value)
  File "/usr/src/asyncio-mongo/tests/base.py", line 19, in call_maybe_yield
    rv = yield from rv
  File "/usr/src/asyncio-mongo/tests/test_objects.py", line 48, in test_MongoOperations
    self.assertEqual(result.get('_id'), doc.get('_id'))
AssertionError: None != ObjectId('53488de8fa8c910140f857d9')

----------------------------------------------------------------------
Ran 30 tests in 1.742s

FAILED (failures=1)
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e562e948>
/usr/lib/python3.4/importlib/_bootstrap.py:2127: ImportWarning: sys.meta_path is empty
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e562e9a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e562eb88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e562eca8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660408>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660468>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e56604c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660528>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660588>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e56605e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660648>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e56606a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660708>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660768>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e56607c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660828>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e56608e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660948>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e56609a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660a68>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660ac8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660b28>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660b88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660be8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361d68>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361dc8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e5660c48>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361e28>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361e88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361ee8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361f48>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e4361fa8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f0048>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f00a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f0108>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f0168>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f01c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f0228>
sys:1: ResourceWarning: unclosed <socket object at 0x7f14e42f0288>

Third time:

./usr/src/asyncio-mongo/tests/test_collection.py:49: DeprecationWarning: Please use assertTrue instead.
  self.assert_(isinstance(self.db.test, Collection))
./usr/src/asyncio-mongo/tests/test_collection.py:84: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(ix, "hello_world")
..........{'_id': ObjectId('53488f25fa8c910155e0220d'), 'oh': 'hai', 'lulz': 123}
...........F......
======================================================================
FAIL: test_MultipleCursorIterations (tests.test_queries.TestMongoQueries)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/src/asyncio-mongo/tests/base.py", line 32, in inner
    run_now(func, *args, **kwargs)
  File "/usr/src/asyncio-mongo/tests/base.py", line 26, in run_now
    asyncio.Task(call_maybe_yield(func, *args, **kwargs))
  File "/usr/lib/python3.4/asyncio/base_events.py", line 208, in run_until_complete
    return future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 243, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 281, in _step
    result = coro.send(value)
  File "/usr/src/asyncio-mongo/tests/base.py", line 19, in call_maybe_yield
    rv = yield from rv
  File "/usr/src/asyncio-mongo/tests/test_queries.py", line 30, in test_MultipleCursorIterations
    self.assertEqual(len(res), 450)
AssertionError: 0 != 450

----------------------------------------------------------------------
Ran 30 tests in 1.791s

FAILED (failures=1)
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba2d9a8>
/usr/lib/python3.4/importlib/_bootstrap.py:2127: ImportWarning: sys.meta_path is empty
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba2d948>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba2db88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba2dca8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f408>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f468>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f4c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f528>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f588>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f5e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f648>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f6a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f708>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f768>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f7c8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f828>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f8e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f948>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5f9a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fa68>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fac8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fb28>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fb88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fbe8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fc48>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fca8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fd08>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fd68>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fdc8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fe28>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fe88>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5fee8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f215ba5ff48>
sys:1: ResourceWarning: unclosed <socket object at 0x7f2158eea588>
sys:1: ResourceWarning: unclosed <socket object at 0x7f2158eea5e8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f2158eea648>
sys:1: ResourceWarning: unclosed <socket object at 0x7f2158eea6a8>
sys:1: ResourceWarning: unclosed <socket object at 0x7f2158eea708>
sys:1: ResourceWarning: unclosed <socket object at 0x7f2158eea768>

[A. Jesse Jiryu Davis]

To fix test_index_info, I think you need to sort the results before comparing them. The test was relying on luck to succeed. I can't tell what's wrong with test_MongoOperations or test_MultipleCursorIterations just by looking at them, there may be serious bugs there. =)

The resource warnings should eventually get cleaned up too, IMO. Consider a Pool.__del__ method like PyMongo's:

https://github.com/mongodb/mongo-python-driver/blob/master/pymongo/pool.py#L526

[Don Brown]

Well, thanks to all the help, the tests now pass. Unfortunately, I couldn't figure out how to get rid of those resource warnings. Looks like asyncio_redis had a similar issue, only their fix freezes the tests for me, no matter how I tried to restructure them. Will close this for now as the tests do consistently pass.