QuickCheck tests fail

Issue #2 resolved
Alexander Dorofeev
created an issue

{{{ single insert: [Failed]

Falsifiable with seed 7934048139634871320, after 1 tests. Reason: Exception: 'CouchError (Just 409) "Conflict: Document update conflict."' }}}

On each prop test.

Comments (25)

  1. John Lenz repo owner
    • changed status to open

    Ok, I have all the tests running, but I was working still against the couchdb I run on my debian stable box. It is kind of late tonight so I don't think I will do any more tonight, but tomorrow (Wednesday) I will install couchdb on my desktop machine (Ubuntu oneiric) and see if I can reproduce the error.

    Also, figure out a way of passing the server name on the test command line and default it to localhost or something like that.

  2. Alexander Dorofeev reporter

    Just two cents:

        single insert: [Failed]
    >>> Falsifiable with seed 6511397483682974178, after 11 tests. Reason: Exception: 'CouchError (Just 409) "Conflict: Document update conflict."'

    Test fails after random number of iteractions.

  3. John Lenz repo owner

    Ok, I installed couchdb on my desktop and now am getting the same error. I suspect it is because quickcheck runs sets in parallel... and when I went over the network it was slow enough that none actually ran in parallel...

  4. John Lenz repo owner

    No, what I think is happening is that the object keys are being reused at the same time so two threads get different revisions. For example, adding some prints this is what I get...

    Clearing "otest1_1"
      Revision "9-8261894d0bbf25ade912865d8daa5f8b"
    Clearing "otest1_0"
      Revision "3-f6dd2c41351754c2af1800ebcdb4694f"
    Clearing "otest1_-1"
      Revision "7-4ce0af5d23d1c7225bfdd0935a2c9bfc"
    Adding "otest1_1"
        single insert: [Failed]
    Falsifiable with seed 8281320049974887153, after 1 tests. Reason: Exception: 'CouchError (Just 409) "Conflict: Document update conflict."'
             Properties  Total      
     Passed  0           0          
     Failed  1           1          
     Total   1           1          
    $ curl http://localhost:5984/testcouchenum/otest1_1

    Notice how the revision for otest1_1 is different than the revision when it was cleared... something else came along and added otest1_1 between the clear and the attempt an inserting...

  5. John Lenz repo owner

    Hmm, I guess I do too but only about 1 out of every 10 times I run the tests. 9 out of 10 pass with no errors and really occasionally I get the error conflict error. I have probably run it 50 times by now :)

  6. John Lenz repo owner

    Hmm, test-framework itself must be forking because this is what the couchdb log looks like when a conflict occurs

    [Thu, 22 Dec 2011 03:54:57 GMT] [info] [<0.12319.1>] - - 'HEAD' /testcouchenum/otest_1_-800 200
    [Thu, 22 Dec 2011 03:54:57 GMT] [info] [<0.12319.1>] - - 'DELETE' /testcouchenum/otest_1_-800?rev=3-b42c2d45b7335fe0a0164111bbedad19 200
    [Thu, 22 Dec 2011 03:54:57 GMT] [info] [<0.12319.1>] - - 'PUT' /testcouchenum/otest_1_-1 201
    [Thu, 22 Dec 2011 03:54:57 GMT] [info] [<0.12320.1>] - - 'PUT' /testcouchenum/otest_1_-1 409

    Notice the last two entries have different process ids (12319 versus 12320) so two different processes are trying to run a test at the same time. This is really weird because this is now only a single action....

  7. Alexander Dorofeev reporter

    CouchDB uses mochiweb (with pool). [<0.12320.1>] - is just erlang process id.

    When you update doc - CouchDB update database (and views). This takes time and does in the background. 200/OK sent to you just after verify data and consistency (revision).

    As we see - Couch accept new request 'PUT' /testcouchenum/otest_1_-1 with new process.

    So. As I said earlier - I think, it's just "time" problem. Try to add delay to tests.

  8. John Lenz repo owner

    Well, I narrowed down the error. It looks like it really is a bug not just a problem with the tests, perhaps a bug in http-enumerator. See the new test I created, which fails 95% of the time.

    It happens when you call couchDelete followed by a single call couchPut (see the new test I added). The single call to couchPut causes two PUTs to go out over the wire (verified with wireshark, two puts go out on two different ports) If you add any call like a call to couchRev for something else, it works 100% of the time.

    (This was why it only showed up sometimes in the bigger tests, since it only happened if couchDelete was followed by couchPut.)

    Not sure if it is a problem in http-enumerator or our use of it, but somehow a single call to couchPut is generating two actual HTTP requests.

  9. John Lenz repo owner

    Here is what happens from a dump in wireshark of the new conflicttest, in order of timestamps

    • Port 52099 > 5984 DELETE /testcouchenum/conflicttest?rev=109-a...
    • Port 5984 > 52099 HTTP/1.1 200 OK
    • Port 52099 > 5984 PUT /testcouchenum/conflicttest
    • Port 52100 > 5984 TCP SYN
    • Port 5984 > 52100 ACK
    • Port 52100 > 5984 PUT /testcouchenum/conflicttest
    • Port 5984 > 52100 ACK
    • Port 5984 > 52099 One packet of a 201 response
    • Port 52099 > 5984 RST (Reset connection, second packet of the 201 response never sent)
    • Port 5984 > 52100 HTTP 1.1 409 Conflict

    Somehow http-enumerator is starting a PUT request and then sending a RST packet before reading the response. It then sends the PUT again on a different port.

  10. Alexander Dorofeev reporter

    As I see, Michael buried up to their ears in conduit and dependent packs and he probably will not do anything. I can send pull request to him (or you - if you like github). But, as I think it will require testing.

  11. John Lenz repo owner

    Actually, I am traveling for Christmas to my parents. I won't be back home until Tuesday, so can't test anything till then. Also, I plan on uploading the new version of couchdb-enumerator to hackage Tuesday or Wednesday... we can think if there is anything else to add or change...

  12. Alexander Dorofeev reporter

    Just two little (and handy - I'm using couchdb-enumerator in production) functions in Generic. And some docs for views. I will send pull request to you.

    Sorry about line endings. I store files with unix line endings. But Mercurial always sends system line endings. I run servers under linux, but prefers Windows as desctop system.

    After that, I think, we can relax and let's see - what will come with a conduit.

  13. John Lenz repo owner

    Pulled your changes. I assume the change from RequestBodyEnumChunked to RequestBodyLBS was because of aeson 0.5? Also, I see you updated the lower bound to 0.5 on aeson. Did enough change that we can't support both 0.4 and 0.5?

    Also, it looks like conduit is progressing along faster than thought. I plan on switching this package (and all my code) over to conduits once it lands in warp and yesod, which looks close in the future. But I think I will at least wait and not do anything until http-conduit is declared an a released state and uploaded to hackage.

    I do plan on releasing this version of couchdb-enumerator tomorrow (Thrusday)

  14. Alexander Dorofeev reporter

    Yep. Aeson 0.5 uses text instead blaze-builder. Current (non-conduit) version of http-enumerator uses blaze-builder. But future version will be use text. According to my observations, blaze-builder is used less and less.

  15. John Lenz repo owner

    Ok, uploaded to hackage. Thanks for the help.

    As far as conduits, I plan on converting over now that http-conduit is out, but won't do anything until probably the middle of January. I want to wait until WAI and yesod are converted over and released before I make a serious push to convert my code to conduits. Also, would be nice to make sure the API is mostly stable... although from what I read on the mailing list and such it seems like it is stable already.

  16. Log in to comment