If latest test passed but recent test failed, PR marked as FAILED.

Issue #12356 closed
Andrew Somerville
created an issue

Screenshot from 2016-02-10 14.44.47.png

Comments (8)

  1. Kyle Anderson

    @Andrew Somerville It is a little confusing. Perhaps a different icon to display some failed tests but most recent is successful. For some Users it would be valid to differentiate between a PR with no failed builds vs. a PR with some failures in its history.

    Curious what their intention was. Hopefully somebody from the team with elaborate.

  2. Daniel Tao staff

    The idea here is that you may have multiple builds and if any one of them fails then, as a whole, the commit they're associated with does not "pass" all builds. For instance the Bitbucket team has separate builds for Python and JavaScript tests, so we see the failure icon if either of those builds fails.

    The build status API does allow you to update the status of an existing build. The presence of three separate build icons in the screenshot shows that Shippable is essentially posting a new build status (each w/ a unique key) for each run, when—if I'm understanding correctly—they should probably be updating the existing build if it's being re-run.

    We can reach out to Shippable and see if this is something they can fix.

  3. Matias Goldberg

    Is there a way build status can be changed from the web interface?

    We're using Jenkins and Jenkins sometimes fails because of an MSVC-Jenkins compatibility problem. We think we've fixed it (workaround it) but it is extremely annoying that these commits are marked as FAILED when they should be removed, and Jenkins plugin doesn't appear to support updating the status from its interface.

    I understand that the API is supposed to be "if one test failed, then some platform is failing and mark the commit as failure" (i.e. Windows passes, OSX passes, Linux fails), but it leaves us helpless in Bitbucket when the CI server breaks, with no way to fix it from Bitbucket's side.

  4. Marcus Bertrand staff

    We've recently made numerous changes to our UI/Integrations that should have resolved or changed the characteristics of this issue. If you're still seeing any issues across the latest major versions of any browsers, please raise a new issue on this issue tracker.

    For any feature requests around the build status feature, please raise a new issue here as a feature request and we'll comment on it.

    Cheers,
    Marcus

  5. Log in to comment