Detect when the fail exception was caught in the test
Sometimes a test can catch the
pytest.fail.Exception itself which makes it look like the test was fine while it did really time out.
One improvement is to use an exception which is a subclass of
BaseException instead. Then in the
pytest_runtest_call hook catch this exception and change it to a
Another improvement is to mark the test item as timed out in the signal handler. Then e.g.
pytest_runtest_call call can detect when the test cheated and caught the exception, after which it can fail it anyway with an appropriate message.