Failures in the Yaml filter go undetected

[Mihai Nita]

There are multiple failures that go under the radar in the yaml filter.

Go to the full console output of an okapi-snapshot build and search for "FAIL:" (no quotes) (for instance https://okapi.ci.cloudbees.com/job/okapi-snapshot/459/consoleFull)

Looks like this is the output of some System.err.println calls in YamlParserTest.java

• In general reporting should use a logger, not System.out or System.err
• Failures in unit test should not just log, but should use the unit assert, and really fail

[Chase Tingley]

@jhargrave could probably provide context, but my guess about what's going on here is that these are alternate list syntaxes that snakeyaml doesn't support. We don't want to fail our own build because of that, although the FAIL message doesn't provide much value either. Probably convert them to real unittests, and then mark @Ignore?

[Mihai Nita]

I agree than real tests and @Ignore (an a comment why) would be better (and a less noisy build output :-)

Mihai

[Mihai Nita]

Easy way to find "chatty" modules or test units:

Edit okapi/deployment/logbind-logback/src/main/resources/logback.xml and change this:

    <root level="INFO">

to this:

    <root level="OFF">

Then run mvn -q install

Everything you see on screen is produced by some form of System.out.print / System.err.print, or some SLF4J binding problem

[Mihai Nita]

Other than filters/yaml (the worst offender :-), connectors/deepl uses System.err.println in 10 test units.

And I think there are some problems with filters/pdfbox using log4j. I am digging a bit into that.

Want me to file separate bugs?

[Mihai Nita]

I have changed the print to logging, but I have still left that in place.

Options:

• This is not yet implemented, it is normal to fail for now: create issue / feature, disable the test (with @Ignore) and add a TODO(issue_number)
• This is a real bug, this should not fail: create issue if there is none, add a TODO(issue_number). Disable the test or keep in place (?)
• This is expected to fail, and if it does not fail it is an error: use @Test(expected = <ExpectedException>.class), see https://github.com/junit-team/junit4/wiki/exception-testing

[Mihai Nita]

Fixed by commit 678c8a743d

[Mihai Nita]

Disabled test (with @Ignore), as decided in teleconference (https://groups.google.com/forum/#!topic/okapi-devel/H-mAsJtTh2I)