Commits

Matt Oswald committed 87c6547

updates for Log and Warn, as well as calling out xUnit++ runs being concurrent

Comments (0)

Files changed (5)

Assertions.wiki

-== Assert and Check ==
-
-xUnit++ has a robust suite of test assert methods. While most C++ testing frameworks use preprocessor macros to implement only a few checks, xUnit++ provides two class instances with many built-in methods. And since these tests are not implemented with macros, you get much nicer feedback while editing (assuming your editor can provide such detail) and compiling.
-
-The two objects are {{{Assert}}} and {{{Check}}}. With one exception, they both offer the same check methods. The main difference between the two is that failed check methods called from the {{{Assert}}} object will immediately halt the test, while failed {{{Check}}} check methods will be reported after the test runs to completion.
-
-{{{
-#!c++
-Check.Fail();   // failure is logged, test continues
-Assert.Fail();  // stops the test here
-Check.Fail();   // test never executes this line
-}}}
-
-==== The Methods ====
-
-As stated before, {{{Assert}}} and {{{Check}}} share the same check methods, with one exception: {{{Assert}}} offers {{{Throws}}} while {{{Check}}} does not.
-
-* {{{Contains}}} asserts that a container contains some value. Overloads exist for raw strings and {{{std::string}}}.
-* {{{ContainsPred}}} is an alternative form of {{{Contains}}} which takes a predicate instead of a value.
-* {{{DoesNotContain}}}} asserts that a container does not contain some value. Overloads exist for raw strings and {{{std::string}}}.
-* {{{DoesNotContainPred}}} is an alternative form of {{{DoesNotContain}}} which takes a predicate instead of a value.
-* {{{DoesNotThrow}}} asserts that the given code does not throw any exceptions. Anything that resolves to the equivalent of {{{void (*)()}}} will work.
-* {{{Equal}}} tests object values for equality. There are overloads for raw strings, {{{std::string}}}, floating point types (with //precision//, instead of //tolerance//), and iterator ranges.
-* {{{Empty}}} will assert that the container object is empty. {{{TContainer::empty()}}} will be used if it exists, otherwise the container will be converted to a range using {{{std::begin}}} and {{{std::end}}}.
-* {{{Fail}}} will automatically fail the test.
-* {{{False}}} will fail the test if the supplied parameter resolves to {{{true}}}.
-* {{{InRange}}} checks that the given value fits within the range [{{{min}}}, {{{max}}}).
-* {{{NotEmpty}}} is the opposite of {{{Empty}}}.
-* {{{NotEqual}}} is the opposite of {{{Equal}}}.
-* {{{NotInRange}}} checks that the given value does not fit within the range [{{{min}}}, {{{max}}}).
-* {{{NotNull}}} checks that the value is not equal to {{{nullptr}}}.
-* {{{NotSame}}} verifies that the supplied objects are not the same //object instance//.
-* {{{Null}}} checks that the value is equal to {{{nullptr}}}.
-* {{{Same}}} checks that the supplied objects are the same //object instance//.
-* {{{Throws}}} is an {{{Assert}}}-only member that checks that the supplied code throws a specific exception. If it succeeds, it will return the exception to your test.
-{{{
-#!c++
-auto ex = Assert.Throws<std::exception>([]() { throws std::runtime_error(""); });
-}}}
-
-==== Printing Values ====
-
-Some methods may try to print the values of the objects using {{{to_string}}} with argument-dependent lookup (Koenig lookup). To take advantage of this, implement a {{{to_string}}} function within your object's namespace.
-
-{{{
-#!c++
-namespace NS
-{
-    class A {};
-    std::string to_string(const A &a) { return "A"; }
-}
-}}}
-
-==== Custom Messages ====
-
-If you want to add a custom message to failing tests, use the overloaded {{{operator <<}}}.
-
-{{{
-#!c++
-Assert.Fail() << "This is an example message " << some_value;
-}}}
-
-==== File and Line Info ====
-
-Normally, when a test fails the test runner will report the file and line number for the test itself. This is typically sufficient for most tests, as tests should really only assert one thing at a time. However, if you want to be specific about which check failed, each check method accepts an optional {{{xUnitpp::LineInfo}}} object. The easiest way to do this is to pass the {{{LI}}} macro as the final parameter to the test.
-
-{{{
-#!c++
-Assert.Equal(0, 1, LI) << "0 is never equal to 1!";
-}}}
 == How does xUnit++ compare to ...? ==
 
-xUnit++ is far from the only unit testing framework for C++, and is certainly not perfect. However, it stands up quite well when compared to other systems. Most systems utilize preprocessor macros to implement a small subset of checks, and most also require you to provide your own test runner.
+xUnit++ is far from the only unit testing framework for C++, and is certainly not perfect. However, it stands up quite well when compared to other systems. Most systems utilize preprocessor macros to implement a small subset of checks, and most also require you to provide your own test runner. Perhaps most importantly, only xUnit++ uses {{{std::async}}} to run tests concurrently!
 
 |=|= xUnit++ |= Boost UTF |= Google Test |= MSTest |= UnitTest++ |
 | | | | | | |
 |= Implementation | Static Library | Any! (See Note 1.) | User-provided (See Note 2.) | Requires Visual Studio 2012+ | Static Library |
 |= Test Implementation | Member functions | Preprocessor macros | Preprocessor macros | Static member functions | Preprocessor macros |
 |= Test Runner | Provided | User-supplied | User-supplied | Visual Studio | User-supplied |
+|= Threading | Uses {{{std::async}}} to run tests concurrently! | //n/a// | //n/a// | //n/a// | //n/a//  |
 |= Fatal Failures | {{{Assert.Equal}}} | {{{BOOST_REQUIRE_EQUAL}}} | {{{ASSERT_EQ}}} | {{{Assert::AreEqual}}} | {{{CHECK_EQUAL}}} |
 |= Failures | {{{Check.Equal}}} | {{{BOOST_CHECK_EQUAL}}} | {{{EXPECT_EQ}}} | //n/a// | //n/a// |
-|= Warnings | //n/a// | {{{BOOST_WARN_EQUAL}}} | //n/a// | //n/a// | //n/a// |
+|= Warnings | {{{Warn.Equal}}} | {{{BOOST_WARN_EQUAL}}} | //n/a// | //n/a// | //n/a// |
 |= Per-Check Messages | Use an overloaded stream operator.\\{{{Assert.Equal(a, b) << "message"}}} | Append {{{_MESSAGE}}} to any check macro.\\{{{BOOST_REQUIRES_EQUAL_MESSAGE(a, b, "message")}}} | Use an overloaded stream operator.\\{{{ASSERT_EQ(a, b) << "message"}}} | Additional parameter\\{{{Assert::AreEqual(a, b, "message")}}} | //na// |
+|= Logging | {{{Log.<level> << "msg" << value;}}} | {{{BOOST_TEST_MESSAGE("msg" << value);}}} | //n/a// | //n/a// | //n/a// |
 |= Attributes | Up to 8 per test. | //n/a// | //n/a// | Unlimited per module, fixture, or method. | //n/a// |
 |= Timed Tests | Halt long-running tests. | //n/a// | //n/a// | //n/a// | Fail tests that went over time limit. |
 | | | | | | |
 [[InstallAndSetup.wiki|//Installation and Setup//]]\\
 [[Tests.wiki|//Tests//]]\\
 [[SuitesAndAttributes.wiki|//Suites and Attributes//]]\\
-[[Assertions.wiki|//Assert and Check//]]\\
+[[TestEvents.wiki|//Test Events: Assert, Check, Warn, and Log//]]\\
 [[RunningTests.wiki|//Running tests//]]\\
 [[Compare.wiki|//How does xUnit++ compare to ...?//]]
 
+== Assert and Check ==
+
+xUnit++ has a robust suite of test assert methods. While most C++ testing frameworks use preprocessor macros to implement only a few checks, xUnit++ provides two class instances with many built-in methods. And since these tests are not implemented with macros, you get much nicer feedback while editing (assuming your editor can provide such detail) and compiling.
+
+There are three ways to assert test events: {{{Assert}}}, {{{Check}}}, and {{{Warn}}}. With one exception, they offer the same check methods. {{{Assert}}} will halt the test immediately if any check fails, {{{Check}}} will log the failure but continue, and {{{Warn}}} will not cause the test to  fail by itself, but the "error" will be logged.
+
+{{{
+#!c++
+Warn.Fail();    // warning message logged, test is not a failure (yet)
+Check.Fail();   // failure is logged, test is marked as failing, test execution continues
+Assert.Fail();  // stops the test here
+Check.Fail();   // test never executes this line
+}}}
+
+==== The Methods ====
+
+As stated before, the test objects share the same check methods, with one exception: {{{Assert}}} offers {{{Throws}}} while {{{Check}}} and {{{Warn}}} do not.
+
+* {{{Contains}}} asserts that a container contains some value. Overloads exist for raw strings and {{{std::string}}}.
+* {{{ContainsPred}}} is an alternative form of {{{Contains}}} which takes a predicate instead of a value.
+* {{{DoesNotContain}}}} asserts that a container does not contain some value. Overloads exist for raw strings and {{{std::string}}}.
+* {{{DoesNotContainPred}}} is an alternative form of {{{DoesNotContain}}} which takes a predicate instead of a value.
+* {{{DoesNotThrow}}} asserts that the given code does not throw any exceptions. Anything that resolves to the equivalent of {{{void (*)()}}} will work.
+* {{{Equal}}} tests object values for equality. There are overloads for raw strings, {{{std::string}}}, floating point types (with //precision//, instead of //tolerance//), and iterator ranges.
+* {{{Empty}}} will assert that the container object is empty. {{{TContainer::empty()}}} will be used if it exists, otherwise the container will be converted to a range using {{{std::begin}}} and {{{std::end}}}.
+* {{{Fail}}} will automatically fail the test.
+* {{{False}}} will fail the test if the supplied parameter resolves to {{{true}}}.
+* {{{InRange}}} checks that the given value fits within the range [{{{min}}}, {{{max}}}).
+* {{{NotEmpty}}} is the opposite of {{{Empty}}}.
+* {{{NotEqual}}} is the opposite of {{{Equal}}}.
+* {{{NotInRange}}} checks that the given value does not fit within the range [{{{min}}}, {{{max}}}).
+* {{{NotNull}}} checks that the value is not equal to {{{nullptr}}}.
+* {{{NotSame}}} verifies that the supplied objects are not the same //object instance//.
+* {{{Null}}} checks that the value is equal to {{{nullptr}}}.
+* {{{Same}}} checks that the supplied objects are the same //object instance//.
+* {{{Throws}}} is an {{{Assert}}}-only member that checks that the supplied code throws a specific exception. If it succeeds, it will return the exception to your test.
+{{{
+#!c++
+auto ex = Assert.Throws<std::exception>([]() { throws std::runtime_error(""); });
+}}}
+
+==== Printing Values ====
+
+Some methods may try to print the values of the objects using {{{to_string}}} with argument-dependent lookup (Koenig lookup). To take advantage of this, implement a {{{to_string}}} function within your object's namespace.
+
+{{{
+#!c++
+namespace NS
+{
+    class A {};
+    std::string to_string(const A &a) { return "A"; }
+}
+}}}
+
+==== Custom Messages ====
+
+If you want to add a custom message to failing tests, use the overloaded {{{operator <<}}}.
+
+{{{
+#!c++
+Assert.Fail() << "This is an example message " << some_value;
+}}}
+
+==== File and Line Info ====
+
+Normally, when a test fails the test runner will report the file and line number for the test itself. This is typically sufficient for most tests, as tests should really only assert one thing at a time. However, if you want to be specific about which check failed, each check method accepts an optional {{{xUnitpp::LineInfo}}} object. The easiest way to do this is to pass the {{{LI}}} macro as the final parameter to the test.
+
+{{{
+#!c++
+Assert.Equal(0, 1, LI) << "0 is never equal to 1!";
+}}}
+
+==== Extra Logging ====
+
+If you need more logging output within the tests, use the {{{Log}}} object. Three levels of logging are implemented: Debug, Information, and Warning:
+
+{{{
+#!c++
+Log.Debug << "This is a debug-level message: " << some_value;
+Log.Info << "This is an info-level message: " << some_value;
+Log.Warn << "This is a warning-level message, and will automatically mark the running test has having a warning status. " << some_value;
+}}}
 
 There are two types of tests: Facts and Theories. Facts are one-shot tests that assert (hopefully) one thing. Theories are tests that either can not be completely verified with a single assertion, or need to have multiple inputs verified separately.
 
+xUnit++ supports running tests concurrently, and will also randomize the test execution with each run. Once a Theory is declared, all instances of that Theory are treated as individual tests.
+
 ==== Facts ====
 
 {{{