Anonymous avatar Anonymous committed fbaa333

Unless --cavalier is given, lint loaded tests for sanity.

Comments (0)

Files changed (6)

   are now created where the system wants them to be created (but you can set
   the `TMP` environment variable to "." if you really want them to be created
   in the current directory.)
+* `py-falderal` now complains and stops early if there were no documents
+  specified, no functionality definitions were found in any of the documents,
+  no implementations were found for some defined functionality, or if no
+  tests were found in any of the documents, unless `--cavalier` is passed on
+  the command line.
 
 Version 0.8 "Ukranian Village" (current released version):
 
 having both kinds of expectations would allow non-deterministic combinations
 of the two to be matched.)
 
-### lint test documents by default
-
-Unless some command line option is given (call it `--cavalier`, maybe,) if
-any of the following is true, we should abort quickly with an error message:
-
--   Parsing the given documents resulted in no tests.  (This is implied by:
-    there were no documents given.)
--   For one or more functionalities, there were no implementations specified.
-
 ### catch KeyboardInterrupt, report on all results so far
 
 When a test suite takes a long time to run, it would be very nice if

src/falderal/driver.py

 from falderal.objects import Document, FalderalSyntaxError
 
 
+##### Exceptions #####
+
+class FalderalLintingError(ValueError):
+    pass
+
+
+##### Main #####
+
 def main(args):
     parser = OptionParser()
 
     parser.add_option("-b", "--substring-error",
                       action="store_true", default=False,
                       help="match expected errors as substrings")
+    parser.add_option("--cavalier",
+                      action="store_true", default=False,
+                      help="don't perform sanity linting before running tests")
     parser.add_option("-d", "--dump",
                       action="store_true", default=False,
                       help="print out info about parsed tests, don't run them")
             documents.append(Document.load(filename))
         for document in documents:
             tests += document.parse_blocks_to_tests(functionalities)
-    except FalderalSyntaxError as e:
+
+        if not options.cavalier:
+            if not documents:
+                raise FalderalLintingError("No test documents were specified")
+            if not functionalities:
+                raise FalderalLintingError(
+                    "No functionalities were found in any of the test documents"
+                )
+            for name in functionalities:
+                functionality = functionalities[name]
+                if not functionality.implementations:
+                    raise FalderalLintingError(
+                        "No implementations were found for the functionality '%s'" %
+                        functionality.name
+                    )
+            if not tests:
+                raise FalderalLintingError(
+                    "No tests were found in any of the test documents"
+                )
+
+    except (FalderalSyntaxError, FalderalLintingError) as e:
         # if options.show_full_exception: do that, else
         sys.stderr.write('%s: %s\n' % (e.__class__.__name__, str(e)))
         return 1
 
-    # XXX lint: check for no tests, or no implementations of a functionality
-    # that is being tested, or a functionality that is not being tested, and
-    # break with an error unless some option to suppress this is present
 
     if options.dump:
         print "Functionalities:"
 bin/falderal -t || exit 1
 
 cd tests
-for TEST in test1 test2 test3 test4 test5 test9 test10 test11 test-utf8 test-crlf; do
+
+FIRST_TESTS="
+test1 test2 test3 test4 test5 test9 test10 test11 test-utf8 test-crlf
+"
+for TEST in ${FIRST_TESTS}; do
+    echo ${TEST}...
+    ../bin/falderal --cavalier ${TEST}.markdown > ${TEST}.actual 2>&1
+    diff -u ${TEST}.expected ${TEST}.actual || exit 1
+done
+
+# tests for listing
+LINTING_TESTS="test-no-tests"
+for TEST in ${LINTING_TESTS}; do
     echo ${TEST}...
     ../bin/falderal ${TEST}.markdown > ${TEST}.actual 2>&1
     diff -u ${TEST}.expected ${TEST}.actual || exit 1

tests/test-no-tests.expected

+FalderalLintingError: No functionalities were found in any of the test documents

tests/test-no-tests.markdown

+Falderal Test for No Tests Found
+--------------------------------
+
+Since these blocks are not indented correctly, `py-falderal` should not
+load tests from them.  (Ideally it should produce warnings though.)
+
+  -> Functionality "Cat" is implemented by
+  -> shell command "python cat.py"
+
+Cat cats.
+
+| meow
+= meow
+
+        | purr
+        | prrr
+        | prreow
+        = purr
+        = prrr
+        = prreow
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.