OLD | NEW |
1 # Expect Tests | 1 # Expect Tests |
2 Expect Tests is a test framework which: | 2 Expect Tests is a test framework which: |
3 * Is parallel by default | 3 * Is parallel by default |
4 * Collects coverage information by default | 4 * Collects coverage information by default |
5 * Allows easy test-case generation | 5 * Allows easy test-case generation |
6 * Is compatible with unittest | 6 * Is compatible with unittest |
7 * Provides easy test globbing and debugging | 7 * Provides easy test globbing and debugging |
8 | 8 |
9 You can run the test suite with 'nosetests expect_tests/test' in the root | 9 You can run the test suite with 'nosetests expect_tests/test' in the root |
10 directory. | 10 directory. |
11 | 11 |
12 ## Fiddly details | 12 ## Quick user manual |
13 | 13 |
14 If you have a package that you'd like to exclude from expect_tests' search, | 14 ### Writing tests |
15 simply define a variable `_expect_tests_stop_walk = True` in the package's | 15 Tests are subclasses of unittests.TestCase only. expect_tests looks for tests in |
16 `__init__.py` and expect_tests won't search further in that directory tree. | 16 files named like '*\_test.py'. The coverage information for file foo.py is only |
| 17 collected from tests located in 'tests/foo\_test.py'. |
| 18 |
| 19 ### Invocation |
| 20 The simplest expect_tests invocation is: |
| 21 |
| 22 expect_tests (list|test|train) <path> |
| 23 |
| 24 where <path> can point either to a Python (sub)package's directory, or to a |
| 25 directory containing Python packages. In the latter case, all tests in all |
| 26 packages in the directory will be considered. |
| 27 |
| 28 * list: just output the full list of tests on stdout |
| 29 * test: run the tests |
| 30 * train: run the test and update their expectations instead of checking |
| 31 against them. |
| 32 |
| 33 ### Filtering tests |
| 34 It is possible to run an action on a subset of test instead of all of them. This |
| 35 is achieved by appending a filter after the path specification: |
| 36 |
| 37 expect_tests (list|test|train) <path>:<filter glob> |
| 38 |
| 39 <filter glob> applies to the full test names, as output by 'list'. It does not |
| 40 apply to the package path. |
| 41 |
| 42 Example: Suppose you have the following structure: |
| 43 |
| 44 root/ |
| 45 root/package1 |
| 46 root/package1/__init__.py |
| 47 root/package1/foo.py |
| 48 root/package1/tests/__init__.py |
| 49 root/package1/tests/foo_test.py # contains test TestFoo.test\_feature |
| 50 root/package1/subpackage |
| 51 root/package1/subpackage/__init__.py |
| 52 root/package1/subpackage/subfoo.py |
| 53 root/package1/subpackage/tests/__init__.py |
| 54 root/package1/subpackage/tests/subfoo_test.py # contains TestSubFoo.test\_f
eature |
| 55 root/package2/... # with same structure as package1 |
| 56 |
| 57 Then (supposing the current directory is the parent of 'root/') |
| 58 |
| 59 $ expect_tests list root |
| 60 package1.tests.foo_test.TestFoo.test_feature |
| 61 package1.subpackage.tests.subfoo_test.TestSubFoo.test_feature |
| 62 package2.tests.foo_test.TestFoo.test_feature |
| 63 package2.subpackage.tests.subfoo_test.TestSubFoo.test_feature |
| 64 |
| 65 $ expect_tests list root/package1 |
| 66 package1.tests.foo_test.TestFoo.test_feature |
| 67 package1.subpackage.tests.subfoo_test.TestSubFoo.test_feature |
| 68 |
| 69 $ expect_tests list 'root:package1*' # less efficient than root/package1 |
| 70 package1.tests.foo_test.TestFoo.test_feature |
| 71 package1.subpackage.tests.subfoo_test.TestSubFoo.test_feature |
| 72 |
| 73 $ expect_tests list 'root/package1:*TestSubFoo*' |
| 74 package1.subpackage.tests.subfoo_test.TestSubFoo.test_feature |
| 75 |
| 76 |
| 77 ### Fine-tuning and advanced topics |
| 78 |
| 79 Having trouble debugging a test? You can use the 'debug' action instead of |
| 80 'test' to get a debugging prompt when entering tests. That way you can step |
| 81 through the code if necessary. |
| 82 |
| 83 You can make expect_tests ignore a subpackage by adding a .expect\_tests.cfg |
| 84 file in the directory containing the package, with the following content: |
| 85 |
| 86 [expect_tests] |
| 87 skip=packagetoignore1 |
| 88 packagetoignore2 |
| 89 |
| 90 Some Python code, like the Appengine sdk, requires some special setup to be able |
| 91 to work. In order to support that, you can create a .expect\_tests\_pretest.py |
| 92 file in the directory containing the top-level package containing tests. This |
| 93 code will be execfile'd just before any operation (list/run/train) in this |
| 94 directory. |
OLD | NEW |