OLD | NEW |
(Empty) | |
| 1 README: xdg-utils tests |
| 2 |
| 3 Created 6/27/2006 by Tom Whipple <tom.whipple@intel.com> |
| 4 |
| 5 |
| 6 RUNNING XDG-UTILS TESTS |
| 7 ----------------------- |
| 8 At a minimum, you must have your current directory be the same as the |
| 9 directory containing this README. Then, simply execute: |
| 10 |
| 11 $ ./testrun |
| 12 WARNING: guessed XDG_TEST_DIR to be /home/tw/portland/xdg-utils/tests |
| 13 TEST_LIST: generic_bogus_arg-1-1 ... generic_version-1-50 |
| 14 ... |
| 15 FAIL: test_user_mime_install |
| 16 NORESULT: test_system_mime_install |
| 17 71 of 92 tests passed. (114 attempted) |
| 18 See xdg-test.log for details. |
| 19 NOT OK! |
| 20 |
| 21 FAIL indicates (not suprisingly) a test failure. |
| 22 NORESULT indicates that the test prerequisites failed for some reason. |
| 23 (e.g. the install phase of an uninstall test failed) |
| 24 UNTESTED means that something needed was not found. This is fine and should |
| 25 be ignored. These tests are not counted in the total, only attempted. |
| 26 (e.g. test requires root, but we are not running as root) |
| 27 |
| 28 NOTE: The test runner makes guesses about appropriate values of XDG_TEST_DIR |
| 29 and PATH. These values can be overriden explicitly. |
| 30 |
| 31 To run tests individually, or as smaller groups do something like |
| 32 |
| 33 $ ./testrun xdg-mime |
| 34 |
| 35 OR |
| 36 |
| 37 $ ./testrun xdg-mime/t.10-user_mime_install |
| 38 |
| 39 OR (if you have defined XDG_TEST_DIR and PATH correctly) |
| 40 |
| 41 $ xdg-mime/t.10-user_mime_install |
| 42 |
| 43 |
| 44 BACKUPS |
| 45 ------- |
| 46 THESE TESTS CHANGE YOUR USER ENVIRONMENT. Effort is made to keep pollution |
| 47 to a minimum, but we make no guarantees!! Back up your environment/system |
| 48 early and often. This is especially critical if you run tests as root. You |
| 49 have been warned. |
| 50 |
| 51 |
| 52 INTERACTIVE TESTS |
| 53 ----------------- |
| 54 Because it is difficult to verify the way things appear to the user, some |
| 55 tests are interactive and require the user to verify or perform actions. |
| 56 This is sometimes annoying, so interactive tests can be disabled with the |
| 57 -I flag. Note that if you run tests non-interactively, some tests |
| 58 (xdg-email) may generate strange errors on the screen, since the test |
| 59 cleans up support files before the email client tries to read them. Use |
| 60 the -C option to work around this. |
| 61 |
| 62 |
| 63 PRIVILEGED TESTS |
| 64 --------------- |
| 65 BACK UP YOUR SYSTEM. See above. |
| 66 |
| 67 Some tests require root (e.g. those commands with a --system option). So, |
| 68 tests in this group return UNTESTED if they are not run as root. |
| 69 |
| 70 The test runner will ask for the root password in order to run these tests |
| 71 as the root user. |
| 72 |
| 73 |
| 74 CLEANUP |
| 75 ------- |
| 76 Tests should clean up after themselves. However, this sometimes fails, |
| 77 so use 'sudo make tests-clean' or 'make tests-userclean'. |
| 78 (Note that you must have generated a makefile via 'cd .. && ./configure' |
| 79 at some point.) |
| 80 |
| 81 |
| 82 DIRECTORY STRUCTURE |
| 83 ------------------- |
| 84 xdg-* tests for each util |
| 85 include "library" code used by most tests |
| 86 generic generic tests to be run on most utilities. |
| 87 See xdg-mime/t.apply_generic |
| 88 |
| 89 |
| 90 WRITING XDG-UTILS TESTS |
| 91 ----------------------- |
| 92 See xdg-mime/t.10-user_mime_install as an example. |
| 93 |
| 94 Each test is as follows |
| 95 |
| 96 test_function() { -- Tests are functions for TET integration. |
| 97 |
| 98 test_start "test description" -- required to begin a test |
| 99 test_purpose "verbose text" -- optionally provide a verbose description. |
| 100 not used. |
| 101 test_init -- optionally begin a prerequisite section. |
| 102 assertions that fail here cause NORESULT |
| 103 rather than FAIL |
| 104 <pre-assertions> |
| 105 test_procedure -- required to begin the actuall test |
| 106 assertions |
| 107 <test-assertions> |
| 108 |
| 109 test_result -- required to generate result codes. |
| 110 Must be last. |
| 111 } |
| 112 |
| 113 run_test test_function |
| 114 - OR - |
| 115 repeat_test test_function NVARS V1 ... VN V1val1 ... V1valM ... VNval1 ... VNval
M |
| 116 -- one of the above is required. |
| 117 see include/testcontrol.sh for detail |
| 118 |
| 119 |
| 120 For questions or feedback, please use the Portland mailinglist at |
| 121 http://lists.freedesktop.org/mailman/listinfo/portland |
| 122 |
| 123 Test results can be submitted on |
| 124 http://portland.freedesktop.org/testreport.html |
OLD | NEW |