Commit Graph

1 Commits

Author SHA1 Message Date
Cas Cremers
4a1898db92 src/regression-tests: Consistent regression testing.
There is now a script

  src/regression-tests/regression-test.py

that should in the future be the default for running regression tests
instead of the ad-hoc approach we are currently using. The goal is to
ultimately have more reliable and consistent regression testing.

The script takes as input "tests.txt" and tries to perform tests from
that. This is effectively a collection of inputs to the scyther-linux
binary. The results are writting to the 'results' directory, as
test-X.out and test-X.err, where those correspond to stdout and stderr,
respectively. Additionally, a measurement of wall-clock time in seconds
is written to test-X.time.

For now, we are using the timer to ensure all tests terminate. It would
be nicer to use a less environment-dependent way of enforcing
termination.
2013-04-26 14:29:22 +02:00