135 lines
5.3 KiB
Plaintext
135 lines
5.3 KiB
Plaintext
Perl-based TAP tests
|
|
====================
|
|
|
|
src/test/perl/ contains shared infrastructure that's used by Perl-based tests
|
|
across the source tree, particularly tests in src/bin and src/test. It's used
|
|
to drive tests for backup and restore, replication, etc - anything that can't
|
|
really be expressed using pg_regress or the isolation test framework.
|
|
|
|
The tests are invoked via perl's 'prove' command, wrapped in PostgreSQL
|
|
makefiles to handle instance setup etc. See the $(prove_check) and
|
|
$(prove_installcheck) targets in Makefile.global. By default every test in the
|
|
t/ subdirectory is run. Individual test(s) can be run instead by passing
|
|
something like PROVE_TESTS="t/001_testname.pl t/002_othertestname.pl" to make.
|
|
|
|
By default, to keep the noise low during runs, we do not set any flags via
|
|
PROVE_FLAGS, but this can be done on the 'make' command line if desired, eg:
|
|
|
|
make check-world PROVE_FLAGS='--verbose'
|
|
|
|
When a test fails, the terminal output from 'prove' is usually not sufficient
|
|
to diagnose the problem. Look into the log files that are left under
|
|
tmp_check/log/ to get more info. Files named 'regress_log_XXX' are log
|
|
output from the perl test scripts themselves, and should be examined first.
|
|
Other files are postmaster logs, and may be helpful as additional data.
|
|
|
|
The tests default to a timeout of 180 seconds for many individual operations.
|
|
Slow hosts may avoid load-induced, spurious failures by setting environment
|
|
variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds greater than 180.
|
|
Developers may see faster failures by setting that environment variable to
|
|
some lesser number of seconds.
|
|
|
|
Data directories will also be left behind for analysis when a test fails;
|
|
they are named according to the test filename. But if the environment
|
|
variable PG_TEST_NOCLEAN is set, the data directories will be retained
|
|
regardless of test status. This environment variable also prevents the
|
|
test's temporary directories from being removed.
|
|
|
|
|
|
Writing tests
|
|
-------------
|
|
|
|
You should prefer to write tests using pg_regress in src/test/regress, or
|
|
isolation tester specs in src/test/isolation, if possible. If not, check to
|
|
see if your new tests make sense under an existing tree in src/test, like
|
|
src/test/ssl, or should be added to one of the suites for an existing utility.
|
|
|
|
Note that all tests and test tools should have perltidy run on them before
|
|
patches are submitted, using perltidy --profile=src/tools/pgindent/perltidyrc
|
|
|
|
Tests are written using Perl's Test::More with some PostgreSQL-specific
|
|
infrastructure from src/test/perl providing node management, support for
|
|
invoking 'psql' to run queries and get results, etc. You should read the
|
|
documentation for Test::More before trying to write tests.
|
|
|
|
Test scripts in the t/ subdirectory of a suite are executed in alphabetical
|
|
order.
|
|
|
|
Each test script should begin with:
|
|
|
|
use strict;
|
|
use warnings FATAL => 'all';
|
|
use PostgreSQL::Test::Cluster;
|
|
use PostgreSQL::Test::Utils;
|
|
use Test::More;
|
|
|
|
then it will generally need to set up one or more nodes, run commands
|
|
against them and evaluate the results. For example:
|
|
|
|
my $node = PostgreSQL::Test::Cluster->new('primary');
|
|
$node->init;
|
|
$node->start;
|
|
|
|
my $ret = $node->safe_psql('postgres', 'SELECT 1');
|
|
is($ret, '1', 'SELECT 1 returns 1');
|
|
|
|
$node->stop('fast');
|
|
|
|
Each test script should end with:
|
|
|
|
done_testing();
|
|
|
|
Test::Builder::Level controls how far up in the call stack a test will look
|
|
at when reporting a failure. This should be incremented by any subroutine
|
|
which directly or indirectly calls test routines from Test::More, such as
|
|
ok() or is():
|
|
|
|
local $Test::Builder::Level = $Test::Builder::Level + 1;
|
|
|
|
Read the documentation for more on how to write tests:
|
|
|
|
perldoc Test::More
|
|
perldoc Test::Builder
|
|
|
|
For available PostgreSQL-specific test methods and some example tests read the
|
|
perldoc for the test modules, e.g.:
|
|
|
|
perldoc src/test/perl/PostgreSQL/Test/Cluster.pm
|
|
|
|
Portability
|
|
-----------
|
|
|
|
Avoid using any bleeding-edge Perl features. We have buildfarm animals
|
|
running Perl versions as old as 5.14, so your tests will be expected
|
|
to pass on that.
|
|
|
|
Also, do not use any non-core Perl modules except IPC::Run. Or, if you
|
|
must do so for a particular test, arrange to skip the test when the needed
|
|
module isn't present. If unsure, you can consult Module::CoreList to find
|
|
out whether a given module is part of the Perl core, and which module
|
|
versions shipped with which Perl releases.
|
|
|
|
One way to test for compatibility with old Perl versions is to use
|
|
perlbrew; see http://perlbrew.pl . After installing that, do
|
|
|
|
export PERLBREW_CONFIGURE_FLAGS='-de -Duseshrplib'
|
|
perlbrew --force install 5.14.0
|
|
perlbrew use 5.14.0
|
|
perlbrew install-cpanm
|
|
cpanm install Test::Simple@0.98
|
|
cpanm install IPC::Run@0.79
|
|
cpanm install ExtUtils::MakeMaker@6.50 # downgrade
|
|
|
|
TIP: if Test::Simple's utf8 regression test hangs up, try setting a
|
|
UTF8-compatible locale, e.g. "export LANG=en_US.utf8".
|
|
|
|
Then re-run Postgres' configure to ensure the correct Perl is used when
|
|
running tests. To verify that the right Perl was found:
|
|
|
|
grep ^PERL= config.log
|
|
|
|
Due to limitations of cpanm, this recipe doesn't exactly duplicate the
|
|
module list of older buildfarm animals. The discrepancies should seldom
|
|
matter, but if you want to be sure, bypass cpanm and instead manually
|
|
install the desired versions of Test::Simple and IPC::Run.
|