Functional Test Suite

The dCache functional test suite provides a quick way to test a dCache instance and provides client and server tests. It is written in python built upon a modular framework, of plugin's. Each test is one plugin, these tests have attributes to allow them to be selected by attribute or run individually, each test also sates which variables it requires and so users can build the queries on a single line.

Installation

The dCacheTestSuite application is a python program which is either to be installed on the client or the server. For some tests you may require the components from a Grid UI.

It can be installed simply from the dCache yum repository which will install the code required to run the suite.

add the following line to your "/etc/yum.repos.d/"

[root@clinton ~]# cat /etc/yum.repos.d/dcache-tools.repo 
[dcache-tools]
name=DCacheTools
baseurl=http://cvs.dcache.org/repository/yum/sl4.4.0/i386/RPMS.tools/
enabled=1

If you are not on SL4 i386 please browse the repositories to set your architecture correctly. Then run the following command.

yum install dCacheTests

Since the test suite uses GSI authenticated client tools to access the server, you should also create a users account and add the users grid certificates to this account for each user of the test suite. Thier is also a online man page which can be accessed with

man dCacheTestSuite

Once this is completed you should check that you can do a globus-url-copy to a suitably set up server before running the test suite. If this works you can run the client tests.

Command Line Use

The test suite has 2 main types of options, options for the test suite, and options for the tests. The test suite options are

 -h, --help              Display help information
 -V, --version           Version
 -l, --list              List available tests
 -L, --List              List available tests grouped
 -a, --all               Run available tests
 -s, --select            Run available tests selected by a key=val pair, eg -s 'locality=clienttest release=1.7.0'
 -b, --busyb             Generate a busybee script for selected, eg -g 'locality=clienttest release=1.7.0'
 -r, --run               run test
 -q, --loglevel          Set Log Level
 -p, --plugin            your tests directory

Other options provide variables for tests, if a test requires and option it should stop and show an error message informing that a variable needs to be set. these options are listed bellow.

 -T, --testhost          Set testhost host
 -v  --vo                VO (Virtual Organisation) to test
 -d, --domain            Domain name to test eg desy.de

Command line options for the test suite

Command option -h --help

Displays the built in help, its probably more usefull to read thsi wiki for new users or to read the man page. This is useful for experienced users though.

Command option -l --list

Lists all available tests in the test suite. these are just presented in a single list. Each test is presented only once.

Command option -L --List

Lists all available tests in the test suite and grouped by the tags on each test. If a test is tagged multiple times then it appears many times this list is thierfore much longer than the simple list. Some important tests maybe tagged many times, For example "test_GsiDcap" is tagged as "release=1.6.6", "release=1.7.0", "release=1.8.0" , "release=future", "security=gsi", "locality=clienttest" as its applicable to current future and previous versions of dCache is a client test and is based upon gsi security.

Command option -a --all

This runs all available tests, originally this was very useful as running each test on a single command line leads to very long command lines, its usefulness has gone down now we have many tests as not all tests are applicable to all scenarios. Examples are a server test cannot be run on a client, and a dcap test is of little use in a host providing only GSI based access.

Command option -s --select

Run available tests selected by the tests tag, one or more tags can be selected with the attribute this parameter requires. this has largely superseded the -a --all option as not all tests can always be run with a single dCache instance. For example

-s 'locality=clienttest'
-s 'locality=clienttest release=1.7.0'
-s "locality=clienttest release=1.7.0 security=gsi"

The first selection would run all client tests, the second line would run all tests which are client tests for release 1.7.0 and this release series, the third line limits the selected tests to only tests that are for release 1.7.0 are client tests and accept gsi authentication.

Command option -b --busyb

Tells the test suite to build a busybee configuration file so that tests can be run on a cron job for a web page. This can be used for continuous monitoring of a server. the tests added to the busybee script are selected using the "-s --select" option detailed above.

Command option -r, --run

Runs tests explicitly, this option takes the attribute of a tests name. The list of test names can be retrieved from the "-l --list" command line. Tests may require variables

-r test_GsiDcap

This option can be used more than once in a command line and in conjuction or separately to the command option -s --select.

Command option -q, --loglevel

Takes an attribute loglevel, this sets the amount of reporting. The parameter is a number between 0 and 50, 0 is the most verbose and 50 is the most silent.

"critical" = 40 -50
"error"    = 30-40
"warning"  = 20-30
"info"     = 10-20
"debug"    = 0-10

All errors of a numbers higher or equal the loglevel set will be displayed to standard out. This logging is slightly different to standard python logging as it was written to work with versions of python before the logging library was available in the python available on our version.

Command option -p, --plugin

This allows users to add directories where non standard tests can be stored.

Command line options for the tests

These options are required by different tests to provide variables that each test requires, if an attribute is missing Errors will occur as shown in the following example.

$  dCacheTestSuite.py -T cork.desy.de  -v dteam -q 0 -s "locality=clienttest release=1.7.0 security=gsi" 
No tests scripts directory specified. Will use basicTests.
/usr/lib/python2.2/site-packages
Error: testDomain not set
Error: Parameters inccorect

Test result

If all the selected tests are passed the test suite will terminate with a return code of zero. this can be evaluated by running the following command.

echo $?

The first test that fails will terminate the test suite, and the return code will be set to a non zero value. If its not clear what failed it is recomended to reduce the loggin level to a low number so that the tests give more verbose logging using the command option -q, --loglevel.

Examples

Running the test suite with out command parameters

$dCacheTestSuite.py 
No tests scripts directory specified. Will use basicTests.
/usr/lib/python2.2/site-packages
 -h, --help              Display help information
 -V, --version           Version
 -l, --list              List available tests
 -L, --List              List available tests grouped
 -a, --all               Run available tests
 -s, --select            Run available tests selected by a key=val pair, eg -s 'locality=clienttest release=1.7.0'
 -b, --busyb             Generate a busybee script for selected, eg -g 'locality=clienttest release=1.7.0'
 -T, --testhost          Set testhost host
 -r, --run               run test
 -v  --vo                VO (Virtual Organisation) to test
 -d, --domain            Domain name to test eg desy.de
 -q, --loglevel          Set Log Level
 -p, --plugin            your tests directory

Typical test operation used in release testing.

dCacheTestSuite.py -T cork.desy.de -t 'cork.desy.de'  -d desy.de  -v dteam -q 0  
        -s "locality=clienttest release=1.7.0 security=gsi"

Testing srm interface.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_srm

Testing gridftp doors interface.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_GlobusUrlCopyGridftp

Testing gridftp doors interface work with parallel transfers.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_GlobusUrlCopyStreams

Testing gridftp doors dont allow listing directories outside the nameserver.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_edgGridftpLsRoot

Testing gsidcap using the interposition library.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_GsiDcap

Testing gsidcap doors with the dccp client.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_dccpGsidcap

Testing dcap doors.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_dccpDcap

Testing dccp test_dccpRCcheck return codes.

dCacheTestSuite.py -T cork.desy.de   -d desy.de  -v dteam -q 0 
    -r test_dccpRCcheck

Installing Building and Packaging the test suite

The dcache test suite is pure python and packaged as pure python. Hence to set it up all you need to do is

python setup.py install

To make an RPM

python setup.py  bdist_rpm