SRM 2.2 Setup
Disclaimer : The author is still learning SRM 2.2 setup so this pages is under heavy development. This said this recipe works.
Introduction
From the perspective of the Poolmanager the order of configuration for working space management.
Pools -> Pool Groups -> Links -> Link Groups -> Working space reservation.
please be aware that if Pools are shared between Pool Groups space reservation will not work correctly and space calculation will give incorrect values and can lead to over booking.
Other components that must be configured include the namespace and "SpaceManagerLinkGroupAuthorizationFileName?" file.
Configuring the dCacheSetup file
First enable space management in the file /opt/d-cache/config/dCacheSetup
vi /opt/d-cache/config/dCacheSetup
Change the following entries to "yes"
srmSpaceManagerEnabled=yes srmImplicitSpaceManagerEnabled=yes
And we must also set
SpaceManagerLinkGroupAuthorizationFileName=/opt/d-cache/etc/spacemanager.cfg
This file could contain the following information.
LinkGroup dteamProductionDisk /dteam/Role=* LinkGroup dteamProductionTape /dteam/Role=production /dteam/soft-valid/Role=production LinkGroup AllUsersTapeSpace #allow anyone :-) */Role=*
Where DteamProductionDisk?, and DteamProductionTape? are storage clases we will use later for reserving space in.
Now we must restart the dcache core
/etc/init.d/dcache restart
Configuring the PoolManager.
To set up SRM 2.2 space management we now we must configure the poolmanager. This can be done either on the file directly or through the admin interface. Due to its interactive nature we will need to use the admin interface and set up the pool manager.
First log in
[root@steyr ~]# ssh -c blowfish -p 22223 -l admin `hostname -f `
admin@steyr.desy.de's password: dickerelch
dCache Admin (VII) (user=admin)
(local) admin >
The default password is
dickerelch
Now we have logged in we must change to the PoolManager domain.
(local) admin > cd PoolManager
Configuring the PoolManager ugroups.
Now lets test for existing ugroups
[ford.desy.de] (PoolManager) admin > psu ls -l ugroup any-protocol unitList : */* (type=Protocol;canonical=*/*;uGroups=1) any-store unitList : dteam:STATIC@osm (type=Store;canonical=dteam:STATIC@osm;uGroups=2) *@* (type=Store;canonical=*@*;uGroups=1) dteam:GENERATED@osm (type=Store;canonical=dteam:GENERATED@osm;uGroups=2) dteam-groups unitList : dteam:STATIC@osm (type=Store;canonical=dteam:STATIC@osm;uGroups=2) dteam:GENERATED@osm (type=Store;canonical=dteam:GENERATED@osm;uGroups=2) world-net unitList : 0.0.0.0/0.0.0.0 (type=Net;canonical=0.0.0.0/0.0.0.0;uGroups=1)
This states the following ugroups exist.
any-protocol any-store dteam-groups world-net
We will be working with "dteam-groups".
Configuring the PoolManager pgroup.
First we must prevent sharing of sharing pools,
psu removefrom pgroup default <poolName>
To check this worked
[ford.desy.de] (PoolManager) admin > psu ls -l pgroup default default linkList : default-link (pref=10/10/-1/10;;ugroups=3;pools=1) poolList :
if it failed it will look like
[ford.desy.de] (PoolManager) admin > psu ls -l pgroup default default linkList : default-link (pref=10/10/-1/10;;ugroups=3;pools=1) poolList : ford_1 (enabled=true;active=17;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled) [ford.desy.de] (PoolManager) admin > psu ls -l pgroup default default linkList : default-link (pref=10/10/-1/10;;ugroups=3;pools=1) poolList : ford_1 (enabled=true;active=6;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)
So now we have no pools in the "default" pgroup (pool group)
[ford.desy.de] (PoolManager) admin > psu ls pgroup ResilientPools default dteam
Since "DteamProductionDiskPoolGroup?" does not exist we have to create it.
psu create pgroup DteamProductionDiskPoolGroup
Now when we confirm that we created it successfully.
[ford.desy.de] (PoolManager) admin > psu ls -l pgroup ResilientPools linkList : poolList : DteamProductionDiskPoolGroup linkList : poolList : default linkList : default-link (pref=10/10/-1/10;;ugroups=3;pools=1) poolList : ford_1 (enabled=true;active=27;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled) dteam linkList : dteam-link (pref=20/20/-1/20;;ugroups=2;pools=1) poolList :
Now we need to add a pool to the pool group DteamProductionDiskPoolGroup? and remove from default.
psu removefrom pgroup default ford_1 psu addto pgroup DteamProductionDiskPoolGroup ford_1
Configuring the PoolManager link.
Now we need to create the link
psu create link DteamProductionDiskLink dteam-groups psu ls -l link
should output
DteamProductionDiskLink readPref : 0 cachePref : 0 writePref : 0 p2pPref : -1 section : None linkGroup : None UGroups : dteam-groups (links=2;units=2)
Now we need to add the pool group to the link
psu add link DteamProductionDiskLink DteamProductionDiskPoolGroup
Configuring the PoolManager linkGroup.
Now we need to create the link group.
psu create linkGroup DteamProductionDisk
To confirm this operation succeeded.
[ford.desy.de] (PoolManager) admin > psu ls -l linkGroup
DteamProductionDisk : [EMPTY]
Attributes:
AccessLatency:
onlineAllowed=false
nearlineAllowed=true
RetentionPolicy:
custodialAllowed=true
outputAllowed=true
replicaAllowed=true
We can change the properties of the link group for example
[ford.desy.de] (PoolManager) admin > psu set linkGroup onlineAllowed DteamProductionDisk true
[ford.desy.de] (PoolManager) admin > psu ls -l linkGroup
DteamProductionDisk : [ DteamProductionDiskLink ]
Attributes:
AccessLatency:
onlineAllowed=true
nearlineAllowed=true
RetentionPolicy:
custodialAllowed=true
outputAllowed=true
replicaAllowed=true
From the help it seems to be clear how to set the other attibutes of a linkGroup
psu set linkGroup custodialAllowed <linkGroup> <true|false> psu set linkGroup replicaAllowed <linkGroup> <true|false> psu set linkGroup nearlineAllowed <linkGroup> <true|false> psu set linkGroup outputAllowed <linkGroup> <true|false> psu set linkGroup attribute <linkGroup> [-r] attribute=value psu set linkGroup onlineAllowed <linkGroup> <true|false>
So we now need to add links to our link group
psu addto linkGroup DteamProductionDisk DteamProductionDiskLink
Repeating the query we can now see the linkGroup "DteamProductionDisk?" contains the link "DteamProductionDiskLink?"
Now to confirm the set up with dcache.
[ford.desy.de] (PoolManager) admin > free DteamProductionDisk : 17179868663
Now we must save our changes
save
Reserving Space
Space can be reserved with the SRM client or through the admin interface.
Reserving Space with the Admin Interface
If you are not in the Space manager in the admin interface change to the space manager domain with ".." followed by "cd SrmSpaceManager".
[ford.desy.de] (SrmSpaceManager) admin > reserve -vog=/dteam -acclat=NEARLINE -retpol=REPLICA -lg=DteamProductionDisk -desc=TAPE 300 3600 10001 voGroup:/dteam voRole:null linkGroupId:0 size:300 created:Tue Feb 19 17:09:14 CET 2008 lifetime:3600000ms expiration:Tue Feb 19 18:09:14 CET 2008 descr:TAPE state:RESERVEDused:0allocated:0 [ford.desy.de] (SrmSpaceManager) admin >
when specifying access latency and retention you must be very carefull that the policy of a space reservation being made has properties that do not contradict link group settings. Code will not error if one breaks this rule.
AccessLatency=ONLINE RetentionPolicy=REPLICA
then LG should have
"onlineallowed=true" "replicaallowed=true"
So now to confirm, the space allocation
[omsynge@clinton ~]$ /opt/d-cache/srm/bin/srm-get-space-metadata srm://ford.desy.de:8443/pnfs/desy.de/data/dteam -space_tokens=10001
WARNING: SRM_PATH is defined, which might cause a wrong version of srm client to be executed
WARNING: SRM_PATH=/opt/d-cache/srm
Space Reservation with token=10001
owner:VoGroup=/dteam VoRole=null
totalSize:300
guaranteedSize:300
unusedSize:300
lifetimeAssigned:3600
lifetimeLeft:3537
accessLatency:NEARLINE
retentionPolicy:REPLICA
A simple example of space reservation through the admin interface.
(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE 300 3600 229927 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 10:59:26 CST 2007 lifetime:3600000ms expiration:Fri Nov 09 11:59:26 CST 2007 descr:TAPE state:RESRVED (SrmSpaceManager) admin >
As the previous version does not explicitly specify a linkGroup the reservation is made against the first match, the following example explicitly reserves space against a single link group.
(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE -lg=custodialPoolsLinkGroup 300 3600 229944 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 11:01:54 CST 2007 lifetime:3600000ms expiration:Fri Nov 09 12:01:54 CST 2007 descr:TAPE state:RESRVED (SrmSpaceManager) admin >
With the SRM client
Using the dCache SRM client
Example set ups by VO
Since the VO's are actively debating and changing their specifications for how they wish to use space reservation, this section details our current understanding but the space token names, their attributes and even the number of them cannot be considered definitive. Please speak to the VO's for definitive details these are just examples.
Atlas
Additions to poolManager.conf for Atlas.
psu create poolgroup AtlasProductionDiskPoolGroup psu create link AtlasProductionDiskLink psu addto linkGroup AtlasProductionDiskPoolGroup psu create linkGroup AtlasProductionDisk psu addto linkGroup AtlasProductionDisk AtlasProductionDiskLink psu create linkGroup AtlasProductionTape
Additions to LinkGroupAuthorizationFile?.conf for Atlas.
LinkGroup AtlasProductionDisk /atlas/Role=production /atlas/soft-valid/Role=production LinkGroup AtlasProductionTape /atlas/Role=production /atlas/soft-valid/Role=production
Define the following space token descriptions:
- TAPE for the CUSTODIAL-NEARLINE storage class
- DISK for the REPLICA-ONLINE storage class
Se need to do this with the space manager probably best done with admin interface. But could be done with client but admin interface can explicitly bind the space resivation to a link group
#space reservation example (SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE 300 3600 229927 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 10:59:26 CST 2007 lifetime:3600000ms expiration:Fri Nov 09 11:59:26 CST 2007 descr:TAPE state:RESRVED (SrmSpaceManager) admin > #space reservation within a specific link group (SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE -lg=custodialPoolsLinkGroup 300 3600 229944 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 11:01:54 CST 2007 lifetime:3600000ms expiration:Fri Nov 09 12:01:54 CST 2007 descr:TAPE state:RESRVED (SrmSpaceManager) admin >
LHCB
LinkGroup LHCb_RAW_DISK /lhcb/lcgprod/Role=* LinkGroup LHCb_RDST_DISK /lhcb/lcgprod/Role=* LinkGroup CUSTODIAL_NEARLINE_TAPE /lhcb/lcgprod/Role=*
Define the following space token descriptions (more info will be given about the space allocated per token):
* LHCb_RAW and LHCb_RDST for the CUSTODIAL-NEARLINE
#space reservation example (SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE 300 3600 229927 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 10:59:26 CST 2007 lifetime:3600000ms expiration:Fri Nov 09 11:59:26 CST 2007 descr:TAPE state:RESRVED (SrmSpaceManager) admin > #space reservation within a specific link group (SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE -lg=custodialPoolsLinkGroup 300 3600 229944 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 11:01:54 CST 2007 lifetime:3600000ms expiration:Fri Nov 09 12:01:54 CST 2007 descr:TAPE state:RESRVED (SrmSpaceManager) admin >
Default Space Token Tag
In order to specify a default space token in the name space, create a space token and create a tag named "WriteToken?" in the directory. Srmcp writes into the files in this directory without explicitly specified space token will use space token from this tag.
Example
if space token name is 123456, create a tag using command:
echo "123456" > ".(tag)(WriteToken)"
