wiki:manuals/SRM_2.2_Setup
Last modified 7 years ago Last modified on 12/10/10 16:23:46

SRM 2.2 Setup

Disclaimer : The author is still learning SRM 2.2 setup so this pages is under heavy development. This said this recipe works.

Introduction

From the perspective of the Poolmanager the order of configuration for working space management.

Pools -> Pool Groups -> Links -> Link Groups -> Working space reservation.

please be aware that if Pools are shared between Pool Groups space reservation will not work correctly and space calculation will give incorrect values and can lead to over booking.

Other components that must be configured include the namespace and "SpaceManagerLinkGroupAuthorizationFileName?" file.

Configuring the dCacheSetup file

First enable space management in the file /opt/d-cache/config/dCacheSetup

vi /opt/d-cache/config/dCacheSetup

Change the following entries to "yes"

srmSpaceManagerEnabled=yes
srmImplicitSpaceManagerEnabled=yes

And we must also set

SpaceManagerLinkGroupAuthorizationFileName=/opt/d-cache/etc/spacemanager.cfg

This file could contain the following information.

LinkGroup dteamProductionDisk
/dteam/Role=*

LinkGroup dteamProductionTape
/dteam/Role=production
/dteam/soft-valid/Role=production

LinkGroup AllUsersTapeSpace
#allow anyone :-)
*/Role=*

Where DteamProductionDisk?, and DteamProductionTape? are storage clases we will use later for reserving space in.

Now we must restart the dcache core

/etc/init.d/dcache restart

Configuring the PoolManager.

To set up SRM 2.2 space management we now we must configure the poolmanager. This can be done either on the file directly or through the admin interface. Due to its interactive nature we will need to use the admin interface and set up the pool manager.

First log in

[root@steyr ~]# ssh -c blowfish -p 22223 -l admin `hostname -f `
admin@steyr.desy.de's password: dickerelch

    dCache Admin (VII) (user=admin)


(local) admin >

The default password is

dickerelch

Now we have logged in we must change to the PoolManager domain.

(local) admin > cd PoolManager

Configuring the PoolManager ugroups.

Now lets test for existing ugroups

[ford.desy.de] (PoolManager) admin > psu ls -l ugroup                        
any-protocol
 unitList :
   */*  (type=Protocol;canonical=*/*;uGroups=1)
any-store
 unitList :
   dteam:STATIC@osm  (type=Store;canonical=dteam:STATIC@osm;uGroups=2)
   *@*  (type=Store;canonical=*@*;uGroups=1)
   dteam:GENERATED@osm  (type=Store;canonical=dteam:GENERATED@osm;uGroups=2)
dteam-groups
 unitList :
   dteam:STATIC@osm  (type=Store;canonical=dteam:STATIC@osm;uGroups=2)
   dteam:GENERATED@osm  (type=Store;canonical=dteam:GENERATED@osm;uGroups=2)
world-net
 unitList :
   0.0.0.0/0.0.0.0  (type=Net;canonical=0.0.0.0/0.0.0.0;uGroups=1)

This states the following ugroups exist.

any-protocol
any-store
dteam-groups
world-net

We will be working with "dteam-groups".

Configuring the PoolManager pgroup.

First we must prevent sharing of sharing pools,

psu removefrom pgroup default <poolName>

To check this worked

[ford.desy.de] (PoolManager) admin > psu ls -l pgroup  default
default
 linkList :
   default-link  (pref=10/10/-1/10;;ugroups=3;pools=1)
 poolList :

if it failed it will look like

[ford.desy.de] (PoolManager) admin > psu ls -l pgroup  default
default
 linkList :
   default-link  (pref=10/10/-1/10;;ugroups=3;pools=1)
 poolList :
   ford_1  (enabled=true;active=17;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)
[ford.desy.de] (PoolManager) admin > psu ls -l pgroup  default
default
 linkList :
   default-link  (pref=10/10/-1/10;;ugroups=3;pools=1)
 poolList :
   ford_1  (enabled=true;active=6;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)

So now we have no pools in the "default" pgroup (pool group)

[ford.desy.de] (PoolManager) admin > psu ls pgroup
ResilientPools
default
dteam

Since "DteamProductionDiskPoolGroup?" does not exist we have to create it.

psu create pgroup DteamProductionDiskPoolGroup

Now when we confirm that we created it successfully.

[ford.desy.de] (PoolManager) admin > psu ls -l pgroup
ResilientPools
 linkList :
 poolList :
DteamProductionDiskPoolGroup
 linkList :
 poolList :
default
 linkList :
   default-link  (pref=10/10/-1/10;;ugroups=3;pools=1)
 poolList :
   ford_1  (enabled=true;active=27;rdOnly=false;links=0;pgroups=1;hsm=[];mode=enabled)
dteam
 linkList :
   dteam-link  (pref=20/20/-1/20;;ugroups=2;pools=1)
 poolList :

Now we need to add a pool to the pool group DteamProductionDiskPoolGroup? and remove from default.

psu removefrom pgroup default ford_1
psu addto pgroup DteamProductionDiskPoolGroup ford_1  

Configuring the PoolManager link.

Now we need to create the link

psu create link DteamProductionDiskLink dteam-groups
psu ls -l link   

should output

DteamProductionDiskLink
 readPref  : 0
 cachePref : 0
 writePref : 0
 p2pPref   : -1
 section   : None
 linkGroup : None
 UGroups :
   dteam-groups  (links=2;units=2)

Now we need to add the pool group to the link

psu add link  DteamProductionDiskLink DteamProductionDiskPoolGroup

Configuring the PoolManager linkGroup.

Now we need to create the link group.

psu create linkGroup DteamProductionDisk 

To confirm this operation succeeded.

[ford.desy.de] (PoolManager) admin > psu ls -l linkGroup                      
DteamProductionDisk : [EMPTY]
    Attributes:
    AccessLatency:
           onlineAllowed=false
           nearlineAllowed=true
    RetentionPolicy:
           custodialAllowed=true
           outputAllowed=true
           replicaAllowed=true

We can change the properties of the link group for example

[ford.desy.de] (PoolManager) admin >  psu set linkGroup onlineAllowed DteamProductionDisk true
[ford.desy.de] (PoolManager) admin > psu ls -l linkGroup                                      
DteamProductionDisk : [  DteamProductionDiskLink ]
    Attributes:
    AccessLatency:
           onlineAllowed=true
           nearlineAllowed=true
    RetentionPolicy:
           custodialAllowed=true
           outputAllowed=true
           replicaAllowed=true

From the help it seems to be clear how to set the other attibutes of a linkGroup

 psu set linkGroup custodialAllowed <linkGroup> <true|false>
 psu set linkGroup replicaAllowed <linkGroup> <true|false>
 psu set linkGroup nearlineAllowed <linkGroup> <true|false>
 psu set linkGroup outputAllowed <linkGroup> <true|false>
 psu set linkGroup attribute <linkGroup> [-r] attribute=value
 psu set linkGroup onlineAllowed <linkGroup> <true|false>

So we now need to add links to our link group

psu addto linkGroup DteamProductionDisk DteamProductionDiskLink

Repeating the query we can now see the linkGroup "DteamProductionDisk?" contains the link "DteamProductionDiskLink?"

Now to confirm the set up with dcache.

[ford.desy.de] (PoolManager) admin > free                                                              
DteamProductionDisk : 17179868663

Now we must save our changes

save

Reserving Space

Space can be reserved with the SRM client or through the admin interface.

Reserving Space with the Admin Interface

If you are not in the Space manager in the admin interface change to the space manager domain with ".." followed by "cd SrmSpaceManager".

[ford.desy.de] (SrmSpaceManager) admin > reserve   -vog=/dteam  -acclat=NEARLINE -retpol=REPLICA   -lg=DteamProductionDisk  -desc=TAPE 300 3600 
10001 voGroup:/dteam voRole:null linkGroupId:0 size:300 created:Tue Feb 19 17:09:14 CET 2008 lifetime:3600000ms  expiration:Tue Feb 19 18:09:14 CET 2008 descr:TAPE state:RESERVEDused:0allocated:0
[ford.desy.de] (SrmSpaceManager) admin > 

when specifying access latency and retention you must be very carefull that the policy of a space reservation being made has properties that do not contradict link group settings. Code will not error if one breaks this rule.

AccessLatency=ONLINE
RetentionPolicy=REPLICA

then LG should have

"onlineallowed=true"
"replicaallowed=true"

So now to confirm, the space allocation

[omsynge@clinton ~]$ /opt/d-cache/srm/bin/srm-get-space-metadata srm://ford.desy.de:8443/pnfs/desy.de/data/dteam -space_tokens=10001
WARNING: SRM_PATH is defined, which might cause a wrong version of srm client to be executed
WARNING: SRM_PATH=/opt/d-cache/srm
Space Reservation with token=10001
                   owner:VoGroup=/dteam VoRole=null
               totalSize:300
          guaranteedSize:300
              unusedSize:300
        lifetimeAssigned:3600
            lifetimeLeft:3537
           accessLatency:NEARLINE
         retentionPolicy:REPLICA

A simple example of space reservation through the admin interface.

(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE 300 3600
229927 voGroup:/atlas voRole:production linkGroupId:53988 size:300 
created:Fri Nov 09 10:59:26 CST 2007 lifetime:3600000ms  expiration:Fri 
Nov 09 11:59:26 CST 2007 descr:TAPE state:RESRVED
(SrmSpaceManager) admin >

As the previous version does not explicitly specify a linkGroup the reservation is made against the first match, the following example explicitly reserves space against a single link group.

(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE -lg=custodialPoolsLinkGroup 300 3600
229944 voGroup:/atlas voRole:production linkGroupId:53988 size:300 
created:Fri Nov 09 11:01:54 CST 2007 lifetime:3600000ms  expiration:Fri 
Nov 09 12:01:54 CST 2007 descr:TAPE state:RESRVED
(SrmSpaceManager) admin >

With the SRM client

Using the dCache SRM client

Example set ups by VO

Since the VO's are actively debating and changing their specifications for how they wish to use space reservation, this section details our current understanding but the space token names, their attributes and even the number of them cannot be considered definitive. Please speak to the VO's for definitive details these are just examples.

Atlas

Additions to poolManager.conf for Atlas.

psu create poolgroup AtlasProductionDiskPoolGroup
psu create link AtlasProductionDiskLink
psu addto linkGroup AtlasProductionDiskPoolGroup
psu create linkGroup AtlasProductionDisk
psu addto linkGroup AtlasProductionDisk AtlasProductionDiskLink
psu create linkGroup AtlasProductionTape

Additions to LinkGroupAuthorizationFile?.conf for Atlas.

LinkGroup AtlasProductionDisk
/atlas/Role=production
/atlas/soft-valid/Role=production

LinkGroup AtlasProductionTape
/atlas/Role=production
/atlas/soft-valid/Role=production

Define the following space token descriptions:

  • TAPE for the CUSTODIAL-NEARLINE storage class
  • DISK for the REPLICA-ONLINE storage class

Se need to do this with the space manager probably best done with admin interface. But could be done with client but admin interface can explicitly bind the space resivation to a link group

#space reservation example
(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production 
-acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE 300 3600
229927 voGroup:/atlas voRole:production linkGroupId:53988 size:300 
created:Fri Nov 09 10:59:26 CST 2007 lifetime:3600000ms  expiration:Fri 
Nov 09 11:59:26 CST 2007 descr:TAPE state:RESRVED
(SrmSpaceManager) admin >

#space reservation within  a specific link group
(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production 
-acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE -lg=custodialPoolsLinkGroup 
300 3600
229944 voGroup:/atlas voRole:production linkGroupId:53988 size:300 
created:Fri Nov 09 11:01:54 CST 2007 lifetime:3600000ms  expiration:Fri 
Nov 09 12:01:54 CST 2007 descr:TAPE state:RESRVED
(SrmSpaceManager) admin >

LHCB

LinkGroup LHCb_RAW_DISK
/lhcb/lcgprod/Role=*

LinkGroup LHCb_RDST_DISK
/lhcb/lcgprod/Role=*

LinkGroup CUSTODIAL_NEARLINE_TAPE
/lhcb/lcgprod/Role=*
Define the following space token descriptions (more info will be given about the space allocated per token):

    * LHCb_RAW and LHCb_RDST for the CUSTODIAL-NEARLINE
#space reservation example
(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE 300 3600
229927 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 10:59:26 CST 2007 lifetime:3600000ms  expiration:Fri 
Nov 09 11:59:26 CST 2007 descr:TAPE state:RESRVED
(SrmSpaceManager) admin >

#space reservation within  a specific link group
(SrmSpaceManager) admin > reserve -vog=/atlas -vor=production -acclat=ONLINE -retpol=CUSTODIAL -desc=TAPE -lg=custodialPoolsLinkGroup 300 3600
229944 voGroup:/atlas voRole:production linkGroupId:53988 size:300 created:Fri Nov 09 11:01:54 CST 2007 lifetime:3600000ms  expiration:Fri 
Nov 09 12:01:54 CST 2007 descr:TAPE state:RESRVED
(SrmSpaceManager) admin >

Default Space Token Tag

In order to specify a default space token in the name space, create a space token and create a tag named "WriteToken?" in the directory. Srmcp writes into the files in this directory without explicitly specified space token will use space token from this tag.

Example

if space token name is 123456, create a tag using command:

echo "123456" > ".(tag)(WriteToken)"