Tertiary Storage related queues in dCache

Scope of this document

dCache can be connected to one ore more tertiary storage systems. This page document described the different queues involved in the interactions between dCache and the back-end storage systems. The description briefly touches on the commands to monitor and manipulate the queues.

Introduction

Operations related to a tertiary storage backend are managed by four queues within dCache :

• Central Restore Queue : receiveing requests for files which are no longer available on disk but have been requested by a client. There is only a single instance of this queue within one dCache instance. The queue resides in the PoolManager. After a request has been added to the queue, the actual request is delegated to the Pool HSM I/O Restore Queue.
• Pool Flush Queue : There is exactly one Pool Flush Queue per Pool. The queue collects all files which are supposed to go to tape on that pool and sorts them by storage group. If certain thresholds per storage are reached, the queue is emptied into the Pool HSM I/O Flush Queue.
• Pool HSM I/O Restore Queue : This queue contains all files which dCache wants to restore from tape to this Pool. The number of maximum allowed active restore processes can be configured.
• Pool HSM I/O Flush Queue : This queue contains all files which dCache wants to flush to tape from this Pool. The number of maximum allowed active flush processes can be configured.

The figure below give a brief idea on how those queues interact. The picture is followed by a more detailed description of the purpose of each of the queues and on how to query and manupulte them

---

General behaviour

Data onto tape

Whenever a file on a pool is supposed to be stored on tape, the file entry is added to the Pool Flush Queue of that pool. The Pool Flush Queue sorts all newly added files according to their storage group (e.g. store:group@hsm) consequently creating a queue per storage group (Pool Flush subgroup). Each of these sub queues can be configued to move its entire content into the Pool HSM I/O Flush Queue after a certain threshold is reached. The threshold is defined with respect to the number of files collected, the total amount of data collected or a certain timeout has expired for this particular subgroup. A single Pool HSM I/O Flush Queue receives entries from all Pool Flush subqueues. The Pool HSM I/O Flush Queue operates first in, first out. Only the first n requests are active and launch an executable resonsible for storing the files to tape. The remaining entries are inactive and wait for the next active request to finish. The Pool Flush queue, its subqueues and the Pool HSM I/O Flush Queue of a Pool operate independently from other Pools.

Data from tape

Whenever a file, which is no longer available on disk, is requested from dCache, the request is forwaded to the 'tape restore subsystem'. Depending on the configuration, this operation may be protected by ACL's which results in the request to be rejected. If the tape restore is permitted, the request is added to the Central Restore Queue within the PoolManager, which delegates the restore operation to a pool which is most appropriate for getting the file back to disk.

---

Queue management in detail

Central Restore Queue

cd PoolManager
rc ls

Pool HSM I/O Restore Queue

#
# for FETCH file
#
cd <poolName>
rh ls
#
# for STORE file
#
st ls

Pool HSM I/O Flush Queue

#
# for FETCH file
#
cd <poolName>
rh ls
#
# for STORE file
#
st ls

Pool Flush Queue

[dcachetogo.dcache.org] (pool1) admin > queue ls -l queue

                   Name : chimera:alpha
              Class@Hsm : chimera:alpha@osm
 Exiration rest/defined : -39 / 0   seconds
 Pending   rest/defined : 1 / 0
 Size      rest/defined : 877480 / 0
 Active Store Procs.    : 0
  00001BC6D76570A74534969FD72220C31D5D
# and 
#
flush ls

---

Last modified by patrick @ Wed May 23 09:53:03 2012