wiki:manuals/dCacheSpecs
Last modified 14 years ago Last modified on 05/01/07 14:38:17

dCache is Managed Storage

dCache w/o HSM and w/o Reslient Module

  • Due to the consequent usage of the java technology, dCache components can run on a variaty of hardware and OS systems. This allows to pick the most appropriate system for various purposes. e.g.
    • high throughput or integrated storage boxes for pool node usage
    • highly threadable systems (multi core) for core components (eg. PoolManager)
    • Well tuned db systems for SRM or chimera
  • Pool nodes are selected based on
    • directory subtrees
    • client IP subnet (external versus internal versus 'close to farm nodes' networks)
    • protocol(s)
    • data flow direction (put, get pool to pool transfers)

This allows for assigning pools to group of people (they possibly paied for it) or assigning pools according to the importance of the data. Moreover the transport protocol may determine the final pool set as well.

e.g. the system could be configured to store incoming data from remote sites to dedicated transfer pools but copy them to internal read pools as soon as they arrive or as soon as they are requested by users.

  • Protect overall performance by providing queues for the various transfer modes.

By providing transfer queues for read write and 'pool to pool' transfers the system may protect itself from being overloaded by bunches of requests. Transfer reequests exceed a certain threshold will be hold and process later.

  • dCache detects areas of hot access to various groups of pools and can take care that data is transfered to less used pools before delivering it to the client.
  • Mechanisms are available to manually transfer data within dCache between pools in order to shutdown pool nodes without inferering with the overall performance and availability of the system. The entry in the file system stays the same.

dCache w/o HSM and wiht Reslient Module

In addition to the featue presented above the follow mechanism is available using the resilient manager module.

The resilient dCache module takes care that for each single dataset, a configurable number of copies of the file is available on different pool nodes. This garuanties that the outtage of one (or more) pools will not effect the overall availability of the system. Scheduled shutdown of pools can be configured in order to allow dCache to handle the replication of datasets well in advance.

dCache with HSM back end connectivity

In addition to the features presented above the HSM connectivity module provides the following mechanisms.

  • If configured, data is collected based on various criteria and flushed into the HSM back end following rules concerning time intervals or number of data collected without a flush.
  • Transparently to the user, data is refetched from the HSM backend if requested by a user but no longer available on disk.

All features decribed in the paragraphs above apply to an HSM based system as well.


Last modified : 1 May, 2007 by patrick