wiki:developers-meeting-20090128
Last modified 12 years ago Last modified on 01/29/09 12:13:37

dCache developers video meeting

Participants : Fermi(Timur,Dmitry,Gene,Vladimir), NDGF(Gerd), Desy(Paul,Tigran,Patrick,Irina)

Prelim

  • Flavia would like to check GFAL / dCache interoperability. She needs help from us. Please check her e-mail
  • The SRM_BUSY reply needs to be followed up on. Please check e-mail from Flavia with Subject "SRM_BUSY"
  • Alex has problems with SL5 unmount. This might be just a matter of the right sl5 release.
  • 1.9.2 seems to have a minor incompatibility in gPlazma. This was detected in the test system and at NDGF. Ted solved this already ?
  • PermissionHandlerDataSource=diskCacheV111.services.PnfsManagerFileMetaDataSource
  • When we are going to start with a new branches/1.9.3 ?
  • Which layout is more suitable for this command edg-gridftp-ls -v

Layout A

[]$ edg-gridftp-ls -v gsiftp://...../pnfs/example.com/data/dir1
drw                 Nov 17 2008 18:07:06 dteamTest
-rw         14      Nov 18 2008 10:47:02 File1
-rw         14      Nov 18 2008 10:47:19 File2
-r-         288     Nov 18 2008 17:43:39 File3
-r-         300     Nov 18 2008 17:43:47 File4
dr-                 Nov 19 2008 11:45:27 DIR_1
[]$ 

Layout B

[ ]$ edg-gridftp-ls -v  gsiftp://...../pnfs/example.com/data/testDir
-rw-r--r--  594     Jan 13 2009 14:45:18 File1
drwxr-xr-x          Jan 12 2009 12:57:01 TestSuiteDir
drwxrwxr-x          Dec 15 2008 16:05:42 MainTestDir
-rw-rw-r--  28      Dec 09 2008 14:02:15 File2
[ ]$ 

Main Topic(s)

  • SRM speed improvements (Chapter II) based on Timur's analysis of messages profiling.

Summary of the discussion of SRM speed improvements topic

  • After consideration of the list of messages used in SRM-DCACHE communication the following changes were proposed:
    1. Make srmls asynchronous and scheduled
    2. Add information about the client side time allocated for reply wait available in the PnfsManager messages. Make PnfsManager consider its queue and provide a failed reply to the requests that could not be executed in time. Modify SRM code to take advantage of this functionality and start returning SRM_BUSY code to srm clients for the requests that could not be executed after certain number of retries due to the pnfs being busy.
    3. Auto Create indexes for SRM DBs (list of indexes was sent by Gerd earlier)
    4. Optimize PutDone?, do not use PnfsManagerGetStorageInfo, rather use PnfsManagerMapMessage, do not release file reservation in space (CancelUseSpace) if the transfer was successful.
  • The following ideas were also discussed, so they were not yet scheduled to be implemented:
    1. Make TURL unique, make doors aware of transfer related work already accomplished by SRM. Make srm somehow pass negotiated StorageInfo, space reservations, pool info, etc to the door.
    2. Chache pnfs manager replies
    3. Combine to srm space reserve and use messages for implicit reservation in preparetoput.
    4. Make dCache services aware of system load information (CPU/Memory/Network etc.) and use it in scheduling.
    5. Make PnfsHandler aware of the pnfs load problems and hold messages or return failed messages when appropriate. Make PnfsHandler capable of sending messages with methods that provide asynchronous message reply notification.
  • Other recommendations:
  1. Allow NFS caching on SRM Node.

Last Modified by Timur @ Wed Mar 3 06:08:01 2021