Ticket #217 (new bugs)

Opened 12 years ago

Improvement of the p2p mechanism (RT 4324)

Reported by: patrick Owned by: paul
Priority: major Milestone: 1.9.3
Component: core Keywords: p2p load balanceing external script
Cc: Sub Version:


This is a script Jon is applying to the US CMS system and seems to work fine. We should integrate his suggestion into the core dCache code.

Jons suggestions: The algorithm is easy - I just sort the costs numerically, and then choose the entry that corresponds to, say, 95% of the pools having a cost less than chosen value. This picks out hot pools and does p2p for those pools, and ignores other ones and lets requests queue up.

Plus, I now adjust the value every 5 minutes so it keeps the pools from going into a p2p frenzy when user load changes. The script has a base value, corresponding to "full" default mover queues, so the costcut can never go too low and start unwanted P2Ps.


Please consider adding such a feature to the dCache base code - it is really useful for me.

I should tell you I also have a spillway script that monitors the queues and opens and closes the gates every so often allowing more transfers to start if I don't detect any changes for the active movers (our users keep connection active long after transfer is done - new so-called lazy download mode in the cms framework).

Also, we need a way to exclude each queue from the cost - not just the restores and stores.

Note: See TracTickets for help on using tickets.