wiki:developers-meeting-20101117
Last modified 10 years ago Last modified on 11/17/10 18:18:17

[part of a series of meetings]

Participants

Tanja, Antje, Paul, Owen, Karsten, Tigran, Christian; Dmitry; Thomas

Agenda

[see box on the right-hand side]

Postcards

Up to two minutes (uninterrupted) per person where they can answer two questions:

  • What I did last week (since the last meeting),
  • What I plan to do in the next week.

No questions until we get through everyone :)

Dmitry: continue to work with Catalin with SRM scalability. Split off the SRM client. When-ever you get the request it takes data from the database. The plan is so SRM doesn't hold any queues in the memory. Tested with the client connecting to. Can get ~400 Hz with current test setup. Not seen any degradation of responsiveness. Also looked at RT tickets.

Gerd: worked a bit on the WebDAV door; fixed the HEAD command. Write new login strategy. Write release notes for 1.9.10-3.

Thomas: Put GSI authentication for xrootd up for review; this will give us an authentication mechanism for xrootd, allowing us to interoperate with the Brian Bockelman thing. Fixed the problem reported against xroot opaque and apply to 1.9.10-branch. It's probably also broken in earlier branches but the impact is likely not important. Also also worked on the new HTTP mover.

Christian: ssh2 server + admin interface is in review. Testing that the documentation is correct. EMI work.

Tigran: finish (or hope to) the threadless movers + a few other things. ACLs in NFS

Karsten: Argus plugin work. Next week working on other plugins for gPlazma2. Currently the plugin allows for blacklisting users by DN. Future work may include a mapping plugin.

Owen: EMI more ETICS stuff. Working with EMI on ticket metrics and request-for-change. We will likely need to introduce a new state in RT. Tidying up releases; helping Christian.

Paul: Testing srmCopy; ...

Antje: Documentation and testing.

Tanja: working on NFS; patch on how to kill movers when the session has expired. Working on tickets.

Plans for patch-releases

Should we make a new patch release?

Two fixes: one from Gerd (http) and Thomas (xrootd). People are waiting on them. Both reports in 1.9.10 so releasing 1.9.10.

The pin-manager timeout patch: should we merge this one into 1.9.5? RB entry

Trunk activity

Progress with new features...

SSH

Christian .. Tigran would like to have it asap so we can try it out. The hope is to have this enabled by default for the next 1.9.12. We need to test whether this works with our existing scripts and Jython.

Include support for new interface asap.

Should we switch the Jython over to using SSH v2 client? It currently uses SSH v1 to establish a connection with dCache.

Just need to test with whatever scripts we have; Gerd / NDGF, Dmitry Fermilab, DESY.

Running as non-root

Document how to switch from running dCache as root to running as non-root user.

Pool code

  1. Commit threadless mover code for pool, for NFS movers; it would be nice to have an additional mover to demonstrate the interface is correct.
  1. http or xrootd (or both, as they're both based on Jetty) mover. This should be trivial: the interface is already there (MoverExecutorService? is the interface).
  1. fix other protocols.

The only problem with having idling threads .

500kB per thread is stack space, unallocated by default. The problem is with 32-bit problem.

The ability to throttle the IO by throttling the number of worker threads is useful.

dcap has the issue that the data connection is identified with the client+file. So, if a client opens 1,000 files there's 1,000 network connections.

Billing record format

What we have now is transfer information, stored in billing, is information from door and pool. For accounting, you need to have both pieces of information.

The change that the door will publishing a single record that will have all the currently published door info + information from the pool (pool name, bytes transferred, file size, time-stamp).

In dCache, we make the assumption that every protocol has a control channel and a data channel.

xroot, http/webdav there is no control channel. Currently we do hacks to retain state in the door for each transfer.

Would be nice to move away from the door publishing billing information and have the pool.

Tigran: I'm happy to do it the other way around, but we need to be careful about breaking compatibility with 1.9.5

Whatever compatibility work-arounds we have in place for now, we can drop after the release of 1.9.12 / next golden release.

Tigran to have a look whether it's possible to keep compatibility with new doors with the old pools.

Issues from yesterday's Tier-1 meeting

Not much to report.

KIT

PIC

ulimit support?

Add support for this. Not agreed was what the default value should be.

Issues from EMI

We currently have poorly defined "request for change" process in EMI.

dCache server: we've got official agreement to move to using the Sun JDK (from OpenJDK). However, this process is being slow and painful.

RT will likely need another state to comply with EMI's RT.

Do we comply with our license?

Owen: ETICS is a pile of steaming poo.

srmCopy

Paul to send Dmitry the information on the problem.

A lot of S2 tests failing randomly. Particularly on the memory-contained machines.

"put overwrite transfer" test.

SRM client

In previous meeting, we agreed Dmitry would look at patches on trunk and see what we need to be back-ported. There is only one thing to merge into 1.9.5: support in the client for async. srmLs. Tigran: this is already back-port to 1.9.5

Left to do is back-port the fix for the shell-script fix. The ability to limit memory usage is part of that patch.

Tigran will merge the shell patch.

Owen will test client against our server, DPM, Castor, etc.

The next Golden release

Current status of tickets? People to update the tickets to make sure the title is accurate and add any comments necessary.

Outstanding RT Tickets

[This is an auto-generated item. Don't add items here directly]

xroot issues

Tigran: to create bug reports for the two issues in xrootd. Talking to Thomas for information.

Review of RB requests

DTNM

Proposed: same time, next week.