wiki:gfd.47
Last modified 6 years ago Last modified on 10/10/12 18:42:30

CURRENT (as of October 2012) GFD.47 SUPPORT IN dCache

Executive summary

Nomenclature

door
client-facing software component: listens for incoming connection and provides support for the control channel and some data connections.
pool
software component responsible for storing files' data

MODE-X is supported for GET and PUT and for active and passive transfers, provided the data connection is between the client and a pool. For other transfers (including STOR and RETR) attempts to use MODE-X will fail.

There is an optional feature where MODE-X data-blocks include a checksum for each block transferred. This is not supported.

A well-behaved client will discover there is no support for per-data-block checksum, so will not sent or expect such checksums. Transfers with a badly behaved client that attempts to use checksums will result in data corruption.

There is no support for the receiver requesting that a datablock is resent; attempts to do this will terminate the transfer.

There is no support for client to set tid value when initiating a transfer. Attempts are ignored; a tid of 0 is always used.

The data-channel implementation is broken as tid is sent as a 64-bit value (spec. says 32-bit).

Support for the EOF command is advertised through the FEAT command but the implementation is broken.

CONTROL CHANNEL

GET command

Supports MODE-X for active and passive transfers provided data is on a pool that is configured to accept redirects.

If (for whatever reason) the door would have to proxy the transfer then MODE-X will fail with a 504 response.

Support for GET options:

mode
supported
pasv
supported
path
supported (required)
port
supported
cksum
not supported (ignored, always as if "cksum=NONE")
tid
not supported (ignored, always as if "tid=0")

PUT command

Supports MODE-X for active and passive transfers provided data is on a pool that is configured to accept redirects.

If (for whatever reason) the door would have to proxy the transfer then MODE-X will fail with a 504 response.

Support for PUT options:

mode
supported
path
supported (required)
pasv
supported
port
supported

Other commands

MODE <mode>
supported
EOF
not supported (500 response)
CKSM
supported
SCKS
supported
OPTS <op> CKSUM <alg>
not supported (501 response)
OPTS PUT EOF
not supported (501 response)
OPTS STOR EOF
does "something":
  1. expects format OPTS STOR EOF <arg> where <arg> is either "0" (disable EOF) or "1" (enable EOF); fails with a 501 response otherwise.
  2. setting this open has no effect (EOF is not supported).
RETR
not supported (fails with 504 response)
STOR
not supported (fails with 504 response)
ERET
not supported (P is normally supported, but fails with 504 response if MODE-X is requested)
ESTO
not supported (A is normally supported, but fails with 504 response if MODE-X is requested)
FEAT
supported. List of features includes: CKSUM, EOF, MODEX, GETPUT. Note that STREAMING is not listed.

DATA CHANNELS USING MODE-X

dCache supports MODE-X as sender and receiver for GET and PUT transfers where the data connection is between the client an a pool.

AS SENDER:

Sends header of 25 bytes (should be 21 bytes)

Transaction ID is sent as 64-bit integer (should be 32-bit). The value is always zero.

Never sends per-block checksum.

Receiver-sent commands:

The length of any one command is limited to 128 bytes. If this limit is exceeded then all connections are terminated and the control channel will receive a 426 response.

Support for receiver-sent commands:

READY supported:

if expected, will start sending data; otherwise, will terminate the transfer by closing all data connections and the control channel will receive a 426 response.

BYE supported:

if expected, the connection is closed; otherwise, will terminate the transfer by closing all data connections and the control channel will receive a 426 response.

CLOSE supported:

if number of connections > 1 then:

sends a block with EOD and SENDER_CLOSES_THIS_STREAM and no data. This is sent after the currently being sent block (if any) has been sent. It then waits for the receiver to send the "BYE" command and then closes the connection.

if number of connections == 1 then:

the command is ignored.

RESEND not supported:

if received will terminate the transfer by closing all connections. The control channel will receive a 426 response.

AS RECEIVER

Expects header of 25 bytes (should be 21 bytes)

Transaction ID read as 64-bit integer (should be 32-bit)

Does not support per-block checksums. If client sends checksum information then uploaded data will be corrupt.

The CLOSE command is never sent.

If the Descriptor has any bits other than EOF, EOD or SENDER_CLOSES_THIS_STREAM then the transfer is terminated by closing all data connections and the control channel will receive a 426 response.

The EOF, EOD or both flag may be set in a Descriptor of an empty block or as part of a data-containing block. The receiver will behave correctly in either case.

Setting SENDER_CLOSES_THIS_STREAM in Descriptor has no effect.

After receiving a block with EOD, the BYE command is sent and the connection is closed. If the sender has already closed the connection before BYE command was sent then the failure to send this command is ignored.

If the sender closes the connection without the last block Descriptor having EOD set then the transfer is terminated by closing all connections and the control channel will receive a 426 response.

If EOF is not received and all data connections are closed (via EOD) then will continue to listen for incoming connections. Transfers may be subject to timeout, resulting in the control channel receiving a 426 response.

Closing a data channel without EOD will terminate the transfer (closing all connections).

After EOF is received, new connections are closed as soon as they are established. Here is a note from code:

From the GridFTP 2 spec: "After receiving EOF block from a sender host, active data receiver host must not try to open any new data channels to that sender host."

This rule is difficult to honor, as we may have a connection establishment "in progress" at the time we received the EOF. At the same time it is not clear if an active receiver is allowed to close a connection before READY has been sent; passive receivers are explicitly allowed to do so.