Last modified 13 years ago Last modified on 05/14/08 18:29:05

Discussion on storage entities in Glue-1.3

The following wiki is an attempt to capture an email thread that includes comments from Maarten Litmaath (ML) and Stephen Burke (SB).

The email was originally in the form of a series of (numbered) questions about the GSSD proposal for representing SRM information within Glue v1.3.

Please note The comments may have been edited slightly to better fit the wiki. Please do not consider the information here as a verbatim copy of what these people have said.

Intro comments

For example, a GlueSA object having GlueSALocalID lhcb:custodial:nearline raises several questions:

is the form

"<VO>:<ret. policy>:<access latency>", or
"<VO>:<space token description>", or
"<space token description>" ?
(perhaps, with a site-local naming policy resulting in the observed


Is the observed form of GlueSALocalID required or just a coincidence?

ML: A LocalID only has be unique locally: a strict format is not required. However, info providers need a repeatable algorithm to determine the exact string for a particular case, and it probably ought to be human- readable, even when that gives rise to potential misuse, e.g. when the VO name was used for GlueSALocalID.

If the GlueSALocalID is of the form <vo>:<ret. policy>:<access latency>, does

this represent aggregated, summary information for all SRM spaces for the stated VO, with the stated retention policy and access latency?

ML: If the SA represents a storage class, it amounts to a summary of all the SRM v2.2 spaces that have that storage class. In that case it is natural to construct the SALocalID as in the example.

Note that the SRM developers seem to have dropped their concerns about publishing the used/reserved/free sizes of individual, static SRM v2.2 spaces, so we should rather have each of them published as a GlueSA, instead of a summary per storage class.

SB: As I've said before I think we really should have an SA per space - e.g. I now use this a lot to look at the free and used space per space token in castor. Is there any fundamental block to doing that for dcache?

If the LocalIDs are required to be derived from a <space token descriptions>, how should the uniqueness, required by GlueSALocalID, be effected?

The following seems natural:

    <VO>:<space token user description>

ML: Different VOs can use the same space token user descriptions, so there has to be a <VO> field to disambiguate them.

Some general queries and comments

1. the expected number of each objects doesn't seem to be defined. From the example, I'd guess there should be:

o precisely one GlueSE;

ML: Yes.

o a GlueSA ("backward compatibility Glue-1.2") object for each supported VO,

all immediate children of the GlueSE;

ML: Yes.

o a GlueSA object for each SRM space, all immediate children of the GlueSE;

ML: Yes, as long as the aforementioned concerns do not reappear; otherwise a summary per storage class.

o a VOInfo object for each VO the space supports, all immediate children of

the GlueSA corresponding to the space.

ML: Yes.

2. From memory, someone (maybe Stephen Burke) mentioned that publishing GlueSchemaVersion? information for each object is now deprecated and that only the top-level site object (or top-level site objects, like GlueSE) should publish objects with GlueSchemaVersion class: child objects are assumed to be of the same GLUE version. If so, could the example be updated to remove the GlueSchemaVersion class, where appropriate?

ML: The gLite 3.0 BDII uses this search filter:


It has been obsoleted by the gLite 3.1 BDII, which uses '*' instead, but there probably remain quite a few gLite 3.0 instances out there, that would be broken by leaving out the GlueSchemaVersion... I would rather leave it in for now.

SB: Actually it's mentioned in the page on Sergio's web site that you pointed to, and I said should be deprecated itself! As Maarten says, for now I think we should continue to publish those attributes.

3. The usage of GlueChunkKey and GlueForeignKey does not appear to be well defined. Could a statement be added that defines precisely when these attributes are to be included?

ML: I leave that to Stephen or Laurence.

SB: The ForeignKey is used to express object-object relations beyond the ones implied by the LDAP tree, as indicated in the schema document, e.g. service to site and service to service. ChunkKey is used as a helper for relations within the tree (as discussed before you can also query for DN elements but the ChunkKey still has its uses). I'm not sure there is any absolute rule for ChunkKey, it depends on what queries you'd be likely to want, but looking at what we publish, for the SE we seem to have chunk keys for the SE ID in every other object, and for the VOInfo there is also one for the SA, i.e. basically we have them for everything.

4. For each attribute, could a statement, stating which fields are required and which are optional, be added? Also, for each attribute, which values are deprecated (particularly for compound and enumerated attribute types)? This is most applicable for deprecated fields, but also applies to others as GLUE v1.3 requires almost no attributes to be published. It would also help defined the time-scale within which deprecated fields are to be removed.

ML: I intended the example to show exactly the fields that we want in LCG/EGEE, whatever the GLUE 1.3 documentation says about the various attributes.

5. The Glue v1.3 specifies that the non-depriated size attributes should be in "GB", whilst some (depricated) attributes are in "KB" or "KByte". Could it be clarified whether "GB" refers to Gigabytes (109 bytes) or Gibibytes (230 Bytes)? Similarly is "kB" referring to kilobytes (103 bytes) or kibibytes (210 bytes)?

ML: LCG has decided that the prefixes denote powers of 10. The GLUE docs should state this explicitly.

SB: The 2.0 schema doc does say that.

Questions about GlueSE object

6. GlueSEUniqueID. This field appears to be a FQDN. Is this a required form of the GlueSEUniqueID or just a coincidence? Can this be documented in the example?

ML: It is a "coincidence" (which has led to misuse, indeed).

SB: And it may be unsafe to change it as I suspect there is code making that assumption (or you could change it anyway and see what breaks in certification!)

7. GlueSEName This appears to duplicate the GlueSEUniqueID. Is this duplication required behaviour? If it is optional, can a site choose simply not to publish a GlueSEName? (I believe GLUE/LDAP schema does not require GlueSEName to be published).

ML: The GlueSEName used to be vital in determining the type of the SE:

    GlueSEName: GR-01-AUTH:srm_v1
    GlueSEName: GR-03-HEPNTUA:disk

ML: The gLite 3.1 GFAL code has not looked at the GlueSEName since a while, and there are more and more SRMs without the type suffix, so it should be OK to drop the attribute, if you want.

8. GlueSEPort This field is marked as deprecated in Glue v1.3; also, the attribute does not appear to be well-defined in the documentation. Could this field either be dropped or better documented?

ML: It was kept because older GFAL versions could refer to it, but by now I think we can drop it.

9. GlueSEArchitecture. Is it required to publish this field? If so, under what circumstances should an SE describe itself as "tape", "multidisk", "disk"? Are any other values permitted? Can this be documented?

ML: AFAIK there are no data management clients that care about the attribute; the idea was that a client might select SEs with the same guaranteed level of quality:

    tape > multidisk > disk

It also allows people to discover what kinds of setups exist out there.

SB: This is part of the great debate about hardware representation, it was supposed to be a minimal amount of information about what the hardware actually is. In practice no-one looks at it, which possibly reinforces the point that people don't need to know about the hardware even if they think they do ... anyway, the values are documented in the schema doc. The reason for disk vs multidisk is that it used to be common to have SEs which were just a single PC with its internal disk used as the storage. Now I guess all real-world SEs probably have disk arrays so they would all be multidisk.

10. GlueSESizeTotal. Does Total size include tape storage; if so, what semantics are expected for the tape component of this value?

ML: In the example I added disk and tape like this:

    T1D0[nearline] + T0D1[online] + T1D1[online or nearline]

That is, the GlueSESizeTotal shows the total amount of distinct data that can be stored in the system.

11. GlueSESizeFree. What does "free" mean in this context? For example, are (non-essential) replicas include in free space? Should space used by staged files be included? Can a site choose to not publish this attribute (technically, it is optional in GLUE).

ML: The free space should be generally usable for new files to be stored. Non-essential replicas will be removed when space is needed, so they should not be counted in the used space. A pinned staged file counts in the used space of the online component of a T1D0 space. Reserved space is not free, since it is not generally usable.

The GlueSESizeFree attribute makes much more sense for a Classic SE; an SRM probably should not publish it.

SB: However, we should check for use in e.g. gstat, which I suspect is taking its free and used space from the SE-level values. Those are numbers which tend to get reported in high-level meetings by people like John Gordon or written in press releases, so it would be nice if they could bear some resemblance to reality!

12. GlueSEUsedOnlineSize What does "used" mean in this context? Does used space include all replicas an SE might use for load-balancing?

ML: No. See above. In the future we may want to have more categories than used, reserved and free.

13. GlueSETotalNearlineSize, GlueSEUsedNearlineSize From what I understand from the SRM MoU, no effort is made to represent to near-line component of storage. For this reason, can the nearline figures be omitted? (Technically, they can, as the attributes are optional in Glue v1.3)

ML: Yes, though having approximate values published would be nice.

14. Is publishing the GlueInformationServiceURL still required? I believe it is no longer in use by end-clients.

ML: Stephen/Laurence?? I doubt it has ever been used at all...

SB: No, I don't think it's used, although equally it does no particular harm to publish it. I don't think we require that resource BDIIs have to be externally accessible so it might not be usable in practice anyway.

"glue-1.2" (backward compatibility) GlueSA objects

15. Is these glue-1.2-backward-compatibility GlueSA objects to support clients that don't understand Glue v1.2 or to allow publishing of information for VOs that choose not to use SRM spaces?

ML: Both. For the latter case the idea is that an SRM v2.2 default space will be published, but that has not been fully sorted yet.

16. Are the values of these glue-1.2-backward-compatibility GlueSA objects required to have backwards compatible semantics? For example, are these values required to be complete independent of any SRM spaces?

ML: See (*) below.

17. The GlueSALocalID appears to be the VO name. Is this a requirement or a coincidence? Can this be documented within the example?

ML: For the 1.2 SA it is a requirement. I will indicate that.

18. GlueSAStateAvailableSpace, GlueSAStateUsedSpace. These fields are deprecated in v1.3. Instead, the values should be published as TotalOnlineSize, etc. Should these new fields be used instead of the deprecated fields?

ML: The 1.2 SA would need the deprecated fields to allow RB/WMS gang matching of jobs to select SEs with sufficient space. Again, this does not make much sense for anything other than Classic SEs. The new attributes would not help a lot either in this case.

SB: We should also check the storage accounting system to see what it looks at.

19. What is the relationship between these backward compatible GlueSA objects and those GlueSA objects representing SRM spaces? For example, should the 1.2-backward-compatible GlueSA represent all space that a VO has access, all space not accounted for by SRM spaces, or some other relationship?

ML: (*) In the example I added all the spaces to obtain the Available and Used sizes, but your question made me realize that it may be better to think of the 1.2 SA as representing what an SRM v1.1 client would experience: the data would be stored outside of any listed space, possibly except for the default space, whose semantics have not been fully decided yet. That is, an SRM v1.1 client might actually write into the v2.2 default space.

20. In the example, there appears to be only one GlueSAAccessControlBaseRule attribute per glue-1.2-backward-compatibility GlueSA. Is this a (proposed) requirement when publishing these objects?

ML: At least the plain VO ACBR must be there. In an earlier version there also was the VO:VO ACBR, but I recently removed it to make it clearer that the 1.2 SA is for backward compatibility. I could put it back in...

21. The GlueSAAccessControlBaseRule attributes appear to be simple VO names, with no "VO:" prefix and no "VOMS-based" authorisation. Is this required behaviour?

ML: See above.

GlueSA objects for an SRM space

22. the GlueSALocalID in the first example is atlas:custodial:nearline, which appears to be of the form <VO>:<Ret. policy>:<Acc.latency>. Is this a required form of GlueSALocalID? If so, what parts (if any) correspond to the space token description? If any part is the SRM space-token description, how is the local uniqueness (that GlueSALocalID requires) enforced?

ML: See discussion near the top of this message.

23. GlueSAStateAvailableSpace, GlueSAStateUsedSpace. Are these deprecated fields required? The same information is available as the new (with GLUE v1.3) *Size attributes. If both the deprecated and new attributes are required, could the precise relationship between them be documented?

ML: I suppose we can remove the deprecated fields here.

SB: As above, we should check the storage accounting.

ML: Of the new ones as many as possible and reasonable should be provided. An T0D1 space should not publish nearline sizes (which would all be zero). A static space should not publish reserved sizes: that attribute is to indicate the total _dynamically_ reserved space.

24. GlueSAAccessControlBaseRule; only attributes with values of the form "VO:<vo-name>" are published, no attributes with just the VO name (which appear in the "Glue-1.2" GlueSA objects). Is this correct? If so, can this be documented?

ML: Correct: the new SAs follow the new ACBR schemes. I will indicate that.

25. The ATLAS example publishes size information for near-line storage (GlueSATotalNearlineSize, etc...). Publishing space information associated with nearline storage seems to run counter to the SRM MoU, which considers only storage with online latency. Whilst individual sites may choose to publish nearline size information, making this compulsory seems to run counter to the spirit of the MoU. If this information is optional, can this be documented?


Questions about VOInfo objects

26. There appears to be precisely one GlueVOInfo object per VO that an SRM space supports. Is this correct? If so, can this be documented?

ML: There can be zero or more. An SA might have different tags for different groups of users, probably with different paths as well. The GLUE 1.3 spec should document the multiplicity.

27. The examples appear to have GlueVOInfoLocalID attributes of the form: <VO>:<string>. Are GlueVOInfoLocalID required to have attribute values of this form or is this a coincidence?

ML: See discussion near the top of this message.

28. Is the <string> the space token description, or derived from the space token description?

ML: Ditto.

29. Assuming VOInfoLocalID is derived from a space token description, which suffers no uniqueness constraint, and given that a VOInfoLocalID is required to be unique, how is this requirement to be maintained?

ML: The space token user description is unique within the VO.

30. The GlueVOInfoName appears to duplicate the information provided by GlueVOInfoLocalID. Is this required behaviour? Can a site publish a different GlueVOInfoName? Can a site refrain from publishing GlueVOInfoName? (Technically, it can, as GlueVOInfoName is an optional attribute)

ML: See earlier discussion.

SB: As a general point, the Name attributes are things you might put in a monitoring display or which would be otherwise helpful to someone looking at the content of the info system, so you can either put something that seems useful or leave it out.

31. Is the attribute GlueVOInfoTag the SRM space token description? If so, can this be clearly stated somewhere?

ML: Yes and OK.

Questions about AccessProtocol objects

32. Can the different valid GlueSEAccessProtocolType attribute values be enumerated somewhere? For example, is it gridftp, GridFTP, Gridftp, gsiftp, GSIFTP, GsiFTP, ...?

ML: Stephen/Laurence?? The GLUE web pages should provide the enumeration. BTW, the winner is: gsiftp.

SB: It's here, although as we discussed this should possibly move to the OGF site. Note that this is something which needs to be consistent between SRM and GLUE because SRMs have to recognise these names for GetTURL. My personal view is that it would be better for SRM to define the names and for GLUE to reference that, but in past discussions it has appeared that the SRM people don't want to do that and would rather keep the GLUE list as definitive.

Questions about ControlProcol objects

33. GlueSEControlProtocolType indicates SRM (as capitals). Is this case-sensitive?

ML: Yes. See previous.

SB: The 2.0 schema now explicitly says that everything is case-sensitive. In the past we've had problems because some applications are case-insensitive (notably ldapsearch!) so people have got sloppy

Questions about SRM-derived Service objects

SB: In fact Service is a separate issue anyway - I've written a dynamic publisher for it which should replace the current static configuration. I'll send a separate mail about that.

34. GlueServiceType: is the correct value SRM (as capitals).

ML: Yes.

35. GlueServiceURI, GlueServiceAccessPointURL: these attributes don't appear to be in Glue-v1.3 documentation, although they do exist in the GLUE-1.3/LDAP schema (marked with "D" for deprecated). Are these fields required? (No Glue v1.3-conforming software should be querying for them.)

ML: Older GFAL versions used such deprecated attributes. By now I think they can be removed.

36. The GlueServiceURI, GlueServiceAccessPointURL attribute values appear to be identical to the GlueServiceEndpoint. Is this a requirement? If so, can this be documented?

ML: Yes, but see previous.