wiki:manuals/NativeHttpAccess
Last modified 9 years ago Last modified on 10/08/09 15:36:43

Native Unsecure Http access to world readable datasets in dCache

This is exerimental

Prerequisites resp. Restrictions and Limitations

  • The current implementation provided unauthenticated http read access to world readable files in dCache.
  • Http is currently configured to not be used in the SRM transfer protocol negotiation.
  • The actual port number of the data transfer connection is randomly chosen and can't be predicted. Techinally this is not an issue but could be one for systems, protected by filewalls. Consequently, pools serving files by http should have ports above 4096 opened in firewalls and portfilters between the client and the pool server. (connection request are from client to server).
  • The http data mover will always run in the default I/O queue and can't be assigned to an alternative one.
  • This server won't give you access to directory listings.

We will improve this prototype in the future and address those limitations.

Installation Instructions

The native http protocol implementation is part of most recent 1.7.0 distributions, though not configured yet. These instructions apply to 1.7.0-35 and below. We hope to have the http server configuration included in one of the upcoming releases.

  1. Create a file named httpDoor.batch in /opt/d-cache/config with the content of the box at the end of this document. You only need to do this on the host you intend to run the http main door (e.g. head node). Please note that this is only the door which does the redirect to the actual data movers. We would expect that the cpu and memory consumption of the door is not significant. The http movers need not to be configured.
  2.     cd /opt/d-cache/jobs
        ./initPackage.sh
    
  3. The default port number of the http door is '8888'. In case you need this changed, add the following line to the /opt/d-cache/config/dCacheSetup.
      httpPort=<newPortNumber>
    
    Make sure 8888 resp. <newPortNumber> is not occupied by any other web server.
  1. Start the service :
        cd /opt/d-cache/jobs
        ./httpDoor start
    
  1. The http service will not be automatically started/stopped by the dcache-core script. You either have to do it by hand or you need to modify dcache-core. Please keep in mind that this file will be overwritten by a subsequent upgrade.

Configuring the PoolManager for special http pools

No special PoolManager configuration is required as long as the http protocol should be handled by the same pools as the other protocols. Otherwise, if special http pools should be provided, an http protocol unit has to be defined

   psu create unit -protocol Http/*
   psu create ugroup http-protocol Http/*
      ***
   psu create link http-read-link  http-protocol .... 
      ***
   psu add link http-read-link http-read-pools
   

The httpDoor Batch file (httpDoor.batch)

#
# $Id: door.batch,v 1.18.2.2 2006/09/22 16:18:27 patrick Exp $
#
set printout default 2
set printout CellGlue none
onerror shutdown
#
check -strong setupFile
#
copy file:${setupFile} context:setupContext
#
#  import the variables into our $context.
#  don't overwrite already existing variables.
#
import context -c setupContext
#
#   Make sure we got what we need.
#
check -strong serviceLocatorPort serviceLocatorHost
check -strong dCapPort

#
create dmg.cells.services.RoutingManager  RoutingMgr
#
#   The LocationManagerc Part
#
create dmg.cells.services.LocationManager lm 
       "${serviceLocatorHost} ${serviceLocatorPort}"
#
#
onerror continue
   set context -c httpPort             8888   
   set context -c httpMaxLogin         1500
   set context -c httpIoQueue          ""
   set context -c httpIoQueueOverwrite denied
onerror shutdown
#
#    http    D o o r
#
create dmg.cells.services.login.LoginManager Http-${thisHostname} 
            "${httpPort} 
             -export 
             diskCacheV111.doors.HttpDoor 
             -prot=telnet -localOk 
             -maxLogin=${httpMaxLogin} 
             -brokerUpdateTime=30 
             -protocolFamily=http 
             -protocolVersion=1.1.0 
#             -poolProxy=PoolManager 
#             -io-queue=${httpIoQueue} 
#             -io-queue-overwrite=${httpIoQueueOverwrite} 
#             -loginBroker=LoginBroker  
"
#
#

Chimera web server

in addition to http door you may start yet another http door for directory listing.

  • on copy attached chimera-httpd-run.sh into /opt/d-cache/libexec/chimera
  • add into /opt/d-cache/config/dCacheSetup file folloing lines:
    chimeraHttpPort=8084 # 
    httpIoDoor=<http door host>:<port>
    
  • start http listing door
     $ chmod +x /opt/d-cache/libexec/chimera/chimera-httpd-run.sh
     $ /opt/d-cache/libexec/chimera/chimera-httpd-run.sh start
    
  • and use it!

On download, IO request will be redirected to httpDoor.

Be aware, the setup and functionality is a subject to change.

Remark

This is actually independed of whether or not the http protocol is used
There have been various reports that on small installations the authentication, especially for the SRM eventually times out. Although this is an indication that your system is sub optimal dimensioned we would recommend to configure gPlazma to use the module instead of the gPlazma cell. Please adjust your dCacheSetup file as follows :

#
useGPlazmaAuthorizationModule=true
useGPlazmaAuthorizationCell=false
#

Please don't forget to uncomment both statements.


Last modified Wed Sep 19 16:31:24 2018 by patrick

Attachments