wiki:PNFS.InstallationNotesOnOpenSolaris_Usage
Last modified 14 years ago Last modified on 08/09/07 14:33:21

Instructions for Setting up and Using pNFS

There are 3 main components of a pNFS configuration; a client, a metadata server (MDS) and a data server (DS). The BFU archives that you have installed allow you to configure and run each of these components. What follows is information on how to get each component configured and running. If you have any problems with these instructions or find any errors in them along the way, send an email to nfsv41 dash discuss at opensolaris dot org.

The DS and the MDS can, but aren't required to be configured to run on the same machine. Running them on the same machine is handy for easy setup; especially if you want to get going without a whole lot of effort.

For the basic setup, we recommend that you have two machines. One to use as the client and another to use as the combined MDS and DS. The following instructions give specific examples in order to create this exact setup, but the examples can be extended in order to create different configurations (e.g. MDS and data server(s) on multiple different machines). None the less, if you are just starting out with pNFS, we recommend for you to try the basic setup first and then build off of that.

Please note that the command line interfaces shown here are prototypical and will be changing in the future. Quick and Easy Reference

This section is a quick reference to the commands you will need to issue in order to get your systems configured and running. More detailed information on what the commands do is also available below!

On the Metadata Server

#sharemgr create <group>
#sharemgr add-share -s <path> <group>
#mdsadm -o add -t auth -a ip=<ip-addr-of-ds>

Note: If the DS is on the same machine as the MDS, you can skip the last command.

On the Data Server

#zpool create <pool> <vdev>
#dservadm addmds 127.0.0.1.8.1
#dservadm addpool <pool>
#dservadm enable

On the Client

#mount -F nfs -o vers=4 <resource> <mount point>

The more detailed information…

This section covers the same information as the "Quick and Easy Reference", but goes into more detail on what each command does. Pre-configuration – Turn on Debugging (optional)

  1. To turn on debugging, add the following entry to /etc/syslog.conf:
    • *.info /var/adm/messages
  2. Set dserv:dserv_debug to 1 in /etc/system

Instructions for Setting up a Metadata Server

Share a file system:

#sharemgr create <group>
#sharemgr add-share -s <path> <group>

mdsadm Command

The configuration of the metadata server (MDS) is done with the mdsadm command. It allows you to input a list of data server "authorization" records and layouts for the MDS. The syntax is as follows:

mdsadm -o <operation> -t <type>
   Valid values for operation include "add"
   Valid values for type include "auth" and "layout"

A data server "authorization" record notifies the metadata server that the given data server will be communicating with it.

A layout determines how the file will be laid out at creation time (e.g. 3 way stripe, 32K stripe unit or interlace) Using the mdsadm Command to Configure a Metadata Server

mdsadm -o add -t auth -a ip=<ip-addr-of-ds>
  • Note: If the DS is on the same machine as the MDS, you can skip this command.
  • A default layout will be generated based on the device information reported via the control-protocol by the data-servers
  • This information is not currently stored persistently. Therefore the above step will need to be done after every reboot of the metadata server.

Instructions for Setting up a Data Server

Create a zpool

There should be at least one pool per data server instance, so if you are planning on having multiple instances on your data server, be sure to create at least one pool per instance. For now we recommend using a single instance of the data server.

#zpool create <pool> <vdev>

Simple (single instance) Data Server Configuration

dservadm Command

The configuration of the data server is done using the dservadm command. Please note that the dservadm command line interface is an interim solution. This command will be replaced with a more complete CLI in the near future. The syntax is as follows:

dservadm create <instance>
    Creates an instance of the data server with the name as given.  This can
    be used to create multiple instances of the data server on one physical
    machine.  By default, there is one instance of the data server.  This
    is referred to as the default instance.  This capability is experimental
    and is not expected to be delivered with the final pNFS implementation.
dservadm destroy <instance>
    Deletes the instance of the data server with the name given.
    This capability is experimental and is not expected to be delivered
    with the final pNFS implementation.
dservadm enable [ -i <instance> ]
    Starts the data server SMF service.  If given an instance name,
    that instance will be started.  If not given an instance name, the
    default instance will be started.
dservadm disable [ -i <instance> ]
    Stops the data server SMF service.  If given an instance name,
    that instance will be stopped.  If not given an instance name,
    the default instance will be stopped.
dservadm addpool [ -i <instance> ] <zpool-name>
    Adds the given zpool to the data server for use in doing I/O.  If
    given an instance name, the zpool will be added to that instance.
    If not given an instance name, the zpool will be added to the default
    instance.
dservadm droppool [ -i <instance> ] <zpool-name>
    Deletes the given zpool from the data server.  This will stop the
    data server from using the zpool to do I/O.  If given an instance
    name, the zpool will be deleted from that instance.
    If not given an instance name, the zpool will be deleted from the
    default instance.
dservadm listpools [ -i <instance> ]
    Lists the zpools that are available to the data server.  If given an
    instance name, the zpools that are available to that instance are
    displayed.  If not given an instance name, the zpools for the
    default instance are displayed.
dservadm addmds [ -i <instance> ] <mds-universal-address>
    Adds the given metadata server universal address to the data server.
    This will allow the data server and metadata server to communicate.
    If given an instance name, the mds is added to that instance.
    If not given an instance name, the mds is added to te default instance.
    There can only be one metadata server added to any instance of the
    data server.
dservadm dropmds [ -i <instance> ]
    Deletes the metadata server.  This will remove the
    metadata server from the data server and will stop the communication
    between the two.  If given an instance name, the mds will be deleted
    from that instance.  If not given an instance name, the mds will be
    deleted from the default instance.
dservadm listmds [ -i <instance> ]
    Lists the metadata server that the data server is in communication
    with.  If given an instance name, the mds that is in use by that
    instance is displayed.  If not given an instance name, the mds being
    used by the default instance are displayed.

Using dservadm to Configure a Data Server

  • Add the address of the MDS for use by the data server:
    # dservadm addmds 127.0.0.1.8.1
       (Note: 127.0.0.1 is the IP address, 8.1 is the port number
       (8.1 or (8*256 + 1) = 2049).  Unless you went out of your way to
       change the port that the Metadata Server is running on, this will
       always be 8.1)
    
  • To list the MDS that has been assigned to the data server:
    #dservadm listmds
    
  • Add the zpool to the list of pools for use by the data server:
    #dservadm addpool <pool>
    
  • To list the zpools that have been added for use by the data server:
    #dservadm listpools
    
  • Start the data server service:

Note: If you are NOT running an MDS on the same machine and haven't shared a file system, you won't be able to enable the data server service. Having a share present on the data server machine is currently required because the data server service is dependent upon the NFS server service and that service isn't enabled until something is shared.

#dservadm enable
  • To determine the state of the data server service:
    #svcs dserv
    STATE          STIME        FMRI
    online         19:29:24    svc:/network/dserv:default
    

Instructions for Setting up a Client

#mount -F nfs -o vers=4 <resource> <mount point>
    The resource will be the shared file system on the metadata server
    (e.g. mds:/export)
    The mount point is where you want the file system mounted on the client.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Last modified Fri Feb 26 22:49:40 2021 by Björn Böttcher < bjoern.boettcher _AT_ desy.de>
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-