/usr/share/doc/coop-computing-tools/manual/chirp.html

<html>

<head>
<title>Chirp User's Manual</title>
</head>

<h1>Chirp User's Manual</h1>
<b>July 2011</b>
<p>
Chirp is Copyright (C) 2003-2004 Douglas Thain and
Copyright (C) 2005- The University of Notre Dame.
All rights reserved.
This software is distributed under the GNU General Public License.
See the file COPYING for details.
<p>
<b>Please use the following citation for Chirp in a scientific publication:</b>:
<dir>
<li>Douglas Thain, Christopher Moretti, and Jeffrey Hemmes, <a href=http://www.cse.nd.edu/~dthain/papers/chirp-jgc.pdf>Chirp: A Practical Global Filesystem for Cluster and Grid Computing</a>, <i>Journal of Grid Computing</i>, Springer, 2008. DOI: 10.1007/s10723-008-9100-5.  (The original is available on <a href=http://www.springerlink.com>http://www.springerlink.com</a>.)
</dir>

<h2>Overview</h2>

Chirp is a system for performing input and output across the Internet.
Using Chirp, an ordinary user can share storage space and data with
friends and colleagues without requiring any sort of administrator
privileges anywhere.
<p>
Chirp is like a distributed filesystem (such as NFS) except that it can be
run over wide area networks and requires no special privileges on
either the client or the server end.  Chirp allows the end user to set
up fine-grained access control so that data can be shared (or not shared)
with the right people.
<p>
Chirp is also like a file transfer system (such as FTP) that provides
streaming point-to-point data transfer over the Internet.  However, Chirp
also provides fine-grained Unix-like data access suitable for direct
access by ordinary programs.
<p>
Chirp also includes several advanced features for authentication tickets, space allocation, and more.
However, each of these features must be explicitly enabled, so you don't
have to worry about them if all you want is simple storage access.
Read on below for more details.
<p>
Begin by <a href=http://www.cse.nd.edu/~ccl/software/manuals/install.html>installing the cctools</a> on your system.
When you are ready, proceed below.

<h2>Running a Chirp Server</h2>

Running a Chirp server is easy.  You may run a Chirp server as any ordinary
user, and you do <b>not</b> need to install the software or even run
the programs as root.  To run a Chirp server, you must do three things:
pick a storage directory, run the server, and then adjust the access control.
<ol>
<p>
<li> <b>Pick a storage directory.</b>  The Chirp server will only allow
access to the directory that you choose.  It could be a scratch directory,
your home directory, or even your filesystem root.   For now, let's store
everything in a temporary directory:
<pre>
/tmp/mydata
</pre>
<p>
<li> <b>Run the server.</b>  Simply run <tt>chirp_server</tt> and direct
it to your storage directory:
<pre>
% chirp_server -r /tmp/mydata &
</pre>
<p>
<li> <b>Adjust the access control.</b>  When first started, the Chirp
server will allow access only to YOU from the same host.  You will
probably want to change this to allow access to other people and hosts.
To adjust the access control, use the <tt>chirp</tt> tool and the
<tt>setacl</tt> command to set the access control list.
For example, to also allow other hosts in your domain to read and
write the server:
<pre>
% chirp localhost
 chirp:localhost:/> setacl . hostname:*.mydomain.edu write
</pre>
</ol>

Now that you have a server running on one machine, let's use some
tools to move data to and from your server.

<h2>Accessing Chirp Servers</h2>

The easiest way to access Chirp servers is by using a tool called
<b><a href=http://www.cse.nd.edu/~ccl/software/parrot>Parrot</a></b>.  Parrot is a
personal virtual filesystem: it "speaks" remote I/O operations on behalf
of ordinary programs.  For example, you can use Parrot with your regular
shell to list and access Chirp servers like so:
<p>
<center>
<table width=90% border=4>
<tr >
<td bgcolor=#eeee77>
<pre>
 % parrot_run tcsh
 % cd /chirp
 % ls
angband.somewhere.edu:9094
dustpuppy.somewhere.edu:9094
peanuts.somewhere.edu:9094
...
 % cd /chirp/peanuts.somewhere.edu
 % cp /tmp/bigfile .
 % ls -la
total 804
drwx------    2 fred   users          4096 Sep 10 12:40 .
drwx------    2 fred   users          4096 Sep 10 12:40 ..
-rw-r--r--    1 fred   users      104857600 Sep 10 12:57 bigfile
-rw-r--r--    1 fred   users           147 Sep 10 12:39 hosts
 % parrot_getacl
unix:fred rwlda
hostname:hedwig rl
...
</pre>
</table>
</center>
<p>
(If you are having difficulting accessing your server,
have a look at "debugging hints" below.)
<p>
Parrot is certainly the most convenient way to access storage, but
it has some limitations: it only works on Linux, and it imposes
some performance penalty.
<p>
You can also attach to Chirp filesystems by using the FUSE package
to attach Chirp as a kernel filesystem module.  Unlike Parrot,
this requires superuser privileges to install the FUSE package,
but will likely work more reliably on a larger number of programs.
You can do this with either <a href=http://fuse.sourceforge.net>Linux FUSE</a>
or <a href=http://code.google.com/p/macfuse>MacFuse</a>.
Once you have downloaded and installed FUSE, simply run <tt>chirp_fuse</tt>
with the name of a directory on which the filesystem should be mounted.
For example:
<center>
<table width=90% border=4>
<tr >
<td bgcolor=#eeee77>
<pre>
% mkdir /tmp/chirp
% chirp_fuse /tmp/chirp
% cd /tmp/chirp
% ls -la 
total 9742
dr-xr-xr-x   0 fred users    6697 Feb 22 13:54 albus.cse.nd.edu:9094
dr-xr-xr-x   0 fred users    6780 Feb 22 13:54 aluminum.helios.nd.edu:9094
dr-xr-xr-x   0 fred users   27956 Feb 22 13:54 angband.cse.nd.edu:9094
dr-xr-xr-x   0 fred users    6466 Feb 22 13:54 angelico.helios.nd.edu:9094
...
</pre>
</table>
</center>

For more portable, explicit control of a Chirp server, use the
Chirp command line tool.  This allows you to connect to a server,
copy files, and manage directories, much like an FTP client:
<p>
<center>
<table width=90% border=4>
<tr >
<td bgcolor=#eeee77>
<pre>
 % chirp
 ...
 chirp::> open myhost.somewhere.edu
 chirp:myhost.somewhere.edu:/> put /tmp/bigfile
file /tmp/bigfile -> /bigfile (11.01 MB/s)
 chirp:myhost.somewhere.edu:/> ls -la
dir      4096 .                                        Fri Sep 10 12:40:27 2004
dir      4096 ..                                       Fri Sep 10 12:40:27 2004
file      147 hosts                                    Fri Sep 10 12:39:54 2004
file 104857600 bigfile                                 Fri Sep 10 12:53:21 2004
 chirp:myhost.somewhere.edu:/>
</pre>
</table>
</center>
<p>
In scripts, you may find it easier to use the standalone commands
<tt>chirp_get</tt> and <tt>chirp_put</tt>, which move single files
to and from a Chirp server.  These commands also allow for streaming
data, which can be helpful in a shell pipeline.  Also, the -f option
to both commands allows you to follow a file, much like the Unix
<tt>tail</tt> command:
<p>
<center>
<table width=90% border=4>
<tr >
<td bgcolor=#eeee77>
<pre>
 % tar cvzf archive.tar.gz ~/mydata
 % chirp_put archive.tar.gz myhost.somewhere.edu archive.tar.gz
 % ...
 % chirp_get myhost.somewhere.edu archive.tar.gz - | tar xvzf
 % ...
 % chirp_get -f myhost.somewhere.edu logfile - |& less
 %
</pre>
</table>
</center>
<p>
You can also write programs that access the Chirp C interface directly.
This interface is relatively
self explanatory: programs written to use this library may perform
explicit I/O operations in a manner very similar to Unix.
For more information, see the <a href=http://www.cse.nd.edu/~ccl/software/howto/code.shtml>HOWTO: Write Code that Uses Chirp</a>

<h2>Finding Chirp Servers</h2>

Now that you know how to run and use Chirp servers,
you will need a way to keep track of all of the servers that
are available for use.
For this purpose, consult the Chirp <a href=http://catalog.cse.nd.edu:9097>storage catalog</a>.
This web page is a list of all known Chirp servers and their locations.
Note that this same list appears if you use Parrot to perform an <tt>ls</tt>
on <tt>/chirp</tt>
<p>
The storage catalog is highly dynamic.  By default, each Chirp server
makes itself known to the storage catalog every five minutes.  The
catalog server records and reports all Chirp servers that it knows
about, but will discard servers that have not reported for fifteen minutes.
<p>
If you <b>do not</b> want your servers to report to a catalog, then
run them with this option:
<pre>
% chirp_server -u -
</pre>
<p>
Alternatively, you may establish your own catalog server.  This can be useful for keeping
your systems logically distinct from the main storage pool, but can also help
performance and availability if your catalog is close to your Chirp servers.
The catalog server is installed in the same place as the Chirp server.
Simply run it on any machine that you like and then direct your Chirp servers to
update the new catalog with the -u option.  The catalog will be published
via HTTP on port 9097 of the catalog machine.
<p>
For example, suppose that you wish to run a catalog server on a machine
named <tt>dopey</tt> and a Chirp server on a machine named <tt>sneezy</tt>:
<pre>
dopey% catalog_server
...
sneezy% chirp_server -u dopey [more options]
</pre>
Finally, point your web browser to:
<pre>
http://dopey:9097
</pre>
Or, set an environment variable and use Parrot:
<pre>
% setenv CATALOG_HOST dopey
% parrot_run tcsh
% ls /chirp
</pre>
And you will see <a href=http://catalog.cse.nd.edu:9097>something like this.</a>
You may easily run multiple catalogs for either scalability or fault tolerance.
Simply give each Chirp server multiple -u options to name each running catalog.
<p>
(Hint: If you want to ensure that your chirp and catalog
servers run continuously and are automatically restarted after an upgrade,
consider using <a href=watchdog.html>Watchdog</a>.)

<h2>Security</h2>

Now that you have an idea how Chirp can be used,
let's discuss security in more detail.
Different sites require different levels of security
and different technological methods of enforcing security.
For these reasons, Chirp has a very flexible security system
that allows for a range of tools and policies
from simple address checks to Kerberos authentiation.
<p>
Security really has two aspects: authentication and authorization.
Authentication deals with the question "Who are you?"
Once your identity has been established, then authorization
deals with the question "What are you allowed to do?"
Let's deal with each in turn.

<h3>Authentication</h3>

Chirp supports the following authentication schemes:
<p>
<table>
<tr><td><b>Type</b><td><b>Summary</b><td><b>Regular User?</b><td><b>Root?</b>
<tr><td><td><td>(non-root)<td>(root)
<tr><td>kerberos   <td> Centralized private key system<td> no <td> yes (host cert)
<tr><td>globus     <td> Distributed public key system<td> yes (user cert) <td> yes (host cert)
<tr><td>unix       <td> Authenticate with local unix user ids. <td> yes <td> yes
<tr><td>hostname   <td> Reverse DNS lookup <td> yes <td> yes
<tr><td>address    <td> Identify by IP address <td> yes <td> yes
</table>
<p>
The Chirp tools will attempt all of the authentication types
that are known and available in the order above until one works.
For example, if you have Kerberos installed in your system,
Chirp will try that first.  If not, Chirp attempts the others.
<p>
Once an authentication scheme has succeeded, Chirp assigns
the incoming user a <b>subject</b> that describes both
the authentication method and the user name within that method.
For example, a user that authenticates via Kerberos might
have the subject:
<pre>
    kerberos:dthain@nd.edu
</pre>
A user authenticating with Globus credentials might be:<br>
(Note that Chirp substitutes underscores for spaces.)
<pre>
    globus:/O=Cooperative_Computing_Lab/CN=Douglas_L_Thain
</pre>
While another user authenticating by local unix ids might be:
<pre>
    unix:dthain
</pre>
While a user authenticating by simple hostnames might be:
<pre>
    hostname:pigwidgeon.cse.nd.edu
</pre>
Take note that Chirp considers all of the subjects as different
identities, although some of them might correspond to the same
person in varying circumstances.
<p>

<h3>Authorization</h3>

Once Chirp has authenticated your identity, you are logged
into a server.  However, when you attempt to read or manipulate
files on a server, Chirp checks to see whether you are
authorized to do so.  This is determined by <b>access control lists</b>
or <b>ACLs</b>.
<p>
Every directory in a Chirp server has an ACL, much like
filesystems such as as AFS or NTFS.  To see the ACL
for a directory, use the Chirp tool and the <tt>getacl</tt>
command:
<pre>
 chirp:host.mydomain.edu:/> getacl 
unix:dthain rwlda
hostname:*.mydomain.edu rwl
</pre>
Or, if you are using Parrot, you can use <tt>parrot_getacl</tt>
to examine ACLs in the same way:
<pre>
% parrot_run tcsh
% cd /chirp/host.mydomain.edu
% parrot_getacl
unix:dthain rwlda
hostname:*.mydomain.edu rwl
</pre>
This ACL indicates that the subject <tt>unix:dthain</tt>
has five access rights, while the subject pattern <tt>hostname:*.mydomain.edu</tt>
has only three access rights. The access rights are as follows:
<p>
<table>
<tr><td><b>r</b> <td> - The subject may read items in the directory.
<tr><td><b>w</b> <td> - The subject may write items in the directory.
<tr><td><b>l</b> <td> - The subject may list the directory contents.
<tr><td><b>d</b> <td> - The subject may delete items in the directory.
<tr><td><b>p</b> <td> - The subject may put new files into the directory. 
<tr><td><b>a</b> <td> - The subject may administer the directory, including changing the ACL.
<tr><td><b>x</b> <td> - The subject may execute programs in the directory.
<tr><td><b>v</b> <td> - The subject may reserve a directory.
</table>
<p>
Access rights often come in combinations, so there are a few aliases for your convenience:<br>
<p>
<table>
<tr><td><b>read</b>    <td> - alias for <b>rl</b><br>
<tr><td><b>write</b>   <td> - alias for <b>rwld</b><br>
<tr><td><b>admin</b>   <td> - alias for <b>rwlda</b><br>
<tr><td><b>none</b>    <td> - delete the entry<br>
</table>
To change an access control list on a directory, use the <tt>setacl</tt> command
in the Chirp command line tool:
<pre>
 chirp:host.mydomain.edu:/> setacl / kerberos:dthain@nd.edu write
 chirp:host.mydomain.edu:/> getacl 
unix:dthain rwlda
hostname:*.mydomain.edu rwl
kerberos:dthain@nd.edu rwld
</pre>
Note that for subject names that contain spaces, you should simply
substitute underscores.  For example, if your subject name is
<tt>/O=Univ of Somewhere/CN=Fred Flint</tt>, then you might issue
a <tt>setacl</tt> command like this:
<pre>
 chirp:host.mydomain.edu:/> setacl / /O=Univ_of_Somewhere/CN=Fred_Flint rwlda
</pre>
Or, you can accomplish the same thing using <tt>parrot_setacl</tt> inside of Parrot:
<pre>
% parrot_run tcsh
% cd /chirp/host.mydomain.edu
% parrot_setacl . /O=Univ_of_Somewhere/CN=Fred_Flint rwlda
</pre>
<p>
The meaning of ACLs is fairly obvious, but there are few subtleties you should know:
<p>
<b>Rights are generally inherited.</b>  When a new directory is created,
it automatically gets the ACL of its parent.  Exception: read about
the <b>reserve</b> right below.
<p>
<b>Rights are generally not hierarchical.</b>  In order to access a directory,
you only need the appropriate permissions on <i>that</i> directory.
For example, if you have permission to write to <tt>/data/x/y/z</tt>,
you do <b>not</b> need any other permissions on <tt>/data</tt>,
<tt>/data/x</tt> and so forth.  Of course, it may be difficult
to discover a deep directory without rights on the parents, but
you can still access it.
<p>
<b>The delete right is absolute.</b>  If you have permission to
delete a directory, then you are able to delete the <i>entire</i>
subtree that it contains, regardless of any other ACLs underneath.
<p>
Note that it is possible to use Chirp to export an existing directory
tree without manually populating every directory with ACLs.  Simply
create an ACL in an external file, and then use the -A option to
tell the Chirp server to use that file as the default ACL.

<h3>Reservation</h3>

The <b>v - reserve</b> right is an important concept that deserves
its own discussion.
<p>
A shared-storage environment such as Chirp aims to allow many
people to read and write common storage space.  Of course, with
many people reading and writing, we need some mechanism to make
sure that everybody does not step on each other's toes.
<p>
The <b>reserve</b> right allows a user to create what is
essentially a fresh workspace for their own use.  When
a user creates a new directory and has the <b>v</b> right
(but not the <b>w</b> right), Chirp will create a new
directory with a fresh ACL that gives the creating user restricted rights.
<p>
A good way to use the reserve right is with a wildcard at
the top directory.  Here's an example.  Suppose that Fred
creates a new Chirp server on the host <tt>bigwig</tt>. Initially, no-one except
Fred can access the server.  The first time it starts, the
Chirp server initializes its root directory with the following ACL:
<pre>
unix:fred rwla
</pre>
Now, Fred wants other users in his
organization to be able to use this storage, but doesn't
want them messing up his existing data.  So, Fred uses
the Chirp tool to give the list (<b>l</b>) and reserve (<b>v</b>)
rights to anyone calling from any machine in his organization:
<pre>
 chirp:bigwig:> setacl / hostname:*.somewhere.edu lv(rwlda)
 chirp:bigwig:> getacl /
unix:fred rwlda
hostname:*.somewhere.edu lv(rwlda)
</pre>
Now, any user calling from anywhere in <tt>somewhere.edu</tt>
can access this server.  But, all that any user can do is
issue <tt>ls</tt> or <tt>mkdir</tt> in the root directory.  For example,
suppose that Betty logs into this server from <tt>ws1.somewhere.edu</tt>.
She can not modify the root directory, but she can create
her own directory:
<pre>
 chirp:bigwig:> mkdir /mydata
</pre>
And, in the new directory, <tt>ws1.somewhere.edu</tt> can
do anything, including edit the access control.  Here is
the new ACL for <tt>/mydata</tt>:
<pre>
 chirp:bigwig:> getacl /mydata
hostname:ws1.somewhere.edu rwlda
</pre>
If Betty wants to authenticate with Globus credentials from
here on, she can change the access control as follows:
<pre>
 chirp:bigwig:> setacl /mydata globus:/O=Univ_of_Somewhere/CN=Betty rwla
</pre>
And, the new acl will look as follows:
<pre>
 chirp:bigwig:> getacl /mydata
hostname:ws1.somewhere.edu rwlda
globus:/O=Univ_of_Somewhere/CN=Betty rwla
</pre>

<h3>Simple Group Management</h3>

Chirp currently supports a simple group management system based on files.
Create a directory on your local filesystem in which to store
the groups.  Each file in the directory will have the name of the
desired groups, and contain a list of the members of the group in
plain text format.  Then, give your Chirp server the -G argument
to indicate the URL of the group directory.  Once the groups are
defined, you can refer to them in access control lists using the
<tt>group:</tt> prefix.

<p>
For example, suppose you wish to have two groups named <tt>group:students</tt> and <tt>group:faculty</tt>.  You could define the groups in the <tt>/data/groups</tt> directory as follows:
<p>
<pre>
/data/groups/students:
unix:astudent
unix:bstudent

/data/groups/faculty:
unix:aprof
unix:bprof
</pre>

Then, run the chirp server with the option <tt>-G file:///data/groups</tt>.  (Notice the URL syntax.)  Then, to make a directory <tt>/homework</tt> that is readable by students and writable by faculty, do this:

<pre>
chirp:bigwig> mkdir /homework
chirp:bigwig> setacl /homework group:students rl
chirp:bigwig> setacl /homework group:faculty  rwld
</pre>

If the groups are to be shared among many Chirp servers, place
the group directory on a web server and refer to it via an <tt>http</tt> URL.

<h3>Notes on Authentication</h3>

Each of the authentication types has a few things you should know:
<p>
<b>Kerberos:</b>  The server will attempt to use the Kerberos identity
of the host it is run on.  (i.e. host/coral.cs.wisc.edu@CS.WISC.EDU)
Thus, it must be run as the superuser in order to access its certificates.
Once authentication is complete, there is no need for the server
to keep its root access, so it will change to any unprivileged user
that you like.  Use the <tt>-i</tt> option to select the userid.
<p>
<b>Globus:</b> The server and client will attempt to perform
client authentication using the Grid Security Infrastructure (GSI)>
Both sides will load either user or host credentials, depending
on what is available.  If the server is running as an ordinary
user, then you must give a it a proxy certificate with grid-proxy-init.
Or, the server can be run as root and will use host certificates
in the usual place.
<p>
<b>Unix:</b>  This method makes use of a challenge-response
in the local Unix filesystem to determine the client's Unix identity.
It assumes that both machines share the same conception of the user database
and have a common directory which they can read and write.
By default, the server will pick a filename
in /tmp, and challenge the client to create that file.  If it can,
then the server will examine the owner of the file to determine the
client's username.  Naturally, /tmp will only be available to clients
on the same machine.  However, if a shared filesystem directory
is available, give that to the chirp server via the -c option.
Then, any authorized client of the filesystem can authenticate to
the server.  For example, at Notre Dame, we use <b>-c /afs/nd.edu/user37/ccl/software/rendezvous</b>
to authenticate via our AFS distributed file system.
<p>
<b>Hostname:</b> The server will rely on a reverse DNS lookup
to establish the fully-qualified hostname of the calling client.
The second field gives the hostname to be accepted.
It may contain an asterisk as a wildcard.
The third field is ignored.
The fourth field is then used to select an appropriate local username.
<p>
<b>Address:</b> Like "hostname" authentication, except the server
simply looks at the client's IP address.
<p>
By default, Chirp and/or Parrot will attempt every authentication type
knows until one succeeds.  If you wish to restrict or re-order
the authentication types used, give one or more
<b>-a</b> options to the client, naming the authentication types to be used,
in order. For example,
to attempt only hostname and kerberos authentication, in that order:
<pre>
   % chirp -a hostname -a kerberos
</pre>

<h2>Advanced Topic: Cluster Management</h2>

Several tools are available for managing a large cluster of Chirp servers.
<p>
First, a <a href=http://www.cse.nd.edu/~ccl/viz>Java visual display applet</a>
gives a graphical view of all servers in a cluster, as well as active network
connections between each client and server.  This tool can be used to quickly
view whether storage is free or used, whether CPUs are idle or busy, and whether
the network is idle or in use.  Clicking on individual nodes shows the same
detailed data as is avaliable in the <a href=http://catalog.cse.nd.edu:9097>catalog page</a>.
<p>
Next, it can be helpful to give a single 'superuser' limited access
to all servers and directories in a cluster, allowing them to fix broken
access controls and solve other problems.  To allow this, the <tt>-P user</tt>
argument can be given to a Chirp server, and will implicitly give the named
user the <b>L</b> and <b>A</b> rights on any directory on that server.
<p>
When managing a large system with many users, it is important to keep track
of what users are employing the cluster, and how much space they have consumed.
We refer to this as <b>auditing</b> the cluster.  To audit a single node,
use the <tt>audit</tt> command of the Chirp tool.  This produces a listing
of all users of a single host.  (You must have the <tt>A</tt> right in the root
directory of the server to run this command.) For example:
<pre>
% chirp ccl01.cse.nd.edu audit
   FILES     DIRS      DATA OWNER
   82842       27    5.0 GB globus:/O=UnivNowhere/CN=Fred
    6153      607  503.4 MB unix:fred
       2        2  200.3 MB hostname:laptop.nowhere.edu
      12        2    3.9 MB unix:betty
</pre>

To audit an entire cluster, run the <tt>chirp_audit_cluster</tt> tool.
This will extract the current list of hosts from your catalog,
run an audit on all hosts in parallel, and then produce several
reports in text files: <tt>audit.users.txt</tt>, <tt>audit.hosts.txt</tt>,
<tt>audit.users.hosts.txt</tt>, and <tt>audit.hosts.users.txt</tt>.
<p>
Often, users of a cluster will wish to replicate commonly used data
across all disks in the system, perhaps to provide fast access to relatively
static data.  The <tt>chirp_distribute</tt> tool can be used to rapidly
move data from one node to all others.  Given a source host and path,
<tt>chirp_distribute</tt> will create a spanning tree and then move data
directly from host to host in parallel.  This is much faster than running
<tt>cp</tt> or <tt>chirp put</tt> directly.  For example, this will copy
the <tt>/database</tt> directory from host <tt>server.nd.edu</tt> to
all hosts in your cluster:
<pre>
% chirp_distribute server.nd.edu /database `chirp_status -s`
</pre>
Another common problem is cleaning up data that has been copied this
way.  To delete, simply run <tt>chirp_distribute</tt> again with the
<tt>-X</tt> option and the same arguments.

<h2>Advanced Topic: Space Management</h2>

When multiple users share a common storage space,
there is the danger that one aggressive user can
accidentally (or deliberately) consume all available storage
and prevent other work from happening.
Chirp has two mechanisms available to deal with this problem.
<p>
The simpler tool is just a free space limit.
If run with the <tt>-F</tt> option, a Chirp server will stop
consuming space when the free space on the disk falls below
this limit.  External users will see a <it>"No space left on device."</tt> error.
For example, <tt>-F 100MB</tt> will leave a minimum of 100MB free on the local disk.
This mechanism imposes little or no performance penalty on the server.
<p>
The more complex tool is a user-level quota and allocation system.
If run with the <tt>-Q</tt> option, a Chirp server will establish
a software quota for all external users.  That is, <tt>-Q 2GB</tt>
will limit external users to consuming a total of 2 GB of storage
within a single Chirp server.
This mechanism imposes some run-time performance penalty,
and also delays server startup somewhere: the Chirp server must
traverse its storage directory to count up the available space.
<p>
With the <tt>-Q</tt> option enabled, external users can <it>allocate</tt>
space before consuming it.  Using the Chirp tools, users may use the <tt>mkalloc</tt>
command to create new directories with an attached space allocation.
For example, <tt>mkalloc /mydata 1GB</tt> will create a new directory <tt>/mydata</tt>
with an allocation of 1GB.  This allocation is a <i>limit</i> that prevents
files in that directory from consuming more than 1GB; it is also a <i>guarantee</i>
that other users of the server will not be able to steal the space.
Such allocations may also be subdivided by using <tt>mkalloc</tt> to create
sub-directories.
<p>
Note: Users employing Parrot can also use the <tt>parrot_mkalloc</tt>
and <tt>parrot_lsalloc</tt> commands in ordinary scripts to achieve
the same effect.
<p>
To examine an allocation, use the <tt>lsalloc</tt> command.
<p>
To destroy an allocation, simply delete the corresponding directory.

<h2>Advanced Topic: Ticket Authentication</h2>

<p>
  Often a user will want to access a Chirp server storing files for cluster
  computing jobs but will have difficulty accessing it securely without
  transferring their credentials with the jobs dispatched to the cluster. To
  facilitate ease-of-use, users typically solve this by giving rights to a
  hostname mask (e.g. <em>*.cse.nd.edu</em>) on the Chirp server.  However,
  this level of access can be innappropriate due to sensitive data.  Instead,
  these users are forced to use difficult authentication methods such as Globus
  or Kerberos for running the Chirp server. They may also use a virtual network
  solution but users typically lack this amount of control on clusters.  To
  provide an easy solution to this problem, Chirp offers its own ticket based
  authentication system which is convenient and simple to setup.
</p>

<p>
  To start, users may create a ticket for authentication using:
</p>

<code>chirp &lt;host:port&gt; ticket_create -output myticket.ticket -subject unix:user -bits 1024 -duration 86400 / rl /foo rwl</code>

<p>
  This command performs multiple tasks in three stages:
</p>

<p>
  First, it creates a ticket which is composed of an RSA Private Key with a key
  (modulus) size of 1024 bits. When we refer to the ticket, we are speaking of
  this Private Key. By default, the ticket file generated is named
  <strong>ticket.MD5SUM</strong> where <strong>MD5SUM</strong> is the MD5
  digest of the Public Key of the ticket.
</p>
  
<p>
  Once the ticket is created, it is registered with the Chirp server with a
  validity period in seconds defined by the duration option (86400, or a day).
  The <code>-subject unix:user</code> switch allows the user to set the ticket
  for another user; however, only the <strong>chirp_server</strong> superuser
  (-P) may set tickets for any subject. For regular users, the -subject option
  is unnecessary as it is by default the subject you possess when registering
  the ticket. Users who authenticate using this ticket in the future will
  become this subject with certain masked rights.
</p>
  
<p>
  Once the ticket is created and registered, we give the ticket a set of
  <em>ACL masks</em>. The ACL mask will mask the rights of the
  ticket-authenticated user with the rights of the subject that registered the
  ticket. For example, if a user named <em>foo</em> (subject is
  <code>unix:foo</code>) has rights <strong>rwl</strong> in the root
  directory of the Chirp server and if a ticket is registered for <em>foo</em>
  with the ACL mask <code>/ rlx</code>, the effective rights of the
  ticket-authenticated user is <strong>rl</strong> in the root directory.
</p>

<p>
  ACL masks are also inherited from parent directories. So, in the above
  example, the root directory has the ACL mask <strong>rl</strong> while the
  foo directory has the ACL mask <strong>rwl</strong>. Other nested directories
  within the root directory also inherit the <strong>rl</strong> mask.
  Similarly, nested directories of the foo directory inherit the
  <strong>rwl</strong> mask. We emphasize that the ACL mask does not give
  rights but limits them. If the user that registers a ticket has no rights in
  a directory, then neither will the ticket authenticated user.
</p>

<h3>Authenticating with a ticket</h3>

<p>
  To authenticate using a ticket, it can be as simple as including the ticket
  file with your job. Tickets that follow the <strong>ticket.MD5SUM</strong>
  template are automatically added to the list of tickets to try when
  authenticating. You can also give specific tickets to authenticate with
  using a comma-delimited list of ticket filenames in either the
  <strong>CHIRP_CLIENT_TICKETS</strong> environment variable or via the
  <strong>-i <tickets></strong> option. Tickets are tried in the order
  they are specified.
</p>

<code>chirp &lt;host:port&gt;</code>

<p>
  The above command will try ticket authentication as a last resort but will
  use tickets it finds in the current directory following the template.
</p>

<code>chirp -a ticket -i file.ticket &lt;host:port&gt;</code>

<p>
  The above command forces ticket authentication and only uses the
  <strong>file.ticket</strong> ticket to authenticate.
<p>

<p>
  Authenticating is this simple. It is important to note that tickets are
  obviously not protected in any way from theft when you distribute the ticket
  with jobs in a distributed computing environment (no ticket system can give
  this guarantee).  Users may want to protect their tickets in basic ways
  by setting a restrictive file mode and by giving tickets a limited duration
  on the server.
</p>

<p>
  Finally, users should be careful to experiment with small key sizes for a
  balance of quick authentication and security. Smaller key sizes may be
  rejected outright by openssl when given a 64 byte challenge to sign. Chirp
  will not authenticate or use smaller challenge sizes if openssl rejects the
  ticket.
</p>

<h3>Manually Registering a Ticket</h3>

<p>
  A ticket is only useful when registered with a server. The ticket_create
  command does this for you automatically but you may also wish to register the
  ticket with multiple servers. To do this, you can manually register a ticket
  that is already created by using the <code>ticket_register</code> command:
</p>

<code>chirp &lt;host:port&gt; ticket_register myticket.ticket unix:user 86400</code>

<p>
  The first argument to <code>ticket_register</code> is the name of the ticket,
  followed by the subject, and finally the ticket duration.  The second option
  (the subject) is optional. As described earlier, specifying the subject
  allows you to register a ticket with a user other than yourself. This is only
  possible if you are authenticated with the server as the super user.
</p>

<h3>Modifying the Rights of a Ticket</h3>

<p>
  You may use the <code>ticket_modify</code> command to change the rights
  a ticket has in a directory. You are restricted to giving rights to a 
  ticket you already possess. Recall, however, that the rights are actually
  a mask that are logically ANDed with the rights the user has at the time.
<p>

<code>chirp &lt;host:port&gt; ticket_modify myticket.ticket / rl</code>

<p>
  The above command changes the ACL mask of <code>myticket.ticket</code>
  to <code>rl</code> in the root directory.
</p>

<p>
  A ticket identifier as returned by <code>ticket_list</code> may also
  be used instead of a ticket filename.
</p>

<h3>Deleting a Ticket</h3>

<p>
  Deleting a ticket unregisters the ticket with the server. Additionally,
  the ticket on the client is deleted.
</p>

<code>chirp &lt;host:port&gt; ticket_delete myticket.ticket</code>

<p>
  A ticket identifier as returned by <code>ticket_list</code> may also
  be used instead of a ticket filename.
</p>

<h3>Listing the Registered Tickets on the Server</h3>

<p>
  To list the tickets registered on a server, use the <code>ticket_list</code>
  command:
</p>

<code>chirp &lt;host:port&gt; ticket_list unix:user</code>

<p>
  The subject argument instructs the command to fetch all the tickets belonging
  to the user. You may also use <code>ticket_list all</code> to list all the
  tickets of all users on the server. The latter command is only executable by
  the Chirp super user. The output is a list of tickets identifiers. You can
  query information about a ticket using these identifiers with the
  <code>ticket_get</code> command.
</p>

<h3>Getting a Registered Ticket's Information from the Server</h3>

<p>
  To check the status of a ticket on a server, you may use the
  <code>ticket_get</code> command:
</p>

<code>chirp &lt;host:port&gt; ticket_get myticket.ticket</code>

<p>
  So long as you own the ticket or are authenticated as the super user, the
  server will return to you information associated with the ticket.  The ticket
  must also exist and must also not have expired.  <code>ticket_get</code>
  takes a client side ticket filename as an argument or a ticket identifier as
  returned by the <code>ticket_list</code> command.
</p>

<p>
  <code>ticket_get</code> prints the subject that owns the ticket,
  the base64 encoded public key of the ticket, the time left
  until the ticket expires in seconds, and a variable number of
  directory and ACL masks. For example, we might have the following
  output:
</p>

<pre>
$ chirp host:port ticket_get myticket.ticket
unix:pdonnel3
LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlHZk1BMEdDU3FHU0liM0RRRUJBUVVBQTRHTkFEQ0Jp
UUtCZ1FEZVoyZWxKYXdlcHBHK0J4SFlaMmlmWFIzNAovU3RhUElta0lmeit4TDZxN21wS2lnMDJQZ2Z5
emdKRWFjMk50NzJrUlBpOEJWYWdkOHdvSGhWc25YZ1YvNjFPCjVkaG13STNLYWRlYjNUbkZXUUo3bFhh
anhmVTZZR1hXb2VNY1BsdjVQUWloWm8yWmFXTUUvQVA4WUtnVVphdXcKelI2RkdZWGd6N2RGZzR6Yk9R
SURBUUFCCi0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo=
5993
/ rl
/foo rwl
</pre>

<p>
  Note that the base64 encoded public key above is wrapped to fit an 80
  character width for this manual. In the actual output, the public key is on
  one line. All of the information is new-line-delimited.
</p>


<h2>Advanced Topic: HDFS Backend Storage for Chirp</h2>

<p>
The Chirp server is able to bind to backend filesystems besides the local
filesystem. In particular, it is able to act as a frontend for the Hadoop HDFS
filesystem. When used on top of Hadoop, Chirp gives you the benefit of a robust
system of ACLs, simple userspace access and POSIX semantics (with some
limitations, discussed below). Perhaps best of all, client jobs will no
longer have any Hadoop or Java (version) dependencies.

<p>
To run a Chirp server as a frontend to Hadoop use the
<tt>chirp_server_hdfs</tt> command, which will set up a number of needed
environment variables and then run <tt>chirp_server</tt> with the usual arguments.
Give the location of the root storage directory in HDFS with <tt>-r</tt> and a URL like so:
<pre>
% chirp_server_hdfs -r hdfs://headnode.hadoop.domain.edu/mydata
</pre>
<p>
By default, chirp will use whatever default replication factor is
defined by HDFS (typically 3).  To change the replication factor of
a single file, use the <tt>chirp setrep</tt> or <tt>parrot_setrep</tt> commands.  A path of <tt>&amp;&amp;&amp;</tt> will set the replication factor for all new files created in that session.

<h3>Temporary Local Storage</h3>
<p>
Chirp allows you to setup a location to place temporary files such as those
for caching groups, and other items.
You can set this using the <tt>-y path</tt>.
This allows for faster access, POSIX semantics, and less load on HDFS.
By default, Chirp assumes the current directory for temporary storage.

<h3>Limitations</h3>
<p>
Chirp tries to preserve POSIX filesystem semantics where possible despite HDFS
violating certain assumptions. For example, random writes are not possible for
Chirp on HDFS. When the user requests to open a file for writing, Chirp assumes
an implicit <tt>O_APPEND</tt> flag was added. In addition, HDFS does not
maintain an execute permission bit for regular files. Chirp assumes all files
have the execute bit set.
<p>
Chirp also does not allow using the thirdput command or user space management
(<tt>-F</tt>) when using HDFS as a backend.

<h2>Debugging Advice</h2>

Debugging a distributed system can be quite difficult because of the
sheer number of hosts involved and the mass of information to be
collected.  If you are having difficulty with Chirp, we recommend
that you make good use of the debugging traces built into the tools.
<p>
In all of the Chirp and Parrot tools, the <tt>-d</tt> option allows
you to turn on selected debugging messages.  The simplest option
is <tt>-d all</tt> which will show every event that occurs in
the system.
<p>
To best debug a problem, we recommend that you turn on the debugging
options on <b>both</b> the client and server that you are operating.
For example, if you are having trouble getting Parrot to connect
to a Chirp server, then run both as follows:
<pre>
% chirp_server -d all [more options] ...
% parrot_run -d all tcsh
</pre>
Of course, this is likely to show way more information than you
will be able to process.  Instead, turn on a debugging flags
selectively.  For example, if you are having a problem with authentication,
just show those messages with <tt>-d auth</tt> on both sides.
<p>
There are a large number of debugging flags.  Currently, the
choices are: syscall notice channel process resolve libcall
tcp dns auth local http ftp nest chirp dcap rfio cache
poll remote summary debug time pid all.  When debugging problems
with Chirp and Parrot, we recommend selectively using
<tt>-d chirp</tt>, <tt>-d tcp</tt>, <tt>-d auth</tt>, and
<tt>-d libcall</tt> as needed.

</body>
</html>
coop-computing-tools-doc 4.0-1.1 / usr / share / doc / coop-computing-tools / manual / chirp.html
