/usr/share/doc/condor/html/4_4

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<!--Converted with LaTeX2HTML 2008 (1.71)
original version by:  Nikos Drakos, CBLU, University of Leeds
* revised and updated by:  Marcus Hennecke, Ross Moore, Herb Swan
* with significant contributions from:
  Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
<HTML>
<HEAD>
<TITLE>4.4 Hooks</TITLE>
<META NAME="description" CONTENT="4.4 Hooks">
<META NAME="keywords" CONTENT="ref">
<META NAME="resource-type" CONTENT="document">
<META NAME="distribution" CONTENT="global">

<META NAME="Generator" CONTENT="LaTeX2HTML v2008">
<META HTTP-EQUIV="Content-Style-Type" CONTENT="text/css">

<LINK REL="STYLESHEET" HREF="ref.css">

<LINK REL="next" HREF="4_5_Application_Program.html">
<LINK REL="previous" HREF="4_3_Computing_On.html">
<LINK REL="up" HREF="4_Miscellaneous_Concepts.html">
<LINK REL="next" HREF="4_5_Application_Program.html">
</HEAD>

<BODY  BGCOLOR=#FFFFFF >

<DIV CLASS="navigation"><!--Navigation Panel-->
<A NAME="tex2html1831"
  HREF="4_5_Application_Program.html">
<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="next.png"></A> 
<A NAME="tex2html1825"
  HREF="4_Miscellaneous_Concepts.html">
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="up.png"></A> 
<A NAME="tex2html1819"
  HREF="4_3_Computing_On.html">
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="prev.png"></A> 
<A NAME="tex2html1827"
  HREF="Contents.html">
<IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="contents.png"></A> 
<A NAME="tex2html1829"
  HREF="Index.html">
<IMG WIDTH="43" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="index" SRC="index.png"></A> 
<BR>
<B> Next:</B> <A NAME="tex2html1832"
  HREF="4_5_Application_Program.html">4.5 Application Program Interfaces</A>
<B> Up:</B> <A NAME="tex2html1826"
  HREF="4_Miscellaneous_Concepts.html">4. Miscellaneous Concepts</A>
<B> Previous:</B> <A NAME="tex2html1820"
  HREF="4_3_Computing_On.html">4.3 Computing On Demand</A>
 &nbsp; <B>  <A NAME="tex2html1828"
  HREF="Contents.html">Contents</A></B> 
 &nbsp; <B>  <A NAME="tex2html1830"
  HREF="Index.html">Index</A></B> 
<BR>
<BR></DIV>
<!--End of Navigation Panel-->
<!--Table of Child-Links-->
<A NAME="CHILD_LINKS"><STRONG>Subsections</STRONG></A>

<UL CLASS="ChildLinks">
<LI><A NAME="tex2html1833"
  HREF="4_4_Hooks.html#SECTION00541000000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN> Job Hooks That Fetch Work</A>
<UL>
<LI><A NAME="tex2html1834"
  HREF="4_4_Hooks.html#SECTION00541100000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">1</SPAN> Work Fetching Hooks Invoked by Condor</A>
<LI><A NAME="tex2html1835"
  HREF="4_4_Hooks.html#SECTION00541200000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">2</SPAN> Keywords to Define Job Fetch Hooks in the Condor Configuration files </A>
<LI><A NAME="tex2html1836"
  HREF="4_4_Hooks.html#SECTION00541300000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">3</SPAN> Defining the FetchWorkDelay Expression</A>
<LI><A NAME="tex2html1837"
  HREF="4_4_Hooks.html#SECTION00541400000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">4</SPAN> Example Hook: Specifying the Executable at Execution Time</A>
</UL>
<BR>
<LI><A NAME="tex2html1838"
  HREF="4_4_Hooks.html#SECTION00542000000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">2</SPAN> Hooks for a Job Router</A>
<UL>
<LI><A NAME="tex2html1839"
  HREF="4_4_Hooks.html#SECTION00542100000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">2</SPAN>.<SPAN CLASS="arabic">1</SPAN> Hooks Invoked for Job Routing</A>
</UL>
<BR>
<LI><A NAME="tex2html1840"
  HREF="4_4_Hooks.html#SECTION00543000000000000000"><SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">3</SPAN> Daemon ClassAd Hooks</A>
</UL>
<!--End of Table of Child-Links-->
<HR>

<H1><A NAME="SECTION00540000000000000000"></A><A NAME="sec:hooks"></A>
<A NAME="40801"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN> Hooks
</H1>
A <SPAN  CLASS="textit">hook</SPAN> is an external program or script invoked by Condor.

<P>
Job hooks that fetch work allow sites to write their own programs or scripts, 
and allow Condor to invoke these hooks at the right moments 
to accomplish the desired outcome.
This eliminates the expense of the matchmaking and scheduling provided 
by the <SPAN  CLASS="textit">condor_schedd</SPAN> and the <SPAN  CLASS="textit">condor_negotiator</SPAN>,
although at the price of the flexibility they offer. 
Therefore, job hooks that fetch work allow Condor to more easily and directly
interface with external scheduling systems. 

<P>
Hooks may also behave as a Job Router.

<P>
The Daemon ClassAd hooks permit the <SPAN  CLASS="textit">condor_startd</SPAN> and 
the <SPAN  CLASS="textit">condor_schedd</SPAN> daemons to execute hooks once or on a periodic basis.

<P>
Note that standard universe jobs execute different <SPAN  CLASS="textit">condor_starter</SPAN> and
<SPAN  CLASS="textit">condor_shadow</SPAN> daemons that do not implement any hook mechanisms.

<P>

<H2><A NAME="SECTION00541000000000000000"></A><A NAME="sec:job-hooks"></A>
<A NAME="40834"></A>
<A NAME="40835"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN> Job Hooks That Fetch Work
</H2>

<P>
In the past, Condor has always sent work to the execute machines by
pushing jobs to the <SPAN  CLASS="textit">condor_startd</SPAN> daemon, either from the <SPAN  CLASS="textit">condor_schedd</SPAN>
daemon or via <SPAN  CLASS="textit">condor_cod</SPAN>.
Beginning with the Condor version 7.1.0, the <SPAN  CLASS="textit">condor_startd</SPAN> daemon now has the
ability to pull work by fetching jobs via a system of plug-ins or
hooks.
Any site can configure a set of hooks to fetch work, completely
outside of the usual Condor matchmaking system.

<P>
A projected use of the hook mechanism implements what might
be termed a <SPAN  CLASS="textit">glide-in factory</SPAN>, especially where the
factory is behind a firewall.
Without using the hook mechanism to fetch work,
a glide-in <SPAN  CLASS="textit">condor_startd</SPAN> daemon behind a firewall
depends on CCB to help it listen and eventually receive
work pushed from elsewhere.
With the hook mechanism, a glide-in <SPAN  CLASS="textit">condor_startd</SPAN> daemon
behind a firewall uses the hook to pull work.
The hook needs only an outbound network connection to complete
its task,
thereby being able to operate from behind the firewall,
without the intervention of CCB.

<P>
Periodically, each execution slot managed by a <SPAN  CLASS="textit">condor_startd</SPAN> will
invoke a hook to see if there is any work that can be fetched.
Whenever this hook returns a valid job, the <SPAN  CLASS="textit">condor_startd</SPAN> will
evaluate the current state of the slot and decide if it should start
executing the fetched work.
If the slot is unclaimed and the <TT>Start</TT> expression evaluates to
<TT>True</TT>, a new claim will be created for the fetched job.
If the slot is claimed, the <SPAN  CLASS="textit">condor_startd</SPAN> will evaluate the
<TT>Rank</TT> expression relative to the fetched job, compare it to
the value of <TT>Rank</TT> for the currently running job, and decide
if the existing job should be preempted due to the fetched job having
a higher rank.
If the slot is unavailable for whatever reason, the <SPAN  CLASS="textit">condor_startd</SPAN>
will refuse the fetched job and ignore it.
Either way, once the <SPAN  CLASS="textit">condor_startd</SPAN> decides what it should do with
the fetched job, it will invoke another hook to reply to the attempt
to fetch work, so that the external system knows what happened to that
work unit.

<P>
If the job is accepted, a claim is created for it and the slot moves
into the Claimed state.
As soon as this happens, the <SPAN  CLASS="textit">condor_startd</SPAN> will spawn a
<SPAN  CLASS="textit">condor_starter</SPAN> to manage the execution of the job.
At this point, from the perspective of the <SPAN  CLASS="textit">condor_startd</SPAN>, this claim
is just like any other.
The usual policy expressions are evaluated, and if the job needs to be
suspended or evicted, it will be.
If a higher-ranked job being managed by a <SPAN  CLASS="textit">condor_schedd</SPAN> is matched
with the slot, that job will preempt the fetched work.

<P>
The <SPAN  CLASS="textit">condor_starter</SPAN> itself can optionally invoke additional hooks to
help manage the execution of the specific job.
There are hooks to prepare the execution environment for the job,
periodically update information about the job as it runs, notify when
the job exits, and to take special actions when the job is being evicted.

<P>
Assuming there are no interruptions, the job completes, and the
<SPAN  CLASS="textit">condor_starter</SPAN> exits, the <SPAN  CLASS="textit">condor_startd</SPAN> will invoke the hook to
fetch work again.
If another job is available, the existing claim will be reused and a
new <SPAN  CLASS="textit">condor_starter</SPAN> is spawned.
If the hook returns that there is no more work to perform, the claim
will be evicted, and the slot will return to the Owner state.

<P>

<H3><A NAME="SECTION00541100000000000000"></A><A NAME="sec:job-hooks-hooks"></A>
<A NAME="40861"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">1</SPAN> Work Fetching Hooks Invoked by Condor
</H3> 

<P>
There are a handful of hooks invoked by Condor related to fetching
work, some of which are called by the <SPAN  CLASS="textit">condor_startd</SPAN> and others by
the <SPAN  CLASS="textit">condor_starter</SPAN>.
Each hook is described, including when it is invoked, what
task it is supposed to accomplish, what data is passed to the hook,
what output is expected, and, when relevant, the exit status expected.

<P>
<DL COMPACT>
<DD><A NAME="40865"></A>
</DD>
<DT>Hook: Fetch Work</DT>
<DD><P>
The hook defined by the configuration variable
<TT>&lt;Keyword&gt;_HOOK_FETCH_WORK</TT> <A NAME="41118"></A> <A NAME="41119"></A> is invoked whenever the <SPAN  CLASS="textit">condor_startd</SPAN>
wants to see if there is any work to fetch.
There is a related configuration variable called
<TT>FetchWorkDelay</TT> <A NAME="41125"></A> <A NAME="41126"></A> which determines how long the <SPAN  CLASS="textit">condor_startd</SPAN>
will wait between attempts to fetch work, which is described in detail
in within section&nbsp;<A HREF="#sec:job-hooks-fetch-work-delay">4.4.1</A> on
page&nbsp;<A HREF="4_4_Hooks.html#sec:job-hooks-fetch-work-delay"><IMG  ALIGN="BOTTOM" BORDER="1" ALT="[*]" SRC="crossref.png"></A>.
<TT>&lt;Keyword&gt;_HOOK_FETCH_WORK</TT> is the most important hook in the whole
system, and is the only hook that must be defined for any of the other
<SPAN  CLASS="textit">condor_startd</SPAN> hooks to operate.

<P>
The job ClassAd returned by the hook needs to contain enough
information for the <SPAN  CLASS="textit">condor_starter</SPAN> to eventually spawn the work.
The required and optional attributes in this ClassAd are identical to
the ones described for Computing on Demand (COD) jobs in
section&nbsp;<A HREF="4_3_Computing_On.html#sec:cod-application-attributes">4.3.3</A> 
on COD Application Attributes, 
page&nbsp;<A HREF="4_3_Computing_On.html#sec:cod-application-attributes"><IMG  ALIGN="BOTTOM" BORDER="1" ALT="[*]" SRC="crossref.png"></A>.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>ClassAd of the slot that is looking for work.

<P>
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>ClassAd of a job that can be run.
  If there is no work, the hook should return no output.

<P>
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>Ignored.
</DD>
</DL>

<P>
<A NAME="40879"></A>
</DD>
<DT>Hook: Reply Fetch</DT>
<DD><P>
The hook defined by the configuration variable
<TT>&lt;Keyword&gt;_HOOK_REPLY_FETCH</TT> <A NAME="41137"></A> <A NAME="41138"></A> is invoked whenever
<TT>&lt;Keyword&gt;_HOOK_FETCH_WORK</TT> <A NAME="41142"></A> <A NAME="41143"></A> returns data and the <SPAN  CLASS="textit">condor_startd</SPAN>
decides if it is going to accept the fetched job or not.

<P>
The <SPAN  CLASS="textit">condor_startd</SPAN> will not wait for this hook to return before
taking other actions, and it ignores all output.
The hook is simply advisory, and it has no impact on the behavior of the
<SPAN  CLASS="textit">condor_startd</SPAN>.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>Either the string <code>accept</code> or <code>reject</code>.

<P>
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>A copy of the job ClassAd and the slot ClassAd
  (separated by the string <code>-----</code> and a new line).

<P>
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>Ignored.
</DD>
</DL>

<P>
<A NAME="40887"></A>
</DD>
<DT>Hook: Evict Claim</DT>
<DD><P>
The hook defined by the configuration variable
<TT>&lt;Keyword&gt;_HOOK_EVICT_CLAIM</TT> <A NAME="41153"></A> <A NAME="41154"></A> is invoked whenever the <SPAN  CLASS="textit">condor_startd</SPAN>
needs to evict a claim representing fetched work.

<P>
The <SPAN  CLASS="textit">condor_startd</SPAN> will not wait for this hook to return before
taking other actions, and ignores all output.
The hook is simply advisory, and has no impact on the behavior of the
<SPAN  CLASS="textit">condor_startd</SPAN>.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>A copy of the job ClassAd and the slot ClassAd
  (separated by the string <code>-----</code> and a new line).

<P>
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>Ignored.
</DD>
</DL>

<P>
<A NAME="40894"></A>
</DD>
<DT>Hook: Prepare Job</DT>
<DD><P>
The hook defined by the configuration variable
<TT>&lt;Keyword&gt;_HOOK_PREPARE_JOB</TT> <A NAME="41164"></A> <A NAME="41165"></A> is invoked by the <SPAN  CLASS="textit">condor_starter</SPAN> before
a job is going to be run.
This hook provides a chance to execute commands to set up the job
environment, for example, to transfer input files.

<P>
The <SPAN  CLASS="textit">condor_starter</SPAN> waits until this hook returns before
attempting to execute the job.
If the hook returns a non-zero exit status, the <SPAN  CLASS="textit">condor_starter</SPAN> will
assume an error was reached while attempting to set up the job
environment and abort the job.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>A copy of the job ClassAd.

<P>
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>A set of attributes to insert or update into the job ad.  For example,
  changing the <TT>Cmd</TT> attribute to a quoted string changes the executable 
  to be run.

<P>
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>0 for success preparing the job, any non-zero value on failure.
</DD>
</DL>

<P>
<A NAME="40902"></A>
</DD>
<DT>Hook:  Update Job Info</DT>
<DD><P>
The hook defined by the configuration variable
<TT>&lt;Keyword&gt;_HOOK_UPDATE_JOB_INFO</TT> <A NAME="41176"></A> <A NAME="41177"></A> is invoked periodically during the
life of the job to update information about the status of the job.
When the job is first spawned, the <SPAN  CLASS="textit">condor_starter</SPAN> will invoke this
hook after <TT>STARTER_INITIAL_UPDATE_INTERVAL</TT> <A NAME="41183"></A> <A NAME="41184"></A> seconds
(defaults to 8).
Thereafter, the <SPAN  CLASS="textit">condor_starter</SPAN> will invoke the hook every 
<TT>STARTER_UPDATE_INTERVAL</TT> <A NAME="41190"></A> <A NAME="41191"></A> seconds (defaults to 300,
which is 5 minutes).

<P>
The <SPAN  CLASS="textit">condor_starter</SPAN> will not wait for this hook to return before
taking other actions, and ignores all output.
The hook is simply advisory, and has no impact on the behavior of the
<SPAN  CLASS="textit">condor_starter</SPAN>.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>A copy of the job ClassAd that has been augmented with additional
  attributes describing the current status and execution behavior of
  the job.

<P>
The additional attributes included inside the job ClassAd are:
<DL>
<DT><STRONG><TT>JobState</TT></STRONG></DT>
<DD>The current state of the job.
  Can be either <TT>"Running"</TT> or <TT>"Suspended"</TT>.

<P>
</DD>
<DT><STRONG><TT>JobPid</TT></STRONG></DT>
<DD>The process identifier for the initial job directly spawned by the
  <SPAN  CLASS="textit">condor_starter</SPAN>.

<P>
</DD>
<DT><STRONG><TT>NumPids</TT></STRONG></DT>
<DD>The number of processes that the job has currently spawned.

<P>
</DD>
<DT><STRONG><TT>JobStartDate</TT></STRONG></DT>
<DD>The epoch time when the job was first spawned by the <SPAN  CLASS="textit">condor_starter</SPAN>.

<P>
</DD>
<DT><STRONG><TT>RemoteSysCpu</TT></STRONG></DT>
<DD>The total number of seconds of system CPU time (the time spent at
  system calls) the job has used.

<P>
</DD>
<DT><STRONG><TT>RemoteUserCpu</TT></STRONG></DT>
<DD>The total number of seconds of user CPU time the job has used.

<P>
</DD>
<DT><STRONG><TT>ImageSize</TT></STRONG></DT>
<DD>The memory image size of the job in Kbytes.
</DD>
</DL>

<P>
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>Ignored.
</DD>
</DL>

<P>
<A NAME="40925"></A>
</DD>
<DT>Hook:  Job Exit</DT>
<DD><P>
The hook defined by the configuration variable
<TT>&lt;Keyword&gt;_HOOK_JOB_EXIT</TT> <A NAME="41212"></A> <A NAME="41213"></A> is invoked by the <SPAN  CLASS="textit">condor_starter</SPAN>
whenever a job exits, either on its
own or when being evicted from an execution slot.

<P>
The <SPAN  CLASS="textit">condor_starter</SPAN> will wait for this hook to return before
taking any other actions.
In the case of jobs that are being managed by a <SPAN  CLASS="textit">condor_shadow</SPAN>, this
hook is invoked before the <SPAN  CLASS="textit">condor_starter</SPAN> does its own optional file
transfer back to the submission machine, writes to the local user log
file, or notifies the <SPAN  CLASS="textit">condor_shadow</SPAN> that the job has exited.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>A string describing how the job exited:
  
<UL>
<LI><code>exit</code> The job exited or died with a signal on its own.
</LI>
<LI><code>remove</code> The job was removed with <SPAN  CLASS="textit">condor_rm</SPAN> or as the result of
    user job policy expressions (for example, <TT>PeriodicRemove</TT>).
</LI>
<LI><code>hold</code> The job was held with <SPAN  CLASS="textit">condor_hold</SPAN> or the
    user job policy expressions (for example, <TT>PeriodicHold</TT>).
</LI>
<LI><code>evict</code> The job was evicted from the execution slot for
    any other reason (<TT>PREEMPT</TT> <A NAME="41233"></A> <A NAME="41234"></A> evaluated to TRUE in the
    <SPAN  CLASS="textit">condor_startd</SPAN>, <SPAN  CLASS="textit">condor_vacate</SPAN>, <SPAN  CLASS="textit">condor_off</SPAN>, etc).
  
</LI>
</UL>

<P>
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>A copy of the job ClassAd that has been augmented with additional
  attributes describing the execution behavior of the job and its
  final results.

<P>
The job ClassAd passed to this hook contains all of the extra
attributes described above for <TT>&lt;Keyword&gt;_HOOK_UPDATE_JOB_INFO</TT> <A NAME="41244"></A> <A NAME="41245"></A>, and
the following additional attributes that are only present once a job
exits:
<DL>
<DT><STRONG><TT>ExitReason</TT></STRONG></DT>
<DD>A human-readable string describing why the job exited.

<P>
</DD>
<DT><STRONG><TT>ExitBySignal</TT></STRONG></DT>
<DD>A boolean indicating if the job exited due to being killed by a
  signal, or if it exited with an exit status.

<P>
</DD>
<DT><STRONG><TT>ExitSignal</TT></STRONG></DT>
<DD>If <TT>ExitBySignal</TT> is true, the signal number that killed the job.

<P>
</DD>
<DT><STRONG><TT>ExitCode</TT></STRONG></DT>
<DD>If <TT>ExitBySignal</TT> is false, the integer exit code of the job.

<P>
</DD>
<DT><STRONG><TT>JobDuration</TT></STRONG></DT>
<DD>The number of seconds that the job ran during this invocation.
</DD>
</DL>

<P>
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>None.

<P>
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>Ignored.
</DD>
</DL>

<P>
</DD>
</DL>

<P>

<H3><A NAME="SECTION00541200000000000000"></A><A NAME="sec:job-hooks-keywords"></A>
<A NAME="40956"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">2</SPAN> Keywords to Define Job Fetch Hooks in the Condor Configuration files 
</H3> 

<P>
Hooks are defined in the Condor configuration files by prefixing
the name of the hook with a keyword.
This way, a given machine can have multiple sets of hooks, each set
identified by a specific keyword.

<P>
Each slot on the machine can define a separate keyword for the set
of hooks that should be used with <TT>SLOT&lt;N&gt;_JOB_HOOK_KEYWORD</TT> <A NAME="41256"></A> <A NAME="41257"></A>.
For example, on slot 1, the variable name will be
called <TT>SLOT1_JOB_HOOK_KEYWORD</TT>.
If the slot-specific keyword is not defined, the <SPAN  CLASS="textit">condor_startd</SPAN> will
use a global keyword as defined by <TT>STARTD_JOB_HOOK_KEYWORD</TT> <A NAME="41264"></A> <A NAME="41265"></A>.

<P>
Once a job is fetched via <TT>&lt;Keyword&gt;_HOOK_FETCH_WORK</TT> <A NAME="41269"></A> <A NAME="41270"></A>, the
<SPAN  CLASS="textit">condor_startd</SPAN> will insert the keyword used to fetch that job into
the job ClassAd as <TT>HookKeyword</TT>.
This way, the same keyword will be used to select the hooks invoked by
the <SPAN  CLASS="textit">condor_starter</SPAN> during the actual execution of the job.
However, the <TT>STARTER_JOB_HOOK_KEYWORD</TT> <A NAME="41279"></A> <A NAME="41280"></A> can be defined to
force the <SPAN  CLASS="textit">condor_starter</SPAN> to always use a given keyword for its own
hooks, instead of looking the job ClassAd for a <TT>HookKeyword</TT>
attribute.

<P>
For example, the following configuration defines two sets of hooks,
and on a machine with 4 slots, 3 of the slots use the global keyword
for running work from a database-driven system, and one of the slots
uses a custom keyword to handle work fetched from a web service.
<PRE>
  # Most slots fetch and run work from the database system.
  STARTD_JOB_HOOK_KEYWORD = DATABASE

  # Slot4 fetches and runs work from a web service.
  SLOT4_JOB_HOOK_KEYWORD = WEB

  # The database system needs to both provide work and know the reply
  # for each attempted claim.
  DATABASE_HOOK_DIR = /usr/local/condor/fetch/database
  DATABASE_HOOK_FETCH_WORK = $(DATABASE_HOOK_DIR)/fetch_work.php
  DATABASE_HOOK_REPLY_FETCH = $(DATABASE_HOOK_DIR)/reply_fetch.php

  # The web system only needs to fetch work.
  WEB_HOOK_DIR = /usr/local/condor/fetch/web
  WEB_HOOK_FETCH_WORK = $(WEB_HOOK_DIR)/fetch_work.php
</PRE>
<P>
The keywords <TT>"DATABASE"</TT> and <TT>"WEB"</TT> are completely arbitrary, so
each site is encouraged to use different (more specific) names as
appropriate for their own needs.

<P>

<H3><A NAME="SECTION00541300000000000000"></A><A NAME="sec:job-hooks-fetch-work-delay"></A>
<A NAME="40973"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">3</SPAN> Defining the FetchWorkDelay Expression
</H3>

<P>
There are two events that trigger the <SPAN  CLASS="textit">condor_startd</SPAN> to attempt to
fetch new work:

<UL>
<LI>the <SPAN  CLASS="textit">condor_startd</SPAN> evaluates its own state
</LI>
<LI>the <SPAN  CLASS="textit">condor_starter</SPAN> exits after completing some fetched work
</LI>
</UL>

<P>
Even if a given compute slot is already busy running other work, it is
possible that if it fetched new work, the <SPAN  CLASS="textit">condor_startd</SPAN> would prefer
this newly fetched work (via the <TT>Rank</TT> expression) over the work it
is currently running.
However, the <SPAN  CLASS="textit">condor_startd</SPAN> frequently evaluates its own state,
especially when a slot is claimed.
Therefore, administrators can define a configuration variable which controls
how long the <SPAN  CLASS="textit">condor_startd</SPAN> will wait between attempts to fetch new work.
This variable is called <TT>FetchWorkDelay</TT> <A NAME="41302"></A> <A NAME="41303"></A>.

<P>
The <TT>FetchWorkDelay</TT> expression must evaluate to an integer,
which defines the number of seconds since the last fetch attempt
completed before the <SPAN  CLASS="textit">condor_startd</SPAN> will attempt to fetch more work.
However, as a ClassAd expression (evaluated in the context of the
ClassAd of the slot considering if it should fetch more work, and the
ClassAd of the currently running job, if any), the length of the delay
can be based on the current state the slot and even the currently
running job.

<P>
For example, a common configuration would be to always wait 5
minutes (300 seconds) between attempts to fetch work, unless the slot
is Claimed/Idle, in which case the <SPAN  CLASS="textit">condor_startd</SPAN> should fetch
immediately:

<P>
<PRE>
FetchWorkDelay = ifThenElse(State == "Claimed" &amp;&amp; Activity == "Idle", 0, 300)
</PRE>
<P>
If the <SPAN  CLASS="textit">condor_startd</SPAN> wants to fetch work, but the time since the
last attempted fetch is shorter than the current value of the delay
expression, the <SPAN  CLASS="textit">condor_startd</SPAN> will set a timer to fetch as soon as
the delay expires.

<P>
If this expression is not defined, the <SPAN  CLASS="textit">condor_startd</SPAN> will default to
a five minute (300 second) delay between all attempts to fetch work.

<P>

<H3><A NAME="SECTION00541400000000000000"></A><A NAME="sec:job-hooks-example"></A>
<A NAME="40993"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">1</SPAN>.<SPAN CLASS="arabic">4</SPAN> Example Hook: Specifying the Executable at Execution Time
</H3>

<P>
The availability of multiple versions of an application leads to
the need to specify one of the versions. 
As an example, consider that 
the java universe utilizes a single, fixed JVM.
There may be multiple JVMs available, and the Condor job may
need to make the choice of JVM version.
The use of a job hook solves this problem.
The job does not use the java universe, and instead uses the
vanilla universe in combination with a 
prepare job hook to overwrite the <TT>Cmd</TT> attribute of the job ClassAd.
This attribute is the name of the
executable the <SPAN  CLASS="textit">condor_starter</SPAN> daemon will invoke,
thereby selecting the specific JVM installation.

<P>
In the configuration of the execute machine:

<P>
<PRE>
JAVA5_HOOK_PREPARE_JOB = $(LIBEXEC)/java5_prepare_hook
</PRE>
<P>
With this configuration, a job that sets the <TT>HookKeyword</TT> attribute with

<P>
<PRE>
+HookKeyword = "JAVA5"
</PRE>

<P>
in the submit description file causes the <SPAN  CLASS="textit">condor_starter</SPAN>
will run the hook specified by <TT>JAVA5_HOOK_PREPARE_JOB</TT> <A NAME="41324"></A> <A NAME="41325"></A>
before running this job.
Note that the double quote marks are required to correctly define
the attribute.
Any output from this hook is an update to the job ClassAd.  
Therefore, the hook that changes the executable may be

<P>
<PRE>
#!/bin/sh

# Read and discard the job ClassAd
cat &gt; /dev/null
echo 'Cmd = "/usr/java/java5/bin/java"'
</PRE>

<P>
The submit description file for this example job may be
<PRE>
universe = vanilla
executable = /usr/bin/java
arguments = Hello
# match with a machine that advertises the JAVA5 hook
requirements = (JAVA5_HOOK_PREPARE_JOB =!= UNDEFINED)

should_transfer_files = always
when_to_transfer_output = on_exit
transfer_input_files = Hello.class

output = hello.out
error  = hello.err
log    = hello.log

+HookKeyword="JAVA5"
queue
</PRE>Note that the <SPAN  CLASS="textbf">requirements</SPAN> command ensures that this job
matches with a machine that has <TT>JAVA5_HOOK_PREPARE_JOB</TT> defined.

<P>

<H2><A NAME="SECTION00542000000000000000"></A><A NAME="sec:job-hooks-JR-overview"></A>
<A NAME="41010"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">2</SPAN> Hooks for a Job Router
</H2> 

<P>
Job Router Hooks allow for an alternate transformation and/or 
monitoring than the <SPAN  CLASS="textit">condor_job_router</SPAN> daemon implements.
Routing is still managed by the <SPAN  CLASS="textit">condor_job_router</SPAN> daemon,
but if the Job Router Hooks are specified,
then these hooks will be used to transform
and monitor the job instead.

<P>
Job Router Hooks are similar in concept to Fetch Work Hooks,
but they are limited in their scope.
A hook is an external program or script invoked by the <SPAN  CLASS="textit">condor_job_router</SPAN>
daemon at various points during the life cycle of a routed job.

<P>
The following sections describe how and when these hooks are used,
what hooks are invoked at various stages of the job's life, 
and how to configure Condor to use these Hooks.

<P>

<H3><A NAME="SECTION00542100000000000000"></A><A NAME="sec:job-hooks-JR"></A>
<A NAME="41015"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">2</SPAN>.<SPAN CLASS="arabic">1</SPAN> Hooks Invoked for Job Routing
</H3>

<P>
The Job Router Hooks allow for replacement of the transformation engine used
by Condor for routing a job.
Since the external transformation engine is not controlled by Condor,
additional hooks provide a means to update the job's
status in Condor, and to clean up upon exit or failure cases.
This allows one job to be transformed to just about any other type of job
that Condor supports,
as well as to use execution nodes not normally available to Condor.

<P>
It is important to note that if the Job Router Hooks are utilized, 
then Condor will not ignore or work around a failure in any hook execution.
If a hook is configured,
then Condor assumes its invocation is required and will not
continue by falling back to a part of its internal engine.
For example,
if there is a problem transforming the job using the hooks,
Condor will not fall back on its transformation accomplished without the hook
to process the job.

<P>
There are 2 ways in which the Job Router Hooks may be enabled.
A job's submit description file may cause the hooks to be invoked with 
<PRE>
  +HookKeyword = "HOOKNAME"
</PRE>Adding this attribute to the job's ClassAd causes the <SPAN  CLASS="textit">condor_job_router</SPAN>
daemon on the submit machine to invoke hooks prefixed with the defined keyword.
<TT>HOOKNAME</TT> is a string chosen as an example; any string may be used.

<P>
The job's ClassAd attribute definition of <TT>HookKeyword</TT> takes
precedence,
but if not present, hooks may be enabled by defining on the submit machine
the configuration variable 
<PRE>
 JOB_ROUTER_HOOK_KEYWORD = HOOKNAME
</PRE>Like the example attribute above,
<TT>HOOKNAME</TT> represents a chosen name for the hook, 
replaced as desired or appropriate.

<P>
There are 4 hooks that the Job Router can be configured to use.
Each hook will be described below along with data passed 
to the hook and expected output.
All hooks must exit successfully.

<P>
<DL COMPACT>
<DD><A NAME="41025"></A>
</DD>
<DT>Hook: Translate</DT>
<DD><P>
The hook defined by the configuration variable 
  <TT>&lt;Keyword&gt;_HOOK_TRANSLATE_JOB</TT> <A NAME="41342"></A> <A NAME="41343"></A>
  is invoked when the Job Router has determined that a job
  meets the definition for a route.  This hook is responsible for doing the
  transformation of the job and configuring any resources that are external to
  Condor if applicable.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>The first line will be the route that the job matched as
  defined in Condor's configuration files followed by the job ClassAd,
  separated by the string <code>"------"</code> and a new line.
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>The transformed job.
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>0 for success, any non-zero value on failure.
</DD>
</DL>

<P>
<A NAME="41029"></A>
</DD>
<DT>Hook: Update Job Info</DT>
<DD><P>
The hook defined by the configuration variable 
  <TT>&lt;Keyword&gt;_HOOK_UPDATE_JOB_INFO</TT> <A NAME="41347"></A> <A NAME="41348"></A>
  is invoked to provide status on the specified routed job
  when the Job Router polls the status of routed jobs at intervals
  set by <TT>JOB_ROUTER_POLLING_PERIOD</TT> <A NAME="41352"></A> <A NAME="41353"></A>.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>The routed job ClassAd that is to be updated.
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>The job attributes to be updated in the routed job,
   or nothing, if there was no update.
   To prevent clashing with Condor's management of job attributes,
   only attributes that are not managed by Condor should be output
   from this hook.
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>0 for success, any non-zero value on failure.
</DD>
</DL>

<P>
<A NAME="41034"></A>
</DD>
<DT>Hook: Job Finalize</DT>
<DD><P>
The hook defined by the configuration variable 
  <TT>&lt;Keyword&gt;_HOOK_JOB_FINALIZE</TT> <A NAME="41357"></A> <A NAME="41358"></A>
  is invoked when the Job Router has found that the job has completed.
  Any output from the hook is treated as an update to the source job.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>The source job ClassAd, followed by the routed copy Classad that completed,
  separated by the string <code>"------"</code> and a new line.
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>An updated source job ClassAd, or nothing if there was no update.
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>0 for success, any non-zero value on failure.
</DD>
</DL>

<P>
<A NAME="41038"></A>
</DD>
<DT>Hook: Job Cleanup</DT>
<DD><P>
The hook defined by the configuration variable 
  <TT>&lt;Keyword&gt;_HOOK_JOB_CLEANUP</TT> <A NAME="41362"></A> <A NAME="41363"></A>
  is invoked when the Job Router finishes managing the job.
  This hook will be invoked regardless of whether the job
  completes successfully or not,
  and must exit successfully.

<P>
<DL>
<DT><STRONG>Command-line arguments passed to the hook</STRONG></DT>
<DD>None.
</DD>
<DT><STRONG>Standard input given to the hook</STRONG></DT>
<DD>The job ClassAd that the Job Router is done managing.
</DD>
<DT><STRONG>Expected standard output from the hook</STRONG></DT>
<DD>None.
</DD>
<DT><STRONG>Exit status of the hook</STRONG></DT>
<DD>0 for success, any non-zero value on failure.
</DD>
</DL>

<P>
</DD>
</DL>

<P>

<H2><A NAME="SECTION00543000000000000000"></A><A NAME="sec:daemon-classad-hooks"></A>
<A NAME="41044"></A> 
<A NAME="41045"></A> 
<A NAME="41046"></A> 
<A NAME="41047"></A> 
<A NAME="41048"></A>
<BR>
<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">4</SPAN>.<SPAN CLASS="arabic">3</SPAN> Daemon ClassAd Hooks
</H2> 

<P>
The <SPAN  CLASS="textit">Daemon ClassAd Hook</SPAN> mechanism is
used to run executables (called jobs) directly from the
<SPAN  CLASS="textit">condor_startd</SPAN> and <SPAN  CLASS="textit">condor_schedd</SPAN> daemons. 
The output from these jobs is incorporated into the machine ClassAd
generated by the respective daemon.
The mechanism and associated jobs have been identified by various
names, including the <SPAN  CLASS="textit">Startd Cron</SPAN>, dynamic attributes,
and a distribution of executables collectively known as <SPAN  CLASS="textit">Hawkeye</SPAN>.

<P>
Pool management tasks can be enhanced by using a 
daemon's ability to periodically run executables.
The executables are expected to generate ClassAd attributes as their output,
which are incorporated into the machine ClassAd.
Policy expressions may then reference the dynamic attributes.

<P>
Configuration variables related to Daemon ClassAd Hooks are defined
within section&nbsp;<A HREF="3_3_Configuration.html#sec:Config-hooks">3.3.36</A>.
Here is a complete configuration example.
It defines all three of the available types of jobs:
ones that use the <SPAN  CLASS="textit">condor_startd</SPAN>, benchmark jobs,
and ones that use the <SPAN  CLASS="textit">condor_schedd</SPAN>.
<PRE>
#
# Startd Cron Stuff
#
# auxiliary variable to use in identifying locations of files
MODULES = $(ROOT)/modules

STARTD_CRON_CONFIG_VAL = $(RELEASE_DIR)/bin/condor_config_val
STARTD_CRON_MAX_JOB_LOAD = 0.2
STARTD_CRON_JOBLIST =

# Test job
STARTD_CRON_JOBLIST = $(STARTD_CRON_JOBLIST) test
STARTD_CRON_TEST_MODE = OneShot
STARTD_CRON_TEST_RECONFIG_RERUN = True
STARTD_CRON_TEST_PREFIX = test_
STARTD_CRON_TEST_EXECUTABLE = $(MODULES)/test
STARTD_CRON_TEST_KILL = True
STARTD_CRON_TEST_PARAM0 = abc
STARTD_CRON_TEST_PARAM1 = 123
STARTD_CRON_TEST_SLOTS = 1
STARTD_CRON_TEST_JOB_LOAD = 0.01

# job 'date'
STARTD_CRON_JOBLIST = $(STARTD_CRON_JOBLIST) date
STARTD_CRON_DATE_MODE = Periodic
STARTD_CRON_DATE_EXECUTABLE = $(MODULES)/date
STARTD_CRON_DATE_PERIOD = 15s
STARTD_CRON_DATE_JOB_LOAD = 0.01

# Job 'foo'
STARTD_CRON_JOBLIST = $(STARTD_CRON_JOBLIST) foo
STARTD_CRON_FOO_EXECUTABLE = $(MODULES)/foo
STARTD_CRON_FOO_PREFIX = Foo
STARTD_CRON_FOO_MODE = Periodic
STARTD_CRON_FOO_PERIOD = 10m
STARTD_CRON_FOO_JOB_LOAD = 0.2

#
# Benchmark Stuff
#
BENCHMARKS_JOBLIST = mips kflops

# MIPS benchmark
BENCHMARKS_MIPS_EXECUTABLE = $(LIBEXEC)/condor_mips
BENCHMARKS_MIPS_JOB_LOAD = 1.0

# KFLOPS benchmark
BENCHMARKS_KFLOPS_EXECUTABLE = $(LIBEXEC)/condor_kflops
BENCHMARKS_KFLOPS_JOB_LOAD = 1.0

#
# Schedd Cron Stuff 
#
SCHEDD_CRON_CONFIG_VAL = $(RELEASE_DIR)/bin/condor_config_val
SCHEDD_CRON_JOBLIST =

# Test job
SCHEDD_CRON_JOBLIST = $(SCHEDD_CRON_JOBLIST) test
SCHEDD_CRON_TEST_MODE = OneShot
SCHEDD_CRON_TEST_RECONFIG_RERUN = True
SCHEDD_CRON_TEST_PREFIX = test_
SCHEDD_CRON_TEST_EXECUTABLE = $(MODULES)/test
SCHEDD_CRON_TEST_PERIOD = 5m
SCHEDD_CRON_TEST_KILL = True
SCHEDD_CRON_TEST_PARAM0 = abc
SCHEDD_CRON_TEST_PARAM1 = 123
</PRE>
<P>
<A NAME="41059"></A>

<P>

<DIV CLASS="navigation"><HR>
<!--Navigation Panel-->
<A NAME="tex2html1831"
  HREF="4_5_Application_Program.html">
<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="next.png"></A> 
<A NAME="tex2html1825"
  HREF="4_Miscellaneous_Concepts.html">
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="up.png"></A> 
<A NAME="tex2html1819"
  HREF="4_3_Computing_On.html">
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="prev.png"></A> 
<A NAME="tex2html1827"
  HREF="Contents.html">
<IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="contents.png"></A> 
<A NAME="tex2html1829"
  HREF="Index.html">
<IMG WIDTH="43" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="index" SRC="index.png"></A> 
<BR>
<B> Next:</B> <A NAME="tex2html1832"
  HREF="4_5_Application_Program.html">4.5 Application Program Interfaces</A>
<B> Up:</B> <A NAME="tex2html1826"
  HREF="4_Miscellaneous_Concepts.html">4. Miscellaneous Concepts</A>
<B> Previous:</B> <A NAME="tex2html1820"
  HREF="4_3_Computing_On.html">4.3 Computing On Demand</A>
 &nbsp; <B>  <A NAME="tex2html1828"
  HREF="Contents.html">Contents</A></B> 
 &nbsp; <B>  <A NAME="tex2html1830"
  HREF="Index.html">Index</A></B> </DIV>
<!--End of Navigation Panel-->
<ADDRESS>
condor-admin@cs.wisc.edu
</ADDRESS>
</BODY>
</HTML>
condor-doc 7.8.2~dfsg.1-1+deb7u3 / usr / share / doc / condor / html / 4_4_Hooks.html
