%+
% Name:
%   afterglow_spec_1.6.tex
%
% History:
%   2008 Feb 10, created (v1.0), Glenn E. Allen
%   2008 Feb 11, added cntthresh (v1.1), GEA
%   2008 Feb 12, added maskfile (v1.2), GEA
%   2008 Feb 13, added comments about interleaved mode (v1.3), GEA
%   2008 Jul 23, significantly revised (v1.4), GEA
%   2008 Jul 28, added SAMP_CYC (v1.5), GEA
%   2008 Jul 29, added regwidth (v1.6), GEA
%-

\documentclass{article}

\usepackage{changebar}
\usepackage{cxo-memo-logo}
\usepackage[dvips]{graphics}
\usepackage{epsfig}
\usepackage{gea}

\begin{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 1. Header

\memobasic{
   Jonathan McDowell, SDS Group Leader }{
   Glenn E.\ Allen, SDS }{
   Afterglow spec }{
   1.6 }{
   http://space.mit.edu/CXC/docs/docs.html\#aft }{
   /nfs/cxc/h2/gea/sds/docs/memos/afterglow\_spec\_1.6.tex }

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 2. Tool name

\section{Afterglows}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 3. Description

\subsection{ Description }

A cosmic-ray ``afterglow'' is produced when a large amount of charge is
deposited on a CCD by a cosmic ray. Most of the charge is clocked off of the
CCD in a single frame. However, a small amount can be captured in charge
traps that release the charge relatively slowly. As a result, a sequence of
events can appear in a single detector pixel over a few frames as the
trapped charge is released.

To date, two algorithms have been used by the CXC to identify cosmic-ray
afterglows. The first algorithm was implemented in the CIAO tool {\tt
acis\_detect\_afterglow} and used for pipeline processing from the summer of
2000 to the fall of 2004. This algorithm searches for occasions when events
are detected in two or more consecutive frames on the same CCD pixel. While
the events are flagged as potential cosmic-ray afterglows and excluded from
Level 2 event-data files, the corresponding pixels are not included in the
observation-specific bad-pixel file. This algorithm finds many afterglow
events, but at the expense of discarding X-ray events associated with real
astrophysical sources.  The fraction of the source events that are discarded
depends on the brightness and variability of a source.

In an attempt to minimize the loss of source events, another algorithm was
developed and implemented in the CIAO tool {\tt acis\_run\_hotpix}, which is
a wrapper around the tools {\tt acis\_find\_hotpix}, {\tt
acis\_classify\-\_hotpix} and {\tt acis\_build\_badpix}. The second
algorithm searches for detector pixels that have an unusually large number
of events that occur over a short period of time. Suspicious pixels are
added to the observation-specific bad-pixel file only if the neighboring
pixels do not have a significant excess of events.  This condition helps
insure that events associated with dithered sources are not discarded.
Events associated with afterglows are flagged and excluded from Level 2
event-data files. The newer algorithm has been used for pipeline processing
(and reprocessing) since the fall of 2004. While it is relatively gentle on
astrophysical sources, it does let some afterglows ``slip through the
cracks.'' The afterglow detection efficiency depends on the number of events
in the afterglow.  The efficiency declines quickly as the number of events
in an afterglow drops below about eight.

This spec describes a third afterglow-detection algorithm. Like the second
algorithm, it is designed to avoid discarding events associated with real
astrophysical sources.  It is also designed to enhance the detection
efficiency for afterglows that have as few as four events. The principal
change between the second and third afterglow-detection algorithms is that
the third algorithm searches for afterglows using the events in a short,
sliding time window instead of using the events from the entire duration of
an observation (\ie, the algorithm searches in three dimensions instead of
two).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 4. Input

\subsection{ Input }

\begin{enumerate}

  \item
    A Level 1 event-data file (acis*evt1.fits)

  \item
    A Level 1 observation-specific bad-pixel file (acis*bpix1.fits)

  \item
    A Level 1 mask file (acis*msk1.fits)

  \item
    A Level 1 exposure statistics file (acis*stat1.fits)

\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 5. Output

\subsection{ Output }

\begin{enumerate}

  \item
    An updated observation-specific bad-pixel file

\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 6. Parameters

\subsection{ Parameters }

\begin{enumerate}

  \item
    infile,s,a,``'',,,``Name of input event-data file''

  \item
    outfile,s,a``'',,,``Name of output bad-pixel file''

  \item
    badpixfile,s,a,``'',,,``Name of input bad-pixel file''

  \item
    maskfile,s,a,``'',,,``Name of input mask file''

  \item
    statfile,s,a,``'',,,``Name of input exposure-statistics file''

  \item
    expnowindow,i,h,10,1,100,``Number of frames in the sliding time window''

  \item
    probthresh,r,h,0.001,1.0e-10,0.1,``Minimum post-trials significance of
    potential afterglows (\eg, 1 sigma = 0.159, 90\% = 0.1, 2 sigma =
    0.0228, 99\% = 0.01 and 3 sigma = 0.00135)''

  \item
    cntthresh,i,h,2,4,10,``Minimum number of events in an afterglow''

  \item
    \begin{changebar}
    regwidth,i,h,7,3,255,``Size of reference region (\eg, 7~pixels $\times$
    7~pixels)'' 
    \end{changebar}

  \item
    clobber,b,h,``no'',,,``Overwrite output file if it exists?''

  \item
    verbose,i,h,0,0,5,``Amount of messages produced (0$=$none, 5$=$a lot)''

  \item
    mode,s,h,``ql'',,,

\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 7. Processing

\subsection{ Processing }

In the standard ACIS pipeline, the afterglow-detection algorithm is used
after the bias(es) has been searched for bad bias values and before the
event data is searched for hot pixels.  The afterglow-detection algorithm is
summarized below.

\begin{enumerate}

  \item
    Verify that the input files exist.
    %
    If clobber=no, then verify that the output files do not exist.
    %
    Verify that the input event-data file has READMODE=TIMED.  The
    afterglow-detection algorithm is not appropriate for continuous-clocking
    mode event data.
    %
    Verify that the values of the parameters {\tt expnowindow}, {\tt
    probthresh}, {\tt cntthresh}
    %
    \begin{changebar}
    and {\tt regwidth}
    \end{changebar}
    %
    are in the valid ranges for these parameters.
    %
    \begin{changebar}
    Note that {\tt regwidth} must be an odd number.
    \end{changebar}

  \item
    Exclude ``invalid'' pixels%
    %
    \footnote{Here an invalid pixel is one that has one or more of the
    following STATUS bits set in the observation-specific bad-pixel file or
    that has ${\rm SAMP\_CYC} = 0$ in the mask file.

      \vspace*{1.0\baselineskip}
      \hbox{
        \hspace*{0.25in}
        \begin{tabular}{cl}
          Bit & Description \\ \hline
          0 & bad pixel \\
          2 & bias-parity error \\
          3 & bias = 4095 \\
          4 & bias = 4094 \\
          13 & FEP0 problem \\
          16 & bad bias value \\
        \end{tabular}
      }

      \vspace*{1.0\baselineskip}
      \noindent
      %
      Note that the STATUS bits are numbered from 0 to 31. It is not
      necessary to ignore pixels that have bias values of 4096 (\ie, are
      missing data) because biases with such problems are adjusted on the
      ground.  If they are not adjusted, then all events on pixels with a
      bias = 4096 are discarded.
      }
    %
    from the search.

  \item
    To improve the performance of the algorithm, perform more than one pass
    through the data.  In the first pass, potential afterglow events are
    identified as suspicious using a minimum set of criteria.  Events $i$
    and $j$ may be part of an afterglow if the following four conditions are
    satisfied.
    %
    \begin{eqnarray}
      {\rm CCD\_ID}_{i} & = & {\rm CCD\_ID}_{j} \label{eqn01}, \\
      {\rm CHIPX}_{i} & = & {\rm CHIPX}_{j}, \\
      {\rm CHIPY}_{i} & = & {\rm CHIPY}_{j},\ {\rm and} \\
      |{\rm EXPNO}_{i} - {\rm EXPNO}_{j}| & \le & {\tt expnowindow}.
        \label{eqn04}
    \end{eqnarray}

  \item
    During a subsequent pass through the data, the $i$th, $(i+1)$th,
    {\ldots}, $(i+n)$th set of events on a pixel is identified as an
    afterglow if each consecutive pair of events in the set satisfies
    equations~\ref{eqn01}--\ref{eqn04} and if
    %
    \begin{eqnarray}
      N_{\rm evt}^{\rm aft} & \ge & {\tt cntthresh}, \\
      P_{\rm post} & < & {\tt probthresh},\ {\rm and} \\
      P_{\rm ref} & \ge & {\tt probthresh}, \label{eqn07} \\
    \end{eqnarray}
    %
    where the post-trials probability
    %
    \begin{equation}
      P_{\rm post}
        =
	1 -
	\left(
	  1 - P_{\rm pre}
	\right)^{\rm N_{\rm trial}},
    \end{equation}
    %
    the pre-trials probability
    %
    \begin{equation}
      P_{\rm pre}
        =
        1 -
        \left[
          \left(
            \sum_{i=0}^{N_{\rm evt}^{\rm aft}-1}
            \frac{\left( N_{\rm bgd}^{\rm aft} \right)^{i}}{i!}
          \right) +
          \frac{1}{2}
          \frac{
	    \left( N_{\rm bgd}^{\rm aft} \right)^{N_{\rm evt}^{\rm aft}}
	  }{
	    N_{\rm evt}^{\rm aft}!
	  }
        \right]
        e^{-N_{\rm bgd}^{\rm aft}},
    \end{equation}
    %
    $N_{\rm evt}^{\rm aft}$ is the number of events in the potential
    afterglow, the number of background events for the potential afterglow
    %
    \begin{equation}
      N_{\rm bgd}^{\rm aft}
        =
	\frac{ F }{ {\rm SAMP\_CYC}_{\rm aft} }
	\left(
	  \frac{
	    N_{\rm frame}^{\rm aft}
	  }{
	    N_{\rm frame}^{\rm tot}
	  }
	\right),
    \end{equation}
    %
    ${\rm SAMP\_CYC}_{\rm aft}$ is the sample cycle for the pixel on which
    the potential afterglow occurred,
    %
    $N_{\rm frame}^{\rm aft}$ is the number of valid frames%
    %
    \footnote{Here an invalid frame for a CCD is one that is not listed in
    the exposure statistics file.  Also note that for TIMED mode
    observations, frames with ${\rm EXPNO} < 3$ are invalid.}
    %
    in the afterglow,
    %
    $N_{\rm frame}^{\rm tot}$ is the total number of valid frames for the CCD,
    %
    the number of trials%
    %
    \footnote{The estimate of $N_{\rm trial}$ is an upper limit on the
    number of trials.  The actual number of trials includes only the number
    of ``independent'' searches.  Since adjacent windows in the sliding
    EXPNO window overlap, they are not independent.  A lower limit on
    $N_{\rm trial}$ can be obtained by calculating the number of
    nonoverlapping windows.  This value is smaller than equation~\ref{eqn11}
    by a factor of about $({\tt expnowindow + 1})$.  Since a precise value
    for $N_{\rm trial}$ can be difficult to determine, equation~\ref{eqn11}
    is used because it yields the most conservative (\ie, the largest)
    number of trials.
    }
    %
    \begin{equation}
      N_{\rm trial}
        =
	\sum_{k}
	N_{{\rm pix},k}^{\rm ccd}
	\left( N_{{\rm frame},k}^{\rm tot} - {\tt expnowindow} - 1 \right),
	\label{eqn11}
    \end{equation}
    %
    $N_{{\rm pix},k}^{\rm ccd}$ is the number of valid pixels$^{1}$ for the
    $k$th CCD (\ie, $= 1024 \times 1024$ less the number of invalid pixels),
    %
    $N_{{\rm frame},k}^{\rm tot}$ is the total number of valid frames for
    the $k$th CCD, the probability%
    %
    \footnote{Unlike the previous afterglow-detection algorithm, the
    probability $P_{\rm ref}$ is a pre-trials probability instead of a
    post-trials probability.  In this case, it is more difficult for events
    associated with real astrophysical sources to be identified as
    afterglows.}
    %
    that the event fluence in the reference region is consistent with the
    event fluence on the entire node (\ie, that the potential afterglow is
    not part of a dithered source)
    %
    \begin{equation}
      P_{\rm ref}
        =
	\left\{
	  \begin{array}{ll}
            1 -
            \left[
              \left(
                \sum_{i=0}^{N_{\rm evt}^{\rm ref}-1}
                \frac{\left( N_{\rm bgd}^{\rm ref} \right)^{i}}{i!}
              \right) +
              \frac{1}{2}
              \frac{\left( N_{\rm bgd}^{\rm ref} \right)^{N_{\rm evt}^{\rm ref}}}{N_{\rm evt}^{\rm ref}!}
            \right]
            e^{-N_{\rm bgd}^{\rm ref}}
	      & (N_{\rm evt}^{\rm ref} > 0)
	      \\
	    1
	      & (N_{\rm evt}^{\rm ref} = 0)
	  \end{array}
	\right.
    \end{equation}
    %
    the number of background events for the reference region%
    %
    \footnote{
    Equation~\ref{eqn15} is valid only if all of the valid pixels in the
    reference region have the same sample cycle.  If, for example, the
    reference region contains subsets A and B with $N_{\rm pix,A}^{\rm ref}$
    and $N_{\rm pix,B}^{\rm ref}$ valid pixels and sample cycles ${\rm
    SAMP\_CYC}_{\rm A}$ and ${\rm SAMP\_CYC}_{\rm B}$, respectively, then
    equation~\ref{eqn15} becomes
    %
    \begin{equation}
      N_{\rm bgd}^{\rm ref}
        =
	F
	\left(
	  \frac{ N_{\rm pix,A}^{\rm ref} }{ {\rm SAMP\_CYC}_{\rm A} } +
	  \frac{ N_{\rm pix,B}^{\rm ref} }{ {\rm SAMP\_CYC}_{\rm B} }
	\right).
    \end{equation}
    }
    %
    \begin{equation}
      N_{\rm bgd}^{\rm ref}
        =
	F
	N_{\rm pix}^{\rm ref},
	\label{eqn15}
    \end{equation}
    %
    $N_{\rm pix}^{\rm ref}$ is the number of valid pixels%
    %
    \footnote{For the purposes of calculating $N_{\rm pix}^{\rm ref}$, the
    central pixel of the region (\ie, the pixel on which the potential
    afterglow occurred) is invalid as are pixels that lie on a different
    node from the central pixel. Other pixels are considered invalid if they
    satisfy the usual conditions.$^{1}$ 
    }
    %
    in the 
    %
    \begin{changebar}
    {\tt regwidth}~pixel $\times$ {\tt regwidth}~pixel
    \end{changebar}
    %
    reference region surrounding the pixel with the potential afterglow, and
    the nominal fluence $F$ is computed as follows.
    %
    \begin{enumerate}
      \item
        For each 32~pixel $\times$ 32~pixel region $l$ of the node,%
	\footnote{
	Equation~\ref{eqn17} is valid only if all of the valid pixels in the
	region have the same sample cycle.  If, for example, the region
	contains subsets A and B with $N_{\rm evt,A}^{l}$ and $N_{\rm
	evt,B}^{l}$ events on $N_{\rm pix,A}^{l}$ and $N_{\rm pix,B}^{l}$
	valid pixels and sample cycles ${\rm SAMP\_CYC}_{\rm A}$ and ${\rm
	SAMP\_CYC}_{\rm B}$, respectively, then equation~\ref{eqn17} becomes
	%
	\begin{equation}
	  F_{l}
	    =
	    \frac{
	      {\rm SAMP\_CYC}_{\rm A} N_{\rm evt,A}^{l} +
	      {\rm SAMP\_CYC}_{\rm B} N_{\rm evt,B}^{l}
	    }{
	      N_{\rm pix,A}^{l} + N_{\rm pix,A}^{l}
	    }.
	\end{equation}
	}
	%
	\begin{equation}
	  F_{l}
	    =
	    \frac{
	      N_{\rm evt}^{l}
	    }{
	      N_{\rm pix}^{l}
	    },
	    \label{eqn17}
	\end{equation}
        %
        where $N_{\rm pix}^{l}$ is the total number of valid pixels$^{1}$ in
	the region, and
        %
        $N_{\rm evt}^{l}$ is the total number of events on these pixels
	during the entire observation.
	%
      \item
        Select the regions where $F_{l}$ is greater than zero and less than
	two times the mean value of the set of $F_{l}$s.
      \item
        Calculate the median value, $F_{\rm med}$, of the selected values of
	$F_{l}$.
      \item
        Calculate the root-mean-square, $F_{\rm rms}$, of the selected values.
      \item
        Select the regions where $F_{l}$ is greater than zero, is greater
	than or equal to $F_{\rm med} - 2 F_{\rm rms}$, and is less than
	$F_{\rm med} + 2 F_{\rm rms}$.
	%
	\label{stepf}
      \item
        Calculate the median of the selected values.
      \item
        Calculate the root-mean-square of the selected values.
	\label{stepg}
      \item
        Repeat steps~\ref{stepf}--\ref{stepg} an additional nine times
	(\ie, a total of ten times).
      \item
        Set $F$ equal to the value of $F_{\rm med}$ from the last iteration.
    \end{enumerate}
    %

  \item
    Each potential afterglow that satisfies the criteria in equations
    \ref{eqn01}--\ref{eqn07} is written to the outfile with
    %
    \begin{equation}
      {\rm TIME}
        =
	{\rm TIME}_{\rm start} -
	{\rm TIMEPIXR} \times {\rm TIMEDEL} -
	{\rm FLSHTIME}
    \end{equation}
    %
    and
    %
    \begin{equation}
      {\rm TIME\_STOP}
        =
	{\rm TIME}_{\rm stop} +
	(1 - {\rm TIMEPIXR}) \times {\rm TIMEDEL},
    \end{equation}
    %
    where ${\rm TIME}_{\rm start}$ and ${\rm TIME}_{\rm stop}$ are the TIMEs
    in the exposure statistics file that are associated with the start and
    stop EXPNOs of the afterglow and TIMEDEL, TIMEPIXR and FLSHTIME are
    keywords in the exposure statistics file. The contents of the input
    bad-pixel file is also copied to the output file. Note that it is
    possible, albeit unlikely, for more than one afterglow to occur on the
    same pixel during an observation.

\end{enumerate}

Once the afterglow-detection algorithm has been used, the tools {\tt
acis\_build\_badpix} and {\tt acis\_process\-\_events} are used to mark the
pixels adjacent to afterglows as bad and to set the appropriate STATUS bit
(16 of 0--31) for events associated with afterglows.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 8. Caveats

\subsection{ Caveats }

\begin{enumerate}

  \item
    Although it may not be optimum to do so, the afterglow-detection
    algorithm is applied separately to the primary and secondary data for
    interleaved mode observations.

  \item
    The algorithm is not applied to the data for continuous-clocking mode
    observations.

  \item
    The choices of default values for the parameters {\tt expnowindow},
    {\tt cntthresh}
    %
    \begin{changebar}
    and {\tt regwidth}
    \end{changebar}
    %
    may not be optimum.

\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 9. Finish

\end{document}

