%+
% Name:
%   bias_parity_error_spec_0.3.tex
%
% History:
%   2010 Jul 19, created (v0.1), Glenn E. Allen
%   2010 Aug 18, removed maxpix and fixed maxerr (v0.2), GEA
%   2010 Aug 19, added a figure showing the number of errors per frame
%     (v0.3), GEA
%-

\documentclass{article}

\usepackage{cxo-memo-logo}
\usepackage[dvips]{graphics}
\usepackage{epsfig}
\usepackage{gea}
\usepackage{pstricks}

\newrgbcolor{red}{1 0 0}

\begin{document}

% 1. Header

\memobasic{
   Jonathan McDowell, SDS Group Leader }{
   Glenn E.\ Allen, SDS }{
   Bias-parity error spec }{
   0.3 }{
   http://space.mit.edu/CXC/docs/docs.html\#berr }{
   /nfs/cxc/h2/gea/SDS/Docs/Memos/Berr/bias\_parity\_error\_spec\_0.3.tex }

% 2. Tool name

\section{Bias-parity errors}

% 3. Description

\subsection{ Description }

Some observations are adversely affected by bias-parity errors.  To explore
the properties of these errors, 16,304 bias-parity error files were obtained
from the archive for the interval from the beginning of the mission through
July 8, 2010.  Figure~\ref{fig01} shows histograms of the total number and
the ``valid'' number (see sec.~\ref{proc}) of errors for each file.  The
results for the valid errors are also listed in Table~1. The results below
the horizontal line in the middle of the table are the results for the
twelve files that contain errors associated with the ``FEP0 problem.'' The
number of errors, if any, per frame of data is typically very small
(Fig.~\ref{fig02} and Table~2).  The only bias-parity error files that have
more than 10 errors per frame are the twelve files associated with the FEP0
problem.  As shown in Figure~\ref{fig03}, the number of bias-parity errors
appears to be rising with time. There is a correlation between the number of
errors and the number of pixels on which the errors occur
(Fig.~\ref{fig04}). Of the 16,304 files, there are only seven where the
number of valid errors is not equal to the number of pixels affected by the
errors.  Six of these seven files are ones associated with FEP0 problem for
which the number of errors is comparable to or greater than 262,144, the
maximum number of pixels that can be affected by the problem. The seventh
file is for {\tt OBS\_ID} 7649, where all but one of the 7,439 errors are
reported for the pixel $( {\tt CCD\_ID}, {\tt
CHIPX}, {\tt CHIPY} ) = ( 8, 786, 356 )$.%
%
\footnote{These errors are valid and are not associated with the FEP0
problem.}
%

The distribution of the valid errors in chip coordinates is shown in
Figure~\ref{fig05}. Inspection of this figure suggests that there is a
periodic pattern every 32 pixels in {\tt CHIPX}.%
%
\footnote{The FEP reads the bias-parity error data in 32-bit words.}\ \
%
Figures~\ref{fig06} and \ref{fig07} confirm the existence of a periodic
pattern.  As shown in Figure~\ref{fig08}, there is a similar, but much less
pronounced, 32-pixel periodicity in the {\tt CHIPX} values of the errors
associated with the FEP0 problem. A much more prominent feature of the FEP0
errors is that they only occur in columns with odd values of {\tt CHIPX}
(Fig.~\ref{fig09}) and in rows with values of ${\tt CHIPY} \ge 513$
(Fig.~\ref{fig08}).

The remainder of this spec describes how bias-parity error files should be
handled when the tool {\tt acis\_\-build\_\-badpix} is used to create a
bad-pixel file.

% 4. Input

\subsection{ Input }

\begin{enumerate}

  \item
    One or more Level 0 bias-parity error file (acis*berr0.fits)

\end{enumerate}

% 5. Output

\subsection{ Output }

\begin{enumerate}

  \item
    A bad-pixel file that includes, among other things, a list of the pixels
    with valid bias-parity errors (acis*bpix1.fits).

\end{enumerate}

% 6. Parameters

\subsection{ Parameters }

\begin{enumerate}

  \item
    { \tt berrfile,f,a,"",,,"Name(s) of input bias-parity error file(s)" }

  \item
    { \tt berrext,s,h,"BERR",,,"Name of bias-parity error extension" }

  \item
    { \tt pbkfile,f,a,"",,,"Name of input parameter-block file" }

  \item
    { \tt maxerr,i,h,10,{\red{0}},{\red{100}}%
    %
    \footnote{A maximum value of 100 is large enough to be well above the
    maximum number of valid errors per frame for files that are not
    associated with the FEP0 problem (Fig.~\ref{fig02} and Table~2) and
    small enough to be well below the number of errors per frame in the
    first frame associated with the FEP0 problem for the twelve FEP0 {\tt
    OBS\_ID}s in Table~1.}%
    %
    ,"Maximum number of valid
    bias-parity errors per frame" }

  \item
    { \tt outfile,f,a"",,,"Name of output bad-pixel file" }

\end{enumerate}

% 7. Processing

\subsection{ Processing }
\label{proc}

Perform the following tests before processing begins.
%
\begin{itemize}

  \item
    Verify that each {\tt berrfile} exists and has an an extension where the
    value of {\tt EXTNAME} is identical to the value specified by {\tt
    berrext}.  If one or both of these conditions is not true, then exit
    with an error message.

  \item
    Verify that the {\tt pbkfile} exists.  If it does not, then exit with an
    error message.

  \item
    Verify that {\tt maxerr} is in the specified range.

  \item
    If {\tt clobber = no}, then verify that the {\tt outfile} does not
    exist.  If it does, then exit with an error message.

\end{itemize}

Perform the following steps, in sequence, for each bias-parity error file.
Note that each file corresponds to a single CCD for an observation.

\begin{enumerate}

  \item
    Ignore invalid bias-parity errors.  Invalid errors are those for which
    {\red{\tt{DATAMODE = VFAINT} and}} one of the following is true.%
    %
    \footnote{As defined here, all errors associated with the FEP0 problem,
    except for those at $({\tt CCDX},{\tt CCDY}) = (0,{\tt ROWCNT})$, are
    valid.}
    %
    \begin{itemize}

      \item
        { \red $( {\tt CCDX}, {\tt CCDY} ) = ( 0, {\tt ROWCNT} )$ or }

      \item
        { \red $( {\tt CCDX}, {\tt CCDY} ) = ( 1, {\tt ROWCNT} )$ or }

      \item
        $( {\tt CCDX}, {\tt CCDY} ) = ( 1022, 0 )$ or

      \item
        $( {\tt CCDX}, {\tt CCDY} ) = ( 1023, 0 )$.

    \end{itemize}
    %
    Here {\tt DATAMODE} is a keyword in the {\tt berrfile}, {\tt CCDX} and
    {\tt CCDY} are elements of columns with the same names in the {\tt
    berrfile} and {\tt ROWCNT} is a keyword in the {\tt pbkfile}.

  \item
    If a {\tt berrfile} includes one or more valid bias-parity errors, then
    set the {\tt FEP\_ID} equal to the value of the keyword of the same name
    in the {\tt berrfile} and use the information in the binary table of the
    {\tt pbkfile} to find the corresponding {\tt CCD\_ID}.

  \item
    If {\red ${\tt FEP\_ID} = 0$ and} the total number of valid errors per
    frame is greater than {\tt maxerr} for any frame, then the FEP suffered
    from the FEP0 problem.

  \item
    If a {\tt berrfile} includes one or more valid bias-parity errors and
    does not contain errors associated with the FEP0 problem, then include
    in the extension of the {\tt outfile} that is associated with {\tt
    CCD\_ID} one or more entries such that
    %
    \begin{itemize}

      \item
        {\tt SHAPE} is computed as usual,

      \item
        {\tt COMPONENT} is computed as usual,

      \item
        ${\tt CHIPX} = {\tt CCDX} + 1$,

      \item
        ${\tt CHIPY} = {\tt CCDY} + {\red{\tt STARTROW} +} 1$,

      \item
        ${\tt TIME} = {\tt TIME}_{\rm berr}$,

      \item
        ${\tt TIME\_STOP} = {\red {\tt TSTOP}}$, and

      \item
        {\tt STATUS} has bit 2 (of 0-31) set to one.

    \end{itemize}
    %
    Here, {\tt CCDX} and {\tt CCDY} are the coordinates of the bias-parity
    error, ${\tt TIME}_{\rm berr}$ is the {\tt TIME} associated with the
    error, and {\tt STARTROW} and {\tt TSTOP} are keywords in the {\tt
    pbkfile}.

  \item
    If a {\tt berrfile} contains valid bias-parity errors associated with
    the FEP0 problem, then include in the extension of the {\tt outfile}
    that is associated with {\tt CCD\_ID} an entry such that%
    %
    \footnote{An examination of the twelve bias-parity error files that
    include errors associated with the FEP0 problem (\ie\ ones for the {\tt
    OBS\_ID}s that are below the line in Table~1), reveals that the only
    errors reported are errors associated with the FEP0 problem.  There is
    no evidence that there are other valid errors in the files.  Therefore,
    the only entries that should be included in the corresponding bad-pixel
    files are entries for the FEP0 problem.}
    %
    \begin{itemize}

      \item
        {\tt SHAPE} is determined as usual,

      \item
        {\tt COMPONENT} is computed as usual,

      \item
        $1 \le {\tt CHIPX} \le 1024$,

      \item
        $513 \le {\tt CHIPY} \le 1024$,

      \item
        ${\tt TIME} = {\tt TIME}_{\rm berr}$,

      \item
        ${\tt TIME\_STOP} = {\red {\tt TSTOP}}$,
	and

      \item
        {\tt STATUS} has bit 13 (of 0-31) set to one.

    \end{itemize}
    %
    Here, ${\tt TIME}_{\rm berr}$ is the {\tt TIME} associated with the
    first frame in which there are more than {\tt maxerr} valid bias-parity
    errors and {\tt TSTOP} is a keyword in the {\tt pbkfile}.

\end{enumerate}

% 8. Caveats

\subsection{ Caveats }

\begin{enumerate}

  \item
    The values of {\tt TIME} in the bias-parity error files appear to be the
    {\tt TIME}s of the mid points of the frames.  Therefore, perhaps the
    values of {\tt TIME} in the output file should be ${\tt TIME}_{\rm stat}
    - {\tt TIMEPIXR}_{\rm stat} \times {\tt TIMEDEL}_{\rm stat}$ instead of
    ${\tt TIME}_{\rm berr}$, where ${\tt TIME}_{\rm stat}$ is the time
    associated with the relevant {\tt EXPNO}.

\end{enumerate}

% 9. Figures and table

% 9.1. Table 1

\begin{center}
  \begin{tabular}{ccl}
    \multicolumn{3}{c}{Table 1. Histogram of the number of valid bias-parity
      errors} \\
    \hline \hline
    No.\ valid errors & No.\ files & {\tt OBS\_ID}(s) \\
    \hline
    0 & 14079 & {\ldots} \\
    1 & 2055 & {\ldots} \\
    2 & 124 & {\ldots} \\
    3 & 23 & {\ldots} \\
    4 & 6 & 2977, 5771, 9581, 10052, 10806, 11011 \\
    5 & 1 & 4195 \\
    6 & 2 & 9893, 11058 \\
    7 & 1 & 9924 \\
    7439 & 1 & 7649 \\
    \hline
    127003 [3550]\ \ \ \ & 1 & 62340 \\
    136481 [625]\ \ \ \ \ & 1 & 62338 \\
    139559 [139522] & 1 & 965 \\
    139756 [139744] & 1 & 510 \\
    149668 [149668] & 1 & 62502 \\
    186595 [186383] & 1 & 62333 \\
    200635 [4827]\ \ \ \ & 1 & 62327 \\
    238050 [146756] & 1 & 62353 \\
    265784 [118226] & 1 & 18 \\
    336595 [79091]\ \ & 1 & 62363 \\
    1043209 [139813]\ \ & 1 & 1383 \\
    3286241 [101649]\ \ & 1 & 333 \\
    \hline
    Total & 16304 & {\ldots} \\
    \hline \hline
    %
    \multicolumn{3}{l}{\parbox{5.0in}{The numbers inside the square brackets
    in the first column are the number of errors in the first frame that has
    an error.  These values are well above the maximum allowed value for
    {\tt maxerr}.}}
    %
  \end{tabular}
\end{center}

% 9.2. Table 2

\vspace*{1.0\baselineskip}
\begin{center}
  \begin{tabular}{ccl}
    \multicolumn{3}{c}{Table 2. Histogram of the number of valid bias-parity
      errors per frame} \\
    \hline \hline
    No.\ valid errors per frame & No.\ frames & {\tt OBS\_ID}(s) \\
    \hline
    1 & 9613 & {\ldots} \\
    2 & 90 & {\ldots} \\
    3 & 13 & {\ldots} \\
    4 & 1 & 5771 \\
    5 & 1 & 4195 \\
    6 & 2 & 9893, 11058 \\
    7 & 1 & 9924 \\
    \hline
    Total & 9721 & {\ldots} \\
    \hline \hline
    %
    \multicolumn{3}{l}{\parbox{5.0in}{The very large number of frames with
      no valid errors is omitted.  The data in the files that are associated
      with the FEP0 problem are also omitted. Therefore, the values for the
      number of frames are the differences between the red and green
      histograms in Figure~\ref{fig02}. }}
    %
  \end{tabular}
\end{center}

% 9.3. Fig 1

\begin{figure}
  \hbox{\psfig{file=n_errs_hist.ps,angle=-90,width=6.5in}}
  %
  \caption{Histograms of the number of bias-parity errors in the bias-parity
    error files.  The black histogram includes all telemetered bias-parity
    errors.  The red histogram includes only the valid errors (see
    sec.~{\ref{proc}}).  The bin at the right-hand side of the plot
    indicates the number of files that have at least 200 errors.
    %
    \label{fig01}}
\end{figure}

% 9.4. Fig 2

\begin{figure}
  \hbox{\psfig{file=err_rate_hist.ps,angle=-90,width=6.5in}}
  %
  \caption{Histograms of the number of bias-parity errors per frame of data.
    The black histogram includes all telemetered bias-parity errors.  The
    red histogram includes only the valid errors (see sec.~\ref{proc}).  The
    green histogram includes only the valid errors that are associated with
    the FEP0 problem.  The bin at the right-hand side of the plot indicates
    the number for frames that have at least 30 errors.  All twelve of the
    files associated with the FEP0 problem have at least one frame with 30
    or more errors (Table~1).  The dotted, vertical line is the default value for {\tt
    maxerr}.
    %
    \label{fig02}}
\end{figure}

% 9.5. Fig 3

\begin{figure}
  \hbox{\psfig{file=n_errs_v_time.ps,angle=-90,width=6.5in}}
  %
  \caption{The number of valid bias-parity errors as a function of time.
    Each data point is the sum over a 90-day interval.  The day is the
    mission elapsed time (\ie\ is relative to 1998-01-01T00:00:00).  Two
    data points have more than 100 errors and are, therefore, not shown on
    the plot.  The data point for the days from 630 to 720 includes the
    twelve files that are associated with the FEP0 problem.  The data point
    for the days from 3330 to 3420 includes {\tt OBS\_ID} 7649.  Aside from
    these two anomalies, the number of bias-parity errors appears to be
    rising with time.
    %
    \label{fig03}}
\end{figure}

% 9.6. Fig 4

\begin{figure}
  \hbox{\psfig{file=n_errs_v_n_pix.ps,angle=-90,width=6.5in}}
  %
  \caption{The number of pixels on which valid bias-parity errors occur v.\
    the number of valid errors. The dashed line is the line along which the
    number of pixels equals the number of errors.  It is not possible for a
    data point to lie above this line (\ie\ to have more pixels with
    bias-parity errors than errors).  The cluster of points near the upper,
    right-hand corner are those associated with the FEP0 problem.  The
    dotted, horizontal line at 262,144 represents the maximum number of
    pixels that can be affected by the FEP0 problem.  The point near the
    middle of the lower edge is for {\tt OBS\_ID} 7649.  A bias-parity error
    file for this observation had 7,439 errors, all but one of which
    occurred at the location $( {\tt CCD\_ID}, {\tt CHIPX}, {\tt CHIPY} ) =
    ( 8, 786, 356 )$.
    %
    \label{fig04}}
\end{figure}

% 9.7. Fig 5

\begin{figure}
  \hbox{\psfig{file=valid_coords.ps,angle=-90,width=6.5in}}
  %
  \caption{A plot of the chip coordinates at which valid bias-parity errors
    occur. Note that all CCDs are included.  The data in the files that are
    associated with the FEP0 problem are excluded.  A periodic pattern in
    {\tt CHIPX} is evident.
    %
    \label{fig05}}
\end{figure}

% 9.8. Fig 6

\begin{figure}
  \hbox{\psfig{file=n_valid_v_coords_1024.ps,angle=-90,width=6.5in}}
  %
  \caption{Histograms of the number of valid bias-parity errors as a
    function of {\tt CHIPX} (black) and {\tt CHIPY} (red).  Note that all
    CCDs are included.  The data in the files that are associated with the
    FEP0 problem are excluded.  The spikes at ${\tt CHIPX} = 786$ and ${\tt
    CHIPY} = 356$ are associated with the 7,438 errors at this location for
    {\tt OBS\_ID} 7649.  A periodic pattern in {\tt CHIPX} is evident.
    %
    \label{fig06}}
\end{figure}

% 9.9. Fig 7

\begin{figure}
  \hbox{\psfig{file=n_valid_v_coords_32.ps,angle=-90,width=6.5in}}
  %
  \caption{Histograms of the number of valid bias-parity errors as a
    function of {\tt CHIPX} (black) and {\tt CHIPY} (red).  The coordinates
    are modulo 32.  Note that all CCDs are included.  The data in the files
    that are associated with the FEP0 problem are excluded.  The spikes at
    coordinates of 18 and 2 are associated with the 7,438 errors at ${\tt
    CHIPX} = 786$ and ${\tt CHIPY} = 356$ for {\tt OBS\_ID} 7649. Errors
    occur almost exclusively on one of the last four columns in each set of
    32 columns.
    %
    \label{fig07}}
\end{figure}

% 9.10. Fig 8

\begin{figure}
  \hbox{\psfig{file=n_fep0_v_coords_1024.ps,angle=-90,width=6.5in}}
  %
  \caption{Histograms of the number of bias-parity errors as a function of
    {\tt CHIPX} (black) and {\tt CHIPY} (red) for the errors associated with
    the FEP0 problem.  Note that all CCDs are included. A periodic pattern
    in {\tt CHIPX} is evident.  No errors are reported for pixels that have
    ${\tt CHIPY} < 513$.
    %
    \label{fig08}}
\end{figure}

% 9.11. Fig 9

\begin{figure}
  \hbox{\psfig{file=n_fep0_v_coords_32.ps,angle=-90,width=6.5in}}
  %
  \caption{Histograms of the number of bias-parity errors as a function of
    {\tt CHIPX} (black) and {\tt CHIPY} (red) for the errors associated with
    the FEP0 problem.  The coordinates are modulo 32.  Note that all CCDs
    are included.  No errors are reported for pixels that have even values
    of {\tt CHIPX}.
    %
    \label{fig09}}
\end{figure}

% 10. Finish

\end{document}

