Re: ISIS: loading data with proper statistical error

Dear Mike, David and John,

Thanks a lot for your answers.

I should have stressed that I know that there's no statistical error present in my pha files.
I'm working with XMM-Newton and NuSTAR data, and in both cases, their pipelines don't produce any error column in their PHA file.

I just didn't realize that before as XSPEC does calculate the errors when we load in the data.

So my question was actually how do I calculate the errors and then assign them to my data set?

I think that David answered this question.
It seems I should be able to do that with  put_data_counts.


Now, and it's a whole different discussion, how do you think I should calculate these errors?
A lot of bins contain less that 10 counts, so Gaussian statistics (sqrt(N)) would be totally underestimated. 
Poisson statistics would be more correct. Is there a build-in function to calculate the errors (something like what Gherels (1986) described)?

With Poisson errors in each bin, should I worry when I rebin the data that the errors will end up over-estimated?

Thanks again for your help,

Nicolas



Dr. Nicolas Barrière
***********************************************
barriere_at_email.domain.hidden510 - 643 - 4747
Space Sciences Lab
7 Gauss way
Berkeley – CA 94720
USA
***********************************************

On Nov 30, 2012, at 8:03 AM, Michael Nowak wrote:

> 
> On Nov 29, 2012, at 6:11 PM, Nicolas Barrière wrote:
> 
>> I just started to use ISIS one week ago, and although I've been steadily becoming more comfortable with its use, I'm now stuck on a problem that should be very simple.
>> 
>> When I load a spectrum using 
>> id = data_load ("filename.pha");
>> 
>> it seems that the statistical errors are not calculated.
>> 
>> I noticed that when I was trying to plot the data, but also when I do:
>> 
>> data = get_data_counts(id);
>> print (data.err);
>> 
>> 
>> Am I missing something?
> 
> If I'm remembering this all correctly, the default is for ISIS to read the error column in the data file and use that.  So perhaps the data is not properly stored in the column.  
> 
> There are a few caveats, however:
> 
> 1) There is a minimum statistical error controlled by a global variable, Minimum_Stat_Err, which is normally set to 1 (so bins with 0 counts don't
> get set to 0 error).  I normally reset that to 1e-30 for various reasons.  That
> could change things.
> 
> 2)  ISIS does not keep the grouping of the data set - after reading it in, 
> ISIS goes back to unbinned data.
> 
>    group(id;min_sn=4.5,bounds=0.5,unit="kev");
>    notice_values(id,0.5,8;unit="kev");
> 
> will get you something like the usual minimum 20 counts/bin (assuming no
> background), and then notice the 0.5-8 keV range.  Once you rebin, ISIS
> applies the "standard" rules for the statistical error (usually square root of
> counts, but you can override this behavior, and I do, for example, when using
> a separate spectrum as a "correction file" to the background).
> 
> So, if the error bars in the pha file aren't what you expect, and you haven't
> grouped the data after reading it in, it's possible to have the error bars be
> not what you expected.  Regrouping the data or changing the pha file error
> column should change it.
> 
> You can do get_data_counts/put_data_counts combinations to override things,
> but you probably want to avoid that since that itself will be overridden if you do
> any rebinning of the data during analysis.  (And that's one of the main features 
> we use of ISIS - the ability to rebin the data during analysis.)
> 
> If that doesn't do it for you, send me a copy of the data with a little bit more detail
> about how it's deviating from expectations, and I'll take a closer look.
> 
> Cheers,
> 
> Mike Nowak
> 
> 
> 

----
You received this message because you are
subscribed to the isis-users list.
To unsubscribe, send a message to
isis-users-request_at_email.domain.hiddenwith the first line of the message as:
unsubscribe