ChaRT vs. Real

Just wondering how good ChaRT is

Just exactly how good is ChaRT?

I couldn't tell how good ChaRT simulation is by looking at both ChaRT simulation and real data images. So I took up the data products that Jennifer West produced and analyzed for the fun of it. Let me set aside the fact that I need to get a life than doing this for fun.

Anyway, first load up both simulated and real datasets onto IDL and create 2-D image arrays (histogram) in SKY_X and SKY_Y coordinates. I scaled the images so that the intensity ranges from 0 to 1 in both images. Here is the blinking animation of these two frames:

So they look alike. But how does that hold up statistically? Looking alike is one thing; mathematically identical is another. What we want is the latter, of course. So I ran some KS-type analysis in the accumulated PSF profile. For simplicity, I collapse the whole image onto SKY_X or SKY_Y axes and then ran the comparison of cumulative profiles (real and ChaRT). The results are shown below:

KS Analysis on SKY_X cumulative profiles

KS Analysis on SKY_Y cumulative profiles

and the results are amazingly good. Don't be fooled by the low probability here. The KS test used here is from Numerical Recipe and not exactly the best thing there is. The main point is that the profiles look identical, which indicates that the ChaRT simulation is doing pretty well on mimicing the real dataset. There seems to be some differences in its wing part, though. This can be seen in the difference image (i.e., ChaRT - Real image): Difference Image (ChaRT - REAL)

The difference (in the normalized scale) ranges from -4.5 to 9% (-4.5% being the darkest and 9 being the brightest). The image had been smoothed with median 3x3 filter to bring out the residuals. All in all, these profiles -- ChaRT and Real-data -- are not precisely the same, but then it's hard to argue that these profiles are statistically (absolutely) different. More likely the KS statistic shows these profiles are alike. From there on, the users need to decide what the statistics really means.

This page was last updated Dec 30, 2002 by Bish K. Ishibashi. To comment on it or the material presented here, send email to bish@space.mit.edu.