Welcome to Max Tegmark's CMB data analysis center

This site contains links to cosmic microwave background resources across the globe, focusing on data analysis, and is up-to-date through about year 1999. Click on any part of the data analysis pipeline below for details. For theoretical aspects, see the twin site, Wayne Hu's theory center. See also the experiment overview page.

THE DATA ANALYSIS PIPELINE

The figure above schematically illustrates the many steps involved in producing future high-precision constraints on cosmological models. If non-Gaussian CMB fluctuations are present (as in say a topological defect model), then additional processing of the CMB map will be desirable, since the phases will contain information in a addition to that in the power spectrum. The various steps are discussed in more detail below.

The microwave sky

At microwave frequencies, the sky contains a gold mine of information about the early Universe, since the cosmic microwave background (CMB) fluctuations depend sensitively most cosmological parameters - click here for an online review. Unfortunately, this cosmic background signal is contaminated by a various types of foreground emission - a recent review is given in Section 4 of Tegmark & Efstathiou (1996)

Measurement (experimental design issues)

Click here for a summary of all CMB experiments today, with links. When planning and designing experiments with a finite budget, one obviously wants to

Maximize the ability to discriminate between different cosmological models
Maximize the ability to subtract foregrounds
Minimize the susceptibility to systematic errors

Sky coverage: To a good approximation, incomplete sky coverage destroys information about the power spectrum coefficients in two ways:

It increases the sample variance (by a factor which is roughly the inverse square root of the area covered - see Scott etal 1993.)
It smears out features in the power spectrum (of width Delta-l less than the inverse of the smallest angular dimension in radians - see Tegmark 1996).

This gives the following two rules of thumb (both derived and discussed in detail in Tegmark 1997) when deciding what region to cover:

Choose the area such that the signal-to-noise ratio per resolution element (pixel) is of order unity (or a few times greater, since S/N>1 helps checking for systematics).
Avoid "skinny" regions narrower than a few degrees in the smallest direction (since a CDM spectrum has information on scales of order Delta-l = 30).

Pointing strategy: As explained in Wright (1996), one should try to make the scanning strategy as ``complicated'' and ``connected'' as the hardware allows, since this enhances the ability to detect and remove systematic problems and tends to make the noise covariance matrix in the final map more diagonal. Other recent assessments of scanning and chopping strategies are Knox (1996) and Magueijo & Hobson (1996).
1/f-noise: Although obviously undesirable, this is not necessarily a show stopper. As shown in Tegmark (1997), 1/f-noise in the time domain gets converted into something with an angular power spectrum slope between 1/l and white noise as long as the scanning strategy is well connected. Since this rises slower than the CMB spectrum (roughly 1/l^2) towards large scales, you're OK on all scales if the formula of Jansen et al 1996 shows that you're OK on the pixel scale.
Frequency coverage: "The more, the merrier", needless to say. To decide where in a given range to place a given number of channels, one can select a foreground model and a removal method (as in say Tegmark & Efstathiou 1996) and numerically examine which channel placement minimizes the sum of the residual foreground contamination (after subtraction) and the noise. Such analyses are given by Brandt etal (1994) and Dodelson (1996).
Polarization sensitivity: Although the polarization fraction is less than 10% in standard models, measuring it can help break certain degeneracies, especially regarding the reionization redshift, gravity waves and foregrounds. Should one sacrifice overall sensitivity to measure polarization? Zaldarriaga etal (1997) discuss the tradeoffs.

The raw data

Before the raw measured data set can be used to do science, it must of course be carefully calibrated, cleaned and checked for systematic problems, as described below.

Calibration and cleaning

Although perhaps the most unglamorous step in the pipeline, this is often the most time-consuming step as well. For a rather lengthy list of potential problems to worry about, see Tom Herbig's Little Shop of Horrors. An instructive case study (for COBE) is Kogut etal (1996), and ways of facilitating systematic error detection and removal by means of a good pointing strategy are discussed by Wright (1996).

Time-ordered data

The result of this gruesome process is the time-ordered data set (TOD). For future satellite missions, it might be contain as many as 10,000,000,000 numbers. For a total-power measurement, these numbers are simply the positions and temperatures of all the pixels observed, in chronological order, in each channel. For single-difference experiments (like COBE and MAP), the TOD consists of pairs of pixel positions and the temperature difference. For more general chopping schemes, each temperature in the TOD is some linear combination of the temperature across the sky. The cosmological parameters can be measured with the smallest error bars possible by performing a brute force likelihood analysis on the TOD - in principle. In practice, this is numerically unfeasible for large data sets. Analysis of large future data sets will therefore necessarily involve the intermediate step of reducing the TOD to maps.

Mapmaking

Map-making offers a convenient way to distill the cosmological information from the TOD into a much smaller data set. In addition, maps are of course useful for comparing different experiments with one another, for subtracting foregrounds, and to look for spatial features in the data (e.g., systematic problems and non-gaussian signals). By linearity, the TOD data vector y can be written y=Ax+n, where the vector x contains the temperatures in each of the pixels in the map, the vector n denotes the noise and A is some known matrix determined by the pointing strategy. In the past, at least four different map-making methods have been employed for estimating the map x from the TOD y. Ten map-making methods are compared by Tegmark (1997), and it is found that several of them (the COBE method and various variants of Wiener filtering) have the nice property that they retain all the cosmological information from the TOD. This means that the parameters can be measured just as accurately from the map(s) as from the full TOD. The conclusion of this paper is that the TOD should be reduced to maps using the COBE method, since

this has been shown to be numerically feasible (Wright 1996).
other maps (e.g. Wiener-filtered ones) can be computed directly from the COBE method maps by a numerically stable procedure.

Multi-frequency maps

The result of the map-making step is a number of sky maps at different frequencies (those from different channels at the same frequency are of course combined after appropriate systematics checks of the difference maps). For the subsequent analysis, it is often desirable to include maps at additional frequencies from other experiments as well - both from other CMB experiments and from "foreground experiments" such as e.g. DIRBE and IRAS. Maps in corresponding to the "electric" and "magnetic" parts of the the polarization field (Kamionkowski et al 1996, Zaldarriaga et al 1996) are likely to be useful as well.

Foreground removal

To remove contaminating foreground signals, we can can take advantage of all ways in which they differ from the CMB signal:

Their frequency dependence can be used to detect and subtract them.
When they are highly non-Gaussian, we can use this to detect and remove them (e.g., interpreting 5-sigma outliers as point sources and discarding these pixels, which is more effective after band-pass filtering the map).
When they are localized in real space (like the galactic plane, say), we can discard or downweight the polluted regions.
When they are localized in Fourier space, we can downweight the most contaminated modes. Example: for low multipoles, one can downweight high (dust-contaminated) frequency channels, whereas for high l, one can downweight low (radio source contaminated) frequency channels.

A detailed discussion of all these issues is given in Tegmark & Efstathiou (1996), and detailed simulations can be found in Brandt etal (1994). The bottom line is that for future multichannel experiments like MAP and PLANCK (ne COBRAS/SAMBA), an easy-to-implement subtraction scheme can eliminate all foregrounds to a an accuracy of much better than a percent if current foreground estimates are OK . Unfortunately, this is still a big if, as we lack e.g. accurate point source counts between 20 and 200 GHz.

CMB sky map

The result of the foreground removal is the merging of all channels into a single map. As of January 20, 1997, two-dimensional CMB maps have been published for these experiments:

Power spectrum estimation

Why do it?

If the statistical properties of the CMB fluctuations are isotropic and Gaussian (which they are in the standard inflationary models), then all the cosmological information in a sky map is contained in its power spectrum C_l (the variance of its spherical harmonic coefficients, corrected for beam smearing). This means that all the information from even a giant data set (say a map with n=10^7 pixels) can be reduced to just a couple of thousand numbers, greatly facilitating parameter estimation (the next step in the pipeline). Indeed, for future experiments, it has been argued that this data compression step will be necessary to make parameter estimation numerically feasible, just as the TOD must be compressed (via mapmaking) to make power spectrum estimation numerically feasible.

Why not compute some other statistic instead?

The power spectrum has emerged as the standard way to present experimental results in the literature since it has several advantages over say the correlation function:

It allows a model-independent comparison between different experiments (whereas say the correlation function depends on angular resolution).
Since the Boltzmann equation that governs anisotropy formation (click here for an online review) is diagonal in the Fourier (multipole) domain, not in real space, there is a one-to-one correspondence between visible features in the power spectrum and the physical processes one is studying.
Estimating the correlation function and it's covariance matrix is NOT faster than extimating the power spectrum, so the former has no practical advantage.

How do it?

There are two reasons for why a straightforward expansion in spherical harmonics is not the best way to measure the power spectrum:

One always has incomplete sky coverage (even with satellites, the Galactic plane must be discarded).
One wishes to give less weight to noisier pixels (that have been observed fewer times) in order not to destroy information.

Both of these facts spoil the orthogonality of the spherical harmonics. Any quadratic combination of pixels will, appropriately normalized, measure some weighted average of the power spectrum - the weights are known as the window function. The non-orthogonality simply means that it is impossible to obtain an ideal (Kronecker delta) window function. Instead, the best you can do is (Tegmark 1995) to get a Window function whose width is about the inverse of the smallest angular map dimension in radians, which is usually adequate for all practical purposes adequate. There is a simple power spectrum estimation method (Tegmark 1996) that has the following nice properties:

It is unbeatable in the sense that no other method can measure C_l with smaller error bars, and all the cosmological information from the map is retained.
It is quadratic, which makes the statistical properties of the measurements easy to compute and use for estimation of cosmological parameters.
It is computationally faster than rival high-precision methods such as the nonlinear maximum-likelihood technique, with the crucial steps scaling as n^2 rather than n^3, where n is the number of map pixels. This makes it computationally feasible even for future high-resolution CMB experiments with millions of pixels.
It is applicable to any survey geometry whatsoever, with arbitrary regions masked out and arbitrary noise behavior.
It is not a "black-box" method, but quite simple to understand intuitively: it corresponds to a high-pass filtering and edge softening of the original map followed by a straight expansion in truncated spherical-harmonics.

Angular power spectrum

Once computed from the data, the power spectrum can be used to constrain cosmological models.

How compute it theoretically?

Wayne Hu's online review gives an intuitive physical explanation of the physical origin of the features of the power spectrum. Another must-see are the Berkeley movies, showing how the power spectrum changes as you alter the parameters. For purely pragmatic purposes, the bottow line is revealed by the figure above:

The power spectrum is a complicated-looking function.
This is because it depends on virtually all cosmological parameters.
Therefore we can use an observed power spectrum to measure virtually all cosmological parameters.

How compute it in practice?

By using "the fastest Boltzmann code in the west", written by Seljak & Zaldarriaga (1996). You can download it from the CMBFAST web site. It computes a typical spectrum in a minute or two - about 100 times faster than previous codes.

Model testing & parameter estimation

For future high-precision CMB experiments, parameter estimation with a simple chi-squared model fit to the observed power spectrum will give virtually the smallest error bars possible (section 5.4 in Tegmark 1996). For smaller data sets which produce weaker constraints, more accurate results can be obtained from a brute-force likelihood analysis of the sky map, as was done for COBE (Tegmark & Bunn 1995, Hinshaw et al 1996). The big caveat to all of this is that in a non-Gaussian model (such as cosmic strings or textures), the power spectrum is only part of the story, and additional information can be extracted from the phases of the map. The question of how to proceed in such a case is still wide open - for some early work on the subject, see Kogut etal (1996) and Ferreira & Magueijo (1996).

Estimates of cosmological parameters

If a model similar to standard Cold Dark Matter (CDM) turns out to be correct, future CMB missions should be able to measure key cosmological parameters to an accuracy of a few percent or better ( Jungman etal 1996; Bond, Efstathious & Tegmark 1997; Zaldarriaga, Seljak & Spergel 1997). It is important to remember that although certain projects (such as trying to determine the shape of the inflaton potential by measuring the spectral index n, the tensor spectral index and the tensor-to-scalar ratio) require previously untested assumptions about untested high-energy physics, there are many other parameters that can be measured in a robust fashion by assuming little else than that we understand gravity and the behavior of hydrogen and photons at a few thousand degrees ( Hu & White 1996). For instance, the spacing between the power spectrum peaks provides a fairly clean probe of the angle-distance relationship (which fixes a certain combination of Omega and Lambda). Whatever the true power spectrum turns out to look like, it is likely to help us clean up among the profusion of cosmological models that are currently on the market.