The XMM-Newton Serendipitous Source Catalogue: 4XMM-DR10

User Guide to the Catalogue


Release 1.10 10th December 2020 Associated with Catalogue version 1.0

Prepared by the XMM-Newton Survey Science Centre Consortium

This User Guide refers directly to the full FITS and plain-text formats of the catalogue. Users interested in the details of changes to the data processing since the 4XMM-DR9 catalogue release, can refer directly to section 3. Information about the columns contained in the 4XMM-DR10 catalogue are presented in section 4. Brief summaries of some elements of the 4XMM-DR10 catalogue properties are provided in section 5 but a comprehensive evaluation of the catalogue is in Webb et al., (2020).

Should you use the catalogues 4XMM-DR10 or 4XMM-DR10s for your research and publish the results, please use the acknowledgement below and cite the relevant paper, either Webb et al., (2020) or Traulsen et al., (2020).

This research has made use of data obtained from the 4XMM XMM-Newton serendipitous source catalogue compiled by the 10 institutes of the XMM-Newton Survey Science Centre selected by ESA.

Contents


Summary

4XMM-DR10 is the fourth generation catalogue of serendipitous X-ray sources from the European Space Agency's (ESA) XMM-Newton observatory, and has been created by the XMM-Newton Survey Science Centre (SSC) on behalf of ESA. It is an incremental version of the the 4XMM catalogue and contains 343 more observations and 39196 more detections than the preceding 4XMM-DR9 catalogue, which was made public in December 2019. In addition, we provide spectra and lightcurves for more than 14491 more detections than in 4XMM-DR9.

The catalogue contains source detections drawn from a total of 11647 XMM-Newton EPIC observations made between 2000 February 1 and 2019 December 14; all datasets included were publicly available by 2019 December 14 but not all public observations are included in this catalogue. For net exposure time  ≥  1ksec, the net area of the catalogue fields taking account of the substantial overlaps between observations is ~ 1192 deg2.

4XMM-DR10 contains 849991 X-ray detections above the processing likelihood threshold of 6. These X-ray detections relate to 575158 unique X-ray sources. A significant fraction of sources (110446, 19%) have more than one detection in the catalogue (up to 74 repeat observations in the most extreme case).

The catalogue distinguishes between extended emission and point-like detections. Parameters of detections of extended sources are only reliable up to the maximum extent measure of 80 arcseconds. There are 80547 detections of extended emission, of which 18093 are 'clean' (in the sense that they were not flagged).

Due to intrinsic features of the instrumentation as well as some shortcomings of the source detection process, some detections are considered to be spurious or their parameters are considered to be unreliable. It is recommended to use a detection flag and an observation flag as filters to obtain what can be considered a 'clean' sample. There are 748670 out of 849991 detections that are considered to be clean (i.e., summary flag < 3).

For 303023 detections, EPIC time series and 302773 detections, EPIC lightcurves were automatically extracted during processing, and a χ2-variability test was applied to the time series. 6972 detections in the catalogue are considered variable, within the timespan of the specific observation, at a probability of 10-5 or less based on the null-hypothesis that the source is constant. Of these, 5176 have a summary flag  < 3.

The median flux (in the total photon-energy band 0.2 - 12 keV) of the catalogue detections is ~ 2.3 × 10-14 erg/cm2/s; in the soft energy band (0.2 - 2 keV) the median flux is ~ 5.3 × 10-15, and in the hard band (2 - 12 keV) it is ~ 1.2 × 10-14. About 23% have fluxes below 1 × 10-14 erg/cm2/s. The flux values from the three EPIC cameras are, overall, in agreement to ~ 10% for most energy bands. The median positional accuracy of the catalogue point source detections is generally < 1.57 arcseconds (with a standard deviation of 1.44).

With 4XMM-DR10, we also release 4XMM-DR10s, a new version of the stacked catalogue built from 7803 4XMM-DR10 overlapping observations. 4XMM-DR10s contains 1396 stacks (or groups). Most of the stacks are composed of 2 observations and the largest has 352. The catalogue contains 335812 sources, of which 2562133 have several contributing observations. Stacking observations allows yet fainter sources to be detected in sky regions observed more than once, increasing the number of detections and uncovering long-term variability on repeatedly observed objects.

1. Introduction

Pointed observations with the XMM-Newton Observatory detect significant numbers of previously unknown 'serendipitous' X-ray sources in addition to the proposed target. Combining the data from many observations thus yields a serendipitous source catalogue which, by virtue of the large field of view of XMM-Newton and its high sensitivity, represents a significant resource. The serendipitous source catalogue enhances our knowledge of the X-ray sky and has the potential for advancing our understanding of the nature of various Galactic and extragalactic source populations.

The 4XMM-DR10 catalogue is the twelth publicly released XMM-Newton X-ray source catalogue produced by the XMM-Newton Survey Science Centre (SSC) consortium. It follows the 1XMM (released in April 2003), 2XMMp (July 2006), 2XMM (August 2007), 2XMMi (August 2008), 2XMMi-DR3 (April 2010), 3XMM-DR4 (July 2013), 3XMM-DR5 (April 2015), 3XMM-DR6 (July 2016), 3XMM-DR7 (June 2017) and 3XMM-DR8 (May 2018) and 4XMM-DR9 (December 2019) catalogues: 2XMMp was a preliminary version of 2XMM. 2XMMi and 2XMMi-DR3 are incremental versions of the 2XMM catalogue.

The 4XMM-DR10 catalogue is about 5% larger than the 4XMM-DR9 catalogue. In terms of the number of X-ray sources, combining the 4XMM-DR9 and 4XMM-DR9s catalogues gives a catalogue that is similar in size to the Chandra Source Catalogue but 4XMM-DR10+4XMM-DR10s contain twice as many individual sources as the Chandra catalogue. 4XMM-DR10+4XMM-DR10s complement deeper Chandra and XMM-Newton small area surveys, probing a large sky area at the flux limit where the bulk of the objects that contribute to the X-ray background lie. The 4XMM-DR10 catalogue provides a rich resource for generating large, well-defined samples for specific studies, utilizing the fact that X-ray selection is a highly efficient (arguably the most efficient) way of selecting certain types of object, notably active galaxies (AGN), clusters of galaxies, interacting compact binaries and active stellar coronae. The large sky area covered by the serendipitous survey, or equivalently the large size of the catalogue, also means that 4XMM-DR10+4XMM-DR10s is a superb resource for exploring the variety of the X-ray source population and identifying rare source types.

The production of the 4XMM-DR10 and 4XMM-DR10s catalogues has been undertaken by the XMM-Newton SSC consortium in fulfillment of one of its major responsibilities within the XMM-Newton project. The catalogue production process has been designed to fully exploit the capabilities of the XMM-Newton EPIC cameras and to ensure the integrity and quality of the resultant catalogue through rigorous screening of the data.

4XMM-DR10 and 4XMM-DR10s are based on the pipeline configurations 18. This pipeline version contains many changes with respect to the pipeline used to make the previous major version of the catalogue, 3XMM-DR5. Changes include source spectra and light curves created for pn Timing mode and small window data, source detection on pn small window data, energy dependent Charge Transfer Inefficiencies (CTI) and double event energy corrections applied, time and pattern dependent corrections of the spectral energy resolution of pn data, X-ray loading and rate dependent energy (PHA) and CTI corrections for EPIC pn Timing and Burst modes, binning of MOS spectra changed from 15 eV to 5 eV and filtering with XMMEA_EM, which is a bit-wise selection expression, automatically removing “bad events” such as bad rows, edge effects, spoiled frames, cosmic ray events (MIPs), diagonal events, event beyond threshold, etc, instead of XMMEA_SM (which removed all flagged events except those flagged only as CLOSE_TO_DEADPIX), background regions for EPIC spectra and light curves selected from the same EPIC chip where the source is found, observations of solar system objects processed such that X-ray images and spectra correctly refer to the moving target, pileup diagnostic numbers for EPIC sources included, and footprints for EPIC observations based on combined EPIC exposure maps provided as ds9 region files. Other changes carried out specifically for the production of 4XMM include a revised systematic position error, the modelling of the EPIC background and finer binning of EPIC lightcurves. More information on these changes can be found in Webb et al., (2020).

Users of the 4XMM catalogue should be aware that the DETID and SRCID values bear no relation to those in the previous 2XMM series of catalogues. However, a cross-matching is provided in 4XMM-DR10 via the DR3DETID and DR3SRCID columns.

2. User Guide for 4XMM

The extensive User Guide (UG) for the 2XMM catalogue still describes many of the details of the data processing and compilation approach applicable to the 4XMM-DR10 catalogue. However, a significant number of changes to the processing have been implemented for 4XMM and these are described in the 4XMM-DR9 documentation and the Section 3 below. For convenience, Table 1, which gives the energy band definitions, is repeated here.

Table 1:  Energy bands used in 4XMM-DR10 processing
Basic energy bands: 1 = 0.2 -   0.5 keV  
2 = 0.5 -   1.0 keV    
3 = 1.0 -   2.0 keV    
4 = 2.0 -   4.5 keV    
5 = 4.5 - 12.0 keV    
Broad energy bands: 6 = 0.2 -   2.0 keV   soft band, no images made
7 = 2.0 - 12.0 keV   hard band, no images made
8 = 0.2 - 12.0 keV   total band
9 = 0.5 -   4.5 keV   XID band

3.   4XMM-DR10 -- key changes with respect to 4XMM-DR9

3.1 Data selection

XMM-Newton observations considered for inclusion in the 4XMM-DR10 catalogue were those with ODFs available for processing up to 2019 December 14 and all were publicly available as of 2019 December 31. After allowing for a small number of observations which failed in processing for a variety of reasons, Table 2.1 gives the list of the final 11647 observations which are included in the 4XMM-DR10 catalogue.

3.2 Naming convention for the DETID and the SRCID

Starting in 3XMM-DR5, the procedure for attributing the detection identification number (DETID) and the unique source identification number (SRCID), both being unique to each detection and each unique source respectively, has been modified. Previously, identification numbers were re-computed for each catalogue version leading to supplementary columns added to the catalogue with the DETID and SRCID from previous releases.

The DETID is now constructed from the OBS_ID, which always remains the same for an observation, coupled with the source number SRC_NUM (SRC_NUM is the source number in the individual source list for a given observation; Sources are numbered in decreasing order of count rate (i.e. the brightest source has SRC_NUM = 1)) as follow:

DETID = 1 + OBS_ID + SRC_NUM

where the '+' sign indicates string concatenation and where SRC_NUM is zero-padded to form a 4 digit number. The SRCID of a unique source is then determined from the first DETID attributed to that source (i.e. in the observation where the source was first detected) and replacing the first digit '1' by '2'.

Despite the new naming convention that aims at preserving SRCID numbers across catalogue versions, a certain number of SRCID can disappear from one catalogue version to another. This is a normal consequence of the algorithm that groups detections together into unique sources (see section 6 of Rosen et al. 2016). When new data are added and statistics are improved, the algorithm might find a better association of detections into unique sources. As an example, a total of 134 SRCIDs listed in 3XMM-DR7 are absent in 3XMM-DR8.

3.3 Systematic position error

The astrometry of the X-ray detections is improved by using the catcorr task to cross-correlate the X-ray detections with the USNO B1.0, 2MASS or SDSS (DR8) optical/IR catalogues. However, where catcorr fails to obtain a statistically reliable result (poscorrok=false), a systematic error of 1.5′′ was used to create the 3XMM catalogue. To check this value, we cross-matched SDSS quasars with the detection catalogue where poscorrok=false, out to r=30 arcsecs, filtering the latter with SUM_FLAG=0 and EP_EXTENT=0, to keep only the cleanest sample of secure point-like X-ray sources. For more information about what was done, see Webb et al. submitted. We define the combined positional error as σ=(Δ S2+Δ X2/2)0.5, where Delta; X = POSERR and Δ S is the radially-averaged uncertainty in the SDSS positions to which we had already added a systematic 0.1'' in quadrature, and x = r / σ. Our final filtering retained only the 157 QSO-X-ray pairs with x<5.

The expected probability density distribution of x should follow the Rayleigh distribution P(x) = x e-x2. Since this was not the case for the 157 pairs of sources found above, we added an additional positional uncertainty, σ, in quadrature, so that the total positional uncertainty is now σ = (σ2 + Σ2)0.5, looking for the value of Σ that minimizes the difference between the distribution of the x' = r/σ' and the Rayleigh distribution. For 4XMM-DR10 we found Σ = 1.90 +/- 0.01 arcsec, where the uncertainty (1 σ) has been calculated by bootstrap with replacement. This value was then used to replace the 1.5 arcsec systematic error when poscorrok=false.

3.4 Pile up information

As of 4XMM we provide three new columns (PN_PILEUP, M1_PILEUP and M2_PILEUP) quantifying whether each detection may be affected by pile-up in any instrument. A value below 1 corresponds to negligible pile-up (less than a few % flux loss) while values larger than 10 denote heavy pile-up. Pile-up is dependent on time for variable sources. We neglect that here, but note that a variable source is more piled-up than a constant one for the same average count rate, so our pile-up level can be viewed as a lower limit. We also neglect the slight dependence on the source spectrum due to the event grade dependence of pile-up.

Our pile-up levels are not based on a fit of the full images using a pile-up model (Ballet 1999). For point sources, they are based on the measured count rates reported in the catalogue over the full energy band, transformed into counts per frame. The thresholds (at which the pile-up level is set to 1) are set to 1.3 cts/frame for MOS and 0.15 cts/frame for PN.

For extended sources, this method gives inconsistent results. Therefore, for 4XMM-DR10 we set all values of the pile-up for extended sources to the NULL value.

3.5 Hot areas in the detector plane

Warm pixels on a CCD (at a few counts per exposure) are too faint to be detected as such by the automatic processing, but can either push faint sources above detection level, or create spurious sources when combined with statistical fluctuations. This is an intrinsically random process, not visible over a short period of time, but which creates hot areas when projecting all sources detected over 18 years onto the detector plane.

We addressed this by projecting all sources onto CCD coordinates PN/M1/M2_RAWX/Y, keeping only sources above the detection threshold with the current instrument alone. In that way, we can distinguish hot areas coming from different instruments. We proceeded to detect hot pixels or columns in each CCD, using a similar method to the SAS task embadpixfind. For more information see Webb et al. submitted. Many of warm pixels were not present at the beginning of the mission, and some appear for a short amount of time. So we tested each hot area for variability using revolution number, and the same KS-based algorithm used to detect segments of bright columns, compared to the reference established over all sources on all CCDs and all instruments. This resulted in a revolution interval for each hot area.

Sources on a hot area for a particular instrument and within the corresponding revolution interval are flagged with flag 12. The updated table for the 12 flags is given below.


Table 3.1:  Flag Keys
1   Low detector coverage ca_MASKFRAC <  0.5
2   Near other source R ≤ 65 * SQRT (EP_RATE); R(min) = 10", R(max) = 400"
3   Within extended emission R ≤ 3 * EP_EXTENT; R(max) = 200"
4   Possible spurious extended source near bright source Flag 2 is set and EP_CTS(min) = 1000 for the causing source
5   Possible spurious extended source within extended emission R ≤ 160" and fraction of rate wrt causing source is 0.4
6   Possible spurious extended source due to unusal large single-band DET_ML Fraction of ca_b_DET_ML wrt the sum of all ≥ 0.9
7   Possible spurious extended source At least one of the flags 4, 5, 6 is set
8   On bright MOS-1 corner or bright low gain PN column
9   Near bright MOS-1 corner R ≤ CUTRAD = 60" of a bright pixel the corner
10   Detection whose centre lies on any masked column or row due OoT and RGA features.
11   Within region where spurious detections occur Manual flag
12   Detection on hot area

The default value of every flag is F for False. When a flag was set it means it has been changed to T for True.

The task dpssflag sets all flags except the camera-specific flags (i.e., flags 2,3,4,5,6,7) on the summary row (EPIC band 8) which are then propagated backwards to the individual cameras and bands. Flag 12 is not propagated into the SUM_FLAG in 4XMM.

4. Catalogue content and organisation

This section summarises the organisation of the catalogue and gives details of all the columns. Known problems with parameters presented in the catalogue or with products associated with it are listed in Sec. 6.

There are 336 columns in the catalogue; they are grouped together and explained in the links below.

For each observation there are up to three cameras with one or more exposures which were merged when the filter and submodes were the same (2XMM UG, Sec. 2.2). The data in each exposure are accumulated in several distinct energy bands (Table 1). Camera-level measurements can further be combined into observation-level parameters. Consequently, the source parameters can refer to some or all of these levels: on the observation level there are the final mean parameters of the source (prefix 'EP'); on the camera level the data for each of the three cameras (where available) are given (prefix 'PN', 'M1', or 'M2'), and on the energy band level the energy-dependent details of the source parameters are given (indicated by a 'b' in the column name where b = 1,2,3,4,5,8,9). Finally, on a meta-level, some parameters of sources that were detected more than once (prefix 'SC') were combined, see 2XMM UG, Sec. 3.2.4.

The column name is given in capital letters, the FITS data format in brackets and the unit in square brackets. If the column originates from a SAS task, the name of the task is given to the right hand side and a link is set to the SAS package documentation with which the data in the 4XMM-DR9 catalogue was processed. It should be pointed out that the SAS used for the bulk reprocessing (for 4XMM) was from manifest pipeline version 18, which is based on SAS 18. A description of the column and possible cross-references follow.

Entries with NULL are given when no detection was made with the respective camera, that is, ca_MASKFRAC < 0.15 or NULL (i.e., a camera was not used in an observation).

Details of the columns:

Part 1: 14 columns: Identification of the source
This includes the basic static identifiers, IAU name, together with cross-references to the (spatially) nearest detection and source ID in the previous 3XMM-DR4 and 2XMMi-DR3 catalogues, where relevant, including information about the spatial displacements and the number of 'nearby' matches. Five columns (introduced in 3XMM-DR7) are used to provide this information.
Part 2: 11 columns: Details of the observation and exposures
Part 3: 20 columns: Coordinates
The external equatorial and Galactic coordinates and the internal equatorial coordinates as derived from the SAS tasks catcorr and emldetect are given together with the error estimates. Two columns convey information about the absolute astrometric catalogue used for field rectification and whether the field was successfully rectified
Part 4: 228 columns: Source parameters
The parameters of the source detection as derived from the SAS tasks emldetect and srcmatch are given here.
Part 5: 8 columns: Detection flags
This part lists the flags to qualify the detections. The summary flag, which gives an overall assessment for the detection, is followed by particular flags for each camera. A flag each is given if there exists at least one time series or one spectrum for this source. One column is added at the end of the catalogue to provide information about detections arising in fields with high background levels
Part 6: 13 columns: Detection variability
This part gives variability information for those detections for which time series were extracted. This includes six columns which provide measures of the fractional variance of the timeseries
Part 7: 42 columns: Unique source parameters
This part lists the source parameters for the unique sources across all observations (using the prefix 'SC'); these are coordinates, fluxes, hardness ratios, likelihoods, extent information and a variability and a summary flag. The number of detections is given also. Of six columns introduced for 3XMM, two of these relate to a fractional variance measure (and error). The other four provide information about the maximum and minimum measured fluxes from constituent detections (and errors). Two further columns relating to the epochs of the first and last observations contributing to a unique source, replace two columns in 2XMMi-DR3

Columns in the slimline catalogue:

Table 6 lists the 45 columns in the 4XMM-DR10 'slimline' version of the catalogue, all of which are explained in Part 1 or Part 7 of the 4XMM-DR10 column description, except the WEBPAGE_URL column which is described at the end of the table.

5. Catalogue Properties

This section summarises the main properties of the catalogue but does not provide a detailed analysis. A comprehensive evaluation of the catalogue is presented in the 4XMM catalogue papers Webb et al., (2020) or Traulsen et al., (2020).

5.1 Overview

The catalogue contains source detections drawn from 11647 XMM-Newton EPIC observations made between 2000 February 3 and 2019 December 14 and which were publicly available by 2019 December 31. Net exposure times in these observations range from < 1000 up to ~130000 seconds (that is, a full orbit of the satellite). Figure 5.1 shows the distribution of fields on the sky.

The sky area of the catalogue observations corrected for field overlaps with effective exposure > 1 ks ~1192 deg2.

The catalogue contains 849991 X-ray detections with total-band (0.2 -12 keV) likelihood values  ≥  6. These are detections of 575158 unique X-ray sources, that is, 110446 X-ray sources have multiple detections in separate observations (up to 74 detections). Of the 849991 X-ray detections, 80547 are classified as extended with 18093 of these being in regions considered to be 'clean' (SUM_FLAG  < 1).

5.2 Data quality

As part of extensive quality evaluation for the catalogue, each field has been visually screened. Regions where there were obvious deficiencies with the automatic source detection and parameterization process were identified and all detections within those regions were flagged (cf. 2XMM UG, Sec. 3.2.6 but importantly, note Section 3.11). Such flagged detections include clearly spurious detections as well as detections where the source parameters may be unreliable. Each XMM-Newton field is also evaluated to assess the fractional area of the observation that is affected by flagged detections, as reflected by the OBS_CLASS parameter. For most uses of the catalogue it is recommended to use a detection flag (SUM_FLAG, EP_FLAG or SC_SUM_FLAG) and an observation flag (OBS_CLASS) as a filter to obtain what can be considered a 'clean' sample.

Note that no attempt is made to flag spurious detections arising from statistical fluctuations in the background. An updated analysis of the false detection rate are presented in the 4XMM catalogue paper Webb et al., (2020).

5.3 Sensitivity and Photometry

Figure 5.4 presents, for each of the three cameras, the distributions of flux for energy bands 1 to 5 and also for the combined (EPIC) data. These give an indication of the limiting flux available in the catalogues for each of the bands.

5.4 Astrometry

Comparing the astrometry between 3 and 4XMM shows very similar results. A more detailed analysis of these issues are presented in the 4XMM catalogue paper Webb et al., (2020).

6. Known problems and other issues

Please read the Watchouts section of the 4XMM-DR10 documentation for the latest information on 4XMM-DR10 catalogue issues.

References:

Budavari, T & Szalay, A 2009, Ap.J, 679, 301-309 Probabilistic Cross-Identification of Astronomical Sources

Edelson, R., et al. 2002, Ap.J, 568, 610-626 X-Ray Spectral Variability and Rapid Variability of the Soft X-Ray Spectrum Seyfert 1 Galaxies Arakelian 564 and Ton S180

Mateos, S., et al. 2009, A&A, 496, 879-889 Statistical evaluation of the flux cross-calibration of the XMM-Newton EPIC cameras

Pye, J., et al. 1995, MNRAS, 274, 1165-1193, The ROSAT Wide Field Camera all-sky survey of extreme-ultraviolet sources - II. The 2RE Source Catalogue

Read, A., et al. 2011, A&A, 534, A34 A new comprehensive 2D model of the point spread functions of the XMM-Newton EPIC telescopes: spurious source suppression and improved positional accuracy

Rosen, S.R., Webb, N.A., Watson, M.G., et al. 2016, A&A, 590, 1 The XMM-Newton serendipitous survey. VII. The third XMM-Newton serendipitous source catalogue

Vaughan, S., et al. 2003, MNRAS, 345, 1271-1284 On characterizing the variability properties of X-ray light curves from active galaxies

Watson, M., et al. 2009, A&A, 493, 339-373 The XMM-Newton serendipitous survey. V. The Second XMM-Newton Serendipitous Source Catalogue

Document revision history

Release No. Release Date      Comments
1.10 10 December 2020 First release for 4XMM-DR10
1.0 18 December 2019 First release for 4XMM

Appendices

A.1 List of the observations used in the catalogue

List of observations ('fields').

A.2 Catalogue pipeline processing details

4XMM-DR10 was reduced with pipeline : version 18, with the Current Calibration Files (CCFs) of February 2019 pipeline release notes.