The aim of the XMM-Newton Pipeline Processing System (PPS) is to provide a set of data products which are of immediate value for the XMM-Newton observer as well as for the XMM-Newton Science Archive (XSA), where they are also stored for eventual public release. The pipeline products, together with the Observation Data File (ODF), are distributed to observers by the XMM-Newton Science Operations Centre (SOC) via the XSA.
Until March 2012, the XMM-Newton SSC performed routine processing of all XMM pointed (and slew) observation data on behalf of ESA using a dedicated Pipeline Processing System (PPS). In early 2012 the pipeline software and configuration details at that point were transferred to the XMM-Newton XMM-Newton Science Operations Centre at ESAC. From 1st March 2012, the SOC took over responsibility for routine processing.
While the pipeline processing task was transferred to the SOC, the SSC retained responsibility for the development of the cat9.0 pipeline that was to be used for the bulk reprocessing of all XMM-Newton data in December 2012, which would form the input to the 3XMM-DR4 X-ray source catalogue. Following the bulk reprocessing, the cat9.0 pipeline was transferred to the SOC during early 2013 and became the default processing pipeline at the SOC following the release of the 3XMM-DR4 catalogue on 23 July 2013.
The processing pipeline comprises a number of perl modules that perform a range of tasks on XMM-Newton data, acting as wrappers to a number of SAS tasks that perform most of the detailed science data reduction. The pipeline software and module codes are used to determine the data flows, provide some of the filtering logic and augment products, while the entire system is managed via an SQL database. The pipeline data flow and modules are shown in this schematic of the pipeline flow, while the modules from the cat9.0 pipeline can be accessed here.
A new pipeline has been inaugurated whenever a new SAS version has been incorporated or significant pipeline structure/functionality changes were made. Each pipeline is defined by a new label, configuration and release note. The tables below provide a résumé of current and past SSC pipelines.
When important software and/or calibration changes have accumulated, all available XMM ODFs are bulk reprocessed through a stable pipeline and static calibration data to re-impose processing/calibration homogeneity and to build XMM X-ray source catalogues (e.g. 3XMM-DR4).
