CDA - Reprocessing IV
The Archive
- Public Data
- Proprietary Data
- Repro III
- Repro IV
- Repro V
- Privacy Impact Assessment
- Release Notes
- Documentation
- Archive Operations Team
Search and Retrieve Data
Advanced Data Services
Archive User Services
- Synopsis of Services
- Processing Status
- Special Requests
- Dataset Identifiers (DOIs)
- Contributed Datasets
- CXO Papers Repository
Chandra Aggregated Datasets
Publishing Chandra Results
Data Analysis Links
- The archive is fully functional.
-
Reprocessing IV has been completed in December 2014.
- ASCDSVER: 10.13
- CALDBVER: 4.11.5
- Electrical Outage 11/05/21 - 11/06/21
2021-11-05
There will be a planned electrical outage Friday, November 5 2021, 5:30PM - Saturday, November 6 2021, 6:00PM EDT. During this time period ChaSeR and other archive interfaces will be unavailable. - Electrical Outage on 07/28/21
2021-07-26
There will be a planned electrical outage Wednesday, July 28 2021, 8:00PM - 10:00PM EDT. During this time period ChaSeR and other archive interfaces will be unavailable. - Downtime on 6/12/21
2021-06-10
Due to electrical work, the public interfaces of the Chandra Data Archive will not be available on Saturday 6/12 from 4 AM to 9 PM. - Repro V Begins
2020-10-26
The fifth major reprocessing of the archive (Repro V) has started! Details can be found here. - Past Notices
Scope
The fourth reprocessing campaign of the CXC will Include all observations taken after 2000 Jan 29, up to current data. We have no plans to reprocess the first 6 months of the mission, which was the bulk of manual efforts in Repro III. Neither algorithm changes in pipeline tools or calibration upgrades will significantly affect this time-span. On the other hand, there are almost 13,000 ObsIds (see Table) in the queue for reprocessing. The time estimates below reflect the increase in ObsIds to process.
Number of Observations per Repro IV phase
Instrument | Phase 1 | Phase 2 |
---|---|---|
15 Nov 2005 - 31 Dec 2011 | 29 Jan 2000 - 15 Nov 2005 | |
ACIS-S | 3057 | 2858 |
ACIS-I | 1887 | 1340 |
HRC-I | 383 | 387 |
HRC-S | 368 | 471 |
CTI | 557 | 1548 |
TOTAL | 6262 | 6604 |
We plan to process the data in 2 phases.
- Phase 1:Observations starting at Nov. 15, 2005.
These ObsIds have not been processed with Repro III, so we will begin with level 0 processing. This early step will allow us to apply the interpolated spacecraft clock correlation files to the times.
This phase will be executed in two parts: first all of 2011, since it contains data that are still proprietary (Phase 1a), then all observations made between 15 November 2005 and 1 January 2011 (Phase 1b).
- Phase 2:Observations between Jan. 29, 2000 and Nov. 15, 2005
These ObsIds were processed with Repro III and have interpolated spacecraft clock correlation corrections in the archive level 0 products. The processing for these ObsIds will start with the obidet pipeline in order to populate new header keywords, and then continue with aspect level 1 to apply the new PCAD periscope alignment correction and run to completion. A list of science upgrades (highlighted below) includes applying the new ACIS sub-pixel algorithm in level 1.
Phase 1a started on 31 January 2012.
Highlights
- ACIS: Apply the ACIS subpixel resolution algorithm
- ACIS: Improve ACIS cosmic-ray afterglow removal
- ACIS: Correct for serial CTI on the BI CCDs
- ACIS: Add CTI correction header metadata
- HRC: Correct for HRC time-dependent gain
- ACIS: Apply CTI correction for ACIS graded mode data
- ACIS: Apply temperature-dependent CTI correction
- ACIS: Update ACIS time dependent gain
- ACIS: Flag bright bias events
- ACIS: Improve ACIS event file metadata
- HRC: Correct HRC DTF
- ASPECT: Apply PCAD periscope alignment correction
- GENERAL: Update data products to FITS 3.0 compliance
Details
ACIS
- Apply the ACIS subpixel resolution algorithm
Repro IV will apply the ACIS subpixel resolution algorithm to ACIS imaging data to achieve the best spatial resolution. Quantitatively, the size (FHWM) of a bright (but not piled-up), on-axis point source can be reduced by about 20-30%. The new algorithm does not introduce aliasing artifacts for dithered observations, and does not worsen the positional accuracy. For more details, see https://cxc.harvard.edu/ciao/why/acissubpix.html.
- Improve ACIS cosmic-ray afterglow removal
Repro IV will use the new tool acis_find_afterglow (which replaces acis_run_hotpixel) for ACIS hot pixel and cosmic ray afterglow removal. The new tool enhances the detection efficiency for afterglows by using the events in a short, sliding time window instead of using events from the entire observation. The previous tool was effective in finding cosmic-ray afterglows with 8 events or more, but was known to leave residual artifacts, particular for ACIS-I data. The new afterglow tool is effective in finding afterglows with 4 events or more. For more details, see the technical memo by G. Allen in https://space.mit.edu/CXC/docs/docs.html#aft.
- Correct for serial CTI on the BI CCDs
Repro IV will correct for ACIS serial CTI on the BI CCDs. The serial CTI correction affects all of the energy related columns (PHA, PI, ENERGY) and changes number of valid events. The maximum change in pulse height, as a fraction of the total pulse height, is ~ 5-14% on ACIS-S1 and ~ 2-6% on ACIS-S3, depending on the energy (the quoted values are for 5.0+/-0.5 keV, respectively).
The amount of the change depends on the location of the source on the detector. However, for ACIS-S3, the aim point is close to the location where serial CTI is worst. For more information, see CIAO pages.
- Add CTI correction header metadata
Repro IV will be the first processing of all ACIS data that includes the CTI_APP header keyword that encodes the set of CTI corrections applied to the data (N = none, S = serial only, P = parallel only, and B = both serial and parallel). This update was first applied part way through Repro III.
Roughly 4500 ObsIds from Repro III do not include this keyword, so the current state of the archive is mixed. Datasets without this keyword cannot use automatic CalDB selection, requiring the user to identify and specify the necessary calibration files by hand in order to compute response files such as RMFs.
- Apply CTI correction for ACIS graded mode data
Repro IV will be the first processing of that applies parallel CTI corrections to ACIS graded mode data. For more details, see memo
- Apply temperature-dependent CTI correction
Repro IV will be the first processing of ACIS data that applies temperature-dependent CTI corrections to ACIS data. The temperature-dependent CTI correction makes ~ 0.2-0.7% of pulse height change. For more details, see CIAO pages.
- Update ACIS time dependent gain
Repro IV will update the time dependent gain calibrations for all ACIS observations since 2005 Nov 15 by interpolating between the previous and subsequent epoch calibration data (during standard data processing, calibrations for the epoch following the observation are not available and so the time dependent gain correction is applied by extrapolating the prior calibration data). The final PHA corrections are usually < 2% different from the preliminary value.
- Flag bright bias events
Repro IV will be the first processing of all ACIS data that correctly tags events associated with bias image pixels that have bright bias values as bad status. Bright bias values occur when bias telemetry is lost and cannot be repaired on the ground.
- Improve ACIS event file metadata
For Repro IV, we have added a set of keywords copied from the ACIS level 0 parameter block file (pbk0) into the event header. These values are needed by CIAO users to perform the dead area correction algorithm. By providing these values in the event header, users will no longer need the pbk0 data product when doing data analysis (being a level 0 file, the pbk0 file name is time-based rather than ObsId/ObI based, making it non-trivial for users to match to the correct ObsId). Future tool updates will make use of the new header keywords. This will make the users' lives easier, the data more self-contained, and ultimately mean less code to maintain.
HRC
- Correct for HRC time-dependent gain
Repro IV will be the first processing of all HRC data that uses the new algorithms to compute a HRC time-dependent gain maps for both HRC-I and HRC-S. Updates to the gain correction files in the CalDB are expected approximately annually in the future, depending on the behavior of the instrument gain. For further technical details, see the memo by Posson-Brown and Kashyap in memo.
- Correct HRC DTF
Repro IV will be the first processing of all HRC data that uses a revised algorithm to compute a HRC dead-time factor. The previous algorithm did not work robustly if high count-rates resulted in telemetry saturation.
ASPECT
- Apply PCAD periscope alignment correction
Repro IV will be the first processing that applies the PCAD periscope alignment correction. This improves the image reconstruction accuracy (i.e., reduces aspect blurring) by correcting for thermal alignment changes in the fiducial transfer system periscope related to overall spacecraft heating. There are roughly 400 ObsIds with periscope drift corrections > 0.1" (~ 0.1-0.4"). For more details, see Aspect pages.
GENERAL
- Update data products to FITS 3.0 compliance
Repro IV will be the first processing that generates data products compliant with the FITS 3.0 standard. Compatibility with the standard ensures that users will continue to be able to manipulate Chandra data with their favorite analysis software in the future.