• Obsolete CTI analysis

Date: 17 May 2001
From: Dan Schwartz
Subject: Draft plan for implementing CTI ground correction

The CXC has received a recommendation from the PSU/MIT ACIS IPI team to implement a CTI corrector in the ground data analysis. We are drafting the present plan to accomplish this.

This plan is complex, requiring key inputs from the ACIS/IPI, SDS, CAL, and CXC-DS groups, with support from DOSS, and from cdo to keep the users informed.

The present memo:
a). States the tasks involved for an initial CXC capability to correct for CTI effects.
b). Presents very roughly the dependencies and a possible schedule to accomplish a).
c). Introduces the issue of production of rmf, which is seen as key to maintaining the CTI correction capability through further chip damage and/or changing cosmic ray flux in the years ahead.

To be clear, only items a) and b) constitute the plan to develop an immediate CTI correction capability. Item c) affects longer range decisions about calibration fidelity and maintenance, and support for additional ACIS modes.

See the following references:

• ACIS CTI Correction Approaches - Catherine Grant, Leisa Townsley & the ACIS IPI Team (26 Apr 2001; revised 30 Apr 2001)
• ACIS CTI Correction Report, (22 Mar 2001), MIT team
• Townsley & Broos 2001, "Modeling Charge Transfer Inefficiency in ACIS"
• Broos & Townsley 2001, "Notes on RMF Generation"

Thank you,
Dan

A. Tasks

1. Write a spec for the forward-corrector (PSU) algorithm. Glenn Allen will write and iterate with the ACIS analysis and modeling group and CXC-DS. We intend to implement the recommendation to correct the (9) amplitudes so that we also make a grade correction.

2. CXC-DS codes a prototype which our ad hoc evaluation group can use. Current concept by Kenny Glotfelty is to code this in acis_process_events, but this is a design decision. One clear requirement will be to preserve the raw telemetered amplitudes, along with the corrected amplitudes. A further downstream specification and decision will concern whether this correction is implemented in pipelines.

3. Develop an rmf, gain tables, and arf to use with the CTI-corrected data.
3.1 MIT ACIS (Mark Bautz, Gregory Prigozhin, supported by Bev LaMarr and Catherine Grant) will provide binary code which CXC can run to generate events. (This is the process we used for currently on-going work on the backside chips.) The code will simulate the output of the FI chips. As a subtask, Dick Edgar and Norbert Schultz will specify the regions of chip needed to be simulated, and the degree to which events are "CTI perturbed."
3.2 The events which are output from this process need to be run through the prototype CTI correction code to generate simulated events_corrected_for_CTI.
3.3 Dick Edgar and Mike Raley will run the latter simulated events output through the standard fits and FEF generation, which also makes the gain tables. Mike Wise will package files and make rmf as needed to support the evaluation process.
3.4 The ad hoc evaluation group will fit on-orbit data from the on-board cal sources and astronomical objects, and iterate as necessary. This function will both test the code, and determine its efficacy.
3.5 We need to determine how to compute arf. An approach could be to use pre-launch arf. To untangle current discrepancies between FI and BI at low energies we probably first need better rmf to analyze the data.

4. Dale Graessle needs a CALDB architecture to allow the new calibration products for CTI-corrected data to be kept distinct from the those for raw data. (We need to support analysis of non-CTI-corrected data at least for another release or so. For one thing, the correction is not yet suitable and calibrated for all ACIS modes.)

5. When the ACIS modeling and analysis group has completed end to end evaluation, it will provide calibration products to Dale, with supporting rationale. The CXC-DS would schedule the new code and calibration products for a specific release.

Auxiliary task:
CXC cdo web pages discuss the PSU correction when it is posted to the contributed software page, and explain the upcoming release of capability in the CXC Data System.

B. Schedule

1. Glenn Allen has completed a first draft spec. He estimates no more than four weeks to complete iteration through the IPI, SDS, and DS groups.

2. Kenny will be able to generate prototype (and eventually, final) code much faster than any other steps of this process. I infer no more than 1 week turnarounds (but must be coordinated with other CXC-DS tasks). This is serial to task 1.

3. The entire task is arduous, as we know from our work since last October trying to produce S3 products.
3.1, MIT can probably provide the code within 4 to 6 weeks. This task has started and is parallel to 1. and 2.
3.2 This is a new step -- WAG of about a week.
3.3 This is the process that has been taking since last October with the S3 chips. However, we (Dick and Mike) were doing a lot of development of scripts and inventing new methodologies for fitting 10 Gaussians to create the FEF. They will need to do some detailed planning of how to divide the response generation into subarrays, which few tiles to make and test first, etc. Balancing the existence of script tools with new questions to be answered, and other summer activities, this might be planned for 3 months subsequent to receiving the output of 3.1/3.2. So end of September.
3.4 This time is iterative and largely parallel to the fitting task of item 3.3. It is integral to the process of testing the code, so I don't allocate it any serial time.
3.5 Determining arf is a research project. Using existing arf can be the de facto initial approach. I project updating and releasing arf at some time after release of the CTI correction machinery, since we probably need this machinery to analyze and resolve the on-orbit discrepancies.

4. Dale has some concepts in this area. He can start immediately and proceed in parallel. He will work continuously within the group so that there are no surprises in formats, volume, etc.

5. Aiming for October, 2001.

Obviously, the above schedule is very loose. We are working as a collaboration, understanding that we have other tasks from our group and division leaders, but recognizing that Harvey and Roger have placed a high priority on completing this task.

C. Production of rmf

Background motivation:
As is obvious in Section B., and in the on-going work of our ad hoc modeling and analysis group, by far the longest schedule driver is the process of fitting FEF to the simulated events output. Besides the fact that it is an ill-posed mathematical problem, and takes a very long time, it inevitably must be losing some fidelity. We know we are trying to fit multiple Gaussians to peaks which we know are not Gaussian shaped. Although Dick Edgar has been developing scripts which we expect to reduce the schedule by a factor of 3 relative to our current effort, fitting a single set of FEF will always require several labor months of science effort.
There will be many probable or possible changes to the ACIS response over the course of the mission. These include increasing trap density as the CTI degrades continuously or (but hopefully not) by catastrophic events, changes in the pre-filling of traps due to the variation of cosmic ray flux with solar cycle, operation at higher temperatures if or when thermal surfaces degrade and we cannot control to -120 deg C. These issues are totally independent of whether we do CTI correction or not. In addition, if we DO do CTI correction as planned, the response will in principle depend on the ACIS operating mode (although this remains to be quantified). This is because all CTI correction studies and calibrations to date are based on the 3.2 sec exposure full-frame mode. The response will vary due to different amounts of cosmic ray charge pre-filling traps if different exposure times are used. The CTI correction obviously applies only to F and VF modes, so we will always have to maintain separate products for Bright source mode. CC mode is not corrected with current algorithm, and perhaps never will be.

Recommendation:
Produce rmf directly from the simulated events by selecting the events appropriate to the user chosen spatial region, grade, energy range, and any other event selection criteria, and binning by the discrete input energies which exist and the user selected output energy or PHA ranges; Interpolating or extrapolating as necessary in the input energy space to create the exact user specified matrix.

Problem:
To achieve the perceived requirements for spatial and energy resolution, and for numbers of simulated events to give negligible statistical errors in the simulation, 50 to 100 Gigabytes worth of event information may be required.

Solutions to this exist. I offer the following "ingredients" as an "existence theorem," but software research and evaluation should proceed.

• Create standard arrays of events binned by ~50 input energies and 1000 output energy bins. These arrays are to span a range of standard selections of regions and grade selection. Distribute these arrays as standard cal products, along with software which creates rmf from them according to the interface currently used by Observers.
  
• Allow users (e.g., with special unusual needs) to request the rmf(s) which they wish the pipeline, or manual processing, to generate for their datasets.
  
• Allow users access to the raw event files on CXC machines to create any custom rmf needed.
  
• Regenerate simulated events as (an optional) part of the rmf generation process. The paradigm is that users only need 10 to 100 times more simulated events than are in their actual image, so for images with less than several 10^5 events this is certainly feasible. The code to generate the simulated events would have to be provided by the IPI team, which MIT already has done for binary versions of the backside model. Probably source code would have to be provided and engineered into the CXC data system, but this is TBD (to be discussed).