CIAO 4.0 Release Notes

---

(8 Sep 2008) CALDB has been patched to version 3.5.0. The How CALDB 3.5.0 Affects Your Analysis section has information on the release.

(16 Apr 2008) CIAO has been patched to CIAO 4.0.2 on Mac OS X Intel only. This patch enables CIAO to run on Mac OS X 10.5 (Leopard) Intel; it is optional for 10.4 (Tiger) Intel. The patch should not be applied to Linux, Solaris, or Mac PowerPC installations of CIAO.

CIAO 4.0.2 Release Notes

---

CIAO 4.0 is a full installation of the CIAO software. It contains all the enhancements and bug fixes since CIAO 3.4, including those which were made available in the 4.0 Beta releases.

The most notable change to the CIAO tools in version 4.0 is that the ACIS dead area correction is turned on by default; refer to the How CIAO 4.0 and CALDB 3.4.2 Affect Your Analysis section for details. There is a new version of tgextract - named tgextract2 - which provides similar functionality, but allows large and asymmetric regions for better determination of background counts. Significant work has been done on the tool wavdetect, which has two new parameters. The sso_freeze tool has been enhanced so that it can create object-centered aspect solutions which allows an object-centered exposure map to be generated. A number of bug fixes to the Data Model library and several CIAO tools are also included.

The Data Model library now supports an ASCII kernel. This kernel allows users to generate ASCII format files for use within Data Model enabled tools (e.g. dmlist, dmcopy), and apply the suite of Data Model capabilities to those files via its extended filename syntax. The ASCII kernel is considered beta for this release.

The CIAO 4.0 release includes a new, more powerful version of ChIPS, the CIAO plotting package. It can be used during data analysis - e.g. to plot a lightcurve - and to create publication-quality figures. Details and documentation on the new system is available from the ChIPS website.

This release also features an experimental (beta) version of the new Sherpa, the CIAO modeling and fitting package. Details and documentation on the new system are available from the Sherpa website.

CIAO 4.0 is fully GPL compliant. OTS (off-the-shelf) software have been updated to current versions. A set of "smoke test" scripts are included to aid users in verifying installation of the release.

---

• Python and IPython Caveats
• How CALDB 3.5.0 Affects Your Analysis
• How CALDB 3.4.5 Affects Your Analysis
• How CALDB 3.4.3 Affects Your Analysis
• How CIAO 4.0 and CALDB 3.4.2 Affect Your Analysis
• Tools
• Parameter Files
• Data Model
• User Interface
• Libraries
• S-Lang
• Environment Modifications
• Analysis Scripts
• Documentation

---

Python and IPython Caveats

Python

• CIAO 4.0 uses the Python software packaged in the CIAO OTS directory for the Sherpa and ChIPS applications. In the window where CIAO is sourced any other Python installation is overwritten: this may interfere with other Python applications on the user's system. At present, the workaround is to run CIAO in a separate window from other Python applications. The CXC is working to resolve this issue.

IPython

• CIAO 4.0 includes IPython 0.8 in the CIAO OTS directory which is used by Sherpa and ChIPS to provide command-line user interfaces. These programs create an IPython profile in the directory $HOME/.ipython-ciao.

  If IPython users want any personal customizations to be available when running CIAO, they will have to copy them from $HOME/.ipython to $HOME/.ipython-ciao.

---

How CALDB 3.5.0 Affects Your Analysis

ACIS -120 C Imaging and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 May 2008 - 31 July 2008 (Epoch 34):

  acisD2008-02-01t_gainN0005.fits
  acisD2008-02-01t_gainN0006.fits
  acisD2008-05-01t_gainN0002.fits
  acisD2008-05-01t_gain_biN0002.fits
  

  The files are applicable to -120 C focal plane temperature only. The acisD2008-05-01t_gain[_bi]N0002.fits files are the new default proccesing files in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0006 files showing T_GAIN energy shifts
  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

ACIS -110 C Grating Data

• Order-sorting (OSIP) Files for the Back-illuminated Chips (S1, S3)

  New order-sorting files for the back-illuminated chips S1 and S3 at -110 C have been made available to correspond to the ACIS -110C GAIN file released in CalDB 3.4.3 (March 2008):

  acisD1999-09-16osipN0007.fits 
  acisD2000-08-11osipN0007.fits
  

  The file acisD2000-08-11osipN0007.fits has the identical data set in the OSIP table extension, but is effective for the -110 C recalibration period (UTC) 2000-08-11T00:00:00 through 2000-08-12T22:00:00. Grating observations taken during this period should use the new OSIP data as well.

  Beginning a new analysis

  If you are beginning a new analysis, reprocess your data with acis_process_events to apply the newest calibration. The reprocessing will also ensure that the same OSIP files are applied to the data (via tg_resolve_events) and to the response files (via mkgarf) .

  Analyses in progress

  If you have an analysis in progress, use this information to evaluate whether or not the calibration updates will significantly impact your results.

  • Chip S1: the previous OSIP had an acceptance region artificially broadened for the older, poorer S1 gain function. Applying the new OSIP in conjunction with the gain released in CALDB 3.4.3 markedly improves analysis of data on chip S1.

  • Chip S3: changes in the order sorting are within the noise.

  Applying the new calibration means redoing your entire analysis from the level=1 event file, as shown in the Create a New Level=2 Event File thread, including creating a new PHA2 file. You will also have to remake any spectra and response files (ARF, RMF) used in your analysis.

  Users who do not reprocess their data need to be sure to use the same OSIP file that was applied to the data when creating gARFs. The CIAO software will automatically pick up the most recent applicable file, which may be different than the one used by tg_resolve_events.

  The file name is recorded in the evt1a.fits or evt2.fits file header:

  unix% dmkeypar acis_evt2.fits OSIPFILE echo+
  acisD1999-09-16osipN0006.fits
  

  This file is not the same as the new file, acisD1999-09-16osipN0007.fits. Set this filename in the osipfile parameter:

  unix% pset mkgarf osipfile=$CALDB/data/chandra/acis/cpf/osip/acisD1999-09-16osipN0006.fits
  

---

How CALDB 3.4.5 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 February 2008 - 31 April 2008 (Epoch 33):

  acisD2007-11-01t_gainN0005.fits
  acisD2007-11-01t_gainN0006.fits
  acisD2008-02-01t_gainN0002.fits
  acisD2008-02-01t_gain_biN0002.fits
  

  The files are applicable to -120 C focal plane temperature only. The acisD2008-02-01t_gain[_bi]N0002.fits files are the new default proccesing files in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0005 files showing T_GAIN energy shifts
  • ACIS-I3 aimpoint: version N0006 files showing T_GAIN energy shifts

  Note that the plots of the I3 files are identical. The same CTI-corrected T_GAIN data for the FI chips exist in both.

  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

---

How CALDB 3.4.3 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (TGAIN) Files for -120 C Data

  There are new time-dependent ACIS Gain (TGAIN) files for 01 November 2007 - 31 January 2008 (Epoch 32):

  acisD2007-08-01t_gainN0005.fits
  acisD2007-08-01t_gainN0006.fits
  acisD2007-11-01t_gainN0002.fits
  acisD2007-11-01t_gain_biN0002.fits
  

  The files are applicable to -120 C focal plane temperature only. The acisD2007-11-01t_gain[_bi]N0002.fits files are the new default proccesing files in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0005 files showing T_GAIN energy shifts
  • ACIS-I3 aimpoint: version N0006 files showing T_GAIN energy shifts

  Note that the plots of the I3 files are identical.

  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  The differences in these figures are attributed to whether the T_GAINs are derived from CTI-corrected data or non-CTI data.

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

• Time-dependent ACIS Gain (TGAIN) Files for -110 C Data

  A TGAIN correction for the back-illuminated chips (S1 and S3) at the -110 C focal plane temperature has been derived. This file is a flat correction based on the -120 C gains used for the two BI chips. The corresponding gain file is also included in this release:

  acisD1999-09-16t_gainN0005.fits
  acisD1999-09-16gainN0006.fits
  

  These plots illustrate the degree of change the -110 C file (red plot) will produce in comparison with the -120 C Epoch 1, 2, and 3 files.

  • S3 aimpoint
  • S3 midpoint
  • S1 midpoint

  Users working with ACIS data taken at -110 C on ACIS-S1 or S3 should reprocess to apply the TGAIN calibration to the data.

  Creating RMFs: if the -110 C TGAIN is applied to the data, users must use the tool mkacisrmf when creating RMFs for S1 and S3, otherwise the response file will be wrong. RMFs for the front-illuminated chips should still be made with the tool mkrmf. The mkacisrmf why topic and various threads have details on how to run the tool.

---

How CIAO 4.0 and CALDB 3.4.2 Affect Your Analysis

ACIS Imaging Data

• ACIS Dead Area Correction

  The ACIS dead area correction is turned on by default in this release. For complete details on the dead area correction, read the ACIS Dead Area Correction why topic.

  The following CIAO response tools and scripts can apply the dead area correction:

  • mkarf
  • mkinstmap
  • mkgarf
  • mkwarf
  • specextract
  • fullgarf
  • psextract
  • acisspec

  The pbkfile and dafile parameter defaults have been updated to reflect that the correction is applied by default. Refer to the help file for each tool for details, e.g. ahelp mkwarf.

  Turning Off the Correction

  If you are have an analysis in progress and do not want to apply the dead area correction, it is possible to "turn off" the correction. To do so, set the dafile parameter to "NONE" and leave the pbkfile parameter blank. For example:

  unix% pset mkgarf pbkfile="" dafile=NONE
  

• Time-dependent ACIS Gain (TGAIN) Files

  There are new time-dependent ACIS Gain (TGAIN) files for 01 August 2007 - 31 October 2007 (Epoch 31):

  acisD2007-05-01t_gainN0005.fits
  acisD2007-05-01t_gainN0006.fits
  acisD2007-08-01t_gainN0002.fits
  acisD2007-08-01t_gain_biN0002.fits
  

  The files are applicable to -120 C data only. The acisD2007-08-01t_gain[_bi]N0002.fits files are the new default proccesing files in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0005 files
  • ACIS-I3 aimpoint: version N0006 files.

  As shown in both of the figures above, the T_GAIN correction at the I3 aimpoint is currently diminishing with time.

  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

ACIS Grating Data

• ACIS Dead Area Correction

  The ACIS dead area correction is turned on by default in this release. For complete details on the dead area correction, read the ACIS Dead Area Correction why topic.

  The following CIAO response tools and scripts can apply the dead area correction:

  • mkarf
  • mkinstmap
  • mkgarf
  • mkwarf
  • specextract
  • fullgarf
  • psextract
  • acisspec

  The pbkfile and dafile parameter defaults have been updated to reflect that the correction is applied by default. Refer to the help file for each tool for details, e.g. ahelp mkwarf.

  Turning Off the Correction

  If you are have an analysis in progress and do not want to apply the dead area correction, it is possible to "turn off" the correction. To do so, set the dafile parameter to "NONE" and leave the pbkfile parameter blank. For example:

  unix% pset mkgarf pbkfile="" dafile=NONE
  

• Time-dependent ACIS Gain (TGAIN) Files

  There are new time-dependent ACIS Gain (TGAIN) files for 01 August 2007 - 31 October 2007 (Epoch 31):

  acisD2007-05-01t_gainN0005.fits
  acisD2007-05-01t_gainN0006.fits
  acisD2007-08-01t_gainN0002.fits
  acisD2007-08-01t_gain_biN0002.fits
  

  The files are applicable to -120 C data only. The acisD2007-08-01t_gain[_bi]N0002.fits files are the new default proccesing files in the CALDB.

  These plots illustrate the degree of change the new files will produce in comparison with the old files.

  • ACIS-I3 aimpoint: version N0005 files
  • ACIS-I3 aimpoint: version N0006 files.

  As shown in both of the figures above, the T_GAIN correction at the I3 aimpoint is currently diminishing with time.

  • ACIS-S3 aimpoint: no CTI correction applied
  • ACIS-S3 aimpoint: with CTI correction applied

  Users working with ACIS data taken during this period may wish to reprocess to improve the TGAIN calibration in their data. The DATE-OBS header keyword records the observation start date.

  Note that unless you are fitting a spectra with oxygen emission lines, the gain refinement is unlikely to have an effect on the spectrum larger than the uncertainties in determining the gain.

---

Tools

acis_build_badpix

• Bug fix: the tool would fail crash when doing CALDB lookup if the user supplied a bad obsfile value.

acis_process_events

• The algorithm used to set STATUS bit 23 for TIMED VFAINT mode events that may be potential background events was changed to ignore pixels that are known to be bad. Previously set VFAINT status bits are unset before the check is done again.

  There is also a warning message if check_vf_pha is set for data that wasn't taken in VFAINT mode.

• Bug fix: STATUS bit 4, which is used to identify events on bad pixels, was being set for events on bad pixels and events adjacent to bad pixels. Now the bit is set only for events on bad pixels. The number of events for which any bad-pixel STATUS bit is set should remain the same because STATUS bit 5, which is used to identify events adjacent to bad pixels, was (and still is) set properly.

• Bug fix: the stop time of afterglow badpixels was not being correctly used. The afterglow badpixels are now marked as bad (bit 16) until the stop time, not the end of the observation.

• Bug fix: after a file is reprocessed, the values of PHA_RO are equal to the summed pulse height of the original, unadjusted event island (PHAS).

acis_run_hotpix

• Bug fix: the dmkeypar commands used by this script now have echo="no" set to quiet screen output.

aconvolve

• Bug fix: the "edges" treatment parameters for double precision sliding cell convolution were being ignored. Also fixed problems with convolutions of 3D images.

• The kernel parameter has been removed.

acrosscorr

• The kernel parameter has been removed.

apowerspectrum

• The kernel parameter has been removed.

arestore

• The kernel parameter has been removed.

ascii2fits

• The ascii2fits script has been removed from CIAO. This simple script relied on the VARMM (Variable, Math and Macro S-Lang) library, which is not available in CIAO 4.0. In the future, its functionality will be replaced by the Data Model ASCII kernel.

asphist

• The tool only prompts for the dtffile parameter if the input file contains HRC data.

axbary

• The tool prompts for parameters in the same order as they appear in the parameter file.

• Supporting functions for barycenter operations have been updated.

• Bug fix: corrected a issue that appeared when the nput filename is > 49 characters and the output filename is < 49.

celldetect

• The kernel parameter has been removed.

csmooth

• The kernel parameter has been removed.

destreak

• The tool has been updated to better handle cases when the streak rate is unusually high; see the description of the max_rowloss_fraction parameter in the help file. The change is relatively minor and should affect only those observations that have a high streak row-loss fraction on ACIS-S4.

dmappend

• Bug fix: the tool no longer crashes when appending to a file that contains only an image extension.

dmarfadd

• Bug fix: tool would fail if the input stack file does not exist. It now prints an error message.

• Fixed crash that could happen when more than 12 files were input.

• Cleaned up warnings when grating keywords are missing (not required to create valid output).

• The kernel parameter has been removed.

dmcontour

• The kernel parameter has been removed.

dmcoords

• When running the tool in parameter mode, the "sim" and "displace" values get added to the values obtained from the aspect solution file. If "Set SIM/DISPLACE [x y z ax ay az]" is used in interactive mode, however, those values are taken as absolute, even if an aspect solution file was specified.

• Bug fix: if you feed dmcoords a chip coordinate which is outside the valid range of 0.5:1024.5, it now prints a warning about "Bad chip coordinates: Negative or zero value(s)".

dmdiff

• Bug fix: the tool no longer truncates the tolerance range on a header key with a real value to an integer. This was effectively making the tolerance more restrictive.

• Bug fix: the error printed when trying to compare two columns that have different datatypes and/or different cases in the column name has been improved.

• The limit of 300 columns per file has been removed.

• Bug fix: the tool would not detect differences between the vector arrays of the input files, e.g. comparing columns CHIP(CHIPX,CHIPY).

dmextract

• Bug fix: a parse error would occur when the grid syntax was used in the input file filter, e.g. "[bin col=grid(file[filter]);weight]", and a background file was specified. (Even though the grid syntax was part of the input file filter, it was the parsing of the background file that causes the failure.)

dmfilth

• The calculation of the POISSON distribution has been updated to be more numerically stable. Previously, the tool would fail if pixels had more than ~100 counts/pixel and this distribution was used.

• The DIST, POISSON, and GLOBAL distribution arrays have all been updated from static to dynamic allocation.

• A normalized Gaussian distribution is substituted for the Poisson distribution when average background is greater than 500 counts/pixel to avoid numerical problems.

dmimg2jpg

• The log and powerlaw scaling routines were rewritten to give better results for normalized (fluxed) images. Changes to the powerlaw routine are subtle, but users should see marked improvement in the log scaling routine.

• The scalefunction options now include "asinh".

dmimgpick

• Bug fix: if the input image was not square, dmimgpick would have a problem related to how the tool checks if the pixel being "picked" is outside the image and the tool would fail with a segmentation fault. This bug was seen on Linux and Mac OS X platforms.

dmimgthresh

• Bug fix: running the tool with cut=<value>: or cut=:<value> would treat the unspecified limit as "0".

dmjoin

• The joinfile parameter can take a stack of input files. The tool checks the consistency of the number and type of columns in the stack.

dmmakereg

• Bug fix: using region="bounds(...)" and kernel=ascii produces a region file formatted for use into ds9, e.g. box(xmid,ymid,width,height,angle).

• Bug fix: a bug existed when the shapes that are being printed don't all have the same number of points (e.g. one is a circle and one is a polygon).

dmmerge

• Bug fix: the tool checks the length of text columns before merging the files and exits if they do not match.

dmpaste

• The kernel parameter has been removed.

dmsort

• The kernel parameter has been removed.

dmstat

• Images with dmBTYE datatype (8-bit unsigned integer) are now supported.

dmtcalc

• The kernel parameter has been removed.

dmtype2split

• dmtype2split no longer assumes an output block name of "SPECTRUM". It is necessary to specify the output block name, e.g.

  dmtype2split infile="pha2.fits[#row=4:4]" outfile="HEG_p1.pha[SPECTRUM]" 
  

  If no block name is given, the input filename is used to name it.

• Bug fix: when writing out vector columns, only the first component was copied to the output file. Now both components of the vector are written to the output as scalar columns.

• Bug fix: the tool no longer writes incorrect CHANTYPE and BACKSCAL header keyword values. The values are copied from the input file.

get_src_region

• There are a number of bug fixes for the case where all the values in a floating-point image were between 0 and 1, such as the "threshold" value getting cast to an integer and Null/NaN/DSS-region checking not working correctly.

• The tool would fail on Linux when there were no "bright" pixels in the image.

• The kernel parameter has been removed.

hrc_dtfstats

• The infile parameter can take a stack of input files. The stack is combined into a single output file. All the records from all the DTF files in the stack (which pass the GTI filter in the gtifiles parameter) are used to compute average quantities for the output file.

• The lookupTab parameter was added to provide header merging rules.

• Bug fix: if the tool was run with the wrong input file, e.g. an evt2 instead of the dtf file, it would appear to run but created no output.

hrc_process_events

• Bug fix: the tool would fail if given a level 2 file as input, rather than printing an error message.

lightcurve

• The lightcurve tool has been removed from CIAO. The dmextract should be used for creating lightcurves; this has been the recommendation since CIAO 3.0 (August 2003).

mkacisrmf

• Two new parameters have been added to mkacisrmf: asolfile and obsfile.

  If the aspect solution file(s) for the observation are provided in the asolfile parameter, the average dither values are used to derive the correct instrument coordinates for an input WMAP in DETX,DETY coordinates.

  The obsfile value serves only for CALDB lookup. The value defaults to the wmap parameter value, so users don't have to specify both.

• Bug fix: if the WMAP file is empty (i.e. all values are zero), the tool issues a warning.

• Bug fix: the tool will generate the correct RMF for a non-standard channel grid, e.g. channel="69:128:1".

mkarf

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

  Users who wish to continue analysis without the dead area correction should set both of these parameter values to "NONE".

mkgarf

• The obsfile parameter is no longer redirected to the value of the asphistfile by default. The recommended input to this parameter is now a level 2 event file or similar FITS file.

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

  Users who wish to continue analysis without the dead area correction should set both of these parameter values to "NONE".

mkinstmap

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

  Users who wish to continue analysis without the dead area correction should set both of these parameter values to "NONE".

mkoif

• The mkoif tool has been removed from CIAO.

mkwarf

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

  Users who wish to continue analysis without the dead area correction should set both of these parameter values to "NONE".

• Bug fix: a check for invalid chip coordinates has been added to mkwarf. The edge of the WMAP (in detector coordinates) could get mapped to chip coordinates at 1024.5 which was rounded to 1025. The tool then tried to find the FEF region to convert chipy=1025 (for which there is none) and thus failed.

• Added support for WMAPs in TDET coordinates in addition to DET coordinates.

• The combination of the TDET and single pixel changes causes mkwarf to skip bad pixels. This is a problem if the observation is over a bad column or node boundary, in which case the ARF is significantly over-estimated.

psf_project_ray

• Bug fix: when the evtfile parameter was used, only the first 10 characters of the DETNAM header keyword value were copied to the output.

specextract

• Two new parameters have been added to specextract to allow the ACIS dead area correction to be taken into account: pbkfile and dafile.

  The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile parameter has no default value, and the dafile parameter default is "CALDB". If a stack of input filenames is given to specextract, there must be the same number of parameter block files specified.

  Users who wish to continue analysis without the dead area correction should set both of these parameter values to "NONE".

sso_freeze

• Two new parameters have been added to sso_freeze: asolfile and ocsolfile. They can be used to create an object-centered aspect solution file, which may then be used to generate an exposure map or response file (ARF) for the data.

tg_create_mask

• In order to use the parameter sX_width_heg/meg/leg to set the output width, sX_zero_rad must be set. If sX_zero_rad is not set, a warning will be displayed to tell the users that the radius and the width are computed by using input_psf_tab.

tg_resolve_events

• tg_resolve_events cannot accept a stack of files for the input, output, or regionfile parameters. The tool now checks that only one filename is supplied for each.

tgdetect

• Bug fix: a problem would occur when running this tool with snr_thresh=3 because that is the default value of the celldetect thresh parameter. celldetect is called by tgdetect, and a logic bug was causing a section of code not to execute when the values are equal. The bug has been fixed.

tgextract

• Bug fix: the tool now writes the DETCHANS header keyword in the output PHA2 file. This will eliminate an error seen by XSpec users.

tgextract2

• New Tool!

  tgextract2 is an alternative to tgextract, providing the same functionality with a different parameter interface and slightly different output format. It allows large and asymmetric regions for better determination of background counts.

  • The tool will optionally compute the BACKSCALE (BG_AREA column) from the region and events, enabling support of large regions which extend off detector or non-uniform regions.

  • The BG_AREA and BACKSCAL are output as arrays.

    Note that Sherpa 3.4 cannot use the BACKSCAL array properly; it will read in only the first value in the array. ISIS [http://space.mit.edu/CXC/ISIS/] reads in the whole array correctly.

  • Tool parameters and output columns are consistent with the dmextract tool, where applicable.

  This tool is not back-compatible with tgextract.

vtpdetect

• Bug fix: when vtpdetect is given an image which has been blocked by a factor other than 1, it no longer double-corrects the radii.

• The kernel parameter has been removed.

wavdetect

• An alternative algorithm for specifying the allowed number of false sources detected has been added to this tool. Based on extensive new simulations to determine the dependence of the number of pixels per false source on source significance and background (expressed in the new sigcalfile parameter), the tool can now determine the significance threshold for source detection itself, if the user inputs directly the desired false source rate (e.g., desired number of false sources per field per scale). Both threshold algorithms are fully supported, and users need make no changes if they still wish to use the current algorithm. The new algorithm is restricted to symmetric wavelets but can be applied to non-Chandra data.

• An alternative algorithm for specifying the allowed number of false sources detected has been added to this tool. There are two new parameters related to the new algorithm: falsesrc and sigcalfile.

  falsesrc sets the number of false sources allowed per scale in the image. That number is combined with image size and scale information, using additional simulation results contained in the file defined by the sigcalfile parameter, to determine the threshold significance for each pixel; consequently the value given for the parameter sigthresh is ignored. If falsesrc is set to a negative number, the code uses sigthresh instead to determine thresholds.

  The data file for the false-source algorithm is set in the sigcalfile parameter. The file contains simulation results that are used with the false-source algorithm. This file is included with with the release and may be used for both Chandra and non-Chandra data. The user may not specify "CALDB" for this parameter.

• The calibrated range of the new wavdetect is 0.1 to 1e-8 false srcs per pixel. Depending on the significance settings and the input image size, users can go outside of this range in which case the results become unpredictable.

• The kernel parameter has been removed.

wtransform

• (wavdetect calls wtransform, hence the identical release notes for this tool.)

• An alternative algorithm for specifying the allowed number of false sources detected has been added to this tool. Based on extensive new simulations to determine the dependence of the number of pixels per false source on source significance and background (expressed in the new sigcalfile parameter), the tool can now determine the significance threshold for source detection itself, if the user inputs directly the desired false source rate (e.g., desired number of false sources per field per scale). Both threshold algorithms are fully supported, and users need make no changes if they still wish to use the current algorithm. The new algorithm is restricted to symmetric wavelets but can be applied to non-Chandra data.

• An alternative algorithm for specifying the allowed number of false sources detected has been added to this tool. There are two new parameters related to this addition: falsesrc and sigcalfile.

  falsesrc sets the number of false sources allowed per scale in the image. That number is combined with image size and scale information, using additional simulation results contained in the file defined by the sigcalfile parameter, to determine the threshold significance for each pixel; consequently the value given for the parameter sigthresh is ignored. If falsesrc is set to a negative number, the code uses sigthresh instead to determine thresholds.

  The data file for the false-source algorithm is set in the sigcalfile parameter. The file contains simulation results that are used with the false-source algorithm. This file is included with with the release and may be used for both Chandra and non-Chandra data. The user may not specify "CALDB" for this parameter.

• The kernel parameter has been removed.

---

Parameter Files

ciao.par

• The filtwin_init and max_savestates parameters have been removed from ciao.par, since the filtwin application is no longer part of CIAO.

hrc_dtfstats

• New lookupTab parameter provides header merging rules.

mkacisrmf

• Two new parameters: asolfile (aspect solution file) and obsfile (observation file for CALDB lookup).

mkarf

• The pbkfile and dafile parameter defaults have been updated to reflect that the ACIS dead area correction is "on" by default in CIAO 4.0.

mkgarf

• The obsfile parameter is no longer redirected to the value of the asphistfile.

• The pbkfile and dafile parameter defaults have been updated to reflect that the ACIS dead area correction is "on" by default in CIAO 4.0.

mkinstmap

• The pbkfile and dafile parameter defaults have been updated to reflect that the ACIS dead area correction is "on" by default in CIAO 4.0.

mkwarf

• The pbkfile and dafile parameter defaults have been updated to reflect that the ACIS dead area correction is "on" by default in CIAO 4.0.

specextract

• Two new parameters to allow the ACIS dead area correction to be taken into account: pbkfile and dafile.

sso_freeze

• Two new parameters for use in creating an object-centered aspect solution file: asolfile and ocsolfile.

wavdetect

• Two new parameters related to detecting false sources: falsesrc and sigcalfile.

wtransform

• Two new parameters related to detecting false sources: falsesrc and sigcalfile.

kernel parameter

• The kernel parameter has been removed from the following tools:

  • aconvolve
  • acrosscorr
  • apowerspectrum
  • arestore
  • celldetect
  • csmooth
  • dmarfadd
  • dmcontour
  • dmpaste
  • dmsort
  • dmtcalc
  • get_src_region
  • vtpdetect
  • wavdetect
  • wtransform

---

Data Model

ASCII Kernel

• New Functionality!

  The Data Model library now supports an ASCII kernel. This kernel allows users to generate ASCII format files for use within Data Model enabled tools (e.g. dmlist, dmcopy), and apply the suite of Data Model capabilities to those files via its extended filename syntax. For this release, three ASCII file formats - from raw tabular data, to a multi-extension FITS-like style - are supported, allowing the user to create ASCII files with varying levels of complexity. Several option tags can be used to customize the format definition for elements such as separator characters, line skipping, and comment designation to meet specific needs.

  For this release, we are supporting ASCII table files only. While there is limited image handling capability in this release, full support will be added in a future release. The ASCII Kernel is being released as a beta.

  For more information on the ASCII kernel, refer to:

  • ahelp dmascii
  • Using ASCII Files in CIAO thread
  • Using the ASCII Kernel manual

Bug Fixes

• The DM would not complain when given a component in a FITS region file that did not exist (e.g. "file.fits[component=99]" when there was no component numbered 99).

• In vector filtering cases where the components are non-adjacent dissimilar type columns, e.g. (tg_lam,pi), the dmcopy tool and the DM table filtering function would produce different results due to a difference in filtering logic.

• Corrected problems transforming a vector of arrays. This had been listed on the dmcopy and dmlist bug pages.

Enhancements

• The DM will rename transform descriptors if another descriptor already has the name. Includes changes where default image axis names are redefined after all file descriptors are generated to avoid clashes between default names and those defined in the file.

• If a file contains vector columns with single range subspace, the subspace is applied to each vector component. The output file contains individual subspaces for each component.

Transform Library

• Three new commands have been added to the Transform library:

  • copy_transform
  • apply_transform
  • get_transform_type

---

User Interface

firstlook and filtwin

• The firstlook and filtwin GUIs have been removed from CIAO.

---

Libraries

cxcparam

• There are updates to the parser to allow spaces around commas in the parameter file, e.g. "infile, s, a, ...", in order to be compatible with some FTOOLS parameter files.

dslib

• The WCS routines have been updated to handle images with physical coordinates but no world coordinates.

psflib

• New routines to access the PSF/ECF info:

    psfGetInit()
    psfGetFree()
    psfGetRadius()
    psfGetECF()
  

• Updated to use either the old (one table) or new (multiple blocks) style of ECF tables.

regionlib

• Bug fix: using region="bounds(...)" and kernel=ascii produces a region file formatted for use into ds9, e.g. box(xmid,ymid,width,height,angle).

tcdlib

• Added new routines for double precision FFT and Slide convolutions.

• Significant speed up in convolutions.

• Bug fix: the "edges" treatment parameters for double precision sliding cell convolution were being ignored. Also fixed problems with convolutions of 3D images.

---

S-Lang

VARMM replaced

• The VARMM (Variable, Math and Macro) S-Lang library has been replaced in CIAO 4.0 by the Crates and Transform libraries.

  Crates wraps the CIAO Datamodel (DM) library, and provides a scripting interface with all of the capabilities of the Data Model (including filtering) from either Python or S-Lang. The Transform library provides a high-level interface for performing World Coordinate System transformations. Both support direct interactive access by the user and are called internally by applications that need to use these functions directly.

  Users who need assistance in updating S-lang scripts to run in CIAO 4.0 should contact the CXC HelpDesk.

paramio

• Bug fix: the paccess() function returns a NULL when the parameter file cannot be found.

---

Environment Modifications

Wrapper Scripts

• In an continuing effort to make CIAO less intrusive on the users environment, variables that impact system libraries and modules such as LD_LIBRARY_PATH, PYTHONPATH, and SLANG_SCRIPT_PATH are no longer set when the CIAO setup scripts (e.g. ciao.csh) are sourced. They are instead set only when a CIAO program or script is run. This was done by wrapping the CIAO executables in scripts. The wrapper scripts are in the $ASCDS_INSTALL/bin directory, and the executables they call are in $ASCDS_INSTALL/binexe.

  This new setup will allow the user to safely run non-CIAO programs in your CIAO window.

---

Analysis Scripts

acis_bkgrnd_lookup v2.0

• The script has been updated to to run under S-Lang v2. Code updates were made to replace the VARMM (Variable, Math and Macro S-Lang) library with the cxcdm (CXC Data Model) module. Otherwise, acis_bkgrnd_lookup functionality is unchanged.

acis_fef_lookup v1.21

• The script has been updated to to run under S-Lang v2. Otherwise, acis_fef_lookup functionality is unchanged.

acisspec v4.1

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

• The kernel parameter was removed from dmtcalc and dmpaste.

add_grating_orders v2.4

• It is necessary to specify an output block name of "SPECTRUM" in the dmtype2split commands used by the script. Quotation marks were also added to a dmhedit command so that the correct pathname written in the ANCRFILE header keyword in the output PHA file.

• The kernel parameter was removed from dmtcalc and dmpaste.

add_grating_spectra v3.3

• The kernel parameter was removed from dmtcalc and dmpaste.

analyze_ltcrv.sl v1.6

• The plotting routines have been removed from the script, since they will not work in ChIPS 4.0. Otherwise, analyze_ltcrv.sl functionality is unchanged. The plotting functionality will be replaced soon.

• The script has been updated to use the cxcdm module in place of VARMM (Variable, Math and Macro S-Lang library).

fullgarf v4.1.1

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

• An additional parameter change was made in mkgarf. The obsfile value is set explicitly to the name of the event file instead of being redirected to the asphistfile parameter value.

get_sky_limits v1.9

• The script now uses the cxcdm module to inspect the input file. This uncovered a few bugs for images with no physical or world-coordinate systems. It also means that the case of the axis names, as reported on-screen and saved to the dmfilter parameter, may change.

lc_clean.sl v1.10

• The plotting routines have been removed from the script, since they will not work in ChIPS 4.0. Otherwise, lc_clean.sl functionality is unchanged. The plotting functionality will be replaced soon.

• The script checks for the histogram module and defaults to the cxcdm module if histogram is unavailable. This replaces the CIAO 3.4 use of VARMM (Variable, Math and Macro S-Lang library).

merge_all v3.8

• Replaced an echo command with pget to assign "oldtab" variable

psextract v4.0.1

• The ACIS dead area correction is "on" by default in CIAO 4.0. The pbkfile and dafile parameter defaults have been updated to reflect this. The pbkfile parameter is now automatic (i.e. not hidden) and there is no default value. The default value of the dafile parameter has been changed to "CALDB".

  Note that psextract applies the same pbkfile to the source and background datasets. If this is not appropriate for your analysis, you will have to run mkarf independently to create an ARF with the dead area correction. (In this case, make sure to "turn off" the correction during the psextract run, i.e. pbkfile=NONE and dafile=NONE.)

Missing Scripts

• Due to dependencies on ChIPS or Sherpa, a number of CIAO scripts have been removed temporarily from the contributed scripts package. They will be replaced once the script has been rewritten to work with the new version of the applications.

  CIAO scripts with some ChIPS or Sherpa dependency (e.g. for plotting):

  • monitor_photom
  • regions.sl
  • show_tgain_corr
  • show_wgt.sl
  • spectrum.sl
  • tg_osort_img
  • tg_scale_reg

  Sherpa-specific scripts:

  • chart_spectrum.sl
  • paramest.sl
  • setplot.sl
  • sherpa_paramset.sl
  • sherpa_plotfns.sl
  • sherpa_utils.sl
  • simspec

Obsolete Scripts

• The following obsolete scripts have been removed from the contributed scripts package:

  • acis_expmap
  • color_image
  • sstats.sl

---

Documentation

ChIPS Website

• There is a new website dedicated to the ChIPS plotting package: http://cxc.harvard.edu/chips.

Sherpa Website

• There is a new website dedicated to the Beta version of the new Sherpa package: http://cxc.harvard.edu/sherpabeta.

Download pages

• There are known problems with the CIAO download pages in Internet Explorer on the Mac OS X platform. We recommend that you choose an alternate browser - such as Safari (distributed with Mac OS X) or Mozilla Firefox [http://www.mozilla.org/products/firefox/] - to avoid download problems.

Ahelp

• The options to have ahelp open a web browser and display an HTML version of the help page have been removed. These options were "-i" (display HTML fom the CIAO website) and "-w" (display HTML packaged with CIAO).

  Users who wish to have a copy of the help pages in HTML when they are not online should download the CIAO Website [http://cxc.harvard.edu/ciao4.0/download/web.html].

• Documentation on the S-Lang Run-Time Library is no longer packaged as ahelp files. Users can access this info from within slsh via "help", e.g.:

  slsh> help assoc_get_keys;
  assoc_get_keys
  
   SYNOPSIS
    Return all the key names of an Associative Array
  
  ...
  

Analysis Threads

• The CIAO Science Threads are in the process of being tested and updated for the CIAO 4.0 release. Please check the "Last Update" information at the top of each thread to see if it has been reviewed yet.

Missing Threads

• Due to dependencies on ChIPS, a number of CIAO threads have been removed temporarily from the website. Users will need to run these threads in CIAO 3.4 if these threads are crucial to the analysis.

  • Introduction to Plotting Using ChIPS

  • Measure Grating Dispersion Distance

  • Processing ACA Monitor Window Data

  • Create an Order-Sorting Image

  • Displaying the FEF Regions Covered by a Source

  • Calculating Spectral Weights

Obsolete Threads

• The following obsolete threads have been removed from the website.

  • Extracting Extended Source Spectra and Responses: use specextract for these analyses

  • Correcting Responses for ACIS Contamination: the information is available from the ACIS QE Degradation why topic

  • Updating dmgroup Syntax for CIAO 3: refer to the dmgroup help file for syntax

  • Calculating Statistics of Images: the obsolete script used in this thread, sstats.sl, was removed from CIAO; refer to the dmstat help file for syntax

  • Creating Accurate RA, Dec Coordinates for src2 Files: the detections in the src2.fits files are not sufficient for direct use in analysis. Users interested in spatial analysis should perform their own detection runs with one of the three Chandra Detect tools: celldetect, wavdetect, and vtpdetect.

---