What's New for CIAO 4.17

CALDB 4.12.0 includes updates to:

• Revised set of ACIS "Phase 2 Response" (P2_RESP) files for observations with warm focal-plane temperatures on front-illuminated CCDs, effective over the -120C to -105C temperature range for ACIS-I0, -I1, -I2, -I3, and -S2. These files are used as input to the mkacisrmf CIAO tool, which is also used by specextract.
• HRC-S time-dependent quantum efficiency uniformity includes a new epoch to account for changes in HRC's operating voltages. This will affect the exposure maps and ARF.

CALDB 4.11.6 includes updates to the HRC-I time dependent quantum efficiency since 2011 and includes new epochs to account for changes in HRC's operating voltages. This will affect the exposure maps and ARF.

The CIAO contributed scripts package version 4.17.0 has been released. It includes a few new scripts including

• fine_astro which automates the typical steps needed to apply a fine astrometric correction to Chandra datasets.
• The mkrprm script (make radial profile response matrix) approximates the aperture correction sometimes needed when extracting radial profiles.
• The patch_hrc_ssc script identifies HRC observations that suffer from Secondary Science Corruption (SSC) and updates the corrupted dead time values.

Along with additional bug fixes and enhancements.

The CXC is pleased to announce the availability of CIAO 4.17. This release includes several important updates and bug fixes as well as support for the latest operating systems

Notable changes include

• dmcopy bug fix triggered when using opt=all to copy all blocks in a FITS file, when a time filter has been applied.
• collect_chandra_dois is a new script to help users gather lists of OBS_ID values to create a new Dataset Object Identifier (DOI).
• Updates to aconvolve, csmooth, and others to use double precision when computing FFT based convolutions. This improves numeric accuracy especially when working with very large images or images with a large range of pixel values.

CIAO 4.17 comes with Sherpa v4.17.0 and ds9 v8.6.

CIAO 4.17 is available with Python 3.11 (only) for the following platforms:

• 64-bit Linux distributions, CentOS8 / Ubuntu 22.04 era and newer
• macOS 12-15 (Monterey through Sequoia) (both Intel x86 and Apple Silicon ARM processors).

CIAO 4.17 can be installed using conda or using the ciao-install script.

See the release notes for more details and a complete list of changes.

SAOImageDS9 version 8.6 has been released. It includes improved SAMP support, new Hue Saturation Value (HSV) and Hue Lightness Saturation (HLS) frames (similar to the traditional tri-color Red Green Blue (RGB) frame), and improved support for multi-dimensional compressed images.

CALDB version 4.11.5 has been released. Changes include new and updated HRC-S QE and QEU files.

CALDB version 4.11.3 has been released with new HRC-S QEU files affecting observations after 2024 March 4.

Version 4.16.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.16 users are encouraged to update this version of the package. This update provides minor enhancements and bug fixes to many scripts but also introduces the sherpa_contrib.diag_resp function to simplify creating 'perfect' responses (i.e. flat ARF and diagonal RMF) for a variety of X-ray instruments in Sherpa. A complete list of changes is available on the contributed scripts history page.

CALDB version 4.11.2 has been released. The changes includes a new HRC-I QE file affecting HRC-I observations made since 27 September 2023.

CALDB version 4.11.1 has been released. Changes include

• ACIS: Improved time and temperature dependent CTI correction for the ACIS-S2 and -S3 CCDs.
• Updated HRC-S time dependent QEU files.

Version 4.16.1 of the Contributed Scripts and Modules tarfile has been released; download and install the updated package from with Conda or ciao-install, with directions on Scripts page. The update is driven by the release of the CSC 2.1 with revisions to search_csc and obsid_search_csc and several minor bugs have been fixed.

The CXC is pleased to announce the availability of CIAO 4.16. This release includes several important updates and bug fixes as well as support for the latest operating systems including native support for macOS ARM/M1/M2/M3 systems.

Notable changes include

• Native build for macOS ARM/M1/M2/M3 architecture
• CIAO is now only distributed as a set of conda packages. The ciao-install script has been re-written to automate the installation of conda and CIAO packages.
• Updates to tgextract and mkgrmf for higher order (>1) ....stuff....
• acis_process_events support for temperature dependent CTI corrections.
• Several tools, including dmextract, have been updated to copy the WCS from the input image to the output table.
• Improved handling for non Chandra/CIAO file with incomplete or incompatible metadata (keywords).

CIAO 4.16 comes with Sherpa v4.16.0 and ds9 v8.5.

CIAO 4.16 is available for the following platforms:

• 64-bit Linux distributions, CentOS8 / Ubuntu 20.04 era and newer
• macOS 11-14 (Big Sur through Sonoma)
  • Apple silicon ARM/M1/M2/M3
  • Intel/x86_64

CIAO 4.16 can be installed using conda or using the ciao-install script.

See the release notes for more details and a complete list of changes.

CALDB version 4.11.0 has been released. Changes include

• ACIS: Improved time and temperature dependent CTI correction for the four ACIS-I CCDs.
• HRC-I: Improved time dependent QE calibrations.

Version 4.16.0 of the CIAO Scripts Package has been released. This release includes

• energy_hue_map to create "true color" images (rather than the traditional "tri-color" images)
• several alternative/adaptive binning tools including hexgrid, vtbin, centroid_map, pathfinder, and mkregmap
• updates to dax to improve menus and user feedback
• blanksky improvements for VFAINT mode handling
• specextract improvements to handling MARX generated event files and dealing with ancillary file.
• srcflux has a new option to create regions better suited for point sources especially in crowded fields.
• Several scripts have been updated for changes to sherpa plotting.

For more details see the Analysis Script section of the release notes.

CALDB version 4.10.7 has been released with new time-dependent HRC-S QEU files.

CIAO 4.15.2 contains updates to the Obsvis observation planning software and calibration including updates for ACIS subarray observations, ACIS dither amplitude, and backwards compatibility fixes.

Version 4.15.3 of the Contributed Scripts and Modules has been released; download the updated package from the Scripts page. CIAO 4.15 users, especially those using the ACIS blank sky background files, are encouraged to download the latest version. This update includes a bug fix in specextract when dealing with ACIS blank sky background files affecting the normalization (via the AREASCAL keywords), updates to the search_csc and obsid_search_csc scripts to allow access to the Master source polygons for extended sources as well as file downloads for the "current" database view, and some internal dax updates.

Version 4.15.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.15 users are encouraged to update this version of the package. This update provides minor enhancements and bug fixes to many scripts but also introduces the get_dither_parameters script and better support of converting XSpec XCM files to Sherpa with convert_xspec_script. A complete list of changes is available on the contributed scripts history page.

CALDB version 4.10.4 has been released with new time-dependent ACIS Gain (TGAIN) files for February 2022-January 2023 (Epochs 89+90+91+92) and updates to the Epochs 87+88 TGAIN files. Going forward, these calibration products will be released on an annual, rather than semi-annual, basis.

The CXC is pleased to announce the availability of CIAO 4.15.1. This release includes:

1. Fixes to support Numpy 1.24.

2. SAOImage DS9 8.4.1 with fixes to plotting from the Prism window.

Download and upgrade instructions are available on the Download page.

The CXC is pleased to announce the availability of ciao-contrib 4.15.1. This release includes:

1. Fixes to support Numpy 1.24.

2. color_color tool to compute hardness ratio diagrams.

3. combine_spectra bug fixes when setting the src_arfs parameter.

4. specextract bug fixes including pixel mask filtering, history, and improving support for blank sky files.

5. Fixed plotting issue with sherpa_contrib.profiles routines

6. convert_xspec_user_model bug fixes and initial support for compiling with udmget for FORTRAN models.

Download and upgrade instructions are available on the Download page.

(31 jan 2023) These issues have been resolved with the CIAO 4.15.1 patch.

An update for CIAO is planned for January to address the following issues:

1. NumPy 1.24 has just been released onto conda-forge and it is incompatible with CIAO. A fix is to ensure that NumPy version 1.23 is used instead by specifying numpy<1.24 when installing CIAO, as described on the Watch Out page. This is only a problem for conda users, as ciao-install comes with an older version of NumPy.

2. SAOImage DS9 version 8.4 does not let users plot data with the prism option (that lets users view the contents of table data). An updated version (8.4.1) is available from the SAOImage DS9 home page.

The CXC is pleased to announce the availability of CIAO 4.15. This release includes several important updates and bug fixes as well as support for the latest operating systems.

Notable changes include

• wavdetect updates to improve robustness of detecting low significance sources and to remove artificial duplicate sources.
• tg_resolve_events now properly resolves HRC events in the HEG/MEG overlapping region.
• Sherpa has been upgraded to use the XSPEC 12.12.1c models library.
• dmimgcalc can now be used to specify an arbitrarily long expression by using stacks.
• Improved uniformity including scripts adding HISTORY records, images recognizing NULL values, and copying world coordinate systems (WCS) from images to tables.

CIAO 4.15 is available for the following platforms:

• 64-bit Linux distributions, CentOS7 / Ubuntu 18.04 era and newer
• macOS x86 and ARM (M1/M2, using Rosetta2 emulation), Catalina through Ventura

CIAO 4.15 can be installed using conda or using the ciao-install script.

See the release notes for more details and a complete list of changes.

The CXC is also pleased to announce the release of the CIAO 4.15.0 contributed scripts. This release includes several new scripts and important updates, including

• psf_contour creates source regions based on simulated PSFs.
• bkg_fixed_counts creates background regions which enclosed a specified number of counts.
• aplimits computes upper limits for false detection and missed detections.
• statmap computes a simple statistic (min, max, mean, median, etc) of the a column in an input table based on the input map file (eg as output from dmnautilus or dmradar.
• specextract now, by default, uses the center of the extraction region when computing PSF fractions and unweighted ARFs. The new resp_pos parameter allows users to select different algorithms to determine the response position, including the previous MAX algorithm (location of pixel with maximum value).
• srcflux supports user plugins to perform custom analysis.
• dax added an interactive tool to compute grating coordinates.

For more details and a complete list see the Analysis Scripts release notes.

Highlights include:

• New layout views: Basic and Advanced
• New Illustration: users can add graphics for presentations
• Fade between images, save as GIF movies
• Improvements in access to regions' analysis results via XPA and SAMP.

For a complete list of changes see the SAOImageDS9 version 8.4 Release Notes page.

MARX 5.5.2 incorporates the latest calibration updates.

Visit the MARX website for details.

CALDB version 4.10.2 has been released with a new ACIS contamination model, which affects observations from 2020 and beyond.

Version 4.14.4 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. This update provides advice to users of recent warm-temperature observations in specextract if necessary and enhancements to the convert_xspec_user_model script.

Version 4.13.3 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. search_csc and obsid_search_csc has been updated to accomodate queries to the CSC 2.1 database—which is presently undergoing processing— by using the catalog=current argument.

CALDB version 4.9.8 has been released with new time-dependent HRC-S QEU files.

Version 4.14.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.14 users are encouraged to update this version of the package. This update provides a minor bug fixes and several enhancements to dax, chandra_repro, and srcflux. A complete list of changes is available on the contributed scripts history page.

CALDB version 4.9.7 has been released with new time-dependent ACIS Gain (TGAIN) files for August 2021-January 2022 (Epochs 87+88) and updates to the Epochs 85+86 TGAIN files.

Version 4.14.1 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.14 users are encouraged to update this version of the package. This minor update is primarily driven by bug fixes of regressions in specextract affecting the usage of blanksky background files and region parsing when handling user-reduced data of Solar System object observations. A complete list of changes is available on the contributed scripts history page.

CIAO 4.14 contains several important bug fixes and enhancements.

• conda is the preferred installation method which now requires using the conda-forge channel. Users can still install CIAO with the ciao-install.
• There are two bug fixes related to creating ARFs and exposure maps affecting several tools: mkarf, mkgarf, mkwarf, mkinstmap. There is also a bug fix for dmextract which affects how exposure time is computed for lightcurve bins.
• There is a new adaptive binning tool: dmradar, the polar counterpart of dmnautilus.
• sherpa has several upgrades including support updating to XSPEC 12.12.0, changes to sample flux, updates to how filtering and grouping are done, and continued development of the plotting commands.

The contributed scripts package includes an updated version of specextract which has been upgraded to support parallel processing. The new find_mono_energy script can be used to compute an appropriate monochromatic energy for an energy band given an ARF and RMF and a spectral model. merge_obs, reproject_obs, and flux_obs now create individual field-of-view (FOV) files along with a merged FOV file. dax now installs several common tasks into the new ds9 analysis button bar

See the release notes for more details.

SAOImage DS9 v8.3 has been released. It includes a number of enhancements including:

• Individual color bars per frame (optional), along with individual horizontal and vertical graphs.
• Additional matplotlib colormaps.
• New notebook widget lets users write text that can be saved separately and is included in the ds9 backup file.
• Analysis tasks can now be added directly to the ds9 button bar.
• In addition to various bug fixes.

For more information check out the full set of release notes.

CALDB version 4.9.6 has been released, with new time-dependent ACIS Gain (TGAIN) files for -120 C ACIS observations since 01 February 2021. New HRC QE/QEU files have been updated with the latest information of detector degradations from calibration observations and predictive files have been added to coincide with the most recent HRC-I and HRC-S high-voltage changes. New HRC-S time-dependent gain maps have been included, providing better background event filtering, as the prior version resulted in excessive removal of valid events.

Version 4.13.3 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. chandra_repro has been updated to accomodate the new CalDB 4.9.6 for reprocessing HRC-S observations.

Version 4.13.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.13 users are encouraged to update this version of the package. This update provides a minor bug fixes to specextract and several enhancements to dax. A complete list of changes is available on the contributed scripts history page.

CALDB version 4.9.5 has been released with new time-dependent ACIS Gain (TGAIN) files for August 2020-January 2021 (Epochs 83+84) and updates to the Epochs 81+82 TGAIN files. New time- and temperature-dependent CTI corrections for ACIS-S3 non-graded observation modes and graded CC-mode observations for observations since January 2020 has been added.

Version 4.13.1 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.13 users are strongly encouraged to update this version of the package.

While there are several minor bug fixes, the main drive behind this release is to address the ACIS frame-store shadow, which were flagged as badpixels starting with CIAO 4.12/CALDB 4.9.0 and ASCDSVER 10.8.3 for the standard processing pipeline. However, due to an oversight, the newly flagged status bit is not automatically recognized by the tools which create the instrument responses, and the contributed scripts have been updated to apply the necessary filter prior to generating the responses. For more information, see: Caveats about ACIS Frame Store Shadow Bad Pixels

CIAO 4.13 is primarily a bug fix and maintenance release. Noteable changes include:

• conda is now the preferred installation method. Users can still install CIAO with the ciao-install.
• sherpa has several upgrades including support updating to XSPEC 12.10.1s, improved integration with jupyter by providing rich display elements (HTML tables, plots), improvements to plotting, bug fixes to PHA data handling (specifically variable BACKSCAL arrays), and flux calculations.
• Several tools have significant updates including dmcoords support for Chandra grating coordinates, wcs_match improvements to provide an interactive mode, and a new tool asp_offaxis_corr which applies the boresight corrections directly to the aspect solution.

The contributed scripts package includes update to srcflux which now computes combined fluxes from multiple observations, chandra_repro updates to use new asp_offaxis_corr tool to apply boresight correction to the aspect solution, and several other new utilities and helpful updates.

See the release notes for more details.

The Chandra CALDB 4.9.4 has been released. It includes:

• Version 14 of the ACIS contamination model.
• Updated HRC-S time dependent QUE files.
• Updated HRC-S/LETG PI background filter file

See the How CALDB 4.9.4 Affects Your Analysis section of the release notes for more details.

SAOImage DS9 v8.2 has been released. It includes a number of enhancements including:

• Support for various TTK themes including Dark Mode.
• Significant improvement to plots including new Plot Control Panel to make adjusting plot properties easier.
• A new data exploration tool called Prism is now available which has much of the same functionality as the CIAO prism application.
• New support for Chandra and HST footprints.
• New blending modes for masks.
• Users can now import FITS tables directly into the catalog tool.

For more information check out the full set of release notes.

MARX has been updated to match CALDB 4.9.4 released with CIAO 4.13. See Release Notes.

CALDB version 4.9.3 has been released with new time-dependent ACIS Gain (TGAIN) files for February-July 2020 (Epochs 81+82) and updates to the Epochs 79+80 TGAIN files.

Version 4.12.4 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

This update provides a significantly enhanced version of dax, with a list of changes available on the contributed scripts history page. dax, the DS9 Analysis eXtensions, is a collection of CIAO tasks that have been integrated with SAOImage DS9 using DS9's analysis menu interface. dax allows users to access many CIAO tools and applications using a simplified graphical user interface. With dax users can perform various image processing operations (eg adaptive and non-linear smoothing), compute net counts and fluxes, perform source detection, and even modeling and fitting of spectra and 2D images using Sherpa.

CIAO 4.12.1 has been released. With the release of Sherpa 4.12.1, a patch to Sherpa in CIAO 4.12 fixes issues which had the potential of impacting users by either stopping an analysis session or providing incorrect results. Apart from Sherpa, CIAO 4.12.1 is identical to CIAO 4.12.

CALDB version 4.9.2.1 has been released, with new time-dependent ACIS Gain (TGAIN) files for -120 C ACIS observations since 29 January 2000. New predictive HRC QE files have also been added to account for reduction in QE for LETG observations and better calibrated energy scales in addition to improved HRC-S time-dependent gain maps have been included, which requires enhancements to hrc_process_events introduced in CIAO 4.12.

Version 4.12.3 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. While there are several minor bug fixes, the main drive behind this release are changes to CalDB 4.9.2.1.

Version 4.12.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page. CIAO 4.12 users are strongly encouraged to update this version of the package.

While there are several minor bug fixes, the main drive behind this release are the CXC's move to using https rather than FTP to provide data access and the underlying changes to the SDP software to DS 10.8.3 on 2020 March 24 which has changed how the aspect solution is generated for new observations and in anticipation of Repro V of the Chandra archive.

CALDB version 4.9.1 has been released. It includes

• New time-dependent ACIS Gain (T_GAIN) files for November 2019-January 2020 (Epochs 79+80) and updates to the Epochs 77+78 TGAIN files.
• New ACIS Gain Map (DET_GAIN) files has been added, correcting for the reduction in gain found in the mid-chip regions of the front-illuminated CCDs.
• New predictive HRC-S QEU has been added to account for the reduction in the detector's QE.

Due to the format changes, CIAO 4.12 users must use CALDB 4.9.0 or later. Similarly CIAO 4.11 and earlier users must not update to CALDB >4.9.0; doing so can lead to less accurate calibrations being applied.

CIAO 4.12 has been released. Significant updates include

• Removal of the ChIPS plotting package. The CXC has prepared a ChIPS to Matplotlib conversion guide to assist users to convert their sherpa and plotting scripts.
• Updates to the event processing tools to support new calibrations. This includes marking the bottom few rows of each ACIS CCD as bad pixels due to the frame store shadow.
• An experimental conda distribution provides an alternative approach to installing CIAO.
• Sherpa updates include plotting improvements, updated models to XSpec 12.10.1n, and more.
• Sundry updates and enhancements to various other tools, libraries, and python modules & packages.

CALDB version 4.9.0 has been released. It includes

• An update to the ACIS badpixel files to include the frame store shadow region (the bottom few rows of each CCD). This affect TIMED mode datasets.
• A format change to the ACIS temperature dependent CTI calibration files (no effect on current calibration).
• An update to the HRC degap files to address a technical issue with recently introduced keywords.

Due to the format changes, CIAO 4.12 users must use CALDB 4.9.0 or later. Similarly CIAO 4.11 and earlier users must not update to CALDB 4.9.0; doing so can lead to less accurate calibrations being applied.

SAOImage DS9 v8.1 has been released. Higlights include

• New support for overlaying masks based on world, physical or image coordinates. Each mask is rotated, scaled, and centered upon the specified coordinate system. There is no limit to the number of masks which can be displayed.
• An all new plot tool is now available. Each plot tool dialog can contain one or more graphs, arranged in row, column, or grid order. In addition, a new strip plot layout is also available. Support for line, bar, or scatter graphs is provided. Each graph has its own set of display parameters and can contain one or more data sets. Each plot dialog maybe printed as a postscript image, or exported as tiff, png, jpeg, or gif.
• The movie feature now supports animated GIFS. 3D data cubes, or a sequence of multiple frames can be animated, and saved as an animated GIF to be displayed on the desktop, in twitter or on a web site.
• The MacOS Aqua port now supports 'Dark Mode' for MacOS Mojave and Catalina users. The appearance of all dialogs, menus, and displays will be rendered based on the current MacOS appearance settings.
• Contours have been reimplemented to provide high quality resolution and have been threaded to take advantage of multiple CPUs, providing fast rendering, especially in the case of 3D data cubes.

MARX 5.5.0 has been reelased. This marx release includes the following changes:

• Update CalDB files that are shipped with marx to CalDB version 4.9.0.
• Change column type of PHA and PI columns written by marx2fits to float32 to match the files that CIAO writes.
• User-compiled sources (SourceType="USER") have a function called user_create_ray which is called with a pointer to ray properties (direction, time, and energy). All ray properties used to be undefined before calling this function and the user function could return -1 for energy and time to have marx assign energy and time. Now, energy is initialized before user_create_ray is called, to allow user code that assigns locations based on the ray energy. With this change, marx.par must contain a valid specification for the source spectrum, since an energy will be drawn, even if the user source later assigns a different value. For backwards-compatibility, if user_create_ray sets the energy to a negative value, marx will assign the energy drawn before to the variable again.

CALDB version 4.8.5 has been released. The contamination model for the optical blocking filter has been updated to version N0013, affecting recent ACIS-S and ACIS-I observations, particularly of extended source, since 2018 due to an increased build up of the absorbing layer than had been predicted in earlier models.

Version 4.11.5 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

This update is primarily driven by the release of the Chandra Source Catalog 2.0, affecting the scripts search_csc and obsid_search_csc. This release also includes several bug fixes; of note is the handling of non-Chandra lightcurve files with deflare, which was broken during the migration to matplotlib.

CALDB version 4.8.4.1 has been released with corrected time-dependent ACIS Gain (TGAIN) files for August 2018-July 2019 (Multi-Epochs 75+76 and 77+78) that were introduced in CalDB 4.8.3 and 4.8.4.

A previously undetected error in the calibration products for the T_GAIN files in these combined epochs has be corrected, which affected ACIS chips 0, 1, 2, 3 and 6 (I0, I1, I2, I3, and S2). This error produced alternating columns of zero correction values for the CalDB T_GAIN files in their DELTPHA2 and DELTPHA1 vector columns. Therefore, the newly corrected T_GAIN affects observations taken since DATE-OBS=2018-05-02T12:00:00. Other observations are unaffected by these new calibration files.

Users with affected observations will want to run chandra_repro and reprocess the data to correct the T_GAIN calibration, as this error was propagated through to the SDP. The DATE-OBS header keyword records the observation start date. The error was detected by 05 September 2019, and corrected in processing on 2019-09-06T17:31:43 (UTC). Any data sets processed in SDP after that date are unaffected by the error. All affected ObsIDs will be reprocessed for the archive using the correctioned T_GAIN files released in CalDB 4.8.4.1.

DS Operations have determined that 302 OBS_IDs have been affected in processing by the errors in T_GAIN, including 127 ACIS-I observations, and 175 ACIS-S observations that used ACIS-6 (S2). Among these are seven ACIS/Grating observations which are on the public archive. All of these OBS_IDs will be reprocessed for the archive using the correctioned T_GAIN files released in CalDB 4.8.4.1. Users and PIs will be notified in a timely fashion of these errors and their appropriate reparations to the data.

CALDB version 4.8.4 has been released with new time-dependent ACIS Gain (TGAIN) files for February-April 2019 (Epochs 77+78) and updates to the Epochs 75+76 TGAIN files. New predictive and revised HRC-S QEU files have also been added to account for the reduction in QE from recent HRC-S/LETG calibration observations.

CALDB version 4.8.3 has been released, with new time-dependent ACIS Gain (TGAIN) files for August 2018-January 2019 (Epochs 75+76) and updates to the Epochs 73+74 TGAIN files. New predictive HRC-S QEU and degap files have also been added to account for reduction in QE for LETG observations and better calibrated energy scales.

Version 4.11.3 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Several updates have been made to the scripts and module package, the most important of which and bug was fixed in srcflux which had been over-estimating the net_photflux and net_flux values by ~5 to 15% for typical cases.

The CIAO Data Analysis in DS9 system (dax), has also been overhauled, with the following highlights:

• files created by the tools can now be saved to a user-specified directory, so users can access the data products
• plots are now created with DS9 rather than using ChIPS
• a number of new tasks have been added, as well as improvements made to existing tasks, such as: users can now choose which statistic to use in spectral its with Sherpa; there has been improvements to supporting DS9 regions in CIAO tools; and a number of tools will now use the data as shown in DS9 (e.g. smoothing) rather than the input file

Version 4.11.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Highlights of the updates to the scripts and module include:

• chandra_repro added support for event files with no exposure time and improved generation of FOV files.
• deflare updated to use matplotlib, improving support for running script remotely and on headless systems.
• find_chandra_obsid workaround for CERTIFICATE_VERIFY_FAILED errors seen on MacOS and certain OpenSSL builds.
• blanksky_image bug fixed in parsing the data model subspace to recognize chip filters applied to input file.
• sherpa_marx bug fixed so the output flux is appropriately normalized for use with MARX.

CIAO 4.11 has been released. It includes support for latest operating systems, various bug fixes, and off-the-shelf package updates. The popular Project jupyter notebook system and matplotlib are now included in CIAO; many users will also find it easier to use pip3 to install additional 3rd party Python packages into their CIAO distribution. CIAO 4.11 includes all the sherpa 4.10.2 changes (including XSpec models v12.10.0e).

CALDB version 4.8.2 has been released. It includes the annual updates to the HRC gain and QEU files. It also includes the PIMMS files for use in Cycle 21 proposal planning.

SAOImage DS9 version 8.0 has been released. It includes full support for FITS WCS papers I, II, and III along with legacy support of several non-standard WCS conventions. The command parsers have also been rewritten with improved error reporting.

Version 4.10.3 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

The primary reason for this release are updates to find_chandra_obsid and search_csc due to a change in the URL used for the name resolver provided by the CADC to avoid problems with certain name searches and added support to fall over to curl or wget as the Chandra Data Archive has moved to using https. Minor enhancements and bug fixes have been made for an assortment of scripts.

CALDB version 4.8.1 has been released. The contamination model for the optical blocking filter has been updated to version N0012, affecting recent ACIS-S and ACIS-I observations, particularly those since 2016, due to a slower build up of the absorbing layer than had been predicted in earlier models.

CALDB version 4.8.0 has been released, with new time-dependent ACIS Gain (TGAIN) files for February 2017 - July 2018 (Epochs 73+74) and updates to the Epochs 71+72 TGAIN files. Headers have also been updated for early-epoch non-CTI corrected TGAIN files; this particular update does not affect typical use cases.

CALDB version 4.7.9 has been released. The contamination model for the optical blocking filter has been updated to version N0011, affecting recent ACIS-I observations, particularly those since 2016, due to a slower build up of the absorbing layer than had been predicted in earlier models. New HRC-I QE and HRC-S QEU files have been added, accounting for degradation of the detectors' QE, affecting all HRC observations.

Version 4.10.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

The primary reason for this release are updates to find_chandra_obsid due to the new location and security updates to the Chandra Footprint Server. Unfortunately, not all versions of CIAO 4.10 contain a Python that supports SSL, and in these cases the script will try to use the curl and wget command-line tools to access the data.

CIAO 4.10 has been released . It includes various bug fixes, support for newer compilers, and updated off-the-shelf packages. It also includes all the sherpa 4.9.1 changes (including XSpec models v12.9.1n), and new support for pixel mask based filtering.

SAOImage DS9 v7.6 has been released. It includes new ports for 32bit and 64bit Windows platforms and macOS HighSierra. It includes new features such as elliptical Gaussian smoothing and secure https:// support; along with other improvements to XPA support, improvements to compressed FITS formats, better handling of data cubes, HEAPIX image support and multiple other bug fixes.

CALDB version 4.7.8 has been released, with new time-dependent ACIS Gain (TGAIN) files for November 2017 - January 2018 (Epochs 71+72) and updates to the Epochs 69+70 TGAIN files. New degap corrections for HRC-S have also been introduced—applicable to all HRC-S observations, particularly those with gratings inserted—and may be applied with chandra_repro or hrc_process_events.

CALDB version 4.7.7 has been released, with new gain maps and corresponding background spectrum files for HRC-I observations occuring on or later than 2017-09-17 and new HRC-I particle background files for the years 2015-2017.

CALDB version 4.7.6 has been released, with new time-dependent ACIS Gain (TGAIN) files for February - July 2017 (Epochs 69+70) and updates to the Epochs 67+68 TGAIN files. The "Group G" ACIS blanksky background files for CTI-corrected, non-graded mode observations taken since 2012 have been revised, fixing issues related to a bug in a tool used to generate the background files.

CALDB version 4.7.5.1 has been released, with new time- and temperature-dependent CTI correction files for ACIS-S3 and new "Group G" ACIS blanksky background files for CTI-corrected, non-graded mode observations taken since 2012.

Version 4.9.4 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Updates have been made to the scripts and module:

• dax Photometry (srcflux) task now allows user to select the method used to estimate the PSF fraction for their analysis.
• chandra_repro internal fix added to return more informative error messages when handling observations with OBC aspect solutions.
• srcflux modified with internal optimization for cases where a large number of positions are used; this does not affect the output.
• mktgresp updated so temporary aspect histogram files can be cleaned up with Python 3.5.
• apply_fov_limits, get_fov_limits, fluximage, flux_obs, and merge_obs enhanced with new error handling/messages.
• ciao_contrib.runtool support for running the tools axbary, evalpos, mean_energy_map, pileup_map, and wavdetect has been moved to the CIAOToolDirect class.

A new Gallery of CIAO Examples has been published. The gallery includes simple examples of CIAO filtering and binning syntax as well as more advanced examples showing how to create unsharpen masks and how to compute the Gaussian Gradient Magnitude of an image.

Version 4.9.3 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Updates have been made to the scripts and module:

• download_chandra_obsid, download_obsid_caldb, and find_chandra_obsid changed the default FTP server used from ftp://cdaftp.cfa.harvard.edu to ftp://cda.cfa.harvard.edu, as the prior server is being taken out of service in June 2017.
• ecf_calc, deflare, and find_chandra_obsid scripts and the lightcurves.py module have been updated for Python 3.5 compatibility.

CALDB version 4.7.4 has been released, with with new time-dependent ACIS Gain (TGAIN) files for August 2016 - January 2017 (Epochs 67+68) and updates to the Epochs 65+66 TGAIN files. The HRC-S badpixel and QEU files and the HRC-I QE files have also been updated.

Version 4.9.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Updates have been made to the scripts and module:

• chandra_repro fixed screen typos and re-worked applying history log to output files.
• srcflux modified to run fluximage with background=none to avoid double counting the background in srcflux. This change only affects HRC-I observations.
• simulate_psf applies spatial filter to output ray files to match input file sky range for easier visualization.
• mktgresp updated to support input of Type I PHA files.
• tgsplit updated to append the REGION block to the produced Type I PHA file.
• combine_spectra and combine_grating_spectra recomputes the STAT_ERR column of the co-added spectrum using the Gehrels' variance function.
• reproject_obs bug fixed that caused runtime failure in Python 3.5 with multi-asol observations.
• install_marx and ecf_calc updated for Python 3.5 compatibility.
• download_obsid_caldb bug fixed where the script fails when the output directory parameter contains a trailing "/".
• blanksky
  • random parameter added to pass a random seed to reproject_events in order to create reproducible results.
  • The DY_AVG, DZ_AVG, and DTH_AVG header keywords are added to the tailored blanksky background file that match the reference events file.
• specextract bug fixed causing script to exit prematurely when co-adding 3 sets of spectra and responses when bkgresp=no.
• sherpa_contrib.xspec.xsconvolve module has been updated to work with Sherpa in CIAO 4.8 or greater.

Read the release notes

Download CIAO 4.9 and CALDB 4.7.3

CIAO 4.9 is distributed for the following platforms:

• Linux 64-bit (CentOS 6.8/RHEL 6)
• Linux 64-bit (Ubuntu 14.04 LTS)
• Mac macOS 10.12 (Sierra)
• Mac OS X 10.11 (El Capitan)
• Mac OS X 10.10 (Yosemite)
• Mac OS X 10.9 (Mavericks)

CIAO is no longer available for 32-bit Linux operating systems or for older 64-bit Linux machines (CentOS 5 era). CIAO is also no longer available for older versions of OS X (notably 10.6, 10.7 and 10.8). CIAO 4.7 is still available for users who are unable to upgrade. More details on the Platform Support page.

This is primarily a maintenance release: bug fixes, supporting new compilers and OTS upgrades. Notable changes and improvements in CIAO 4.9:

• CIAO pre-built binaries are now provided for a larger number of operating systems: Linux Fedora based systems (such as RedHat, CentOS, Scientific Linux), Linux Ubuntu based systems (such as LinuxMint), and individually for each Apple release: Mavericks, Yosemite, ElCapitan, and Sierra.

• Beta support for Python 3.5. Many parts of CIAO have been updated to work with either Python 2.7 or Python 3.5.

• CIAO includes version 7.5 of SAOImage DS9.

  

  Problems Selecting or Creating Regions in DS9?

  If you are unable to select or create region in DS9, users are reminded that they now need to change the "Edit" mode setting in order to select or create regions in recent versions of DS9. Please see the watchout page for more information on this.

• The major updates for Sherpa are:

  • Improvements to the WStat statistic, save_all function, and internal documentation (expanding use of Python docstrings in lieu of ahelp in Sherpa).
  • Support for Python 3.5 added.
  • XSPEC models have been updated to version 12.9.0o

  Sherpa is also available as a standalone system, accessible from the Sherpa GitHub respository or from the standalone Sherpa page.

• Several tools have been updated; this includes bug fixes and added support for table inputs to dmellipse and expanded file comparison using dmdiff.

• There have been several new scripts released since the CIAO 4.8 release, including: blanksky, blanksky_image, correct_periscope_drift, and simulate_psf. The specextract script has been modified so that if a background response is created, the background ARF will always be weighted; the background RMF will be determined by the weight_rmf parameter.

Changes in CALDB 4.7.3 include

• ACIS OBF Contamination Model Version 10
• ACIS T_GAIN Epochs 65 and 66
• HRC-I GMAP for 2016-09-20 and Later Observations
• HRC-I PIBGSPEC for 2016-09-20 and Later Observations
• PIMMS Cycle 19 Effective Areas

MARX 5.3.2 is also released, upgraded to use the calibration data in CALDB 4.7.3. This is the version that should be used for cycle 19 proposal preparation, especially for users planning to observe with the ACIS detector, where the contamination is rapidly changing.

Install MARX on OSX

OSX users should be aware that there continues to be an incompatibility between MARX and the default C compiler, clang affecting both manual and install_marx installations. While MARX compiles successfully with clang, several problematic issues have been seen in the resulting MARX build. The MARX developers suggest using a true gcc compiler, obtainable at http://hpc.sourceforge.net/ for Max OS X, as noted in Known Bugs and Limitations—Max OS X, and setting the appropriate PATH and CC environmental variables.

Version 4.8.4 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Three new scripts have been added:

• blanksky creates an unscaled blanksky background file compatible with a given event file from an observation, while scaling factors are calculated for each chip and stored in the file header.
• blanksky_image creates scaled background and background-subtracted images for a given blanksky background file and a reference image.
• correct_periscope_drift corrects small (~0.1 arcsec) alignment drifts seen over the course of recent, long (>50 ks) observations. Applicable for sub-pixel analysis of bright, on-axis (<2 arcmin) point sources with minimal pileup.

Updates have been made to the scripts:

• specextract
  • binarfcorr parameter added to explicitly specify detecter pixel size used by PSF correction.
  • Additional checks and warnings added for warm observations.
  • Bug fixed to avoid inadvertent failure when weight=no and mkacisrmf can be used for the observation.
• srcflux
  • Systems with MARX installed can now use psfmethod=marx to simulate a mono-chromatic PSF that is used for aperture correction.
  • Multiple event files can be used as input, but the results are not merged.
  • The default model and absmodel have been updated to separate the absorption model.
  • A new ${root}_summary.txt file is created with a copy of the final summary information.
• chandra_repro changed where pix_adj=default for ACIS data in CC-mode will now run acis_process_events with pix_adj=NONE.
• mktgresp can now create responses for ACIS-I observations with gratings inserted.
• tgsplit fixed an error when using tgextract2 pha2 files which were created with error=gehrels.
• monitor_photom script cleaned up to avoid warnings thrown by newer NumPy versions.
• dax tasks updated to request regions in a specific format.
• install_marx updated to correct a problem in the auto-generated bash-shell setup script.

CALDB version 4.7.2 has been released, retracting the N0005 version of the ACIS grade file introduced in CALDB 4.6.5. This change only affects graded CC-mode observations taken since 2009 November 1. The N0005 grade files mapped the flight grade 66 events as good ASCA grade 2 events in the level 2 events files, but further experience using these events has shown that they only introduce background noise to the resulting spectra in this mode. Subsequently, the previous grade mapping will be used for CC-mode data, where the flight grade 66 events will be filtered out in the level 2 events files.

Version 5.3 of MARX (the Chandra on-orbit simulator) is now available to download. Previous version of MARX (5.0-5.2) contain a bug that affects the PSF for off-axis source simulations. If simulating any source more than 3 arcmin from the aimpoint, please upgrade to MARX 5.3. For details on the issue, see MARX PSF wrong for off-axis sources (#21).

Note that on-axis sources and simulations conducted with SAOTrace or ChaRT are unaffected by this bug.

Version 4.8.3 of the Contributed Scripts and Modules tarfile has also been released; download the updated package from the Scripts page.

A new script, simulate_psf, simplifies the interface to MARX, allowing to simulate a source PSF for an existing observation while using the header information in the file of the existing observation to setup the simulation and projection. The script also supports projection of ChaRT/SAOTrace-generated rayfiles to the detector-plane.

The install_marx has been updated to pick up the latest available version of MARX and specextract has included additional error messages.

Install MARX on OSX

OSX users should be aware that there continues to be an incompatibility between MARX and the default C compiler, clang affecting both manual and install_marx installations. While MARX compiles successfully with clang, several problematic issues have been seen in the resulting MARX build. The MARX developers suggest using a true gcc compiler, obtainable at http://hpc.sourceforge.net/ for Max OS X, as noted in Known Bugs and Limitations—Max OS X, and setting the appropriate PATH and CC environmental variables.

CALDB version 4.7.1 has been released has been released with new time-dependent ACIS Gain (TGAIN) files for November 2015-January 2016 (Epoch 64) and updates to the Epoch 63 TGAIN files.

Download CIAO 4.8.1/4.8.2

CIAO 4.8.2 is only distributed for Mac OS X 64-bit 10.11 (El Capitan) and is identical to the CIAO 4.8.1 patch, with a fix on a bug introduced by the previous patch affecting the OS X 10.11 (El Capitan) pre-compiled build.

Details of the changes that were previously introduced in CIAO 4.8.1 may be found in the CIAO 4.8.1/4.8.2 Release Notes.

Download CIAO 4.8.1

The primary changes introduced in the CIAO 4.8.1 update are changes to acis_process_events—which fixes a memory allocation issue when handling large data sets that can cause the tool to break—and the upgrade to DS9 7.4. Details of the changes may be found in the CIAO 4.8.1 Release Notes.

Version 4.8.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

A new Python model function has been added:

• sherpa_contrib.utils.renorm changes the normalization of a source model so that the total predicted signal matches the observed data, as an alternative to guessing.

Updates have been made to the scripts and modules:

• fluximage, merge_obs, flux_obs, and reproject_obs "random" parameter added to control the randomization of reprojected events from HRC-I background files.
• readout_bkg "random" parameter added to control the stream of random CHIPY values.
• convert_xspec_user_model support added for models which specify an initialization string in the model parameter file.
• sherpa_contrib.chart module has been updated to be compatible with ChaRT v2.
• sherpa_contrib.utils.estimate_weighted_expmap updated to work with CIAO 4.8.

Read the release notes

Download CIAO 4.8 and CALDB 4.7

CIAO 4.8 is distributed for the following platforms:

• Linux 64-bit
• Mac OS X 64-bit 10.11 (El Capitan)
• Mac OS X 64-bit 10.8 (Mountain Lion) compatible with 10.9 (Mavericks) and 10.10 (Yosemite)

CIAO is no longer available for 32bit Linux operating systems or for older 64bit Linux machines (CentOS 5 era). CIAO is also no longer available for older versions of OSX (notably 10.6 and 10.7). CIAO 4.7 is still available for users who are unable to upgrade. More details on the Platform Support page.

Notable changes and improvements in CIAO 4.8:

• This is primarily a maintenance release: bug fixes, supporting new compilers and OTS upgrades.

• CIAO includes version 7.3.2 of SAOImage ds9: users may find that they are unable to select or create region files using this version due to a change in ds9; please see the watchout page for more information on this.

• The major updates for Sherpa are: the XSPEC models have been updated to version 12.9.0d; the wstat statistic (based on the XSPEC version) which extends the cstat statistic to include a Poisson background component; user statistics now have access to the background of a data set; and this is the first version to be based on the Sherpa GitHub repository.

• ChIPS has been updated to support XQuartz version 2.7.8 on OS-X systems.

• There have been several updates to acis_process_events and tg_resolve_events to allow for better calibrations of ACIS+HETG data taken in Continous Clocking (CC) mode. chandra_repro has been upgraded to take advantage of these changes.

• Updates to reproject_aspect (including wcs_match and wcs_update) and to axbary have significantly simplified their respective threads: Correcting Absolute Astrometry and Apply Barycenter Correction.

• The CXC datamodel has improved support for filtering on string columns and supporting NULL values for logical (True/False) data-types. Filtering and area calculations involving complex polygons (sides intersect) have also been improved.

• There have been several new scripts released since the CIAO 4.7 release, including: readout_bkg, install_marx, download_obsid_caldb, detilt, dewiggle, symmetrize, splitobs, gti_align, and multi_chip_gti. The add_grating_orders and add_grating_spectra scripts have been removed in this release; the combine_grating_spectra script should be used instead.

Changes in CALDB 4.7.0 include

• ACIS T_GAIN Epoch 63 Updates
• ACIS SUBPIX Header Mod for CC-mode
• ACIS QE for CC-mode Obs_IDs before 01 Nov 2009
• HRC-I GMAP for 2015-09-27 and Later Observations
• HRC-I PIBGSPEC for 2015-09-27 and Laer Observations
• HRC-I BKGRND Event Lists for 2014, 2015
• PIMMS Cycle 18 Effective Areas

CALDB version 4.6.9 has been released with new, time-dependent ACIS Gain (TGAIN) files for May-July 2015 (Epoch 62) and updates to the Epoch 61 TGAIN files. Newly generated position-dependent gain maps (DET_GAIN) and corrseponding RMF calibration files (P2_RESP) for the back-illuminated ACIS-S1 have been added, primarily affecting the order-sorting of events in ACIS-S/LETG observations and generated RMFs below 0.5 keV on the S1 chip. The Group F ACIS background files have been also updated, correcting a data type issue that clashed with background files from other groups, triggering warning messages; there is no change in the actual data, otherwise.

For HRC-S/LETG observations, newly calculated line spread functions have been added to the CALDB to improve the signal-to-background ratio in spectral extraction, particularly for narrow region. The change manifests itself in the generated grating RMF. Subsequently, QE calibration files for HRC-S and HRC-I have also been updated since there will be no overall change in the HRC/LETG effective area because changes in the RMFs are compensating changes in the HRC QE, but there are changes of up to ~12% (largest at low-energies) in effective area when the HRC-S or HRC-I are used without a grating.

A new version of the Scripts and Modules Package is also available, fixing a bug in tgsplit affecting CC-mode gratings data and adding several new scripts, including readout_bkg which models the out of time contribution of bright and extended source to the ACIS background. A full list of the new scripts can be found at the scripts history page.

Both updates are available via ciao_install or they can be downloaded individually:

• download the updated package from the Scripts page;
• Download CALDB 4.6.9

The CXC is also pleased to announce the release of ChaRT v2, an interface to the most recent version of SAOTrace. There have been several changes, most importantly, the ability to account for spacecraft dither which is necessary to perform sub-pixel analysis. A summary of changes and details on preparing a ChaRT simulation are described in: Preparing to Run ChaRT.

CALDB version 4.6.8 has been released with new, time-dependent ACIS Gain (TGAIN) files for Febraury-April 2015 (Epoch 61) and updates to the Epoch 60 TGAIN files. Revised ACIS TGAIN files have also been released for the period November 2013-October 2014, primarily affecting data on ACIS-S1 and to a lesser extent, ACIS-I3. Users working with ACIS data taken during these periods, particularly with gratings inserted, may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. This release requires CIAO 4.5 or later for full compatibility.

A new version of the Scripts and Modules Package is also available, fixing bugs in combine_spectra and ecf_calc; and adding multi-ObI observation support to merge_obs/reproject_obs/flux_obs.

Both updates are available via ciao_install or they can be downloaded individually:

• download the updated package from the Scripts page;
• Download CALDB 4.6.8

Users can now find a collection of information and resources on the Chandra PSF at PSF Central.

It is an active work in progress and we expect to add additional information and links as time goes on.

Version 4.7.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Three new scripts have been added:

• splitobs automates the steps necessary to separate multi-ObI and interleaved-mode observations into individual directories compatible with chandra_repro.
• gti_align synchronizes the input GTIs to the ACIS exposure boundaries.
• multi_chip_gti creates a single multi-chip GTI from individual GTIs.

Fixes and updates have been made to the scripts:

• chandra_repro checks the file names in the primary and secondary directories against the file names in the level 1 event file header. Support added for FAINT_BIAS datasets.
• specextract has enhanced error and warning messages and better supports preset ardlib parameters.
• srcflux saves additional per-source data products that is created and used by the script. The spectral model parameters can now take special tokens: %NRAO_NH% and/or %BELL_NH% to replace values reported by the COLDEN tool for the source's location. The per-chip live time is also now used rather than just the LIVETIME value fo the aimpoint chips.
• mktgresp has three new parameters. The orders parameter allows the creation of responses for an arbitrary set of orders, independent of the input pha2 file; and by default, the responses for different orders are created in parallel, controlled by the parallel and nproc parameters.
• combine_spectra fixed for a bug when checking that all input files have the same channel grid.
• coords.chandra Python module includes a new routine to convert from sky(x,y) coordinates to Chandra coordinates.
• dax spectral analysis fixed to allow for background inclusion.

CALDB version 4.6.7 has been released has been released with new time-dependent ACIS Gain (TGAIN) files for November 2014-January 2015 (Epoch 60) and updates to the Epoch 59 TGAIN files.

Users can now follow CIAO on Facebook, Twitter, and Google+. Topics will include

• Announce CIAO and CALDB releases, along with other CXC software such as MARX and SAOTrace.
• Updates when documentation pages are updated: new bugs posted, changes to the analysis threads, updated help files.
• Various CXC and CIAO related announcements: proposal deadlines, dates for science meetings, changes in platform support, etc.
• Other items of interest to the CIAO community.

Users are also reminded of the CIAO RSS feed to receive major announcements and of the CIAO YouTube channel with various CIAO and ds9 tutorials and demos.

Read the release notes

Download CIAO 4.7 and CALDB 4.6.5

Significant changes in CIAO 4.7 include:

• This is primarily a maintenance release: bug fixes, supporting new compilers and OTS upgrades.
• Sherpa now supports stacks of datasets: package for manipulating a stack of related datasets and simultaneously fitting a model to them. It provides stack-enabled (i.e. vectorized) versions of the key Sherpa commands used to load data, set source models, get and set parameters, fit, and plot.
• Sherpa is also using the newer XSPEC model library version 12.8.2e, and under the hood is making use of more robust library interfaces.
• CIAO includes version 7.3.2 of SAOImage ds9. Users will notice some changes to the interface. This is discussed more in the watchout page.
• dmcoords now automatically makes use of the DY_AVG, DZ_AVG, and DTH_AVG keywords in the file headers to correct for the mean SIM drift during an observation. The asolfile parameter is no longer necessary nor required for data processed with ASCDSVER greater than 8.4.2 or has been recalibrated with chandra_repro.
• There have been several new scripts released since the CIAO 4.6 release, including monitor_photom (generates lightcurve from ACA optical monitor data) and combine_grating_spectra (replaces add_grating_orders and add_grating_spectra). There have also been bug fixes and enhancements of existing scripts.

Version 4.6.7 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Two new scripts have been added:

• combine_grating_spectra replaces the add_grating_orders and add_grating_spectra scripts to combine spectra from positive and negative grating orders and co-add spectra across multiple observations or sources. The script is compatible with both Type I and II PHA files and will automatically handle response and background files.
• tgsplit converts a Type II PHA file to a set of Type I source and background PHA files. Positive and negative ordered background spectra are also combined.

Fixes and updates have been made to the scripts:

• specextract has been fixed to avoid failures when creating an unweighted RMF with a weighted ARF for early mission, warm focal-plane (-110C) observations.
• combine_spectra has been fixed to avoid failures if any of the background is zero in the same channel as the input spectra when background scaling method is based on counts. Users can also now specify file types not to combine.
• dax has had several tool names renamed and moved to different menus for clarity. A new Net Counts tool is added to obtain the net counts in each individual region.

Version 4.6.6 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

One new scripts and two new Python modules have been added:

• convert_xspec_user_model is an experimental script that compiles XSpec user-models (additive, multiplicative, and convolution models only) for use in Sherpa.
• sherpa_contrib.xspec.local provides the Python lmod function to load compiled XSpec user-models into Sherpa.
• sherpa_contrib.xspec.xsconvolve is an experimental module providing an interface to the XSpec convolution models in Sherpa.

Fixes and updates have been made to the scripts:

• chandra_repro now creates a FOV file with the other output products and the STATFILT header keyword is added to event files created with 32-bit status.
• combine_spectra updates the ONTIME/LIVETIME/EXPOSURE keywords in the combined ARF.
• fluximage, flux_obs, reproject_obs, and merge_obs have been updated with improved error messages and handling of data with missing keywords. Additionally, merge_obs and reproject_obs is updated to only reproject data if the new tangent point is more than 0.05 arcsec away from the current tangent point.
• specextract has been updated to include a new tmpdir parameter; improved error handling for problems with response creation and corrupt ardlib.
• srcflux has been updated to include a new tmpdir parameter and binsize hidden parameter to support sub-pixel analysis for small regions.
• search_csc and obsid_search_csc have corrected MSBS (master source basic summary) column.
• acis_clear_status_bits fixed for bug affecting 32-bit systems where the CLSTBITS header keyword was being written incorrectly.

XImage-based color lookup-tables have also been updated to contain 256 colors and the bluebase2 lookup-table has been updated to replace invalid values.

CALDB version 4.6.3 has been released has been released with new time-dependent ACIS Gain (TGAIN) files for May-July 2014 (Epoch 58) and updates to the Epoch 57 TGAIN files. New predictive HRC-S QEU files for 2014-2020 is also added to account for reduction in QE for LETG observations and fixes errors selecting the correct HRC-S QEU files for observations made in 1999 and 2014.

CALDB version 4.6.2 has been released has been released with new time-dependent ACIS Gain (TGAIN) files for February-April 2014 (Epoch 57) and updates to the Epoch 56 TGAIN files. The contamination model for the optical blocking filter has been updated to version N0009, affecting observations since early-mid 2013, but is otherwise unchanged from the N0008 model introduced in CALDB 4.5.9 for observations made prior to 2013.

Version 4.6.5 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Three new scripts have been added:

• monitor_photom creates optical light curves from the aspect camera of observations with adat files.
• summarize_status_bits provides an on-screen summary of the status bits set for rows and their meaning.
• apply_fov_limits creates and image from an event file bounded by the sky limits of an FOV file.

Fixes and updates have been made to the scripts:

• srcflux fixed so that background photon flux values in the BG_PHOTFLUX column are correctly calculated, so that the NET_PHOTFLUX_APER and corresponding errors columns are background subtracted.
• combine_spectra rewritten to make use of Python temporary files and include advanced-user options.
• fluximage, flux_obs, and merge_obs have been updated to use event file GTIs to calculate an observation's sky area coverage; for HRC-I background subtraction, scaling by particle flux is an option in addition to exposure time.
• specextract has been updated to better handle multiple aspect solution files for an ObsID and fixed issues with outroots using nested directory paths. Header keywords have been added for galactic HI column densities derived from COLDEN
• download_chandra_obsid has support added to download evt1a and adat file types.

And a new Python module has been added:

• ciao_contrib.region.check_fov.FOVFiles module allows users to query a stack of FOV files to find which observations include a given celestial coordinate.

CALDB version 4.6.1.1 has been released fixing an indexing bug only affecting HRC-I analysis using particle background files performed since 2014 March 26, for HRC-I observations taken between 2012 January 01 to 2014 January 01.

CALDB version 4.6.1 has been released with new time-dependent ACIS Gain (TGAIN) files for November 2013-January 2014 (Epoch 56). Existing HRC-I background files have been upgraded to version N0002, and includes additional files to cover the years 2010-2013, using the latest calibration data. HRC-I and HRC-S RMF file headers have been fixed, where an errant header keyword could cause segmentation faults when using dmcopy with opt=all.

Of particular note, the 2D PSF libraries for ACIS and HRC are retired.

A new version of the Scripts and Modules Package is also available, introducing the ecf_calc script and fixing minor bugs.

• ecf_calc extracts the encircled counts fraction of a source at a given position as a function of radius.
• fluximage cleanup behavior changed for handling instrument maps and for HRC-I data with particle background subtraction.
• chips_contrib.helix module fixed to support more arguments with with the load_colormap_cubehelix() and get_cubehelix() functions.

Both updates are available via ciao-install or they may be downloaded individually:

• download the updated package from the Scripts page;
• Download CALDB 4.6.1

Sherpa has been patched in CIAO 4.6.1 to fix a problem doing analysis in wavelength space (Angstroms) with XSPEC models.

Read the CIAO release notes

Download CIAO 4.6 and CALDB 4.5.9

Notable changes in CIAO 4.6 include:

• Two new tools have been added to help automate the detection of the 0th order location in grating observations: tg_choose_method and tgdetect2
• Sherpa template models have been enhanced to include interpolation between the individual table models. Users can now use templates with all the fit methods (previously only gridsearch was valid).
• The new srcflux script automates the steps necessary to get net counts, rates, and fluxes, with confidence intervals.
• The ACIS parameter-block filename parameter, pbkfile, in mkarf, mkgarf, mkwarf, eff2evt, and mean_energy_map is no longer used. The information is now encoded in the archived data products created with ASCDSVER 8.4.2 and later or reprocessed with chandra_repro. Users with older data who choose not to reprocess their data, can use the new r4_header_update script.
• The suite of hotpixel tools: acis_run_hotpix, acis_find_hotpix, and acis_classify_hotpix has been withdrawn from CIAO. The acis_detect_afterglow tool is now the best tool available to identify cosmic ray afterglows without losing a large fraction of events from bright sources.
• A problem affecting the area calculation when complex regions are selected, has been corrected. This problem affected regions which include multiple overlapping shapes. Simple regions were not affected.

CALDB version 4.5.9 has been released with new ACIS calibration files from an updated QE contamination model that accounts for recent changes in the depth and composition of contaminant on the optical blocking filter. New time-dependent ACIS Gain (TGAIN) files for August-October 2013 (Epoch 55) and a time-dependent HRC-I gain map, and corresponding background PI spectrum for observations, since September 16, 2013 are also included in this release. This release requires CIAO 4.5 or later for full compatibility. The release is available via ciao_install.

Download CALDB 4.5.9

Version 4.5.5 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Minor fixes have been made to the scripts and module:

• search_csc and obsid_search_csc
  • Fixed the retrieval of data products.
  • Fixed use of the combination column definitions in the columns parameter.
• coords.format.deg2dec fixed for problems with small negative-valued declinations.

One new script has been added:

• list_datasetid will provide the LaTeX dataset identifier macros needed by AAS managed publication.

CALDB version 4.5.8 has been released with new, time-dependent ACIS Gain (TGAIN) files for May-July 2013 (Epoch 54). This release requires CIAO 4.5 or later for full compatibility. The release is available via ciao_install.

Download CALDB 4.5.8

Version 4.5.4 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Significant updates have been made to the scripts:

• specextract
  • Ancillary files will use the input file header keyword values by default, and look for the files in their standard sub-directories, unless the files are explicitly specified.
  • The correct parameter has been renamed correctpsf
• chandra_repro
  • The default PI=0:300 filter for non-grating HRC-S data has been removed. Users may apply the DM filter to their input file, if desired.
  • Minor bug fix for when two non-alternating mode event files are found in the secondary/ directory.
  • The creation of the output directory is delayed until the inputs have been verified first.
• download_chandra_obsid and find_chandra_obsid
  • Limited support of mirror sites added for data download.
• merge_obs, reproject_obs, and flux_obs.
  • Improved handling of DM filters included with input files.
  • The bands parameter description has been updated, since the low- and high-energy values are required for ACIS data, while they are not used with HRC observations.
• ahelp documentation added for Python routines used by the scripts, primarily those used by download_chandra_obsid and find_chandra_obsid.
  • identify_name
  • cda_data
  • cda_search
  • coords_format
  • coords_utils

Two new scripts and two Python modules have been added:

• search_csc and obsid_search_csc allows command-line access to the Chandra Source Catalog. Both scripts can be used to retrieve data products and used to create ASCII files of arbitrary sets of CSC columns.
• chips_contrib Python modules
  • The chips_contrib.pyplot—and its dependent, chips_contrib.decorators— module has been introduced, providing limited emulation of the matplotlib pyplot module, providing pyplot commands to be used in ChIPS.

CALDB version 4.5.7 has been released with new, time-dependent ACIS Gain (TGAIN) files for February-April 2013 (Epoch 53) and a new set of non-CTI-corrected ACIS blank-sky background files (Group F) have been added for back-illuminated GRADED-mode observations. The previously released, CTI-corrected, Group F blank-sky files have been upgraded with pseudo-SKY and -DET coordinates to allow for easier visualization. The T_GMAP files for non-gratings HRC-S data have been fixed, where the time-dependent gain maps will be applied with chandra_repro or hrc_process_events by default, instead of the time-independent gain files.

This CalDB release requires CIAO 4.5 or later for full compatibility. The new CTI files will not be used by CIAO 4.4.1 or earlier.

A new version of the Scripts and Modules Package is also available, fixing a bug in merge_obs when stacks of ancillary files are given.

Both updates are available via ciao_install or they can be downloaded individually:

• download the updated package from the Scripts page;
• Download CALDB 4.5.7

Version 4.5.2 of the Contributed Scripts and Modules tarfile has been released; download the updated package from the Scripts page.

Significant updates have been made to the scripts:

• merge_obs, reproject_obs, and flux_obs.
  • Improved handling of HRC data.
  • Event files with OBS_ID keyword of Merged are automatically skipped.
  • Interleaved-mode data can now be combined without explicitly listing e1 and e2 observations.
  • Improved handling of missing ancillary files.
  • The asol argument is checked to make sure an aspect histogram is not used by mistake.
• chandra_repro
  • Interleaved-mode observations reprocessed without the user needing to split the e1 and e2 observations into separate directories.
  • New recreate_tg_mask parameter added. By default, the existing region appended to the original evt2 file is now used rather than recreated using tgdetect and tg_create_mask.
  • When reprocessing an observation using gratings, the ARF and RMF files for each order and arm are now created. They are saved into a tg sub-directory of the output directory.

Three new scripts have been added:

• tgmask2reg and reg2tgmask assist in making custom region extraction masks for tgextract2.
• mktgresp will take a Type II PHA file produced by tgextract or tgextract2 and will create the ARF and RMF for each order and grating arm.

CALDB version 4.5.6 has been released with new, time-dependent ACIS Gain (TGAIN) files for November 2012-January 2013 (Epoch 52) and a new set of CTI-corrected ACIS blank-sky background files (Group F). New ACIS CTI files have been added for observations taken during the period of 2005 September 16 - 2005 October 15, which was affected by a uncontrolled focal plane temperature. Furthermore, the calibration validity end dates have been reset for a previously unidentified set of five observations, on top of the 38 observations fixed in the CALDB 4.5.0 release. The EVTSPLT and TGAIN file selection are fixed for ObsID 6318.

This release requires CIAO 4.5 or later for full compatibility. The new CTI files will not be used by CIAO 4.4.1 or earlier. The release is available via ciao_install.

Download CALDB 4.5.6

Read the CIAO release notes and CALDB release notes for detailed information.

Download CIAO 4.5 and CALDB 4.5.5

Notable changes and improvemetns in CIAO 4.5 include:

• The new tg_findzo tool helps users locate the zeroth order location for heavily piled sources and in the observations that have intentionally masked out the zero order events.
• Various updates related to creating merged datasets, especially larger mosaics that exceed the standard image sizes. This includes updates to reproject_events and dmmerge to adjust the limits based on the data and to dmextract to keep a consistent definition of BACKSCAL regardless of image size.
• sherpa updates includes a comprehensive review of how grouping and filtering interact when both source and background datasets are in use. Also, a new sample_flux routine is available to allow users to compute the flux (including unabsorbed) due to a sub-component of a Sherpa model; this functionality has also been made available in the modelflux tool
• In ChIPS it is now possible annotate plots directly either via the GUI or by right-clicking on plots; text, points, lines, ellipses, boxes, polygon, and even color bars can be added.
• Numerous bug fixes and enhancements were made to many parts of the system including speed enhancements to wavdetect, fixes to ChIPS clobbering existing files, and Obsvis upgrades for number of allowed CCDs.
• As previously announced, the mkpsf tool has been withdrawn from CIAO; users should use ChaRT and MARX to simulate 2D models of the point spread function.

Changes in CALDB 4.5.5 include

• ACIS T_GAIN Epoch 51
• ACIS time-dependent QEU files version N0007
• HRC-I GMAP and PIBGSPEC for Sept 2012
• HRC-S QE versions N0012
• PIMMS Effective Area CY15 files
• Header correction for ACIS Group E BKGRND files
• Header changes to specific ACIS CTI and DET_GAIN files for optimized file selection.

The CIAO scripts package has also been updated for changes in CIAO 4.5. The main changes are:

• the package is now labelled by a number rather than a date;
• the check_ctiapp.sh script has been removed as it is no-longer needed (repro-3 is complete)
• the merge_all, mkBgReg.pl, and mkSubBgReg.pl scripts are deprecated (seethe merge_obs and diffuse emission threads for information on their replacements).

A new version of the Scripts and Modules Package has been released. The main change is the replacement of "merge_all" by merge_obs and reproject_obs, but there are also changes to chandra_repro and the addition of a large number of color look-up tables.

CALDB version 4.5.3 has been released with fixes to the ACIS blank-sky background files released in CALDB version 4.5.2.

Both updates are available via ciao_install or they can be downloaded individually:

• download the updated package from the Scripts page;
• Download CALDB 4.5.3

CALDB version 4.5.2 has been released with upgraded ACIS blank-sky background files, as well as new time-dependent ACIS Gain (TGAIN) files for May - July 2012 (Epoch 50). This release is available via ciao-install.

Download CALDB 4.5.2

The 9th Chandra/CIAO workshop has been scheduled for 22 - 24 April 2013. This three day workshop will feature a series of talks and hands-on sessions for both beginner and advanced CIAO users. Free registration will be required and will open shortly.

CALDB version 4.5.1 has been released with upgraded HRC-S time-dependent gain maps, to be used in reprocessing observations taken in 2012. This release is available via ciao-install.

Download CALDB 4.5.1

Download CIAO 4.4.1, CALDB 4.5, and the Scripts package

The CIAO 4.4.1 release consists of the following changes:

1. Version 2 of the Sherpa package for CIAO 4.4 has been released. Sherpa v2 provides critical bug fixes related to reading in files and is updated to use XSPEC 12.7.1 models. The following new XSPEC additive models have been also added:

   • xseplogpar
   • xsgadem
   • xsvgadem
   • xslogpar
   • xsoptxagn
   • xsoptxagnf
   • xspexmon

2. The CIAO tools package has been updated to version 2. The only change made is to update the internal leap-second library used by CIAO.

CALDB version 4.5.0 has been released with a new focal plane temperature upper-limit, as well as new HRC GAPLOOKUP (degap) tables. Furthermore, the calibration validity end dates have been reset for the 38 affected ACIS ObsIDs, listed in the CalDB Caveats page and five HRC ObsIDs affected by similar problems with their GMAP and HRC-S QEU. This release is available via ciao-install.

Download CALDB 4.5.0

A new version of the Scripts and Modules Package has been released; download the updated package from the Scripts page.

• In chandra_repro, the default value of the check_vf_pha parameter has been changed from yes to no, so that the ACIS VFAINT background-filtering step will no longer be applied to relevant data sets by default. This change was made because the VFAINT filtering is known to remove valid events for sources bright enough to be affected by pile up.

  The auxiliary files - such as bad-pixel, mask, aspect solution - are now copied when the file system does not support soft links. In earlier versions the script would exit with an error in this case.

• The download_chandra_obsid script can now accept CIAO stack files for its obsids and filetypes arguments.

CALDB version 4.4.10 has been released with an upgraded ACIS QE contamination model (version N0007), as well as new time-dependent ACIS Gain (TGAIN) files for Feb - April 2012 (Epoch 49). This release is available via ciao-install.

Download CALDB 4.4.10

A new version of the Scripts and Modules Package has been released; download the updated package from the Scripts page.

• The chandra_repro script has been updated to:

  1. warning messages produced by acis_process_events and hrc_process_events are now written out rather than being hidden;
  2. soft link archive files to output directory rather than copying.

CALDB version 4.4.8 has been released with new time-dependent ACIS Gain (TGAIN) files for 01 Nov 2011 - 31 January 2012 (Epoch 48) . This release is available via ciao-install.

Download CALDB 4.4.8

A new version of the Scripts and Modules Package has been released; download the updated package from the Scripts page.

• The find_chandra_obsid script lets you search the publicly-available Chandra observations by position (RA and Dec, J2000) or name (using the CADC name resolver). It is a command-line wrapper for the Chandra Footprint Service.

  A list of matching observations, including relevant information is returned, or the data can be directly downloaded from the archive (download_chandra_obsid can also be used). Examples:

  unix% find_chandra_obsid 'ngc 1333'
  unix% find_chandra_obsid 12:45:32 -32:30:10
  unix% find_chandra_obsid 350.86 58.81 download=ask 
  

  The ciao_contrib.runtool module has been updated to include support for find_chandra_obsid.

• The combine_spectra script has been updated to:

  1. works around a bug in addresp (no TLMIN4 keyword in the output RMF file) so that combined RMF files can be used in Sherpa and X-Spec;
  2. minor improvements to the handling of temporary files.

• The fluximage script has been updated to:

  1. support setting badpixfile=NONE if you wish to create an exposure map with no bad pixels;
  2. create all temporary files in the directory given by the tmpdir parameter, so that the script can now be run from a directory in which you do not have write permission;

  3. add a COMMENT line to the headers of the files listing the version of the script that was run (to aid in provenance tracking); for instance:

     unix% dmlist 0.5-7.0_2.3_bin8_flux.img header,clean,raw | grep COMMENT | grep fluximage 
     COMMENT      =  fluximage version 14 February 2012 /                     
     

• The specextract script has been updated so that it will not error out if you set outroot to be a stack and had correct=yes and weight=no.

• Two Python modules have been added to support finding (ciao_contrib.cda.search) and downloading (ciao_contrib.cda.data) publicly-available data in the Chandra Data Archive. These modules are used by the find_chandra_obsid and download_chandra_obsid scripts.

• Two Python modules have been added to support some common coordinate tasks:

  1. coords.resolver provides simple access to the CADC name resolver;
  2. coords.format converts between many sexadecimal and decimal degree formats for Right Ascension and Declination values.

A new version of the Scripts and Modules Package has been released; download the updated package from the Scripts page.

• fluximage:

  • HRC-I background is now correctly filtered before subtraction (the application of the STATUS filter was lost in the update of the script for CIAO 4.4).

    If you have used fluximage with HRC-I data in CIAO 4.4, re-run the script to apply the correct background-subtraction. The fluximage command is recorded in the file header:

    unix% dmhistory ciao44_bin32_thresh.img fluximage
    fluximage infile="hrcf00144N005_evt2.fits" outroot="ciao44"
    binsize="32" bands="::1.1" asolfile="" badpixfile="" maskfile=""
    pbkfile="" dtffile="" normalize="no" expmapthresh="1.5%"
    parallel="yes" tmpdir="/tmp" cleanup="no" clobber="yes"
    verbose="1"
    

  • only perform HRC-I background subtraction if the HRC background CALDB package has been installed.
  • setting badpixfile=CALDB uses the bad pixel file from the CALDB rather than the per-observation version.
  • the HRC-I instrument map is now created with the same binning as the exposure map rather than the native HRC pixel scale.

• chandra_repro: improved error checking on the input directory.

A new version of the Scripts and Modules Package has been released; download the updated package from the Scripts page.

• splitroi: a new script for splitting the source and background regions created by the roi tool. See the An Image of Diffuse Emission thread for an example of its use.

• The crates_contrib.utils module has three new routines:

  • make_image_crate
  • add_colvals
  • add_keyval

  The make_image_crate and write_columns routines now accept dictionaries or structured arrays as input, and will convert arrays of np.int32 values to np.int64 to work around a CRATES bug.

Read the CIAO release notes and CALDB release notes for detailed information.

Download CIAO 4.4 and CALDB 4.4.7