CIAO 4.9 Release Notes

---

CIAO 4.9 is distributed for the following platforms:

• Linux 64 bit (CentOS 6.8 / Red Hat Enterprise 6)
• Linux 64 bit (Ubuntu 14.04)
• Apple OS X 10.9 (Mavericks)
• Apple OS X 10.10 (Yosemite)
• Apple OS X 10.11 (El Capitan)
• Apple macOS 10.12 (Sierra)

Users can install CIAO with either Python 2.7 or with Python 3.5. Both Python 2.7 and Python 3.5 versions can be installed but must be installed into separate directories.

CIAO is no longer available for 32bit Linux operating systems or for older 64bit Linux machines (CentOS 5 era). CIAO is also no longer available for older versions of OSX (notably 10.6, 10.7, nor 10.8). CIAO 4.7 is still available for users who are unable to upgrade. More details on the Platform Support page.

Notable changes and improvements in CIAO 4.9:

• CIAO pre-built binaries are now provided for a larger number of operating systems: Linux Fedora based systems (such as RedHat, CentOS, Scientific Linux), Linux Ubuntu based systems (such as LinuxMint), and individually for each Apple release: Mavericks, Yosemite, ElCapitan, and Sierra.

• Beta support for Python 3.5. CIAO have been updated to work with either Python 2.7 or with Python 3.5.

• This is primarily a maintenance release: bug fixes, supporting new compilers and OTS upgrades.

• CIAO includes version 7.5 of SAOImage ds9. Users are reminded that they now need to change the Edit mode setting in order to select or create regions in recent versions of ds9. Please see the watchout page for more information on this.

• Sherpa has seen: improvements to the WStat statistic, save_all function, and internal documentation (the Python docstrings); minor bug fixes; and updated to support Python 3.5. The XSPEC models have been updated to version 12.9.0o. As a reminder, Sherpa is also available as a stand alone system, accessible from the Sherpa GitHub repository or from the standalone Sherpa page.

• Several tools have been updated. This includes adding support for table as inputs to dmellipse and expanded comparisons using dmdiff. There have also been various bug fixes.

• There have been several new scripts released since the CIAO 4.8 release, including: blanksky, blanksky_image, correct_periscope_drift, and simulate_psf.

---

• How CALDB 4.7.8 Affects Your Analysis
• How CALDB 4.7.7 Affects Your Analysis
• How CALDB 4.7.6 Affects Your Analysis
• How CALDB 4.7.5.1 Affects Your Analysis
• How CALDB 4.7.4 Affects Your Analysis
• How CALDB 4.7.3 Affects Your Analysis
• Installation
• Python 3
• Tools
• Parameter Files
• ChIPS
• Sherpa
• Graphical User Interfaces
• Analysis Scripts
• Python Modules
• Libraries
• Environment
• Documentation

---

How CALDB 4.7.8 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data

  The new, combined time-dependent ACIS Gain (T_GAIN) file for August-October 2017 (Epoch 71) and November 2017-January 2018 (Epoch 72) are introduced. The combined Epochs 69+70 file has also been updated since CALDB 4.7.6. Therefore, the new T_GAIN affects observations taken since 2017 May 02. Other observations are unaffected by these new calibration files.

  Users working with ACIS data taken since 2016 November 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.

  See the time-dependent ACIS T_GAIN why page for more information.

  The gain corrections are at the nominal and expected levels of less than 1% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.

  Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.

HRC-S Data

• HRC-S GAPLOOKUP (degap) Upgrades

  The HRC-S calibration team has developed a correction to the existing GAPLOOKUP table for the degap solutions (previously released in CalDB 4.5.0), which introduced a small offset in the positions of photons near the aimpoint. This new version N0004 DEGAP file corrects the issue, by including a shift of +2.5 pixels (approximately 0.018 Å) to all degap corrections (in CRSV) near the aimpoint position.

  The new degap solution reduces a wavelength shift derived from HRC-S/LETG observations of Capella. A full explanation of the new degap correction is given on the HRC Calibration Web page at DEGAP Update for HRC-S2; a collection of degapping memos, reports, and presentations; and a general overview on HRC degapping can be found on the HRC Degap why page.

  It is recommended that all archival HRC-S data be reprocessed with the new GAPLOOKUP tables, including the more recent HRC-S data that have already been reprocessed with the previous GAPLOOKUP table.

---

How CALDB 4.7.7 Affects Your Analysis

HRC Imaging Data

• HRC-I Background Event Files

  New HRC-I background event files are included in this release, being generated using the latest level 1 calibration data, including SAMP-based gains (GMAPs) and the latest DEGAP corrections. The newly introduced background files cover the years 2015-2017. All users performing HRC-I imaging or spatial analyses, e.g. when creating a fluxed image, are encouraged to download and use the background files.

  The HRC-I background event files are not included in the main CALDB tarfile. There is a separate HRC background event tarfile available via ciao-install or from the the the CALDB Download page.

  For instructions on using the background event files, follow the The HRC-I Background Event Files thread.

• HRC-I Gain Maps

  The latest time-dependent HRC-I Gain Map (GMAP) and the corresponding PI background spectrum—to be applied to observations after 2017 September 17, have been released.

  Note that HRC-I gain maps are only useful to observers doing hardness ratio or PI spectral analysis studies with HRC-I.

  As the HRC gain continues to slowly decline, the gain maps are updated annually. The change in gain correction is less than 1% on-axis and roughly 5-10% off-axis. Towards the edge of the microchannel plate, the change is up to 15%. Technical details on the gain maps are described in the gain maps section of the HRC calibration website and the in the HRC-I Gain Map Why Document.

  The corresponding background spectrum applicable to the same HRC-I GMAP time period have been generated.

  Users working with HRC-I data may wish run chandra_repro to reprocess the data and improve the gain calibration.

---

How CALDB 4.7.6 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data

  The new, combined time-dependent ACIS Gain (T_GAIN) file for February-April 2017 (Epoch 69) and May-July 2017 (Epoch 70) are introduced. The combined Epochs 67+68 file has also been updated since CALDB 4.7.4. Therefore, the new T_GAIN affects observations taken since 2016 November 01. Other observations are unaffected by these new calibration files.

  Users working with ACIS data taken since 2016 November 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.

  See the time-dependent ACIS T_GAIN why page for more information.

  The gain corrections are at the nominal and expected levels of less than 2% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.

  Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.

• ACIS Blank-sky Background File (Group G ACIS BKGRND)

  A revised set of ACIS blank-sky background files have been released for observations starting January 2012. This set of files removes spurious events that produced visible "hot spots" in images or non-local distribution of extra events in the N0001 version of the 6i, 2s, and 5s files due to a bug in one of the background compilation routines. All background files have been subsequently regenerated.

  acis2sD2012-01-01bkgrnd_ctiN0002.fits
  acis3sD2012-01-01bkgrnd_ctiN0002.fits
  acis5sD2012-01-01bkgrnd_ctiN0002.fits
  acis6sD2012-01-01bkgrnd_ctiN0002.fits
  acis7sD2012-01-01bkgrnd_ctiN0002.fits
  acis8sD2012-01-01bkgrnd_ctiN0002.fits
  
  acis0iD2012-01-01bkgrnd_ctiN0002.fits
  acis1iD2012-01-01bkgrnd_ctiN0002.fits
  acis2iD2012-01-01bkgrnd_ctiN0002.fits
  acis3iD2012-01-01bkgrnd_ctiN0002.fits
  acis6iD2012-01-01bkgrnd_ctiN0002.fits 
  

  The Using the ACIS "Blank-Sky" Background Files thread contains instructions on how to select a file and match it to a specific observation.

  Only CTI-corrected events cases have been included in this release. Stowed background and ACIS back-illuminated chip events, without CTI-corrections for GRADED mode data will be added in future releases.

  Note that the ACIS background files are not included in the main CALDB tarfile. There is a separate background files tarfile available via ciao-install or from the CALDB Download page.

---

How CALDB 4.7.5.1 Affects Your Analysis

ACIS Imaging and Grating Data

• Time- and Temperature-dependent ACIS-S3 (ACIS-7) CTI Correction Files

  New time-dependent corrections to the temperature-dependent ACIS charge transfer inefficiency (CTI) correction files for ACIS-S3 are introduced. The CTI-corrected gain shift at high focal plane temperatures is significantly reduced on ACIS-S3. The new CTI corrections affect non-graded mode ACIS-S3 observations only. There are no CTI corrections for back-illuminated chips in the graded model, and all other observations are unaffected by these new calibration files.

  Users working with ACIS-S3 may wish to run chandra_repro and reprocess the data to improve the CTI gain shift calibration.

  See the ACIS CTI Correction why page for more information.

  The new CTI corrections reduces the systematic drift of the ACIS-S3 gain to within 0.5% of tabulated values of the ACIS External Calibration Source, particularly at lower energies and warmer focal plane temperatures (up to 2 degrees higher than the nominal -120 C operating temperature). Users interested in CCD spectroscopy may benefit in applying the new CTI-corrected gain adjustment and grating spectroscopy benefits as well, in the form of improved order sorting.

• ACIS Blank-sky Background File (Group G ACIS BKGRND)

  

  Issues with 'Period G' 6i, 2s, and 5s files

  (28 Jul 2017) Errors in the background dataset, acis6iD2012-01-01bkgrnd_ctiN0001.fits, have been identified, where the exposure time is only half as long as the other ACIS-I chips while having nearly the same number of events, implying double the background count rate for ACIS-6. Subsequent review has found artificial "hot spots" in the dataset from a bug in one of the algorithms used to generate the background files. This bug has also affected the files:

  • acis2sD2012-01-01bkgrnd_ctiN0001.fits
  • acis5sD2012-01-01bkgrnd_ctiN0001.fits

  but to a lesser degree. USERS SHOULD AVOID USING THESE FILES. Replacement files were introduced in CALDB 4.7.6 with the N0002 version of these files.

  A new set of ACIS blank-sky background files have been released for observations starting January 2012, where the particle background levels were relatively flat during the period starting in 2012 through late 2015, when the background had increased steadily with the decrease in solar activity.

  acis2sD2012-01-01bkgrnd_ctiN0001.fits
  acis3sD2012-01-01bkgrnd_ctiN0001.fits
  acis5sD2012-01-01bkgrnd_ctiN0001.fits
  acis6sD2012-01-01bkgrnd_ctiN0001.fits
  acis7sD2012-01-01bkgrnd_ctiN0001.fits
  acis8sD2012-01-01bkgrnd_ctiN0001.fits
  
  acis0iD2012-01-01bkgrnd_ctiN0001.fits
  acis1iD2012-01-01bkgrnd_ctiN0001.fits
  acis2iD2012-01-01bkgrnd_ctiN0001.fits
  acis3iD2012-01-01bkgrnd_ctiN0001.fits
  acis6iD2012-01-01bkgrnd_ctiN0001.fits 
  

  The Using the ACIS "Blank-Sky" Background Files thread contains instructions on how to select a file and match it to a specific observation.

  Only CTI-corrected events cases have been included in this release. Stowed background and ACIS back-illuminated chip events, without CTI-corrections for GRADED mode data will be added in future releases.

  Note that the ACIS background files are not included in the main CALDB tarfile. There is a separate background files tarfile available via ciao-install or from the CALDB Download page.

---

How CALDB 4.7.4 Affects Your Analysis

ACIS Imaging and Grating Data

• Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data

  The new, combined time-dependent ACIS Gain (T_GAIN) file for August-October 2016 (Epoch 67) and November 2016-January 2017 (Epoch 68) are introduced. The combined Epochs 65+66 file has also been updated since CALDB 4.7.3. Therefore, the new T_GAIN affects observations taken since 2015 November 01. Other observations are unaffected by these new calibration files.

  Users working with ACIS data taken since 2015 November 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.

  See the time-dependent ACIS T_GAIN why page for more information.

  The gain corrections are at the nominal and expected levels of less than 2% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.

  Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.

HRC Grating and Imaging Data

• HRC-S BADPIX and QEU, HRC-I QE

  The HRC-S "bad pixel" map has been updated to exclude certain events near three plate edges which have been found to be mislocated by the current DEGAP model. Some of these events have distorted pulse-height distributions. Two small regions in the existing BADPIX file have been expanded slightly to exclude these events.

  Concurrently, the HRC-S QEU files have been modified to extend the grating ARFs to the full range of good LETG PHA2 spectral data. Since the net quantum efficiency of the HRC-S is time-varying, the QEU files are annually updated, with an additional file corresponding to the 2012 March 29 HRC-S high-voltage change. The N0008 revisions include predictive QEU files up to the year 2023, which account for a consistent ~0.5% per year drop in the QE across the first-order LETG spectrum since the mission launch. A somewhat steeper reduction has been observed in the zeroth-order over time.

  These CIAO tools and scripts automatically apply the HRC-S QEU files when creating response files:

  • mkgarf
  • fullgarf
  • mktgresp
  • mkinstmap
  • fluximage
  • merge_obs

  To generate the appropriate exposure maps or grating ARFs, the N0008 QEU files should be used in tandem with the new HRC-S BADPIX N0004 file, included in this release. Hence, if the N0004 BADPIX has NOT been applied in processing of your data, chandra_repro should be run before proceeding with analysis using the new QEU N0008 files.

  With the HRC-S QEU work and changes, the HRC-I QE has been updated via HRC-I and HRC-S/LETG cross-calibration. The HRC-I QE change affects the results from mkinstmap and fluximage.

---

How CALDB 4.7.3 Affects Your Analysis

ACIS Imaging and Grating Data

• ACIS QE Contamination Model vN0010

  The ACIS QE contamination model has been upgraded to N0010:

  acisD1999-08-13contamN0010.fits 
  

  This version of the file is necessary due to recent monitoring and modeling of the optical depth of the contaminant on the optical blocking filter, requiring a short-term update for the ACIS contamination model—with updated contamination spatial and temporal models—included in the CalDB.

  The new contamination model provides improved fits to standard extended source spectra with stable photoabsorption and and other fitted parameters of their systematic models.

  While the new model is applicable to all ACIS observations throughout the mission, intended to replace all previous versions of the contamination model, the new model will have no significant effect on fitting results for observations preceeding 2009 and there is minimal change between the N0009 and N0010 models before 2005. However, the contamination is particularly noticeable near and below the Oxygen K-edge at 0.535 keV—and essentially gone above 2.0 keV—so the contamination will affect modeling results for most any low-energy source observed after 2005.

  It is therefore suggested that users doing—or having done—spectral analysis of observations with earlier ACIS contamination models, where a significant portion of the spectra are below 1.0 keV, may have an interest in repeating studies with the new contamination model applied to the ARFs/GARFs.

  More information about the N0010 contamination model, and prior versions, can be found in the ACIS QE contamination model Why document. A detailed presentation of the derivation of the new model—and the resulting changes in the estimated time-dependent effective area of ACIS-I and ACIS-S—is in the technical details section of the CALDB 4.7.3 Release Notes.

  These CIAO response tools automatically apply the contamination file when creating ACIS response files:

  • mkarf
  • mkgarf
  • mkwarf
  • mkinstmap

  As well as the scripts which use them:

  • specextract (calls mkwarf and mkarf)
  • fullgarf (calls mkgarf)
  • fluximage (calls mkinstmap)
  • merge_obs (calls mkinstmap)

• Time-dependent ACIS Gain (T_GAIN) Files for -120 C Data

  The new, time-dependent ACIS Gain (T_GAIN) files for February-April 2016 (Epoch 65) and May-July 2016 (Epoch 66) are introduced and updated for Epoch 64 in this release, and therefore, affects observations taken since 2015 November 01. Other observations are unaffected by these new calibration files.

  Users working with ACIS data taken since 2015 November 01 may wish to run chandra_repro and reprocess the data to improve the T_GAIN calibration. The DATE-OBS header keyword records the observation start date.

  See the time-dependent ACIS T_GAIN why page for more information.

  The gain corrections are at the nominal and expected levels of less than 2% of the energy value and users interested in CCD spectroscopy may benefit in applying the new gain adjustment. Grating spectroscopy benefits as well, in the form of improved order sorting. It is unnecessary to apply this adjustment for only doing timing or imaging analysis, although doing so will not have a negative effect.

  Note that only spectra with several hundreds of counts and/or prominent features (in emission or absorption) will show changes from the T-gain refinement that exceed the uncertainties from the gain calibration.

HRC Imaging Data

• HRC-I Gain Maps

  The latest time-dependent HRC-I Gain Map (GMAP) and the corresponding PI background spectrum—to be applied to observations after 2016 September 20, have been released.

  Note that HRC-I gain maps are only useful to observers doing hardness ratio or PI spectral analysis studies with HRC-I.

  As the HRC gain continues to slowly decline, the gain maps are updated annually. The change in gain correction is less than 1% on-axis and roughly 5-10% off-axis. Towards the edge of the microchannel plate, the change is up to 15%. Technical details on the gain maps are described in the gain maps section of the HRC calibration website and the in the HRC-I Gain Map Why Document.

  The corresponding background spectrum applicable to the same HRC-I GMAP time period have been generated.

  Users working with HRC-I data may wish run chandra_repro to reprocess the data and improve the gain calibration.

---

Installation

Supported Platforms

• CIAO 4.9 is supported on:

  • Linux 64 bit (CentOS 6.8 / Red Hat Enterprise 6)
  • Linux 64 bit (Ubuntu 14.04)
  • Apple OS X 10.9 (Mavericks)
  • Apple OS X 10.10 (Yosemite)
  • Apple OS X 10.11 (El Capitan)
  • Apple macOS 10.12 (Sierra)

  Users can install CIAO with either Python 2.7 or with Python 3.5; however, you must not install both into the same directory.

Updates to the ciao-install Installation Script

• Users can select the Python3.5 version of CIAO using the new --python3 flag

  $ bash ./Downloads/ciao-install --python3
  

  There have been numerous updates to the install script to handle the new operating systems as well to support both Python 2.7 and 3.5 installations.

• Some OSX users reported problems downloading the CIAO tar files. This was frequently tracked to the ciao-install script using an incompatible version of ftp. The script now uses /usr/bin/ftp to avoid this problem.

IPython Settings

• Users will be prompted to update their IPython profiles the first time they run sherpa and chips.

  Users will also be prompted to update their IPython profiles whenever they switch between the Python2.7 and Python 3.5 builds.

Remove old parameter files

• With every new CIAO release, some parameter files are changed: new parameters may be added and occasionally old ones removed or renamed. Deleting or renaming the local parameter directory ensures that the correct parameter files will be accessed the first time a tool is run:

  unix% rm ~/cxcds_param4/*
  

Changes to Dynamic/Shared Libraries

• CIAO tools and applications are built using many different software libraries. As these libraries are common to most of the tools, they are compiled such that different programs can dynamically load the libraries; the libraries are shared by the programs.

  The mechanism used by the programs to locate the shared libraries has changed in CIAO 4.9. The programs are now compiled with the relative location of the libraries built into the executable itself. This means that the programs no longer require the use of the LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (OSX) environment variable to locate these libraries. This change was required for supporting OSX 10.11 (El Capitan) and macOS 10.12 (Sierra) but has now been made to each of the CIAO builds.

  Users who have purposefully overwritten the (DY)LD_LIBRARY_PATH environment variable or who have programs that link against the CIAO libraries (including libraries in the CIAO off-the-shelf, OTS, directory) may need to make changes to their configuration.

---

Python 3

---

Tools

acis_find_afterglow

• Internal cleanup to support different compilers.

acis_process_events

• Update for TDET coordinate calculations in CC mode to set to NULL value when times are invalid.

• Updated PHA_RO (read-out) values for data which has not been CTI corrected.

acis_streak_map

• Now can exclude sources along the read-out streak from the region using the new ssigma parameter.

• Fix bug when infile has 0 events

axbary

• Internal cleanup to support different compilers.

dates

• Added 2016 leap second.

dmdiff

• Fix problem when comments include a "%" character

• Better checking of range() tolerance values

• Better checking for vector (ie virtual) columns (eg "sky"). Now compares the individual physical column names and reports differences in ranges for each physical column separately.

• Image units are now compared

• Multiple transforms (WCS) on a column are now compared separately.

• Fix bug when comparing vector and physical columns

• The string representation of regions are now compared

• Improved checking for structural differences when comparing tables. Including tables with 0 rows and different number of columns (including 0 columns).

• Corrects display of byte data-type value differances

• The ignorepath tolerance option has been extended to also ignore comments

• Other internal code changes.

dmellipse

• Now supports using tables as input. The new xcol, ycol, and zcol parameters specify the column names in the table for the coordinates (x,y) and optionally the per-row weighting (analogous to the pixel value in an image).

• Various internal cleanup.

dmextract

• Fix for binning on columns with long names (> 16 characters) and columns with long descriptions/comments.

• Catch and report problems when exposure (exp) values are invalid (missing columns or non-existent files).

dmhistory

• Show history for partial tool name matches

• Update to handling of ARDLIB and PIXLIB history records.

dmhedit

• Fix bug when comments were uninitialized

dmimgcalc

• return error status when directory/file does not exist.

dmmakereg

• Corrects the string identifying the units of the rotation angle; now correctly written as "deg".

dmtype2split

• Bug fix for invalid output file names.

glvary

• Fix problem when TSTART equals TSTOP. Now a 0 row output file is created.

mkwarf

• Adjusts low energy cutoff used when no FEF file is supplied. (Needed to match HRC RMF energy grid).

tgextract2

• Corrects the problem where the HRC+LETG CALDB bow-tie regionfile was not being used.

wcs_update

• Can now use be used to update an arbitrary FITS extension in the infile.

• The quaternion values in the aspect solution file are now also updated.

• Error message handling improvements.

---

Parameter Files

acis_streak_map

• A new ssigma parameter has been added to specify a limit for excluding sources from a readout streak.

dmellipse

• dmellipse can now work with tables as well as images. The new xcol, ycol, and zcol parameters specify the column names to use for the coordinates and weighting (ie pixel value).

---

ChIPS

Bugs

• Fixed crash on OSX when adding data with transforms associated with it. For example adding an image with a wcs to chips would cause it to crahs. The issue is specific to OSX.

• Fixed a bug that would cause a crash after adding a frame to an existing plot when the gui window is displayed. This is specific to OSX and the gui window must be displayed.

• A user's preferences are now loaded when running python as well as the chips application. Updated ChIPS smoke tests so that the user preferences don't interfere with the smoke test results

---

Sherpa

Statistics

• There have been a number of fixes to the statistic and fitting code that address issues with the WStat statistic, including:

  • Fix problems that could arise when the data was filtered or grouped.

  • The BACKSCAL value was not being handled correctly when it was an array, rather than a scalar, as can happen with grating data.

  • The calc_stat_info function was failing when the WStat statistic was selected.

  • The statistics information returned by calc_stat_info and get_stat_info did not include information about the reduced statistic (rstat) or Q value (qval) when wstat was selected.

  • The calc_stat function was not exiting with an error message when a likelihood statistic - such as Cash, CStat, or WStat - is selected and the background has been subtracted.

Documentation

• There have been a number of documentation updates, either improving or adding to existing docstrings. These include:

  • Documentation for the models has been added. So after

    sherpa> set_source(xsphabs.gal * xsvapec.src)
    

    help on the model components can be found using

    sherpa> help(gal)
    sherpa> help(src)
    

  • Use the correct tolerance value when discussing the optimisation routines (e.g. LevMar).

  • Miscellaneous updates to correct missing or incorrect information, including group_snr, group_adapt_snr, set_rmf, and set_quality.

  

  Sherpa ahelp in CIAO 4.9

  The ahelp documentation for Sherpa has not been updated to match changes in CIAO 4.9, so it is suggested that the Python help system be used; that is use the help command in Sherpa rather than ahelp (this command is not available from the Unix command line, only from Sherpa, ChIPS, or ipython prompts).

XSpec

• The XSPEC models have been updated to version 12.9.0o. The nlapec model is not available, but it can be emulated with the xsapec model by setting the XSPEC XSET variable APECNOLINES to off; for example:

  sherpa> set_source(xsphabs.gal * xsapec.src)
  sherpa> set_xsxset('APECNOLINES', 'on')
  

  although note that this is a global setting, and so will affect all APEC model instances in the Sherpa session.

Python 3.5 support

• The Sherpa code base is now compatible with both Python 2.7 and 3.5. This work included code clean ups to replace deprecated functionality from packages such as NumPy. There are several known differences with the Python 3.5 version in this release:

  1. Numbers displayed in the Python 3.5 version will generally show more decimal places than the Python 2.7 version.

  2. Several functions, such as image_getregion and image_xpaget, return byte strings rather than strings in Python 3.5. The decode method can be used to convert them to strings, as shown below:

     sherpa> image_open()
     sherpa> image_xpaget('version')
             b'sherpa 7.5\n'
     sherpa> image_xpaget('version').decode()
             'sherpa 7.5\n'
     

  3. The order of datasets may be different to the Python 2.7 version; for example, when load_pha is given a PHA2 file, or when fitting multiple data sets. This does not affect the fit results, but may be surprising if the results from Sherpa are expected to be in a certain order.

Bug fixes

• The updates in this version of Sherpa are listed below.

  Improvements and fixes to the save_all function, including:

  • The source expression should now be saved; in previous versions it could sometimes be missed from the output file. Similarly, improvements to storing the background model associated with a PHA data set have been made.

  • There have been improvements to the handling of grouping and quality arrays of PHA data sets: they are now written out as integer values and the values for background data sets are now also stored.

  • Data sets created with the load_arrays function are now written out. This is intended for "small" data sets. It is suggested that FITS files are used for large arrays of if the numeric values may be significantly affected by conversion to text format (i.e. a potential loss in precision).

  • Sessions which use the load_psf function can now be saved when multiple data sets are in use; in previous versions some information may have been lost.

  • When possible, user models created with load_user_model and add_user_pars are now saved to the file. This may not be possible, in which case a place-holder function is used which will display a warning message when the file is evaluated.

    It may still be necessary to edit the file to include necessary imports or set up code for the user model.

  • The script now uses dataset-appropriate functions: for example load_pha or load_image rather than always use load_data, and create_model_component is preferred to eval. Optional arguments to functions are now explicitly named rather than relying on position.

  • The clobber argument now behaves as it does with CIAO tools, in that an existing output file is deleted. Previously it would be appended to.

  • The output file now explicitly imports the sherpa.astro.ui module so that the file can be run from IPython sessions (such as the sherpa environment) with the %run command:

    sherpa> %run bestfit.py
    

    or directly from python, e.g.

    unix% python bestfit.py
    

  • The outfile argument has been added to the save_all function. It can accept any file-like argument such as a file handle or a StringIO object.

  The function load_xstable_model should be used to create an XSPEC table model for additive and multiplicative models (atable and mtable). Support for these models is still available in load_table_model but is now deprecated.

  The covar_matrix argument has been added to the get_draws function, which allows a user-provided covariance matrix to be used.

  Minor change to the axis labels created by the plot_source function for PHA data sets, to better match the other plot types.

  The plot_photon_flux function no-longer errors out with the message:

  TypeError: scales option must be iterable of length ...
  

  Fix the issue where the image_data function would fail with the message

  DS9Err: Could not display image
  

  when the working directory contained a file called x or y.

---

Graphical User Interfaces

---

Analysis Scripts

specextract

• The behavior has been changed when bkgresp=yes so that the calculated background ARF will always be weighted. The background RMF type is still determined by the unweight_rmf parameter.

blanksky

• Script to create an unscaled blanksky background file compatible with a given observation events file. The background scaling factors for each chip are calculated and stored in the BKGSCALE/BKGSCALn header keywords.

blanksky_image

• Script to create scaled background and background-subtracted images given an observation-specific blanksky background file and a reference image.

correct_periscope_drift

• Script to correct small (~0.1 arcsec) intra-observation alignment drifts that can be seen in recent, long (>50 ks) observations. Requires a bright, on-axis (<2 arcmin) point source with minimal pileup.

simulate_psf

• A simplified interface to MARX, to perform both raytrace PSF simulations or projecting ChaRT/SAOTrace rayfiles to the detector-plane for an existing observation.

---

Python Modules

crates

• Add a new add_comments routine.
• when adding image object to IMAGECreate, also add subspace records for X and Y axes.
• generate warning for data columns of unknown type. no CrateData object is created for these columns.
• returning single row bit array column as 2D
• Enable writing of NULL primary image with uint8 type. Improved error message when write_block encounters None value pointer. NOTE: Converts to int2 FITS type (BITPIX=16)

pypixlib

• Improve search logic for obtaining underlying C library cannot rely on (DY)LD_LIBRARY_PATH since this has been deprecated in Mac ElCapitan build.

pyTransform

• wrapper updates for string* elements, removes cross-interpretation as char**

---

Libraries

cxcparam

• Additional error checking for long string values

• Internal cleanup to support different compilers.

datamodel

• Fixes output physical transform definition when filtering images that have no specific physical coordinate system defined.

• Introduces changes to better handle the case where a region filter is entirely outside the boundary of the image.

mtllib

• Internal cleanup

pixlib

• Internal cleanup

region

• Introduces changes to how region bounds are computed when the region is outside the boundary of the dataset.

---

Environment

Off-the-Shelf (OTS) Package Versions

• The following OTS packages are included with CIAO 4.9. For more information on how the OTS packages are built for use with CIAO, refer to the INSTALL_SOURCE file distributed with the software.

  Executables

  • ds9 7.5

  Python

  • either Python 2.7.11 or, as a beta release, Python 3.5.1
    Frameworks on OSX have been disabled to avoid a problem with users picking up the wrong version of NumPy.
  • NumPy 1.11.0
  • ipython 4.2.0
  • six 1.10.0

  Libraries

  • cfitsio v3.380
  • readline 6.3
  • wcssubs 3.9.0
  • XSPEC v12.9.0o (models only)
  • VTK 5.10.1
  • CCfits 2.5
  • fontconfig 2.8.0
  • freetype 2.4.4
  • gsl 2.1
  • vte 0.25.1
  • fftw 3.3.4
  • xpa 2.1.15
  • jpeg 9a
• The ciaover command will now report the version of the Calibration Database as well as the version of the Analysis Scripts package. In addition the full path to the Python executable is shown (even when using the precompiled CIAO version).

Compiler versions

• The following versions of the gcc compiler were used to compile the binary CIAO releases:

  System	Build System	Version
  Linux64	CentOS 6	gcc 4.4.7
  LinuxU	Ubuntu	gcc 4.8.4
  osxm	Mavericks	llvm 6.0
  osxy	Yosemite	llvm 7.0.2
  osxElCap	ElCapitan	llvm 7.0.3
  osxSierra	Sierra	llvm 8.0.0

---

Documentation

---