Synopsis
Introduction to Sherpa, the CIAO modeling and fitting package
Description
Welcome to Sherpa, the CIAO modeling and fitting package. Sherpa enables the user to construct complex models from simple definitions and fit those models to data, using a variety of statistics and optimization methods.
This document provides an introduction to Sherpa; more information is available on the Sherpa website.
Sherpa is designed for use in a variety of modes: as a user-interactive application and in batch mode. The Sherpa command launches an interactive Python session (using the IPython interpreter), in which Sherpa commands (which are Python functions) can be used, as well as Matplotlib commands (to create and modify plots); Crates commands for reading and writing files; as well as other Python functions. It is possible to write Python scripts that import the relevant Sherpa modules, and so run non-interactively.
The Sherpa session can be saved into a platform-independent state file. The session can then be restored at any time.
Starting Sherpa
From the CIAO command line, type:
unix% sherpa [-x|-n|-b|-rcfile|-norcfile] <file>
Any or all of the following options which may be supplied when Sherpa is started:
- -x : launch Sherpa shell in separate display terminal.
- -n : prevents the Sherpa banner from being displayed on startup.
- -b : runs Sherpa in batch mode
- -rcfile : specify a specific .sherpa.rc file to use; refer to "ahelp sherparc" for details.
- -norcfile : do not load any .sherpa.rc file; overrides "-rcfile" if both are set.
- <file>: Python command file to execute
The startup script loads the Sherpa module and the CRATES module, which handles file input and output ("ahelp crates").
The "Starting Sherpa" thread has more details.
The Sherpa Resource File: .sherpa.rc
When Sherpa is started, it processes the $HOME/.sherpa.rc resource file. The resource file defines default behavior for the Sherpa commands and can be customized to the user's preferences; refer to "ahelp sherparc" for details.
Loading Modules
To import the Sherpa, NumPy, and Matplotlib modules in Python without using the "sherpa" startup script:
from sherpa.astro.ui import * from matplotlib import pyplot as plt import numpy as np
It is likely that you will want to include the numpy and matplotlib modules, so they are included in the example above too!
Getting Help
There are several ways to access the Sherpa help files.
Within Sherpa
Within the Sherpa application, the native Python help system can be used:
sherpa> help(<command>)
It is also possible to do a wildcard search for commands, by taking advantage of the native IPython support:
sherpa> plot*? sherpa> *psf?
Alternatively, the ahelp system can be used either directly:
sherpa> ahelp("<command>")
sherpa> ,ahelp <command>or by running the ahelp command-line program:
sherpa> !ahelp <command>
From the CIAO command line
Syntax, description and examples for a specific command:
unix% ahelp <command>
Differences between help and ahelp
For most terms, the ahelp pages are generated from the Python docstrings, and so contain the same information as the help version, but organised slightly differently.
Contributing to Sherpa
Sherpa is developed on GitHub and can be found at https://github.com/sherpa/sherpa. Please consider contributing to Sherpa development - whether it is reporting bugs, providing documentation updates, fixing bugs, or adding new functionality. Using this repository, Sherpa can be installed outside of CIAO, and so used with Python packages that can not be installed into the CIAO environment.
Citing Sherpa
The sherpa module contains a citation function which will display the Zenodo record for Sherpa:
sherpa> import sherpa
sherpa> sherpa.citation('4.16.0')Changes in CIAO 4.16
XSPEC
XSPEC has been updated from 12.12.1c in CIAO 4.15 to 12.13.1 in CIAO 4.16, adding support for the xscglumin convolution model and updating the default parameter values where needed. Unfortunately Sherpa does not provide access to the polarimetry models added in XSPEC 12.12.1. The redshift parameter for XSPEC table models can now be changed to have a maximum outside of the defauult range of 0 to 5 (this addresses a regression in CIAO 4.15), table-model parameters with a negative delta value are now set as frozen by default, and support for models with the ESCALE keyword has been added. Experimental support for creating XSPEC table model files has been aded to the sherpa.astro.io.xstable module.
Filtering and grouping
The CIAO 4.15 release reported the current filter state when notice and ignore calls were made. The 4.16 release adds more commands, such as group and set_analysis.
The group_xxx commands, like group_counts and group_snr, will now default to only grouping the filtered data range. This can change the selected data range used in a fit, and so change fit results. Setting the tabStops argument to "nofilter" will use the original (pre 4.16) behavior. It is suggested that filtering is done before grouping, to make sure the fit is to the requested data range. In the following example a PHA file is first grouped by 20 counts and then filtered:
sherpa> load_pha("3c273.pi")
sherpa> ungroup()
dataset 1: 0.00146:14.9504 Energy (keV) (unchanged)
sherpa> group_counts(20)
dataset 1: 0.00146:14.9504 Energy (keV) (unchanged)
sherpa> notice(0.5, 7)
dataset 1: 0.00146:14.9504 -> 0.438:13.4612 Energy (keV)This results in a selected range of 0.4 to 13.5 keV. This can be compared to doing the filtering first:
sherpa> ungroup() dataset 1: 0.438:13.4612 Energy (keV) (unchanged) sherpa> notice() dataset 1: 0.438:13.4612 -> 0.00146:14.9504 Energy (keV) sherpa> notice(0.5, 7) dataset 1: 0.00146:14.9504 -> 0.4964:7.008 Energy (keV) sherpa> group_counts(20) dataset 1: 0.4964:7.008 Energy (keV) (unchanged)
In this case the final selected range is 0.5 to 7.0 keV. Note that the last bin is unlikely to contain 20 counts.
To match the CIAO 4.15 behaviour we can set tabStops="nofilter", which gives:
sherpa> ungroup() dataset 1: 0.4964:7.008 Energy (keV) (unchanged) sherpa> group_counts(20, tabStops="nofilter") dataset 1: 0.4964:7.008 -> 0.438:13.4612 Energy (keV)
Plotting
There is now experimental support for using the Bokeh Python visualization package instead of matplotlib. The new set_plot_backend call can be used to change the plot backend, or the .sherpa.rc configuration file can be modified. The linewidth option can now be used to change the line width of many plots. Several issues with the plot_source and plot_model commands have been fixed for PHA data. The get_plot_prefs and get_contour_prefs fucntions have been added as a way to access the preferences for all the plot and contour commands.
Linking parameters
Parameters can now be linked using functions which follow the NumPy universal function interface, or can be converted to one using frmopyfunc. The following example will fit the position of the source as an offset and angle from the location 3145,4832:
sherpa> create_model_component("gauss2d", "src")
sherpa> create_model_component("scale2d", "radius")
sherpa> create_model_component("scale2d", "theta")
sherpa> src.xpos = 3145 + radius.c0 * np.cos(theta.c0)
sherpa> src.ypos = 4832 + radius.c0 * np.sin(theta.c0)
sherpa> set_source(src + 0 * (radius + theta))Due to limitations in CIAO 4.16 the source model does not know to treat the radius and theta models as parameters, hence the need to include them in the source expression but multiplied by zero so that they do not directly contribute.
Response files
The save_arf and save_rmf commands have been added. New routines have been added to the sherpa.astro.instrument module to convert a RMF to a 2D image. The plot_rmf command has been added to help visualize a RMF (the response for 5 delta functions, selected to span the RMF response, are shown), along with the associated get_rmf_plot function. RMF files with multiple matrices are currently not supported by Sherpa and their use will trigger a warning (as only the first response will be used).
Random numbers and simulations
The default behavior in CIAO 4.15 is for random numbers to be controlled by the global NumPy random seed. That is, calling np.random.seed to a fixed value should generate repeatable results. This remains the case for CIAO 4.16, but the new set_rng call can instead be sent a NumPy random generator, such as created by np.random.default_rng(), instead.
The plot_pvalue call has been changed so that it is less likely to get stuck in certain areas of parameter space. The ratios field of the get_pvalue_results structure can be displayed to see if this happens (e.g. large areas where the ratio is close to zero). The fake_pha call now matches the fake call and accepts an optional method parameter.
Backgrounds
The calc_bkg_stat, calc_bkg_stat_info, and get_bkg_stat_info have been added. The get_stat_info command can now handle cases where different datasets have different numbers of background components.
Model documentation
The model names - like powlaw1d, gauss1d, xshabs, and xsvapec - now provide more useful information when used with the help command. However, the help command still only provides information on the model itself - such as the parameter names - when used on the instance name - e.g.
sherpa> set_source(xsphabs.gal * xsvapec.src) sherpa> help(src)
Saving state
The output of the save_all command has seen some minor improvements: better handling of XSPEC table models, include set_psf calls when needed, and improved support for datasets created with load_arrays rather than read in from a file. A warning has been added to the documentation of the restore call to point out that it should only be used with a trusted file.
Bugs
See the bugs pages on the Sherpa website for an up-to-date listing of known bugs.