CIAO 1.1 Release Notes
Contents:
- Introduction
- CIAO 1.1 tool list
- List of Tools with modified parameter files
- Tool/Application Notes
- Known Bugs/Issues
-
Introduction
Detailed specifications and implementation information on the UI software contributed to CIAO1.1 can be found on the team web page at http://hea-www.harvard.edu/uinfra/uimain.html.
A well-configured CXCDS environment should be all that is required to use the analysis applications. Please check the notes from previous releases for information on features not explicitly covered below.
Supported Platforms:
Solaris 2.6, 7
Redhat 5.2, 6.0
Slackware 3.5>> we are working on Slackware 4.0
-
CIAO 1.1 tool list
The following is a list of applications and software tools included in CIAO1.1:
Applications sherpa filterwindow firstlook chips prism ACIS acis_format_events acis_process_events acis_sort_time HRC hrc_process_events Gratings tg_resolve_events tg_create_mask tgextract Datamodel dmcopy dmlist dmsort dmextract dmappend dmmerge dmregrid dmwritefef dmhedit dmmakepar dmreadpar dmrega2fits dmstat dmimgcalc dmarfadd dmgti Detect celldetect vtpdetect wavdetect tgdetect TCD csmooth aconvolve Response mkrmf mkpsf mkexpmap mkinstmap mkarf mkgarf asphist asp_apply_sim Timing lightcurve -
Tools with modified Parameter files
The following tools have had their parameter files modified in CIAO1.1. This means that when using CIAO1.1 the parameter files a user may have around from CIAO1.0 must be deleted in order for the new version to be accesed by the software.
- dmlist - added verbose parameter
-
Tool/Application Notes
Data Model ========== o Enhancements to the DM interface: - Handles ASCII region files produced by SAOTNG dmcopy evt.fits[sky=region(saoimage.reg)] out.fits - Handles physical coordinates in FITS images created by dmcopy. If im.fits is a blocked IMAGE made by dmcopy "evt.fits[bin x=500:1080:4,y=400:1080:2]" im.fits then "im.fits[#1=100:120,#2=200:240]" is a filtered image with LOGICAL (blocked) pixels 100-120, 200-240, and "im.fits[x=900:1080,y=800:880]" is approximately the same filtered image, since the use of "x" and "y" is taken to mean PHYSICAL (unblocked) pixels. Recall that in tables (eg event lists), there is no blocking: logical and physical coordinates are the same. - Improved recording of region filters in the output file's data subspace. o Bug fixes: - Bug fixed which caused wrong data subspace (GTI) for evt2 files - CONTINUE keyword was not handled correctly - Block numbering corrected for binned blocks - Fixed bug which caused keywords to be copied out of order. - Fixed bug with limited precision in subspace keywords - Fixed bug with intersecting data subspaces when ranges overlapping - Amended order of new FITS CTYPE keywords to work around IRAF bug - Fixed bug which caused dmcopy to add multiple cases of the FITS literature reference and the long keyword warning - The default name of a binned image block has "_IMAGE" appended to the source table name. - Units added to GTIs. - TDIM handled correctly - External file filtering improved - Column selection on vector cols "[cols sky]" now works - Fixed bug in checksums for GTIs - Fixed bug losing coordinate info on binning event lists. User Interface - M Noble ======================== o IMAGING Either SAOtng or DS9 may be used for imaging ($HOME/.CXCdefaults) Pending further sherpa testing, DS9 will become the default as of this release. o CHIPS - Batch mode plotting now supported in CHiPS (no X display needed) - There is no longer any command/token overlap between the sherpa and chips parsers. - All chips commands are now accessible via XPA, through the addition of an XPA server thread within the chips application. - New "type" command allows user to inspect data that has been loaded into CHiPS plots. o FIRSTLOOK - 1.0 beta version available w/in CIAO1.1, supporting most of of grating- and non-grating analysis threads. - Looks for unzipped versions of files when .gz cannot be found - Integrated tooltips (aka "help bubbles") - New "Help->About" functionality, with connection to README o FILTWIN - 1.0 beta version available w/in CIAO1.1, supporting a good deal of iterative filter recording/saving/updating. - Can communicate directly with ChIPS and imager, as well as receive commands from FirstLook. - New "Help->About" functionality, with connection to README o PRISM - Separate vector columns for plotting individual components against other scalar columns in table - Integrate tooltips (aka "help bubbles") and eliminate quickhelp. o TOOLAGENT - Ability to stop tools after launching (from w/in GUI) - Updated for CIAO1.1 list of tools - Feedback to user when tool completes (from w/in GUI) - Context sensitive help, so that when tool is loaded into GUI param editor, clicking Help gives help specific to that tool o Online Help - Expanded online-help by inclusion of more documentation for sherpa, CHiPS, and various tools - Added support for cross-referencing and/or grouping related CXCDS tools and/or application commands. - Look on local machine for browser before launching another to display help. o Portability - Extended portability to Redhat6.0 and Solaris2.7. We now support five platforms in all, the entire suite of which are: Solaris 2.6, 2.7 Redhat Linux 5.2, 6.0 Slackware Linux 3.5 - More scripting has been implemented to support automated porting and packaging, including an experimental obsvis "subpackage". o Regression Testing The UI regression tests continue to improve, and may now be used to test portable releases, FITS- and IRAF-file compatibility, several new visualization features, AND portions of sherpa functionality. Data Analysis Tools - KJG ========================= o General (i.e in libraries): - header library Updated to get TSTART and TSTOP from the GTI subspace and recompute ONTIME, LIVETIME and EXPOSURE when merging multiple files. - Error Library This has had a fix applied to a bug that shows up in one of the Linux builds. In addition many new error codes have been defined, and there is some new functionality for looking for errors in a list without copying them. o New development: - Exposure Map Software Suite The exposure map code suite consists of the following tools: 1. mkinstmap -- generate an instrument map 2. mkexpmap -- generate an exposure map 3. mkarf -- create an imaging ARF 4. mkgarf -- create a grating ARF Each of these tools require access to calibration data such as effective areas, grating efficiencies, detector QEs, bad pixel lists, etc. These four programs access calibration data via ARDLIB, the analysis reference data library. With the exception of mkinstmap, these programs also require as input an aspect histogram (generated by asphist). The histogram provides a condensed history of the telescope dither. mkinstmap is used by mkexpmap to generate an exposure map. It is essentially the off-axis effective area function of the mirror projected onto a specified detector and multiplied by the detector's QE and bad pixel list. It may be computed at a single energy or summed across a range of energies with user specified weights. mkexpmap transforms the instrument map to the sky and then moves it around on the sky in accordance with the aspect histogram. The resulting object is called an exposure map. Each cell of the exposure map represents the effective exposure given to that region of the sky. mkarf produces a type-I OGIP compatible ARF for a specified off-axis angle and detector/sky region. (Regions will appear in the next release. Until then, the user should selected events in a region big enough ensure that the PSF fraction is 1). mkgarf produces a type-I OGIP compatible ARF for grating analysis. The grating ARF is computed for a specified order, detector, and grating arm. The dmaddarf program could be used to combine grating ARFS for different detector chips. - asp_apply_sim This new tool applies the sim factors to the raw aspect offsets, then writes out the results to an aspoff table. The infile, which contains the raw aspect offsets and the sim factors, was produced by the tool 'asp_calc_offsets'. The output columns include 'time x_offsets y_offsets roll_offsets stf_y stf_z stf_roll', where the columns of 'time stf_y stf_z stf_roll' are copied directly from the input file and the columns of 'x_offsets y_offsets roll_offsets' are the aspect offsets with the sim factors. Some header keywords, such as the detector name, the instrument name, the roll_nom and the sim values, are required for the pixel library to convert the chips coordinates to the focal plane coordinates or vice versa. - asphist This tool takes aspect offsets files for an observation and will create a binned histogram of the aspect motion during the observation using the octree algorithm. The output is a table of the duration of the bin and position of the spacecraft during that duration. The GTIs are also appended to the output table as a subspace. - dmarfadd Combine ARFs (Auxiliary Response Files) and will scale the spectral response when energy bins between two files overlap. The scaling factor is based on the exposure time. The user can only combine ARF of the same order and Grating. If there is no grating in place, there is no order restriction. The ARF must be on the same energy grid. Positive and negative orders of the same magnitude can be added. KNOWN LIMITATIONS: The maximum number of files that can be added is 8. - dmimgcalc This tool performs elementary arithmetic on pairs of images (or binned event lists). Pixel-by-pixel addition, subtraction, multiplication, division, and comparison are supported. This tool is presently very basic and is not as richly functional as other dm tools. - dmregrid: This new tool regrids each input image within the input dataset based on the binning specification and the other parameters, which together specify the offsets, minimum and maximum channels, bin scales, rotation angle, and center of rotation. Each of these parameters can either be a stack or a single value; if a single value is given, it is applied to each input file. For a stack of input files, the sum of the regridded images will be output to the output file. KNOWN BUG : If the binning specification is given by a stack in which the second (or later) item results in a greater number of output pixels than the first binning specification, a segmentation fault results. The fix for this bug will appear in the next release/patch. - lightcurve Bin source and background events into specified time intervals and computes the net count rate for each bin. The purpose of the lightcurve tool is to bin source and background events into specified time intervals and computes the net count rate for each bin. The results computed by lightcurve can then be used to plot light curve of the source. lightcurve has the same functionalities as that of ltcurv in PROS. However, lightcurve has one new binning option besides letting the user select the total number of bins, or the bin length. This new binning option allows the user to set the average number of counts per bin, and then the program will set the total number of bins and bin length accordingly. LIMITATIONS: o The input file must be a time-sorted event file. o The output file is in fits format only. - mkpsf The mkpsf program is one of the Point Spread Function (PSF) tools. The program retrieves an image from a FITS format library file based on information entered by the user and contained in the input file. The program retrieves an image based on energy, x and y coordinates specified by the user, bins it according to the input file, rotates it based on roll provided in the input file, and saves it in a FITS format image file. - mkrmf Generate 2-D (at present) response matrix file (RMF) based on a FITS embedded redistribution function file (i.e, ECD table) and with inputs of MATRIX grid specification (i.e. intrinsic photon energy grids and PHA channels). The output MATRIX/EBOUND FITS file is encoded on-disk in format of "legacy" or CXC. 1> Changed the tool name from rmfcalc to mkrmf 2> Made low_thresh parameter (in mkrmf.par) visible to user 3> Added an offset header keyword in EBOUND extension which is extracted from the input ecd-table FITS file 4> Updated pha (or pi) bin method in linear space for a bin step step = (max-min+1)/num_bin 5> Retained energy bin method the same, or step = (max-min)/(num_bin-1) 6> min/max/num_bin in pha axis must make the bin step an integer, otherwise mkrmf exists with error 7> the pha grids must be binned linearly otherwise mkrmf exits with error o Updates to Existing Tools: - dmlist, dmcopy small changes have been made for CIAO 1.1, although the internals have been substantially improved and there have been a lot of minor bug fixes. - dmreadpar Fixed a bug in the template code that caused an infinite loop to be entered. - celldetect Cell placement algorithm has changed slightly so that results of celldetect will not vary with different region filters. The new results are likely to be (slightly) different from earlier results. Bug 1562 has been fixed, which caused celldetect to exit with no error message when the block specifier (eg: "[EVENTS]") for an event list was omitted. - mtl_build_gti The limits syntax has been updated to be more user friendly, both in referencing columns and defining new columns. A new parameter, userlimit, was added, and the output MTL is optional. - tgdetect Because tgdetect uses celldetect, it is, in principle affected by the celldetect change. However, no differences in results have yet been noted. - tgextract 1> Fixed the checksum error bug. 2> Changed the binning order when the bin unit is Angstrom, eV or keV. Now, the output is in desending order of wavelength if bin unit is Angstrom, and the output is in ascending order of energy if the bin unit is in eV or keV. 3> Added X, Y columns (these are the x,y coordinates of the center of the sources) in the output spectrum. As a result, the input files to tgextract must have a region extension that provides the x, y coordinate of the center of the sources. 4> Replaced the output BACKGROUND column with BACKGROUND_UP and BACKGROUND_DOWN. 5> Redefined the equation for calculating the value of header keword BACKSCAL. Note that there are no changes in the interfaces. o SHERPA New Functionality and Bug Fixes 1> PSFFROMFILE function can be used now as a 2D source model (i.e. the ability to fit psf profiles to sources to determine if they are point-like or extended) or an instrument model. The choice of sliding cell convolve or FFT convolve is given to the user. 2> Fakeit has been added to allow simulations of 1D spectra. 3> Powell algorithm have been greatly revised and improved. 4> Grid method now automatically determines grid points in log space if the linear space grid spacing is too large, and issues a warning to the user that this is being done. 5> Guessing an initial parameters ranges was improved to avoid working within too small parameter space. We still advice to always use at least two optimization methods to confirm the final results. (see Optimization section of the Manual for advice on fitting methods). 6> A bug in the uncertainty algorithm has been fixed. Also the user will see warning if the uncertainty algorithm finds a better minimum. 7> The confidence ranges can be displayed. 8> A problem with automatic replacement of the instrument model files when the new pha file is read into Sherpa has now been fixed. 9> Some plotting commands have been changed to differentiate between CHIPS and Sherpa commands (check the Manual for details). Note "lplot" replaces "plot" command. -
Known Bugs/Know Limitations
1. aconvolve: The center parameter doesn't work properly. When the user selects the option center=yes, the fft of the data (as well as the kernel) shows the maximum response in the corner, not at the center. 2. csmooth and aconvolve: The FFT code (for convolution) in some cases for axes with odd number runs into a segmentation fault. The work around is to crop the image to a slightly smaller size. For example for an image infile.fits with 726x751 pixels size that produced a segmentation fault the user can crop the image to 725x750 pixels and then rubn csmooth, or he/she could run csmooth on a cropped image directly: csmooth infile="infile.fits[1:725,1:750]" 3. dmarfadd The maximum number of files that can be added is 8. 4. dmregrid There is a known bug in dmregrid. If the binning specification is given by a stack in which the second (or later) item results in a greater number of output pixels than the first binning specification, a segmentation fault results. The fix for this bug will appear in the next release/patch. 5. lightcurve LIMITATIONS: o The input file must be a time-sorted event file. o The output file is in fits format only. 6. Data Model filtering Numbers in filters can't end with a decimal point [x=15.] fails, use [x=15.0] or [x=15] instead. ========================================= SHERPA known bugs ========================================= 1> Model definition: -------------------- a> Two dimensional models used in fitting binned data use the model value from the central position on the bin. These models are not integrated over the bin. Therefore for rapidly changing model over the bin the result can be incorrect. Note also that the result will strongly depend on the binning applied to the data. (Bug 1559) b> If there is no data or the filter ignores the entire data set (ignore all) then this data set is empty and any source definition causes Floating exception error and crash. (Bug 1733) 2> PSFFROMFILE: --------------- For PSFFROMFILE to work on two-dimensional images, one cannot apply region filters within Sherpa. Such filters should be applied outside of Sherpa (e.g. with DataModel) before Sherpa analysis. (Bug 1677) 3> Known problems with the Image display in Sherpa: --------------------------------------------------- a> The displayed images from Sherpa have the Y axis flipped (Bug 1603) b> Composite filters (adding or subtracting regions) entered interactively using the Imager are not applied correctly to the data. The user should use the command line input if the aplied filter is composed of 2 or more filter regions. (Bug 1604) c> "Image source" when the source model is PSFFROMFILE causes Segmentation Fault. (Bug 1732) 4> Plotting problems: --------------------- a> Updated plotting definitions are not remembered after the new type of plot is being introduce. For example "lplot data" "lplot data" remembers the modification made to the first plot, but "lplot data" "lplot fit" does not remember the modifications to the data plot. In the order: "lplot data" "lplot fit" "lplot data" the last "lplot data" command also does not remember any modification to the first "lplot data", due to fit plot made in between. (Bug 1040) b> The axes of the multiple plots are not aligned automatically. (Bug 1612) =========================================================================== UI Known bugs 12/16/99 ------------------------ GENERAL ------- 1597: There are colormap issues with GUI applications that may lead to the error message 1629: ExitAll does not exit sherpa PRISM ----- 1530: PRISM shows header keywords in a different format than dmlist 1540: In PRISM, the TWM window maneger does not support Motif window titlebar Example: expand a column data in PRISM, and note that when running under TWM the absence of a title bar prevents the user from knowing just what column is being viewed (because the window titlebar is being used to display the column name, as well as its vectorness/arrayness). 1541: the GUI status/message area cannot ben cleaned out from old messages 1574: PRISM cannot generate multiple plot draw areas if multiple rows are selected for plotting 1616: PRISM does not have "editing" capability 1656: In PRISM the expand window "seems" to allow edits of data matrix but in fact does not 1721: Prism crashes if a block is reopened after expanding vector cols PRISM crashes if you reopen a block while a vector column is expanded. Occassional prism gets into a state where when you try to expand the SRC_POS column you get both vector column dialog box and the "column is not expandable" warning dialog. 1731: When running sherpa, 1) launching imager, 2) suspending sherpa to get a unix prompt, 3) running prism, prism will not come up. 1757: The TSTART/TSTOP keyword are rounded in the header of files seen through PRISM. Workaround: use >dmlist file header FIRSTLOOK --------- 1709 OIF tree window is sometimes hidden The OIF tree window sometimes opens up BEHIND the firstlook window (at least under CDE). 1754: The "STOP" button in First look is not working FILTWIN ------- 1751 : filtwin: File menu could be clearer The File->Restore menu option in FiltWin sometimes confuses users into thinking they can use it to load arbitrary data files into FiltWin. This is not so, as the menu is intended to be used to restore previously saved "saved filter state," ie the filters described w/in the filtwin display during a previous "File->Save filter description" operation. CHIPS ----- 1528: CHIPS window can be killed by process that didn't create it. Example: Prism -> plot two columns -> Chips window comes up. Then run filter window, and then immediately exit it. When filter window dies, it takes CHiPs window with it. 1613: log scale ticks in chips While using sherpa with the x scale set to log, the command: "ticks x 0.0002" generated the following error: Small tick spacing must be <= big spacing When the scale was changed to linear linear, the command worked but the results were poor. The curve disappeared from the screen and 3 sets of overplotted tickmarks appeared. Further comments on the bug: In the majority of situations, log ticks are OK. It is only in specific situations where possibly zero or only one tick is displayed. Axes with range/domain like (1,10) or (1,10000), etc work fine. Odd situations like axes (logarithmic obviously) with a range/domain of (1.7, 2.9) or (90000,90010) often display few or no ticks. This is a difficult problem to solve in SM. Unless I can get help with an algorithm to figure out ticksize values to feed into SM, this bug cannot be fixed. A query to the smusers mail group went unanswered. 1744: Exiting chips kills ds9, even when ds9 was launched as a separate application TOOLAGENT --------- 1754 : INDEF not supported by parameter interface gui Bug Description : The gui parameter interface does not support INDEF as a value, as a result tools like csmooth do not produce correct results when launched from Toolagent (the INDEF parameters get viewed and passed to cxcwrapper as zeros instead of indefs). Workaround: run the tool from the command line IMAGER ------- 1610: Interrution of Sherpa by CTRL-C kills the imager as well