/ ciao4.17 / ahelp / combine_grating_spectra.html

Jump to: Description · Examples · Parameters · TYPE:I vs TYPE:II · Caveats · Changes in the scripts 4.16.2 (August 2024) release · Changes in the scripts 4.14.2 (April 2022) release · Changes in the scripts 4.12.1 (December 2019) release · Changes in the scripts 4.10.3 (October 2018) release · Changes in the script 4.10.1 (April 2018) release · Changes in the scripts 4.9.2 (April 2017) release · About Contributed Software · Bugs · See Also

AHELP for CIAO 4.17

combine_grating_spectra

Context: Tools::Composite

Synopsis

Combine Chandra gratings PHA files and their associated response files.

Syntax

combine_grating_spectra  infile outroot [add_plusminus] [garm] [order]
[arf] [rmf] [bkg_pha] [bscale_method] [exposure_mode] [object]
[clobber] [verbose]

Description

The combine_grating_spectra script combines multiple grating spectra. It can be used to combine the positive and negative orders within a single observation, to combine the orders from multiple observations, or to combine both: observations and positive and negative orders. It will also combine the associated background PHA spectra and source and background ARF and RMF response files.

For TYPE:II PHA files (the default from the tools tgextract, tgextract2, chandra_repro, and from the Chandra archive) users will need to specify the infile parameter and provide a stack of arf and rmf files. The background will be extracted from the TYPE:II file.

For TYPE:I pha files (output of dmtype2split or available from TGCat - the grating catalog and archive), users will only need to specify the infile parameter since the response (ARF and RMF) files will typically be found from the ANCRFILE and RESPFILE keywords in the header of the spectra. If required, the user can override these settings using the arf, rmf, and bkg_pha parameters; however, these files must match the input files and be given in same order. If the background is provided as columns in the spectrum file, the script will attempt to use the data unless the user has specified the bkg_pha parameter.

The output source spectrum will contain the sum of the source spectrum counts and (by default) the sum of the exposure time. The background counts are scaled by the exposure times and areas as discussed below.

The output will always be a TYPE:I pha file regardless of whether TYPE:I or TYPE:II pha files are given as input.

Algorithm Overview:

This script is a Chandra gratings specific version of the more generic combine_spectra script. It identifies the grating arm and spectral order from the input files and runs combine_spectra multiple times to combine each requested grating arm and order from the available input files. If standard TYPE:II pha files are given as input, the script calls the tgsplit script to convert the TYPE:II pha files into TYPE:I pha files and also associates the ARFs and RMFs to the correct spectrum (if they are supplied). If TYPE:I pha files generated by TGCat are given as input, the script calls tgsplit to separate the background from the source (and combines the _UP and _DOWN background components). The output is a set of TYPE:I pha files.

Existing Scripts:

This script replaces the more limited functionality in the now deprecated add_grating_orders and add_grating_spectra scripts.

Examples

Example 1

unix% combine_grating_spectra infile=acisf06471_repro_pha2.fits
outroot=capella6471 add_plusminus=yes

Using the default parameter values, this will combine the plus and minus orders for each of the grating arms and orders in the single PHA2 file. This is equivalent to the add_grating_orders script. Since this observation, ObsID=6471, is an ACIS+HETG observation, it will create 6 combined spectra: HEG 1st order, HEG 2nd order, HEG 3rd order, MEG 1st, MEG 2nd, and MEG 3rd.

unix% /bin/ls capella6471*
capella6471_combo_heg_abs1.pha
capella6471_combo_heg_abs1_bkg.pha
capella6471_combo_heg_abs2.pha
capella6471_combo_heg_abs2_bkg.pha
capella6471_combo_heg_abs3.pha
capella6471_combo_heg_abs3_bkg.pha
capella6471_combo_meg_abs1.pha
capella6471_combo_meg_abs1_bkg.pha
capella6471_combo_meg_abs2.pha
capella6471_combo_meg_abs2_bkg.pha
capella6471_combo_meg_abs3.pha
capella6471_combo_meg_abs3_bkg.pha

The background files are created from the BACKGROUND_UP and BACKGROUND_DOWN columns in the TYPE:II pha file. Since no ARF nor RMFs are supplied, they are not combined.

Example 2

unix% combine_grating_spectra infile=acisf06471_repro_pha2.fits \
outroot=capella6471 add_plusminus=yes arf="tg/*arf" rmf="tg/*rmf"

As above with the addition of specifying the ARF and RMF files. In this example the "*" UNIX wild card is used to locate all the files named "*.arf" and "*.rmf". The script will automatically associate the correct ARF and RMF with the correct row (order/arm) in the PHA file.

Example 3

unix% combine_grating_spectra infile="*/repro/*pha2.fits"
outroot=capella \
garm=HEG order=1 add_plusminus=yes arf=none rmf=none

This example shows combining data for multiple observations. In this case the infile is a stack of TYPE:II pha file. The script will combine the HEG +1 order spectra from the individual observations and the HEG -1 order spectra, and then will combine the plus and minus orders together.

If instead add_plusminus=no is used, then only the HEG +1 orders would be combined.

Since arf=none and rmf=none, no response files are created.

Example 4

unix% combine_grating_spectra infile="*/repro/*pha2.fits" \
outroot=capella garm=HEG order=1 add_plusminus=yes \
arf="*/repro/tg/*arf" rmf="*/repro/tg/*rmf" clob+

Similar to the previous example but now the ARF and RMF files are included. The screen output for this run is shown here:

Input TYPE:II pha file(s).  The output will be one or more TYPE:I pha files
Splitting TYPE:II pha files into TYPE:I
Combining spectra across observations
Prepared to combine 5 spectra

source PHA: capella_tmp00_heg_m1.pha
       ARF: 10599/repro/tg/acisf10599_repro_heg_m1.arf
       RMF: 10599/repro/tg/acisf10599_repro_heg_m1.rmf
    background PHA: capella_tmp00_heg_m1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_tmp01_heg_m1.pha
       ARF: 11931/repro/tg/acisf11931_repro_heg_m1.arf
       RMF: 11931/repro/tg/acisf11931_repro_heg_m1.rmf
    background PHA: capella_tmp01_heg_m1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_tmp02_heg_m1.pha
       ARF: 5955/repro/tg/acisf05955_repro_heg_m1.arf
       RMF: 5955/repro/tg/acisf05955_repro_heg_m1.rmf
    background PHA: capella_tmp02_heg_m1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_tmp03_heg_m1.pha
       ARF: 6471/repro/tg/acisf06471_repro_heg_m1.arf
       RMF: 6471/repro/tg/acisf06471_repro_heg_m1.rmf
    background PHA: capella_tmp03_heg_m1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_tmp04_heg_m1.pha
       ARF: 9638/repro/tg/acisf09638_repro_heg_m1.arf
       RMF: 9638/repro/tg/acisf09638_repro_heg_m1.rmf
    background PHA: capella_tmp04_heg_m1_bkg.pha
               ARF: None
               RMF: None

The following files were created:
  capella_combo_heg_m1.pha
  capella_combo_heg_m1_bkg.pha
  capella_combo_heg_m1.arf
  capella_combo_heg_m1.rmf
Prepared to combine 5 spectra

source PHA: capella_combo_tmp00_heg_p1.pha
       ARF: 10599/repro/tg/acisf10599_repro_heg_p1.arf
       RMF: 10599/repro/tg/acisf10599_repro_heg_p1.rmf
    background PHA: capella_combo_tmp00_heg_p1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_combo_tmp01_heg_p1.pha
       ARF: 11931/repro/tg/acisf11931_repro_heg_p1.arf
       RMF: 11931/repro/tg/acisf11931_repro_heg_p1.rmf
    background PHA: capella_combo_tmp01_heg_p1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_combo_tmp02_heg_p1.pha
       ARF: 5955/repro/tg/acisf05955_repro_heg_p1.arf
       RMF: 5955/repro/tg/acisf05955_repro_heg_p1.rmf
    background PHA: capella_combo_tmp02_heg_p1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_combo_tmp03_heg_p1.pha
       ARF: 6471/repro/tg/acisf06471_repro_heg_p1.arf
       RMF: 6471/repro/tg/acisf06471_repro_heg_p1.rmf
    background PHA: capella_combo_tmp03_heg_p1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_combo_tmp04_heg_p1.pha
       ARF: 9638/repro/tg/acisf09638_repro_heg_p1.arf
       RMF: 9638/repro/tg/acisf09638_repro_heg_p1.rmf
    background PHA: capella_combo_tmp04_heg_p1_bkg.pha
               ARF: None
               RMF: None

The following files were created:
  capella_combo_heg_p1.pha
  capella_combo_heg_p1_bkg.pha
  capella_combo_heg_p1.arf
  capella_combo_heg_p1.rmf
Combining plus and minus orders
Prepared to combine 2 spectra

source PHA: capella_combo_heg_m1.pha
       ARF: capella_combo_heg_m1.arf
       RMF: capella_combo_heg_m1.rmf
    background PHA: capella_combo_heg_m1_bkg.pha
               ARF: None
               RMF: None
source PHA: capella_combo_heg_p1.pha
       ARF: capella_combo_heg_p1.arf
       RMF: capella_combo_heg_p1.rmf
    background PHA: capella_combo_heg_p1_bkg.pha
               ARF: None
               RMF: None

The following files were created:
  capella_combo_heg_abs1.pha
  capella_combo_heg_abs1_bkg.pha
  capella_combo_heg_abs1.arf
  capella_combo_heg_abs1.rmf
Created output source spectrum 'capella_combo_heg_abs1.pha'
    with background spectrum 'capella_combo_heg_abs1_bkg.pha'
    with source ARF 'capella_combo_heg_abs1.arf'
    with source RMF 'capella_combo_heg_abs1.rmf'

Example 5

combine_grating_spectra infile=@tgcat.lis out=tw_hya arf=@tgcat_arf.lis
rmf=@tgcat_rmf.lis object="TW Hya" clob+ add+

This example shows the combination of +/- HEG and MEG spectra of TW Hya spectra that have been downloaded from the TGCat catalog. The input spectra, ARF, and RMF files are input as stacks. The infile stack, tgcat.lis, is shown here:

% cat tgcat.lis
tgcat/obs_5_tgid_3522/heg_-1.pha.gz
tgcat/obs_5_tgid_3522/heg_1.pha.gz
tgcat/obs_5_tgid_3522/meg_-1.pha.gz
tgcat/obs_5_tgid_3522/meg_1.pha.gz
tgcat/obs_7435_tgid_4053/heg_-1.pha.gz
tgcat/obs_7435_tgid_4053/heg_1.pha.gz
tgcat/obs_7435_tgid_4053/meg_-1.pha.gz
tgcat/obs_7435_tgid_4053/meg_1.pha.gz
tgcat/obs_7436_tgid_4054/heg_-1.pha.gz
tgcat/obs_7436_tgid_4054/heg_1.pha.gz
tgcat/obs_7436_tgid_4054/meg_-1.pha.gz
tgcat/obs_7436_tgid_4054/meg_1.pha.gz
tgcat/obs_7437_tgid_4057/heg_-1.pha.gz
tgcat/obs_7437_tgid_4057/heg_1.pha.gz
tgcat/obs_7437_tgid_4057/meg_-1.pha.gz
tgcat/obs_7437_tgid_4057/meg_1.pha.gz
tgcat/obs_7438_tgid_4055/heg_-1.pha.gz
tgcat/obs_7438_tgid_4055/heg_1.pha.gz
tgcat/obs_7438_tgid_4055/meg_-1.pha.gz
tgcat/obs_7438_tgid_4055/meg_1.pha.gz

Since the TGCat spectra include the background in the source spectra, the bkg_pha file is left blank. The script will run tgsplit to separate the background and compute the _UP and _DOWN parts, match the ARF and RMF files, and then combine the 5 observations +/- HEG and MEG first order spectra. The final output will be

...
Created output source spectrum 'tw_hya_combo_meg_abs1.pha'
    with background spectrum 'tw_hya_combo_meg_abs1_bkg.pha'
    with source ARF 'tw_hya_combo_meg_abs1.arf'
    with source RMF 'tw_hya_combo_meg_abs1.rmf'

Created output source spectrum 'tw_hya_combo_heg_abs1.pha'
    with background spectrum 'tw_hya_combo_heg_abs1_bkg.pha'
    with source ARF 'tw_hya_combo_heg_abs1.arf'
    with source RMF 'tw_hya_combo_heg_abs1.rmf'

Example 6

combine_grating_spectra infile=@tgcat.lis out=tw_hya garm=HEG
bkg_pha=none arf=@tgcat_arf.lis rmf=@tgcat_rmf.lis object="TW Hya"
clob+ add+

This is similar to the previous example except only HEG grating will be combine and setting bkg_pha=none instructs the tool to omit the background from the combined data products. The final outputs will now just be

Created output source spectrum 'tw_hya_combo_heg_abs1.pha'
    with source ARF 'tw_hya_combo_heg_abs1.arf'
    with source RMF 'tw_hya_combo_heg_abs1.rmf'

Parameters

name	type	ftype	def	min	max	reqd	stacks
infile	file	input				yes	yes
outroot	file	output				yes
add_plusminus	boolean		no			no
garm	string		all			no
order	string		all			no
arf	file	input				no	yes
rmf	file	input				no	yes
bkg_pha	file	input				no	yes
bscale_method	string		counts			no
exposure_mode	string		sum			no
object	string					no
clobber	boolean		no			no
verbose	integer		1	0	5	no

Detailed Parameter Descriptions

Parameter=infile (file required filetype=input stacks=yes)

Source PHA files to combine

The infile may be either stack of TYPE:II pha files or a stack of TYPE:I pha files. A single TYPE:II pha file can be input when the user simply wants to add positive and negative orders in the same observation. The tool will not work with a mixture of TYPE:I and TYPE:II files.

Parameter=outroot (file required filetype=output)

Root name for output files

The script will always create TYPE:I pha files and if requested the combined background, ARF, and RMF files.

The script will create the following output files based on the available inputs:

${outroot}_combo_${part}_${order}.pha : Source spectrum
${outroot}_combo_${part}_${order}.arf : Source ARF file
${outroot}_combo_${part}_${order}.rmf: Source RMF file
${outroot}_combo_${part}_${order}_bkg.pha : Background spectrum

Where ${part} is the grating arm: heg, meg, or leg. The ${order} value is the integer order number (1, 2, 3) preceded by either "p" for the plus side, "m" for the minus side, or "abs" for when the positive and negative orders have been added (add_plusminus=yes).

Parameter=add_plusminus (boolean not required default=no)

Should the plus and minus side orders be combined?

The tool works by first combining all the spectra with the same order and grating arm. After they are combined the script can then optionally add the plus and minus orders for each absolute value of order and arm.

Parameter=garm (string not required default=all)

Which grating ARM to combine?

This parameter allows the user to filter the input on a specific grating arm: HEG, MEG, or LEG. The values are case sensitive. The default, "all", is to combine all grating arms that are found in the input files.

Parameter=order (string not required default=all)

Which grating order to combine

This parameter can be used to filter the input on a specific signed order: -3, -2, -1, 1, 2, 3. The default, "all", will combine all orders found in infile.

If add_plusminus=yes, then the order is taken to be the absolute value.

Parameter=arf (file not required filetype=input stacks=yes)

Source ARF files to combine; enter list or '@stack'

For TYPE:II pha files, the arf input files must be specified if they are to be combined. The headers of the arf files will be inspected and will be matched to the rows in the infile files. See ahelp tgsplit for more information.

For TYPE:I infiles, if this parameter is blank then the script will locate the ARF file using the ANCRFILE keyword in each of the input files. If given, then the order of the ARF files must match the order of the infile parameter. The parameter can be set to "none" and the keyword value will not be used.

Parameter=rmf (file not required filetype=input stacks=yes)

Source RMF files to combine; enter list or '@stack'

For TYPE:II pha files, the rmf input files must be specified if they are to be combined. The headers of the rmf files will be inspected and will be matched to the rows in the infile files. See ahelp tgsplit for more information.

For TYPE:I infiles, if this parameter is blank then the script will locate the RMF file using the RESPFILE keyword in each of the input files. If given, then the order of the RMF files must match the order of the infile parameter. The parameter can be set to "none" and the keyword value will not be used.

Note: RSP-style responses which have been multiplied by an ARF are not supported.

Parameter=bkg_pha (file not required filetype=input stacks=yes)

Background PHA files to combine; enter list or '@stack'

For TYPE:II pha files this parameter is ignored. The background is taken from the tgextract/tgextract2 output.

For TYPE:I pha files, if this parameter is blank then the script will locate the background file using the BACKFILE keyword in each of the input files. If given, then the order of the background files must match the order of the infile parameter. The parameter can be set to "none" and the keyword value will not be used.

Parameter=bscale_method (string not required default=counts)

How are BACKSCAL and background counts computed?

There are 3 different algorithms to combine the background counts compute the background scaling (BACKSCAL) values.

'asca'

When simply subtracting the background, the 'asca' method can be used. The combined background counts are an exposure and area weighted combination:

f = {(bkg_exp1 + ... + bkg_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ... + src_expN(src_backscalN/bkg_backscalN)]}*
f_I = f*{(src_expI/bkg_expI)*(src_backscalI/bkg_backscalI)}  
BKG_COUNTS = f_1*bkg_counts1 + f_2*bkg_counts2 + ... f_N*bkg_countsN
BKG_BACKSCAL = (src_exp1 + ... + src_expN)/[src_exp1(src_backscal1/bkg_backscal1) + ...  + src_expN(src_backscalN/bkg_backscalN)]  
SOURCE_COUNTS = src_counts1 + src_counts2 + ... + src_countsN
SOURCE_EXPOSURE = src_exp1 + src_exp2 + ... + src_expN
SOURCE_BACKSCAL = 1.0

'time'

The 'time' methods computes the total counts (unweighted) and the provides and exposure time weighted BACKSCAL value.

BKG_COUNTS = bkg_counts1 + bkg_counts2 + ... bkg_countsN
SS = [(src_exp1*src_backscal1) + ... + (src_expN*src_backscalN)] / (src_exp1 + ... + src_expN)
BB = [(bkg_exp1*bkg_backscal1) + ... + (bkg_expN*bkg_backscalN)] / (bkg_exp1 + ... + bkg_expN)
BKG_BACKSCAL = BB / SS
SOURCE_COUNTS = src_counts1 + src_counts2 + ... + src_countsN
SOURCE_EXPOSURE = src_exp1 + src_exp2 + ... + src_expN
SOURCE_BACKSCAL = 1.0

'counts'

The 'counts' method gives background counts and backscale values that are correctly weighted when the background is going to be modeled rather than subtracted. That is the background has been combined such that the errors on the background are correct.

r_i = (src_exp_i*src_backscal_i)/(bkg_exp_i*bkg_backscal_i)
m1 = (bkg_counts1*r_1) + ... + (bkg_countsN*r_N)
m2 = (bkg_counts1*r_1^2) + ... + (bkg_countsN*r_N^2)

BKG_COUNTS = (m1*m1) / m2
BKG_BACKSCAL = SOURCE_BACKSCAL * ((src_exp1+...+src_expN)/(bkg_exp1+...+bkg_expN))*(m1/m2)

SOURCE_COUNTS = src_counts1 + src_counts2 + ... + src_countsN
SOURCE_EXPOSURE = src_exp1 + src_exp2 + ... + src_expN
SOURCE_BACKSCAL = (src_exp1*src_backscal1) + ... + (src_expN*src_backscalN) / (src_exp1 + ... + src_expN)

Parameter=exposure_mode (string not required default=sum)

The default, exposure_mode=sum, should always be used unless the user is combining spectra from multiple sources in the same observation. If combining data from multiple sources in the same observation then set exposure_mode=avg (average).

When using add_plusminus=yes, the tool internally will combine the exposure times correctly using the average values.

Parameter=object (string not required default=)

The value for the OBJECT keyword in the output files

Parameter=clobber (boolean not required default=no)

OK to overwrite existing output file?

Parameter=verbose (integer not required default=1 min=0 max=5)

Debug Level(0-5)

TYPE:I vs TYPE:II

TYPE:I pha files have a single spectrum. Each row in file represents 1 channel.

TYPE:II pha files have a spectrum from each row in the file. For Chandra gratings PHA files, each row represents a different grating arm (heg, meg, leg) and order (+/-1, 2, 3).

Caveats

Any grouping flags which may be present in input source or background PHA spectra will be ignored by the script.
Combining background spectra with wildly varying spectral extraction region areas may yield misleading uncertainty estimates; i.e., some extractions will be over-represented while others will be under-represented.
If the background rates contributing to a source are significantly different in the individual spectra to be combined, it is recommended that these spectra remain separate and be modeled simultaneously - otherwise, the modeling results of the combined source spectrum could be biased towards the observation(s) with the highest background rate(s).
RSP-style responses which have been premultiplied by an ARF are not supported.
Statistical errors on the combined counts in the combined source and background spectra are only recomputed using the Gehrel's approximation if the input file has a STAT_ERR column.
Non-Chandra files are not supported.
Users cannot combine ACIS and HRC datasets.
Users cannot combine different orders, for example combine 1st, 2nd, and 3rd. Users can only combine the plus and minus orders, for example +1 and -1 orders.
Users cannot combine data from different grating arms, for example MEG and HEG data.
Users should not combine ACIS CC-mode data with ACIS TIMED mode data. Combining TIMED mode data with different readout modes (FAINT, VFAINT, GRADED) is all right.

Changes in the scripts 4.16.2 (August 2024) release

Fix a bug when trying to combine PHA2 files that do not contain background spectra.

Changes in the scripts 4.14.2 (April 2022) release

The final output files now have "_combo_" appended to the output root name. This is to avoid a problem when using mktgresp and combine_grating_spectra using the same outroot.

Changes in the scripts 4.12.1 (December 2019) release

Internal cleanup. Removes TG_M and SPEC_NUM keywords from combined files since values are merged.

Changes in the scripts 4.10.3 (October 2018) release

Recognizes when outroot is a directory and adjusts file names so they no longer begin with an underscore or period.

Changes in the script 4.10.1 (April 2018) release

Fixed a bug when the TYPE:II grating spectra do not have background.

Changes in the scripts 4.9.2 (April 2017) release

If the input spectra contain a STAT_ERR column, the script will now recompute the error using the Gehrel's approximation.

About Contributed Software

This script is not an official part of the CIAO release but is made available as "contributed" software via the CIAO scripts page. Please see this page for installation instructions.

Bugs

Problems when setting outroot the same as mktgresp: Users cannot use the same outroot value for both combine_grating_spectra and mktgresp. Depending on the other parameters, this can result in the same file names being created by both scripts for different data products which can then lead to unusual errors. It may cause combine_grating_spectra to error out with a message about missing files.