Prev
Index
Next
kadi: Thu Apr 2 08:37:19 2026
File: /proj/sot/ska/data/kadi/logs/daily.0/kadi_validate.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| kadi |
OK
|
0.09 | 1.5 |
Thu Apr 2 08:37:19 2026 |
No errors
File contents:
2026-04-02 08:36:08,689 run_validators: Validating pitch
2026-04-02 08:36:08,690 get_command_sheet_exclude_intervals: Getting exclude times from https://docs.google.com/spreadsheets/d/19d6XqBhWoFjC-z1lS1nM6wLE_zjr4GYB1lOvrEGCbKQ/export?format=csv&id=19d6XqBhWoFjC-z1lS1nM6wLE_zjr4GYB1lOvrEGCbKQ&gid=1681877928
_get_data: Getting data for conlofp between 2026:078:12:20:38.689 to 2026:092:12:51:38.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/ccdm8eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/ccdm8eng/CONLOFP.h5
_get_msid_data_from_cxc: Slicing CONLOFP arrays [246:1089504]
interpolate: Interpolating index for conlofp
interpolate: Slicing on indexes
2026-04-02 08:36:09,997 tlm: Fetching telemetry for ['pitch_comp'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for pitch_comp between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_comp_data: Getting computed values for pitch_comp
_get_data: Getting data for conlofp between 2026:078:12:05:38.689 to 2026:092:13:06:38.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/ccdm8eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/ccdm8eng/CONLOFP.h5
_get_msid_data_from_cxc: Slicing CONLOFP arrays [5768:1095904]
interpolate: Interpolating index for conlofp
interpolate: Slicing on indexes
_get_data: Getting data for aopcadmd between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/AOPCADMD.h5
_get_msid_data_from_cxc: Slicing AOPCADMD arrays [275:1089504]
_get_data: Getting data for orbitephem0_x between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/orbitephem0/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/orbitephem0/ORBITEPHEM0_X.h5
_get_msid_data_from_cxc: Slicing ORBITEPHEM0_X arrays [5186:9228]
_get_data: Getting data for orbitephem0_y between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for orbitephem0 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/orbitephem0/ORBITEPHEM0_Y.h5
_get_msid_data_from_cxc: Slicing ORBITEPHEM0_Y arrays [5186:9228]
_get_data: Getting data for orbitephem0_z between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for orbitephem0 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/orbitephem0/ORBITEPHEM0_Z.h5
_get_msid_data_from_cxc: Slicing ORBITEPHEM0_Z arrays [5186:9228]
_get_data: Getting data for solarephem0_x between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/solarephem0/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/solarephem0/SOLAREPHEM0_X.h5
_get_msid_data_from_cxc: Slicing SOLAREPHEM0_X arrays [5186:9228]
_get_data: Getting data for solarephem0_y between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for solarephem0 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/solarephem0/SOLAREPHEM0_Y.h5
_get_msid_data_from_cxc: Slicing SOLAREPHEM0_Y arrays [5186:9228]
_get_data: Getting data for solarephem0_z between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for solarephem0 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/solarephem0/SOLAREPHEM0_Z.h5
_get_msid_data_from_cxc: Slicing SOLAREPHEM0_Z arrays [5186:9228]
_get_data: Getting data for aoattqt1 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/AOATTQT1.h5
_get_msid_data_from_cxc: Slicing AOATTQT1 arrays [5973:1095904]
_get_data: Getting data for aoattqt2 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for pcad3eng 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/AOATTQT2.h5
_get_msid_data_from_cxc: Slicing AOATTQT2 arrays [5973:1095904]
_get_data: Getting data for aoattqt3 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for pcad3eng 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/AOATTQT3.h5
_get_msid_data_from_cxc: Slicing AOATTQT3 arrays [5973:1095904]
_get_data: Getting data for aoattqt4 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_msid_data_from_cxc: Using times_cache for pcad3eng 890309417.873 to 891522257.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/AOATTQT4.h5
_get_msid_data_from_cxc: Slicing AOATTQT4 arrays [5973:1095904]
fetch: Setting bads because of gaps in aoattqt1 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
fetch: Setting bads because of gaps in aoattqt2 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
fetch: Setting bads because of gaps in aoattqt3 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
fetch: Setting bads because of gaps in aoattqt4 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
interpolate: Interpolating index for orbitephem0_x
interpolate: Slicing on indexes
interpolate: Interpolating index for orbitephem0_y
interpolate: Slicing on indexes
interpolate: Interpolating index for orbitephem0_z
interpolate: Slicing on indexes
interpolate: Interpolating index for solarephem0_x
interpolate: Slicing on indexes
interpolate: Interpolating index for solarephem0_y
interpolate: Slicing on indexes
interpolate: Interpolating index for solarephem0_z
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt1
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt2
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt3
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt4
interpolate: Slicing on indexes
interpolate: Interpolating index for pitch_comp
interpolate: Slicing on indexes
2026-04-02 08:36:14,433 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:14,433 get_states: Getting commanded states ['pitch', 'pcad_mode'] between 2026:078:12:36:09.191 - 2026:092:09:10:04.041
2026-04-02 08:36:14,433 get_cmds: Getting commands from 2026:078:12:36:09.191 to 2026:092:09:10:04.041 for scenario=None
2026-04-02 08:36:14,434 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:14,435 get_cmds: Recent commands not in cache: updating local cmd_events, loads and getting recent commands
2026-04-02 08:36:14,435 read_cmd_events_from_sheet: Getting cmd_events from https://docs.google.com/spreadsheets/d/19d6XqBhWoFjC-z1lS1nM6wLE_zjr4GYB1lOvrEGCbKQ/export?format=csv
2026-04-02 08:36:14,787 get_cmd_events_from_sheet: Writing 170 cmd_events to /home/aca/.kadi/cmd_events.csv
2026-04-02 08:36:14,796 get_occweb_page: Getting OCCweb FOT/mission_planning/PRODUCTS/APPR_LOADS/2026/MAR with cache=False
2026-04-02 08:36:14,975 get_load_cmds_from_occweb_or_local: Already have /home/aca/.kadi/loads/MAR0926A.pkl.gz
2026-04-02 08:36:14,980 get_load_cmds_from_occweb_or_local: Already have /home/aca/.kadi/loads/MAR1626B.pkl.gz
2026-04-02 08:36:14,986 get_load_cmds_from_occweb_or_local: Already have /home/aca/.kadi/loads/MAR2326A.pkl.gz
2026-04-02 08:36:14,991 get_load_cmds_from_occweb_or_local: Already have /home/aca/.kadi/loads/MAR2426A.pkl.gz
2026-04-02 08:36:14,996 get_load_cmds_from_occweb_or_local: Already have /home/aca/.kadi/loads/MAR3026A.pkl.gz
2026-04-02 08:36:15,000 get_occweb_page: Getting OCCweb FOT/mission_planning/PRODUCTS/APPR_LOADS/2026/APR with cache=False
2026-04-02 08:36:15,039 update_cmd_events_and_loads_and_get_cmds_recent: Including loads MAR0926A, MAR1626B, MAR2326A, MAR2426A, MAR3026A
2026-04-02 08:36:15,042 update_cmd_events_and_loads_and_get_cmds_recent: Load MAR0926A has 1954 commands with RLTT=2026:067:21:28:00.022
2026-04-02 08:36:15,046 update_cmd_events_and_loads_and_get_cmds_recent: Load MAR1626B has 2222 commands with RLTT=2026:075:02:35:52.870
2026-04-02 08:36:15,050 update_cmd_events_and_loads_and_get_cmds_recent: Load MAR2326A has 2027 commands with RLTT=2026:081:20:44:00.000
2026-04-02 08:36:15,053 update_cmd_events_and_loads_and_get_cmds_recent: Load MAR2426A has 1443 commands with RLTT=2026:083:16:59:59.000
2026-04-02 08:36:15,057 update_cmd_events_and_loads_and_get_cmds_recent: Load MAR3026A has 2068 commands with RLTT=2026:088:21:40:04.656
2026-04-02 08:36:15,089 update_cmd_events_and_loads_and_get_cmds_recent: Including cmd_events:
Command at 2026:064:13:40:06.000
Dither at 2026:065:13:30:52.501
2026-04-02 08:36:15,092 update_cmd_events_and_loads_and_get_cmds_recent: Processing CMD_EVT Command at 2026:064:13:40:06.000 with 1 commands
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Adding 1 commands from CMD_EVT Command at 2026:064:13:40:06.000
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Processing CMD_EVT Dither at 2026:065:13:30:52.501 with 1 commands
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Adding 1 commands from CMD_EVT Dither at 2026:065:13:30:52.501
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Processing MAR0926A with 1954 commands
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Adding 1954 commands from MAR0926A
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Processing MAR1626B with 2222 commands
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Adding 2222 commands from MAR1626B
2026-04-02 08:36:15,093 update_cmd_events_and_loads_and_get_cmds_recent: Processing MAR2326A with 2027 commands
2026-04-02 08:36:15,094 update_cmd_events_and_loads_and_get_cmds_recent: Adding 2027 commands from MAR2326A
2026-04-02 08:36:15,094 update_cmd_events_and_loads_and_get_cmds_recent: Processing MAR2426A with 1443 commands
2026-04-02 08:36:15,094 update_cmd_events_and_loads_and_get_cmds_recent: Removing 1465 cmds in SCS slots [128, 129, 130, 131, 132, 133] from MAR2326A due to RLTT in MAR2426A
2026-04-02 08:36:15,096 update_cmd_events_and_loads_and_get_cmds_recent: Adding 1443 commands from MAR2426A
2026-04-02 08:36:15,096 update_cmd_events_and_loads_and_get_cmds_recent: Processing MAR3026A with 2068 commands
2026-04-02 08:36:15,096 update_cmd_events_and_loads_and_get_cmds_recent: Adding 2068 commands from MAR3026A
2026-04-02 08:36:15,497 load_pars_dict: Loaded /proj/sot/ska3/aca/data/kadi/cmds2.pkl with 189299 pars
2026-04-02 08:36:15,514 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:15,517 get_cmds: Getting commands from 2026:071:12:36:09.191 to 2026:078:12:36:09.191 for scenario=None
2026-04-02 08:36:15,517 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:15,518 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:15,519 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:23,106 get_plot_figure: Compressing Telem data for state pitch
2026-04-02 08:36:25,360 get_plot_figure: Creating Telem scatter plot for state pitch
2026-04-02 08:36:25,560 get_plot_figure: Compressing State data for state pitch
2026-04-02 08:36:27,346 get_plot_figure: Creating State scatter plot for state pitch
2026-04-02 08:36:27,487 get_plot_html: Creating HTML for state pitch
2026-04-02 08:36:27,856 run_validators: Validating off_nom_roll
_get_data: Getting data for conlofp between 2026:078:12:20:38.689 to 2026:092:12:51:38.689
interpolate: Interpolating index for conlofp
interpolate: Slicing on indexes
2026-04-02 08:36:28,047 tlm: Fetching telemetry for ['roll_comp'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for roll_comp between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_comp_data: Getting computed values for roll_comp
_get_data: Getting data for conlofp between 2026:078:12:05:38.689 to 2026:092:13:06:38.689
interpolate: Interpolating index for conlofp
interpolate: Slicing on indexes
_get_data: Getting data for aopcadmd between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_data: Getting data for orbitephem0_x between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for orbitephem0_y between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for orbitephem0_z between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for solarephem0_x between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for solarephem0_y between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for solarephem0_z between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for aoattqt1 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for aoattqt2 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for aoattqt3 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
_get_data: Getting data for aoattqt4 between 2026:078:12:09:08.689 to 2026:092:13:03:08.689
fetch: Setting bads because of gaps in aoattqt1 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
fetch: Setting bads because of gaps in aoattqt2 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
fetch: Setting bads because of gaps in aoattqt3 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
fetch: Setting bads because of gaps in aoattqt4 between 2026:092:09:10:09.166 to 2026:092:13:03:08.116
interpolate: Interpolating index for orbitephem0_x
interpolate: Slicing on indexes
interpolate: Interpolating index for orbitephem0_y
interpolate: Slicing on indexes
interpolate: Interpolating index for orbitephem0_z
interpolate: Slicing on indexes
interpolate: Interpolating index for solarephem0_x
interpolate: Slicing on indexes
interpolate: Interpolating index for solarephem0_y
interpolate: Slicing on indexes
interpolate: Interpolating index for solarephem0_z
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt1
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt2
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt3
interpolate: Slicing on indexes
interpolate: Interpolating index for aoattqt4
interpolate: Slicing on indexes
interpolate: Interpolating index for roll_comp
interpolate: Slicing on indexes
2026-04-02 08:36:29,246 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:29,246 get_states: Getting commanded states ['off_nom_roll', 'pitch', 'pcad_mode'] between 2026:078:12:36:09.191 - 2026:092:09:10:04.041
2026-04-02 08:36:29,247 get_cmds: Getting commands from 2026:078:12:36:09.191 to 2026:092:09:10:04.041 for scenario=None
2026-04-02 08:36:29,247 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:29,247 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:29,248 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:29,250 get_cmds: Getting commands from 2026:071:12:36:09.191 to 2026:078:12:36:09.191 for scenario=None
2026-04-02 08:36:29,251 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:29,251 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:29,252 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:33,866 get_plot_figure: Compressing Telem data for state off_nom_roll
2026-04-02 08:36:35,759 get_plot_figure: Creating Telem scatter plot for state off_nom_roll
2026-04-02 08:36:35,784 get_plot_figure: Compressing State data for state off_nom_roll
2026-04-02 08:36:37,567 get_plot_figure: Creating State scatter plot for state off_nom_roll
2026-04-02 08:36:37,620 get_plot_html: Creating HTML for state off_nom_roll
2026-04-02 08:36:37,645 run_validators: Validating dither
_get_data: Getting data for conlofp between 2026:078:12:20:38.689 to 2026:092:12:51:38.689
interpolate: Interpolating index for conlofp
interpolate: Slicing on indexes
2026-04-02 08:36:37,839 get_cmds: Getting commands from 2026:078:12:16:08.689 to 2026:092:12:36:08.689 for scenario=None
2026-04-02 08:36:37,839 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:37,840 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:37,841 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:37,908 add_exclude_interval: dither: excluding interval 2026:079:11:19:59.000 - 2026:079:11:32:59.000: SRDC
2026-04-02 08:36:37,909 tlm: Fetching telemetry for ['aodithen'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for aodithen between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad8eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad8eng/AODITHEN.h5
_get_msid_data_from_cxc: Slicing AODITHEN arrays [35:136188]
interpolate: Interpolating index for aodithen
interpolate: Slicing on indexes
2026-04-02 08:36:38,100 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:38,101 get_states: Getting commanded states ['dither'] between 2026:078:12:36:15.904 - 2026:092:09:10:00.312
2026-04-02 08:36:38,101 get_cmds: Getting commands from 2026:078:12:36:15.904 to 2026:092:09:10:00.312 for scenario=None
2026-04-02 08:36:38,101 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:38,101 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:38,102 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:38,104 get_cmds: Getting commands from 2026:071:12:36:15.904 to 2026:078:12:36:15.904 for scenario=None
2026-04-02 08:36:38,104 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:38,105 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:38,105 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:38,840 get_plot_figure: Compressing Telem data for state dither
2026-04-02 08:36:39,058 get_plot_figure: Creating Telem scatter plot for state dither
2026-04-02 08:36:39,059 get_plot_figure: Compressing State data for state dither
2026-04-02 08:36:39,285 get_plot_figure: Creating State scatter plot for state dither
2026-04-02 08:36:39,331 get_plot_html: Creating HTML for state dither
2026-04-02 08:36:39,354 run_validators: Validating pcad_mode
2026-04-02 08:36:39,366 get_cmds: Getting commands from 2026:078:12:16:08.689 to 2026:092:12:36:08.689 for scenario=None
2026-04-02 08:36:39,366 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:39,367 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:39,367 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:39,409 add_exclude_interval: pcad_mode: excluding interval 2026:079:11:19:59.000 - 2026:079:11:32:59.000: SRDC
2026-04-02 08:36:39,410 tlm: Fetching telemetry for ['aopcadmd'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for aopcadmd between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad3eng/AOPCADMD.h5
_get_msid_data_from_cxc: Slicing AOPCADMD arrays [275:1089504]
interpolate: Interpolating index for aopcadmd
interpolate: Slicing on indexes
2026-04-02 08:36:39,685 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:39,685 get_states: Getting commanded states ['pcad_mode'] between 2026:078:12:36:08.729 - 2026:092:09:10:04.412
2026-04-02 08:36:39,685 get_cmds: Getting commands from 2026:078:12:36:08.729 to 2026:092:09:10:04.412 for scenario=None
2026-04-02 08:36:39,685 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:39,686 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:39,687 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:39,689 get_cmds: Getting commands from 2026:071:12:36:08.729 to 2026:078:12:36:08.729 for scenario=None
2026-04-02 08:36:39,689 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:39,689 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:39,690 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:44,476 get_plot_figure: Compressing Telem data for state pcad_mode
2026-04-02 08:36:46,237 get_plot_figure: Creating Telem scatter plot for state pcad_mode
2026-04-02 08:36:46,241 get_plot_figure: Compressing State data for state pcad_mode
2026-04-02 08:36:47,996 get_plot_figure: Creating State scatter plot for state pcad_mode
2026-04-02 08:36:48,033 get_plot_html: Creating HTML for state pcad_mode
2026-04-02 08:36:48,055 run_validators: Validating simpos
2026-04-02 08:36:48,061 tlm: Fetching telemetry for ['3tscpos'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for 3tscpos between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/sim_mrg/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/sim_mrg/3TSCPOS.h5
_get_msid_data_from_cxc: Slicing 3TSCPOS arrays [8:33989]
interpolate: Interpolating index for 3tscpos
interpolate: Slicing on indexes
2026-04-02 08:36:48,298 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:48,298 get_states: Getting commanded states ['simpos'] between 2026:078:12:36:40.504 - 2026:092:09:09:36.225
2026-04-02 08:36:48,298 get_cmds: Getting commands from 2026:078:12:36:40.504 to 2026:092:09:09:36.225 for scenario=None
2026-04-02 08:36:48,298 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:48,299 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:48,299 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:48,301 get_cmds: Getting commands from 2026:071:12:36:40.504 to 2026:078:12:36:40.504 for scenario=None
2026-04-02 08:36:48,302 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:48,303 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:48,303 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:48,451 get_plot_figure: Compressing Telem data for state simpos
2026-04-02 08:36:48,514 get_plot_figure: Creating Telem scatter plot for state simpos
2026-04-02 08:36:48,518 get_plot_figure: Compressing State data for state simpos
2026-04-02 08:36:48,580 get_plot_figure: Creating State scatter plot for state simpos
2026-04-02 08:36:48,589 get_plot_html: Creating HTML for state simpos
2026-04-02 08:36:48,611 run_validators: Validating obsid
2026-04-02 08:36:48,618 tlm: Fetching telemetry for ['cobsrqid'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for cobsrqid between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/ccdm10eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/ccdm10eng/COBSRQID.h5
_get_msid_data_from_cxc: Slicing COBSRQID arrays [138:544752]
interpolate: Interpolating index for cobsrqid
interpolate: Slicing on indexes
2026-04-02 08:36:48,897 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:48,897 get_states: Getting commanded states ['obsid'] between 2026:078:12:36:09.754 - 2026:092:09:10:04.156
2026-04-02 08:36:48,898 get_cmds: Getting commands from 2026:078:12:36:09.754 to 2026:092:09:10:04.156 for scenario=None
2026-04-02 08:36:48,898 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:48,898 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:48,899 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:48,901 get_cmds: Getting commands from 2026:071:12:36:09.754 to 2026:078:12:36:09.754 for scenario=None
2026-04-02 08:36:48,901 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:48,902 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:48,902 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:51,019 get_plot_figure: Compressing Telem data for state obsid
2026-04-02 08:36:51,902 get_plot_figure: Creating Telem scatter plot for state obsid
2026-04-02 08:36:51,905 get_plot_figure: Compressing State data for state obsid
2026-04-02 08:36:52,782 get_plot_figure: Creating State scatter plot for state obsid
2026-04-02 08:36:52,798 get_plot_html: Creating HTML for state obsid
2026-04-02 08:36:52,821 run_validators: Validating letg
2026-04-02 08:36:52,828 tlm: Fetching telemetry for ['4ootgsel', '4ootgmtn'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for 4ootgsel between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/tel2eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/tel2eng/4OOTGSEL.h5
_get_msid_data_from_cxc: Slicing 4OOTGSEL arrays [9:34047]
_get_data: Getting data for 4ootgmtn between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Using times_cache for tel2eng 890310137.873 to 891521537.8730001
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/tel2eng/4OOTGMTN.h5
_get_msid_data_from_cxc: Slicing 4OOTGMTN arrays [9:34047]
interpolate: Interpolating index for 4ootgsel
interpolate: Slicing on indexes
interpolate: Interpolating index for 4ootgmtn
interpolate: Slicing on indexes
2026-04-02 08:36:53,318 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:53,318 get_states: Getting commanded states ['letg'] between 2026:078:12:36:40.504 - 2026:092:09:09:58.006
2026-04-02 08:36:53,318 get_cmds: Getting commands from 2026:078:12:36:40.504 to 2026:092:09:09:58.006 for scenario=None
2026-04-02 08:36:53,318 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:53,319 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:53,319 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:53,321 get_cmds: Getting commands from 2026:071:12:36:40.504 to 2026:078:12:36:40.504 for scenario=None
2026-04-02 08:36:53,322 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:53,322 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:53,323 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:53,330 get_cmds: Getting commands from 2026:048:12:36:40.504 to 2026:078:12:36:40.504 for scenario=None
2026-04-02 08:36:53,330 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:53,331 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:53,331 _merge_cmds_archive_recent: Merging cmds_recent with archive commands from 2026:048:12:36:40.504
2026-04-02 08:36:55,213 load_idx_cmds: Loaded /proj/sot/ska3/aca/data/kadi/cmds2.h5 with 1797152 commands
2026-04-02 08:36:55,226 get_matching_block_idx: Selecting commands from cmds_arch[0:]
2026-04-02 08:36:55,549 get_matching_block_idx: Matching blocks for (a) recent commands and (b) existing HDF5
2026-04-02 08:36:55,549 get_matching_block_idx: Match(a=0, b=0, size=1)
2026-04-02 08:36:55,549 get_matching_block_idx: Match(a=253, b=1, size=1)
2026-04-02 08:36:55,549 get_matching_block_idx: Match(a=1213, b=2, size=1)
2026-04-02 08:36:55,549 get_matching_block_idx: Match(a=1217, b=3, size=16)
2026-04-02 08:36:55,549 get_matching_block_idx: Match(a=1234, b=19, size=8395)
2026-04-02 08:36:55,549 get_matching_block_idx: Match(a=9629, b=8414, size=0)
2026-04-02 08:36:55,549 get_matching_block_idx: Diffs between (a) recent commands and (b) existing HDF5
2026-04-02 08:36:55,549 get_matching_block_idx: ('equal', 0, 1, 0, 1)
2026-04-02 08:36:55,549 get_matching_block_idx: ('delete', 1, 253, 1, 1)
2026-04-02 08:36:55,549 get_matching_block_idx: ('equal', 253, 254, 1, 2)
2026-04-02 08:36:55,549 get_matching_block_idx: ('delete', 254, 1213, 2, 2)
2026-04-02 08:36:55,549 get_matching_block_idx: ('equal', 1213, 1214, 2, 3)
2026-04-02 08:36:55,549 get_matching_block_idx: ('delete', 1214, 1217, 3, 3)
2026-04-02 08:36:55,549 get_matching_block_idx: ('equal', 1217, 1233, 3, 19)
2026-04-02 08:36:55,549 get_matching_block_idx: ('delete', 1233, 1234, 19, 19)
2026-04-02 08:36:55,549 get_matching_block_idx: ('equal', 1234, 9629, 19, 8414)
2026-04-02 08:36:55,552 get_cmds: Getting commands from archive + recent: start < recent loads start + 3 days for scenario=None
2026-04-02 08:36:55,710 get_plot_figure: Compressing Telem data for state letg
2026-04-02 08:36:55,764 get_plot_figure: Creating Telem scatter plot for state letg
2026-04-02 08:36:55,766 get_plot_figure: Compressing State data for state letg
2026-04-02 08:36:55,821 get_plot_figure: Creating State scatter plot for state letg
2026-04-02 08:36:55,828 get_plot_html: Creating HTML for state letg
2026-04-02 08:36:55,853 run_validators: Validating hetg
2026-04-02 08:36:55,859 tlm: Fetching telemetry for ['4ootgsel', '4ootgmtn'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for 4ootgsel between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_data: Getting data for 4ootgmtn between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
interpolate: Interpolating index for 4ootgsel
interpolate: Slicing on indexes
interpolate: Interpolating index for 4ootgmtn
interpolate: Slicing on indexes
2026-04-02 08:36:55,874 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:55,875 get_states: Getting commanded states ['hetg'] between 2026:078:12:36:40.504 - 2026:092:09:09:58.006
2026-04-02 08:36:55,875 get_cmds: Getting commands from 2026:078:12:36:40.504 to 2026:092:09:09:58.006 for scenario=None
2026-04-02 08:36:55,875 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:55,875 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:55,876 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:55,878 get_cmds: Getting commands from 2026:071:12:36:40.504 to 2026:078:12:36:40.504 for scenario=None
2026-04-02 08:36:55,878 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:55,878 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:55,879 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:55,886 get_cmds: Getting commands from 2026:048:12:36:40.504 to 2026:078:12:36:40.504 for scenario=None
2026-04-02 08:36:55,886 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:55,886 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:55,887 _merge_cmds_archive_recent: Merging cmds_recent with archive commands from 2026:048:12:36:40.504
2026-04-02 08:36:55,889 get_cmds: Getting commands from archive + recent: start < recent loads start + 3 days for scenario=None
2026-04-02 08:36:56,046 get_plot_figure: Compressing Telem data for state hetg
2026-04-02 08:36:56,102 get_plot_figure: Creating Telem scatter plot for state hetg
2026-04-02 08:36:56,103 get_plot_figure: Compressing State data for state hetg
2026-04-02 08:36:56,158 get_plot_figure: Creating State scatter plot for state hetg
2026-04-02 08:36:56,166 get_plot_html: Creating HTML for state hetg
2026-04-02 08:36:56,190 run_validators: Validating sun_pos_mon
2026-04-02 08:36:56,198 tlm: Fetching telemetry for ['aopssupm'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for aopssupm between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad8eng/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/pcad8eng/AOPSSUPM.h5
_get_msid_data_from_cxc: Slicing AOPSSUPM arrays [35:136188]
interpolate: Interpolating index for aopssupm
interpolate: Slicing on indexes
filter_bad: Filtering bad values for aopssupm
2026-04-02 08:36:56,531 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:56,531 get_states: Getting commanded states ['sun_pos_mon'] between 2026:078:12:50:04.104 - 2026:092:09:09:57.493
2026-04-02 08:36:56,531 get_cmds: Getting commands from 2026:078:12:50:04.104 to 2026:092:09:09:57.493 for scenario=None
2026-04-02 08:36:56,531 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:56,532 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:56,532 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:56,534 get_cmds: Getting commands from 2026:071:12:50:04.104 to 2026:078:12:50:04.104 for scenario=None
2026-04-02 08:36:56,535 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:56,535 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:56,536 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:56,549 get_cmds: Getting commands from 2026:048:12:50:04.104 to 2026:078:12:50:04.104 for scenario=None
2026-04-02 08:36:56,549 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:56,550 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:56,550 _merge_cmds_archive_recent: Merging cmds_recent with archive commands from 2026:048:12:50:04.104
2026-04-02 08:36:56,552 get_cmds: Getting commands from archive + recent: start < recent loads start + 3 days for scenario=None
2026-04-02 08:36:56,571 get_cmds: Getting commands from 2025:263:12:50:04.104 to 2026:078:12:50:04.104 for scenario=None
2026-04-02 08:36:56,572 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:56,572 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:56,573 _merge_cmds_archive_recent: Merging cmds_recent with archive commands from 2025:263:12:50:04.104
2026-04-02 08:36:56,576 get_cmds: Getting commands from archive + recent: start < recent loads start + 3 days for scenario=None
2026-04-02 08:36:57,148 get_plot_figure: Compressing Telem data for state sun_pos_mon
2026-04-02 08:36:57,365 get_plot_figure: Creating Telem scatter plot for state sun_pos_mon
2026-04-02 08:36:57,368 get_plot_figure: Compressing State data for state sun_pos_mon
2026-04-02 08:36:57,585 get_plot_figure: Creating State scatter plot for state sun_pos_mon
2026-04-02 08:36:57,599 get_plot_html: Creating HTML for state sun_pos_mon
2026-04-02 08:36:57,621 run_validators: Validating dpa_power
2026-04-02 08:36:57,629 tlm: Fetching telemetry for ['dpa_power'] between 2026:078:12:36:08.689 and 2026:092:12:36:08.689
_get_data: Getting data for dpa_power between 2026:078:12:21:08.689 to 2026:092:12:51:08.689
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/dp_acispow128/TIME.h5
_get_msid_data_from_cxc: Reading /proj/sot/ska3/aca/data/eng_archive/data/dp_acispow128/DP_DPA_POWER.h5
_get_msid_data_from_cxc: Slicing DP_DPA_POWER arrays [120:33855]
interpolate: Interpolating index for dpa_power
interpolate: Slicing on indexes
2026-04-02 08:36:57,918 get_states: Using kadi.commands.states to get cmd_states
2026-04-02 08:36:57,918 get_states: Getting commanded states ['dpa_power'] between 2026:078:12:36:34.816 - 2026:092:09:10:01.081
2026-04-02 08:36:57,918 get_cmds: Getting commands from 2026:078:12:36:34.816 to 2026:092:09:10:01.081 for scenario=None
2026-04-02 08:36:57,918 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:57,919 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:57,920 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:36:57,922 get_cmds: Getting commands from 2026:071:12:36:34.816 to 2026:078:12:36:34.816 for scenario=None
2026-04-02 08:36:57,922 get_cmds: Cache key: (None, None, 30.0, None)
2026-04-02 08:36:57,923 get_cmds: Getting recent commands from cache cache_key=(None, None, 30.0, None)
2026-04-02 08:36:57,923 get_cmds: Getting commands from recent only: start and stop are within recent commands for scenario=None
2026-04-02 08:37:17,041 get_plot_figure: Compressing Telem data for state dpa_power
2026-04-02 08:37:17,619 get_plot_figure: Creating Telem scatter plot for state dpa_power
2026-04-02 08:37:17,891 get_plot_figure: Compressing State data for state dpa_power
2026-04-02 08:37:17,957 get_plot_figure: Creating State scatter plot for state dpa_power
2026-04-02 08:37:18,026 get_plot_html: Creating HTML for state dpa_power