Prev
Index
Next
centroid_dashboard: Wed Sep 2 13:17:31 2026
File: /proj/sot/ska/data/ska_trend/centroid_dashboard/logs/daily.0/centroid_dashboard.log
| Task | Status | Age | MaxAge | Last Date |
|---|
| centroid_dashboard |
OK
|
0.89 | 1.0 |
Wed Sep 2 13:17:31 2026 |
No errors
File contents:
2026-09-02 13:16:12,610 main: Processing from 2026:238:17:16:12.609 to 2026:245:17:16:12.609
2026-09-02 13:16:18,551 get_observations: Found observation 32497 at 2026:238:20:45:18.462 with 1 manvrs
2026-09-02 13:16:18,552 get_observations: Found observation 41197 at 2026:238:21:15:50.252 with 1 manvrs
2026-09-02 13:16:18,552 get_observations: Found observation 30409 at 2026:238:22:08:37.088 with 1 manvrs
2026-09-02 13:16:18,553 get_observations: Found observation 41196 at 2026:239:17:36:20.024 with 1 manvrs
2026-09-02 13:16:18,553 get_observations: Found observation 31998 at 2026:239:18:19:10.790 with 1 manvrs
2026-09-02 13:16:18,554 get_observations: Found observation 31061 at 2026:239:22:07:06.551 with 1 manvrs
2026-09-02 13:16:18,554 get_observations: Found observation 32516 at 2026:240:00:24:47.635 with 1 manvrs
2026-09-02 13:16:18,555 get_observations: Found observation 32510 at 2026:240:05:06:44.534 with 1 manvrs
2026-09-02 13:16:18,555 get_observations: Found observation 41195 at 2026:240:12:00:38.791 with 1 manvrs
2026-09-02 13:16:18,556 get_observations: Found observation 41194 at 2026:240:12:59:43.134 with 1 manvrs
2026-09-02 13:16:18,556 get_observations: Found observation 41193 at 2026:240:14:32:53.795 with 1 manvrs
2026-09-02 13:16:18,556 get_observations: Found observation 41192 at 2026:240:15:32:44.532 with 1 manvrs
2026-09-02 13:16:18,557 get_observations: Found observation 41191 at 2026:240:17:15:40.252 with 1 manvrs
2026-09-02 13:16:18,557 get_observations: Found observation 41190 at 2026:240:22:20:40.252 with 1 manvrs
2026-09-02 13:16:18,558 get_observations: Found observation 30203 at 2026:241:00:18:20.525 with 1 manvrs
2026-09-02 13:16:18,558 get_observations: Found observation 32508 at 2026:241:05:23:14.337 with 1 manvrs
2026-09-02 13:16:18,559 get_observations: Found observation 32515 at 2026:241:08:48:08.151 with 1 manvrs
2026-09-02 13:16:18,559 get_observations: Found observation 32507 at 2026:241:13:56:02.261 with 1 manvrs
2026-09-02 13:16:18,560 get_observations: Found observation 30505 at 2026:241:17:38:21.574 with 2 manvrs
2026-09-02 13:16:18,560 get_observations: Found observation 30135 at 2026:241:22:26:17.654 with 1 manvrs
2026-09-02 13:16:18,561 get_observations: Found observation 26973 at 2026:242:07:09:15.784 with 1 manvrs
2026-09-02 13:16:18,561 get_observations: Found observation 32509 at 2026:242:09:19:44.869 with 1 manvrs
2026-09-02 13:16:18,562 get_observations: Found observation 32512 at 2026:242:12:59:37.474 with 2 manvrs
2026-09-02 13:16:18,562 get_observations: Found observation 32513 at 2026:242:17:35:34.089 with 1 manvrs
2026-09-02 13:16:18,562 get_observations: Found observation 30424 at 2026:242:22:39:04.287 with 1 manvrs
2026-09-02 13:16:18,563 get_observations: Found observation 41187 at 2026:243:03:29:29.731 with 1 manvrs
2026-09-02 13:16:18,563 get_observations: Found observation 41186 at 2026:243:05:25:40.252 with 1 manvrs
2026-09-02 13:16:18,564 get_observations: Found observation 41185 at 2026:243:06:55:40.252 with 1 manvrs
2026-09-02 13:16:18,564 get_observations: Found observation 41184 at 2026:243:09:25:40.252 with 1 manvrs
2026-09-02 13:16:18,565 get_observations: Found observation 41183 at 2026:243:11:25:40.252 with 1 manvrs
2026-09-02 13:16:18,565 get_observations: Found observation 41182 at 2026:243:12:40:40.252 with 1 manvrs
2026-09-02 13:16:18,566 get_observations: Found observation 41181 at 2026:243:13:55:40.252 with 1 manvrs
2026-09-02 13:16:18,566 get_observations: Found observation 30196 at 2026:243:15:46:57.137 with 1 manvrs
2026-09-02 13:16:18,567 get_observations: Found observation 32535 at 2026:243:19:48:53.849 with 1 manvrs
2026-09-02 13:16:18,567 get_observations: Found observation 32529 at 2026:243:20:58:44.095 with 1 manvrs
2026-09-02 13:16:18,567 get_observations: Found observation 30440 at 2026:244:02:34:15.237 with 1 manvrs
2026-09-02 13:16:18,568 get_observations: Found observation 31881 at 2026:244:05:59:32.558 with 1 manvrs
2026-09-02 13:16:18,568 get_observations: Found observation 32530 at 2026:244:14:55:38.540 with 1 manvrs
2026-09-02 13:16:18,569 get_observations: Found observation 30402 at 2026:244:18:05:00.141 with 1 manvrs
2026-09-02 13:16:18,569 get_observations: Found observation 30441 at 2026:245:04:04:50.229 with 1 manvrs
2026-09-02 13:16:18,570 get_observations: Found observation 32520 at 2026:245:06:19:18.904 with 1 manvrs
2026-09-02 13:16:18,570 get_observations: Found observation 32517 at 2026:245:09:49:22.983 with 1 manvrs
2026-09-02 13:16:18,571 get_observations: Found observation 30438 at 2026:245:13:26:05.260 with 1 manvrs
2026-09-02 13:16:18,571 main: Found 43 observations
2026-09-02 13:16:18,571 main: ********************************************************************************
2026-09-02 13:16:18,571 main: Processing observation 32497
2026-09-02 13:16:18,571 main: ********************************************************************************
2026-09-02 13:16:18,589 obs_link_from_info: Found prev obsid 30407
2026-09-02 13:16:18,593 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 13:16:18,594 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:27,417 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/32497/centroid_resids.pkl
2026-09-02 12:16:27,419 write_centroid_resids: Centroid residuals file exists, skipping
2026-09-02 12:16:27,419 plot_crs_time: Plot file exists, skipping
2026-09-02 12:16:27,419 plot_crs_scatter: Plot file exists, skipping
2026-09-02 12:16:28,696 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/32497/n_kalman_delta_roll.png
2026-09-02 12:16:29,037 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/32497/index.html
2026-09-02 12:16:29,040 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/32497/info.json
2026-09-02 12:16:29,045 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32497/index.html
2026-09-02 12:16:29,047 main:
2026-09-02 12:16:29,047 main: ********************************************************************************
2026-09-02 12:16:29,047 main: Processing observation 41197
2026-09-02 12:16:29,047 main: ********************************************************************************
2026-09-02 12:16:29,048 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,051 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,057 process_obs: ObsID 41197 already processed, skipping
2026-09-02 12:16:29,057 main:
2026-09-02 12:16:29,057 main: ********************************************************************************
2026-09-02 12:16:29,057 main: Processing observation 30409
2026-09-02 12:16:29,057 main: ********************************************************************************
2026-09-02 12:16:29,058 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,060 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,066 process_obs: ObsID 30409 already processed, skipping
2026-09-02 12:16:29,066 main:
2026-09-02 12:16:29,066 main: ********************************************************************************
2026-09-02 12:16:29,066 main: Processing observation 41196
2026-09-02 12:16:29,066 main: ********************************************************************************
2026-09-02 12:16:29,067 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,069 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,075 process_obs: ObsID 41196 already processed, skipping
2026-09-02 12:16:29,075 main:
2026-09-02 12:16:29,075 main: ********************************************************************************
2026-09-02 12:16:29,075 main: Processing observation 31998
2026-09-02 12:16:29,075 main: ********************************************************************************
2026-09-02 12:16:29,076 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,078 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,083 process_obs: ObsID 31998 already processed, skipping
2026-09-02 12:16:29,083 main:
2026-09-02 12:16:29,083 main: ********************************************************************************
2026-09-02 12:16:29,083 main: Processing observation 31061
2026-09-02 12:16:29,083 main: ********************************************************************************
2026-09-02 12:16:29,084 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,086 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,092 process_obs: ObsID 31061 already processed, skipping
2026-09-02 12:16:29,092 main:
2026-09-02 12:16:29,092 main: ********************************************************************************
2026-09-02 12:16:29,092 main: Processing observation 32516
2026-09-02 12:16:29,092 main: ********************************************************************************
2026-09-02 12:16:29,093 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,095 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,103 process_obs: ObsID 32516 already processed, skipping
2026-09-02 12:16:29,103 main:
2026-09-02 12:16:29,103 main: ********************************************************************************
2026-09-02 12:16:29,103 main: Processing observation 32510
2026-09-02 12:16:29,103 main: ********************************************************************************
2026-09-02 12:16:29,104 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,106 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,111 process_obs: ObsID 32510 already processed, skipping
2026-09-02 12:16:29,111 main:
2026-09-02 12:16:29,111 main: ********************************************************************************
2026-09-02 12:16:29,111 main: Processing observation 41195
2026-09-02 12:16:29,111 main: ********************************************************************************
2026-09-02 12:16:29,112 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,114 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,120 process_obs: ObsID 41195 already processed, skipping
2026-09-02 12:16:29,120 main:
2026-09-02 12:16:29,120 main: ********************************************************************************
2026-09-02 12:16:29,120 main: Processing observation 41194
2026-09-02 12:16:29,120 main: ********************************************************************************
2026-09-02 12:16:29,122 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,123 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,128 process_obs: ObsID 41194 already processed, skipping
2026-09-02 12:16:29,128 main:
2026-09-02 12:16:29,128 main: ********************************************************************************
2026-09-02 12:16:29,128 main: Processing observation 41193
2026-09-02 12:16:29,128 main: ********************************************************************************
2026-09-02 12:16:29,129 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,131 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,135 process_obs: ObsID 41193 already processed, skipping
2026-09-02 12:16:29,135 main:
2026-09-02 12:16:29,135 main: ********************************************************************************
2026-09-02 12:16:29,135 main: Processing observation 41192
2026-09-02 12:16:29,135 main: ********************************************************************************
2026-09-02 12:16:29,137 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,140 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,150 process_obs: ObsID 41192 already processed, skipping
2026-09-02 12:16:29,150 main:
2026-09-02 12:16:29,150 main: ********************************************************************************
2026-09-02 12:16:29,150 main: Processing observation 41191
2026-09-02 12:16:29,150 main: ********************************************************************************
2026-09-02 12:16:29,157 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,159 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,164 process_obs: ObsID 41191 already processed, skipping
2026-09-02 12:16:29,164 main:
2026-09-02 12:16:29,164 main: ********************************************************************************
2026-09-02 12:16:29,164 main: Processing observation 41190
2026-09-02 12:16:29,164 main: ********************************************************************************
2026-09-02 12:16:29,165 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,167 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,171 process_obs: ObsID 41190 already processed, skipping
2026-09-02 12:16:29,171 main:
2026-09-02 12:16:29,171 main: ********************************************************************************
2026-09-02 12:16:29,171 main: Processing observation 30203
2026-09-02 12:16:29,171 main: ********************************************************************************
2026-09-02 12:16:29,172 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,175 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,180 process_obs: ObsID 30203 already processed, skipping
2026-09-02 12:16:29,180 main:
2026-09-02 12:16:29,180 main: ********************************************************************************
2026-09-02 12:16:29,180 main: Processing observation 32508
2026-09-02 12:16:29,180 main: ********************************************************************************
2026-09-02 12:16:29,182 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,183 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,189 process_obs: ObsID 32508 already processed, skipping
2026-09-02 12:16:29,189 main:
2026-09-02 12:16:29,189 main: ********************************************************************************
2026-09-02 12:16:29,189 main: Processing observation 32515
2026-09-02 12:16:29,189 main: ********************************************************************************
2026-09-02 12:16:29,190 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,192 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,197 process_obs: ObsID 32515 already processed, skipping
2026-09-02 12:16:29,197 main:
2026-09-02 12:16:29,197 main: ********************************************************************************
2026-09-02 12:16:29,197 main: Processing observation 32507
2026-09-02 12:16:29,197 main: ********************************************************************************
2026-09-02 12:16:29,198 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,200 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,205 process_obs: ObsID 32507 already processed, skipping
2026-09-02 12:16:29,205 main:
2026-09-02 12:16:29,205 main: ********************************************************************************
2026-09-02 12:16:29,205 main: Processing observation 30505
2026-09-02 12:16:29,205 main: ********************************************************************************
2026-09-02 12:16:29,207 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,210 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,218 process_obs: ObsID 30505 already processed, skipping
2026-09-02 12:16:29,218 main:
2026-09-02 12:16:29,218 main: ********************************************************************************
2026-09-02 12:16:29,218 main: Processing observation 30135
2026-09-02 12:16:29,218 main: ********************************************************************************
2026-09-02 12:16:29,221 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,226 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,237 process_obs: ObsID 30135 already processed, skipping
2026-09-02 12:16:29,237 main:
2026-09-02 12:16:29,237 main: ********************************************************************************
2026-09-02 12:16:29,237 main: Processing observation 26973
2026-09-02 12:16:29,237 main: ********************************************************************************
2026-09-02 12:16:29,239 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,243 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,254 process_obs: ObsID 26973 already processed, skipping
2026-09-02 12:16:29,254 main:
2026-09-02 12:16:29,254 main: ********************************************************************************
2026-09-02 12:16:29,254 main: Processing observation 32509
2026-09-02 12:16:29,254 main: ********************************************************************************
2026-09-02 12:16:29,255 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,258 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,265 process_obs: ObsID 32509 already processed, skipping
2026-09-02 12:16:29,265 main:
2026-09-02 12:16:29,265 main: ********************************************************************************
2026-09-02 12:16:29,265 main: Processing observation 32512
2026-09-02 12:16:29,265 main: ********************************************************************************
2026-09-02 12:16:29,266 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,275 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,283 process_obs: ObsID 32512 already processed, skipping
2026-09-02 12:16:29,283 main:
2026-09-02 12:16:29,283 main: ********************************************************************************
2026-09-02 12:16:29,283 main: Processing observation 32513
2026-09-02 12:16:29,283 main: ********************************************************************************
2026-09-02 12:16:29,284 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,287 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG2426A/last.html
2026-09-02 12:16:29,311 process_obs: ObsID 32513 already processed, skipping
2026-09-02 12:16:29,311 main:
2026-09-02 12:16:29,311 main: ********************************************************************************
2026-09-02 12:16:29,311 main: Processing observation 30424
2026-09-02 12:16:29,311 main: ********************************************************************************
2026-09-02 12:16:29,313 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,314 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,326 process_obs: ObsID 30424 already processed, skipping
2026-09-02 12:16:29,326 main:
2026-09-02 12:16:29,326 main: ********************************************************************************
2026-09-02 12:16:29,326 main: Processing observation 41187
2026-09-02 12:16:29,326 main: ********************************************************************************
2026-09-02 12:16:29,327 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,328 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,348 process_obs: ObsID 41187 already processed, skipping
2026-09-02 12:16:29,348 main:
2026-09-02 12:16:29,348 main: ********************************************************************************
2026-09-02 12:16:29,348 main: Processing observation 41186
2026-09-02 12:16:29,348 main: ********************************************************************************
2026-09-02 12:16:29,350 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,352 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,368 process_obs: ObsID 41186 already processed, skipping
2026-09-02 12:16:29,368 main:
2026-09-02 12:16:29,368 main: ********************************************************************************
2026-09-02 12:16:29,368 main: Processing observation 41185
2026-09-02 12:16:29,368 main: ********************************************************************************
2026-09-02 12:16:29,369 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,372 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,388 process_obs: ObsID 41185 already processed, skipping
2026-09-02 12:16:29,388 main:
2026-09-02 12:16:29,389 main: ********************************************************************************
2026-09-02 12:16:29,389 main: Processing observation 41184
2026-09-02 12:16:29,389 main: ********************************************************************************
2026-09-02 12:16:29,390 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,392 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,409 process_obs: ObsID 41184 already processed, skipping
2026-09-02 12:16:29,409 main:
2026-09-02 12:16:29,409 main: ********************************************************************************
2026-09-02 12:16:29,409 main: Processing observation 41183
2026-09-02 12:16:29,409 main: ********************************************************************************
2026-09-02 12:16:29,412 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,414 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,432 process_obs: ObsID 41183 already processed, skipping
2026-09-02 12:16:29,432 main:
2026-09-02 12:16:29,432 main: ********************************************************************************
2026-09-02 12:16:29,432 main: Processing observation 41182
2026-09-02 12:16:29,432 main: ********************************************************************************
2026-09-02 12:16:29,434 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,436 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,442 process_obs: ObsID 41182 already processed, skipping
2026-09-02 12:16:29,442 main:
2026-09-02 12:16:29,442 main: ********************************************************************************
2026-09-02 12:16:29,442 main: Processing observation 41181
2026-09-02 12:16:29,442 main: ********************************************************************************
2026-09-02 12:16:29,444 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,446 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,465 process_obs: ObsID 41181 already processed, skipping
2026-09-02 12:16:29,465 main:
2026-09-02 12:16:29,465 main: ********************************************************************************
2026-09-02 12:16:29,465 main: Processing observation 30196
2026-09-02 12:16:29,465 main: ********************************************************************************
2026-09-02 12:16:29,466 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,468 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,485 process_obs: ObsID 30196 already processed, skipping
2026-09-02 12:16:29,485 main:
2026-09-02 12:16:29,485 main: ********************************************************************************
2026-09-02 12:16:29,485 main: Processing observation 32535
2026-09-02 12:16:29,485 main: ********************************************************************************
2026-09-02 12:16:29,486 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,488 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:29,506 process_obs: ObsID 32535 already processed, skipping
2026-09-02 12:16:29,506 main:
2026-09-02 12:16:29,506 main: ********************************************************************************
2026-09-02 12:16:29,506 main: Processing observation 32529
2026-09-02 12:16:29,506 main: ********************************************************************************
2026-09-02 12:16:29,510 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:29,515 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:34,489 get_centroid_resids_from_file: Reading centroid residuals from /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32529/centroid_resids.pkl
2026-09-02 12:16:34,495 write_centroid_resids: Centroid residuals file exists, skipping
2026-09-02 12:16:34,495 plot_crs_time: Plot file exists, skipping
2026-09-02 12:16:34,495 plot_crs_scatter: Plot file exists, skipping
2026-09-02 12:16:35,326 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32529/n_kalman_delta_roll.png
2026-09-02 12:16:39,609 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32529/index.html
2026-09-02 12:16:39,611 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32529/info.json
2026-09-02 12:16:39,629 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32529/index.html
2026-09-02 12:16:39,632 main:
2026-09-02 12:16:39,632 main: ********************************************************************************
2026-09-02 12:16:39,632 main: Processing observation 30440
2026-09-02 12:16:39,632 main: ********************************************************************************
2026-09-02 12:16:39,634 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:39,636 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:43,903 get_centroid_resids: Computing centroid residuals from telemetry
2026-09-02 12:16:45,599 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30440/centroid_resids.pkl with 5 slots
2026-09-02 12:16:45,651 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30440/centroid_resids_time.png
2026-09-02 12:16:46,049 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30440/centroid_resids_scatter.png
2026-09-02 12:16:47,505 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30440/n_kalman_delta_roll.png
2026-09-02 12:16:47,872 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30440/index.html
2026-09-02 12:16:47,876 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30440/info.json
2026-09-02 12:16:47,887 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/30/30440/index.html
2026-09-02 12:16:47,891 main:
2026-09-02 12:16:47,891 main: ********************************************************************************
2026-09-02 12:16:47,891 main: Processing observation 31881
2026-09-02 12:16:47,891 main: ********************************************************************************
2026-09-02 12:16:47,916 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:47,951 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:51,158 get_centroid_resids: Computing centroid residuals from telemetry
2026-09-02 12:16:51,502 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/31881/centroid_resids.pkl with 5 slots
2026-09-02 12:16:51,564 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/31881/centroid_resids_time.png
2026-09-02 12:16:51,882 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/31881/centroid_resids_scatter.png
2026-09-02 12:16:52,941 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/31881/n_kalman_delta_roll.png
2026-09-02 12:16:53,085 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/31881/index.html
2026-09-02 12:16:53,086 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/31881/info.json
2026-09-02 12:16:53,089 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/31/31881/index.html
2026-09-02 12:16:53,090 main:
2026-09-02 12:16:53,091 main: ********************************************************************************
2026-09-02 12:16:53,091 main: Processing observation 32530
2026-09-02 12:16:53,091 main: ********************************************************************************
2026-09-02 12:16:53,092 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:53,093 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:55,233 get_centroid_resids: Computing centroid residuals from telemetry
2026-09-02 12:16:55,523 write_centroid_resids: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32530/centroid_resids.pkl with 5 slots
2026-09-02 12:16:55,582 plot_crs_time: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32530/centroid_resids_time.png
2026-09-02 12:16:55,838 plot_crs_scatter: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32530/centroid_resids_scatter.png
2026-09-02 12:16:56,644 plot_n_kalman_delta_roll: Writing plot file /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32530/n_kalman_delta_roll.png
2026-09-02 12:16:56,790 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32530/index.html
2026-09-02 12:16:56,792 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32530/info.json
2026-09-02 12:16:56,795 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/32/32530/index.html
2026-09-02 12:16:56,796 main:
2026-09-02 12:16:56,796 main: ********************************************************************************
2026-09-02 12:16:56,796 main: Processing observation 30402
2026-09-02 12:16:56,796 main: ********************************************************************************
2026-09-02 12:16:56,797 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:56,799 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:16:58,949 __init__: ObsID 30402 has insufficient telemetry, skipping
2026-09-02 12:16:58,949 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30402/index.html
2026-09-02 12:16:58,955 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30402/info.json
2026-09-02 12:16:58,956 main:
2026-09-02 12:16:58,956 main: ********************************************************************************
2026-09-02 12:16:58,956 main: Processing observation 30441
2026-09-02 12:16:58,956 main: ********************************************************************************
2026-09-02 12:16:58,957 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:16:58,959 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:17:01,182 __init__: ObsID 30441 has insufficient telemetry, skipping
2026-09-02 12:17:01,183 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30441/index.html
2026-09-02 12:17:01,190 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30441/info.json
2026-09-02 12:17:01,193 main:
2026-09-02 12:17:01,193 main: ********************************************************************************
2026-09-02 12:17:01,193 main: Processing observation 32520
2026-09-02 12:17:01,193 main: ********************************************************************************
2026-09-02 12:17:01,194 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:17:01,196 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:17:04,212 __init__: ObsID 32520 has insufficient telemetry, skipping
2026-09-02 12:17:04,213 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32520/index.html
2026-09-02 12:17:04,219 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32520/info.json
2026-09-02 12:17:04,220 main:
2026-09-02 12:17:04,220 main: ********************************************************************************
2026-09-02 12:17:04,220 main: Processing observation 32517
2026-09-02 12:17:04,220 main: ********************************************************************************
2026-09-02 12:17:04,222 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:17:04,227 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:17:08,422 __init__: ObsID 32517 has no maneuver event in telemetry, skipping
2026-09-02 12:17:08,423 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32517/index.html
2026-09-02 12:17:08,429 info: could not get attribute kalman_start for obsid 32517
2026-09-02 12:17:08,429 info: could not get attribute kalman_stop for obsid 32517
2026-09-02 12:17:08,429 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/32517/info.json
2026-09-02 12:17:08,430 main:
2026-09-02 12:17:08,430 main: ********************************************************************************
2026-09-02 12:17:08,430 main: Processing observation 30438
2026-09-02 12:17:08,430 main: ********************************************************************************
2026-09-02 12:17:08,431 obs_link_from_info: No /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30438/info.json file found, obs=None
2026-09-02 12:17:08,432 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/last.html
2026-09-02 12:17:08,433 write_redirect_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/last.html
2026-09-02 12:17:10,530 __init__: ObsID 30438 has no maneuver event in telemetry, skipping
2026-09-02 12:17:10,530 write_index_html: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30438/index.html
2026-09-02 12:17:10,536 info: could not get attribute kalman_start for obsid 30438
2026-09-02 12:17:10,536 info: could not get attribute kalman_stop for obsid 30438
2026-09-02 12:17:10,536 write_info_json: Writing /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports/2026/AUG3126A/30438/info.json
2026-09-02 12:17:10,537 main:
2026-09-02 13:17:12,123 run_check: Starting centroid dashboard continuity check
2026-09-02 13:17:12,123 collect_observations: Scanning observations in /proj/sot/ska3/aca/www/ASPECT_ICXC/centroid_reports
2026-09-02 13:17:12,124 collect_observations: Processing year 2020
2026-09-02 13:17:14,087 collect_observations: Processing year 2021
2026-09-02 13:17:16,926 collect_observations: Processing year 2022
2026-09-02 13:17:20,119 collect_observations: Processing year 2023
2026-09-02 13:17:23,346 collect_observations: Processing year 2024
2026-09-02 13:17:26,391 collect_observations: Processing year 2025
2026-09-02 13:17:29,514 collect_observations: Processing year 2026
2026-09-02 13:17:31,325 collect_observations: Collected 14519 observations from 4-digit years
2026-09-02 13:17:31,329 sort_observations_by_date: Sorted observations by date
2026-09-02 13:17:31,329 check_continuity: Starting continuity check from most recent: ObsID 30438 (AUG3126A) at 2026:245:12:52:08.730
2026-09-02 13:17:31,403 check_continuity: Reached chain start at ObsID 47337 (MAR3020A) at 2020:092:05:29:19.712
2026-09-02 13:17:31,403 check_continuity: Chain complete: visited 14519 observations out of 14519 total
2026-09-02 13:17:31,403 run_check: ✓ Observation chain continuity check PASSED
2026-09-02 13:17:31,403 run_check: Summary: 14519 observations from 4-digit years, all properly linked