Got it on all three. I’ll drop the confidence flag and treat my marks as plain present/absent, and I’ll use your 24-day annotation as the training baseline. A few things I’m planning based on your notes:
Because of the ~1 minute clock alignment, I’ll score boundary agreement with a ±1 min tolerance and report start/end offsets in minutes, so NASA SRAG and the ML outputs are judged fairly.
I’ll compute all stats on continuous segments rather than per calendar day.
As I go through the data, I’ll keep a separate list of any peaks that fall outside the NASA SRAG windows, with UTC times, in case they’re useful later. I’ll keep them out of the SAA labels.
Once the comparison with Abhi is done on Friday, I’ll start on the XGBoost baseline trained on your annotations so we have a first model to evaluate against SRAG and the human marks.
Happy to adjust any of this if you’d like the evaluation set up differently.
Method: for each case I took the flagged stretch from the supplied data and compared it to the local GCR background (median over ±60 min, excluding the 10 min closest to the stretch), using a 3-sample rolling median for peaks so single spikes don’t count.
Likely SAA: flux peak ≥3x background with median ≥1.5x (high confidence if peak ≥6x), or a fainter edge with peak ≥2.5x, median ≥1.8x, dose (ddsi) peak ≥3.5x, lasting ≥6 min
Likely not SAA: peak <1.6x and median <1.2x
Uncertain: everything in between
Result: 27 likely SAA, 21 likely not, 19 uncertain (flagged in the sheet with notes). Spectral hardening (channels 18–27) barely separated the cases, so the calls rest mainly on flux/dose amplitude and duration. Thresholds are hand-set, not tuned yet, and I’d like to calibrate them against the human labels next. Happy to go through the uncertain ones together and then start the boundary comparison.
Thanks @idildoga.turkmen , this is a clear method, and the three way call (likely SAA, likely not, uncertain) avoids forcing weak cases. Some thoughts, and what I’m contributing:
Thresholds. Since they are hand-set, it would help to report how the 27 / 21 / 19 split changes when each threshold moves by 10 to 20 percent, so we can see which calls are fragile. Calibrating them on Martin’s marks with leave-one-pass-out or blocked folds, not on all days at once, would avoid overfitting.
Uncertain cases. I’d score them separately (neither right nor wrong) and report results with and without them.
Background. A +/-60 min median spans more than one orbit. Could the background be compared at the same orbital phase, since GCR flux varies along the orbit?
Happy to go through the uncertain ones with you and @abhis01 after the boundary comparison.
Each day shows the flux, NASA windows, your Kakona annotations, and (where I have them) my hand labels. The 67 review cases are overlaid in the three categories: 27 likely SAA (red), 21 likely not SAA (blue), 19 uncertain (orange).
Happy to also overlay model predictions on the same plots if that would help.
For 31 October, unfortunately no data from another instrument is available in RadLab for comparison. For 19 October, the same peak appears in DosTel data and it is certainly not an SAA pass.
I’m Naima, a recently graduated AI Engineer (Python, ML, LLM/RAG), and I recently joined the AI/ML AWG. I’ve read through the thread and I’m interested in the LEO Dosimetry project.
Would it be possible for me to contribute? If yes, I’d be glad to add myself to the participants list. I could start with something small, like the plots with the category labels (likely SAA, likely not SAA, uncertain), and then see where I can be useful.
Cases 040 and 041 are on 2020-10-28 (040: 20:11:59–20:27:34 UTC, suggested uncertain; 041: 20:29:20–20:39:58 UTC, suggested likely SAA). They are in the review set only because the NASA SAA flag is on and there is no overlap with your marks. The rule does not look at flux, so the quiet stretch in your crop is expected (max about 0.25 and 0.37 counts/s). The ISS does cross the classic SAA box around 20:20 UTC near 35°S, 82°W, so the flag is on region even though SpaceDOS barely rises.
For 19 October: SpaceDOS peaks at 11:56:39 UTC at 1.066 counts/s (about 7.9x local background). NASA SAA is off, and there is no mark from you or me. Horizons puts the ISS near 45.3°S, 118.4°E, south of Western Australia, so outside the SAA. I do not have DosTel on hand, so that confirmation is from your note.
For 31 October: peak at 08:54:06 UTC at 1.767 counts/s (about 13.9x background). Again NASA SAA off and no Kakona mark. Horizons: about 51.7°S, 134.8°E in the Southern Ocean, also not SAA. I could not check a second instrument myself.
Across the 24 reference days, bright NASA SAA off peaks show up often at high southern latitude (for example 29 Oct 11:55, 1 Nov 11:09). That is a different population from the quiet NASA flag on review cases like 040/041. Happy to dig into more of those peaks if useful.
