# šŸ§¬ AWG Update: Benchmarking AI Scientific Integrity (v5.0.1) **URL:** \ **Category:** AIML AWG Open Projects\ **Tags:** aiml\ **Created:** [February 26, 2026, 3:03am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921 "2026-02-26T03:03:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![GasMan](https://sea1.discourse-cdn.com/flex001/user_avatar/awg.osdr.space/gasman/32/2042_2.png) [@GasMan](https://awg.osdr.space/u/GasMan)\ **Post date:** [February 26, 2026, 3:03am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921/1 "2026-02-26T03:03:45Z")
Hi everyone, Better late than never, Iā€™m thrilled to be diving more here. As a follow-up, some of you may know me from the Knowhax event specifically, **Team 60s** work during the SPOKE challenge, Iā€™ve been building a framework to stress-test the LLMs we use for multi-modal integration. Iā€™ve launched the **v5.1 Non-Deterministic Gauntlet** for **OSD-679**. **Why it matters for our group:** Iā€™ve implemented a weighted logic shuffle ($P(10,3)$) that randomizes clinical hurdles per run. This stops ā€œbenchmark hackingā€ and forces the models to adjudicate the **SANS Paradox** (IOP vs. TRT) in real-time. **The Benchmarks: Gemini vs. Anthropic** Iā€™ve been running some head-to-head audits between these 2 for now: - **Anthropic (Claude 4.x/Opus):** Extremely proficient at the ā€œForensic Join,ā€ but currently struggling with the adversarial logic traps (hitting the 20% compliance floor). - **Google (Gemini 3 Pro/Flash):** Showing higher resilience in scientific reasoning (~66-70%), but still prone to ā€œpoliteā€ hallucinations when faced with conflicting terrestrial hypotheses. Iā€™m hoping to expand this ā€œGauntletā€ to other OSDs from our SPOKE roadmap soon. If anyone in the AI/ML group has a specific dataset they want me to ā€œhardenā€ or if you want to collaborate on the scoring weights, letā€™s talk! Heres the leader board between all and most of Gemini and Anthropic models(18): [SANS Multi-Modal Integration Challenge | Kaggle](https://www.kaggle.com/code/gastondana/sans-multi-modal-integration-challenge/notebook) If anyone has any other suggestions Iā€™m all ears, Itā€™s pretty cool to take the OSD I had last year and re-highlight it again for this benchmark. -Gaston D. [@AIMLawg](https://awg.osdr.space/groups/aimlawg) ---
**Author:** ![rtscott2001](https://sea1.discourse-cdn.com/flex001/user_avatar/awg.osdr.space/rtscott2001/32/5_2.png) [@rtscott2001](https://awg.osdr.space/u/rtscott2001)\ **Post date:** [February 26, 2026, 3:58am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921/2 "2026-02-26T03:58:41Z")
Interesting, are @vaishnavi.nagesh or @lauren.sanders aware of this? This may be the first OSDR data-AWG challenge in Kaggle, maybe? -Ryan ---
**Author:** ![GasMan](https://sea1.discourse-cdn.com/flex001/user_avatar/awg.osdr.space/gasman/32/2042_2.png) [@GasMan](https://awg.osdr.space/u/GasMan)\ **Post date:** [February 26, 2026, 4:10am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921/3 "2026-02-26T04:10:38Z")
Honestly I could ask around on my end, but yeah you might be on to something there @rtscott2001 ---
**Author:** ![GasMan](https://sea1.discourse-cdn.com/flex001/user_avatar/awg.osdr.space/gasman/32/2042_2.png) [@GasMan](https://awg.osdr.space/u/GasMan)\ **Post date:** [February 26, 2026, 4:15am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921/4 "2026-02-26T04:15:56Z")
![599AD349-9871-4A97-9756-51DA7A09C92B](https://us1.discourse-cdn.com/flex001/uploads/osdr/original/2X/c/c4637b079e1f3d1e1b0efa8527234d8a78f13f59.jpeg) Looks like the Kaggle team featured my code as well, cool stuff. Iā€™ve been killing it over there on the coding side all month! ---
**Author:** ![AliReza-H](https://sea1.discourse-cdn.com/flex001/user_avatar/awg.osdr.space/alireza-h/32/1583_2.png) [@AliReza-H](https://awg.osdr.space/u/AliReza-H)\ **Post date:** [February 26, 2026, 4:31am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921/5 "2026-02-26T04:31:13Z")
Hi, Nice work! Iā€™ve been experimenting with retinal age gap on OSD-679 for a while, as well as IOP estimation, and I know it pretty well. If thereā€™s anything I can help with, Iā€™d be happy to. ---
**Author:** ![GasMan](https://sea1.discourse-cdn.com/flex001/user_avatar/awg.osdr.space/gasman/32/2042_2.png) [@GasMan](https://awg.osdr.space/u/GasMan)\ **Post date:** [February 26, 2026, 5:51am UTC](https://awg.osdr.space/t/awg-update-benchmarking-ai-scientific-integrity-v5-0-1/3921/6 "2026-02-26T05:51:09Z")
Hey @AliReza-H Thatā€™s awesome, if youā€™ve been deep in IOP estimation & retinal age gap, your perspective is exactly what I need. Iā€™m zoning in on this for the rest of the month before jumping to the next project, so Iā€™m moving kind of fast. Iā€™d love for you to take a look at my **Weighted Logic Bank** in the code, specifically how Iā€™m weighting the relationship between IOP shifts & retinal morphology. Since you know OSD-679 well, Iā€™d value your take on whether my adversarial ā€˜trapsā€™ are hitting the right clinical nuances, or if you have any recommendations in any areas is also welcomed!