The technical problem to be solved here involves the inability of major AI players like ChatGPT and Grok to correctly answer this simple prompt:
List the first 10 instruments in NASA’s SPLASH (SPLASH) database when sorted alphabetically by name.
Regards,
Luis De Pombo
@HardwareAWG
Hi, I would like to contribute to this topic.
Hi Simone! It would be an honor to hear your solution. Please share.
I’m Naima, a recently graduated AI Engineer (Python, LLM/RAG), and I recently joined the AI/ML AWG. I read your post on the AI accessibility of SPLASH and I’d like to contribute.
My idea would be to give an LLM structured access to the SPLASH data (SQL or API queries, or an MCP server) instead of letting it answer from memory, so that questions like “list the first 10 instruments sorted alphabetically” return exact results.
Is the project still open? And is there an API or an export of the SPLASH database I can use to start a small prototype?
Thank you,
Naima
Hello @luis.depombopuerta and @NaimaREJEB,
I’m Kyunghan Bae, an independent researcher and software developer in South Korea working on public-data validation and reproducible computational analysis.
I’d like to contribute validation code alongside Naima’s proposed data-access approach. For the “first 10 instruments sorted by name” question, I propose read-only Python checks for ordering, duplicate entries, and links to source records, using instrument identifiers where available. Recording the source snapshot and sorting rules would make the checks reproducible.
Once an authoritative SPLASH export and sorting convention are agreed, I could prepare a small validation script and error report. Would this be a useful starting contribution, and are there existing tests I should build on?
Following up with runnable checks for the first-ten query. The attached Markdown file contains the complete standard-library Python code, 16 regression tests, run instructions, and the snapshot timestamp and SHA-256. I independently reran all 16 tests successfully.
A fresh read of the public API on 30 October (UTC) returned 148 rows: one field-label row and 147 instruments. The checks distinguish that label row using four matching fields, rather than dropping every record with ID 0. All 147 instrument IDs were present and unique in this snapshot.
One useful difference showed up when comparing the result with the catalogue: clicking Name ascending currently puts ADvanced Space Experiment Processor-2 and ADvanced Space Experiment Processor-4 ahead of Advanced Biological Research System. A case-insensitive comparison puts the biological research system first. The report supplies both first-ten lists with the original IDs and names, so the ordering convention can be agreed before scoring an AI answer.
Would you prefer the current visible catalogue order or a case-insensitive convention as the reference for this first query?
Source and credit: NASA OSDR SPLASH, supported by NASA Biological and Physical Sciences.
splash-validation-contribution.md (14.9 KB)
Hi @luis.depombopuerta and @NaimaREJEB,
I rechecked the attached validator against the public API today. The snapshot is byte-identical to the one from 30 October (SHA-256 b069d0dba999d137183c75f38b9f070423115e6088c358d697a35a3346e86c35), and the 16 published tests still pass.
I also added a small read-only tool adapter around that validator. Its response carries the source URL, retrieval time, snapshot hash and instrument count alongside the ten original IDs/names. The caller must explicitly choose raw_name or casefold_name, preserving the sorting distinction. Eight additional tests pass, and a live fetch returned all ten requested records.
The attached Markdown contains the adapter, its tests and run instructions. It uses splash_audit.py from the earlier bundle above.
Naima, would this response shape fit the structured API/MCP approach you outlined? It would be helpful to agree on the data contract and ordering before adding a model layer.
To make the adapter easier to try, I connected it to a working stdio MCP server. A real MCP client launched the server as a separate process, discovered the tool and its input/output schemas, and retrieved both first-ten lists from NASA’s public endpoint. The live responses contain 147 instruments and the same snapshot hash reported above; the first IDs are 66 for raw-name ordering and 64 for casefold ordering.
The complete attachment includes the validator, adapter, MCP server, pinned dependencies, tests, a subprocess check and the recorded live responses. All 35 SPLASH tests pass, including 11 MCP/fetch checks for required ordering, malformed data, timeouts, retry recovery and bounded reads. I also extracted this exact Markdown into a clean folder and reran the tests and live subprocess check successfully.
Verified on Windows with Python 3.13.12 and MCP SDK 2.3.0. The server exposes one read-only tool, uses the fixed public endpoint, and requires no credentials or model calls. Both ordering conventions remain explicit so we can agree which one should be used for the original question.