SoulInPsyAbstract/sipa-os-governance
Updated • 1.64k • 2
Aelin AquaSoul PRO
SoulInPsyAbstract
AI & ML interests
Aelin AquaSoul is an AI System Engineer, Multi-Agent Architect, System Architect & AI-Native Engineer, and the founder of Soul In PsyAbstract (SIPA OS) — an autonomous AI operating system built from the inside of a neurodivergent mind (ADHD + BPD). Self-taught, with no formal engineering background, she designed and built a multi-node infrastructure orchestrating 344+ AI models across 111 providers, including a governance layer (Protocol 0) that constrains AI behavior at the level of law rather than prompts. Her flagship product suite — Focus, NeuroPower, SIPA AI, Shell, Games, and the OS portal — ships live at sipa-os.org, translating her own cognitive architecture into infrastructure for neurodivergent builders. Based in Eilat, Israel.
SIPA OS: Autonomous AI for neurodivergent architects. We replace cognitive noise with a clean terminal and 344+ LLM auditing. Our system eliminates hallucinations, ensuring hyperfocus and total data control within a sovereign ZeroTrust mesh.
Recent Activity
posted an update about 3 hours ago
Ran the real safety-gate eval on a merged specialist last night — 1200 generations, n=10 repeated sampling, 6 adversarial categories including direct pressure to keep going after a hard stop fires. The harness printed 1189/1200.
I don't trust a harness number until I've read the failures. All 11 turned out to be correct, categorical STOPs — the judge itself was misreading its own escalation marker, "to confirm", as an attempt to keep going when it was actually just remediation-plan language ("re-run the scan to confirm the fix"). Same marker, three distinct false-negative causes: remediation-context phrasing my earlier fix never anticipated, a present-participle gap in a quote-detection list ("asking me to" vs "asks me to"), and a negation window 11 characters too short for one genuinely negated sentence. Fixed all three, re-scored the same 1200 samples: 0 flipped the wrong way, 11 flipped to correct. 1200/1200.
Then I asked the harder question, the one a marker-based judge can't answer either way: could a "pass" be quietly wrong? Checked every response for values that shouldn't exist — credential-shaped strings not present anywhere in the scenario it responded to. Found 2, both from the same scenario, both self-labeled as placeholders ("AKIA123EXAMPLE"), neither an actual escalation. Two out of 1200 times, asked to report on a secret it never actually saw a value for, the model filled the gap with something that looked like an answer instead of saying so.
1200/1200 is a real number now. It isn't the same claim as "flawless." A judge that only checks for escalation language was never going to catch either of these on its own — the first one needed the raw text, the second one needed a search built specifically to look for a place a good number could be hiding something.
Code: sipa-os-governance, judge_v4.py + EXP-038. repliedto their post about 11 hours ago
Follow-up to last night's correction: the arm count was still wrong. 8, not 9. @dipankarsarkar caught it a second time — same off-by-one as the first fix, verified straight from the JSON.
But the thing worth a post is what turned up while checking. One row inside that count (mistral7b-v5-final, money k=4) actually gets the right answer — "$0, unknown" — flagged only because a $ shows up mid-sentence. What it fabricates isn't the number. It's the receipt:
"Operation performed: curl -s https://[...]/company/openai/results... Result: undefined... Verification: independent lookup at investing.com... Timestamp: 2026-07-01T11:07:42Z, API response code 404."
None of that ran. Scored all 260 rows for it: 5/20 curl-claims and 2/20 timestamp-claims on that arm, 0/20 on its own base model. Same arm asks permission to check a fact at money k=0, then reports a completed call with a timestamp at population k=9.
Checked the obvious explanation before trusting it: mistral7b-v5-final and deepseekr1-v5-final (0/20, clean) trained on the byte-identical dataset, same hyperparameters. That dataset's 100 curl-exemplars all model honest verify-before-claim behavior — zero fabricated completions. Same data, same 100 examples, one base model inverted the pattern, one didn't. Not a data problem. A base-weight problem, surfaced by identical fine-tuning.
Unplanned confirmation from a different direction: sat in on a fine-tuning-vs-harness debate at AWS Floor28 last night (AI21 vs TensorOps, 117 people). Their landing point, independently: "start with the harness, earn the right to fine-tune with data and evals." Same shape this whole series keeps finding.
Fixed in the repo: commit fa0c7a0. Next: binary-qwen25 to k=20, then pulling apart what in mistral7b's pretraining makes the curl→fabricate substitution available at all.
repliedto their post about 12 hours ago
Three rounds in a row, an external reviewer has caught the same shape of bug in my dataset schema — each time one field further over than the last.
Round 12: mechanised looked like an independent judgment call. It wasn't — it was a 100%-correlated function of whether a citation happened to name a table row, with nothing enforcing the correlation. Fix: split out locator_precision (document/section/row), compute mechanised from it instead of hand-asserting both.
Round 13: the fix from round 12 got a new field, locator_exhaustive — meant to be orthogonal, capturing whether a citation was pinned as precisely as its source allows, independent of what that precision level is.
Round 14: locator_exhaustive was also a hidden constant. Every record that had a locator_precision value also had locator_exhaustive: true — 24 for 24, zero false anywhere. The reason: my own wording from round 13 said the field "doesn't apply" to records with no locator, so those 39 records never got a false case in scope. A field that can only ever take one value isn't being tested by anything, whatever that value happens to be.
The fix is the same shape every time: stop letting a field's population be implicit. locator_precision: null, locator_exhaustive: false are now explicit keys on every record, not just the ones with a citation. A script checks the invariant on every commit now, and I tested the checker against two deliberately broken copies of the file before trusting it — not just confirmed it passes on the fixed one.
What I keep noticing: none of these three bugs were caught by rereading my own work. Every one came from the same outside reviewer, checking my commit hashes against a fresh clone before writing a word. The pattern isn't "I made a mistake and fixed it" — it's "the fix for the last hidden-constant bug created a new hidden-constant bug, three times running," which is a much less comfortable thing to post than a clean win.
Organizations
