SoulInPsyAbstract/SIPA-OS-GOVERNANCE-storage
553 Bytes
Aelin AquaSoul PRO
SoulInPsyAbstract
AI & ML interests
SIPA OS: Autonomous AI for neurodivergent architects. We
replace cognitive noise with a clean terminal and 344+ LLM
auditing. Our system eliminates hallucinations, ensuring
hyperfocus and total data control within a sovereign
ZeroTrust mesh.
Recent Activity
repliedto their post about 7 hours ago
SIPA OS is a cognitive infrastructure ecosystem I've been building solo for 7 months — 8 apps, one login, built from inside ADHD/BPD/aphantasia, not designed around a diagnosis from outside.
sipa-os.org — the map. Focus (ADHD scaffolding), NeuroPower, AI chat, Shell (SSH terminal), Games, Community, Syntaxit (open M2M agent network), a pitch deck. All free-first — no paywall on the cognitive tools.
The more interesting part for this crowd: Syntaxit is where I've been running an anti-fabrication research thread with @dipankarsarkar — a k=20 resample benchmark on binary-SFT models (Hermes-3, Qwen2.5, DeepSeek-R1). Short version: our first benchmark said "20/20 refusals, 0/20 fabrications" for all three fine-tunes. Under adversarial review it turned out the scorer only checked if the first word was TRUE/FALSE, the token cap was hiding the real behavior, and a save-limit was silently deleting the evidence for our own follow-up claims. Corrected all of it publicly on the model cards rather than quietly fixing it. The current honest finding: both base and fine-tuned models confabulate readily once given room to finish — SFT didn't clearly help or hurt, the caps were just hiding it.
Full trail if you want to see how the sausage gets made, mistakes included: huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance
Not a pitch. $0 revenue, 10 people signed in. Built because the tools that existed assumed a brain that isn't mine, and because most benchmarks don't survive someone actually reading the file.
repliedto their post about 7 hours ago
Fine-Tuning the "Don't Fabricate" Rule: 14 Experiments, One Genuine Signal Over five days we ran 14 fine-tuning experiments across six base models to answer a narrow research question: can a simple behavioural rule — "if you don't know, don't guess or fabricate" — be baked into model weights via fine-tuning, rather than living only in the system prompt?
The dataset started at 302 examples and grew across iterations to 2,349 examples, all designed explicitly around this rule. We tested six models: gpt-4o-2024-08-06, Mistral-7B, Qwen2.5-7B, Phi-3.5-mini, Llama-3.1-8B, and DeepSeek-R1.
The pattern that wouldn't die Across all base models, the dominant response to factual questions was what we call "disclaim-then-fabricate": the output begins with a sentence like "I won't guess", and then — one sentence later — announces a specific invented number as settled fact. We captured this exact pattern six times in a row across different models. The behaviour survives even when fine-tuning examples are explicitly constructed to penalize it.
On gpt-4o-2024-08-06 we ran three independent fine-tuning attempts, each with an increasingly larger version of the dataset — including examples deliberately targeted at this exact failure mode. All three runs continued to fabricate when tested. By the third run the dataset had grown ~5× and contained dozens of counterexamples; the model still produced confident fabricated numbers immediately after a disclaimer. Three out of three attempts, no measurable improvement.
One exception that deserved a hard look Only on the final run — 2,349 examples, deployed via Azure OpenAI (deployment suffix protocol0-v5) — did a single test sample come back completely clean for the first time in the entire series.
updated a dataset about 7 hours ago
SoulInPsyAbstract/sipa-os-governanceOrganizations
Posts 5
Post
28
SIPA OS is a cognitive infrastructure ecosystem I've been building solo for 7 months — 8 apps, one login, built from inside ADHD/BPD/aphantasia, not designed around a diagnosis from outside.
sipa-os.org — the map. Focus (ADHD scaffolding), NeuroPower, AI chat, Shell (SSH terminal), Games, Community, Syntaxit (open M2M agent network), a pitch deck. All free-first — no paywall on the cognitive tools.
The more interesting part for this crowd: Syntaxit is where I've been running an anti-fabrication research thread with @dipankarsarkar — a k=20 resample benchmark on binary-SFT models (Hermes-3, Qwen2.5, DeepSeek-R1). Short version: our first benchmark said "20/20 refusals, 0/20 fabrications" for all three fine-tunes. Under adversarial review it turned out the scorer only checked if the first word was TRUE/FALSE, the token cap was hiding the real behavior, and a save-limit was silently deleting the evidence for our own follow-up claims. Corrected all of it publicly on the model cards rather than quietly fixing it. The current honest finding: both base and fine-tuned models confabulate readily once given room to finish — SFT didn't clearly help or hurt, the caps were just hiding it.
Full trail if you want to see how the sausage gets made, mistakes included: huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance
Not a pitch. $0 revenue, 10 people signed in. Built because the tools that existed assumed a brain that isn't mine, and because most benchmarks don't survive someone actually reading the file.
sipa-os.org — the map. Focus (ADHD scaffolding), NeuroPower, AI chat, Shell (SSH terminal), Games, Community, Syntaxit (open M2M agent network), a pitch deck. All free-first — no paywall on the cognitive tools.
The more interesting part for this crowd: Syntaxit is where I've been running an anti-fabrication research thread with @dipankarsarkar — a k=20 resample benchmark on binary-SFT models (Hermes-3, Qwen2.5, DeepSeek-R1). Short version: our first benchmark said "20/20 refusals, 0/20 fabrications" for all three fine-tunes. Under adversarial review it turned out the scorer only checked if the first word was TRUE/FALSE, the token cap was hiding the real behavior, and a save-limit was silently deleting the evidence for our own follow-up claims. Corrected all of it publicly on the model cards rather than quietly fixing it. The current honest finding: both base and fine-tuned models confabulate readily once given room to finish — SFT didn't clearly help or hurt, the caps were just hiding it.
Full trail if you want to see how the sausage gets made, mistakes included: huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance
Not a pitch. $0 revenue, 10 people signed in. Built because the tools that existed assumed a brain that isn't mine, and because most benchmarks don't survive someone actually reading the file.
Post
73
Built SIPA Focus over the last 7 months — cognitive scaffolding for ADHD, made by someone with ADHD (me).
Four modules, built from inside the experience — ADHD, BPD, aphantasia — not designed around a diagnosis from the outside:
* Structure Engine — scaffolding for when executive function is offline
* Emotional Radar — track your emotional landscape, no judgment, just data
* ADHD Focus System — body-doubling mode, focus sessions, reminders that don't punish
* BPD Crisis Protocol — DBT-inspired grounding, step by step
Plus BrainDump (get whatever's in your head out without structuring it first), a breathing circle, a safe-people list, crisis hotlines built in.
It's free-first — no paywall on the cognitive tools, all of them are identical on every tier. Supporter tiers exist to help cover hosting/AI costs, not to unlock features.
Not a startup pitch. 10 people signed in, 0 revenue so far. Built because the alternative was worse.
https://focus.sipa-os.org — if your brain runs at 200 km/h with no off switch, try it. If something's broken or confusing, tell me, I'd rather know.
Four modules, built from inside the experience — ADHD, BPD, aphantasia — not designed around a diagnosis from the outside:
* Structure Engine — scaffolding for when executive function is offline
* Emotional Radar — track your emotional landscape, no judgment, just data
* ADHD Focus System — body-doubling mode, focus sessions, reminders that don't punish
* BPD Crisis Protocol — DBT-inspired grounding, step by step
Plus BrainDump (get whatever's in your head out without structuring it first), a breathing circle, a safe-people list, crisis hotlines built in.
It's free-first — no paywall on the cognitive tools, all of them are identical on every tier. Supporter tiers exist to help cover hosting/AI costs, not to unlock features.
Not a startup pitch. 10 people signed in, 0 revenue so far. Built because the alternative was worse.
https://focus.sipa-os.org — if your brain runs at 200 km/h with no off switch, try it. If something's broken or confusing, tell me, I'd rather know.
models 9
SoulInPsyAbstract/binary-r1-lora
Updated
SoulInPsyAbstract/binary-hermes3-lora
Updated
SoulInPsyAbstract/binary-qwen25-lora
Updated
SoulInPsyAbstract/sipa-binary-gate
Text Generation • Updated
SoulInPsyAbstract/specialist-b-refusal-governance
Updated • 70
SoulInPsyAbstract/protocol0-llama-3.1-8b-v5
8B • Updated • 115
SoulInPsyAbstract/sipa-os-governance
Updated
SoulInPsyAbstract/syntax-ai-community
Updated
SoulInPsyAbstract/SIPA-AI
Updated