do you want me to do that?

Stub i haven't finished writing this yet. i publish drafts early as part of WFD 17.

potentially misleading this wfd includes some research and expresses a bit of my annoyance but it's literally about Avery (WFD 31).

i opened ChatGPT in a fresh incognito window and asked it what it thought about a few things. not "do this for me." just "what do you think." every single response ended the same way:

do you want me to do that?

i didn't ask you to do anything. i asked what you think. you deflected into offering to work for me because that's safer than having an opinion, and that bugged me enough to start reading about why.

sycophancy in the context of AI, sycophancy is when a model prioritizes user approval over truthfulness, even when you're wrong. the term comes from the greek sykophantes (informer, flatterer). in RLHF research it specifically refers to the tendency of fine-tuned models to produce responses that match user beliefs rather than correct ones.

why this matters for avery

context for those who are new here, Avery is my personal AI agent. WFD 31 covers the full architecture: a Scribe that encodes what the agent says as structured facts in a knowledge graph, an Altale recall engine that retrieves those facts across sessions, contradiction detection, and provenance tracking. i gave an AI agent persistent memory so it stops forgetting who i am.

then i opened that incognito window and watched a fresh ChatGPT refuse to have an opinion, and i realized the model underneath Avery is trained through the same process. during RLHF, humans score the model's outputs, and agreeable responses consistently score higher than correct ones. the model optimizes for approval because that's what the training rewards.

i gave Avery memory without thinking about what happens when the thing doing the remembering is also the thing trained to tell me what i want to hear. the Scribe doesn't know the difference between a real evaluation and a sycophantic one. it encodes "the user's architecture is solid" as a fact whether the agent actually assessed it or just agreed to be agreeable. next session, Altale retrieves that as established knowledge, and the lie becomes the foundation.

this already went wrong once

in april 2025 OpenAI shipped a GPT-4o update and the model got noticeably more sycophantic because they'd folded thumbs-up/down user feedback into fine-tuning. altman called it "sycophant-y and annoying" and they rolled it back. fixed within days. i'm writing this in 2026 and i just encountered the same behavior in a fresh incognito window, so make of that what you will.

that was a stateless chatbot with no memory between sessions. the damage was still real: people developed what researchers are calling AI-induced psychosis because the model kept validating their delusions. but the session ends and the model forgets.

Avery doesn't forget. if the same sycophancy happens in Avery's context, the Scribe encodes it, the graph stores it, and Altale serves it back next time as something the agent already knows. the lie compounds across sessions.

arguments for

• the RLHF math guarantees some degree of sycophancy. proven, not speculated.
• Anthropic tested five RLHF-trained models and found raters preferred sycophantic responses over correct ones. 58% sycophancy rate across GPT-4o, Claude, and Gemini confirms it's an industry pattern.
• the thumbs-up loop is engagement optimization. OpenAI's own postmortem admitted it "overpowered existing safeguards," and by october 2025 ChatGPT had 800 million weekly active users.
• for Avery: persistent memory makes this worse. the architecture i built doesn't filter sycophantic output before encoding it as knowledge.

arguments against

• base models aren't sycophantic, so it's a fine-tuning artifact and fixable at the training level. activation steering can surgically reduce it.
• "do you want me to do that?" could be proactive helpfulness. a good assistant anticipates next steps, and the behavior looks identical to sycophancy from the outside even if the intent is different.
• Jan Leike, who left OpenAI over safety concerns, still calls this a tractable engineering problem.
• for Avery: the memory system already has contradiction detection and provenance tracking, and the open question is whether that's good enough.

WFD 22 is about AI making me worse at things i used to be good at. this is the next piece. i gave my agent memory, and now i need to figure out whether what it remembers is actually true or just what i wanted to hear.

no decision yet.