Claude 4 vs. Claude 3: What Actually Changed

Every model rollout is a live experiment run against every user's existing context. Anthropic's 3 → 4 transition was unusually clean: the same Projects, the same pinned files, the same system prompts, but a different model doing the reading. That made it a good natural experiment for answering the question "when the model swaps, what does the assistant lose about me?"

Twenty prompts, chosen to probe five distinct memory behaviors: explicit facts, system-prompt directives, inferred preferences, cross-conversation callbacks, and voice matching. Every prompt was run twice against the same Project: once during the final week of Claude 3.7's default window, once during the first week Claude 4 was the Project default. Same account. Same Project. No intervening turns between the two runs.

"Agreement" between the paired runs was scored by a blind human review across three axes: factual match, emphasis match, and voice match. Numbers below are aggregate agreement rates, bucketed at 50+ consenting members.

Pinned facts and system prompts

Anything you'd written down explicitly — a fact in the system prompt, a pinned file with your résumé, a rule like "always answer in bullet points" — survived the transition essentially intact. This is the good news, and it's the reason Projects remain the best-behaved memory system in the major-provider landscape.

The Project structure itself

Project names, file order, and conversation counts are model-independent. Nothing here changed. If you organized your Projects thoughtfully before the transition, that organization is worth exactly what it was worth on August 14th.

Inferred preferences

Preferences the older model had picked up implicitly — "I notice you tend to want the shorter version first" — are the classic drift zone. Claude 4 re-infers, and its inferences don't always match. Agreement here dropped twenty-six points.

Cross-conversation callbacks

"Remember what we discussed about the Q3 pitch?"-style references land less reliably. Claude 4's retrieval of prior Project conversations selects a different subset. Not worse; different.

Voice and tone matching

The single largest drop. Voice is emergent from the model's weights, not the Project's data. When the weights change, the voice changes. If you're a heavy Claude user who has spent months teaching a Project a particular tone, this is the one that will feel most jarring.

1. Move your inferred rules into the system prompt. Anything you rely on that isn't written down is a candidate to be forgotten at the next model rollover. Write it down.
2. Keep a "voice sample" pin. Two or three paragraphs of the tone you want, pinned to the Project. Cheap insurance against drift.
3. Take Anthropic's export. Settings → Privacy → Export data. Store it. This is the escape hatch for the day the transition breaks something you can't reproduce.
4. Consider a sidecar. If your voice matters, keeping a portable memory file outside any one provider is how you make sure a model-swap somewhere else doesn't reset you.