Part IV: From LLM Wiki to the AI Scientist

Chapter 11: My Activity as AI's External Memory — Bidirectional Flow with Obsidian × LLM

Written: 2026-04-28 Last updated: 2026-06-11

11.1 What Does Research Mean in the AI Age?

Post #7 "Brain Augmentation" [12] opens with: "In the age of AI, what matters in research is not 'knowing more myself' but 'building an environment where an AI scientist can sustain self-improving knowledge creation.'"

That's the starting point for this chapter. Building an external memory system isn't a productivity tool. It's building an environment for doing research with AI.

What that environment actually looks like is what this chapter covers.

11.2 Vault Structure — In Practice

Turn Chapter 10's three layers into a real folder structure:


vault/
├── raw/                      # L1: original sources
│   ├── papers/               # paper PDFs, arXiv links
│   ├── daily-notes/          # daily notes
│   ├── meeting-notes/        # meeting records
│   └── ideas/                # idea clippings
│
├── wiki/                     # L2: LLM-managed knowledge base
│   ├── people/               # researchers, colleagues
│   ├── papers/               # paper summaries + annotations
│   ├── projects/             # project status pages
│   ├── concepts/             # technical concepts
│   └── moc/                  # Maps of Contents
│
├── decisions/                # human-owned decisions, abandoned paths, commitments
├── logs/                     # agent operation logs and diff summaries
│
├── CLAUDE.md                 # L3: vault structure (for Claude)
└── AGENTS.md                 # L3: vault structure (for Codex)

Recent LLM Wiki guides often add two practical directories: inbox/ and logs/ [6]. inbox/ holds unprocessed URLs, PDFs, and transcripts; logs/ records what each ingest/query/lint operation changed. This prevents the instruction from becoming "read anything and write anywhere." Work starts in inbox/, outputs land in wiki/, and rationale is recorded in logs/.

S8 adds one more directory: decisions/ [12]. This is where the human records what was chosen, what was rejected, and why. LLMs can summarize alternatives, but the decision itself should not disappear into generated wiki text. In practice, decisions/ is the research equivalent of HANDOFF.md and TASKS.md: it preserves state across agents and across weeks.

CLAUDE.md example (L3):


# My Research Vault

This vault contains my research knowledge organized in three layers.

## Structure
- raw/papers/: Paper PDFs and arXiv links. READ ONLY.
- raw/daily-notes/: Daily notes. READ ONLY.
- wiki/papers/: Paper summaries with my annotations. YOU CAN EDIT.
- wiki/concepts/: Technical concept pages. YOU CAN EDIT.
- wiki/moc/: Map of Contents pages. YOU CAN EDIT.

## Operations
When I say "ingest [paper]": read raw/papers/[paper], create/update wiki/papers/[paper].md
When I say "what do I know about [X]": search wiki/ for relevant pages and summarize.
When I ask a research question: search wiki/ AND raw/, synthesize an answer.
Figure 11.1: Real folder layout of a research vault — raw (originals), wiki (LLM-managed knowledge base), CLAUDE.md/AGENTS.md (schema). The three layers mapped to actual directories. illustration by author Gemini assisted

11.3 Backlinks and Tags — The LLM's Index

Obsidian's backlinks ([[link]]) and tags (#tag) express relationships between markdown files. For the LLM, this is its index [7].

Backlink usage:


# Soft Robotic Hand Papers

Related: [[Gelsight Sensor]] [[Tactile Sensing]] [[Dexterous Manipulation]]

## Key Papers
- [[Tactile Sensing Survey]] — Tactile Sensing Overview...
- [[Dexterous Manipulation Paper]] — Dexterous Grasping with Force Feedback...

When the LLM is asked "list papers related to soft robotic hand," it follows backlinks from [[Soft Robotic Hand Papers]] to find related pages.

Tag usage: Build a consistent tag system and the LLM uses it as filter criteria. Examples: #reviewed, #keystone, #todo-summarize.

More tags is not better. In an LLM Wiki, tags are both search UI and agent instruction. A small set of state-like tags such as #todo, #draft, #reviewed, #keystone, and #conflict works better than vague tags like #interesting, #important, or #misc.

S8's four-month observation is that tags survive only when they drive action. A tag such as #reviewed changes whether a claim can be cited. #conflict triggers a synthesis pass. #todo-summarize creates a queue. Tags like #important mostly decay because they do not tell the next agent what to do.

11.4 How to Ask the LLM

Figure 11.2: Three question patterns for an LLM Wiki — Search, Synthesis, Connection. Each draws on a different part of the vault. illustration by author Gemini assisted

Three effective question patterns [4]:

Pattern 1: Search


"Show me the list of tactile sensing papers I read in 2026"

→ Search wiki/papers/, apply date filter

Pattern 2: Synthesis


"Summarize the 3 most common themes in my daily notes from the last 6 months"

→ Read all of raw/daily-notes/, extract patterns

Pattern 3: Connection


"Is there a connection between VLA models and tactile sensing in my wiki?"

→ Read wiki/concepts/VLA.md + wiki/concepts/TactileSensing.md, analyze intersections

Implementation posts after April 2026 emphasize a fourth pattern: audit [10].

Pattern 4: Audit


"Find pages created by last week's ingests that lack source links or duplicate the same concept."

→ Read wiki/ and logs/ together; clean broken links, duplicate pages, unsourced claims, and stale summaries

11.5 The Counter-Argument: You Still Have to Write Yourself

Post [12] offers a counterbalance: "If you delegate everything to AI, your thinking doesn't get externalized — it evaporates. Writing by hand is a verification of understanding. It's not the same as reading an LLM summary."

This is valid. The goal of an external memory system is not to replace your thinking but to amplify it. Finding the balance:

  • LLM handles collection, connection, organization
  • Human provides judgment, interpretation, new perspective
  • Maintain the habit of writing your own thoughts in daily notes
  • Important claims keep human-approved source links
  • LLM-written wiki pages are checked by diff and source, not trusted blindly

From Maker[5]'s 4-month usage report: "The LLM finds connections. Whether those connections are meaningful is still up to me."

S8 puts this as a division of labor. The LLM can broaden alternatives, surface contradictions, and draft summaries. The human writes the decision note: why this path, why not the other path, what evidence would change the decision. Without that human-owned layer, the vault becomes fluent but directionless.

Figure 11.3: Bidirectional flow of an external memory system — humans deposit daily notes, papers, and ideas into the vault; the LLM returns summaries, relationships, patterns, and new connections. illustration by author Gemini assisted

11.6 Bidirectional Flow

The key to an external memory system is bidirectional flow:

Human → LLM (input):

  • Write daily notes (raw/daily-notes/)
  • Add paper PDFs (raw/papers/)
  • Clip ideas (raw/ideas/)
  • When asking, state your perspective explicitly ("My research direction is X; in that context, I want to know about Y")

LLM → Human (output):

  • Summaries and synthesis (wiki/papers/ updates)
  • Discovering related papers (backlink traversal)
  • Identifying patterns (daily notes analysis)
  • Proposing new connections

The framing from [12]: "AI is leverage. Leverage amplifies your capacity — but without a foundation, there is nothing to amplify."

Post #7's conclusion [12]: "This is not just a homepage. It is a space where AI and I think together, and farther ahead, a research environment for an AI scientist that may itself generate the next layer of knowledge."

That gives the operating principle: external memory is not a pretty note app; it is state storage for a research loop. A good vault answers four questions. What did we ingest last time? Which claims still have weak sources? Which ideas became projects? Which experiments failed, and why were they abandoned? If the vault can answer those, it is ready to become the AI Scientist loop in Chapter 12.

S8's operational checklist is a compact test:

  1. Every generated claim points to a source or is marked unresolved.
  2. Every ingest leaves a logs/ entry.
  3. Every important fork leaves a human-written decisions/ note.
  4. Conflicts are tagged and revisited instead of smoothed over.
  5. Weekly lint checks for orphan pages, unsourced claims, stale todos, and duplicate concepts.

References

  1. terryum, "Brain Augmentation," post #7, 2026-03-10. [Um, 2026]
  2. terryum, "New Leverage," terryum-ai, 2026. [Um, 2026]
  3. terryum, "The value of writing yourself," terryum-ai, 2026. [Um, 2026]
  4. Karpathy, Andrej, "LLM Wiki," gist, 2026. [Karpathy, 2026]
  5. Aimaker, "AI-powered second brain — 4-month report," 2026. [Maker, 2026]
  6. Starmorph, "LLM Wiki guide," 2026. [Starmorph, 2026]
  7. Okhlopkov, "Claude Code 4-month setup retrospective," 2026. [Okhlopkov, 2026]
  8. Bsukistory, "AI automation perspective (Korean)," 2026. [Brunch, 2026]
  9. Anthropic, "Claude memory documentation," 2026. [Anthropic, 2026]
  10. WebEdge, "Karpathy's LLM Knowledge Base System: Full Breakdown of His CLAUDE.md Schema," 2026-04-11. [WebEdge, 2026]
  11. Cognition, "llm-wiki: the reference implementation of Karpathy's self-building AI memory pattern," 2026-04-16. [Cognition, 2026]
  12. terryum survey S8, "LLM Wiki to AI Scientist," 2026. [Um, 2026]