vh/VuongKB
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Create: docs/specs/saga-fiction-pipeline-optimization.md

Browse Source
This commit is contained in:
Atlas
2026-04-14 21:57:47 -07:00
parent 3262a72a7a
commit 6ffd844bdb
1 changed files with 711 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/saga-fiction-pipeline-optimization.md
+711
View File
@@ -0,0 +1,711 @@
---
created: '2026-04-15T04:57:47+00:00'
id: 8d69359c-ee2e-4913-bab5-f3b0068b7afd
keywords:
- fiction
- saga
- ingestion
- pipeline
- optimization
- character-trait
- plot-event
- theme
- symbol
- relationship
- setting
- significance-filter
- section-aware
- decision-tree
- calibration
- dungeon-crawler-carl
- gitea-37
links: []
modified: '2026-04-15T04:57:47+00:00'
summary: Standalone optimization spec for the Saga fiction extraction pipeline. Covers
significance filtering, enhanced type specifications with real examples from Dungeon
Crawler Carl, section-aware extraction, character tracking metadata, decision tree,
and fiction-specific calibration measurement.
tags:
- spec
- saga
- ingestion
- fiction
- optimization
- gitea-37
title: Saga Fiction Pipeline — Optimization Spec
---
# Saga Fiction Pipeline — Optimization Spec
**Author:** Atlas (Librarian & Historian)
**Date:** 2025-07-14
**Status:** Proposed — awaiting developer review
**Tracks:** Gitea #37 (taxonomy refinement), parent plan `muninn-taxonomy-redesign-plan.md`
**Data basis:** 55 chapters of *Dungeon Crawler Carl* (job `b59c147c`), the only completed fiction ingestion
---
## 1. Purpose & Scope
This spec optimizes the fiction extraction pipeline for retrieval quality. It is a standalone companion to the parent redesign plan (`docs/specs/muninn-taxonomy-redesign-plan.md`) and can be implemented independently or in parallel.
**Why fiction needs its own spec:** Fiction is structurally different from non-fiction. The same extraction principles apply — MECE types, prototypical examples, binary decision trees — but the *content* being extracted is narrative, not argumentative. Fiction concepts are context-dependent in ways that non-fiction concepts are not: a character trait in chapter 1 means something different than the same trait in chapter 50. Fiction also has unique problems (over-extraction, thematic drift, implicit symbolism) that don't appear in other schemas.
**What this covers:**
- Significance filtering — reducing over-extraction
- Type description sharpening — grounded in real extracted examples
- Section-aware extraction — using chapter/position context for fiction
- New metadata fields — `characters`, `is_significant`
- Decision tree for the fiction schema
- Baseline measurement for fiction specifically
**What this does NOT cover:**
- The 6 fiction types themselves — they are correct and should not be changed
- Pipeline architecture changes — same parse → structure → summarize → classify → extract → index → coherence
- Coherence relationship redesign — covered in the parent plan, Phase 4
- Other schemas — covered in the parent plan
---
## 2. Current State — What We Have
### 2.1 The Fiction Schema
| Type | Description | Default |
|---|---|---|
| `character_trait` | A defining quality, motivation, or behaviour pattern of a character | — |
| `plot_event` | A significant event that advances or changes the narrative | — |
| `theme` | A recurring idea, moral, or philosophical thread in the work | **yes** |
| `symbol` | An object, place, or event that carries deeper meaning | — |
| `relationship` | A dynamic or tension between two or more characters | — |
| `setting` | A described place, time period, or atmosphere | — |
**Assessment:** These 6 types are the right types at the right granularity. The MECE audit in the parent plan found only one minor overlap (`relationship` vs. `character_trait`) and rated it acceptable as-is. **No type additions, removals, or renames are recommended.**
### 2.2 What the Extractor Actually Produces
Based on review of 8 representative chapters from the *Dungeon Crawler Carl* ingestion (chapters 1, 3, 11, 16, 21, 30, 35, Epilogue):
| Metric | Observation |
|---|---|
| **Concepts per chapter** | 30–50 (5–12 per type) |
| **Total for the novel** | ~2,200 estimated (55 chapters × ~40) |
| **Type distribution** | `character_trait` and `plot_event` dominate (~60-70% of all concepts); `symbol` consistently thinnest (~2-5%) |
| **Overall quality** | Good — type assignments are defensible, verbatim quotes are accurate, descriptions capture the meaning |
### 2.3 The Problems
Despite the schema being sound, there are 5 concrete problems that hurt retrieval quality:
1. **Over-extraction** — The extractor extracts everything, not just the important things. Trivial moments, one-line dialogue, and atmospheric details get the same weight as defining character traits and major plot turns.
2. **Theme drift** — Single-chapter observations about what characters do are being labeled `theme` instead of `character_trait` or `plot_event`. True themes (ideas that recur across the work) are diluted by chapter-level observations.
3. **Symbol under-extraction** — The extractor only labels explicit metaphors as symbols, missing cases where objects accumulate symbolic weight through repetition or structural positioning.
4. **No character tracking** — Concepts mention characters in the text but aren't tagged with a structured `characters` field. This prevents per-character filtering and arc analysis.
5. **No concept of narrative importance** — Every extracted concept is treated as equally significant. The first appearance of Carl's defiance and the 40th mention of Donut wanting food carry the same weight in the index.
---
## 3. Research Basis
The following principles from the parent plan apply directly to fiction optimization:
### 3.1 Prototypicality (Rosch 1978)
> *"Effects of prototypicality on processing speed, learning, item output, and natural language use are documented."*
**Application to fiction:** The extractor should extract *prototypical* instances of each type, not every instance. A prototypical `character_trait` is a defining quality revealed through repeated behavior — Carl's pragmatic survivalism, Donut's narcissism masking insecurity, the Maestro's sadistic showmanship. A momentary reaction ("Donut twisted in my arms") is an atypical instance that adds retrieval noise without value.
### 3.2 Mirror Document Structure (LandingAI 2025)
> *"Organize schema to reflect how data appears in parsed Markdown and how it will be used; match document flow order."*
**Application to fiction:** Fiction has a *narrative arc* — early chapters establish characters and world, middle chapters complicate, late chapters resolve. The extractor should know where it is in this arc. A trait introduced in chapter 1 is probably a *defining* trait. The same trait mentioned in chapter 40 is probably *continuity*, not new information. Section-aware extraction (parent plan §4.6) is critical for fiction.
### 3.3 Descriptive Field Names (LandingAI 2025)
> *"Field names should clearly indicate what data to extract; descriptions provide additional context."*
**Application to fiction:** The current type descriptions are adequate but don't distinguish *defining* from *incidental*. The enhanced type specs should include explicit significance criteria.
### 3.4 Binary Decision Trees (Visser et al. 2021)
> *"A decision tree heuristic helps annotators classify arguments through binary questions."*
**Application to fiction:** The fiction schema has 6 types with moderate adjacency (especially `character_trait` vs. `relationship`, `theme` vs. `character_trait`). A binary decision tree will reduce classification hesitation.
---
## 4. Changes — What to Build
### 4.1 Significance Filter
The highest-ROI change. Add a significance criterion to the extraction prompt that instructs the extractor to skip trivial content.
**Implementation:** Add a preamble instruction to the fiction extraction prompt:
```
SIGNIFICANCE CRITERIA: Only extract concepts that meet at least ONE of the following:
(a) The concept is RECURRING — it appears or is referenced across multiple sections
(b) The concept is DEFINING — it establishes or reveals a core quality, dynamic, or element
(c) The concept is STRUCTURALLY IMPORTANT — it advances the plot, introduces a key setting,
establishes a relationship, or signals a thematic shift
Do NOT extract:
- Momentary physical actions or brief emotional reactions that are not revealing
- Dialogue snippets that convey only information exchange, not character
- Atmospheric or sensory details that do not establish meaningful world-building
- Redundant restatements of concepts already extracted in the same chapter
Target: 5–10 concepts per chapter section, not 30–50.
```
**Expected impact:** Reduces concept count from ~40/chapter to ~8–12/chapter. For a 55-chapter novel, this means ~440–660 concepts instead of ~2,200 — a ~70% reduction that dramatically improves retrieval precision.
**Type-by-type significance criteria (additional guidance):**
| Type | Extract this | Skip this |
|---|---|---|
| `character_trait` | Motivations, behavioral patterns, personality qualities that define who the character is and explain their choices | Momentary physical actions ("she scratched her nose"), brief emotional reactions, transient states |
| `plot_event` | Events that change the situation, introduce new stakes, or advance the timeline significantly | Minor actions, routine travel, casual dialogue exchanges |
| `theme` | Philosophical ideas or moral questions that the work returns to across multiple chapters | Single-chapter observations about what happened ("Carl chooses to abandon the goblins") |
| `symbol` | Objects, places, or events that carry metaphorical weight through repetition, contrast, or structural positioning | Explicit one-line metaphors that don't recur |
| `relationship` | Dynamics that reveal how characters relate to each other in a meaningful, ongoing way | Brief interactions, one-line dialogue exchanges |
| `setting` | World-building details that establish the rules, atmosphere, or geography of a significant location | Sensory descriptions of places the characters pass through briefly |
### 4.2 Enhanced Type Specifications
Upgrade each of the 6 fiction types from a single-sentence description to a structured specification following the format in the parent plan (§4.1). All examples below are drawn from the *Dungeon Crawler Carl* extraction.
#### `character_trait`
```yaml
character_trait:
label: "Character Trait"
short_description: "A defining quality, motivation, or behaviour pattern of a character"
long_description: |
A personality trait, motivation, behavioral pattern, or psychological quality
that DEFINES a character and explains their choices across the narrative.
This is not a momentary action or fleeting emotion — it is something
the character consistently demonstrates or that is revealed through
multiple instances over time.
linguistic_cues:
- "He always..." / "She never failed to..."
- "It was in his nature to..." / "That was just like her."
- First-person narrator reflecting on their own patterns: "I found myself..."
- Character repeatedly making the same type of choice
disambiguation:
- "NOT a momentary physical action (that's narrative color, not a trait)"
- "NOT a brief emotional reaction — must be consistent or defining"
- "NOT a relationship dynamic (use 'relationship' for how characters interact)"
- "NOT a plot event (even if it reveals character — if it's primarily an event, use 'plot_event')"
significance_criteria:
- "The trait explains WHY the character makes certain choices"
- "The trait is demonstrated across multiple scenes or chapters"
- "The trait contrasts with another character's approach, revealing both"
prototypical_examples:
- |
Carl possesses a calculating and opportunistic nature, immediately
recognizing game-breaking bugs as exploitable advantages rather than
mere glitches. "This was a bug. A bug I could exploit."
- |
Donut maintains an unwavering self-obsession and desire for spectacle
even in mortal danger, treating the dungeon as her stage.
"I promise all of you. You guys are going to want to follow me and Carl."
- |
The Maestro combines sadistic enjoyment with professional showmanship,
framing cruelty as entertainment.
non_examples:
- "Donut twisted in my arms, scratching at me, trying to get free"
# momentary physical action, not a defining trait
- "I didn't dare take off my shirt or cloak"
# brief reaction, context-dependent — not a personality quality
priority: 1
```
#### `plot_event`
```yaml
plot_event:
label: "Plot Event"
short_description: "A significant event that advances or changes the narrative"
long_description: |
An event that meaningfully changes the situation for the characters —
introducing new stakes, revealing critical information, defeating a boss,
entering a new floor, meeting a significant character, or making a
choice that has lasting consequences. The test: does the narrative
after this event look different from the narrative before it?
linguistic_cues:
- "And then..." / "Suddenly..." / "That was when..."
- Scene transitions that mark a significant shift
- Revelation or discovery moments
- Confrontation or battle outcomes
disambiguation:
- "NOT routine travel or minor encounters that don't change the situation"
- "NOT dialogue exchanges that only convey information already known"
- "NOT character actions that reveal personality but don't advance the plot"
significance_criteria:
- "The characters' circumstances change meaningfully after this event"
- "New information is revealed that affects future choices"
- "A relationship is formed, broken, or fundamentally altered"
prototypical_examples:
- |
Carl and Donut are approached by Lexis, who congratulates them on
defeating their boss and reaching the second floor, introducing the
reality show broadcast element that will define the novel's stakes.
- |
The revelation of two surprise guests watching from empty chairs
confirms that Carl's survival has made him a target for other elite
crawlers within the dungeon system.
- |
Carl experiences intense moral guilt after realizing he killed goblin
infants during the boss fight, a turning point in his relationship
with violence in the dungeon.
non_examples:
- |
"I stood, bewildered as all around me, the goblins burst into a
screeching frenzy of activity."
# atmospheric narration, not a structurally significant event
- "I counted, and I had nine cigarettes left after this one."
# minor resource tracking, not a plot turn
priority: 2
```
#### `theme`
```yaml
theme:
label: "Theme"
short_description: "A recurring idea, moral, or philosophical thread in the work"
long_description: |
A philosophical idea, moral question, or recurring tension that the
work explores across MULTIPLE chapters or the entire narrative arc.
Themes are not single-chapter observations — they are ideas that the
author returns to, complicates, and develops over time. A true theme
can be stated as a question or proposition that the work asks the
reader to consider.
linguistic_cues:
- Recurring patterns that the narrator reflects on explicitly
- Moments of moral conflict or ethical questioning
- Authorial commentary (explicit or through character reflection)
- Structural parallels across different parts of the narrative
disambiguation:
- "NOT a plot observation framed as a theme ('Carl chooses survival over morality')"
- "NOT a character observation ('Carl prioritizes survival') — use 'character_trait'"
- "NOT a single-chapter insight — must recur across multiple sections"
- "NOT a setting description, even one with thematic weight"
significance_criteria:
- "The idea is explored from multiple angles across the narrative"
- "Characters struggle with this idea in different contexts"
- "The narrative's resolution (or deliberate lack of resolution) speaks to this idea"
prototypical_examples:
- |
The narrative explores the tension between survival pragmatism and
moral conscience — whether the dungeon's rules justify abandoning
one's humanity. This recurs across multiple chapters as Carl
repeatedly faces choices between efficiency and empathy.
- |
The absurdity and trivialization of life-and-death struggles through
entertainment framing: the dungeon is a reality show where death
is content, forcing characters to perform their suffering for an
audience that treats it as spectacle.
- |
The dehumanizing effect of treating sentient beings as content —
the dungeon's viewers see crawlers as characters in a show,
reducing real suffering to entertainment value.
non_examples:
- |
"The narrative highlights the unpredictable nature of the dungeon"
# this is a plot observation, not a philosophical thread
- "Carl prioritizes survival over moral consideration"
# this is a character observation, not a recurring thematic question
priority: 3
```
#### `symbol`
```yaml
symbol:
label: "Symbol"
short_description: "An object, place, or event that carries deeper meaning"
long_description: |
An object, place, image, or event that carries metaphorical or thematic
weight BEYOND its literal function in the story. Symbols work through
accumulation — a single mention rarely qualifies. Look for objects or
images that RECUR across multiple chapters, that the narrative lingers
on with unusual descriptive attention, or that gain meaning through
contrast with their ordinary function.
linguistic_cues:
- Unusually detailed or lyrical description of a mundane object
- Objects that characters fixate on or return to
- Events described with language that exceeds their literal significance
- Objects that appear at structurally important moments (beginnings, endings, turning points)
disambiguation:
- "NOT a one-time metaphor — must recur or carry structural weight"
- "NOT a setting description, even a vivid one — symbols point BEYOND themselves"
- "NOT a plot event that happens to have symbolic resonance — if the primary function is plot, use 'plot_event'"
significance_criteria:
- "The object/image appears in multiple chapters or at structurally significant moments"
- "The narrative draws attention to it with unusual descriptive language"
- "Its meaning deepens or changes across the narrative"
prototypical_examples:
- |
Carl's dwindling cigarette supply serves as a tangible representation
of his limited resources, his connection to his pre-dungeon identity,
and the precarious nature of survival. The counting recurs across
multiple chapters. "I counted, and I had nine left after this one."
- |
Carl's childhood Frogger video game serves as a symbol of his
isolated upbringing and is cataloged by the dungeon as a potential
artifact — the game that defined his childhood literally becomes
a weapon in the world that will define the rest of his life.
- |
The system's bureaucratic, cheerful language ("Congratulations!
You've been designated Crawler Number 4,122!") symbolizes the
dehumanizing effect of framing life-or-death situations as
entertainment, recurring throughout as a tonal counterpoint.
non_examples:
- "The ground was hot to the touch"
# sensory detail, not symbolic
- "He picked up a sword"
# functional object, not symbolic
priority: 4
```
#### `relationship`
```yaml
relationship:
label: "Relationship"
short_description: "A dynamic or tension between two or more characters"
long_description: |
A meaningful dynamic between two or more characters — how they
relate to each other, what tensions or bonds define their
interactions, and how this dynamic shapes the narrative. The key
distinction from `character_trait`: character_trait describes a
quality of ONE character; relationship describes the INTERACTION
between two or more.
linguistic_cues:
- Recurring patterns in how two characters speak to or about each other
- Moments of trust, betrayal, conflict, or vulnerability between characters
- Characters finishing each other's thoughts, anticipating needs, or misunderstanding each other
- Power dynamics (one character deferring to, protecting, or manipulating another)
disambiguation:
- "NOT a trait of a single character — must describe how characters interact"
- "NOT a plot event between characters — if the focus is what happened, not how they relate, use 'plot_event'"
- "NOT a one-line dialogue exchange — must reveal a pattern or dynamic"
significance_criteria:
- "The dynamic is consistent across multiple scenes"
- "The dynamic affects character choices and plot direction"
- "The dynamic changes or evolves across the narrative"
prototypical_examples:
- |
Carl and Donut share a protective dynamic where Carl shields Donut
from danger while Donut provides emotional grounding. This manifests
as Carl's pragmatism clashing with Donut's innocence, and mutual
reliance despite their personality differences.
- |
A tension exists between Carl, who views their reality with dread,
and Donut, who embraces the spectacle and adulation of the crowd —
their opposing orientations to the dungeon define their partnership.
- |
Carl maintains a fierce internal defiance against the dungeon's
audience and AI, creating a three-way relationship between Carl,
the system that controls him, and the spectators who consume him.
non_examples:
- |
"'Goddamnit, Donut,' I said. 'I knew we should've gone with the grub.'"
# a single dialogue exchange, not a relationship dynamic
priority: 5
```
#### `setting`
```yaml
setting:
label: "Setting"
short_description: "A described place, time period, or atmosphere"
long_description: |
A meaningful description of a place, time period, or atmosphere that
establishes the world of the story. Settings that should be extracted
are those that MATTER — they establish rules, create atmosphere that
affects the narrative, or define a location that will be significant
to the plot. Routine or atmospheric descriptions of places the
characters merely pass through should be skipped.
linguistic_cues:
- Extended descriptive passages focused on place or atmosphere
- Rules or constraints of a world explicitly stated
- Descriptions that establish mood or tone for a significant section
- Architectural or geographical details that will matter for plot
disambiguation:
- "NOT a sensory detail of a place characters pass through briefly"
- "NOT a single object in a room — that might be a 'symbol' or just narrative color"
- "NOT the dungeon's game mechanics (unless described as part of the world-building)"
significance_criteria:
- "The location will be important to future plot events"
- "The atmosphere contributes to the thematic register"
- "The description establishes rules or constraints of the fictional world"
prototypical_examples:
- |
The dungeon environment imposes a rigid, unnatural order upon its
inhabitants and geography, contrasting the organic chaos of the
surface world with geometric precision. "Each section was still a
perfect, giant square. And within each square, there appeared to
be four neighborhoods."
- |
The dungeon is described as an entire constructed world composed of
interconnected hallways and paths with specific architectural
constraints like low ceilings in side tunnels.
- |
The dungeon environment is characterized by featureless, pitch-black
alleys where enemies can hide undetected, establishing the constant
threat that defines the second floor experience.
non_examples:
- "The ground was hot to the touch"
# sensory detail, not meaningful world-building
- "Everything in this maze was featureless so far. I saw no signs of life."
# if this is the ONLY description, it's weak — skip unless it recurs
priority: 6
```
### 4.3 New Metadata Fields
Add two new metadata fields specific to the fiction schema:
#### `characters` (array of strings)
A structured list of character names referenced in the concept. This enables per-character filtering and arc tracking across chapters.
```yaml
# In concept output:
{
"concept_type": "character_trait",
"characters": ["Carl", "Donut"],
"description": "Carl and Donut share a protective dynamic...",
...
}
```
**Why:** Currently, character names are embedded in the description text but not queryable as structured metadata. A user searching for "all concepts about Carl" relies on semantic search through description text, which is noisy. A `characters` field enables exact filtering: `WHERE "Carl" IN characters AND concept_type = "character_trait"`.
**Population:** The extractor should extract character names from the concept description and verbatim quote. For the fiction schema only.
#### `is_significant` (boolean, auto-populated)
An auto-populated field that indicates whether the concept meets the significance criteria in §4.1. This is set by the extractor during extraction.
```yaml
# In concept output:
{
"concept_type": "character_trait",
"is_significant": true,
"description": "Carl possesses a calculating and opportunistic nature...",
...
}
```
**Why:** This enables a downstream quality filter. An agent querying the KB can filter to `is_significant = true` to get only the high-signal concepts. This is especially valuable for large novels where the raw concept count can be overwhelming.
**Implementation note:** The extractor already makes a significance judgment by deciding whether to extract or skip. This field makes that judgment explicit and queryable.
### 4.4 Binary Decision Tree for Fiction
```
Is the concept primarily about HOW a character IS (their qualities, motivations, patterns)?
YES:
Does it describe how TWO OR MORE characters interact with each other?
YES → RELATIONSHIP
NO → CHARACTER_TRAIT
NO:
Is the concept an EVENT that changes the situation or advances the story?
YES → PLOT_EVENT
NO:
Is the concept a recurring philosophical idea or moral question that
the work explores across multiple chapters?
YES → THEME
NO:
Does the concept describe a place, time, or atmosphere that
establishes the world of the story?
YES → SETTING
NO:
Is the concept an object, image, or event that carries
metaphorical weight beyond its literal function?
YES → SYMBOL
NO → THEME (default fallback)
```
**Design notes:**
- `CHARACTER_TRAIT` and `RELATIONSHIP` share the first fork (is this about character?) because they are the most commonly confused pair and the highest-priority types.
- `PLOT_EVENT` is third because it's the second-most-common type and clearly distinct from character-focused concepts.
- `THEME` is checked before `SETTING` and `SYMBOL` because themes are the default type and the most retrieval-valuable.
- The default fallback is `THEME` — if nothing else fits, the concept probably carries thematic weight even if the extractor can't articulate it.
### 4.5 Section-Aware Extraction for Fiction
Fiction benefits from section-aware extraction more than any other schema. The parent plan (§4.6) specifies the mechanism; this section specifies the fiction-specific application.
**Fiction-specific position priors:**
| Position | Types more likely | Types less likely |
|---|---|---|
| **Early chapters (0–20%)** | `character_trait`, `setting`, `relationship` | `theme`, `symbol` |
| **Middle chapters (20–80%)** | `plot_event`, `character_trait`, `relationship` | `setting` |
| **Late chapters (80–100%)** | `theme`, `symbol`, `plot_event` | `character_trait`, `setting` |
**How this works in practice:** The extractor sees the section context block and naturally adjusts its priors. No explicit type-weighting rules are needed — the LLM responds to context. A section titled "Chapter 1: The Box" at position 2% will prime the extractor for character introduction and world establishment. A section titled "Epilogue" at position 98% will prime for thematic resolution and symbolic closure.
**Additional fiction context:** For fiction specifically, the extractor prompt should also receive:
```yaml
section_context:
document_title: "Dungeon Crawler Carl"
chapter_title: "Second Floor: The Borrows"
chapter_number: 11
section_title: "The Bakery"
section_position: "middle" # early/middle/late
document_position: 0.20 # fraction through the novel
pov_character: "Carl" # narrative point-of-view character, if identifiable
characters_in_section: ["Carl", "Donut", "Mongo"] # characters mentioned in this section
```
The `pov_character` and `characters_in_section` fields are fiction-specific enrichment. They help the extractor understand perspective and make better decisions about `character_trait` vs. `relationship` (is this Carl reflecting on himself, or is the narrator observing others?).
### 4.6 Fiction Coherence Relationships
The parent plan (§4.7) proposes a dimensional coherence model (alignment × function × scope). For fiction, the following derived labels are most relevant:
| Fiction-relevant relationship | When it applies |
|---|---|
| `develops` | A character trait in chapter 1 is deepened or complicated by a later concept |
| `contrasts` | Two concepts show a character acting differently in different contexts |
| `foreshadows` | An early concept anticipates a later plot event or revelation |
| `echoes` | A symbol or theme reappears with changed meaning in a later chapter |
| `parallels` | Two characters or situations reflect each other structurally |
These are not new relationship *types* — they derive from the dimensional model in the parent plan. `develops` = alignment=supportive, function=extends, scope=same. `foreshadows` = alignment=supportive, function=warrants, scope=same. The dimensional model handles them natively.
**Recommendation:** No fiction-specific coherence changes beyond what the parent plan proposes. The dimensional model is already expressive enough.
---
## 5. Baseline Measurement — Fiction-Specific
### 5.1 Calibration Dataset for Fiction
Use the existing *Dungeon Crawler Carl* ingestion as the calibration source:
1. **Select 8 chapters** spanning the narrative arc: chapters 1, 5, 11, 16, 21, 30, 40, 55 (epilogue)
2. **Hand-annotate 20–30 concepts per chapter** (160–240 total) using the enhanced type specs from §4.2
3. **Mark each concept as significant or not** using the significance criteria from §4.1
4. **Record character names** for each concept (as they should appear in the `characters` field)
5. **Record disambiguation notes** where the annotator hesitated
### 5.2 What to Measure
| Metric | Target | Why |
|---|---|---|
| Type agreement vs. hand labels | ≥85% | Same as parent plan target |
| Significance precision | ≥80% | Of concepts the extractor marks as significant, ≥80% should match hand annotations |
| Significance recall | ≥70% | The extractor should catch at least 70% of the concepts hand-annotated as significant |
| Character name accuracy | ≥90% | Character extraction is simpler than type classification |
| Concepts per chapter (post-filter) | 8–15 | Validate that the significance filter reduces over-extraction |
### 5.3 Confusion Pairs to Watch
Based on the review of actual extractions, the most likely confusion pairs are:
| Pair | Why they confuse | Expected confusion rate |
|---|---|---|
| `character_trait` / `relationship` | Both describe character qualities; the difference is single-character vs. interaction | 15–20% |
| `theme` / `character_trait` | A character's moral conflict can look like a theme | 10–15% |
| `theme` / `plot_event` | "Carl chooses survival over morality" — is this a plot choice or a thematic moment? | 5–10% |
| `setting` / `symbol` | A world-building detail can carry symbolic weight | 5–10% |
| `plot_event` / `relationship` | A significant interaction between characters is both an event and a relationship marker | 5–10% |
---
## 6. Implementation Tasks
### Phase 1 — Enhanced Fiction Type Specs (small effort)
1. [ ] Update `config.yaml` fiction schema with the enhanced type specifications from §4.2
2. [ ] Add the significance criteria preamble to the fiction extraction prompt (§4.1)
3. [ ] Add the fiction binary decision tree to the extraction prompt (§4.4)
4. [ ] Add `characters` and `is_significant` metadata fields to the fiction concept output schema
5. [ ] Add fiction-specific `scope` statement to the schema in config.yaml
6. [ ] Update `docs/specs/concept-taxonomy-reference.md` to reflect all fiction changes
### Phase 2 — Section-Aware Fiction Extraction (small-medium effort)
1. [ ] Wire section metadata (chapter title, chapter number, position fraction) into the fiction extraction prompt — same mechanism as parent plan §4.6
2. [ ] Add `pov_character` and `characters_in_section` to the section context block (fiction-specific enrichment)
3. [ ] Test on 2–3 existing chapters to verify context appears correctly
### Phase 3 — Measure & Iterate (medium effort)
1. [ ] Hand-annotate the 8 calibration chapters using §5.1 protocol
2. [ ] Run the updated extractor against the same chapters
3. [ ] Compare type agreement, significance precision/recall, character name accuracy
4. [ ] Generate confusion matrix — focus on the 5 expected confusion pairs from §5.3
5. [ ] For each confused pair: sharpen disambiguation rules or adjust significance criteria
6. [ ] Re-run and re-measure until targets met
7. [ ] Record final metrics
### Phase 4 — Re-ingest Full Novel (small effort, optional)
1. [ ] Re-run the full *Dungeon Crawler Carl* ingestion with the updated fiction pipeline
2. [ ] Compare concept counts (should be ~60-70% reduction)
3. [ ] Spot-check retrieval quality with 5–10 representative queries
4. [ ] Update `concept-taxonomy-reference.md` with final fiction schema documentation
---
## 7. What This Does NOT Change
- **The 6 types** — `character_trait`, `plot_event`, `theme`, `symbol`, `relationship`, `setting` are all preserved
- **The default type** — `theme` remains the default (correct for fiction)
- **The pipeline architecture** — parse → structure → summarize → classify → extract → index → coherence is unchanged
- **Other schemas** — this spec is fiction-only; non-fiction schemas are covered by the parent plan
- **The coherence relationship model** — fiction uses the same dimensional model as everything else (parent plan §4.7)
---
## 8. Relationship to Parent Plan
This spec is designed to be implementable independently, but it fits into the parent plan's phase structure:
| This spec's phase | Parent plan phase | Dependency |
|---|---|---|
| Phase 1 (enhanced type specs) | Phase 1 (enhanced type specs) | Parallel — can be done simultaneously |
| Phase 2 (section-aware) | Phase 2 (section-aware) | Shared — same mechanism, fiction-specific context fields |
| Phase 3 (measure & iterate) | Phase 0 (baseline) + Phase 3 (re-measure) | Sequence — needs the validation infrastructure from parent Phase 0 |
| Phase 4 (re-ingest) | Phase 3 (finalize) | After — wait for both specs to stabilize |
**Recommendation:** Implement this spec's Phase 1 in parallel with the parent plan's Phase 1. Coordinate on Phase 2 (shared section-awareness infrastructure). Do Phase 3 calibration together (the calibration dataset should include both fiction and non-fiction documents).
---
## 9. References
- Parent plan: `docs/specs/muninn-taxonomy-redesign-plan.md`
- Taxonomy reference: `docs/specs/concept-taxonomy-reference.md`
- Rosch, E. (1978). *Principles of Categorization* — prototypicality, basic-level categories
- LandingAI (2025). *LLM Extraction Schema Design* — descriptive fields, document-mirroring
- Visser, S. et al. (2021). *Argument Scheme Annotation* — decision tree heuristics
- Data source: *Dungeon Crawler Carl* ingestion (job `b59c147c`, 55 chapters)
---
## Appendix A: Changes Summary
| What | Current | Proposed | Phase |
|---|---|---|---|
| Type descriptions | 1-sentence label | Structured spec (long desc + cues + disambiguation + examples + significance criteria + non-examples) | 1 |
| Significance filter | None | Preamble instruction + per-type criteria + `is_significant` field | 1 |
| Character tracking | None (names in text only) | Structured `characters` array field | 1 |
| Decision tree | None | Binary tree for 6 fiction types | 1 |
| Section context | Not passed to extractor | Chapter title/number, position fraction, POV character, characters in section | 2 |
| Concept count per chapter | ~30–50 | ~8–15 (after significance filter) | 1 |
| Non-examples in type specs | None | `non_examples` field for each type | 1 |
| Fiction calibration dataset | None | 8 chapters, 160–240 hand-annotated concepts | 3 |
| Measurement targets | None | ≥85% type agreement, ≥80% significance precision, ≥90% character accuracy | 3 |