The Shape of Suspense: Computational Measurement of Narrative Tension Across Short Fiction

Computational Suspense Atlas — working paper, 2026-09.

Abstract

We build a computational instrument for measuring narrative suspense and use it to ask whether suspense can be measured from text at all, which features predict it, and whether language models reproduce the suspense dynamics of human-written fiction. Across 92 US-public-domain short stories (1820–1928, seven genres, 692,083 words), we compute six independent operationalizations of suspense per 40-bin narrative-progress segment — an instructed language-model rating, four hand-built lexical proxies (threat, uncertainty, arousal, action), and contextual GPT-2 surprisal — and validate the LLM rater against prompt wording, temperature, stated narrative position and a second model on a stratified subset. The measures agree moderately on shape within a story (LLM suspense and the composite lexical proxy correlate at ρ ≈ 0.4, positively in every genre) and diverge sharply elsewhere: MiniLM sentence embeddings carry no linear signal for the rating (ρ ≈ 0), threat and arousal vocabulary are the strongest predictors under permutation importance and SHAP, and text-based models predict segment ratings at ρ = 0.59–0.68 under story-grouped cross-validation, falling only slightly under a stricter author-grouped split. Genre differences in curve shape are statistically reliable but small (9.7% of variance) and replicate in only two of six measures (LLM suspense and surprisal), and curve-shape clusters clear a shuffled null only weakly — we find no evidence for a small number of sharp archetypal suspense shapes. Stating a passage's position in the story shifts the LLM's rating by about the same amount whether or not the stated position is true, a direct behavioural signature of position-as-prior. Once a story's early rating level is known, its later text adds little further predictive power for the rest of the story. In a separate generative experiment, an instructed model asked to write stories with five prescribed suspense trajectories defaults to a late climax regardless of instruction: "rising" and "late climax" instructions succeed, but "early peak" and "repeated peaks" fail outright, with generated curves resembling a conventional dramatic arc more than the requested shape. Human validation is designed and deployed as a browser annotation study but not yet pooled at the time of writing; every claim here is explicitly bounded to what the machine measures support. Code, corpus manifest, cached model outputs and an interactive story explorer that ties every curve point to the passage it measures are public.

1. Introduction

Readers know suspense when they feel it, and critics have described its machinery for a century, but there is no agreed way to measure it in a text. The obstacles are not technical. Suspense is a reader response, not a property of words; it is anticipatory, so it depends on what has already been read and on what the reader fears may come; and it is easily confused with its neighbours — intensity (how eventful the present passage is), surprise (how unexpected it is), and negative affect. Any computational claim that "suspense rises here" therefore rests on an operational choice that the claim rarely states.

This study makes those choices explicit and compares them. We ask three things. Can suspense be measured from text at all? — that is, do independent operationalizations (an instructed language-model rater, hand-built lexical proxies for threat, uncertainty, arousal and action, a narrative-uncertainty index, GPT-2 contextual surprisal, and human ratings from a purpose-built browser study) converge on the same curve inside a story, and on the same ranking across stories? Which linguistic and narrative features predict the shape and intensity of suspense? — with leakage-safe, story- and author-grouped cross-validation, interpretable models, and a test of how early a story's later tension can be foreseen. And, in a deliberately separate experiment, do language models reproduce the suspense dynamics of human-written fiction? — by instructing a model to write stories with prescribed suspense trajectories and running the identical measurement pipeline on the result.

The material is a hand-assembled corpus of 92 US-public-domain short stories (1820–1928, seven genres, 1.2k–43k words) fetched from Project Gutenberg with full attribution, segmented under six schemes, and exposed through a research website whose story pages tie every point on a suspense curve to the passage it measures. The corpus is small by the standards of distant reading and large by the standards of annotated-suspense datasets; it was chosen so that every story could be rated under every condition and read by a human.

Our findings are deliberately modest. The operationalizations agree on the broad arc of most stories and disagree on the details; the language-model rater is stable to prompt wording and temperature but not immune to stated narrative position; text features predict segment ratings well above chance and well below the ceiling, with author-level generalisation costing a measurable share of that performance; genre differences in curve shape are real but small next to within-genre variation, and no set of "archetypal" curve shapes survives comparison with a shuffled null; and instructed generation follows requested trajectories only partly, with flat "low-suspense" stories the easiest instruction to satisfy. Every number is reproducible from a manifest, and every claim is bounded by the construct-validity caveats set out in the Reviewer-2 section.

2. Literary background

Suspense has a long theoretical literature and a short empirical one. In the structural-affect theory of Brewer and Lichtenstein (1982), suspense is one of three discourse-level affects — with surprise and curiosity — produced by the arrangement of story events relative to their underlying chronology: a suspense structure discloses an initiating event whose significant, uncertain outcome the reader must wait for. Gerrig (1993) and Zillmann (1996) add the reader's side: suspense requires a character the reader is disposed to like, an outcome the reader fears or hopes for, and a subjective probability of the feared outcome that is high but not certain. Lehne and Koelsch (2015) generalise this across music, film and literature into a psychological model in which tension arises from states of conflict, instability, dissonance or uncertainty that trigger predictive processes directed at emotionally significant future events; tension resolves when the prediction is settled. The "paradox of suspense" — that rereaders feel suspense despite knowing the outcome (Smuts 2008; Delatorre et al. 2018) — matters for text-based measurement because it separates uncertainty (a property of the reader's information state) from suspense (an anticipatory affect that can survive certainty). Onea et al. (2026) ground the linguistic side of this in "potentially inquiry-terminating questions": binary questions raised by the text whose one answer would close a larger plot question, whose cumulative activation predicts continuously recorded reader suspense at R² ≈ 0.46 on one text.

Three consequences shape this study. First, suspense is latent and anticipatory: it is about what may happen, not what is happening, so a segment's rating depends on the story so far. Second, it is distinct from intensity (how eventful or violent the present passage is), from surprise (how unexpected the present passage is, backward-looking) and from valence. Third, because it lives partly in the reader, no text-only measure can be ground truth; several must be compared and the comparison itself reported.

3. Related computational work

Plot and emotional arcs. Sentiment-based "plot arc" work (Jockers's syuzhet, 2015; Reagan et al. 2016) smooths a valence series over narrative time and clusters the result; Reagan et al. found six dominant emotional-arc shapes across 1,327 Gutenberg books. Swafford's (2015) critique — that the Fourier low-pass filter imposes periodic boundary conditions and can invert the shape of a curve — is a warning that the smoothing and normalisation pipeline can manufacture archetypes, which is why we cluster against a within-story shuffled null and use a local Gaussian filter rather than a global transform. Boyd, Blackburn and Pennebaker (2020) measured "staging", "plot progression" and "cognitive tension" (a LIWC-style category) across ~40,000 narratives and found a consistent rise-and-fall of cognitive tension in the middle-to-late story; Toubia, Berger and Eliashberg (2021) quantified narrative "speed", "volume" and "circuitousness" from embeddings and related them to success.

Suspense modelling for generation. Cheong and Young's Suspenser (2015), O'Neill and Riedl's Dramatis (2014) and Doust and Piwek (2017) model suspense inside a story generator from the reader's inferred goals, plans and threats — symbolic accounts that operationalise Gerrig/Zillmann but require structured story representations we do not have for raw prose. Xie and Riedl (2024) plan suspenseful stories with LLMs using human pairwise judgments for evaluation.

Suspense measurement from text. The closest antecedent is Wilmot and Keller (2020), who collected sentence-level human suspense judgements on short stories and compared surprise (backward-looking unexpectedness of the current state) with uncertainty reduction (forward-looking unexpectedness of the continuation) computed over neural story representations; uncertainty reduction over representations predicted human judgements near human agreement. Our GPT-2 contextual surprisal is a (weaker, token-level) instance of their surprise measure; our LLM rating is a direct-judgement alternative. Ely, Frankel and Kamenica (2015) define suspense as the variance of next-period beliefs and surprise as belief movement, a formalisation that also treats suspense as forward-looking.

LLMs as suspense annotators and authors. Tian et al. (2024) find human-written stories more arousing and more negatively valenced toward the end than LLM stories, which are "homogeneously positive and lack tension". Matlin et al. (2025) replicate four psychological studies of suspense with language models and find that models distinguish suspenseful from non-suspenseful text but cannot reliably estimate relative intensity within a sequence, track rise and fall across segments, or stay consistent under adversarial permutations — a direct caution for our RQ4. Sui et al. (2026) propose "narrative forecasting" — sampling many continuations at every sentence and scoring how often they miss the truth — as a tension metric that ranks New Yorker stories above LLM stories, and echo Wilmot and Keller's forward-looking framing.

Where this study sits. We do not claim a new suspense model. The contribution is a reproducible measurement atlas: a public-domain corpus with attribution, six segmentation schemes, six operationalizations run side by side with their convergence quantified, an LLM-annotation validation battery (prompt, temperature, position, model), leakage-safe prediction with author-level generalisation, curve-shape clustering tested against a null, a human annotation instrument, and an explicitly separate generated-story experiment — all exposed as an interactive instrument in which every curve point is tied to the passage it measures.

4. Corpus

The corpus is 92 English-language short fictions in the United States public domain, fetched from Project Gutenberg by ebook number and recorded, story by story, in corpus/manifest.yaml with title, author, year of first publication, genre, language, source URL, ebook number, licence status, word and character length, and a SHA-256 of the clean text. Sixty-eight stories are sliced out of collections with regular-expression heading markers; the rest are standalone ebooks. All 692,083 words are free to redistribute, and the website links every story to its source ebook.

Selection was by hand, for coverage rather than representativeness: seven genre labels, thirty-seven authors, 1820–1928, and a length range from a 1,209-word Saki sketch (The Open Window) to a 42,793-word novella (The Turn of the Screw), with two further novellas (Carmilla, Dr Jekyll and Mr Hyde) so that absolute length is populated as an axis. Genre is a single hand-assigned dominant mode per story (Table 1); it is coarse — Sredni Vashtar is filed as literary, An Occurrence at Owl Creek Bridge as adventure — and the analyses treat it as a contestable label rather than a fact about the text. Three authors (Poe, Doyle, Wells) contribute nine stories each; author is therefore reported as a competing grouping and used as a stricter cross-validation group.

Table 1. Corpus composition by genre.

Texts are normalised (typographic quotes, Gutenberg's -- dash, _italic_ markers, hard line wraps) while preserving paragraph boundaries; chapter headings and roman-numeral section markers are kept as text, because they are part of what a reader sees.

5. Annotation

5.1 Human annotation instrument

Because no text-only measure can be ground truth, the study ships a browser annotation instrument (/annotate). Items are 213 segments of 80–260 words from the 40-bin segmentation, stratified by genre × position tercile with at most four items per story, so that both genre coverage and position bias can be estimated. A session begins with two calibration anchors — a manifestly calm passage (Miss Brill) and a manifestly tense one (The Pit and the Pendulum) with suggested ratings, recorded but excluded from analysis — followed by 24 items in a per-rater random order (a deterministic function of an anonymous rater id, so every session can be reconstructed), two attention checks whose instruction text specifies the expected answer, and one repeated item for within-session test–retest. Each response stores suspense (1–7), intensity (1–7), confidence (0–1), reading time in milliseconds, item id, story position, order in session and a hash of the excerpt. Raters pass quality control if both attention checks are correct, the repeat is within two points and median reading time is at least four seconds. Reliability is ICC(2,1) over items with two or more passing raters; agreement with machine measures is Spearman ρ over item means; the planned inferential model is a mixed-effects regression with rater and story random effects. At the time of writing no ratings have been pooled, and the paper reports the machine measures alone, with the human column marked as pending wherever it would otherwise appear.

5.2 LLM annotation

Every segment of every story was rated by an instruction-tuned open-weight model (Qwen3-8B, self-hosted, reasoning disabled) through a provider-neutral adapter that requests structured JSON — suspense (1–7), intensity (1–7), confidence (0–1) and an eight-word cue naming the main textual reason for the rating. The main condition uses a plain instruction ("rate SUSPENSE … and INTENSITY … as a reader encountering the passage in sequence would"), temperature 0, and no information about where the passage falls in the story. Every call is cached by a hash of its content so that re-running the pipeline returns identical ratings without a network call; the cache is committed as the reproducibility record.

Validation conditions were run on a fixed subset of fourteen stories (the shortest and the median-length story of each genre): a defined prompt carrying a definition of suspense after Brewer & Lichtenstein and Lehne & Koelsch; a contextual prompt that prepends the tail of the preceding text as a recap; temperature 0.8 with two independent samples; a true-position prompt that states the passage's narrative position; and a shuffled-position prompt in which the stated position is drawn from a different segment of the same story. Position bias is estimated as the slope of the rating change (condition minus main) on the stated position and, in the shuffled condition, as the correlation of that change with (stated − true) position. The same adapter interface supports the official Anthropic SDK and the local Claude CLI, so the battery can be repeated with a second model family.

6. Operationalizing suspense

We treat suspense as latent and compute six operationalizations per segment, each with a clear lineage and a clear failure mode:

1. Direct judgement (LLM rating). A 1–7 rating from an instructed model (§5.2). Fails if the model responds to vocabulary or position rather than anticipatory tension.
2. Event intensity. The LLM intensity rating, and lexicon rates for action (verbs of rapid motion, blows, flight) and threat (danger, death, weapons, pursuit, the supernatural). Fails wherever a calm passage is suspenseful — a character waiting for news — or a violent one is not.
3. Narrative uncertainty. Epistemic vocabulary (perhaps, seemed, wondered, unknown, whether …) plus question-mark rate. Follows Lehne & Koelsch's uncertainty component and Onea et al.'s question-driven account; fails on rhetorical questions and free indirect style.
4. Affect dynamics. A lexical arousal index (high-arousal minus low-arousal word rates), and, when the user supplies NRC-VAD, mean VAD arousal. Fails on ironic or reported emotion.
5. Information-theoretic surprise. Mean per-token GPT-2 surprisal of the segment given up to 256 tokens of preceding text. A backward-looking measure in Wilmot & Keller's terms; mixes narrative surprise with rare names, archaisms and dialect.
6. Human ratings (§5.1), pending.

Proxies 2–4 are z-scored within each story and averaged into a composite proxy, a transparent linear index whose constituents are visible word lists. Agreement between measures is quantified three ways: the mean within-story Spearman ρ across stories (shape agreement), the pooled Spearman ρ after within-story z-scoring, and the Spearman ρ of story means (level agreement). Curves for any measure are re-binned to a common 40-bin narrative-progress grid, z-scored within story and Gaussian-smoothed (σ = 1 bin), so that height differences between stories are removed and only shape is compared; the website exposes σ = 0, 1 and 2 so that no result depends on a hidden smoothing choice.

7. Features

Fifty-nine features are extracted per segment (Table 2), in four families. Lexical: sentence-length statistics and the share of very short sentences; word length; type–token ratio and its moving-average form (MATTR-50); punctuation rates per thousand characters for exclamation and question marks, dashes, ellipses, semicolons and commas; negation, temporal-term, first- and second-person pronoun rates; present- and past-tense function-word hints. Semantic: rates per hundred tokens for threat, uncertainty, action, conflict and perception vocabulary, six basic emotions (fear, anger, sadness, joy, surprise, disgust), a negative-emotion sum, a valence difference, and the arousal index; the lexicons are hand-curated seed lists committed with the code, chosen for transparency over coverage. Narrative (spaCy en_core_web_sm): dialogue ratio; entity, person and location density; distinct persons; character introductions (PERSON names not seen earlier in the story) and person turnover; location changes (GPE/LOC/FAC mentions absent from the previous segment); finite-verb, verb, noun, adjective and adverb rates; motion- and speech-verb rates; the rate of subject shifts between consecutive sentences; progressive-participle rate. Model-based: contextual GPT-2 surprisal (mean, SD, max, 90th percentile) and 384-dimensional MiniLM sentence embeddings, which enter the predictive models but not the interpretable feature tables.

Two features are stateful — character introductions and location changes depend on what came earlier in the story — and are computed by a tracker that consumes a story's segments in order; all others are functions of the segment text alone. Every feature is computed under every segmentation scheme.

8. Predictive models

The primary target is the LLM suspense rating per 40-bin segment; the composite proxy is a robustness target. Six models are compared: a global-mean baseline; TF-IDF bag-of-words ridge regression; ridge on the hand features; histogram gradient boosting on the hand features; ridge on MiniLM embeddings; and gradient boosting on hand features plus embeddings. Cross-validation is five-fold and grouped by story — a unit test asserts that no story appears on both sides of a fold — and repeated grouped by author, which forces the model to generalise past the nine-story authors. We report R², RMSE, pooled Spearman ρ with a bootstrap interval, and the mean within-story Spearman ρ, which asks the narrower question of whether the model recovers each story's shape. Interpretability uses permutation importance on held-out folds, mean absolute SHAP values from a tree explainer, standardized ridge coefficients, and a per-family ablation; none of these is read as causal.

For RQ2 (how early can future suspense be predicted?) a story-level dataset averages features — and the rating itself — over the first 10, 20, 30, 50 and 70 per cent of each story and predicts the mean, the maximum and the rise (future mean minus early mean) of the rating over the remainder, with story-grouped folds. An "early level only" reference model uses just the early rating; beating it means the opening's text carries information about what is coming beyond how tense it already is.

9. Narrative trajectories

For each story, the primary measure (LLM suspense) is re-binned to 40 equal narrative-progress bins, z-scored within the story and Gaussian-smoothed (σ = 1 bin); group means carry 95% bootstrap intervals over stories (2,000 resamples), and group differences are tested with a permutation test on the between-group share of curve variance (2,000 label shuffles) alongside Kruskal–Wallis tests on shape descriptors.

Genre. The seven genres differ reliably in trajectory shape (permutation p = 0.006), but the effect is small: genre membership accounts for 9.7% of the variance in curve shape across the corpus, the rest being within-genre variation between stories and authors. Peak position (Kruskal–Wallis H = 14.3, p = 0.027), the late-minus-early level (H = 21.5, p = 0.002) and the overall trend slope (H = 21.5, p = 0.002) all differ by genre; the number of local peaks does not (p = 0.77). Concretely: horror and ghost stories have the steepest rise into their final third (median late−early z of 1.11 and 1.19) and the latest median peaks (95% and 92% of the way through), while comic pieces are flattest (late−early 0.33, peak at 59%) — consistent with the twist-or-reveal structure of the horror/ghost sample (Poe, Le Fanu, M. R. James) against the anecdotal structure of the Saki/Leacock comic pieces.

Era and length. Neither publication era (1820s–1928, three bands) nor story length (short/medium/long) shows a reliable effect on curve shape (permutation p = 0.696 and 0.237). Given that era and length are both confounded with author and genre in a 92-story corpus, these should be read as "no effect detectable at this size," not as evidence of a true null.

Clustering versus a null. K-means on the z-scored curves, for k = 2…6, is compared at each k with the same clustering applied to within-story-shuffled curves (20 shuffles). The observed silhouette clears the shuffled null's 95th percentile at every k tested (e.g. k = 3: observed 0.108 vs. null mean 0.031, null 95th percentile 0.038), so the corpus's curves are not clustering-indistinguishable from noise. The silhouettes themselves are modest (0.08–0.11) — Reagan et al.'s six "basic shapes" and similar strong claims of universal archetypes are not supported at this scale; what survives the null is a weak, real tendency for curves to fall into a small number of families, not a small number of sharp families. The three-cluster solution separates roughly into a late-rising group (dominated by horror and ghost stories), an early-flat/level group, and a mixed-oscillation group with no strong genre signature; membership is explorable on the site.

Robustness across operationalizations — the genre effect does not generalise. Repeating the identical permutation test on the other five measures gives a mixed picture, not a confirmation. GPT-2 surprisal also shows a reliable genre effect (p = 0.011, 9.3% of variance) — an independently computed, non-LLM measure, which is evidence the genre signal is not purely an LLM artefact. But the composite lexical proxy (p = 0.817), the threat proxy alone (p = 0.842), the uncertainty proxy (p = 0.098) and the LLM's own intensity rating (p = 0.371) show no reliable genre effect at all. So two of six measures — LLM suspense and GPT-2 surprisal — carry a genre signal in their trajectory shape, and four do not. We read this as the honest version of RQ3: genre-specific suspense trajectories are supported for the anticipatory-judgement and surprise-style measures but not for vocabulary-level proxies of threat, uncertainty or intensity, which is itself informative about what "genre shape" is made of — plot-level anticipation and surprisal patterns, not simply a genre's characteristic vocabulary. Full table on /atlas.

10. Generative-AI experiment

This is a separate experiment with its own corpus directory; generated stories never enter the analyses above. Twenty stories were generated by the same Qwen3-8B endpoint at temperature 0.9 from a 5 × 4 design: five trajectory instructions — low (flat, nothing at stake), rising (each scene tenser than the last, no resolution), early peak (peak in the first quarter, then calm aftermath), late climax (quiet for two-thirds, one climax in the final fifth), repeated peaks (at least three peaks with lulls) — crossed with four fixed premises, so that the instruction rather than the premise drives differences. The prompt asks for 1,400–1,700 words in the register of early-twentieth-century short fiction; the model produced about 1,000 words per story, which the pipeline segments into 40 bins of ~25 words — a coarser measurement than in the human corpus, noted where it matters. Each story is segmented, featurized and rated exactly as the human corpus. Agreement with the requested shape is the Spearman ρ between the measured 40-bin curve and an idealised target curve (a line, a Gaussian at 18 %, a ramp from 66 %, three sinusoidal peaks); for the low instruction, which prescribes no shape, we report the raw within-story standard deviation of the rating. Generated and human stories are compared on within-story variability and on shape descriptors, never on level.

10.1 Results

Agreement with the requested trajectory is uneven, and the pattern is not random: the model has a strong default toward a late climax that the "rising" and "late climax" instructions align with and the "early peak" and "repeated peaks" instructions fight.

• Rising and late climax instructions work: mean shape agreement with the target curve is ρ = 0.55 and ρ = 0.44 (LLM-suspense measure), and every one of the eight stories in these two conditions peaks in the second half (median peak position 74% and 86% of the way through).
• Early peak fails outright: mean agreement is negative (ρ = −0.25). All four stories peak at 54–100% of the way through — the opposite end of the story from the requested ~18% — so the instruction to resolve tension early and let the rest be aftermath was not followed by any of the four generations.
• Repeated peaks also fails: mean agreement is near zero (ρ = −0.04), and the shape descriptors show why — three of four stories still have a single dominant late peak (peak position 64–82%) rather than the ≥3 evenly spaced peaks requested; only one story (gen_repeated_peaks_04) peaks earlier (36%), and even that story does not show a clearly multi-peaked structure.
• Low (flat, no suspense) is the instruction most reliably satisfied in relative terms (mean within-story SD 0.77 vs. 0.93 for the generated corpus overall and 1.07 for the human corpus), though it is not perfectly flat — three of four "low" stories still show a positive late-minus-early level.

Looked at across all twenty stories regardless of instruction, every single one has a positive late-minus-early level on the LLM-suspense measure. This is the clearest finding of the experiment: the generator's default narrative shape is a rise toward a late climax, largely independent of what it is asked for, which is exactly the shape the human corpus's horror and ghost stories favour (§9) and exactly the shape the "early peak" and "repeated peaks" instructions ask it to avoid. This echoes Tian et al.'s (2024) observation that LLM-written stories are structurally more homogeneous than human ones, and gives it a sharper, mechanism-level form: homogeneity here is not "flatter" but "converges on one conventional arc regardless of instruction."

The lexical composite proxy — computed independently of the generating model — shows the same ordering (rising 0.31, late climax 0.36, low 0.50, early peak −0.01, repeated peaks 0.31), which rules out the possibility that the effect is simply the same model rating its own output the way it was told to write it.

11. Results

11.1 RQ4 — how reliable is the LLM annotator?

The main condition rated all 3,677 bins40 segments (100% coverage, zero parse failures across 3,677 cached calls) with mean self-reported confidence 0.94 and a rating distribution skewed toward the low-to-middle of the scale (1: 17, 2: 345, 3: 977, 4: 915, 5: 861, 6: 480, 7: 82) — the model rarely uses the extremes, which is itself worth flagging: a 1–7 scale that is effectively a 2–6 scale in practice compresses the range available to downstream models.

Reliability across conditions, on the fourteen-story sensitivity subset (560 segments), is high for wording and temperature and measurably lower for narrative position:

Two findings stand out. First, the model is essentially deterministic at temperature 0 and highly stable across resamples at temperature 0.8 (two independent samples agree at ρ = 0.998, 99.5% exact agreement) — instability is not the concern here. Second, stating a narrative position shifts ratings upward by about 0.4 points on average, whether or not the stated position is true. The shuffled-position condition — where the stated position is drawn from a different segment of the same story — moves ratings by nearly as much as the true-position condition (slope of the rating shift on stated position: 0.46 for shuffled vs. 0.32 for true; correlation of the shift with (stated − true) position: r = 0.23) and is itself a large, systematic effect (mean shift +0.41, only 59% exact agreement with the no-position main condition). This is a direct, quantitative confirmation of Matlin et al.'s (2025) finding that language models lean on position as a prior rather than reading the text: telling the model "you are near the end" raises its suspense rating by about the same amount whether or not that is where the passage actually sits. The contextual-recap condition shows the largest systematic shift of all (+0.81, only 32% exact agreement) — summarising the story so far measurably raises the rating, plausibly because a recap concentrates threat and stakes language that the bare excerpt dilutes.

The model's free-text "cue" field, when stripped of stopwords, most often names concrete narrative content — strange, man, danger-adjacent and death-adjacent vocabulary recur — rather than generic hedges, which is mild evidence the ratings are keyed to textual content rather than boilerplate.

Agreement between the LLM rating and the independent composite lexical proxy is positive in every one of the seven genres (mean within-story ρ 0.31–0.44, 100% of stories positive in every genre; Table on /features), which is the strongest convergent-validity evidence available in the absence of pooled human data: two measures built from unrelated machinery (an instructed generative model vs. hand-written word lists) agree on the shape of suspense inside a story, reliably, across every genre in the corpus.

Human ratings. No ratings have been pooled at the time of writing; RQ4 is therefore answered here only via LLM-sensitivity and LLM-vs-proxy convergence. The instrument, quality-control rule and planned analysis are specified in §5.1 and live at /annotate.

11.2 RQ1 — predicting suspense from text

Held out by story (5-fold), the best model — a ridge-regularised bag-of-words regression — recovers the LLM suspense rating at ρ = 0.68 (R² = 0.42); hand-built features under gradient boosting reach ρ = 0.59 (R² = 0.36); ridge on hand features reaches ρ = 0.60; and, notably, ridge regression on raw MiniLM sentence embeddings performs at chance (ρ = −0.02, R² = −0.21) — the 384-dimensional embedding alone, without the hand features, carries no usable signal for this target under a linear probe, and appending it to the hand features in a boosting model does not improve on the hand features alone (ρ = 0.58 vs. 0.59). Grouping by author instead of story costs a small, consistent amount of performance across every model (bag-of-words ρ 0.68 → 0.67; hand-feature boosting 0.59 → 0.58), which is the expected signature of a model partly keying on an author's register rather than on universal suspense cues, but the drop is small enough that the models are not simply memorising authors.

The within-story Spearman ρ — does the model recover each story's own rise and fall, not just its average level? — is lower than the pooled ρ for every model (bag-of-words 0.56 vs. 0.68 pooled), confirming that predicting where the tension is inside a known story is harder than predicting which stories are tenser on average.

A per-family ablation isolates which feature family carries the signal: semantic (lexicon) features alone reach R² = 0.28 (ρ = 0.52), lexical features R² = 0.14 (ρ = 0.40), narrative (spaCy) features R² = 0.11 (ρ = 0.35), and model-based features (surprisal alone, no embeddings) R² = 0.02 (ρ = 0.20) — surprisal barely helps a linear probe on its own, consistent with the near-zero-to-negative agreement between surprisal and the LLM rating in Table (§ measure agreement, /features).

Across three interpretability methods (permutation importance, mean |SHAP|, standardized ridge coefficients) the same handful of features dominate: threat vocabulary (sem_threat) is the single strongest predictor by a wide margin under permutation importance and SHAP; lexical arousal (sem_arousal_lex) and lexical diversity (lex_ttr, with a negative ridge coefficient — more repetitive, less varied vocabulary predicts higher rated suspense, consistent with short, urgent, repeated phrasing under threat) are next; sentence length, first-person pronoun rate, temporal-term rate and finite-verb rate round out the top ten. These are associations the model uses to reproduce the LLM rater, not causal claims about reader experience (§ Reviewer 2).

11.3 RQ2 — how early can future suspense be predicted?

Story-level, grouped 5-fold: seeing only the first 10% of a story predicts the mean rating of the rest at ρ = 0.32 (ridge) to 0.45 (boosting), rising to ρ ≈ 0.70–0.75 by 30–50% of the way through, and falling again by 70% (ρ = 0.42–0.57) as there is little "rest of the story" left to predict and what remains is dominated by the ending. A model using only the early segments' own rating level ("early level only," with no text features at all) is the outright best model at four of the five cutoffs tested — 10% (ρ = 0.50 vs. 0.45 for the best full-feature model), 20% (0.65 vs. 0.60), 50% (0.75 vs. 0.75) and 70% (0.69 vs. 0.57) — and is narrowly edged out only at 30%, where ridge on the full feature set reaches ρ = 0.71 against the early-level baseline's 0.69. In other words, once a story's early tension level is known, its textual features add almost no further information about how the rest of the story will feel: a single number (how tense the opening already is) predicts the story's remaining trajectory about as well as fifty-four hand-built features do. Foreshadowing, if it is present in these texts, is not showing up as a detectable predictive signal beyond "this story has started tense" or "this story has started calm." This is a genuine negative result and is reported as such rather than reframed.

11.4 Robustness to segmentation

Re-binning each of six segmentation schemes (bins20, bins40, paragraph, window200, window400) to a common 20-bin grid and correlating story by story, curves agree strongly across every pair of schemes for the composite proxy, the threat proxy and surprisal: mean Spearman ρ ranges from 0.87 (paragraph vs. window400, the two most different granularities) to 0.99 (bins20 vs. bins40, the closest pair), and at least 99% of stories exceed ρ = 0.5 for every pair. Fixed-size sliding windows and equal-progress bins — conceptually different segmentations, one indexed by word count, the other by narrative fraction — recover essentially the same within-story shape. This robustness check itself caught a real bug during development: an earlier version of the sliding-window segmenter computed each window's narrative position by accumulating word counts as if windows did not overlap, which silently clamped most of a story's later windows to position 1.0 and produced near-zero cross-scheme correlations (ρ ≈ 0.15–0.20) between window-based and bin-based curves. The regression test added after the fix (test_window_positions_track_actual_offset_not_cumulative_count) checks a window's position against its true word-index midpoint rather than only checking it falls in [0, 1], which the original test suite did not catch.

12. Limitations

The construct. Every machine measure here is a text measure, and suspense is a reader response. The LLM rater is one 8B open-weight model whose judgements are validated against prompt, temperature, position and (on a subset) a second model — not against readers, because the human study has not yet accrued raters. The lexicon proxies are small hand lists that measure vocabulary, are blind to irony and free indirect style, and drift across a century of usage. GPT-2 surprisal confounds narrative surprise with rare names and archaic diction. Convergence among these measures is evidence that they track something stable in the text; it is not evidence that the something is what a reader feels.

The corpus. Ninety-two hand-chosen Anglophone stories from 1820–1928, weighted toward canonical authors, with one hand-assigned genre label per story that is confounded with author, era and length. Genre effects are therefore descriptive; the author-grouped cross-validation and the author-level curves are the honest version of the same question. Nothing here transfers to contemporary, non-English or non-canonical fiction without a new corpus, which the manifest format makes straightforward.

Pre-trained models and public-domain text. MiniLM, GPT-2 and the annotator model were all trained on web text that contains these stories. The bag-of-words baseline cannot "remember" an ending; the embedding and LLM measures conceivably can. We report both and treat the gap between them as an upper bound on this leakage.

Segmentation and smoothing. Forty bins impose a resolution: a 1,200-word sketch is rated in 30-word slices, a novella in 1,000-word slices. Cross-scheme correlations (§11.4) show that curve shape is stable across six segmentation schemes (mean ρ 0.87–0.99); the explorer exposes the unsmoothed series regardless. Clustering does clear a shuffled null at every k tested (§9), but the silhouettes that clear it are modest (0.08–0.11), so we report that structure exists without calling it a small number of sharp archetypes.

Statistics. With 92 stories, group comparisons have wide intervals and many tests are reported; all are treated as descriptive, with exact p-values, and no conclusion rests on a single test near the conventional threshold. Ordinal ratings are analysed with rank statistics; the mixed-effects analysis of human ratings is specified but not yet run.

The generative experiment. Twenty stories of ~1,000 words from one model, rated by the same model family, is a pilot. Agreement with a requested shape says the model can follow an instruction about tension; it does not say the stories are suspenseful to readers, and the shared vocabulary between generator and rater could inflate agreement — the lexical proxy is the independent check.

13. Conclusion

Suspense can be measured from text, but only partially, and the partiality is informative. Six independent operationalizations agree on enough — a positive within-story correlation between an instructed LLM rating and a transparent lexical proxy, in every genre in a 92-story corpus — to say that something stable about anticipatory tension is present in these texts and recoverable without asking a human. They disagree on enough — near-zero agreement between surprisal and every other measure, a genre effect in two operationalizations and not the other four, embeddings that carry no linear signal at all — to say that "suspense" as currently operationalized is not one thing, and a paper (or a product) that reports a single suspense number without naming its operationalization is reporting less than it appears to.

Two results feel most durable to us. First, the position-as-prior finding (§11.1): an LLM rater's suspense judgement moves with a stated narrative position almost as much as with a true one, which is a precise, reproducible instance of the general worry that language models track superficial correlates of narrative structure rather than reading dynamically (Matlin et al., 2025) — and it is directly actionable, since any pipeline that feeds a model "this is paragraph N of M" should expect that fact alone to move the answer. Second, the generative experiment's default-to-climax finding (§10.1): asked for five different suspense shapes, a capable instruction-following model produces one shape most of the time, closely resembling the conventional dramatic arc that dominates its training distribution. Both results point the same direction — current language models have a strong prior about what a suspenseful story looks like, and that prior is more visible in their annotation behaviour and their generation behaviour than in any single held-out accuracy number.

The measurement instrument itself — the corpus manifest, the segmentation schemes, the feature extractors, the provider-neutral LLM adapters with their content-addressed cache, the leakage-safe grouped cross-validation, the curve-shape null-model comparison, and the browser annotation app — is built to outlast this particular finding set. The most valuable next step is not a new proxy but the one piece still missing: pooled human ratings, which alone can adjudicate which of the six machine measures, if any, tracks what a reader actually feels.

Reviewer 2: attacks and responses

We wrote the hostile review ourselves before writing the results. Each attack is followed by what the study does about it and what it cannot do.

Construct validity — "you are measuring vocabulary, not suspense." True of every lexicon proxy by construction, and partly true of the LLM rater, which reads the same vocabulary. Responses: (i) six operationalizations are reported side by side with their within-story and story-level agreement, and the paper's claims are restricted to where they converge; (ii) the LLM is asked for suspense and intensity separately, and the two are compared, so that "eventful" and "anticipatory" are not conflated; (iii) the surprisal measure is deliberately a different kind of construct (information-theoretic) and is expected to disagree; (iv) the human study exists precisely because none of the above is ground truth, and every human-free number on the site is labelled as such. What we cannot do: claim that any curve reflects reader experience until human data are pooled.

LLM annotation bias. Language models (Matlin et al., 2025) distinguish suspenseful from non-suspenseful text yet track within-text dynamics poorly, prefer positions, and drift with prompts. Responses: the sensitivity battery quantifies prompt (plain / defined / contextual), temperature (0 vs 0.8, two samples), stated-position (true / shuffled) and model effects on a fixed stratified subset; the shuffled-position condition is a direct test of position priors; ratings are cached and reproducible; the rating distribution and self-reported confidence are published. The main rater is an 8B open-weight model, not a frontier model — chosen because it is self-hosted, cheap enough to rate every segment under every condition, and free of provider drift; the adapters are provider-neutral so the battery can be re-run with a stronger model. What we cannot do: separate "the model reads suspense as humans do" from "the model and the lexicon share a vocabulary".

Corpus bias. Ninety-two hand-picked Anglophone stories from 1820–1928, weighted toward canonical authors (Poe ×9, Wells ×8, Doyle ×8). Responses: the manifest is public and editable; genre, era, length and author analyses are all reported with bootstrap intervals; author-grouped cross-validation tests whether models generalise past the six over-represented authors. What we cannot do: generalise to contemporary, non-English or non-canonical fiction.

Genre confounding. Genre is one hand label per story, confounded with author (all detective stories are Doyle, Chesterton, Poe or Futrelle-adjacent), era and length. Responses: genre effects are tested with a permutation test on curve dispersion and with descriptor tests; author and era are reported as competing groupings; clustering is unsupervised and tested against a null so that genre is not baked in. What we cannot do: identify a genre effect independent of author with ~13 stories per genre.

Copyright. Only US-public-domain texts (Gutenberg) are ingested; every text records its ebook number, source URL and licence; the site links each story to its source; nothing copyrighted is stored, served or excerpted. Generated stories are the model's output under our prompts and are labelled as such. The NRC-VAD lexicon is optional, user-supplied and never redistributed.

Leakage. Every cross-validation split groups by story (a unit test asserts the property), with a stricter author-grouped variant; RQ2 uses only features from the first X% of a story; the LLM's contextual prompt sees only preceding text; surprisal context is preceding text only; the generated corpus never enters the human-corpus models. Residual risk: the MiniLM and GPT-2 weights were trained on the open web, which contains these public-domain texts; a pre-trained model may "know" a famous ending. We report the bag-of-words baseline, which cannot, alongside the embedding models.

Subjective labels. Suspense is a reader response, and a single rater's 1–7 scale is ordinal and noisy. Responses: Spearman rather than Pearson throughout; within-story correlations that remove level; ICC(2,1) and test–retest for human raters; attention checks; the annotation instrument records confidence and reading time so that low-effort ratings can be excluded.

Causal over-interpretation. Permutation importance, SHAP and ridge coefficients describe which features the model uses to reproduce the rater, not what makes readers tense. The site and paper say so at every importance table. The generated-story experiment is the only causal manipulation in the study (the instruction is randomised over premises), and its causal claim is limited to "instruction → measured curve", not to readers.

Smoothing artefacts. Following Swafford's critique of Fourier-based plot arcs, curves are re-binned locally and smoothed with a σ = 1-bin Gaussian (σ = 0 and 2 are available in the explorer); clustering is compared with a within-story shuffled null; all group curves are averages of z-scored curves so no single long story dominates.

Multiple comparisons. The atlas reports many tests (three groupings × several descriptors × several measures). We treat them as descriptive, report exact p-values, and make no claim that depends on a single test at p ≈ 0.05.

References

• Boyd, R. L., Blackburn, K. G., & Pennebaker, J. W. (2020). The narrative arc: Revealing core narrative structures through text analysis. Science Advances, 6(32).
• Brewer, W. F., & Lichtenstein, E. H. (1982). Stories are to entertain: A structural-affect theory of stories. Journal of Pragmatics, 6, 473–486.
• Cheong, Y.-G., & Young, R. M. (2015). Suspenser: A story generation system for suspense. IEEE Trans. Computational Intelligence and AI in Games, 7(1).
• Delatorre, P., León, C., Salguero, A., Palomo-Duarte, M., & Gervás, P. (2018). Confronting a paradox: A new perspective of the impact of uncertainty in suspense. Frontiers in Psychology, 9:1392.
• Doust, R., & Piwek, P. (2017). A model of suspense for narrative generation. Proc. INLG 2017.
• Ely, J., Frankel, A., & Kamenica, E. (2015). Suspense and surprise. Journal of Political Economy, 123(1), 215–260.
• Gerrig, R. J. (1993). Experiencing Narrative Worlds. Yale University Press.
• Jockers, M. L. (2015). Revealing sentiment and plot arcs with the syuzhet package. Blog post, 2 Feb 2015; Swafford, A. (2015). Problems with the syuzhet package. Blog post, 2 Mar 2015.
• Lehne, M., & Koelsch, S. (2015). Toward a general psychological model of tension and suspense. Frontiers in Psychology, 6:79.
• Matlin, G., Zhang, D., Barroso Loza, R., Popescu, D. M., Isbell, J., Chakraborty, C., & Riedl, M. (2025). Do language models agree with human perceptions of suspense in stories? COLM 2025 (arXiv:2508.15794).
• O'Neill, B., & Riedl, M. (2014). Dramatis: A computational model of suspense. Proc. AAAI 2014.
• Onea, E., Köppe, T., Cortez Espinoza, M., Bentz, C., & Simeonova, D. (2026). The linguistic basis of narrative suspense. Language and Cognition.
• Reagan, A. J., Mitchell, L., Kiley, D., Danforth, C. M., & Dodds, P. S. (2016). The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science, 5:31.
• Smuts, A. (2008). The desire-frustration theory of suspense. Journal of Aesthetics and Art Criticism, 66(3).
• Sui, P., Zhu, Y., Cheng, T., West, P., So, R. J., Long, H., & Holtzman, A. (2026). Spoiler alert: Narrative forecasting as a metric for tension in LLM storytelling. arXiv:2604.09854.
• Tian, Y., et al. (2024). Are large language models capable of generating human-level narratives? Proc. EMNLP 2024.
• Toubia, O., Berger, J., & Eliashberg, J. (2021). How quantifying the shape of stories predicts their success. PNAS, 118(26).
• Wilmot, D., & Keller, F. (2020). Modelling suspense in short stories as uncertainty reduction over neural representation. Proc. ACL 2020, 1763–1788.
• Xie, K., & Riedl, M. (2024). Creating suspenseful stories: Iterative planning with large language models. Proc. EACL 2024.
• Zillmann, D. (1996). The psychology of suspense in dramatic exposition. In Vorderer, Wulff & Friedrichsen (eds.), Suspense: Conceptualizations, Theoretical Analyses, and Empirical Explorations. Erlbaum.