Utterance Rules, Processing Order & Review Tuning Dials

IMI Library — Reference · 2026-05-31 · Brian Borg · v1.0

What this document is. A recording becomes a list of utterance rows in two layers. First, the pipeline runs a fixed 7-rule turn-based consolidation (Step 4.5) that turns raw transcript turns into clean rows. Second, the review page lets a reviewer re-derive those rows live with four tuning dials, without reprocessing the audio.

This doc explains the rules, the order each is applied during processing, and exactly what each dial does to the displayed utterances. A short Known issues section at the end records what is currently broken (2026-05-31) so the descriptions aren't mistaken for current behavior.

1. The two layers

Everything below lives in one engine, written twice and kept in lockstep:

Three columns on the media row carry the data between the two layers:

ColumnWhat it holdsMutable?
consolidation_inputsThe immutable raw inputs the rules run on (raw transcript utterances, speaker map, music candidates, topic shifts)No — fixed at processing time
pipeline_originalThe neutral consolidated rows — the pipeline's own output at dial position zeroNo — the baseline snapshot
utterancesWhat the Review UI displays. Equals pipeline_original until a dial is moved, then holds the re-derived rowsYes — re-derived by the dials

2. Processing order — where each rule gets its inputs

The rules all execute at one point (Step 4.5, consolidation), but each depends on a signal produced by an earlier step. If an earlier step is missing or fails, the rule that depends on it has nothing to act on.

01
Transcode — any format → canonical MP4
02
Extract audio — MP4 → mono 16 kHz MP3
03
Transcribe (AssemblyAI) — raw utterances, speaker labels, topic detection → the rules' raw input
2.5
YAMNet segmentation — music vs speech regions → feeds Rule 1
3.5
Voiceprint match — speaker map + per-utterance identity (who is Swami) → feeds Rules 3, 4, 7 and tiny-absorb
04
Enrichment (Claude) — tags, topic-shift timestamps → feeds Rule 6
4.5
Turn-based consolidation the 7 rules run here → writes utterances, pipeline_original, consolidation_inputs
push
Write to database — the three columns land on the media row
review
Tuning dials (browser) — re-run the same engine live on consolidation_inputs
06
Ingest — human-approved record published
Key point

The order of the steps is fixed, but the rules themselves are applied in a single pass at Step 4.5 in the order below: regions are carved first (Rules 1–2), then the speech timeline is walked turn by turn (Rules 3–7), then a cleanup pass absorbs tiny mis-diarized fragments.

3. The seven rules

Applied in this order, in consolidate_turn_based(). "Acts on" is the signal the rule reads; "Needs" is the upstream step that must have produced it.

Rule 1 Songs come first

Each music region becomes a single row labeled "Song." Any transcribed speech that falls inside a music region is removed — lyrics don't become visitor turns.

Acts on: YAMNet music regions  ·  Needs: Step 2.5  ·  one row per region, never zero

Rule 2 Overlapping speech is one row

A stretch where multiple people talk over each other collapses to a single row labeled "Multiple speakers" rather than a tangle of half-caught turns.

Acts on: overlap regions  ·  Needs: overlap detection (deferred; inert when none present)

Rule 3 A visitor's first speech is one row

When someone who isn't Swami speaks, their first stretch of speech becomes one row — the only row they get for that turn.

Acts on: effective speaker identity  ·  Needs: Step 3.5 voiceprint

Rule 4 Swami's response is one row

Swami's reply is a single row. It starts when he begins and ends when a different speaker takes the seat, a song begins, or the recording ends. Silence inside the answer stays part of his row.

Acts on: effective speaker identity + music starts  ·  Needs: Step 3.5 voiceprint

Rule 5 Back-and-forth is absorbed

When the same visitor makes a short interjection during Swami's answer (under 15 seconds), it's folded into Swami's row in time only — the interjection's text is dropped, and no extra row appears. A longer re-engagement is treated as a new turn instead (Rule 7).

Acts on: interjection duration vs the 15s cap  ·  Tuned by: the "Other speeches" dial

Rule 6 Long Swami stretches may split

If Swami speaks for 20 minutes or more and Claude detected topic shifts inside that stretch, the single row is split into one row per topic segment so a long talk is navigable.

Acts on: stretch length + topic-shift timestamps  ·  Needs: Step 4 Claude tags  ·  Tuned by: the "Swami speeches" dial

Rule 7 A new visitor restarts the cycle

When a different non-Swami speaker takes over, the cycle restarts — Rules 3 through 6 repeat for the new exchange.

Acts on: a change in effective speaker  ·  Needs: Step 3.5 voiceprint

Cleanup pass — tiny-row absorption

After the seven rules, a final pass removes visitor rows shorter than 10 seconds whose speaker the voiceprint never identified. These are usually a 1–2 second sliver of Swami's own voice that AssemblyAI mis-labeled as a new speaker. The sliver is absorbed back into the surrounding Swami rows so one continuous answer doesn't fracture into three. Identified visitors — even short ones — are always kept.

4. The review tuning dials

Four dials, each from −5 to +5, with 0 = the pipeline's own output, verbatim. Moving a dial re-runs the engine on the stored consolidation_inputs with one constant replaced by the dial value. The dials only change what is displayed; they never touch the immutable inputs, so any setting is reversible by returning to neutral. Source: dialsToParams() in review-ui/js/tuning.js.

Dial (screen label)Acts onToward +Toward −
Music
more speech ↔ more song
Rule 1 — which audio is carved out as song Carves more audio as song Carves less; shaky "music" reverts to speech
Swami vocal match
looser ↔ stricter
Voiceprint match threshold (0.80 at neutral) Stricter, up to 0.92 — fewer turns accepted as Swami Looser, down to 0.65 — more turns accepted as Swami
Swami speeches Rule 6 — splitting long Swami stretches Splits less → fewer Swami rows (never splits at +5) Splits readily → more Swami rows
Other speeches Rule 5 + tiny-absorb thresholds Absorbs more interjections → fewer visitor rows Absorbs nothing → every interjection survives

Dial-by-dial detail

Music Rule 1

Decides which YAMNet candidates are carved out as song. Neutral passes every detected music candidate and leaves the speech-reject density at its baseline.

musicConfCut = music < 0 ? (0.62 − music·0.066) : 0   |   musicDensityDelta = music · 0.4
+ (song)Raises the speech-reject density so songs whose lyrics were transcribed still count as music. More audio shows as "Song." − (speech)Raises the confidence bar so the shakiest "music" (often Swami's talk mislabeled) drops back to speech.

Swami vocal match voiceprint

Re-thresholds the stored per-utterance voiceprint similarity. An utterance is treated as Swami only when its match score clears the threshold.

vpThreshold = vp ≥ 0 ? (0.80 + vp·0.024) : (0.80 + vp·0.03)   →   +5 = 0.92, −5 = 0.65
+ (strict)Fewer borderline utterances are accepted as Swami; they fall back to their raw label. − (loose)More borderline utterances are accepted as Swami.

Swami speeches Rule 6

Controls how readily a long Swami stretch splits into multiple rows at topic boundaries — the split gate and the topic-shift confidence required to act.

swamiStretchMs = (20 + swami·4) min  (∞ at +5)   |   swamiConfidenceCut = 0.5 + swami·0.1
+Bigger gate, higher confidence bar → splits less → fewer Swami rows. At +5 a stretch never splits. Smaller gate, lower bar → splits at more topic boundaries → more Swami rows.

Other speeches Rule 5 + tiny-absorb

Controls how aggressively short visitor interjections and unidentified slivers are absorbed versus kept as their own rows.

interjectionCapMs = 15000 + other·9000  (0–60s)   |   tinyAbsorbMs = 10000 + other·4000  (0–30s)
+Larger caps → more visitor interjections absorbed into Swami → fewer visitor rows. Caps shrink toward 0 → nothing absorbed → every visitor interjection survives as its own row.
Caption mismatch to fix

The on-screen captions for Swami speeches and Other speeches currently read "fewer entries" on the left (−) and "more entries" on the right (+) — the opposite of what the engine does (+ yields fewer rows, − yields more; the module's own header says "strict (+5) = fewer rows"). The Music and Swami-vocal-match captions match the engine. The two inverted captions are on the fix list.

5. How re-derivation works

When a dial moves, the browser calls deriveRows(consolidation_inputs, pipeline_original, dials):

Because the inputs are immutable, returning every dial to 0 always restores the pipeline's original rows.

6. Known issues (2026-05-31)

The behavior above is the intended design. As of this date the following are broken and on the fix list — noted here so the descriptions aren't read as current reality. Full analysis: imi-library-wiki/wiki/lessons/2026-05-31-tuning-view-missing-columns-zeroes-utterances.md.