Meeting and Conversation Intelligence — Technical reference

This reference owns the implementation contracts of BND-MEETING. Read the package guide for usage, configuration, dependencies and upgrades. Generic execution, isolation and admission remain governed by Platform Documentation.

Data model and interfaces

DataModelPackage: DM-MEETING

Owned entities: meeting, participant, media processing state, transcript version, utterance, speaker hypothesis, agenda, topic, meeting type, summary version.

InterfacePackage contributions: IF-MEETING-WORKSPACE.

Each business object and source reference remains tenant- and purpose-scoped. The package owns schema evolution and declared migration behavior; the platform enforces schema registration, authorization and versioned installation. Consumers reference this model rather than creating competing canonical copies.

Workflows

WorkflowPackage: WFP-MEETING-LIFECYCLE

• Meeting Intake
• Media Preparation
• Local ASR and Alignment
• Diarization and Speaker Resolution
• Transcript Quality Review
• Meeting Classification
• Agenda and Topic Structuring
• Domain Analysis Fan-Out
• Summary and Minutes
• Pre-Meeting Briefing
• Real-Time Assistance
• Longitudinal Meeting Update

These are named definitions within the workflow package, not separate invented workflow IDs. Workflow runs resolve package versions, actor and purpose, authorized inputs, retries, review gates and output schemas through the environment lock. Temporal provides durable execution and Kafka carries declared lifecycle/domain events.

Policies, evaluations and tools

POL-MEETING-PROCESSING, EVAL-ASR-DIARIZATION-MEETING, media, ASR, diarization, alignment, translation, and real-time tools.

Qualification covers the selected model/runtime profile, languages, access scope, failure handling and the package's domain-specific outcomes. Tool execution uses the platform gateway; package code cannot increase its own permissions or write directly to external systems.

Owned agents

M01 — Meeting Structure Agent

Agent ID: effority.meeting-structure

Purpose: Creates the canonical analytical structure of a meeting.

Triggers: Quality-approved transcript, transcript correction, calendar update, or participant correction.

Inputs: Transcript turns, meeting metadata, agenda, participants, authorized object candidates, and configured meeting taxonomy.

Outputs: Meeting type, nested topic segments, actual-versus-planned agenda, topic titles, topic summaries, entities, and multi-object assignments.

Tools: transcript.read, calendar.metadata_read, business-object.search, graph.read, and topic.write.

Review and limits: Ambiguous object assignments remain suggested or unresolved. It does not create domain findings.

Primary metrics: Boundary accuracy, meeting-type accuracy, topic coherence, object-assignment accuracy, and correction rate.

M04 — Meeting Summary and Minutes Agent

Agent ID: effority.meeting-summary

Purpose: Produces role-appropriate meeting summaries and minutes grounded in validated topics and artifacts.

Triggers: Completion of core meeting agents, human review update, or explicit regeneration.

Inputs: Topic structure, validated decisions, actions, risks, questions, object links, recipient permissions, and summary template.

Outputs: Executive, detailed, speaker-authorized, internal, or external minutes with citations, exclusions, and source-version manifest.

Tools: artifact.read_validated, evidence.resolve, template.read, recipient.policy_check, and document.render_draft.

Review and limits: External minutes remain drafts until approval. Restricted content is omitted rather than summarized indirectly.

Primary metrics: Coverage, citation faithfulness, omission rate, recipient-policy violations, and user edit rate.

M05 — Pre-Meeting Briefing Agent

Agent ID: effority.pre-meeting-briefing

Purpose: Prepares the user for an upcoming meeting with relevant history, open commitments, risks, objections, documents, questions, and role-specific priorities.

Triggers: Calendar schedule, user request, material context change, or configured preparation window.

Inputs: Upcoming meeting, participant and object scope, recent validated history, open actions, effective documents, role playbook, and user permissions.

Outputs: Briefing, suggested agenda, questions to clarify, evidence links, preparation checklist, and private recommendations.

Tools: calendar.read, graph.read, timeline.read, finding.read_validated, document.read, and recommendation.read_private.

Review and limits: The briefing never exposes content unavailable to the user or presents inferred personal traits.

Primary metrics: Source coverage, stale-context rate, briefing use, question usefulness, and permission violations.

M06 — Real-Time Assistance Agent

Agent ID: effority.realtime-assistance

Purpose: Produces private low-latency reminders and provisional meeting assistance during an explicitly authorized live session.

Triggers: Authorized live-session start, closed speech segment, agenda threshold, explicit user request, or approved safety event.

Inputs: Provisional transcript, temporary speaker labels, authorized agenda, open questions, relevant short context, and real-time policy.

Outputs: Provisional action marker, agenda reminder, clarification question, private sales or project suggestion, and preliminary summary.

Tools: realtime.transcript_read, agenda.read, finding.read_open, and allowlisted suggestion templates.

Review and limits: Outputs are visibly provisional. The agent does not display live ethics allegations except an authorized predefined E4 safety protocol. The final batch pipeline replaces provisional analysis.

Primary metrics: End-to-end latency, suggestion precision, dismissal rate, distraction rate, provisional-to-final agreement, and session-policy compliance.

M08 — Longitudinal Evolution Agent

Agent ID: effority.longitudinal-evolution

Purpose: Reconstructs how decisions, commitments, risks, objections, acceptances, opportunities, projects, and relationships change over time.

Triggers: New validated finding, review outcome, source correction, periodic snapshot, or user query.

Inputs: Temporal graph, validated artifacts, source versions, previous snapshots, object validity, and configured time window.

Outputs: Change event, recurrence, reopening, aging, fulfillment, nonfulfillment, contradiction, trend vector, and evidence-backed change narrative.

Tools: timeline.read, graph.read, finding.read_history, snapshot.read, and artifact.derive.

Review and limits: The agent calculates a trend only when the configured evidence threshold is met. It never collapses a person into a general score.

Primary metrics: Change-detection accuracy, temporal-link precision, contradiction coverage, stale-snapshot rate, and invalidation correctness.

Implementation profiles

AI Pipeline Orchestration

The meeting workflow is implemented as a durable state machine:

DISCOVERED
  → POLICY_CHECKED
  → INGESTED
  → MEDIA_PREPARED
  → SPEECH_SEGMENTED
  → TRANSCRIBED
  → DIARIZED
  → SPEAKERS_RESOLVED
  → QUALITY_CHECKED
  → TOPICS_SEGMENTED
  → CONTEXT_RETRIEVED
  → FINDINGS_EXTRACTED
  → FINDINGS_VALIDATED
  → PUBLISHED
  → SYNC_PENDING / COMPLETE

Side states: BLOCKED, QUARANTINED, NEEDS_REVIEW, DEGRADED,
RETRY_SCHEDULED, FAILED_PERMANENTLY, RETENTION_EXPIRED.

Every transition is idempotent; persists its input, output, model version, policy, and metrics; and emits an internal platform event. Temporal retries failed Activities and records terminal workflow failure with a diagnosis and operator action. Workers acquire expiring leases so another process can resume from the last checkpoint after a failure. The Kafka-compatible broker independently owns event-delivery retries and dead-letter queues; consumers use idempotency keys and never treat broker delivery as workflow state.

The derivation graph records which items depend on each word, turn, document, and finding. A human correction emits an invalidation event; the orchestrator recalculates only affected topics, citations, findings, summaries, and reports and marks prior exports as based on an earlier version.

Audio, VAD, and ASR Implementation

Acoustic Preparation

1. FFmpeg validates the container and codecs and extracts every channel without altering the original.
2. It generates a linear PCM derivative at 16 kHz and 16-bit. ASR receives a mono mix; diarization and identity also retain separate channels.
3. The service measures clipping, signal-to-noise ratio, silence, useful duration, effective frequency, and discontinuities.
4. Noise and echo reduction produces an additional derivative. The quality selector compares the original and enhanced versions to prevent filtering from destroying phonemes.
5. Files are divided into overlapping blocks at speech boundaries; timestamps are always expressed relative to the original.

Voice Activity Detection

VAD runs locally with Silero VAD on ONNX Runtime. It processes short windows, returns speech probability, and applies domain-specific thresholds. Post-processing adds padding, joins short pauses, and avoids cutting words. Telephony, videoconferencing, and physical rooms use independent profiles.

The result includes speech intervals, long silences, probability, profile, and version. Intervals guide ASR and diarization batching but never delete original audio.

Whisper Inference

The production backend runs Whisper through faster-whisper and CTranslate2. The installation remains compatible with OpenAI’s canonical runtime for reference tests.

• large-v3 is the accuracy profile for post-meeting processing.
• turbo is the low-latency and real-time profile.
• Quantized medium or small models run on edge or CPU nodes.
• GPU uses FP16; compatible hardware can use BF16.
• CPU uses INT8; the hybrid profile uses int8_float16 when language evaluation permits it.
• Batching adapts to free memory and segment duration.
• Decoding uses beam search, language detection, word timestamps, and temperature fallback for anomalous segments.
• Glossaries are injected as limited, versioned context and never as mandatory text.
• Repetition, hallucination over silence, anomalous compression ratios, and low average probability trigger reprocessing with conservative parameters.

Whisper does not perform diarization. The ASR result contains words, timings, log probabilities, language, language probability, temperature, compression ratio, no-speech probability, and exact model.

Selecting Between Source and Local Transcripts

When a platform provides subtitles or a transcript, the system calculates a source score from provenance, coverage, timestamps, identity, language, and sample comparison with audio. The selector runs one of these routes:

• retain the source as the primary transcript;
• run Whisper and select the local version;
• merge source timestamps or names with local text;
• send discrepancies for review.

No version is destroyed. Every report references the transcript_version_id used.

Diarization and Speaker-Identification Implementation

Diarization uses pyannote.audio with the local speaker-diarization-community-1 pipeline. Component telemetry is disabled in private installations, and weights are imported into the internal registry during packaging.

Implemented flow:

1. speaker-activity segmentation and overlap detection;
2. embedding extraction by speech window;
3. agglomerative clustering with optional min_speakers, max_speakers, or expected-count bounds;
4. smoothing of very short turns;
5. preservation of simultaneous-speech regions;
6. confidence calculation by turn and cluster;
7. alignment of diarized turns with ASR words.

Alignment assigns each word to the speaker with the greatest temporal overlap. In ties or genuine overlap, it retains multiple attribution or UNKNOWN instead of forcing an identity. A later optimizer minimizes unlikely speaker changes without merging distinct voices.

Identity resolution applies an ordered cascade:

1. dedicated audio channel or provider identifier;
2. participant list, organizer, and access metadata;
3. spoken introduction and explicit names detected in the text;
4. turn consistency with named questions and answers;
5. voice-embedding similarity only when the tenant enabled the restricted biometric module;
6. human confirmation.

Biometric embeddings are encrypted with a separate key, stored outside the text index, never cross tenants, have their own retention, and are never exposed to search. A human correction applies to one turn, an interval, the full meeting, or an authorized historical cluster according to the reviewer’s selection.

Transcript Quality and Confidence

Visible confidence is not the raw probability from one model. The quality-service combines:

• language probability and stability;
• token log probability;
• no-speech probability;
• compression ratio and repetition;
• VAD coverage;
• acoustic quality;
• agreement among versions;
• diarization confidence;
• density of out-of-vocabulary terms;
• historical human-correction volume for the profile.

A supervised calibrator, versioned by language and domain, converts these variables into HIGH, MEDIUM, LOW, and UNUSABLE bands. Thresholds are determined on the gold corpus. LOW excerpts are reprocessed; UNUSABLE excerpts are shown as unintelligible and do not feed sensitive findings without review.

Topic Segmentation and Multi-Object Classification

Topic segmentation combines three signals:

1. embedding changes across windows of turns;
2. changes in entities, objects, and vocabulary;
3. boundary classification by a local model.

A consolidation step merges compatible short segments, preserves explicit agenda changes, and supports nested topics. Every topic stores its start, end, suggested title, summary, entities, candidate objects, and confidence.

Assignment to an account, project, or opportunity uses:

• calendar and CRM identifiers;
• participants and active relationships;
• names, aliases, and codes;
• similarity to object descriptions;
• context from previous meetings;
• a multi-label classifier;
• exclusion rules based on validity or permissions.

The resolver retains multiple objects when appropriate. A low-margin decision between candidates remains pending confirmation instead of choosing arbitrarily.

Real-Time Implementation

The live module receives audio through WebRTC or streaming PCM and maintains two lanes:

• provisional: VAD, partial ASR, and temporary speaker labels with low latency;
• final: segment closure, stable timestamps, revised diarization, and incremental analysis.

Partial hypotheses are replaced when a segment closes but remain separate from published findings. Live suggestions use limited context, small models, and allowlisted actions. After the meeting, the accuracy-oriented batch pipeline runs and reconciles differences.

Longitudinal Analysis Implementation

Every decision, risk, commitment, objection, and acceptance is materialized as a temporal event. An incremental job builds snapshots by meeting, week, and reporting period.

The longitudinal engine:

• compares current state with the previous snapshot;
• detects reopening, repetition, and aging;
• links a promise to later fulfillment or nonfulfillment;
• preserves contradictions among sources;
• calculates a trend only when enough history exists;
• generates a change narrative with citations from both moments;
• invalidates the trend if identity, project, or base transcript changes.

Health indicators are calculated as vectors of explainable signals. The interface presents every component, time window, and piece of evidence; it never reduces a person to an aggregate score.

Meeting summaries use M04. Scheduled and distributable multi-domain reports are owned by Reports.

Integration boundaries

Only Overview and Audio & Transcript are owned here. Findings, Decisions & Actions, Context, Files & Versions and Activity are contributed by other packages.

Owned screen contracts: SCR-MTG-01, SCR-MTG-02.

See solution workflows for cross-package behavior. Screens, agents and workflows keep one canonical owner even when other bundles invoke or enrich them.