Align

Accurate word timestamps for any transcript.

Word-timestamp refinement for Apple's SpeechAnalyzer pipeline.

SDKs, install and examples: https://github.com/Desert-Ant-Labs/desert-ant-core/blob/main/docs/models/align.md

Corrects the word-level timings that Apple's SpeechTranscriber and SpeechAnalyzer return, without replacing them. Align observes the same audio the analyzer already receives, runs a small Core ML cascade on the CPU and Neural Engine, and returns the familiar result surface with tightened audioTimeRange values. The models are tiny (700KB compiled Core ML) and refine a typical result in a few milliseconds on device.

Apple: "world" 2.61-3.04s ➜ Align: "world" 2.57-2.98s

Try it


Platforms	iOS, macOS, tvOS, visionOS
Weights	v1.0.0

Install

Swift (requirements)

.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "3.1.0")

Then add the Align product to your target.

Files

File	Format	Size	Contents
`align_coarse.mlmodelc`	Compiled Core ML (FP32)	530KB	Coarse stage: searches a 241-frame (2.4 s) context, fixed batch-16
`align_fine.mlmodelc`	Compiled Core ML (FP16)	300KB	Fine stage: searches an 81-frame (0.8 s) crop centered on the coarse prediction
`mel_filters.bin`	Float32 filter bank	40KB	Log-mel filter bank the runtime frontend needs
`calibrator.bin`	Gradient-boosted trees	70KB	Correction calibrator over coarse/fine uncertainty features
`refiner_config.json`	JSON	tiny	Frontend, lexical, and language config the runtime needs
`coarse.pt`	PyTorch checkpoint	500KB	Coarse-stage weights (for retraining / other runtimes)
`fine.pt`	PyTorch checkpoint	500KB	Fine-stage weights (for retraining / other runtimes)

The compiled .mlmodelc stages, mel_filters.bin, calibrator.bin, and refiner_config.json are exactly what the Swift SDK bundles. The .pt checkpoints are the training-run weights.

Architecture

A two-stage coarse-to-fine cascade over a log-mel spectrogram, refining one boundary at a time:

Frontend: an Accelerate/vDSP log-mel spectrogram of the same audio Apple transcribes.
Coarse stage: a compact convolutional model searches a 2.4 s context around Apple's proposed boundary and predicts a distribution over frames.
Fine stage: a second model re-searches a 0.8 s crop recentered on the coarse prediction for a tighter estimate.
Lexical conditioning: UTF-8 byte features of the neighboring words plus a language id let a single model cover all nine languages.
Calibrator: a small gradient-boosted-tree policy maps coarse/fine uncertainty features to a final correction, fit only on the validation split to reduce large regressions.
Structural fallback: boundaries whose correction would be invalid, hit the search-window edge, or lack streaming context keep Apple's original timestamp.

Each stage runs fixed batch-16 on CPU + Neural Engine. Total parameters are 121,141 per stage.

Inputs and outputs

Input: mono audio plus Apple's recognized words with their proposed start/end times.
Output: the same words with corrected start/end times, or Apple's original time when a correction is not structurally safe.

Accuracy

Measured on v1.0.0 over group-held-out recordings, against forced-alignment references built with Qwen3-ForcedAligner (Apache-2.0) averaged with a MIT-licensed second aligner. No speaker in an evaluation split appears in training.

All nine languages

Condition	Apple raw error	Align error	Reduction
Clean	124.2ms	43.9ms	65%
Noisy	88.3ms	33.4ms	62%

Macro-averaged over the nine languages, so a language with more test data cannot carry the figure on its own.

Public benchmark, English

A 500-clip sample of each official LibriSpeech test-clean and test-other split. 67 training speakers against 67 evaluation speakers, zero overlap.

Engine	Split	Raw	Refined	Reduction	Within 50ms
Apple SpeechAnalyzer	test-clean	106.4ms	20.2ms	81%	37% to 95%
Apple SpeechAnalyzer	test-other	111.6ms	24.8ms	78%	35% to 92%

For editing work the p90 matters more than the mean: on test-clean it falls from 230.7ms to 33.0ms, about one frame of 30fps video. Large errors are what a viewer notices when a caption slips or a clip cuts mid-word.

Against hand-corrected boundaries

258 word boundaries across 10 recordings, corrected by hand against the waveform rather than by another aligner. The only figure here not measured against machine references.

System	Error	Within 50ms
Raw Whisper	100.8ms	43%
WhisperX	53.5ms	67%
Align	45.0ms	76%

Core ML parity

The shipped Core ML stages are checked against the PyTorch weights on real audio crops, comparing the decoded correction rather than raw logits: fine 0.27ms mean and 1.98ms p99, coarse 0.0001ms mean and 0.0003ms p99. The coarse stage ships FP32 because at FP16 its p99 reached 3.6ms, above the 3ms acceptance threshold: the softmax-expectation decode amplifies FP16 rounding when the predicted distribution is broad.

Languages

English, Spanish, French, Italian, Portuguese, German, Japanese, Korean, and Chinese. A locale outside this set is passed through unchanged.

Limitations

References are machine forced-alignment estimates, not human annotations, so the figures show a large, consistent reduction of Apple's timing error rather than sample-accurate ground truth.
A learned correction is not guaranteed to improve every boundary; the structural fallback keeps Apple's timestamp when a correction looks unsafe but cannot catch every plausible-looking error.
Japanese, Korean, and Chinese were the weakest languages by a wide margin before v1.0.0. A reference-building defect had emptied nearly all of their training data; v1.0.0 rebuilds it and those three now improve their proposals by 33%, 55%, and 51%, where previously they made timings worse than the input.
Spoken numbers are the weakest remaining case. On a small sample, refinement moved digit boundaries further from the reference than leaving them alone, so treat them as unimproved until a larger sample settles it.

Built on

FLEURS (CC BY 4.0): multilingual training audio.
Qwen3-ForcedAligner-0.6B (Apache-2.0): primary word-boundary references for all nine languages.
OWSM-CTC v4 1B (CC BY 4.0): gross alignment-outlier check where validation agreement is stable.
Genuine Apple SpeechAnalyzer proposals collected on macOS 26.

See THIRD_PARTY_NOTICES.md. None of these systems are redistributed here.

License

Desert Ant Labs Source-Available License. Free for most apps, and a commercial license is required at scale. Full terms are at the link. Licensing: licensing@desertant.com.

See THIRD_PARTY_NOTICES.md.

Citation

@software{align_2026,
  title  = {Align: Word-timestamp refinement for Apple's SpeechAnalyzer pipeline},
  author = {Desert Ant Labs},
  year   = {2026},
  url    = {https://huggingface.co/desert-ant-labs/align},
}

Downloads last month: -; Downloads are not tracked for this model. How to track