Built a from-scratch TTS pipeline (FastSpeech2 + HiFi-GAN) on LJSpeech [P]

SummaryAI generated

Data scientist Ashwin3290 used the LJSpeech-1.1 dataset to build a text-to-speech (TTS) pipeline from scratch, incorporating forced alignment, FastSpeech2 acoustic models, and HiFi-GAN encoder. The system inserts natural pauses through silent markers in CTC forced alignment and filters based on duration and energy gating. Under the Whisper base.en evaluation, the test set WER of PostNet is 8.8%, and the character error rate (CER) is 5.1%; based on real audio benchmarks, the WER is 4.9%, with most errors due to differences in digital transcription rather than synthesis failures. In the initial phase of HiFi-GAN fine-tuning, the output was harsh due to initialization issues; later, general checkpoint weights were used to fix this, stabilizing the loss within the range of 1.6-2.2. The related code is hosted on GitHub, and model weights and sample data are located on Hugging Face.

Related eventsRELATED EVENTS

2026-09-09 05:29OpenAI claims huge maths breakthrough on a famed ‘Millennium Problem’
2026-09-09 06:11OpenAI claims to have solved the Navier-Stokes equation problem, but its originality is questioned by the academic community
2026-09-08 22:00Google DeepMind Maps 9 Billion Possible DNA Variants

Key entitiesKEY ENTITIES

FastSpeech2HiFi-GANLJSpeechWhisper

Coverage · reports per dayLANGUAGE SPLIT

Entity relations

SignalsSIGNALS

Keyword heat

LJSpeech1
FastSpeech21
HiFi-GAN1
Whisper1

All reports (1)SOURCES

↗

R r/MachineLearning en 2026-09-07 14:51

Data scientist Ashwin3290 built a TTS pipeline based on the LJSpeech-1.1 dataset, including forced alignment, FastSpeech2 acoustic models, independent PostNet, and fine-tuned HiFi-GAN encoder. The code is hosted on GitHub, and the weights and samples are located on Hugging Face. Under the Whisper base.en scoring, the WER for the PostNet test set is 8.8%, and the CER is 5.1%; the true audio benchmark WER is 4.9%, with the main error originating from numerical transcription differences rather than synthesis failures. The system inserts natural pauses through silent markers in CTC forced alignment and filters based on duration and energy gating. During HiFi-GAN fine-tuning, the output became harsh due to random initialization of the discriminator; this was later corrected using general checkpoint weights, with the loss stabilizing between 1.6 and 2.2. The model can handle unseen words, but it has difficulties with unfamiliar phoneme sequences (such as “Hig…”).