Generate Speech of Your Favourite Narrator Locally With F5-TTS-MLX
Recently a post about generating audiobooks started trending on hn, and some people in the comments wished they could clone their voice and narrate text without sending it off their machine. Itโs never been easier!
For this example, we only need a mac, uv (modern python package manager), ffmpeg for audio processing and optionally chatgpt for transcribing your voice (but you can do it manually or use mlx-whisper, for example). We will be using F5-TTS-MLX, an open-source speech synthesis implementation of F5 TTS model in Apple Silicon array framework
The original implementation has ๐ฌ๐ง๐บ๐ธ๐ซ๐ฎ๐ซ๐ท๐ฎ๐ณ๐ฎ๐น๐ฏ๐ต๐จ๐ณ๐ท๐บ๐ช๐ธ language support. The MLX implementation also includes a duration predictor specifically for English, which simplifies the creation of natural-sounding audio, but itโs not available in other languages. Personally Iโm interested in German, and someone fine-tuned F5 for it, so additionally training a duration predictor can help to translate audio to it, similar to what Lex Fridman did for the interview with Javier Milei, but open source.
