sidetone

Advanced

Chatterbox

Chatterbox is Resemble AI's open text-to-speech model, and the most natural voice you can run yourself: it can speak in a voice of your own from a short recording. The Chatterbox TTS Server wraps it in the OpenAI audio API, which is what sidetone speaks.

Chatterbox only speaks. For voicemail as text, pair it with Speaches: Chatterbox under Text to speech, Speaches under Transcription. The two settings are separate for exactly this.

1. Run the server

The server comes with compose files for NVIDIA, AMD and CPU. It needs a GPU to be quick; on a CPU a sentence takes a while, which is acceptable for prompts you make once, and nothing else in sidetone waits for it.

$ git clone https://github.com/devnen/Chatterbox-TTS-Server.git
$ cd Chatterbox-TTS-Server
# NVIDIA (RTX 20/30/40):
$ docker compose up -d --build
# no GPU:
$ docker compose -f docker-compose-cpu.yml up -d --build

The first start downloads the model, a couple of gigabytes, and the server then answers on port 8004 with a web page of its own at http://host:8004/, where you can try voices and settings by hand. Its README has the other compose files (RTX 50 series, ROCm) and the install without Docker.

2. Choose a voice

The server ships with predefined voices as audio files in its voices/ directory, and the voice is named by that file. GET /v1/audio/voices on the server lists them, and the web page plays them. To have it speak in a voice of your own, put ten to twenty seconds of clean speech as a .wav or .mp3 in reference_audio/, or upload it on the web page, and the server clones it. Whose voice you clone, and whether you may, is yours to answer.

3. Tell sidetone

$ api -X PUT -d '{"tts_url":"http://speech.example.lan:8004/v1","tts_model":"chatterbox","tts_voice":"Emily.wav"}' localhost:8080/v1/settings
  • The address ends at /v1, as with every speech server.
  • The model name is not used by the server; any word will do.
  • The voice is a file name from the server's voices list, or the name of a reference recording you added.
  • No API key.

Press Test text to speech. The first request after a start loads the model and may exceed sidetone's 50 seconds: try once more. Then make your prompts under Sounds → From text…. Chatterbox speaks at a high sample rate; sidetone brings it down to telephone audio, so do not expect the studio quality you hear on the server's web page to survive the phone line, but the cadence and the naturalness do.

Multilingual and Turbo

The server can switch between the original Chatterbox, Chatterbox Multilingual, which speaks 23 languages, and Chatterbox Turbo, a smaller and much faster model. The choice is made on the server's web page and applies to every request, sidetone's included; nothing changes on the sidetone side. For an office that answers in more than one language, Multilingual and a prompt per language is the way.