The media tool now speaks

CoreSpeed's media tool already made images and video. Now it speaks. Three text-to-speech models, a voice designer and a transcriber from Fish Audio are in the same tool, behind the same MCP endpoint, https://api.corespeed.io/mcp. There is nothing new to connect.

Speech works like every other media call. media__generate runs the model. The audio lands in your organization's Drive under tmp/media/<job>/, with a signed link in the response. media__get_result and media__list_jobs find it again later, from any agent in the organization.

What is new

1. fish-audio/tts/drama-3, Fish Audio's most expressive model, made for directed performance: directions in square brackets, sound cues, and whole scenes with one voice per speaker.
2. fish-audio/tts/s2.1-pro, the production model. Same cues, same dialogue turns, 83 languages detected on their own.
3. fish-audio/tts/s2-pro, the previous generation, for when you need its exact sound.
4. fish-audio/voice-design/v1, which makes a new voice from a text description.
5. media__transcribe, speech to text with speaker turns and word timings.
6. media__list_voices, a live search of Fish Audio's public voice library.

What you can do

Act out a scene

Start each turn with <|speaker:0|>, <|speaker:1|> and so on, and pass one voice per speaker, in order. Here is a short script for two voices from the library:

<|speaker:0|>[whispers] Is the demo still running?
<|speaker:1|>[relieved laugh] It finished an hour ago. The agent filed the report, posted it to the channel, and went quiet.
<|speaker:0|>[barely holding it together] So it worked.
<|speaker:1|>[warmly] It worked. Go home.

The agent sends it as one string:

{
  "model": "fish-audio/tts/drama-3",
  "input": {
    "text": "<|speaker:0|>[whispers] Is the demo still running?\n<|speaker:1|>[relieved laugh] It finished an hour ago. The agent filed the report, posted it to the channel, and went quiet.\n<|speaker:0|>[barely holding it together] So it worked.\n<|speaker:1|>[warmly] It worked. Go home.",
    "voice_id": [
      "8b077acc61e8431093ae2776d386d929",
      "a754ca8729d642d9a918a42b5b6a77d1"
    ],
    "format": "mp3"
  }
}

Pick a voice

media__list_voices searches the library by query and language. Each row has a voice_id, and most have a sample_url to listen to first. We found the two voices above with this call:

{ "language": "en", "licensed_only": true, "sort": "popular" }

Use licensed_only: true for commercial work. Those are voices Fish Audio has licensed from their owners. The rest of the library is community uploads, and some of them imitate real people.

Design a voice

Describe the voice you want. Voice design returns candidate clips (two by default, up to four with n), and each one is saved to your Drive:

{
  "model": "fish-audio/voice-design/v1",
  "input": {
    "instruction": "Warm, confident studio narrator in her forties, natural tone, unhurried pace, slight smile in the voice",
    "reference_text": "Welcome back. Today we look at what changed this week, and what it means for your team.",
    "language": "en",
    "n": 2
  }
}

Your own voice

In CoreSpeed, your own voice is an audio file in your Drive. Record 10 to 30 seconds of clear speech as WAV, MP3 or FLAC. Pass the file as reference_audio_url, and the exact words it says as reference_text.

A voice design candidate works as it is. We moved the first one to voices/narrator.wav and gave it a new line:

{
  "model": "fish-audio/tts/s2.1-pro",
  "input": {
    "text": "This week the media tool learned to speak. Your agent can now write a script, cast the voices, and hand you the finished audio in your Drive.",
    "reference_audio_url": "voices/narrator.wav",
    "reference_text": "Welcome back. Today we look at what changed this week, and what it means for your team.",
    "format": "mp3"
  }
}

For a scene, pass arrays instead: one file and one transcript per speaker, in <|speaker:N|> order.

No voice is saved at Fish Audio. The clip is sent with each call. So there is no voice to register at the vendor, and nothing to delete there later. Who can use the voice is decided like any other file: by who can read it in your Drive. Only use a voice you have the speaker's permission to use.

Transcription

media__transcribe takes one media source: a Drive path, a file id or a public https URL, up to 60 minutes and 100 MB. Speakers are marked inline, and cues such as [laughter] stay in the text unless you set tag_audio_events: false. timestamps: true adds word timings in segments and timed speaker_turns. The language is detected, so language is only a hint.

We sent the scene above back through it with timestamps: true. The first two turns came back like this (the second one trimmed):

"speaker_turns": [
  {
    "speaker": "speaker:0",
    "start": 0.08,
    "end": 1.92,
    "text": "[whisper] Is the demo still running?"
  },
  {
    "speaker": "speaker:1",
    "start": 4,
    "end": 10.4,
    "text": "[chuckle] It finished an hour ago. ..."
  }
]

The transcriber heard the whisper and the laugh that Drama 3 performed. That makes it a quick check on a generated clip, as well as a tool for meetings and interviews. Send a whole conversation in one call, because speaker labels match only within one answer.

Get started

If your agent already uses CoreSpeed, ask it for speech. The models come up in media__list_models with query: "fish". If not, give your agent one line:

set up https://corespeed.io/SKILL.md

The agent reads the skill and signs you in. The Media docs list every input.