Media understanding
Ask about an image, a video or an audio file. The question is free.
Tools1
media__understandfrom 0.5 credits / imageAsk a question about a video, image, or audio file and get a text answer. `media` is a Drive path or file id (drive__list, drive__create_upload, a connector download, a mail attachment, a media output), a public https URL fetchable without authentication, or a YouTube watch URL. `mime_type` is inferred from a URL extension, from the origin's Content-Type, or carried by an upload handle or Drive file. One media file per call. A URL is fetched by CoreSpeed (not by the model) and remembered for 48 hours, so repeat questions about the same URL do not re-fetch it. It must be a direct file URL with an exact Content-Length, up to 2 GB — a share page (Google Drive, Dropbox, …) is not a file URL. The first question about a large file may answer `upload_processing` (retryable) while the file is prepared; retry in a few seconds with the same `media` value. For a file that is only on local disk, call drive__create_upload with its exact type and byte size, then PUT the raw bytes to the returned one-use URL with the returned headers. Files up to 2 GB are supported; treat any HTTP 413 as non-retryable for this deployment before parsing the body. VIDEO can be windowed with `start_seconds` / `end_seconds`. `max_cost_credits` converts a credit budget into a window at the per-second rate; it bounds the media input only — answer tokens bill separately. An explicit `end_seconds` overrides it. AUDIO AND IMAGES reject all three parameters because clipping cannot be enforced for them. With no window, the whole file is analysed up to the capability's context limit. `structuredContent.analyzed_seconds` is an estimate — use it as a paging hint, not an exact duration; `truncated` marks an answer cut at the output limit; `usage` carries the metered quantities (media units + answer tokens) the call settled for. Video is sampled at 1 frame per second, so fast action between frames can be missed. Timestamps stated in prose are not guaranteed accurate. Billed per image (oversized images count as more than one) and per second of video or audio, plus the answer's tokens. The question is free. Failures never charge.Ask a question about a video, image, or audio file and get a text answer. `media` is a Drive path or file id (drive__list, drive__create_upload, a connector download, a mail attachment, a media output), a public https URL fetchable without authentication, or a YouTube watch URL. `mime_type` is inferred from a URL extension, from the origin's Content-Type, or carried by an upload handle or Drive file. One media file per call. A URL is fetched by CoreSpeed (not by the model) and remembered for 48 hours, so repeat questions about the same URL do not re-fetch it. It must be a direct file URL with an exact Content-Length, up to 2 GB — a share page (Google Drive, Dropbox, …) is not a file URL. The first question about a large file may answer `upload_processing` (retryable) while the file is prepared; retry in a few seconds with the same `media` value. For a file that is only on local disk, call drive__create_upload with its exact type and byte size, then PUT the raw bytes to the returned one-use URL with the returned headers. Files up to 2 GB are supported; treat any HTTP 413 as non-retryable for this deployment before parsing the body. VIDEO can be windowed with `start_seconds` / `end_seconds`. `max_cost_credits` converts a credit budget into a window at the per-second rate; it bounds the media input only — answer tokens bill separately. An explicit `end_seconds` overrides it. AUDIO AND IMAGES reject all three parameters because clipping cannot be enforced for them. With no window, the whole file is analysed up to the capability's context limit. `structuredContent.analyzed_seconds` is an estimate — use it as a paging hint, not an exact duration; `truncated` marks an answer cut at the output limit; `usage` carries the metered quantities (media units + answer tokens) the call settled for. Video is sampled at 1 frame per second, so fast action between frames can be missed. Timestamps stated in prose are not guaranteed accurate. Billed per image (oversized images count as more than one) and per second of video or audio, plus the answer's tokens. The question is free. Failures never charge.
- Image0.5 credits / image
- Video12 credits / minute
- Audio4.2 credits / minute
- Answer11.25 credits / 1K tokens
