Fix Vertex AI Gemini default mimeType when emitting fileData for HTTPS image URLs

[Jessica Alder (jessicaalder-lgtm)]

Summary

Customers using Vertex AI Gemini models in the playground (and any caller flowing through gateway → lingua → Vertex) get a 400 from Vertex when sending file/image content parts whose URL is an HTTPS link rather than a base64 data URL:

"Unable to submit request because it has an empty mimeType parameter in fileData. Add a value to the parameter and try again."

The same payload against the standalone Gemini provider succeeds. Seems like server-side tolerance difference (v1beta is permissive; Vertex v1 is strict), not a routing difference. Any chat-completions API call to a Vertex Gemini model where the input contains a File message image_url content part with an HTTPS URL.

Root cause

crates/lingua/src/providers/google/convert.rs:380-413 converts a Universal UserContentPart::Image to a Google Part.

Three branches:

Input	Output	mime_type
data:…;base64,… data URL	inline_data	parsed from URL ✓
http(s)://… URL	file_data	caller-supplied media_type (Option<String>, may be None) ✗
anything else	inline_data	media_type.unwrap_or(DEFAULT_MIME_TYPE) ✓

The HTTPS branch was the only one missing a fallback. When the OpenAI chat-completions input converter at crates/lingua/src/providers/openai/convert.rs:3134-3141 sees an HTTPS URL, it sets media_type: None. That None flowed untouched into the Google emit, producing fileData: { fileUri: "...", mimeType: null }. Vertex rejects.

The existing multimodalRequest payload-test snapshot had this buggy output captured (fileData with no mimeType), so it never caught the issue.

Fix

Fallback chain at the Google output boundary:

caller-supplied media_type → URL-inferred mime → DEFAULT_MIME_TYPE

URL inference uses a new util/media::mime_type_from_url helper:

1. response-content-type query param on S3 presigned URLs (handled by existing parse_file_metadata_from_url).
2. File-extension lookup against Vertex/Gemini-supported types (png, jpg, gif, webp, heic, pdf, mp3, wav, mp4, etc.).
3. None if neither matches; caller applies DEFAULT_MIME_TYPE as last resort.

Fixing at the Google output boundary (rather than the OpenAI input boundary) protects all upstream sources. The Universal UserContentPart::Image.media_type is Option<String> by design, so any producer (Anthropic input, direct construction) can legally produce None.

Tests

• New regression test in convert.rs::tests for Image { image: <HTTPS URL>, media_type: None } — fails before the fix, passes after.
• Coverage for caller-supplied media_type precedence, URL-without-extension fallback to DEFAULT_MIME_TYPE, and a regression guard for the unaffected data-URL branch.
• 8 new tests for mime_type_from_url (S3 presigned content-type override, extension matching including uppercase, unknown/missing extensions, data URLs, invalid URLs).
• New multimodalRequestUrlImage payload case in payloads/cases/advanced.ts — exercises the previously-uncovered HTTPS-URL converter branch.
• Updated the existing multimodalRequest > request 1 snapshot, which had captured the buggy pre-fix output (no mimeType on fileData).

Downstream

This is the lingua side of the fix. After this merges, the gateway needs the lingua submodule pointer bumped in the braintrust repo (one-line change in lingua submodule SHA on a follow-up braintrust PR), then deployed.