Skip to content

RD-623: Argmax OSS engine and pipelines (replace legacy WhisperKit transcription)#99

Open
EduardoPach wants to merge 1 commit intomainfrom
eduardo/rd-623
Open

RD-623: Argmax OSS engine and pipelines (replace legacy WhisperKit transcription)#99
EduardoPach wants to merge 1 commit intomainfrom
eduardo/rd-623

Conversation

@EduardoPach
Copy link
Copy Markdown
Collaborator

What does this PR do?

This change integrates the Argmax open-source Swift CLI (argmax-cli from argmaxinc/argmax-oss-swift) into OpenBench as a first-class engine and pipeline set, and retires the old Python WhisperKit transcription module that wrapped a different code path.

Engine

  • Adds ArgmaxOpenSourceEngine (argmax_oss_engine.py): resolves the CLI via optional cli_path, or clone + swift build -c release --product argmax-cli under ARGMAX_OSS_CACHE_DIR (default ~/.cache/openbench/argmax-oss), with optional commit_hash pin.
  • Exposes transcribe and diarize helpers that shell out to argmax-cli with caller-supplied flag lists.

Pipelines

  • Transcription: ArgmaxOpenSourceTranscriptionPipeline — runs transcribe, parses JSON report segments/words (including per-word timings when present).
  • Diarization: ArgmaxOpenSourceDiarizationPipeline — runs diarize, produces RTTM compatible with existing annotation loading.
  • Orchestration: ArgmaxOpenSourceOrchestrationPipeline — diarize then transcribe and merge speaker labels using word timing.

Aliases and removals

  • Legacy whisperkit.py transcription pipeline is removed.
  • Existing whisperkit-* transcription aliases (e.g. tiny, large-v3, large-v3-turbo) now target the OSS transcription pipeline and configs (same alias names for benchmark continuity).
  • New aliases: argmax-oss-diarization, argmax-oss-orchestration-tiny (plus orchestration config pattern consistent with other orchestration entries).

Testing

  • Exercise openbench-cli evaluate (or pipeline smoke) with a whisperkit-* or argmax-oss-* alias after ensuring argmax-cli is built or cli_path is set.

Introduce argmax_oss_engine and wire transcription, diarization, and
orchestration pipelines plus pipeline aliases. Remove legacy whisperkit
transcription module in favor of Argmax OSS.

Made-with: Cursor
@EduardoPach EduardoPach requested review from arda-argmax and dbrkn May 1, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant