Skip to content

feat(asr): Apple Speech 本地 ASR provider 基础(Draft, issue #574)#625

Draft
appergb wants to merge 1 commit into
betafrom
feat/apple-speech-asr-draft
Draft

feat(asr): Apple Speech 本地 ASR provider 基础(Draft, issue #574)#625
appergb wants to merge 1 commit into
betafrom
feat/apple-speech-asr-draft

Conversation

@appergb

@appergb appergb commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

User description

Draft / WIP —— 抢救并重新落地 issue #574 的 Apple Speech 本地 ASR。本 PR 只含「可干净合并的基础」,核心链路接线 + 完整编译验证待续(见下方待办)。

背景

旧分支 feat/issue-574-apple-speech-asr 落后 beta 112 提交,且 coordinator.rs / coordinator/dictation.rs 已被 refactor/split-dictation(#619) 彻底重构、commands.rs 拆成了 commands/ 模块,无法 rebase(整文件冲突)。本 PR 从旧 WIP 抢救出 provider 实现,重新落到当前 beta 结构。

本 PR 已含(可干净合并的基础,已核验符号依赖存在)

  • asr/local/apple_speech_provider.rsSFSpeechRecognizer 批处理识别(16k/mono PCM → 临时 WAV → SFSpeechURLRecognitionRequest),授权流程、objc2 桥接、TempFileGuard RAII、单测齐全
  • asr/local/mod.rs:注册 provider + APPLE_SPEECH_PROVIDER_ID = "apple-speech" + is_apple_speech()
  • build.rs:链接 Speech.framework(macOS)
  • Info.plistNSSpeechRecognitionUsageDescription
  • 前端:LocalAsr 页 / ProvidersSection / Overview / shared + en/ja/ko/zh-CN/zh-TW i18n

待办(合并前必须完成)

  • dictation 接线:在 coordinator/dictation.rs 仿 is_local_qwen3 分支加 is_apple_speech 处理(apple-speech 无需下载模型,最接近 qwen3:buffer → transcribe()
  • coordinator 状态/选择:按需在 coordinator.rs 接入 apple-speech 的 provider 构建
  • 命令:在 commands/local_asr.rs(或新文件)加 apple-speech「是否支持 / 是否已配置」命令 + 平台门控(非 macOS 判不可用)
  • 编译验证cargo check --manifest-path src-tauri/Cargo.toml(CI 门禁,三平台)+ npm run build
  • 真系统验证:macOS 实测 授权弹框 → 录音 → 出字
  • cargo fmt + cargo clippy

约束

  • provider 仅 macOS 编译(#![cfg(target_os = "macos")]);非 macOS 由上层门控为不可用
  • 首版为批处理识别;实时 partial 流式列为后续增量

🤖 Generated with Claude Code


PR Type

Enhancement, Documentation


Description

  • Introduce Apple Speech local ASR provider for macOS

  • Backend implementation using SFSpeechRecognizer with objc2 bridge

  • Frontend UI card for activation and language support (5 i18n locales)

  • Link Speech.framework in build.rs and add NSSpeechRecognitionUsageDescription


Diagram Walkthrough

flowchart LR
  AppleSpeechAsr --> SFSpeechRecognizer
  AppleSpeechAsr --> LocalAsrUI
  LocalAsrUI --> i18n
Loading

File Walkthrough

Relevant files
Enhancement
6 files
apple_speech_provider.rs
New Apple Speech ASR provider with objc2 bridge                   
+433/-0 
mod.rs
Register Apple Speech provider and constants                         
+16/-0   
LocalAsr.tsx
Add Apple Speech activation UI card                                           
+78/-0   
Overview.tsx
Map apple-speech ID to display name                                           
+1/-0     
ProvidersSection.tsx
Add apple-speech preset and hide from dropdown                     
+11/-5   
shared.tsx
Extend AsrPresetId type with apple-speech                               
+1/-0     
Configuration changes
2 files
build.rs
Link Speech.framework for macOS                                                   
+4/-0     
Info.plist
Add NSSpeechRecognitionUsageDescription                                   
+2/-0     
Documentation
5 files
en.ts
Add English i18n strings for Apple Speech                               
+4/-0     
ja.ts
Add Japanese i18n strings for Apple Speech                             
+4/-0     
ko.ts
Add Korean i18n strings for Apple Speech                                 
+4/-0     
zh-CN.ts
Add Simplified Chinese i18n strings for Apple Speech         
+4/-0     
zh-TW.ts
Add Traditional Chinese i18n strings for Apple Speech       
+4/-0     

从旧 WIP 分支抢救 provider 并重新落到当前 beta 结构;只含可干净合并的基础,coordinator/dictation 接线、命令、完整编译验证待后续(见 PR 待办)。

- asr/local/apple_speech_provider.rs:SFSpeechRecognizer 批处理识别 + 授权 + objc2 桥接 + 单测
- asr/local/mod.rs:注册 + APPLE_SPEECH_PROVIDER_ID + is_apple_speech
- build.rs:链接 Speech.framework(macOS);Info.plist:NSSpeechRecognitionUsageDescription
- 前端:LocalAsr / ProvidersSection / Overview / shared + 5 语言 i18n
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ❌

619 - Not compliant

Non-compliant requirements:

  • 此 PR 未对 dictation.rs 做任何拆分或重构,仅添加了 Apple Speech provider

574 - Partially compliant

Compliant requirements:

  • 实现了 Apple Speech provider 核心代码(apple_speech_provider.rs)
  • 添加了 build.rs 链接 Speech.framework
  • Info.plist 添加了 NSSpeechRecognitionUsageDescription
  • 前端 LocalAsr 页面添加了 Apple Speech 卡片
  • i18n 覆盖 en/ja/ko/zh-CN/zh-TW
  • ProvidersSection 和 Overview 等组件添加了 apple-speech 支持

Non-compliant requirements:

  • dictation 接线未完成(待办中)
  • coordinator 状态/选择未接入
  • commands/local_asr 等命令未实现
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Potential Bug

In recognize_file(), the return value of recognitionTaskWithRequest:resultHandler: (the SFSpeechRecognitionTask) is captured as _task but not retained. Under ARC, the raw pointer assignment does not increase the retain count, so the task may be deallocated before the recognition completes, causing the callback to never fire and the rx.recv_timeout() to always time out. The _task variable should be held as a Retained or the object must be explicitly retained to ensure the task stays alive during the blocking wait.

let _task: *mut AnyObject = unsafe {
    msg_send![
        recognizer,
        recognitionTaskWithRequest: request,
        resultHandler: &*block
    ]
};

pull Bot pushed a commit to yimmy23/openless that referenced this pull request Jun 11, 2026
…pen-Less#574)

承接草稿 PR Open-Less#625 的 provider 基础,补完后端接线:
- coordinator: ActiveAsr::AppleSpeech 变体(macOS only)
- asr_setup: build_apple_speech + ensure_asr_credentials 平台门控 +
  build_qa_asr_start QA 接线 + is_keyless_local_asr_provider
- dictation_session/dictation_end: 听写 start + 批处理 transcribe(沿用
  local_qwen 动态超时)
- qa_session: QA transcribe arm
- resources: cancel arm
- commands: active_apple_speech_asr_is_supported(非 macOS 返回 false)+
  keyless 校验白名单 + asr_configured_for_provider + set_active_asr_provider
  非 macOS 拒绝门控

provider 仅 macOS 编译;非 macOS 由门控判不可用。前端 UI/i18n 已随 Open-Less#625 落地。
验证:cargo check(macOS)通过、provider 单测 5/5 通过、npm run build 通过。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@H-Chris233 H-Chris233 added area:asr ASR area platform:macos macOS-specific needs-work Needs additional work labels Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:asr ASR area needs-work Needs additional work platform:macos macOS-specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants