Skip to content

feat(asr): surface sherpa streaming UI and docs#544

Draft
weikeyi wants to merge 3 commits into
Open-Less:betafrom
weikeyi:pr/sherpa-streaming-ui-docs
Draft

feat(asr): surface sherpa streaming UI and docs#544
weikeyi wants to merge 3 commits into
Open-Less:betafrom
weikeyi:pr/sherpa-streaming-ui-docs

Conversation

@weikeyi

@weikeyi weikeyi commented May 28, 2026

Copy link
Copy Markdown
Contributor

User description

概要

  • 胶囊监听 local-asr-token,在录音 / 转写 / 润色阶段显示 sherpa online streaming partial 文本。
  • Local ASR 页面新增 Batch / Streaming 模型标签、最近一次诊断信息,以及 Zipformer streaming 模型元数据。
  • 补齐 sherpa streaming 相关中英日韩繁体文案。
  • 新增 Windows-only sherpa_online_spike example,用于直接验证 Zipformer online 模型的 partial / final 输出和 RTF。
  • 更新 Windows sherpa-onnx ASR 实施计划,记录当前已完成状态和剩余验收项。

验证

  • npm run build
  • cargo check --manifest-path src-tauri/Cargo.toml --example sherpa_online_spike

PR Type

Enhancement, Tests, Documentation


Description

  • Surface sherpa streaming partials in capsule

  • Add local ASR model diagnostics

  • Extend sherpa model catalog metadata

  • Ship Windows streaming probe and plan updates


Diagram Walkthrough

flowchart LR
  A["Sherpa model catalog"] -- "mode metadata" --> B["Local ASR page"]
  B -- "state + diagnostics" --> C["Capsule UI"]
  D["local-asr-token events"] -- "partial text" --> C
  E["Windows spike example"] -- "validates streaming model" --> A
  F["Windows sherpa plan"] -- "documents current status" --> B
Loading

File Walkthrough

Relevant files
Tests
1 files
sherpa_online_spike.rs
Add Windows streaming probe example                                           
+241/-0 
Enhancement
3 files
Capsule.tsx
Show live streaming text in capsule                                           
+50/-4   
localAsr.ts
Add streaming model and status metadata                                   
+36/-0   
LocalAsr.tsx
Display sherpa modes and diagnostics                                         
+56/-2   
Documentation
6 files
en.ts
Add sherpa streaming and diagnostics copy                               
+9/-1     
ja.ts
Add sherpa streaming and diagnostics copy                               
+9/-1     
ko.ts
Add sherpa streaming and diagnostics copy                               
+9/-1     
zh-CN.ts
Add sherpa streaming and diagnostics copy                               
+9/-1     
zh-TW.ts
Add sherpa streaming and diagnostics copy                               
+9/-1     
windows-sherpa-onnx-asr-plan.md
Update sherpa rollout status and plan                                       
+524/-293

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Compile error

str::ends_with does not accept a predicate function, so this whitespace check will fail to compile. That means the new Windows spike example cannot be built until the spacing logic is rewritten with a valid check.

if !text.is_empty() && !text.ends_with(char::is_whitespace) {
    text.push(' ');

@appergb

appergb commented May 28, 2026

Copy link
Copy Markdown
Collaborator

我需要看到你具体的表现形式,具体的表现形式就是我需要具体的表现和具体的显示形式。

@weikeyi

weikeyi commented May 28, 2026

Copy link
Copy Markdown
Contributor Author

看不懂你的意思

@H-Chris233

Copy link
Copy Markdown
Collaborator

看不懂你的意思

意思是请给出演示图片和视频(对于任何 UI 相关改动

@weikeyi

weikeyi commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

本次涉及的主要前端改动为胶囊里的实时文字显示。
QQ20260530-213342-HD

@H-Chris233

Copy link
Copy Markdown
Collaborator

@appergb 我们是否应该合入这个pr?

@appergb

appergb commented May 31, 2026

Copy link
Copy Markdown
Collaborator

@H-Chris233 NO 这个应该给开关. 其次是他的P R没有详细的写出它更改了哪些部分 再然后是如果要有显示应该在胶囊的上方单独弹出窗口进行显示,并且这个窗口的显示应该是从横向往上,然后可以扩大到这种动态的窗口,然后里面做字符,然后字符就变动的时候窗口变动大小

@H-Chris233

Copy link
Copy Markdown
Collaborator

对于这种UI改动的功能请给开关

@H-Chris233 H-Chris233 marked this pull request as draft June 1, 2026 09:16
@H-Chris233 H-Chris233 added area:asr ASR area area:ui UI area needs-work Needs additional work stale-candidate Candidate for staleness labels Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:asr ASR area area:ui UI area needs-work Needs additional work Review effort 3/5 stale-candidate Candidate for staleness

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants