Merge dev into main#35
Merged
Merged
Conversation
…l, smart model picker
Config & paths
- Move all CLI runtime data to ~/.config/nexus-cli/ via NEXUS_RUNTIME_ROOT env var
set at startup; backend keeps ~/.config/nexus/ untouched
- Fix Viper config lookup to use runtimepath.ResolveRoot() instead of hardcoded
~/.config/nexus, preventing the CLI from leaking the backend's model config
- Migrate legacy ~/.nexus_secret → ~/.config/nexus-cli/secret.key on first run
- DB default path now ~/.config/nexus-cli/data/nexus.db (persistent, not /tmp)
Credentials & model persistence
- API keys stored AES-256-GCM encrypted in SQLite, never in YAML
- Model selection persisted to DB (key "model") so it survives YAML resets
- stripRuntimeSecrets() clears all provider secrets + RuntimeRoot before YAML save
- loadCredsIntoConfig() loads model from DB when YAML has no model set
Startup UX
- No default provider on first run; welcome screen shows ctrl+p hint when unconfigured
- CLI starts without error when nothing is configured (skip provider validation)
- SDK default used internally for client init; TUI shows empty model string to user
- Banner shows "(not configured)" instead of "model: /" when no provider set
- ParseModelIdentifier("") returns empty ModelIdentifier instead of Anthropic default
TUI: web search panel
- New SearchPanel component (ctrl+p → Web Search) matching ConfigPanel design
- Supports Tavily, Exa, Jina AI, LangSearch, SearXNG, DuckDuckGo
- List mode, key-edit mode, mode-select mode with masked input
- SaveSearchKey / SaveSearchMode / LoadSearchConfig wired through Workspace interface
TUI: provider config & model picker
- ctrl+v pastes from clipboard in all secret fields; ctrl+r toggles reveal (was ctrl+v)
- Model picker only shows providers that have credentials configured
- Ollama: auto-detected at startup via /api/tags + /api/show per model (generative only,
embedding models filtered); result cached in DB under "ollama:models"
- Ollama re-probed automatically on save/delete of its endpoint field
- Picker refreshes live when startup probe completes (ModelListMsg sent after cache write)
- Ollama endpoint configurable in provider settings (default: http://localhost:11434)
- 8s timeout for Ollama live fetch to handle slow /api/show chains
…nce (#34) * feat(tui): three-mode execution badge + pair programming mode tracking Header now shows ● execute (muted) / ◈ plan (orange) / ◎ pair (lighter orange) via ExecutionMode() instead of the binary PlanMode() check. AddToolProgress intercepts enter/exit_pair_programming_mode alongside the existing plan mode tools — updates pairDepth counter, suppresses tool rows. * fix(tui): render read_file content as clean markdown/code in detail panel The engine's FormatTextWithLineNumbers embeds "File:/Lines:" header and N→ line-number prefixes in the content metadata field. Glamour cannot parse that as markdown, producing raw symbol noise in the detail sidebar. parseReadContent strips the header block and N→ prefixes, returning the clean file body plus the 0-based start line. detailBody and inlinePreview now use the clean body; code files pass startLine as offset to renderCodeBody so line numbers still reflect actual file positions. * fix(tui): reset all session state when deleting the active session Deleting the active session from the sessions panel now immediately: - clears m.activeSession and resets to the welcome screen - clears the chat (messages, tool selections, planDepth, pairDepth) - clears lastTurnErr / lastErr / busy so no stale error leaks into new sessions Clear() also resets planDepth and pairDepth so mode badges start clean on every session switch. * feat(tui): replay session transcript on resume Previously, loading a previous session showed only "Resumed session" with an empty chat. Now the full conversation is replayed from the stored transcript: user messages, assistant text, thinking blocks, and completed tool rows (with their input/result metadata for the detail sidebar). The conversion happens in buildSessionHistory (cmd/cli/tui.go), which pairs each ToolUseContent with its matching ToolResultContent, then sends the result as []HistoryEntry in SessionLoadedMsg. Also exports sdk.ToolResultContent from pkg/sdk/types.go. * feat(tui): full-fidelity session replay with persisted tool metadata buildSessionHistory now reads ToolResultContent.Metadata (already written by buildToolResultMessages in the engine) which carries the complete TUI metadata map: content, execution_duration_ms, lines_added, lines_removed, exit_code, cwd, type, url, title, provider, result_count, etc. The replay loop in model.go copies this map and injects tool_input so all detail panel renderers (file content, diff, bash output, web results) work exactly as during the live session. HistoryTool.Result removed in favour of HistoryTool.Metadata; fallback to {content: rawString} for old sessions that predate this change. * feat(db): add session_files table to track file operations per session Migration 20260607_007_session_files creates session_files with: (session_id, file_path, operation, timestamp_unix, lines_added, lines_removed) Two indexes: by session (for fast per-session lookup) and by path (for cross-session "who touched this file?" queries). Live recording: onProgress writes to session_files whenever write_file, edit_file, or apply_patch completes during an active session. Backfill: when LoadSession runs, if no session_files rows exist for that session the transcript is scanned and rows are inserted retroactively, covering all sessions created before this change. operation values: "create" | "update" (write_file), "edit" (edit_file), "patch" (apply_patch). file_path and line counters come from the ToolResultContent.Metadata already stored in the transcript. * fix(db): add tool_use_id to session_files as pointer to transcript metadata Without tool_use_id, retrieving a diff from session_files required scanning the entire transcript JSON. With tool_use_id stored: session_files.tool_use_id → session_transcript_entries.entry_json → ToolResultContent.Metadata["structured_patch" | "git_diff" | "content"] The migration is updated before it ever ran (table didn't exist in live DB), so no ALTER TABLE needed. A dedicated index on tool_use_id is added for direct lookup by tool call. * feat(db,vector): SQLite optimizations, FTS5 transcript search, and HNSW RAG backend DB / SQLite: - Add perf pragmas: 20 MB page cache, 128 MB mmap, WAL temp in RAM, autocheckpoint - Run PRAGMA optimize on Close for query-planner housekeeping - Fix UpsertSessionFile to INSERT OR IGNORE + unique partial index on tool_use_id - Fix HasSessionFileEntry and HasNamespace to use SELECT EXISTS instead of COUNT(*) - Fix DeleteSession to rely solely on FK CASCADE (single DELETE FROM session_metadata) - Fix GetTeamAgents to use raw SQL SELECT DISTINCT instead of full-struct GORM scan - Add migration 008: dedup session_files + mailbox unread/history partial indexes - Add migration 009: FTS5 virtual table session_transcript_fts with insert/delete triggers that stay in sync with CASCADE deletes from session_metadata Vector / RAG: - Add BackendHNSW: pure-Go HNSW store (github.com/coder/hnsw), no CGO, no external service - Per-namespace persistence: <slug>.hnsw (graph) + <slug>.meta.json (text + metadata) - O(log n) ANN search vs previous O(n) brute-force; scores normalized to cosine similarity - Hybrid keyword blend when HybridWeight > 0 + QueryText set - Wire HNSW backend into CLI via buildRAGService; activates only when RAG_EMBEDDING_URL + RAG_EMBEDDING_MODEL env vars are present - Add HNSWDataDir helper to runtimepath - Add tests: upsert/search/persistence/delete and hybrid keyword ranking - Add complete database schema doc (docs/database-schema.md) * feat(tui): keyboard-first interaction, updated tests and roadmap - Replace mouse-click hint with ctrl+t keyboard hint in thinking block footer - Replace HandleMouseDown/Up tool detail zone click with HasSelectedTool + ToggleDetails (tool detail pane is now keyboard-driven, not mouse-zone-click-driven) - Update golden snapshots to match new rendering - Update TUI roadmap: mark config isolation, credentials DB, clipboard paste as done; expand in-progress and upcoming sections * fix: correct critical issues from codebase audit (C1-C5, M1) C1 — Session leak in task manager: Add committed bool + defer pattern; session.Close() is called if RegisterTools fails before the goroutine takes ownership. C2 — HNSW partial write undetected: Replace two separate error checks with errors.Join so both saveErr and metaErr are always surfaced to the caller. C3 — FTS5 migration errors silently ignored: Replace `_ = err` with log.Printf in both migrateSQLiteVectorFTS5 and migrateSQLiteTranscriptFTS5; startup no longer fails but the operator sees a warning when hybrid search degrades to LIKE scan. C4 — JSON metadata unmarshal silently ignored: Replace `_ = json.Unmarshal(...)` with explicit error logging in hnsw_store.go and sqlite_store.go; corrupted metadata is visible in logs instead of silently returning nil. C5 — context.Background() hardcoded in sqlite_backend: Add dbCtx() helper returning context.WithTimeout; DeleteSession uses 10s timeout, AppendTranscriptEntries and ReplaceTranscript use 30s. Full ctx propagation on the Backend interface is tracked as L-A. M1 — Embedding dimension never validated: Both embedOpenAI and embedOllama now check that every returned vector is non-empty and that all vectors in a batch share the same dimension. Also fix: HNSW hybrid search result order hnswBlendKeyword was modifying scores but not re-sorting, causing keyword-boosted records to be returned out of order. Add sort.Slice descending by score after blending. Caught by TestHNSWStore_HybridKeywordBlend. Add docs/audit/codebase-audit-2026-06.md with full audit findings split into NOW (fixed) and LATER (community issues L-A through L-M). * fix(agent): sub-agent tool failures due to missing InputSchema and field aliases Three root causes identified from runtime observation (agents completing in 20-52ms, never making LLM calls): 1. Missing InputSchema on AgentTool.Definition(): The 'agent' tool had no JSON Schema, forcing the LLM to guess field names from description text. With spawn_agent (which has a full schema) registered in the same registry, the LLM was cross-contaminating field names. Fix: add InputSchema with type enum, task, maxTurns, run_in_background, fork, isolation, and tools properties — matching the Description contract exactly. 2. 'agent_type' alias not handled: Call() only read parsedInput["type"], not parsedInput["agent_type"]. The LLM used "agent_type" (spawn_agent convention) causing agentType=="" → fast return "type is required" every time. Fix: accept "agent_type" as fallback alias for "type". 3. Error message for missing type hid valid values: "Error: type is required" gave the LLM nothing to self-correct with. Fix: include the full list of available agent types in the error response, matching the existing behavior for "unknown agent type". Also improve wait_agent error hint when the agent_id looks like a tool_use_id (UUID format), helping the LLM distinguish spawn_agent IDs from tool_use_ids. * fix(providers,tui): file logging + orphaned tool_result sanitization Redirect TUI stdlib log output to ~/.config/nexus-cli/logs/cli.log so errors are observable in TUI mode instead of silently discarded. Strip orphaned tool_result blocks before sending to OpenAI-compat APIs to prevent invalid_request_message_order errors (z-ai/GLM-4.5 etc.) when parallel agent failures leave tool_results without a matching assistant tool_call. Sanitizer is a no-op when no assistant tool_calls exist in the conversation, preserving valid single-turn tool results. * fix(agent): ToolUses tracking and memory cleanup in async agent manager Pass cumulative toolUses count through RunConfig.Callback so AsyncAgent.ToolUses stays accurate turn-by-turn instead of remaining 0 during execution. Sync final ToolUses from RunResult after RunAgent() completes to cover any missed updates. Call Cleanup() in Shutdown() after all goroutines finish to release memory held by completed/failed/cancelled agents. Removed lazy cleanup from StartAgent() since it would break wait_agent by deleting completed agents before the LLM retrieves them. * feat(tui): ESC to interrupt agent + SearXNG URL configuration ESC (and ctrl+c) now cancels the running agent turn immediately by cancelling the per-submit context, stopping the API call in progress. The footer shows "interrupting…" while waiting for the goroutine to drain. context.Canceled errors are suppressed so no red error banner appears after a deliberate user interrupt. SearXNG is now configurable from the web search panel: pressing Enter on SearXNG opens an "Instance URL" field (not masked, not a secret). The URL is persisted to the DB under "SEARXNG_BASE_URL" and applied as an env var at startup via loadCredsIntoConfig, so NewSearXNGProvider() picks it up on every run. The mode selector can then be set to "searxng" to route all web searches through the configured self-hosted instance. * fix(tui): sidebar scrolling — preserve position on streaming updates Root cause: SetSize() reset detailKey to "" which forced GotoTop() on every streaming update (chat height grows → SetSize called → detailKey="" → next render sees detailKey≠key → GotoTop). Removed the reset. Rewrote renderToolDetail cache logic with three distinct cases: - New tool selected (detailToolID changed): reset to top - Same tool, content grew (streaming) or size changed: preserve yOffset - Size only changed, identical content: re-layout preserving yOffset Also fixed ctrl+o auto-switching focus to uiFocusMain when the sidebar opens, so arrow keys scroll immediately without requiring an extra Tab press. Closing the sidebar returns focus to the editor input. * fix(agent): isolate async sub-agent context from parent turn Background sub-agents spawned via spawn_agent were running with the parent session's turn context. When that turn ended, defer cancel() fired and killed the still-running sub-agent's API calls and permission prompts, producing 'permission denied: prompt failed: context canceled'. Three changes: - async.go runAgent: replace config.Context with agent.Ctx so the goroutine uses its own independent context regardless of parent state - runner.go RunConfig: add PermissionMode field to let callers override the session's permission mode after creation - spawn_agent.go: set PermissionMode=bypass so background agents auto- approve tools without blocking on interactive prompts that no longer have a valid TUI context * feat(agent): track and expose sources from sub-agent tool calls Sub-agents now automatically collect every file path, URL, and search query they consult during execution. Sources are deduplicated and attached to RunResult.Sources as []SourceRef{Type, Value}. The parent agent receives sources in the tool_result data payload (agent tool, wait_agent, fork mode, worktree mode). wait_agent also appends a formatted source list at the end of its Content string so the parent LLM sees them inline without parsing JSON. Extracted automatically from: read_file, write_file, edit_file, glob, grep, web_search, web_fetch, web_crawl, web_map, browser_navigate, browser_open, wikipedia, scholarly_search, langsearch. * feat(agents): implement resume_agent — session resumption for sub-agents Sub-agents now persist their session ID in RunResult and AsyncAgent after each run. A new resume_agent tool reopens the persisted session via Engine.OpenSession and submits a new task into the existing conversation history, so the agent retains full context of everything it read, fetched, and wrote previously. Changes: - engine.go: add OpenSession(ctx, sessionID) via optional sessionRestorer interface - session.go: add GetSessionID() accessor - runner.go: add SessionID to RunResult, ResumeFromSessionID to RunConfig; RunAgent branches on ResumeFromSessionID to restore instead of create - async.go: add SessionID field to AsyncAgent, captured from RunResult on completion - wait_agent.go: expose session_id in result JSON + update description - resume_agent.go: new tool accepting session_id (or agent_id) + task; supports sync (blocking) and async (background) modes - sdk/client.go: register resume_agent alongside spawn_agent * fix(tui): session list timestamps, preview, and silent load errors Three bugs fixed in the session browser (Ctrl+S): 1. UpdatedAt/CreatedAt not propagated — sessions always showed "—" for age. cmd/cli/tui.go was discarding the int64 unix timestamps from state.SessionInfo instead of converting them to time.Time for tui.SessionInfo. 2. Session load errors silently swallowed — when RestoreSessionState failed (checkpoint mismatch, compaction boundary error, etc.) the error was stored in m.lastErr which is never rendered, leaving the user with a blank chat and no feedback. Now also sets m.lastTurnErr so the status bar shows the failure. 3. No session preview — the session picker only showed an 8-char ID, age, and turn count with no context about what the session was about. Like Codex, the first user message is now extracted during SaveSessionState, stored in metadata.Additional["canonical_transcript"]["first_user_message"], surfaced in SessionInfo.Preview, and rendered below the meta line in the picker. Search now also matches against the preview text. * fix(providers): z-ai uses x-api-key auth header, not Authorization: Bearer api.z.ai returns 'x-api-key header is required' on 401, meaning the endpoint uses Anthropic-style authentication despite serving OpenAI-compat request bodies. - Add zAiAdapter that embeds openAICompatAdapter (same /chat/completions body and response format) but overrides applyAuthHeaders to send x-api-key - Update adapterForProvider to route APIProviderZAi to zAiAdapter - Update BuildAuthHeaders in config.go to use x-api-key for ZAi - Update the provider test to match the new auth header * feat(tui): full session deletion and session title display DeleteSession now removes associated browser artifacts from storage (screenshots, downloads) and cleans up in-memory plan state/files. Session list shows the first user message line as the primary title with ID/age/turns as secondary metadata, matching Codex's approach. * feat(appdir): session-scoped directory layout All session data now lives under sessions/{session_id}/ directly in the app root (~/.config/nexus-cli/). Screenshots go to sessions/{id}/images/, downloads to sessions/{id}/tools/, and plan files to sessions/{id}/plans/. Deleting a session is now two calls: store.DeleteSession for the DB and appdir.DeleteSessionDir (os.RemoveAll) for all physical files — no more per-namespace artifact listing. New appdir package centralises all path resolution for cmd/cli. Session directories are created via EnsureSessionDir at session open/resume. * feat(appdir): session-scoped artifacts with web/images/audio structure Web-scraped content moves from global artifacts/web/ into sessions/{id}/artifacts/web/, making it session-scoped and cleaned up automatically on session delete. Browser screenshots move to sessions/{id}/screenshots/ (renamed from images/ to avoid ambiguity). Adds key builders and store functions for: - GeneratedImageKey / StoreGeneratedImageRef → sessions/{id}/artifacts/images/ - AudioKey / StoreAudioRef → sessions/{id}/artifacts/audio/ - WebArtifactKey / StoreWebArtifactRef → sessions/{id}/artifacts/web/ Threads sessionID through the fetch pipeline (Fetch → fetchViaHTTP → persistArtifact) so web fetches land in the right session directory. Storage GC reaper no longer needed for these namespaces. * fix(auth): z-ai x-api-key always missing on fresh configuration Root causes fixed: - loadRuntimeOptions applied overrides.Model AFTER loadCredsIntoConfig, so the credential lookup used the wrong provider (anthropic default) and returned an empty API key for z-ai - SaveProviderField called reloadClient with no model set, building a keyless client that raced with SetModel's correct reload - CreateSession and LoadSession goroutines could grab w.client before SetModel's reloadClient goroutine finished (reloadMu added to serialise) Also fixed: - LangSearch API key not restored from DB on startup - LangSearch missing from 'nexus config --search' and config summary - pendingSubmitMsg dropped when Enter pressed before session created - /cli binary added to .gitignore * fix(tui): robust configuration loading and input isolation Refactors the TUI and configuration logic to ensure immediate application of API keys and prevent accidental file path insertions during setup. - Fixes circular dependency in loadCredsIntoConfig. - Triggers client reload on all sensitive configuration changes in TUI. - Normalizes Z.ai provider identifiers (zai, z-ai, z.ai) across the stack. - Strictly disables file completions during configuration states. - Refactors TUI chat components into specialized files for better maintainability. * fix(cli): parse permission mode case-insensitively during client reload * feat(permissions): add session-level auto-approvals for tools * feat(permissions): persist session-level tool approvals to permissions.json * fix(state): resolve checkpoint mismatch on session load and delete * perf(tui): optimize keystroke and scrolling rendering performance
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release of dev branch features and bug fixes to main, including TUI rewrite and checkpoint mismatch/performance fixes.