Skip to content

Use Unicode placeholders for kitty graphics inside tmux#7

Open
ibehnam wants to merge 8 commits intomil-ad:mainfrom
ibehnam:unicode-placeholders-tmux
Open

Use Unicode placeholders for kitty graphics inside tmux#7
ibehnam wants to merge 8 commits intomil-ad:mainfrom
ibehnam:unicode-placeholders-tmux

Conversation

@ibehnam
Copy link
Copy Markdown

@ibehnam ibehnam commented Mar 6, 2026

Summary

  • When running inside tmux, plots render as full-screen overlays instead of staying inside the tmux pane. This is because the current tmux passthrough approach sends raw graphics escape sequences to the outer terminal, which renders images at absolute cursor positions without pane awareness.
  • This PR implements the kitty graphics protocol Unicode placeholder method. Image data is transmitted via tmux passthrough (a=T,U=1), but display uses U+10EEEE placeholder characters with diacritical marks encoding row/column positions. The image ID is encoded in the foreground RGB color. tmux treats these as normal text, so images stay inside their panes, scroll correctly, and get clipped at pane boundaries.
  • Both Ghostty and Kitty support Unicode placeholders natively.
  • The existing direct display path (outside tmux) is unchanged.

Changes

  • utils.py: Add get_char_cell_width() and num_required_cols() helpers (mirrors existing height/lines helpers)
  • backend.py: Add display_kitty_unicode_placeholder() with two-phase approach:
    1. Transmit image data via tmux passthrough
    2. Output Unicode placeholder characters as regular text
  • backend.py: Route to Unicode placeholders when TMUX env var is set

Prerequisites

Requires allow-passthrough in tmux config:

set -g allow-passthrough on

Test plan

  • Plot renders inside the tmux pane (not as a full-screen overlay)
  • Switching panes hides the plot correctly
  • Scrolling moves the plot with terminal content
  • Works in both Ghostty and Kitty
  • Plot still works correctly outside tmux (direct display, unchanged)

When running inside tmux, images displayed via the kitty graphics
protocol appear as overlays on the entire terminal screen rather than
being confined to the tmux pane. This is because tmux passthrough
sends raw graphics escape sequences to the outer terminal, which
renders images at absolute cursor positions without pane awareness.

Fix this by implementing the kitty graphics protocol Unicode placeholder
method (U+10EEEE). Image data is still transmitted via tmux passthrough,
but display uses placeholder characters with diacritical marks encoding
row/column positions and the image ID encoded in the foreground RGB
color. tmux treats these as normal text, so images stay inside their
panes, scroll correctly, and get clipped at pane boundaries.

This approach works with both Ghostty and Kitty, which support Unicode
placeholders natively. The existing direct display path (outside tmux)
is unchanged.

Requires `set -g allow-passthrough on` in tmux.conf.
ibehnam and others added 6 commits March 5, 2026 20:21
When IPython runs on a remote machine over SSH from a tmux pane,
the TMUX env var is not forwarded. Fall back to checking if TERM
starts with "tmux" or "screen", which SSH does forward.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TIOCGWINSZ returns 0 for pixel dimensions over SSH, causing
ZeroDivisionError. Fall back to 16x8 pixel cell size defaults.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Large single-passthrough DCS sequences can be silently dropped over SSH.
Wrapping each chunk separately is the standard pattern for kitty+tmux.
Also flush between image transmission and placeholder output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Without q=2 on continuation chunks, kitty sends a response for each,
filling the pty input buffer and causing a deadlock over SSH.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-chunk passthrough caused hangs. Revert to single DCS passthrough
wrapping all kitty graphics chunks, which is the same structure that
works locally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mil-ad
Copy link
Copy Markdown
Owner

mil-ad commented Mar 6, 2026

Thanks for the PR Behnam jan. I'll take a look soon

Over SSH, TIOCGWINSZ returns 0 for pixel dimensions, causing the
fallback cell sizes to overestimate placeholder grid rows/cols.
Add env var overrides for precise control and improve fallbacks
from 16/8 to 24/12 to better match modern terminal cell sizes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ibehnam
Copy link
Copy Markdown
Author

ibehnam commented Mar 7, 2026

You're welcome Milad jan!

@mschubert
Copy link
Copy Markdown

I can confirm that tmux anchoring is broken and this fixes it for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants