Convert templates to Colortemplate v3 syntax

[lifepillar]

Colortemplate v3 is finally (!) ready for prime time and has been merged into master. You may still checkout the v2 branch if you want to keep using Colortemplate v2. Please read the documentation if you upgrade to v3.

This PR brings all the templates to the new syntax introduced in v3. The syntax is fully described in Colortemplate's help file.

In terms of template syntax, the main difference is that Variant no longer exists: each highlight group definition is self-contained, as long time ago @romainl suggested. So, the new syntax is totaly declarative: highlight group definitions can be freely reordered (although in this PR I've tried to modify the templates as little as possible).

In terms of generated code, the major difference is that gui attributes, (best) cterm attributes and term attributes are merged in a single, complete, hi statement. I'm open to change the format of the output, but after a long time working with color schemes I've come to the conclusion that it's best to provide the most complete definition regardless of the actual environment.

In several places I've noted missing highlight groups and in few cases some ambiguities that I've encountered (highlight groups defined twice for the same variant). I've marked those with TODO in the templates.

Everything builds with the current Colortemplate master and, in theory, the color schemes should remain functionally unchanged. It would be unrealistic, though, to assume that these changes do not introduce any issues. As an example, I'm pretty sure that some ctermul attribute needs to be fixed.

Colortemplate v3 has still some rough edges, especially regarding error reporting, and it has not been optimized for performance yet. All in all, the UX should be pretty similar to what you're used to, though.

Let me know what you think.

[lifepillar]

Ok, let me look into possible solutions.

Btw, Colortemplate checks for compatibility with some imported libraries, so

I would be lucky if it explicitly error me about it.

You will get an informative message if an updated version of Colortemplate requires a newer library than you have installed. This reminds me that I need to add requirements for all the imported scripts.

[lifepillar]

So, the simplest thing would be to add Colortemplate to the vim-devel package (I would add it as an optional plugin under vim-devel/opt): then, you would just clone vim-devel and you have everything you need: packadd colortemplate and start working. If I do this, I would also add other development plugins to vim-devel, such as StylePicker or vim9asm. Then vim-devel would start looking like a real package :-D

The less appealing (to me) alternative is to copy (subtree-merge or something) the vim-devel libraries that Colortemplate needs into the Colortemplate plugin. I like this less because I would end up with two copies of the same libraries, as I need vim-devel separately anyway for other purposes. The only potential advantage (?) of this approach is that you would have to install a plugin and not a package, so that is more likely to work with plugin managers (but with packages, why would you need a plugin manager?).

In either case, you would clone one repo. Let me know what you prefer.

[lifepillar]

Also, the first solution is easy to maintain because a plugin fits in a package, so I can simply use Git subtree merge to update things. The other way round is a bit more involved: I'd probably end up just manually copying the needed scripts into Colortemplate.

[lifepillar]

Small updates:

• I've made Colortemplate an optional plugin of the vim-devel package. So, now you need to clone just one repo.
• I've fixed ctermul values in the color schemes of this PR, and another issue for when t_Co is empty.

My plan is now to test this PR using the scripts from #281. If no further issues are encountered, imo this will be safe enough to merge.

[lifepillar]

The scripts from #281 were very useful to compare 2520 screen dumps (two for each combination of color scheme, sample script, t_Co value and valid background) and found some visual differences (not too many, luckily—I hope I haven't missed any).

Note: The reference version of the color schemes I used for comparisons is from Vim 9.1.1128, so some differences may be due to changes merged into Vim after that patch.

Summary report:

Colorscheme	background	t_Co	Script	Differences
lunaperche	light	8	diff, misc	String cterm attributes undefined in reference (Colortemplate v2 bug?)
lunaperche	dark	8,16	diff, misc	String cterm attributes undefined in reference (Colortemplate v2 bug?)
retrobox	dark,light	0	diff, misc	Function is now bold
retrobox	dark,light	8,16	diff	DiffDelete was bold in reference
retrobox	dark,light	0	quickfix	QuickfixLine is now reverse instead of underlined
wildcharm	dark	gui,256	quickfix,spell	EndOfBuffer was defined twice in reference
wildcharm	dark,light	gui,256,16,8	various scripts	VertSplit is defined differently

Additional notes:

1. sample_windows.vim also resulted in different screen dumps for almost all color schemes, but afaict the differences were due to different items displayed in command-line completion (so, a textual difference, not a color scheme difference). That script should probably be updated to be more robust wrt to what is displayed on screen.
2. The difference in wildcharm likely not an issue of this PR? See VertSplit foreground color equal to background color for some color schemes #272
3. The remaining differences look like a bug (or shortcoming) in Colortemplate v2. Colortemplate v3 now requires a default definition for each highlight group, which is used for all environments unless overridden. So, for example, since QuickfixLine is not defined in _tcozero, term is set to the same value as in the default definition, while before it was probably left undefined, hence inheriting from Vim's default color scheme.

I'm now leaving it to you to decide how to proceed from here. My suggestion would be to review and merge #281 first, then double-check the report above, and finally merge this PR.

[lifepillar]

Ok, it's actually 2610 screen dumps, and there are a few more differences than I've reported above (I was mistakenly comparing some screen dumps with themself in some cases 🤦), but basically all of them are for 0- and 8-color environments. Most likely, they can be fixed by providing definitions for some missing highlight groups in _tco_zero and some more explicit 8-color definitions for some color schemes. I'll see if I can fix all of those discrepancies and report back.

[lifepillar]

Reporting back as promised. Some changes I've made to fix some discrepancies:

• Added Float, Function, Number and QuickFixLine to _tco_zero. They are all defined to generate term=NONE`. Accordingly, default definitions for those highlight groups have been added to the color schemes in which they were not present.

I consider the remaining (few) visual differences wrt the color schemes in master fixes of shortcomings in master. Specifically:

• lunaperche did not define String for dark/light 8-color terminals. This is fixed in this PR.
• lunaperche did not define Visual for 8/16-color terminals (except for the combination 16/light), falling back to the default color scheme. This is fixed in this PR.
• retrobox's template removes bold for Error in 8/16-color terminals (both backgrounds), but that definition was not applied to the color scheme. This is fixed in this PR.
• wildcharm (dark) has this definition:

Variant: gui 256
EndOfBuffer                    colorNonT      none
Variant: gui 256 16
EndOfBuffer                    color08        none

So, it is not clear what the foreground should be for gui and 256. By comparing with the light background and considering what the intent could have been here, I've opted for colorNonT by default, and color08 for 16-color terminals. This results in a different EndOfBuffer background in dark wildcharm than what is currently in Vim.

Finally, for wildcharm, the version in Vim's runtime has a NONE background for VertSplit, but the version in this repo has a solid VertSplit. I assume that this is a change that has not been pushed to Vim yet. I've kept the solid appearance.

With all that, I'm taking this PR out of draft mode as this is imo ready to be merged.

[habamax]

I've created colortemplate-v3 branch and merged this PR there.

@romainl I believe @lifepillar should have access to the repository, cc @chrisbra

[habamax]

I've updated habamax colorscheme -- now with v3 it is possible to target gui and termguicolors separately, so I tried to come up with "transparent" Visual and Diff colors:

GUI + termguicolors

Same with 256c and lesser

[habamax]

I have updated most of the colorschemes in colortemplate-v3 branch.
@romainl @neutaaaaan pls check it.

I am going to merge it to master by the end of next week if there are no objections.

[habamax]

@k-37 could you please check unokai in colortemplate-v3?

[k-37]

Hi @habamax, thank you for bringing this up to my attention.

These lines are marked as TODO in unokai.colortemplate:

; TODO: missing definitions
CursorIM        -> omit  ; TODO: TBD
Cursor          -> omit  ; TODO: TBD
lCursor         -> omit  ; TODO: TBD

But last two are defined as:

; Variant: gui
Cursor                  /gui black          white
lCursor                 /gui color00        colorlC

Can CursorIM be omitted so default value can be used?

[habamax]

Can CursorIM be omitted so default value can be used?

The -> omit part does this.

We technically don't need to do it as far as I understand but it should be harmless - Cursor/lCursor/CursorIM only has effect in gui anyways.

@lifepillar could you clarify it?

[k-37]

I should have been more clear, can we remove those lines then?

They are marked as TODO but, as I see it, nothing needs to be done about them.

So they maybe define 256, 16, 8. If that is the case this might be more clear:

; Variant: 256 16 8
CursorIM        -> omit
Cursor          -> omit
lCursor         -> omit

[habamax]

I think you can remove it.

So they maybe define 256, 16, 8. If that is the case this might be more clear:

It defines the highlight for all specified environments:

Environments: gui 256 16 8 0

And then later you redefine it for a specific one:

Cursor                  /gui black          white

[lifepillar]

In Colortemplate v3, providing a default definition for each highlight group is mandatory. In those cases where a default definition is not desired, one can use SomeHighlightGroup -> omit as the default (and provide definitions for specific environments). So, for example, with

Cursor          -> omit
Cursor                  /gui black

the Cursor highlight group is defined only in GUI Vim. You cannot just write:

Cursor                  /gui black

to achieve the same: this is an error, because no default definition for Cursor is given (/gui &co are called “overriding definitions“… they must override something!).

Cursor/lCursor/CursorIM only has effect in gui anyways.

I wasn't sure about that, so I added the TODOs for others to double-check. Are you sure that no (modern) terminal has gained the ability to tweak the cursor's color?

[lifepillar]

To elaborate further: in Colortemplate, omit is used to suppress an attribute—that is, to skip its definition altogether in the output (which is different from setting an attribute to NONE).

When used in a linked highlight group, G -> omit basically means that the (linked) highlight group definition for G should be suppressed. In other words, the default definition for G is… no definition.

[habamax]

Are you sure that no (modern) terminal has gained the ability to tweak the cursor's color?

Pretty much sure, although not 100%. Based on foot, alacritty, konsole, gnome's new console, gnome terminal, iterm2, osx terminal, windows terminal, kitty.

They can probably change cursor color using escape sequences, however it is not related to vim's Cursor highlight.

[habamax]

Merged into master. Additionally, there is colortemplate-v2 branch, just in case.