Unicode characters clipped #1110

Activity

1. ychin commented on Oct 18, 2020

   

   ychin

   on Oct 18, 2020

   Last edited by ychin

   Member

   More actions

   Please don't turn off Core Text Renderer. As you said it's deprecated and doesn't work properly and the option will be removed soon.

   #858 should fix this but we are still ironing some issues with it.

   Ultimately though, with how Vim rendering works, every character has to be aligned on the column, and it doesn't really do well on things like 𝒫(Ω) because the "𝒫" just takes up too much space. Even with the space it will still look a little weird with it poking into the next character.
2. ychin commented on Oct 18, 2020

   

   ychin

   on Oct 18, 2020

   Member

   More actions

   Alternatively, you can use the new-ish feature and do call setcellwidths([[0x1d4ab, 0x1d4ab, 2]]).

   0x1d4ab is the unicode number for "𝒫", and doing this allows you to make it take up two columns instead of one to reduce any clipping and make it look nicer as well. You can use Vim command ga when cursor is on a character to see what the code is.
3. postylem commented on Oct 18, 2020

   

   postylem

   on Oct 18, 2020

   Author

   More actions

   I see, thanks for the response! For now, is there a way to get macvim to just show the characters overlapping, rather than clip them (that is, to get the same behaviour as in terminal vim, in my second screenshot)?

   Custom-adjusting the width with setcellwidths() for each glyph I want to use for my primary use-case it isn't useful, because it doesn't work for the glyphs which are put there by tex-conceal (it seems it only works for glyphs that are typed in as unicode characters, not ones that are displayed to 'conceal' tex code... I don't know enough about that to understand why).
4. ychin commented on Oct 18, 2020

   

   ychin

   on Oct 18, 2020

   Member

   More actions

   I see, thanks for the response! For now, is there a way to get macvim to just show the characters overlapping, rather than clip them (that is, to get the same behaviour as in terminal vim, in my second screenshot)?

   No, not for now until the pull request I mentioned above gets merged.

   because it doesn't work for the glyphs which are put there by tex-conceal (it seems it only works for glyphs that are typed in as unicode characters, not ones that are displayed to 'conceal' tex code... I don't know enough about that to understand why).

   Examples? I think it should work regardless how the text gets there. If you are using Terminal Vim for example, you can shift-select to select the raw text and paste it here.
5. postylem commented on Oct 19, 2020

   

   postylem

   on Oct 19, 2020

   Last edited by postylem

   Author

   More actions

   Ah it does work, I was messing up the syntax. Thanks!
   There's still a small difference betwen the characters that show up because of tex-conceal, and the characters that are entered directly, when there is a newline after (see below).
   I'll continue to work with this for now, setting a list of characters to display as double width as I need.

   ---

   For reference:
   To make both 𝓟 (0x1d4d0) and 𝓐 (0x1d4df) be double-wide, I call
   :call setcellwidths([[0x1d4d0, 0x1d4d0, 2], [0x1d4df, 0x1d4df, 2]])

   This is before:
   
   As you can see,

   • glyphs displayed by tex-conceal show clipping if followed by a ascii character or newline, and overlap if followed by another conceal-replaced glyph
   • glyphs which were directly entered to the file are only clipped by newline

   and this is after calling setcellwidths for those two unicode codepoints
   

   ---

   For comparison, this is what it looks like in terminal Vim (iTerm2). No clipping ever, just overlap.
   

   Note: if I call :call setcellwidths([[0x1d4d0, 0x1d4d0, 2], [0x1d4df, 0x1d4df, 2]]) in terminal vim I get some odd things:
   
   It seems not to be playing very well with the conceal stuff.
6. ychin commented on Oct 20, 2020

   

   ychin

   on Oct 20, 2020

   Last edited by ychin

   Member

   More actions

   So I'm confused. Both screenshots look similar. Are they both done in the terminal? Can you provide more info? It's hard to debug through screenshots.

   Info include:

   • How you are producing the screenshots and vimrc / commands to reach each screenshot. (The before and after)
   • Copy paste the actual text and past it here.

   If one screenshot is GUI and the other one is terminal that just means it's a terminal issue, not conceal-related. (And I could repro that in terminal as well)

   I think the actual issue is setcellwidth is buggy, and tbh should the discussion should be at Vim itself instead of this issue.
7. postylem commented on Oct 22, 2020

   

   postylem

   on Oct 22, 2020

   Last edited by postylem

   Author

   More actions

   Sorry to be confusing. To clarify: the first two screenshots in my previous post were with macvim, and the second two were with terminal vim (iTerm2).

   The setcellwidth solution works for me. But just to document clearly the behaviour I was decribing, for the benefit of anyone reading this later, here goes:

   Here is a .tex document:

   \documentclass[11pt]{standalone}
   \usepackage{fontspec,mathrsfs}
   \setmainfont{STIXGeneral-Regular}
   \begin{document}
   Unicode characters: 𝓟(Ω) 
                       𝓟₂ and 𝓟𝓐
                       𝓟 and 𝓐
   
   Concealed latex   : $\mathscr{P}(\Omega)$
                       $\mathscr{P}_2$ and $\mathscr{P}\mathscr{A}$
   		    $\mathscr{P}$ and $\mathscr{A}$
   \end{document}

   When I use macvim to edit this file, I see the following (with tex-conceal), where

   • underlying unicode wide characters overlap the next character unless it is a newline
   • tex-conceal rendered wide characters are clipped when followed by an asci character or a newline, but not if followed by another non-ascii char (like ₂ or 𝓐).

   

   and after running my cursor across the text:

   

   Now, if I run :call setcellwidths([[0x1d4d0, 0x1d4d0, 2], [0x1d4df, 0x1d4df, 2]]) to set the cellwidths to double for the two offending characters (U+1D4DF: MATHEMATICAL BOLD SCRIPT CAPITAL P, and U+1D4D0: MATHEMATICAL BOLD SCRIPT CAPITAL A), then I see
   
   This is a workable fix for me. I'll close the issue. Thank you.

   There is still a question about why the unicode characters rendered from tex-conceal show a slightly different behaviour than the plain unicode characters, but I suppose that's due to something related to the fact that when I pass my cursor over the text the clipping shows up (I can't pass the cursor over the concealed text, because it becomes un-concealed when the cursor is on that line).

   ---

   My possibly relevant vimrc lines

   " tex-conceal (KeitaNakamura/tex-conceal.vim) settings 
   set conceallevel=2
   let g:tex_conceal='abdmg'
   let g:tex_conceal_frac=1
   hi Conceal ctermbg=none
8. postylem

   closed this as completedon Oct 22, 2020