fix: use utf8proc to detect emojis #29645
Conversation
|
I think this new solution should be superior but it wouldn't hurt if some emoji experts/connoisseurs tested this PR. |
github-actions
bot
added
build
|
How is this change breaking precisely? |
Previously, neovim counted a codepoint as an emoji if it were part of the group "Emoji", "Emoji_Presentation", "Emoji_Modifier", "Emoji_Modifier_Base" and "Emoji_Component" in unicode.org/reports/tr51. In this PR, neovim counts an emoji if it is part of the "Extended_Pictographic". I am not 100% sure what this entails in practice. Here's alist of changed codepoints.Note that this list is not exhaustive as all emojis in |
|
Could also count as "fix" (not breaking) if we declare utf8proc as the source of truth for this (which arguably we should)? |
More accurately, it will check if the codepoint has the "Extended_Pictographic" property, which according to https:// unicode.org/reports/tr51/#Emoji_Properties_and_Data_Filesis described as: "The Extended_Pictographic characters contain all the Emoji characters except for some Emoji_Component characters." This should in most cases align with what people refer to when they say "emoji".
dundargoc
force-pushed
the
build/utf8proc/emoji
branch
from
198f051
to
a6e5e5e
Compare
dundargoc
changed the title
|
It's strange to replace |
|
So it turns out it's not possible to determine from codepoint alone if a character is an emoji, as there are codepoints that can be both text and emoji depending on variant selector(?). Closing for the time being until we have a better solution in mind. |
More accurately, it will check if the codepoint has the
"Extended_Pictographic" property, which according to
https:// unicode.org/reports/tr51/#Emoji_Properties_and_Data_Filesis
described as:
"The Extended_Pictographic characters contain all the Emoji characters
except for some Emoji_Component characters. "This should in most cases
align with what people refer to when they say "emoji".