Skip to content

Handle surrogate pair when converting from LSP encoding #1785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Handle surrogate pair when converting from LSP encoding #1785

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6912f9e
Handle surrogate pair when converting from LSP encoding
bullno1 Jun 6, 2025
ecd1bb8
Perform a range check before accessing the high byte
bullno1 Jun 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions ycmd/completers/language_server/language_server_protocol.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -823,6 +823,13 @@ def UTF16CodeUnitsToCodepoints( line_value, code_unit_offset ):
# is one-past-the-end of the string in unicode codepoints
return len( line_value ) + 1

if byte_offset_utf16 < len( value_as_utf16_bytes ):
hi_byte_at_offset = value_as_utf16_bytes[ byte_offset_utf16 + 1 ]
if 0xdc <= hi_byte_at_offset <= 0xdf:
# If we are in the middle of a surrogate pair, then advance the offset
# further to skip the pair
code_unit_offset += 1

bytes_included = value_as_utf16_bytes[ : code_unit_offset * 2 ]
return len( bytes_included.decode( 'utf-16-le' ) )

Expand Down
11 changes: 11 additions & 0 deletions ycmd/tests/language_server/language_server_protocol_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,3 +204,14 @@ def test_CodepointsToUTF16CodeUnitsAndReverse( self ):
equal_to( code_units ) )
assert_that( lsp.UTF16CodeUnitsToCodepoints( line_value, code_units ),
equal_to( codepoints ) )

def test_CodepointsToUTF16CodeUnitsSurrogate( self ):
for line_value, codepoints, code_units in [
( '😉', 1, 1 ),
( 'f😉', 2, 2 ),
]:
with self.subTest( line_value = line_value,
codepoints = codepoints,
code_units = code_units ):
assert_that( lsp.UTF16CodeUnitsToCodepoints( line_value, code_units ),
equal_to( codepoints ) )
Loading