Skip to content

fix: full unicode filename handling in 'writeUTF8' and 'sizeUTF8' - #21

Open
mehradotdev wants to merge 1 commit into
photopea:masterfrom
mehradotdev:fix.unicode-filename
Open

fix: full unicode filename handling in 'writeUTF8' and 'sizeUTF8'#21
mehradotdev wants to merge 1 commit into
photopea:masterfrom
mehradotdev:fix.unicode-filename

Conversation

@mehradotdev

Copy link
Copy Markdown

Note: This PR is just for reference. Added a lot of comments to explain the code changes. Feel free to close it. If it doesn't suit your coding style.

Summary

Handle full Unicode code points correctly when encoding ZIP entry names.
This changes writeUTF8 and sizeUTF8 so they process full Unicode code points instead of raw UTF-16 code units.

It adds a small helper:

readUnicodeCodePoint(str, ci) //'ci' means character index.

and uses it from both UTF-8 writer functions.

Why this is needed?

The current implementation iterates over str.charCodeAt(ci), which reads UTF-16 code units one at a time. That works for BMP characters, but it breaks for characters represented by surrogate pairs.

Examples that currently fail:

  • rare-kanji-𠮷.txt
  • emoji-😀.txt

Those names are encoded as invalid UTF-8 byte sequences, and when parsed back they become mojibake instead of the original filename.

No other ZIP logic is changed.

Benefit

  • Correct round-trip handling for full Unicode filenames
  • Fixes ZIP entry names containing emoji and other non-BMP characters
  • Keeps existing behavior for ASCII and BMP text unchanged

Example behavior

BMP names like 日本語.txt work as expected even before this change however Non-BMP are broken.

Before:

  • rare-kanji-𠮷.txt round-trip as rare-kanji-í¡í¾·.txt
  • emoji-😀.txt round-trip as emoji-í ½í¸.txt

After:

  • rare-kanji-𠮷.txt round-trips correctly
  • emoji-😀.txt round-trips correctly

Suggested validation

Encode and parse archives containing names like:

  • 日本語.txt
  • 한국어.txt
  • emoji-😀.txt
  • rare-kanji-𠮷.txt

Then verify the parsed keys exactly match the original names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant