Syntax Look Get Syntax Look
Product menu

Syntax Look

Why your file previews as garbled characters

Last updated:

You preview a file and the text is mostly right, but every accented letter, curly quote and dash has turned into a small cluster of nonsense. café becomes café. An em dash becomes â€".

Nothing is damaged. The bytes are being read with the wrong alphabet.

Why it happens

Text files do not record what encoding they use. A reader has to guess, and almost everything now guesses UTF-8, which is the right guess nearly all of the time.

But plenty of files were not written that way. Anything exported from an older Windows tool is often Windows-1252, where one byte is one character. In UTF-8 those same bytes are the start of a multi-byte sequence, so the reader assembles them into the wrong characters and prints exactly the mojibake above.

The reverse case exists too. A UTF-16 file read as UTF-8 shows a null byte between every letter, which usually renders as boxes or gaps.

The tell

The pattern tells you which one you have.

  • Latin text that is fine except for accents and punctuation: a single-byte encoding read as UTF-8. Windows-1252 nearly always.
  • Every second character missing or boxed: UTF-16 read as UTF-8.
  • A stray  at the very start: a UTF-8 byte order mark being shown rather than consumed.

What Syntax Look does

It checks for a byte order mark first, which settles UTF-8 and UTF-16 outright. Failing that it reads as UTF-8, and when the bytes are not valid UTF-8 it falls back to Windows-1252 rather than printing replacement characters.

That covers the common case: an old export opens and reads correctly instead of looking corrupted.

If a file still previews wrongly, it is likely in an encoding outside that set—a legacy Mac Roman file, or one of the East Asian multi-byte encodings. Converting it once is the durable fix:

iconv -f SHIFT-JIS -t UTF-8 old.txt > new.txt

Keep the original until you have checked the result.