https://txtfetch.com/glossary/character-encoding/
Character encoding
The rulebook mapping bytes to characters, so reading a file with the wrong one can turn every character wrong, even though every byte survives intact.
definition
Character encoding is a mapping between numeric byte values and the characters they represent, letting a computer store and interpret text as bytes.
Also called: text encoding, charset
in-plain-terms
Text isn't stored as letters. It's stored as bytes, and an encoding is the agreed rule for which byte sequences represent which characters. UTF-8 is the modern default and can represent any Unicode character using one to four bytes. Legacy single-byte encodings like Windows-1252 only cover a few hundred characters each, and different encodings assign different characters to the same byte value.
Reading a byte stream with the wrong encoding doesn't lose data, since the original bytes are still there. It produces the wrong characters instead. That's the entire mechanism behind mojibake: correct bytes, wrong rulebook applied when decoding them.
why-it-matters
- A file with no declared encoding forces a reader to guess. Reliable detection handles most real files, but any guesser can occasionally pick wrong on a short or unusual byte sequence.
- Re-decoding with the right encoding fully recovers the original text when the bytes are intact. Mojibake is reversible in a way a genuinely undecodable byte, which becomes a replacement character, is not.
how-to-check
- Scan extracted text for mojibake or replacement-character patterns. This is a quick signal that the wrong encoding was assumed somewhere in the pipeline. Scan extracted text for encoding damage
related-terms
faq
- What's the difference between an encoding and a font?
- An encoding maps bytes to characters, deciding which character a given byte sequence represents. A font then maps each character to a visual glyph shape. Getting the encoding wrong produces the wrong characters entirely; a font issue only affects how a correct character looks.
- Why does the same file sometimes decode differently in different tools?
- Because a byte stream doesn't announce its own encoding. A reader that assumes the wrong one, or guesses wrong when none is declared, produces different, wrong characters than a reader that assumes correctly. This happens even though both readers see identical bytes.
related-reading
Characters & encoding
See the term in real output.
Drop a file into the free reader and watch it happen.
Open the file reader →