HowToCodePage

UTF-8 and character sets

Why 'one character = one byte' is wrong, what code points are, and how UTF-8 became the default for the web.

Characters are not bytes

A code point is a number assigned to a character by the Unicode standard (for example, A is U+0041, é is U+00E9, 😀 is U+1F600). An encoding decides how that number is stored as bytes. The same character can be stored differently by UTF-8, UTF-16, or UTF-32.

⚠️
Mixing encodings is the classic cause of mojibake (garbled text like é). Always declare and agree on one encoding end to end.

How UTF-8 works

UTF-8 is a variable-width encoding: ASCII characters (0–127) take exactly one byte and are identical to ASCII, which is why legacy English text keeps working. Other characters take 2–4 bytes.

Code point rangeUTF-8 bytes
U+0000 – U+007F1 byte (same as ASCII)
U+0080 – U+07FF2 bytes
U+0800 – U+FFFF3 bytes
U+10000 – U+10FFFF4 bytes
A      → 0x41            (1 byte)
é      → 0xC3 0xA9      (2 bytes)
中      → 0xE4 0xB8 0xAD  (3 bytes)
😀  → 0xF0 0x9F 0x98 0x80 (4 bytes)

UTF-8 is the web default

  • Declare it: <meta charset="utf-8"> in HTML, and Content-Type: text/html; charset=utf-8 over HTTP.
  • For APIs, JSON text is defined by the spec to be UTF-8; do not add a BOM.
  • Read files with an explicit encoding: open(f, encoding='utf-8') in Python, not the platform default.
💡
The Byte Order Mark (BOM) is meaningless for UTF-8 and often breaks parsers (JSON, shell scripts). Save UTF-8 without BOM.

FAQ

Is UTF-8 the same as Unicode?
No. Unicode is the character table (code points); UTF-8 is one way to encode those code points into bytes.
Why do I get • instead of •?
The • (U+2022) was written as UTF-8 but read as Latin-1. Re-decode as UTF-8 to fix it.

Base64 encoding URL encoding (percent-encoding)

Last refreshed 2026-09-17.