3. Encoding Results

Unicode Character Inspector

Character Encoding Formulas

Character encoding converts abstract characters into bytes. Unicode assigns a code point such as \(U+0041\) or \(U+1F44B\), and an encoding such as UTF-8 or UTF-16 decides how that code point becomes bytes or code units.

UTF-8 is variable length. It uses one to four bytes depending on the code point range:

UTF-16 uses 16-bit code units. Characters in the Basic Multilingual Plane use one code unit. Supplementary characters use a surrogate pair:

UTF-32 uses one 32-bit value per code point:

Base64 encodes three bytes into four printable characters:

Percent encoding writes each byte as a percent sign followed by two hexadecimal digits:

Unicode normalization converts equivalent text into a consistent representation:

Complete Guide to Character Encoding

Character encoding is the system that turns human-readable text into bytes computers can store, transmit, and interpret. A letter, symbol, emoji, mathematical sign, Arabic word, Hindi phrase, or Chinese character is not stored as a visual drawing. It is represented by a code point and then encoded into bytes. Without a correct encoding, text can become unreadable, corrupted, or misinterpreted.

Unicode is the central standard for modern text. It assigns code points to characters across languages and scripts. For example, the Latin capital letter A is \(U+0041\), the euro sign is \(U+20AC\), and the waving hand emoji is \(U+1F44B\). A code point is an abstract number. It does not by itself say how many bytes are stored. That is the job of an encoding such as UTF-8, UTF-16, or UTF-32.

UTF-8 is the dominant web encoding because it is compact for ASCII text, supports the full Unicode range, and avoids many byte-order problems. ASCII characters from 0 to 127 encode as one byte in UTF-8. Many European and Middle Eastern characters use two bytes. Many Asian characters use three bytes. Emoji and other supplementary characters usually use four bytes. This variable-length design makes UTF-8 efficient and backward-compatible with ASCII for basic English text.

UTF-16 is common inside several programming environments, including JavaScript strings. UTF-16 stores text as 16-bit code units. Characters in the Basic Multilingual Plane use one code unit. Supplementary characters such as many emoji use two code units called a surrogate pair. This is why a JavaScript string length can be larger than the number of visible characters. The emoji 👋 is one Unicode code point but two UTF-16 code units.

UTF-32 is simpler conceptually because it uses one 32-bit unit per Unicode code point. Its simplicity comes with size cost. A basic English letter that needs one byte in UTF-8 takes four bytes in UTF-32. UTF-32 is useful for some internal processing and teaching, but it is not usually the best storage or web interchange format.

ASCII is the historical 7-bit encoding for basic English letters, digits, punctuation, and control characters. ASCII contains 128 values from 0 to 127. UTF-8 was designed so that ASCII bytes mean the same thing in UTF-8. That compatibility is one reason UTF-8 became practical for web adoption. However, ASCII cannot directly encode characters such as €, é, नमस्ते, مرحبا, or emoji.

Latin-1, also known as ISO-8859-1 in many contexts, extends the idea of single-byte text to 256 values. It can represent many Western European characters but not the full range of Unicode. Windows-1252 is a related legacy encoding used historically in Windows environments. Many mojibake problems happen when text encoded in one legacy encoding is decoded as another.

Mojibake is the visible corruption that happens when bytes are decoded with the wrong character encoding. For example, UTF-8 bytes for one character can appear as two or three strange characters if interpreted as a legacy single-byte encoding. The fix is not to visually repair the text by hand; the correct fix is to decode the original bytes using the correct encoding whenever possible.

A byte order mark, or BOM, is a special sequence at the start of some text files. It can signal the byte order for UTF-16 or UTF-32. UTF-8 does not need byte order, but some files still include a UTF-8 BOM. BOM handling can matter when reading CSV files, source code, configuration files, and data exports. This tool lets you add or strip BOMs for encoding and decoding experiments.

Endianness describes byte order. In UTF-16LE, the least significant byte appears first. In UTF-16BE, the most significant byte appears first. The same idea applies to UTF-32LE and UTF-32BE. If a file is decoded with the wrong endianness, the result may look completely broken. Understanding endianness is important in binary formats, networking, file parsing, and systems programming.

Base64 is not a character encoding like UTF-8. It is a binary-to-text encoding that represents bytes using printable characters. It is useful when binary data must pass through systems designed for text. Base64 is common in email, data URLs, tokens, and APIs. It is not encryption. Anyone can decode Base64 if they have the encoded text.

Percent encoding is used in URLs. A byte is written as a percent sign followed by two hexadecimal digits, such as %E2%82%AC for the UTF-8 bytes of the euro sign. URL encoding is byte-oriented, so the text must first be encoded into bytes, usually UTF-8, and then each unsafe byte is written using percent notation.

HTML entities are another representation layer. The character © can be written as ©, ©, or ©. Entities are useful when text must appear inside HTML without being interpreted as markup, or when a developer wants to show special characters safely. HTML escaping is also a security habit when displaying user text in a web page.

Unicode normalization is essential for reliable text comparison. The character é can be represented as one precomposed character or as the letter e followed by a combining accent. These may look the same but have different code point sequences. NFC, NFD, NFKC, and NFKD normalize text in different ways. Search, sorting, file names, usernames, and duplicate detection can behave incorrectly if normalization is ignored.

Grapheme clusters are what users often think of as characters. A single visible emoji can contain multiple code points joined together. A flag emoji uses regional indicator symbols. A family emoji may use several people emojis joined with zero-width joiners. A letter with a combining mark can be multiple code points but one visible unit. This is why the tool reports characters, code points, UTF-16 code units, grapheme clusters, and bytes.

JavaScript strings use UTF-16 code units internally. This means string.length reports code units, not Unicode code points or grapheme clusters. This matters when validating usernames, limiting posts, slicing emoji text, or storing multilingual text. Cutting a string at a random code-unit boundary can split a surrogate pair and create invalid text.

Developers should usually store and exchange text as UTF-8 unless there is a specific reason not to. HTML pages, APIs, JSON files, CSV exports, XML, Markdown, source files, logs, and databases are easier to handle when UTF-8 is used consistently. Legacy encodings may still appear in old files, regional datasets, archived systems, or exports from older software.

Character encoding is also a mathematical topic. It uses base conversion, binary representation, hexadecimal notation, bit masks, byte grouping, modular ranges, and variable-length coding. When a code point is encoded as UTF-8, its bits are distributed into byte templates. When bytes are displayed as hexadecimal, each byte is split into two 4-bit nibbles. Encoding is not magic; it is structured arithmetic.

This tool is useful for students learning Unicode, developers debugging mojibake, teachers preparing computer science lessons, publishers checking multilingual content, SEO teams working with URLs, and anyone who needs to see the actual bytes behind text. It is intentionally transparent: you can see the text, the bytes, the code points, and the formulas.

This page is not an official exam score calculator. There is no universal score guideline, score table, or next exam timetable for character encoding conversion. It can support computer science, web development, digital publishing, data handling, cybersecurity basics, and applied mathematics, but official exam schedules and grading rules must come from the relevant school, course provider, or exam board.

Reference Links

Useful references: Unicode Standard, WHATWG Encoding Standard, MDN TextEncoder, MDN TextDecoder, and MDN String normalize.

How to Use Character Encoding Converters

1. Choose a tool mode. Encode text, decode bytes, inspect characters, normalize text, or use escape tools.
2. Select an encoding. Choose UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE, ASCII, Latin-1, or Windows-1252 style.
3. Enter text or bytes. Encode mode expects text. Decode mode expects hex, binary, decimal, Base64, or percent-encoded bytes.
4. Set BOM and error options. Add BOM, strip BOM, replace unsupported characters, or use strict mode.
5. Run conversion. Click Convert Encoding and review all outputs.
6. Inspect Unicode details. Review each character’s code point, decimal value, UTF-8 bytes, UTF-16 units, and HTML entity.
7. Export results. Copy the main output, copy all details, download CSV, or print/save as PDF.

Score, Course, and Exam Table Note

Character Encoding Converter FAQ