Unicode tag characters and hidden text smuggling

01What the tag block is

Unicode allocates U+E0000 to U+E007F as the tag characters. They were introduced to carry language tags inline and were deprecated almost immediately. Each printable ASCII character has a corresponding tag character at exactly 0xE0000 above its code point: the letter A, at U+0041, has a tag counterpart at U+E0041.

This makes encoding trivial. Any ASCII string can be converted character by character into a tag-block string that no browser, editor or terminal displays. Decoding is the same operation in reverse, so anything hidden this way is fully and losslessly recoverable.

02Why it matters

The technique is commonly called ASCII smuggling. A block of text that looks like a normal paragraph can carry a second, invisible paragraph inside it. Because the hidden portion is real text, anything that reads the string rather than looking at it will see the whole thing.

That includes language models. An instruction hidden in the tag block is invisible to the person pasting the text and fully legible to the model receiving it, which makes it a practical prompt-injection vector. It also survives copy and paste, which is how it typically travels.

Unlike a zero-width space, there is no legitimate reason for a tag character to appear in modern prose. Its presence is not an artefact. Something put it there.

03How to remove it

Delete the entire range. In a regular expression with the unicode flag, /[\u{E0000}-\u{E007F}]/gu matches every tag character. There is no case where preserving one is correct in ordinary text.

Worth stripping at the same time are the high variation selectors, U+E0100 to U+E01EF. These were designed to select alternate glyph forms for CJK ideographs, and they can also be chained to encode data. In text that is not deliberately typesetting rare Han variants, they carry no meaning.

The low variation selectors, U+FE00 to U+FE0F, are a different matter. U+FE0F is what makes a heart render as a colour emoji rather than a monochrome glyph, so removing that range breaks legitimate content.

04Testing whether a cleaner handles it

Encode a word into the tag block, append it to a sentence, and run the result through the tool. If the reported removal count is zero and the output still contains code points at or above U+E0000, the payload survived.

This is a useful discriminator. Many cleaners implement a short list of well-known zero-width characters and stop there, which leaves the one range that can carry a complete message.

Questions