Uppercase and Lowercase in Unicode: Why Case Is Harder Than It Looks

Case is not universal

Many writing systems have no case at all. Latin, Greek, Cyrillic and Armenian have uppercase and lowercase forms, and so do a few smaller scripts such as Coptic, Cherokee and Georgian. Chinese, Japanese, Korean, Arabic, Hebrew, Hindi and Thai have no case, so a case converter leaves text in these scripts unchanged.

Case is also younger than writing. Latin capitals are the older forms, and lowercase letters developed in the early Middle Ages. Georgian is a recent example: the modern capital-like letters called Mtavruli were added to Unicode in version 11 in 2018, and Cherokee lowercase letters were added in 2015, in Unicode 8.0, because Cherokee was first encoded with capital forms only.

One letter, several letters: the German ß

The German letter ß, the sharp s, is lowercase only. It has no capital in everyday typing, and its uppercase form is the two letters SS. That means that converting Straße to capitals gives STRASSE, and that the text becomes one character longer. Programming languages follow this: in JavaScript, 'ß'.toUpperCase() returns 'SS'.

A capital sharp s, ẞ, exists: it was added to Unicode in version 5.1 in 2008 as U+1E9E, and the Council for German Orthography accepted it in 2017 as an alternative form in the official rules. It is still rarely used in practice, and standard case conversion produces SS. Other characters expand in the same way: the ligature fi becomes FI and the letter ʼn becomes ʼN. Code that assumes that uppercase and lowercase strings have equal length will fail.

Turkish and the dotted i

In Turkish and Azerbaijani, the letter i has two forms: the dotted i and the dotless ı. The capital of the dotted i is İ, with a dot, and the capital of the dotless ı is I. In most other languages, the capital of i is I and the lowercase of I is i., so a program that applies the English rule to Turkish text produces wrong words.

Programmers know this as the Turkish i problem. In JavaScript, 'i'.toLocaleUpperCase('tr') returns İ, while 'i'.toUpperCase() returns I. In Java, calling toUpperCase() without a locale on a Turkish computer has caused real bugs, for example in the processing of file extensions and keywords. The fix is to use a fixed locale for identifiers and a user locale only for text shown to people.

Greek final sigma

Greek has two lowercase forms of the letter sigma: σ in the middle of a word and ς at the end. Both have the same capital, Σ. Converting to uppercase is simple, but converting back is context-sensitive: the program must look at the position of the letter in the word to decide between σ and ς.

Unicode describes this in its rules for special casing, and modern engines follow them: in JavaScript, 'ΟΔΟΣ'.toLowerCase() returns 'οδος' with a final ς. Simple tools that map each letter on its own produce οδοσ, which looks wrong to Greek readers.

Title case letters and digraphs

Unicode contains a few digraph letters that have a separate title case form. The letter Dž, in Latin script for Croatian and Serbian, has three forms: DŽ in capitals, dž in lowercase and Dž in title case, which is used when only the first letter of a word is capitalized. The three forms are separate characters, U+01C4, U+01C6 and U+01C5.

Dutch ij is a different case. Unicode has the ligature characters IJ and ij, but they are rarely used in normal text. Instead, Dutch spelling has a rule: in a word that begins with ij, both letters are capitals, as in IJsselmeer. The converter on this page applies this rule when the page language is Dutch.

Case folding and comparison

Case folding is related to lowercasing, but it is designed for comparison, not for display. For example, the German ß folds to ss, so that the strings Straße and STRASSE compare as equal under full case folding. Python offers str.casefold() for this purpose, and it is more aggressive than lower().

If you compare user input, such as a username or an email address, do not use a plain uppercase or lowercase conversion. Use case folding, or the comparison function of your database, and normalize the text to a standard Unicode form first. Otherwise, strings that look identical can compare as different.

What this means for a case converter

A good case converter uses the language of the page when it converts, relies on the Unicode data of the browser and accepts that the output can be longer than the input. The converter on this page uses the toLocaleUpperCase and toLocaleLowerCase functions of the browser, so the German ß becomes SS, the Greek final sigma is handled correctly in lowercase, and the language of the page sets the rules.

Some limits remain. The converter does not know the language of a pasted text, only the language of the page, so Turkish text pasted into an English page is converted with English rules. When you work with such text, check the letters İ and ı after conversion.