Text Encoding Menu in 2021

Back in December 2014, I wrote a post called Character Encoding Menu in 2014. Now seems like a good time to take a new snapshot for the record. Copypaste from the old intro still applies: This post is about a UI feature that I wish no one would have to use. Happily, it is indeed almost unused. Still, I made it more usable in the case when it is used.

The 2014 Menu

To recap, at the end of 2014 the Character Encoding menu in Firefox looked like this:

The 2021 Menu

In mid-January 2021, the Text Encoding menu in Firefox looks like this:

There are four differences made as four distinct steps over time:

  1. The menu is now called “Text Encoding” (as in Safari) as opposed to “Character Encoding” on the assumption that the new name suggests the purpose better to users who are not familiar with the terminology.
  2. The three Japanese encodings have been replaced with a single item that autodetects among those three.
  3. The “Auto-Detect” submenu is gone. Instead of there being three locale-specific detection modes, chardetng is now enabled for everyone without UI to turn it off.
  4. There is a new first item called “Automatic”. This item runs chardetng even for labeled pages with UTF-8 allowed as a possible outcome and with the top-level domain of the page ignored.

Where to Go from Here

Does it make sense to have items other than “Automatic”? Probably not. After all, Chrome and Firefox for Android don’t have the menu at all.

For users who have telemetry enabled, we collect data about whether the item “Automatic” was used at least once in given Firefox subsession, whether an item other than “Automatic” was used at least once in a given Firefox subsession, and a characterization of how the encoding that is being overridden was determined (from HTTP, from meta, from chardetng running without the user triggering it, from chardetng as triggered by the user by having chosen “Automatic” previously, etc.). If things go well, the telemetry can be analyzed when Firefox 87 is released (i.e. when 86 has spent its time on the release channel). The current expectation for this is 2021-03-23.

Additionally, the intent is to start honoring the encoding declaration in an XML declaration in text/html soon. The spec says not to do this, which is why Firefox does not do this at present, but the plan is to change the spec. This will eliminate a reason for Firefox to differ from Chrome and Safari on the encoding topic.

The expectation is that the telemetry will show that the single menu item “Automatic” can replace the whole menu. Obviously, that would involve renaming the item to something like “Override Text Encoding”. Hopefully, there won’t be a Text Encoding menu in 2022.