Accept-Charset Is No More

Now that Firefox 10 has been released, none of the major browsers send only Chrome sends the Accept-Charset HTTP header.

During the Firefox 4 development cycle, I noticed that IE, Chrome and Safari were not sending the Accept-Charset HTTP header in their HTTP requests. This meant that the Web had to work even without browser sending that header.

It was already obvious that Accept-Charset was obsolete as a feature and was a waste of bytes in each HTTP request. If your server has the capability to vary what character encoding it uses, it could use UTF-8. All browsers support UTF-8. There is no way for any non-UTF-8 encoding to represent something more, since everything gets converted to UTF-16 internally. Thus, the browser and the server negotiating a suitable encoding is pointless.

There was indication that Opera was going to drop Accept-Encoding and indeed they have dropped it by now. In the case of Firefox, the change was not allowed in Firefox 4. Ms2ger removed Accept-Charset and landed the code for Firefox 6.

However, it turned out that Yahoo! Search and Yahoo! Babelfish malfunctioned when the browser identified itself as Firefox but did not send Accept-Charset. Since the sites worked in IE, Chrome and Safari, they obviously had to have code paths that functioned without Accept-Charset. However, the code paths were conditional on the UA string instead of the presence of Accept-Charset. Sigh.

To give Yahoo! time to fix their stuff, the change was backed out before release from Firefox 6. And then Firefox 7. And Firefox 8. And Firefox 9. The change was finally shipped in Firefox 10. Yahoo! Search had been fixed. Yahoo! Babelfish is still broken as of 2012-02-06.

Anyway, now none of Firefox, Opera, Safari, Chrome or IE send the Accept-Charset header. Yay for smaller HTTP requests.

What Can We Learn?

• As a server talking to a browser, always use UTF-8. No exceptions. Use the lang attribute for CJK disambiguation.
• It is hard to take cruft out of the HTTP request boilerplate. Resist the urge to put new cruft there.
• It is a bad idea to assume that a certain feature is present because the UA string looks a certain way. Check for the presence of the feature instead!
• If you defer a browser change in order to give a site the time to make a trivial adjustment, you may end up deferring the change many times.

Correction About Chrome

Contrary to what I originally wrote, Chrome still sends Accept-Charset as of 2012-02-06. Right before I wrote this post, I tested something encoding-related in Chrome, thought I had tested this in Chrome, too, but it turns out I only tested this in Safari 5.1.3, IE9, Firefox 10 and Opera 11.61 and failed to test Chrome. How embarrassing. Sorry.