librelist archives

adventures in i18n

From:

Joshua Bronson

Date:

2011-03-21 @ 22:29

Dear Flask users list,

I've been playing with i18n for the first time and I thought I'd mention my
experience in case it's of interest.

Before I say more, Babel integration with Flask and Jinja2 works like a
charm. Thank you to those responsible!

So on with the story. The site I'm internationalizing is static. The first
thing I decided was that I wanted each translation of the site mounted under
/<locale>/. So all the content for the English language version is under
/en/, the Chinese version is under /zh/, etc. So if you're looking at the
page /en/foo, and you want to view it in Chinese, you can just change the
url to /zh/foo.

The main reason for this is not to make it easier to switch between
translations though, it's to make the site more cacheable to avoid
unnecessary load. Some internationalized sites do this differently: they put
different translations of the same page at the same url, and rely on
sessions to store the user's locale. For instance, no matter what language
you're viewing github in, the training page is always at
https://github.com/training. Note that this is not a page that changes based
on whether you're logged in[1], so unless I'm mistaken, they're preventing
pages like this from being served by something like Varnish with this
design.

So I've got things working so that each translation of the site is mounted
under /<locale>/, and the same controller and template that render /en/foo
renders /zh/foo. If the page "foo" contains a link to another page "bar",
/en/foo will link to /en/bar, while /zh/foo will link to /zh/bar. But how
does a user coming into the site root for the first time get to the right
translation? Content negotiation. If their browser sends an Accept-Language
header, we can use request.accept_languages.best_match(AVAILABLE_LOCALES) to
find the best matching localization available, and fall back on
BABEL_DEFAULT_LOCALE, at which point we return a redirect. On all pages

Now that I've got this working more or less to my satisfaction, I invite you
to check it out at http://1.bravenewsoftware.appspot.com/. The code is at
https://github.com/jab/bravenewsoftware.org. The interesting parts are the
"set_locale" app.before_request function (http://goo.gl/vIW5O) and the
"get_locale" babel.localeselector function (just above). The
"localize_this_url" function (a few lines above that) is also handy for
generating the footer links to the other available translations of the
current page (http://goo.gl/zAOOu). These links even work for custom error
handlers (which of course are localized), including on pages that give 404:

http://1.bravenewsoftware.appspot.com/zh/notfound

Tip: I found
https://addons.mozilla.org/en-us/firefox/addon/quick-locale-switcher/ to be
a handy way to test the content negotiation.

One remaining annoyance I noticed is that if a browser sends only something
like "en-us" as its Accept-Language header, without also the more general
"en" (I'm glaring at you, Safari), and you call
request.accept_languages.best_match(["en"]), it will return None instead of
"en". This is inferior to babel's built-in negotiation (http://goo.gl/pi7ra
):

>>> preferred = ['zh_cn']
>>> available = ['zh']
>>> babel.Locale.negotiate(preferred, available)
<Locale "zh">

Would it make sense for werkzeug to be smarter about this, or should you
just use babel if you want better negotiation?

Thanks for reading.

-Josh

[1] except for the log in/log out controls at the top, which can be taken
care of with Javascript and a cookie to keep the server response static.

On Mon, Mar 21, 2011 at 6:29 PM, Joshua Bronson <jabronson@gmail.com> wrote:

> I've been playing with i18n for the first time and I thought I'd mention my
> experience in case it's of interest.
>

Figured out a nicer way to do the routing since last I wrote. Just thought
I'd send along the interesting code in case it might help anyone else:

From

https://github.com/jab/getlantern.org/blob/master/appengine/application/__init__.py
:

# matches all available language codes
LC = '/<any(%s):langcode>' % ', '.join(LOCALEMAP)

# / -> /<langcode>/
url('/', 'views.render_page')

# /<DEFAULT_PAGE> -> /<langcode>/
url('/' + DEFAULT_PAGE, 'views.render_page')

# /<langcode>/<DEFAULT_PAGE> -> /<langcode>/
for i in LOCALEMAP:
    url('/%s/%s' % (i, DEFAULT_PAGE), 'views.render_page')

# /<langcode>/ shows default page
url(LC + '/', 'views.render_page')

# /<page>/ -> /<langcode>/<page>
url('/<page>', 'views.render_page')

# /<langcode>/<page>
url(LC + '/<page>', 'views.render_page')




From
https://github.com/jab/getlantern.org/blob/master/appengine/application/views.py
:

def render_page(langcode=None, page=DEFAULT_PAGE):
    # don't allow access to error templates (which begin with a number) or
    # non-base-extending templates (which begin with underscore)
    if page[:1] not in lowercase:
        raise NotFound
    try:
        return render_template(page + '.html')
    except TemplateNotFound:
        raise NotFound



And from

https://github.com/jab/getlantern.org/blob/master/appengine/application/setup_app.py
:

@app.before_request
def set_locale():
    view_args = req.view_args or {}
    page = view_args.get('page')
    langcode = view_args.get('langcode')
    g.locale_requested = langcode and LOCALEMAP[langcode]
    g.locale_negotiated = Locale.negotiate(imap(itemgetter(0),
        req.accept_languages), LOCALEMAP, sep='-')
    g.locale = g.locale_requested or g.locale_negotiated
    if not g.locale:
        g.locale = DEFAULT_LOCALE
        g.locale_unsatisfied = True
    if req.endpoint == 'render_page' and g.locale_requested is None:
        return redirect(url_for_page(page, langcode=g.locale.language))



(All code distributed under the terms of the GPLv2.)

On Sat, Apr 30, 2011 at 10:20 PM, Joshua Bronson <jabronson@gmail.com>wrote:

> Figured out a nicer way to do the routing since last I wrote.
>

For the record, just wanted to say I switched over to using blueprints for
this as documented in

http://flask.pocoo.org/docs/patterns/urlprocessors/#internationalized-blueprint-urls(wish
I'd seen this earlier!) and it's totally the way to go.