Software Internationalization (I18n) With GNU Gettext

foss.
my 2009
Welcome
Software Internationalization (i18n) with GNU gettext
Muhammad Najmi Ahmad Zabidi
Department of Computer Science

KICT, International Islamic University Malaysia,
MALAYSIA
25th October 2009
najmi@kict.iiu.edu.my
Created with LATEX
Muhammad Najmi () foss.my 2009 October 26, 2009 1 / 48

foss.my 2009
About
Agenda
1 About
2 Internationalization
3 gettext
4 Task of developer/translator
5 Flow of the translation process
6 Codes
7 Tools
8 Route of PO to MO
9 Issues with package generated by gettext
10 Conclusion

foss.my 2009
About
About the speaker

KDE Subversion committer and original translator to ms MY

foss.my 2009
About
About the speaker

Translation CVS committer for Tuxpaint project

foss.my 2009
About
About the speaker

Occasionally sending translation for GNU ms MY, (Munsyi
Project), lead by Sharuzzaman

foss.my 2009
About
About the speaker

Occasionally sending translation for GNU ms MY, (Munsyi
Project), lead by Sharuzzaman

foss.my 2009
Internationalization
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
What actually internationalization is:

Software going global

foss.my 2009

Software package getting “world” acceptance, thus people willing
to localize it

foss.my 2009

Software package getting “world” acceptance, thus people willing
to localize it
Needs i18n support first!

foss.my 2009
Localization vs Internationalization
The differences. . .
Localization
Done by translator
It can be by the
user/Developer
Submit to package
maintainer
Get credit

foss.my 2009
Localization vs Internationalization
The differences. . .
Localization
Developer
Done by translator
Prepare package
It can be by the
Merge package
user/Developer
Update package
Submit to package
maintainer Announce new
package
Get credit
Give proper credit

foss.my 2009
gettext
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
gettext
gettext
gettext’s features
Part of GNU packages
Enables internationalization of software
Enables the creation of Portable Object (PO) file
Portable Object?

foss.my 2009
gettext
Features
Gettext features
Why gettext?
Supports major characters encoding . . .
UTF-8 for an example
KDE,Gnome, Squirrelmail use it
Relatively to update and maintain

foss.my 2009
Task of developer/translator
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Task of developer/translator
What developers and translators should do?

foss.my 2009
Flow of the translation process
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Flow of Translation

foss.my 2009
PO file
Portable Object (PO) files
PO file’s features
It’s a raw, untranslated file
Created automatically using gettext package
Ready to be translated

foss.my 2009
MO file
Machine Object (MO) files
MO file’s features
Compiled file, derived from PO
It’s a binary, thus machine readable

foss.my 2009
Location of MO file
Default & customized location
By default Linux refers to

/usr/share/locale
Customized directory
najmi@notre-dame:/var/www$ tree ms
ms
‘-- LC_MESSAGES
|-- greetings.mo
|-- index.mo
‘-- messages.mo

foss.my 2009
Creating MO file
Generating Machine Object file
How?
Use msgfmt - message format
This will generate default name, message.mo, unless you specify
it
You can trash it out, given you just want to check the localized
stats to /dev/null
It can be reverted back, use msgunfmt

foss.my 2009
Locale
Intro to locale
Locale
Locale is the local setting of a particular country, race, venue etc
For Bahasa Melayu, the assigned locale is ms MY, where ms is for
Malay, and MY is for Malaysia
As for now, since Indonesia is using “id” , so sometimes ms is just
fine
Date sequence, for .eg dd/mm/yyyy is also fall under locale . We
also do not have Daylight Saving Time(DST), that is also locale
Locale can be simply viewed by typing $locale

foss.my 2009
Charset
Some stuffs on chartset
Charset?
Different charset support different language, characters
Includes Roman, Arabic, CJK and etc
UTF-8, UTF-16 etc
I don’t really know on this stuffs, only know Roman based and
Arabic based chars

foss.my 2009
Codes
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Codes
Sample of C++ file
How’s the C++ file looks like?
# include <iostream>
# include < l i b i n t l . h>
# include <l o c a l e >
using namespace s t d ;
i n t main ( )
{
s e t l o c a l e ( LC ALL , ” ” ) ;
b i n d t e x t d o m a i n ( ” h e l l o ” , ” / u s r / share / l o c a l e ” ) ;
textdomain ( ” h e l l o ” ) ;
cout<< g e t t e x t ( ” H e l l o , w o r l d ! ” ) <<e n d l ;
cout<< g e t t e x t ( ”How are you? ” ) <<e n d l ;
return 0;
}

foss.my 2009
Codes
Sample
Executable output
$ ./hello
Hello, world!
How are you?

foss.my 2009
Codes
Sample
Executable output
$ ./hello
Hello, world!
How are you?
$ export LC_ALL=ms_MY.UTF-8

foss.my 2009
Codes
Sample
Executable output
$ ./hello
Hello, world!
How are you?
$ export LC_ALL=ms_MY.UTF-8
$ ./hello
Assalamualaikum, dunia!
Awak apa khabar?

foss.my 2009
Codes
xgettext
About xgettext
Part of gettext
Extracting translatable strings from source code
xgettext -d lang lang.php
This will create lang.po, with respect to strings in lang.php

foss.my 2009
Codes
Sample of the POT file
How’s the raw POT file looks like?

# SOME DESCRIPTIVE TITLE .
# C o p y r i g h t (C) YEAR THE PACKAGE’ S COPYRIGHT HOLDER
# T h i s f i l e i s d i s t r i b u t e d under t h e same l i c e n s e as t h e PACKAGE package .
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
# , fuzzy
msgid ” ”
msgstr ” ”
” P r o j e c t−Id−V e r s i o n : PACKAGE VERSION\n ”
” Report−Msgid−Bugs−To : \n ”
”POT−C r e a t i o n−Date : 2009−10−23 22:08+0800\n ”
”PO−Revision−Date : YEAR−MO−DA HO: MI+ZONE\n ”
” Last−T r a n s l a t o r : FULL NAME <EMAIL@ADDRESS>\n ”
” Language−Team : LANGUAGE <LL@li . org>\n ”
”MIME−V e r s i o n : 1.0\n ”
” Content−Type : t e x t / p l a i n ; c h a r s e t =CHARSET\n ”
” Content−T r a n s f e r−Encoding : 8 b i t \n ”
# : h e l l o . cpp : 1 1
msgid ” H e l l o , w o r l d ! ”
msgstr ” ”
# : h e l l o . cpp : 1 2
msgid ”How are you ? ”
msgstr ” ”

foss.my 2009
Codes
Sample of the translated PO file
How’s the translated PO file looks like?
# t r a n s l a t i o n o f h e l l o . po t o Malay
# Muhammad Najmi b i n Ahmad Z a b i d i <n a j m i . zabidi@gmail . com>, 2009.
msgid ” ”
msgstr ” ”
” P r o j e c t−Id−V e r s i o n : h e l l o \n ”
”PO−Revision−Date : 2009−10−19 18:32+0800\n ”
” Last−T r a n s l a t o r : Muhammad Najmi b i n Ahmad Z a b i d i <n a j m i . zabidi@gmail . com>\n ”
” Language−Team : Malay <t r a n s l a t i o n −team−ms@lists . s o u r c e f o r g e . net>\n ”
” Content−Type : t e x t / p l a i n ; c h a r s e t =UTF−8\n ”
” X−Generator : KBabel 1.11.4\ n ”
# : h e l l o . cpp : 1 1
msgid ” H e l l o , w o r l d ! ”
msgstr ” Assalamualaikum , dunia ! ”
# : h e l l o . cpp : 1 2
msgid ”How are you ? ”
msgstr ” Awak apa khabar ? ”

foss.my 2009
Codes
gettext in Python
python!
import g e t t e x t
g e t t e x t . b i n d t e x t d o m a i n ( ’ p i t o n ’ , ’ / u s r / share / l o c a l e ’ )
g e t t e x t . t e x td o m a i n ( ’ p i t o n ’ )
= gettext . gettext
# ...
p r i n t ( ’ python i s s i m p l e ’ )
p r i n t ( ’ as s i m p l e as t h i s ’ )

foss.my 2009
Codes
gettext in Python
Traslatable strings from Python file

# t r a n s l a t i o n o f p i t o n . po t o Malay
# C o p y r i g h t (C) YEAR THE PACKAGE ’S COPYRIGHT HOLDER
# T h i s f i l e i s d i s t r i b u t e d under t h e same l i c e n s e as t h e PACKAGE package .
#
# Muhammad Najmi b i n Ahmad Z a b i d i <n a j m i . zabidi@gmail . com>, 2009.
msgid ” ”
msgstr ” ”
” P r o j e c t−Id−V e r s i o n : p i t o n \n ”
”PO−Revision−Date : 2009−10−24 20:57+0800\n ”
” Last−T r a n s l a t o r : Muhammad Najmi b i n Ahmad Z a b i d i <n a j m i . zabidi@gmail . com>\n ”
” Language−Team : Malay <kedidiemas@yahoogroups . com>\n ”
” Content−Type : t e x t / p l a i n ; c h a r s e t =UTF−8\n ”
” X−Generator : KBabel 1.11.4\ n ”
# : p i t o n . py : 6
msgid ” python i s s i m p l e ”
msgstr ” python mudah ”
# : p i t o n . py : 7
msgid ” as s i m p l e as t h i s ”
msgstr ” semudah i n i ”

foss.my 2009
Codes
gettext in Python
Python executable

foss.my 2009
Codes
gettext in Python
Python executable
$ python piton.py
python mudah
semudah ini

foss.my 2009
Codes
gettext in Python
Python executable
$ python piton.py
python mudah
semudah ini
$ export LC_ALL=C

foss.my 2009
Codes
gettext in Python
Python executable
$ python piton.py
python mudah
semudah ini
$ export LC_ALL=C
$ python piton.py
python is simple
as simple as this

foss.my 2009
Codes
PHP source code
Let’s see for PHP
<?
$language=ms MY ;
s e t l o c a l e ( LC ALL , ” $language ” ) ;
bindtextdomain ( ” i n d e x ” , ” / u s r / share / l o c a l e / ” ) ;
textdomain ( ” i n d e x ” ) ;
bind textdomain codeset ( ” i n d e x ” , ’UTF−8 ’ ) ;
echo ( ” Welcome\n ” ) ;
echo ” ” ;
echo ( ” Bye ” ) ;
echo g e t t e x t ( ” The s e t t i n g s o f ” ) . $language . ( ” i s working f i n e ” ) ;
?>

foss.my 2009
Codes
PO file generated from the PHP file. . .
Let’s see for PHP
The following is the example of snipped file - without header

# : l a n g . php : 8
msgid ” Welcome\n ”
msgstr ” ”
# : l a n g . php : 1 0
msgid ” Bye ”
msgstr ” ”
# : l a n g . php : 1 1
msgid ” The s e t t i n g s o f ”
msgstr ” ”
# : l a n g . php : 1 1
msgid ” i s working f i n e ”
msgstr ” ”

foss.my 2009
Codes
Ideal PHP code for i18n
Avoid confusing translator
Try to put strings in a minimal line

Avoid separation of sentence
<?
$language=ms MY ;
s e t l o c a l e ( LC ALL , ” $language ” ) ;
b i n d t e x t d o m a i n ( ” i n d e x ” , ” / u s r / share / l o c a l e / ” ) ;
textdomain ( ” index ” ) ;
b i n d t e x t d o m a i n c o d e s e t ( ” i n d e x ” , ’UTF−8 ’);
echo ( ” Welcome\n ” ) ;
echo ” ”;
echo ( ” Bye ”);
/ / echo ( ” The s e t t i n g s o f ” ) . $language . ( ” i s working f i n e ” ) ;
echo s p r i n t f ( ( ” The s e t t i n g s o f %s i s working f i n e ” ) , $language ) ;
?>

foss.my 2009
Codes
Ideal PHP code for i18n
Use “sprintf”
Use the following . . .

echo s p r i n t f ( ( ‘ ‘ The s e t t i n g s o f %s i s working f i n e ’ ’ ) , $language ) ;
instead of . . .
echo ( ‘ ‘ The s e t t i n g s o f ’ ’ ) . $language . ( ‘ ‘ i s working f i n e ” ) ;

foss.my 2009
Codes
Sample confusing strings
Separated into two
echo ( ‘ ‘ The s e t t i n g s o f ’ ’ ) . $language . ( ‘ ‘ i s working f i n e ” ) ;
will generate. . .
# : l a n g . php : 1 1
msgid ‘ ‘ The s e t t i n g s o f ’’
msgstr ‘ ‘
# : l a n g . php : 1 1
msgid ’ ’ i s working f i n e ‘ ‘
msgstr ’ ’ ‘ ‘ ’ ’
which separates sentence into two different strings. . .

foss.my 2009
Codes
The ideal coding
echo s p r i n t f ( ( ‘ ‘ The s e t t i n g s o f %s i s working f i n e ’ ’ ) , $language ) ;
since this will generate. . .
msgid ‘ ‘ The s e t t i n g s o f %s i s working f i n e ’ ’

msgstr ‘ ‘

foss.my 2009
Codes
Plural issues
What if we have plural noun. . .

Plural is when you have different noun for singular and plural
Singular Plural
cat cats
Singular-Malay Plural-Malay

foss.my 2009
Codes
Plural issues

Singular Plural
cat cats
boy boys

foss.my 2009
Codes
Plural issues

Singular Plural
cat cats
boy boys
man men

foss.my 2009
Codes
Plural issues

Singular Plural
cat cats
boy boys
man men
basikal basikal-basikal

foss.my 2009
Codes
Plural issues

Singular Plural
cat cats
boy boys
man men
pensil pensil-pensil

foss.my 2009
Codes
Plural issues

Singular Plural
cat cats
boy boys
man men
pensil pensil-pensil
komputer komputer-komputer

foss.my 2009
Codes
Plural issues
However, as far as I know, plural issue in Malay languge is non trivial

issue, so for the sake of easiness (perhaps, lazy?) plural = 0

foss.my 2009
Codes
Using ngettext in PHP
ngettext
<?php
$n = 2 ;
echo s p r i n t f ( n g e t t e x t ( ”%d boy i s e a t i n g ” , ”%d boys are e a t i n g ” , $n ) , $n ) ;
echo ” ” ;
$n =1;
echo s p r i n t f ( n g e t t e x t ( ”%d c a t f a l l s ” , ”%d c a t s f a l l ” , $n ) , $n ) ;
echo ” ” ;
$n =1;
echo s p r i n t f ( n g e t t e x t ( ”%d t i n g k a p d i t u t u p ” , ”%d t e t i n g k a p d i t u t u p ” , $n ) , $n ) ;
echo ” ” ;
$n =4;
/ / i f i t goes w i t h o u t parameter %d , no s p r i n t f i s f i n e
echo n g e t t e x t ( ” F i l e i s good ” , ” F i l e s are good ” , $n ) ;
?>
2 boys are eating

1 cat falls
1 tingkap ditutup
Files are good

foss.my 2009
Codes
Using ngettext in PHP
Sample PO
# : n p l u r a l . php : 3
# , php−f o r m a t
msgid ”%d boy i s e a t i n g ”
m s g i d p l u r a l ”%d boys are e a t i n g ”
msgstr [ 0 ] ” ”
msgstr [ 1 ] ” ”
# : n p l u r a l . php : 6
msgid ”%d c a t f a l l s ”
m s g i d p l u r a l ”%d c a t s f a l l ”
msgstr [ 0 ] ” ”
msgstr [ 1 ] ” ”
# : n p l u r a l . php : 9
msgid ”%d t i n g k a p d i t u t u p ”
m s g i d p l u r a l ”%d t e t i n g k a p d i t u t u p ”
msgstr [ 0 ] ” ”
msgstr [ 1 ] ” ”
# : n p l u r a l . php : 1 3
msgid ” F i l e i s good ”
m s g i d p l u r a l ” F i l e s are good ”
msgstr [ 0 ] ” ”
msgstr [ 1 ] ” ”

foss.my 2009
Tools
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Tools
Internationalization tools
Internationalization tools
gted (very recent)

html2po / po2html

foss.my 2009
Tools
Localization tools
Localization tools
POedit
Kbabel (now lokalize)
VI? Emacs ... the list may infinite up to n # of tools
Incoming . . . wish for PO Live Edit, prototype is webl10n by
gandalf
Pootle

foss.my 2009
Route of PO to MO
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Route of PO to MO
PO to MO journey
Translate
PO file − > MO file − > Working Interface

foss.my 2009
Route of PO to MO
PO to MO journey
Translate
Compile

foss.my 2009
Route of PO to MO
PO to MO journey
Translate
Compile
Invoke

foss.my 2009
Issues with package generated by gettext
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Issues
What are the issues?
Software may become bloated

foss.my 2009
Issues
So, the user may not install everything in the first place!

foss.my 2009
Issues
Mozilla has its own way. . .
I saw moz2po and po2moz . . . a good news, perhaps?
Bug filed #501988 seems a good news

foss.my 2009
Issues
I also have problem with UTF-8 on console.. garbled characters

foss.my 2009
Issues
I also have problem with UTF-8 on console.. garbled characters
Then, I export LC ALL=C, of course I’m not happy with this
But most of the time, it works. . .

foss.my 2009
Conclusion
Agenda
1 About
3 gettext
6 Codes
7 Tools
8 Route of PO to MO
10 Conclusion

foss.my 2009
Conclusion
gettext rules

foss.my 2009
Conclusion
fin()
Questions?

?È@ñË@ Õ»YJ« Éë
najmi{at} kict.iiu.edu.my

Software Internationalization (I18n) With GNU Gettext

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Software Internationalization (I18n) With GNU Gettext

Enviado por

Direitos autorais:

Formatos disponíveis

foss.

Software Internationalization (i18n) with GNU gettext

Muhammad Najmi Ahmad Zabidi

Department of Computer Science

25th October 2009

Muhammad Najmi () foss.my 2009 October 26, 2009 1 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 2 / 48

About the speaker

Muhammad Najmi () foss.my 2009 October 26, 2009 3 / 48

About the speaker

Muhammad Najmi () foss.my 2009 October 26, 2009 3 / 48

About the speaker

Muhammad Najmi () foss.my 2009 October 26, 2009 3 / 48

About the speaker

Muhammad Najmi () foss.my 2009 October 26, 2009 3 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 4 / 48

What actually internationalization is:

Muhammad Najmi () foss.my 2009 October 26, 2009 5 / 48

What actually internationalization is:

Muhammad Najmi () foss.my 2009 October 26, 2009 5 / 48

What actually internationalization is:

Muhammad Najmi () foss.my 2009 October 26, 2009 5 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 6 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 6 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 7 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 8 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 9 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 10 / 48

What developers and translators should do?

Muhammad Najmi () foss.my 2009 October 26, 2009 11 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 12 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 13 / 48

Portable Object (PO) files

Muhammad Najmi () foss.my 2009 October 26, 2009 14 / 48

Machine Object (MO) files

Muhammad Najmi () foss.my 2009 October 26, 2009 15 / 48

Default & customized location

By default Linux refers to

Muhammad Najmi () foss.my 2009 October 26, 2009 16 / 48

Generating Machine Object file

Muhammad Najmi () foss.my 2009 October 26, 2009 17 / 48

Locale can be simply viewed by typing $locale

Muhammad Najmi () foss.my 2009 October 26, 2009 18 / 48

Some stuffs on chartset

Muhammad Najmi () foss.my 2009 October 26, 2009 19 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 20 / 48

How’s the C++ file looks like?

Muhammad Najmi () foss.my 2009 October 26, 2009 21 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 22 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 22 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 22 / 48

xgettext -d lang lang.php

This will create lang.po, with respect to strings in lang.php

Muhammad Najmi () foss.my 2009 October 26, 2009 23 / 48

How’s the raw POT file looks like?

Muhammad Najmi () foss.my 2009 October 26, 2009 24 / 48

How’s the translated PO file looks like?

Muhammad Najmi () foss.my 2009 October 26, 2009 25 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 26 / 48

Traslatable strings from Python file

Muhammad Najmi () foss.my 2009 October 26, 2009 27 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 28 / 48

Muhammad Najmi () foss.my 2009 October 26, 2009 28 / 48