Yamyam: 1版をインポートしました

2018-06-28T19:45:55Z

1版をインポートしました

wiki>Xiaoxiangquan: /* Font Loading */

2011-08-22T15:46:37Z

‎Font Loading

新規ページ

{{Page/Header|<blender version>|<previous page>|<next page>}}

=Garlic's Internationalization Design=

We use [http://www.gnu.org/software/gettext GNU Gettext] to translate strings. Here is the standard workflow.

==GNU Gettext Workflow==
- Firstly, a programmer writes a piece of code containing strings that he wants them to be translated into end users' local language. The original code is like:
<source lang="c">
ot->name = "Quit";
</source>

To use GNU Gettext, the programmer wrote like this:
<source lang="c">
/* header for gettext */
#include <libintl.h>
...
ot->name = gettext("Quit");
</source>

- Then the programmer ran the command xgettext, which will scan the source files specified, and gather the words in gettext(""). Finally, xgettext will gennerate a .pot file, whose suffix means a PO Template.

- A po is a translation file with items like:
msgid "Wave"
msgstr "Vague"
Actually, this is a po for French, generally named fr.po. Translators use msgmerge command to merge the old PO and the newly updated POT, then all the new changes will be available to see in the PO. If there is no "Quit" item in the old PO, now there will be:
msgid "Quit"
msgstr ""
Then translators can open the PO and fill untranslated strings.

- After the PO is translated, anyone can execute a msgfmt command to compile it as a MO file, and replace the old one used by the application. And the repository maintainer could update the MO to the trunk, too.

- At runtime, the gettext("Quit") calling will find the "Quit" entry in the MO and replace the string with the mapped one. Finally, users see a localized message if he has set application's locale and language properly.

Note that If there is no target entry or just an empty one,
the calling will return the original message.

Now as you can see, this work has great need for cooperations.

==More about Gathering and Translating==

In the process above, the gettext(str) act as two roles: mark the string to be gathered by xgettext command, and do the real translate at runtime. Actually, we can seperate these two stages as follows:

<source lang="c">
#define Mark(msgid) msgid
#define Trans(msgid) gettext(msgid)
...
ot->name = Mark("Quit");
ot->name = Trans(ot->name);
</source>

In this style, the Mark() do nothing. But if we add "Mark" as a keyword for xgettext, the command will notice that "Quit" should be gathered. And at runtime, the Trans() calling will do the real thanstation.

Note that the xgettext is not intelligent enough to notice the "Quit" through Trans() calling, as the ot->name is not an evident string. Even the following syntax will not pass:
<source lang="c">
#define SubStr "DEF"
...
/* cannot recognize string constants concatenated with macro */
ot->name = gettext("ABC" SubStr "GHI");
</source>

While simple string constants concat could be handled well:
<source lang="c">
/* xgettext works well to gather "ABCDEFGHI" */
ot->name = gettext("ABC" "DEF" "GHI");
</source>

Further more, format string can be handled well only if the translated string contains the same number and type of format elements, or else a runtime error may occur:
<source lang="c">
/* make sure the translated "Hello, %s!" contains "%s" */
printf( gettext("Hello, %s!"), gettext("world") );
</source>

==Garlic's Gettext Framework==
===Supporting Files===
Let's see the related file tree first:
/po: Contains all PO and POT files, as well as some updating tools.
---/blender.pot: The PO Template
---/*.po: All translations
---/POTFILES.in: All source files containing messages to be handled
---/update_pot.py: Script to update the blender.pot.
---/update_po.py: Script to update the *.po.
---/update_mo.py: Script to compile the *.po.
/release/bin/.blender/locale: Contains all compiled translations.
----------------------------/*/LC_MESSAGES/blender.mo:
Compiled translation for single language.

====Update POT====
The update_pot.py will scan all the source files listed in the POTFILES.in. The command is like:
<source lang="bash">
# --files-from specifies the files list
# --keyword specifies what indicates a target string
# --output specifies output file
# --from-code specifies string encode
xgettext --files-from=POTFILES.in
--keyword=_ --keyword=N_
--output=blender.pot
--from-code=utf-8
</source>
Generally, xgettext takes gettext() ans _() as default keywords, but it's not bad to repeat for clarity.
As you can see, in Garlic we also use another keyword, N_(). See details later.

====Update PO====
The update_po.py will merge the blender.pot with all the PO files. After that, all additions, deletions and modifications of items will be available in the PO for translators. The command is like:
<source lang="bash">
# --lang specifies the target language
# zh_CN.po is the old PO
# blender.pot is the updated PO Template
msgmerge --update --lang=zh_CN zh_CN.po blender.pot
</source>

====Update MO====
The update_mo.py will compile all the PO, generates MO files and put them under /release/bin/.blender/locale. The command is like:
<source lang="bash">
# --statistics makes translation status printed
# zh_CN.po is the source PO
# -o specifies the target MO file path
msgfmt --statistics zh_CN.po -o LOCALE_DIR/zh_CN/LC_MESSAGES/blender.mo
</source>

===The _() and N_() Macro===
We supply a decorated gettext(), named BLF_gettext(), to do the real translation at runtime, which is in /source/blender/blenfont/intern/blf.c:
<source lang="c">
const char* BLF_gettext(const char *msgid)
{
#ifdef INTERNATIONAL
if( msgid!=NULL && strlen(msgid)>0 )
return gettext( msgid );
return "";
#else
return msgid;
#endif
}
</source>

Taking into account that it will be called thousands of times and we programers are lazy, a macro _() is supplied for short:
<source lang="c">
#define _(msgid) BLF_gettext(msgid)
</source>
Please always try to use _(). It will make the code clearer, and translation related issues won't disturbe the logic semantics too much.

However, sometimes we cannot call a function like BLF_gettext(). For example, there is piece of code in /source/blender/editors/screen/screen_ops.c:
<source lang="c">
static EnumPropertyItem modal_items[] = {
{KM_MODAL_CANCEL, "CANCEL", 0, "Cancel", ""},
{KM_MODAL_APPLY, "APPLY", 0, "Apply", ""},
{KM_MODAL_STEP10, "STEP10", 0, "Steps on", ""},
{KM_MODAL_STEP10_OFF, "STEP10_OFF", 0, "Steps off", ""},
{0, NULL, 0, NULL, NULL}};
</source>
We cannot call a function in these statements, but we need to let the xgettext command know what to collect. So, we seperate the gathering and translating stages as mentioned above. To achieve this we define a macro that do nothing. See it in /source/blender/blenfont/BLF_api.c:
<source lang="c">
#define N_(msgid) msgid
</source>

With this macro we can rewrite the code as:
<source lang="c">
static EnumPropertyItem modal_items[] = {
{KM_MODAL_CANCEL, "CANCEL", 0, N_("Cancel"), ""},
{KM_MODAL_APPLY, "APPLY", 0, N_("Apply"), ""},
{KM_MODAL_STEP10, "STEP10", 0, N_("Steps on"), ""},
{KM_MODAL_STEP10_OFF, "STEP10_OFF", 0, N_("Steps off"), ""},
{0, NULL, 0, NULL, NULL}};
</source>

Now, all the marked strings can be gathered to POT. But where to do the real translating at runtime? The answer is any one place before its content takes effect. In this example, it's just in the same function:
<source lang="c">
keymap= WM_modalkeymap_add( keyconf, "Standard Modal Map", modal_items);
</source>
We rewrite it as:
<source lang="c">
keymap= WM_modalkeymap_add( keyconf, "Standard Modal Map",
RNA_enum_items_gettexted(modal_items));
</source>

The RNA_enum_items_gettexted() is defined in /source/blender/makesrna/intern/rna_access.c. It traverses over a EnumPropertyItem array, and try translating all name and description field:
<source lang="c">
/* make every name and description field translated by gettext */
EnumPropertyItem* RNA_enum_items_gettexted(EnumPropertyItem *item)
{
if( item )
{
int i;
for(i=0; item[i].identifier; i++)
{
if( item[i].name )
item[i].name = _(item[i].name);
if( item[i].description )
item[i].description = _(item[i].description);
}
}
return item;
}
</source>

===The Special makesrna Issues===
Most strings outside /source/blender/makesrna could be handled easily by _(). Both gathering and translating work well. But it's different for those inside the makesrna folder.

When compiling, these files will linked as an executable makesrna, which will gennerate a lot of rna_*_gen.c somewhere depending on system. For linux, it's under [Build Path]/source/blender/makesrna/intern. These generated files will be compiled and linked into the final blender application.

So there is no need to translate strings when running makesrna. We'll just use N_() to mark all strings we are interested in. To do the real translating, we should access the datas in generated files. This can be done by a extern declaration. See the makesrna/RNA_access.h for example:
<source lang="c">
...
/* Types */

extern BlenderRNA BLENDER_RNA;
extern StructRNA RNA_Action;
extern StructRNA RNA_ActionConstraint;
extern StructRNA RNA_ActionFCurves;
extern StructRNA RNA_ActionGroup;
...
</source>
Each of these StructRNAs contains a linked list of properties with name and description fields which should be translated. If it is an ENUM property, the item field would be an array of EnumPropertyItems mentioned above, so we also need to process it with RNA_enum_items_gettexted().

Finally, we implement a RNA_types_init_gettext() in the makesrna/intern/rna_access.c to do this traversal processing.

===Language Selection===
The application decides which translation to work with by both the locale and language settings. The language options are listed in an EnumPropertyItem array named language_items, in the /source/blender/makesrna/intern/rna_userdef.c. Users' selection is stored in the global U.language. So we can set the environment accordingly. This is all done in /source/blender/blenfont/intern/blf_lang.c. The most important function, BLF_lang_set(), will try setting the locale, LANG and LANGUAGE environment values to the specified language.

This function also set the domain binding for gettext:
<source lang="c">
void BLF_lang_set(const char *str)
{
...
if( str[0]!=0 )
{
BLI_setenv("LANG", str);
BLI_setenv("LANGUAGE", str);
}
...
setlocale(LC_ALL, str);
...
textdomain(DOMAIN_NAME);
bindtextdomain(DOMAIN_NAME, global_messagepath);
...
}
</source>

===Font Loading===
Blender is rendered by OpenGL, so after other issues settled, the last problem is how to render international texts such as Chinese on the UI. The main point is to load a proper font when calling uiStyleInit() in /source/blender/editors/interface/interface_style.c. Finally we find [http://unifoundry.com/unifont.html| Unifont] a good glyphs that support all useful utf8 characters.

However, the TTF takes more than 10MB space. To make blender as lightweight as possible, we decide to gzip it. As a result, the application has to ungzip it at runtime. This work is done by BLI_ungzip_to_mem() in /source/blender/blenlib/intern/fileops.c:
<source lang="c">
/*
* gzip the file in from_file and write it to memery to_mem,
* at most size bytes.
*
* return the unziped size
*/
int BLI_ungzip_to_mem(const char *from_file, char *to_mem, const int size)
{
gzFile gzfile;
int readsize;

gzfile = gzopen( from_file, "rb" );
readsize = gzread( gzfile, to_mem, size);

if (readsize < 0)
readsize = EOF;

return readsize;
}
</source>

The get_datatoc_bunifont_ttf() function in /source/blender/editors/datafiles calls this function and return the font data been read. Then the uiStyleInit() finished loading the unifont TTF:
<source lang="c">
font->blf_id= BLF_load_mem_unique(
"default",
(unsigned char *)get_datatoc_bunifont_ttf(),
datatoc_bunifont_ttf_size);
</source>

Presently Garlic takes unifont as the default font, while not the old bfont. Because the Language Selection interface needs almost all languages, so any single language TTF may not work well.

After these preparation, international texts can be displayed properly, now.

{{Page/Footer|Internationalization|}}

← 古い版	2018年6月28日 (木) 19:45時点における版
(相違点なし)

利用者:Xiaoxiangquan/GoogleSummerOfCode/2011/Internationalization/design - 版の履歴

Yamyam: 1版 をインポートしました

wiki>Xiaoxiangquan: /* Font Loading */

Yamyam: 1版をインポートしました