MG/Translations maintenance

How to manage translations

In case you're happy to help out with translations management: You'd need to install the Python based "transifex-client" (available in Ubuntu, probably Debian, too, via apt, and also via python pip), and configure authentication in ~/.transifexrc as discussed at https://docs.transifex.com/client/introduction Then, while you're in the megaglest-data git repository directory (develop branch), make sure tx status works, outputting many translations but not outputting that it could not authenticate to transifex.com.

To manage translations, there's a script at megaglest-data/glest_game/.tx/tx.sh. I'm not sure I ever used this incarnation of the script but I assume it just works, since Filux will have used it.

If you're not currently preparing an MG release and want to just get all the translation updates, just run git pull, then tx.sh without parameters, otherwise use tx.sh --release. This should downloads all the latest translations from Transifex and overwrites them in your local git repository, and clean those files up a bit (removing superfluous blank spaces, undoing some undesirable HTML entity encoding etc.). Then you git diff to ensure all those translations which were submitted actually make sense and could actually work on the game (e.g. sometimes people just translate but do not test translations in the game using the download feature Softcoder integrated there (see MG/Translations#Testing_your_translation), where the game gets updated translations directly from Transifex) and then some translations may be way too long to fit into the space available on the GUI, or using characters not available in the font that is set (in the main .lng files) to be used for this language.

If things look fine then i guess you just tx.sh --push-all.

If entirely new languages were added then you also need to extend lang_map in data/glest_game/.tx/config - this is a mapping between how MG refers to languages (e.g. the paths and translation file names in git) and how Transifex calls them.

One-liners

Here are some POSIX shell one-liners to help with maintenance of translation files.

Remove BOM off all files

sed -i '1 s/^\xef\xbb\xbf//' data/glest_game/data/lang/*.lng data/glest_game/data/lang/hint/*.lng data/glest_game/scenarios/*/*.lng data/glest_game/tutorials/*/*.lng

Extract all keys to a file

cat data/glest_game/data/lang/*.lng data/glest_game/data/lang/hint/*.lng data/glest_game/scenarios/*/*.lng data/glest_game/tutorials/*/*.lng | awk -F= '/^[^;#]/ {print $1}' | sort | uniq | grep -Ev '^\W*$' > /tmp/all_keys.txt

List unreferenced keys

List keys which are not referenced anywhere in the source code and thus may be leftovers in the translation files.

while read key; do grep -Eq '(lang\.get\("'"$key"'"\)|hasString\("'$key'"\)|setDisplayText\('"'$key'"'\))' `find source/ -type f -name '*.cpp' -o -name '*.h'`; if [ "$?x" = '1x' ]; then echo "Key [$key] could not be found in the source code."; fi; done < /tmp/all_keys.txt

Beware of false positives:

• Keys which end in numbers / counters are usually not found on the source code as is
• ... (to be added)