Discussion:
[tw] [TW5] MultiLang
(too old to reply)
wimm
2016-07-06 10:23:17 UTC
Permalink
Hi,

PoC / show case multi language TW5.1.11

The current implementation for interface language rquires 600+ separate
tiddlers for each piece of text for each language.

my goal

- reduce the number of tiddlers (core plugin was getting very long list of
tiddlers) (partial done, language tiddlers are still present but not used,
a few for long texts will remain).
- immediate language translation switching without reload (partial done,
not yet for already present buttons)
- make code more readable by including original english text in the code
instead of lingo-base and lingo macro (done)
- easier to translate, checking for missing entries (one file per
language), switch back to English to find the search key, for the
translation)
-all translations except for dates, numbers, colors, these also require
rewrite of utils.js
-all languages (partial done see list of languages below)

to accomplish this I created 9 data tiddlers with translations
$:/trnslt-en2, $:/trnslt-nl, etc

for example in the dutch translation tiddler there is a line

close others:sluit anderen


instead of
\define lingo-base() $:/language/Buttons/
<<lingo CloseOthers/Caption>>
tiddler will contain a query on a datatiddler $:\trnslt and shows the query
key of the text


{{$:/trnslt##close others}}


this then tranlates to
sluit anderen
You can try the result at http://tw5multilanguage.tiddlyspot.com

<Loading Image...>


This is WORK IN PROGRESS there is still a lot to do do not use this TW for
storing imported data

Is this the way to go? pull request?
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6a9e4ee7-67b8-4ef4-9945-9e025e889044%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeremy Ruston
2016-07-06 11:00:15 UTC
Permalink
Hi Wimm

That’s an ambitious and interesting piece of work, thank you for undertaking it!
- reduce the number of tiddlers (core plugin was getting very long list of tiddlers) (partial done, language tiddlers are still present but not used, a few for long texts will remain).
I’m not sure that I see the number of shadow tiddlers as a great disadvantage. Are you concerned with performance or something else?
- immediate language translation switching without reload (partial done, not yet for already present buttons)
Doesn’t that already work? As far as I know a restart is needed after installing a new language plugin, but not to switch between them.
- make code more readable by including original english text in the code instead of lingo-base and lingo macro (done)
I would need persuading that this is an improvement. It leads to much longer identification strings, and will surely lead to complexity if we want to change the *English* translation of a string?
- easier to translate, checking for missing entries (one file per language), switch back to English to find the search key, for the translation)
My thinking was that we could build tools to make it easier to translate, hence http://tiddlywiki.com/editions/translators/
This is WORK IN PROGRESS there is still a lot to do do not use this TW for storing imported data
Is this the way to go? pull request?
It’s very interesting; I hope my comments don’t put you off. I’d certainly find it easier to review your changes if you could make a fork on GitHub.

Best wishes

Jeremy
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
Visit this group at https://groups.google.com/group/tiddlywiki <https://groups.google.com/group/tiddlywiki>.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6a9e4ee7-67b8-4ef4-9945-9e025e889044%40googlegroups.com <https://groups.google.com/d/msgid/tiddlywiki/6a9e4ee7-67b8-4ef4-9945-9e025e889044%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/FEA347AA-1C44-4D7A-8099-E0D3D3A23443%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
wimm
2016-07-06 12:19:25 UTC
Permalink
Post by Jeremy Ruston
That’s an ambitious and interesting piece of work, thank you for
Post by Jeremy Ruston
undertaking it!
That's alright, most of the work was done by curl, regular expressions
Post by Jeremy Ruston
using gvim and tiddlywiki to create the new translation tiddlers.
- reduce the number of tiddlers (core plugin was getting very long list of
Post by Jeremy Ruston
tiddlers) (partial done, language tiddlers are still present but not used,
a few for long texts will remain).
I’m not sure that I see the number of shadow tiddlers as a great
disadvantage. Are you concerned with performance or something else?
- immediate language translation switching without reload (partial done,
not yet for already present buttons)
Doesn’t that already work? As far as I know a restart is needed after
installing a new language plugin, but not to switch between them.
this is just a feature that should still exist, adding a new language
would now not require a reload anymore
Post by Jeremy Ruston
- make code more readable by including original English text in the code
instead of lingo-base and lingo macro (done)
I would need persuading that this is an improvement.
My main concern was the indirection, I tried to add a translation of a
diff view plugin when editing tiddlers, and came to this solution.
When I search for a system tiddler I normally use the span title (it is
Post by Jeremy Ruston
hard to search for an icon) in the current implementation you first find
the translation tiddler, then you have to select ''part'' of the
translation tiddler path, to find the tiddler you were looking for.
Using the new trnslt tiddlers, you will find the tiddler (plus the trnslt
tiddlers), if you only find trnslt tiddlers the text was modified in the
En-En translation, in this tiddler you can find the original key to search
again. .
Post by Jeremy Ruston
It leads to much longer identification strings,
That is true, but it will result in a smaller TW file (trnslt json's for 8
languages are 215k)
Post by Jeremy Ruston
and will surely lead to complexity if we want to change the *English*
translation of a string?
English translation strings can be modified in the English to English
translation tiddler. Other translations still work if you do not change the
key. If you modify the text a lot, then you maybe should opt for a new
entry, so everybody knows that their translations probably also need
updating..
Post by Jeremy Ruston
- easier to translate, checking for missing entries (one file per
language), switch back to English to find the search key, for the
translation)
My thinking was that we could build tools to make it easier to translate,
hence htt <http://tiddlywiki.com/editions/translators/>
p://tiddlywiki.com/editions/translators/
<http://tiddlywiki.com/editions/translators/>
I tried the translator GUI, but when importing all translations, I noticed
many translations are missing.
tooling to find missing translations you would need a tiddler containing
something like

\define maketranslt( tid)
{{$:/trnslt-en2##$(currentTiddler)$}}===={{$:/trnslt-nl##$(currentTiddler)$}}
<$list filter="[[$:/trnslt-en2]indexes[]]"> <br><<maketrans>>
</$list>

and search for ==== at the end o the line in your editor or your browser)

if you need multiple translations for one text e.g. Belgium using Dutch and
French

\define maketranslt( tid)
{{$:/trnslt-en2##$(currentTiddler)$}}:{{$:/trnslt-nl##$(currentTiddler)$}}
({{$:/trnslt-fr##$(currentTiddler)$}})
<$list filter="[[$:/trnslt-en2]indexes[]]"> <br><<maketrans>>
</$list>

and copy the output to a $:/trnslt-be tiddler, and extend the choose
language tiddler with one button


This is WORK IN PROGRESS there is still a lot to do do not use this TW for
storing imported data
Post by Jeremy Ruston
Post by Jeremy Ruston
Is this the way to go? pull request?
It’s very interesting; I hope my comments don’t put you off. I’d
certainly find it easier to review your changes if you could make a fork on
GitHub.
It's just a proof of concept, it is one of many local forks, There are
only a few new tiddlers : choose language, translations data tiddlers,
small flags.svg, the remainder is a "global replace" of the lingo
macros..(I'm not a fan of github yet).
Post by Jeremy Ruston
Best wishes
Jeremy
Thanks for your reply
Best regards
WiM
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/11f18aaf-a593-403b-af74-dea8c0231bb4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jed Carty
2016-07-06 11:16:11 UTC
Permalink
I haven't looked through the code so I am going by your description for
this.

For anything like this my two concerns would be that languages need to be
able to be updated independently and that plugins should be able to be
translated also.
It sounds like you have the first one without any trouble, but if each
language is in a single data tiddler can we still have optional additions
for plugins using this system?

I also agree with Jeremy that using the english translation of a string as
the key isn't necessarily a good idea. Problems with changing the english
translation are only one part of that though.
Currently english is the common language for the core development, this
isn't a problem because it is one of the most common languages in the
world, but I would rather avoid anything that would make it a requirement
or discourage non-english speakers from thinking they could contribute.
I am not certain that using english as the key would be a problem in either
of those cases, but that was my first reaction to the idea.
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6dd01dda-00b3-4095-92be-49cabb438e3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
wimm
2016-07-06 11:41:16 UTC
Permalink
Hi Jed

I haven't looked through the code so I am going by your description for
Post by Jed Carty
this.
the TW gives more information
Post by Jed Carty
For anything like this my two concerns would be that languages need to be
able to be updated independently and that plugins should be able to be
translated also.
It sounds like you have the first one without any trouble, but if each
Post by Jed Carty
language is in a single data tiddler can we still have optional additions
for plugins using this system?
Separate tiddlers for a plugin translation are possible, that was the
original problem I tried to resolve, before refactoring the complete TW

$:/trnslt-pluginname-en
$:/trnslt-pluginname-nl
etc...
Post by Jed Carty
I also agree with Jeremy that using the english translation of a string as
the key isn't necessarily a good idea. Problems with changing the english
translation are only one part of that though.
I thought of that.. There is an English to English translation tiddler, to
allow for modification of the English txt, as long as you leave the part
before the : untouched, so the same mechanism works for all languages.
Post by Jed Carty
Currently english is the common language for the core development, this
isn't a problem because it is one of the most common languages in the
world, but I would rather avoid anything that would make it a requirement
or discourage non-english speakers from thinking they could contribute.
I am not certain that using english as the key would be a problem in either
Post by Jed Carty
of those cases, but that was my first reaction to the idea.
thanks for your concerns

KR
WiM
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1e26ba2a-e31b-4bac-b94a-2dd6e97304cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mat
2016-07-06 12:22:19 UTC
Permalink
On mobile so brief;

Regarding transl: it is frustrating to search some string and not find the tid because string was in lingo thing.

can the technique seen in aliases.tiddlyspot.com be used to hide inline strings, targetedr via span IDs, i.e hide the english and display the translation? It relies on css pseudo elements if wanting to do it purely css.

<:-)
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6b1fef1f-4e71-43d7-866f-b334e6e8f32d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeremy Ruston
2016-07-06 12:33:17 UTC
Permalink
Hi Mat
Post by Mat
can the technique seen in aliases.tiddlyspot.com be used to hide inline strings, targetedr via span IDs, i.e hide the english and display the translation? It relies on css pseudo elements if wanting to do it purely css.
The translation mechanism must work on the server as well as in the browser, so it has to be done in pure JavaScript.

Best wishes

Jeremy
Post by Mat
<:-)
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6b1fef1f-4e71-43d7-866f-b334e6e8f32d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/5B60D0AA-F56D-455F-BE95-B140A1B36C16%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
PMario
2016-07-06 13:14:05 UTC
Permalink
Hi Wimm,

It seems you put a lot of work into it. ... but I have some concerns if it
actually is an improvement, for me it looks like the opposite.
Post by wimm
The current implementation for interface language rquires 600+ separate
tiddlers for each piece of text for each language.
If you want to translate TW, you just need to checkout the github repo and
modify: 62 different files. They create a lot of shadow tiddlers, that's
right. But TW is optimized for small chunks of text.
IMO TW translations are already packed quite a lot.


my goal
Post by wimm
- reduce the number of tiddlers (core plugin was getting very long list of
tiddlers) (partial done, language tiddlers are still present but not used,
a few for long texts will remain).
why? There are a lot of tiddlers that also create dynamic lists. eg:
tiddler-types dropdown. or Fields.multids ...

- If you throw all of them into one big unmanageable file you remove the
dynamic features.

- Every shadow tiddler can be modified separately by the user. eg: You can
add your own comments, tags and what not ....
- If we remove this possibility, imo it's a "no go" regression.
- With the existing mechanism, one tiddler will be affected by user
editing.

If a user edits the "big" translation tiddler, she basically sticks with it
for ever, since it is not updated by the core update mechanism anymore ..


- immediate language translation switching without reload (partial done,
Post by wimm
not yet for already present buttons)
This is status quo.
Post by wimm
- make code more readable by including original english text in the code
instead of lingo-base and lingo macro (done)
Making the core UI code more readable, would be nice, but having a
translate "english to English" table (aka hash table) imo will increase
complexity over time a lot. Not now but in the future.

Eg: just think about this hash table in 5 years from now.
For TW backwards compatibility is important. So you have to stick forever
with the English keys from now on, even if it changes meaning in the
future.

There is a reason, why the tiddler names use namespaces. Many filtered
lists rely on those tiddler namespaces.

Your suggestion{{$:/trnslt##close others}} imo removes the namespace
concept, which influences a lot of filter operators and a core concept.
Post by wimm
- easier to translate, checking for missing entries (one file per
language), switch back to English to find the search key, for the
translation)
As I wrote there are only 62 files to manage and the shorter they are, the
easier it is to find the differences. I use the Meld program to check the
differences between eg: english and german

So using Meld and the github commit history makes it incredibly easy to
follow the language changes. If following the changes is hard, we need
better documentation.

just some thoughts.
-mario
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d83d6d25-5a7d-43f4-a82f-e20789c61e6d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
wimm
2016-07-06 16:39:43 UTC
Permalink
jeremy, mario, jed & mat

maybe just to clarify,
All above changes were made to a single tiddlywiki.html file using some smart vi regular expressions and the internal editor of tw.

The last time I used nodes.js for tw was at least two and a half years ago. That might explain why I don't use github ;-)

long keys:
if the keys get too long, you can easily abbreviate them as long they contain the start of the sentence. please note the translation can contain wikitext including transclusions for e.g. short sentences in the same or other trnslt file.

sv source code (for educational purposes, school..):
@mat if you want the strings in the html source to be translated . you could use for instance a vim command file (generated by a simple tiddler) to replace all english key texts by Swedish(?) an additional sv 2 sv trnslt is required, it would then show the texts. (append the sv mutids and tid files, see tw use en2 and the file to create english to sv translation. use that translation to create sv to sv like the belgium replace nl and fr by sv)

replace the macro for the Belgium nl fr trnslt -from previous post - in something like (not tested I'm om my phone now)
:%s/trnslt##{{trnslt##\$:\/trnslt-en2##\$(currentTiddler)\$/trnslt##{{trsnlt-sv##$(currentTiddler)/g

because all translations for one language are together in one tiddler(strong cohesion) a 4 line tiddler can generate the vim command to do a full translation into sverige. copy command to a file called sv.vim
gvim -s sv.vim mytiddlywiki.html will creare the file you want. use file menu to save.

kr WiM
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/619a3d6f-937c-4830-9e90-17dd2d4a120c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
wimm
2016-07-08 05:16:27 UTC
Permalink
Mario,
To remove your concern about the contents of the English to English
translation in let's say 5 years. We can use the power of Tiddlywiki and
gvim to update all in less than 60 minutes.

So when the translation En-En contains too many modified translations A vim
command very similar to the Svenka translation can be created using TW to
update the English keys in the html source (not tested yet)

create a tiddler (see mutlilanguage.tiddlyspot.com)
\define maketranslt( tid)
:%s/trnslt##{{trnslt##\$(currentTiddler)\$/trnslt##{{trsnlt-en2##$(currentTiddler)/g
<$list filter="[[$:/trnslt-en2]indexes[]]"> <br><<maketrans>>
</$list>

Copy the output to a file called en.vim and use vim-s en.vim tiddywiki.html
to update the html source

At the same time you then need to update the translation files for each
language

For the updated English translation file
\define maketranslt( tid)
{{$:/trnslt-en2##$(currentTiddler)$}}{{$:/trnslt-en2##$(currentTiddler)$}}
<$list filter="[[$:/trnslt-en2]indexes[]]"> <br><<maketrans>>
</$list>

copy the preview to a new data tiddler e.g. translt-en2 (don't use
$:/trnslt-en2 at this moment)

for Dutch
\define maketranslt( tid)
{{$:/trnslt-en2##$(currentTiddler)$}}{{$:/trnslt-nl##$(currentTiddler)$}}
<$list filter="[[$:/trnslt-en2]indexes[]]"> <br><<maketrans>>
</$list>

for all other languages replace *nl* in the Dutch example by the
appropriate language abbreviation.

finally rename all trnslt-xx to $:/trnslt-xx
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/90432ff3-d413-4453-b2c1-2f8399008827%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...