[sip-comm-dev] Localization support


#1

Hi Folks,

I am thinking about redesigning the i18n stuff which has some drawbacks in
the current code. For now, it is a static class in the gui service. I have
managed to override the methods to which I referred in a previous mail,
and that caused the message.properties file to be parsed in ISO-8859-1. It
is now properly read as UTF-8.

This said, I'd like to initiate a discussion about how to fit the i18n
stuff more efficiently into SC, because we will need internationalization
in other places than the GUI, especially with all the cool new modules
coming in ;-).

So, initially the i18n stuff was supposed to be moved to util, but I'd
like to know how you feel about this. There are several questions that
come to mind, for example, regarding translation files : one monolithic
file, or one file for each "subset" of text ? Shall the i18n support be a
service ? Maybe nobody cares :-p ...

Tell me what you think !

Cheers,

Jean

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#2

Hello Jean,

lorchat@sfc.wide.ad.jp wrote:

Hi Folks,

I am thinking about redesigning the i18n stuff which has some drawbacks in
the current code. For now, it is a static class in the gui service. I have
managed to override the methods to which I referred in a previous mail,
and that caused the message.properties file to be parsed in ISO-8859-1. It
is now properly read as UTF-8.

Excellent! Could you please send your patch over so that we could have a
look?

This said, I'd like to initiate a discussion about how to fit the i18n
stuff more efficiently into SC, because we will need internationalization
in other places than the GUI, especially with all the cool new modules
coming in ;-).

So, initially the i18n stuff was supposed to be moved to util, but I'd
like to know how you feel about this. There are several questions that
come to mind, for example, regarding translation files : one monolithic
file, or one file for each "subset" of text ? Shall the i18n support be a
service ? Maybe nobody cares :-p ...

Well, one monolithic file would be difficult to achieve since plugins
won't be able to use it, so we might just as well try and find a way
that would work for everyone. I am thinking of something like defining a
"resources" directory that would be located in the root of all bundles
that need i18n as well as accepting some convention as to how we should
name our resource files.

Ideally a utility that would be handling i18n is going to provide
methods which would look something like:

I18NUtility.getString(Class myCallingClass,
                      String "the.key.of.the.string.i.want");

When called, the method would check what is the currently default
locale, then try and load a resource for it and if no such resource
exists, return a value from the default one.

In terms of programming, a very easy way of implementing
I18NUtility.getString(); would be to use the ClassLoader.getResource()
method. So in case the currently selected locale is Korean, then we'd
have (quite roughly) the following calls:

public String getString(Class callerClass, String key)
{
    ...

    URL resourcesURL = callerClass.getClassLoader()
            .getResource("resources/resources.txt_kr"));

    if ( resourcesURL != null)
    {
        //.. load and use the file that resourcesURL points to
    }
    else
    {
        //asssuming resources/resources.txt is where default resource
        //names are kept
        resourcesURL = callerClass.getClassLoader()
            .getResource("resources/resources.txt"));
    }

    ...

    return value;
}

I don't know whether the "callerClass.getClassLoader().getResource()"
part would work properly from inside a bundle different from the one
that callerClass belongs to, because of all the restrictions that OSGi
imposes on class loading. If it does then the whole thing can be
implemented as a service. If it doesn't, we'd need to pack it as a
lightweight lib and include it inside every bundle that uses it.

How does this sound?

It might also be a good idea to have a bundle that would be handling
management of the current locale - a locale manager. If we decide to
implement the above as a service then both (locale manager and resource
retriever) would most probably be in the same bundle. The locale manager
would have to register a configuration GUI component in the SC config
form and menus so that users could easily switch from one locale to
another. It would also save the currently selected locale to
sip-communicator.xml in case it is different from the host system locale.

The bundle may also export a service that allows querying as well as
persistently setting the default locale.

So what do you think about all this?

Emil

···

Tell me what you think !

Cheers,

Jean

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#3

Hi Emil,

nice to hear from you :wink:

It
is now properly read as UTF-8.

Excellent! Could you please send your patch over so that we could have a
look?

Of course. I'll send it later today, as I am now in an important meeting...
The patch extends ResourceBundle and Properties to support UTF-8 as input.

As you will NOT be able to see in the patch, this used to work pretty well
on the previous architecture, with the monolithic file and the static
class. But you described the problems with a single file better in your
previous mail.

Well, one monolithic file would be difficult to achieve since plugins
won't be able to use it, so we might just as well try and find a way
that would work for everyone. I am thinking of something like defining a
"resources" directory that would be located in the root of all bundles
that need i18n as well as accepting some convention as to how we should
name our resource files.

This suits me fine. It means that when preparing a new bundle, everyone
should think about putting one file (at least !) in the resources
directory. Am I right ? And does it sound feasible ?

Ideally a utility that would be handling i18n is going to provide
methods which would look something like:

I18NUtility.getString(Class myCallingClass,
                      String "the.key.of.the.string.i.want");
When called, the method would check what is the currently default
locale, then try and load a resource for it and if no such resource
exists, return a value from the default one.

This is almost what we have. In fact, we are based on the ResourceBundle
class, and use the getString method of that class. However, you have to do
some kind of initialization first. This is where we have a problem because
the path is all messed up and I do not understand how this works in Java
:-P.

In terms of programming, a very easy way of implementing
I18NUtility.getString(); would be to use the ClassLoader.getResource()
method. So in case the currently selected locale is Korean, then we'd
have (quite roughly) the following calls:

public String getString(Class callerClass, String key)
{
    ...

    URL resourcesURL = callerClass.getClassLoader()
            .getResource("resources/resources.txt_kr"));

    if ( resourcesURL != null)
    {
        //.. load and use the file that resourcesURL points to
    }
    else
    {
        //asssuming resources/resources.txt is where default resource
        //names are kept
        resourcesURL = callerClass.getClassLoader()
            .getResource("resources/resources.txt"));
    }

    ...

    return value;
}

I don't know whether the "callerClass.getClassLoader().getResource()"
part would work properly from inside a bundle different from the one
that callerClass belongs to, because of all the restrictions that OSGi
imposes on class loading. If it does then the whole thing can be
implemented as a service. If it doesn't, we'd need to pack it as a
lightweight lib and include it inside every bundle that uses it.

Well, I had a hell of a headache with paths already, so I am not afraid to
do more testing. What I am afraid of, is whether or not we can have access
to a resources/ directory at the root of all bundles ? And does it sound a
bad idea to use resources/ directory in the bundle directory itself
instead ? It is not a very centralized process though.

Then, about your example, I have a very big concern. I wanted to use
ResourceBundle class because it parses a file and uses hash tables to
store value pairs. This is very efficient, especially when using a lot of
text. Indeed, I do not want to open the file and search for a string each
time. So if we end up using something else than ResourceBundle, we have to
cache it. In addition, ResourceBundle is neat because it is hierarchical,
and can return a resource from another "upper" language in the hierarchy,
if the string is not found in a specific "lower" language. Disclaimer : I
do not intend to classify languages, but usually, at least english has all
the localized strings. And for example, canadian french can resort to
french, or the other way round.

How does this sound?

Sounds good for me. I'll try to figure out how the path nightmare sorts
out using ClassLoader stuff.

It might also be a good idea to have a bundle that would be handling
management of the current locale - a locale manager. If we decide to
implement the above as a service then both (locale manager and resource
retriever) would most probably be in the same bundle. The locale manager
would have to register a configuration GUI component in the SC config
form and menus so that users could easily switch from one locale to
another. It would also save the currently selected locale to
sip-communicator.xml in case it is different from the host system locale.

I planned to get the locale from the xml config file, but your description
of the process is much better than I could have dreamt of...

The bundle may also export a service that allows querying as well as
persistently setting the default locale.

So what do you think about all this?

Well, I now have some pointers to push the effort a bit further. I'll tell
you if I meet any dead-end.

By the way, if someone else has insights to share about this problem, do
feel free to tell !

Cheers,

Jean

···

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net


#4

Hello Jean,

Nice to hear from you too!

lorchat@sfc.wide.ad.jp wrote:

Hi Emil,

nice to hear from you :wink:

It
is now properly read as UTF-8.

Excellent! Could you please send your patch over so that we could have a
look?

Of course. I'll send it later today, as I am now in an important meeting...
The patch extends ResourceBundle and Properties to support UTF-8 as input.

Great.

As you will NOT be able to see in the patch, this used to work pretty well
on the previous architecture, with the monolithic file and the static
class. But you described the problems with a single file better in your
previous mail.

Okay, we'll talk about this once we see the patch but I'm relatively
confident it should also work with the separate files.

Well, one monolithic file would be difficult to achieve since plugins
won't be able to use it, so we might just as well try and find a way
that would work for everyone. I am thinking of something like defining a
"resources" directory that would be located in the root of all bundles
that need i18n as well as accepting some convention as to how we should
name our resource files.

This suits me fine. It means that when preparing a new bundle, everyone
should think about putting one file (at least !) in the resources
directory. Am I right ? And does it sound feasible ?

If they want to use our utility then they should be prepared to have at
least one resource file. If they want to use our utility then they'd
also have to make sure that when their bundle is created, their resource
files are jar-ed inside a "resources" directory directly under the root
of the bundle.

Ideally a utility that would be handling i18n is going to provide
methods which would look something like:

I18NUtility.getString(Class myCallingClass,
                      String "the.key.of.the.string.i.want");
When called, the method would check what is the currently default
locale, then try and load a resource for it and if no such resource
exists, return a value from the default one.

This is almost what we have. In fact, we are based on the ResourceBundle
class, and use the getString method of that class. However, you have to do
some kind of initialization first. This is where we have a problem because
the path is all messed up and I do not understand how this works in Java
:-P.

Can you be more specific? What do you need to do in the initialization
phase? Would it be ok if we did it at bundle startup (i.e. inside the
Activator.start() method)?

<snip>
...
</snip>
Well, I had a hell of a headache with paths already, so I am not afraid to
do more testing. What I am afraid of, is whether or not we can have access
to a resources/ directory at the root of all bundles ?

I think that we can. As long as we get our hands on the right class
loader, we can go check the content of directories stored in its source
bundle. I'll do some testing and let you know.

And does it sound a
bad idea to use resources/ directory in the bundle directory itself
instead ? It is not a very centralized process though.

That would be more difficult to have since it would be the installer's
responsibility to create and fill the directory which means that it
won't work for plugins installed later.

Then, about your example, I have a very big concern. I wanted to use
ResourceBundle class because it parses a file and uses hash tables to
store value pairs. This is very efficient, especially when using a lot of
text. Indeed, I do not want to open the file and search for a string each
time.

True ... I hadn't really thought of that but we can easily make our
ResourceUtility service cache resource files against the ClassLoaders
that were used for testing them.

So if we end up using something else than ResourceBundle, we have to
cache it. In addition, ResourceBundle is neat because it is hierarchical,
and can return a resource from another "upper" language in the hierarchy,
if the string is not found in a specific "lower" language. Disclaimer : I
do not intend to classify languages, but usually, at least english has all
the localized strings. And for example, canadian french can resort to
french, or the other way round.

I agree that it would indeed be great to stick with ResourceBundles, and
I'd have to confess, that I don't really see why we couldn't. Is there
something I am missing?

How does this sound?

Sounds good for me. I'll try to figure out how the path nightmare sorts
out using ClassLoader stuff.

It might also be a good idea to have a bundle that would be handling
management of the current locale - a locale manager. If we decide to
implement the above as a service then both (locale manager and resource
retriever) would most probably be in the same bundle. The locale manager
would have to register a configuration GUI component in the SC config
form and menus so that users could easily switch from one locale to
another. It would also save the currently selected locale to
sip-communicator.xml in case it is different from the host system locale.

I planned to get the locale from the xml config file, but your description
of the process is much better than I could have dreamt of...

Ha, come on!

I'll go for that resource testing now and let you know of my findings.

Cheers
Emil

···

The bundle may also export a service that allows querying as well as
persistently setting the default locale.

So what do you think about all this?

Well, I now have some pointers to push the effort a bit further. I'll tell
you if I meet any dead-end.

By the way, if someone else has insights to share about this problem, do
feel free to tell !

Cheers,

Jean

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@sip-communicator.dev.java.net
For additional commands, e-mail: dev-help@sip-communicator.dev.java.net