6 Feb
2013
6 Feb
'13
9:08 a.m.
Hi list,
Attached 3 patches for decoding UCS2 MT-SMS and incoming USSD.
Main idea: convert UCS2 text to UTF-8, check if terminal supports
UTF-8 and if yes - print, otherwise - print in hex, and notify VTY to
check mobile log (since we can't send UTF text to VTY, if someone
knows how to do it - please tell me).
Waiting for comments.
Thanks,
Pavel
6 Feb
6 Feb
12:09 p.m.
Pavel Baturko wrote:
Attached 3 patches for decoding UCS2 MT-SMS and
incoming USSD.
Main idea: convert UCS2 text to UTF-8, check if terminal supports
UTF-8 and if yes - print, otherwise - print in hex, and notify VTY to
check mobile log (since we can't send UTF text to VTY, if someone
knows how to do it - please tell me).
hi paval,
the patches for osmocombb look good to me, except that the UTF encoded
text is not only displayed on VTY, but also not written to sms.txt file.
look at gsm340_rx_sms_deliver() where linefeeds from sms-text are
removed for the vty output. you could also remove (or replace by '?')
non-ascii characters there. then it can be displayed on VTY too.
you can apply the osmocombb patches, after the libosmocore patch is
applied to master of libosmocore. for applying the libosmocore patch, i
cannot give green light, because holger was recently working on sms
code, so i would ask him.
regards,
andreas
12:40 p.m.
Hi Andreas,
Thanks for comments.
the patches for osmocombb look good to me, except that
the UTF encoded
text is not only displayed on VTY, but also not written to sms.txt file.
you could also remove (or replace by '?')
non-ascii characters there. then it can be displayed on VTY too.
Am I right that
you are proposing to display only ascii part of text
and save it also in sms.txt file instead of status messages like 'SMS
was printed ...'? For all non-ascii SMSs this will looks like '??? ??
?????' at VTY, is it ok?
you can apply the osmocombb patches, after the
libosmocore patch is
applied to master of libosmocore. for applying the libosmocore patch, i
cannot give green light, because holger was recently working on sms
code, so i would ask him.
I'll wait for Holger's answer.
Thanks,
Pavel
1 p.m.
Pavel Baturko wrote:
Am I right that you are proposing to display only
ascii part of text
and save it also in sms.txt file instead of status messages like 'SMS
was printed ...'? For all non-ascii SMSs this will looks like '??? ??
?????' at VTY, is it ok?
i don't know about coding of UTF, but wikipedia reads that all ASCII
characters are UTF-8 characters, so i guess that all characters except
non-ASCII character should be visible. am i wrong?
1:21 p.m.
Hi,
i don't know about coding of UTF, but wikipedia
reads that all ASCII
characters are UTF-8 characters, so i guess that all characters except
non-ASCII character should be visible. am i wrong?
Yes, they will be visible, but
I'm talking about SMSs without ASCII
chars except spaces/numbers/dots/commas/etc. In that case all words
(in national language for example) will not be visible at VTY and user
will see many '?' symbols if change non-ascii chars to '?' and
spaces/comms/etc, and I think such representation of original text
will be not very good (if I correctly understand your first comment).
Thanks,
Pavel
2:21 p.m.
Yes, they will be visible, but I'm talking about
SMSs without ASCII
chars except spaces/numbers/dots/commas/etc. In that case all words
(in national language for example) will not be visible at VTY and user
will see many '?' symbols if change non-ascii chars to '?' and
spaces/comms/etc, and I think such representation of original text
will be not very good (if I correctly understand your first comment).
ok then, why not print your message about encoded type to VTY, but store
message in sms.txt. the gsm340_rx_sms_deliver() function could take two
arguments, one text for the vty and one to store.
7 Feb
7 Feb
11:26 p.m.
Hi,
Attached new version. SMS text will be saved to sms.txt file in any
case (in case of UCS2 SMSs with non-ASCII chars this file should be
opened with UTF-8 encoding) but on VTY user will see notice that SMS
text can't be printed.
Thanks,
Pavel
8 Feb
8 Feb
8:24 a.m.
Hi,
Attached new version. SMS text will be saved to sms.txt file in any
case (in case of UCS2 SMSs with non-ASCII chars this file should be
opened with UTF-8 encoding) but on VTY user will see notice that SMS
text can't be printed.
Thanks,
Pavel
hi pavael,
this looks good to me now. you can commit it as soon as the libosmocore
patch is accepted and committed.
best regards,
andreas
8:48 a.m.
this looks good to me now. you can commit it as soon
as the libosmocore
patch is accepted and committed.
Should I ask someone for committing to libosmocore?
And can I commit to osmocom-bb repo directly (I think I can't)? Or I
should re-ask for committing when libosmocore will be ready?
Thanks,
Pavel
10:39 a.m.
Hi Pavel,
sorry to be the party pooper, but I don't like the libosmocore patch for
two reasons:
1) a popen to the locale command is really ugly. First of all, it will
synchronously block the process issuing that command. Typical
libosmocore-using processes are single-threaded and expect library
calls to be non-blocking. Secondly, I really hate the idea of
spawning other processes, executing other executables. It sounds
lieke an ugly hack. Is there no way to get the information
programmatically, e.g. by using localeconv(3) ?
2) the function "ucs2_to_utf8_decode" is cluttering the global namespace
even further. Yes, it is in principle not a function related to
gsm, but I'd still appreciate if it would start with osmo_ or gsm_
prefix in order to make sure we don't clash with applications or
other libraries using the same symbol name.
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
10:55 a.m.
Hi Harald,
Is there no way to get the information
programmatically, e.g. by using localeconv(3) ?
Another way to get this
information that I know is smthng like that:
+ old_locale = setlocale(LC_ALL, NULL);
+ curr_locale = setlocale(LC_ALL, "");
+ for (i = 0; i < strlen(curr_locale); i++)
+ curr_locale[i] = tolower(curr_locale[i]);
+ have_utf8_locale = ((strstr(curr_locale, "utf-8") != NULL) ||
+ (strstr(curr_locale, "utf8" ) != NULL)) ? 1 : 0;
+ setlocale(LC_ALL, old_locale);
Not sure if it's ok because of temporary changing of current locale.
unfortunately I was not able to find better ways to check locale.
2) the function "ucs2_to_utf8_decode" is
cluttering the global namespace
even further. Yes, it is in principle not a function related to
gsm, but I'd still appreciate if it would start with osmo_ or gsm_
prefix in order to make sure we don't clash with applications or
other libraries using the same symbol name.
I can move it to utils.c, is it
acceptable? And of course add "osmo_" prefix.
Thanks,
Pavel
11:14 a.m.
Hi Pavel,
On Fri, Feb 08, 2013 at 01:55:52PM +0400, Pavel Baturko wrote:
yes, that would be appreciated, thanks.
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
Another way to get this information that I know is
smthng like that:
+ old_locale = setlocale(LC_ALL, NULL);
+ curr_locale = setlocale(LC_ALL, "");
+ for (i = 0; i < strlen(curr_locale); i++)
+ curr_locale[i] = tolower(curr_locale[i]);
+ have_utf8_locale = ((strstr(curr_locale, "utf-8") != NULL) ||
+ (strstr(curr_locale, "utf8" ) != NULL)) ? 1 : 0;
+ setlocale(LC_ALL, old_locale);
Not sure if it's ok because of temporary changing of current locale.
unfortunately I was not able to find better ways to check locale.
the locale setting is only for the current program. And as indicated,
at least so far, most libosmocore users are single-threaded, i.e. nobody
would notice if the locale was different for a very short amount of
time. Furthermore, assuming that the code is not called all that often
(i.e. only once at program startup) I wouldn't be worried about that so
much.
I am not an expert in locale.h matters, but it would somehow be strange
to me if you cannot determine utf-8 support without having to actually
change the locale. Is anyone on this list more familiar with it?
2) the
function "ucs2_to_utf8_decode" is cluttering the global namespace
even further. Yes, it is in principle not a function related to
gsm, but I'd still appreciate if it would start with osmo_ or gsm_
prefix in order to make sure we don't clash with applications or
other libraries using the same symbol name.
I can move it to utils.c, is it
acceptable? And of course add "osmo_" prefix.
11:29 a.m.
Hi Harald,
the locale setting is only for the current program.
And as indicated,
at least so far, most libosmocore users are single-threaded, i.e. nobody
would notice if the locale was different for a very short amount of
time. Furthermore, assuming that the code is not called all that often
(i.e. only once at program startup) I wouldn't be worried about that so
much.
I can add some kind of global var which will indicate UTF-8 support
and init it when app starting with code above (or other code if will
find better one). Then use it in case of SMS/USSD receving.
I am not an expert in locale.h matters, but it would
somehow be strange
to me if you cannot determine utf-8 support without having to actually
change the locale. Is anyone on this list more familiar with it?
I asked some time
ago, but there was no answer.. setlocale function
with NULL argument always returns "C" - Minimal "C" locale - if
locale
was not changed before, with arguments rather than NULL - changes the
locale.
Thanks,
Pavel
1:24 p.m.
Hi Pavel,
On Fri, Feb 08, 2013 at 02:29:17PM +0400, Pavel Baturko wrote:
This sounds reasonable. Please make the global variable static and hide
it to the caller. So only on the first call to the function it will do
the setlocale() trick, then store the result in a variable. All further
calls to the function will then retunr the variable instead of
setlocale() and friends.
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
time.
Furthermore, assuming that the code is not called all that often
(i.e. only once at program startup) I wouldn't be worried about that so
much.
I can add some kind of global var which will indicate UTF-8 support
and init it when app starting with code above (or other code if will
find better one). Then use it in case of SMS/USSD receving.
10 Feb
10 Feb
12:55 p.m.
Hi,
Here are updated versions of patches:
Functions for converting ucs2->utf8 and checking current locale are
moved to util.c.
Function for checking current locale rewrited to use 'setlocale'
instead of external 'locale' cmd call.
Support of UTF is checked just once in 'l23_app_init' and then used
via static var.
Thanks,
Pavel
13 Feb
13 Feb
3:30 p.m.
Hi Harald,
Have you had a chance to look at last versions of patches?
Thanks,
Pavel
On Sun, Feb 10, 2013 at 11:55 AM, Pavel Baturko <pabftk(a)gmail.com> wrote:
Hi,
Here are updated versions of patches:
Functions for converting ucs2->utf8 and checking current locale are
moved to util.c.
Function for checking current locale rewrited to use 'setlocale'
instead of external 'locale' cmd call.
Support of UTF is checked just once in 'l23_app_init' and then used
via static var.
Thanks,
Pavel
27 Feb
27 Feb
5:30 a.m.
Just kindly reminder..
Thanks,
Pavel
On Wed, Feb 13, 2013 at 2:30 PM, Pavel Baturko <pabftk(a)gmail.com> wrote:
Hi Harald,
Have you had a chance to look at last versions of patches?
Thanks,
Pavel
On Sun, Feb 10, 2013 at 11:55 AM, Pavel Baturko <pabftk(a)gmail.com> wrote:
> Hi,
>
> Here are updated versions of patches:
> Functions for converting ucs2->utf8 and checking current locale are
> moved to util.c.
> Function for checking current locale rewrited to use 'setlocale'
> instead of external 'locale' cmd call.
> Support of UTF is checked just once in 'l23_app_init' and then used
> via static var.
>
> Thanks,
> Pavel
4087
days inactive
4108
days old
baseband-devel@lists.osmocom.org
17 comments
3 participants
participants (3)
-
Harald Welte -
jolly -
Pavel Baturko