Hi all,
I noticed that SMS with emoticons on the boundary of the concatenated segments are not displaying correctly on the destination handset.
* - imagine the disaster when that kiss-blowing smiley face thing at the end of your SMS turns out as a diamond with a ? for the recpipient! OMG, the potential butterfly effect is too much to think about... :))
So... This is my analysis:
We have 140 bytes for the message, less the 6 bytes for the user header data in the case of concatenated SMS , leaving 134 bytes.
It's well know that this means 67 characters per segment for an SMS using UCS-2 encoding.
But, it we fill the message with emoticons that are using 4 bytes per 'character', then we have space for 134/4 = 33 and a half. Ooops.
Still, the destination handset should reassemble the message and stick the two "halves" of the emoticon together, right? - except I'm not observing this.
To rule out us doing something wrong in osmo, I wonder might somebody else (who has an unlimited SMS package) from a commercial provider try sending some crafted SM from one emoji-enabled phone to another, something like this:
π±abcdefghijklmnopqrstuvwxyz123456789abcdefghijklmnopqrstuvwxyz123π±56789
That's likely going to get mangled by mailing list or your mua, so it is:
[4 byte emoji]abcdefghijklmnopqrstuvwxyz123456789abcdefghijklmnopqrstuvwxyz123[4 byte emoji]56789
You could also try various messages filled with emojis. Of course, if you bring up your osmo network with SMPP-mirror you can watch the trace in wireshark, you'll see when the last emoji gets chopped in two. You could try the same message on osmo and your commerical network.
If it's actually a problem of the phones, you should get
[4 byte emoji]abcdefghijklmnopqrstuvwxyz123456789abcdefghijklmnopqrstuvwxyz123[two diamonds with ?]56789
Thanks!
Keith.
With my commercial operator:
Tried to compose an SMS with emojis but my current SMS app doesn't seem to support them (FP2 Open). I guess I could set signal as my SMS app...
I sent myself 40 emoticons and at least signal displays them correctly.
Then I sent 40 and quickly changed the default SMS app back to the stock Messaging thing to receive the SMS there, and it also shows all 40 intact. Notably displayed as emoticons, apparently it does support them after all.
~N
On Mon, Nov 12, 2018 at 04:54:52PM +0100, Neels Hofmeyr wrote:
With my commercial operator: [...]
I guess it would make sense to replicate this experiment using a OsmoBTS/BSC on the RAN side but a "commerical core" on the CN side. This way one can take protocol traces of [not only] the Layer3 communication (CP/RP/TP layers) and compare the osmocom vs. non-osmocom signaling.
On 12/11/2018 16:54, Neels Hofmeyr wrote:
With my commercial operator:
Hi Neels, thanks for doing this test, sorry i got distracted from this and did not reply.
I sent myself 40 emoticons and at least signal displays them correctly.
Then I sent 40 and quickly changed the default SMS app back to the stock Messaging thing to receive the SMS there, and it also shows all 40 intact. Notably displayed as emoticons, apparently it does support them after all.
Yes, probably a keyboard issue, more than the app itself. Signal has it's own built in emoji composer, and some keyboards have weird ways to get to the emoji keyboard - Searching about FP2, I came across this phrase "Just keep pressed the bottom right button (the green one) and the option shows. Then move your finger to the left while still touching the screen.", if that makes any sense to you.
Worst case, you could just compose in signal, then copy + paste. No need to switch the App I think.
As Harald said, it would be good to test with a commercial core network, but I don't have one :)
If you could quickly try this with your (or any) phone next time you have it connected to your Osmocom network, that would confirm it is not just my phones. (I really don't see anything out of the ordinary on my protocol traces.)
Given that last time the emojis I posted to ML looked ok in a few MUAs, here's the result of sending some emoji (copied from SMS app to K9 mail - > saved to IMAP draft -> thunderbird.
with latest MSC, no SMPP mode:
Sent:
ππππππππππππππππππππππππππππππππππππππππ
Rcvd:
ππππππππππππππππππππππππππππππππποΏ½οΏ½ππππππ
Thanks!
k/
On 20.11.18 09:04, Keith wrote:
As Harald said, it would be good to test with a commercial core network, but I don't have one :)
I don't know if this helps, but I got curious and sent the 40-emoji-message from a phone connected to a commercial network and captured it via SS7. (So no Osmocom involved.)
The pcap file is attached (addresses removed).
It looks like the 4-byte characters don't get split in half, but the last two bytes get pushed to the next message, and all emojis are received unharmed.
One thing I don't get though: where is it even specified that you can use UTF-16 in SMS? Even the latest 23.038 only lists UCS-2.
-Tobias