Fast SIM cards loosing bytes

Min Xu mxu at sanjole.com
Fri Jan 24 21:37:26 UTC 2014


Dean

Just noticed that you wanted the firmware.  So I am including the
pre-compiled image in case you didn't want to recompile yourself.


On Fri, Jan 24, 2014 at 11:24 AM, Min Xu <mxu at sanjole.com> wrote:

> It's incompatible because due to the way the USB peripheral transfer the
> data out to the host, there is NO mechanism to indicate exactly where the
> next USB transaction should start.  Therefore, when the data is flowing
> very fast, every 64 byte (the bus width) write to the USB peripheral will
> be concatenated together, resulting in the host read to combine multiple
> req_ctx buffers.
>
> The way I solved it is by adding fields to the simtrace_hdr structure:
> (the + indicates the new fields).  This allows me on the host to keep track
> if / where a new req_ctx begins in the received usb data.
>
>  struct simtrace_hdr {
>         u_int8_t cmd;
>         u_int8_t flags;
>         u_int8_t res[2];
> +       u_int16_t seq_num;
> +       u_int16_t offset;
> +       u_int16_t tot_len;
>         u_int8_t data[0];
>  } __attribute__ ((packed));
>
>
> Since the laptop that I was using to create multiple commits is not
> available at the moment, I am attaching the   git  diff   from the
> repository (instead of from my last commits in previous email).  Meaning
> this is a single diff file.
>
> For the host software, the basic change in the process_usb_msg function.
> My implementation is a naive implementation using a much larger buffer to
> copy data to, where if a req_ctx is split across multiple usb transfers, it
> will join those fragments then process:
>
> struct simtrace_hdr {
>     u_int8_t cmd;
>     u_int8_t flags;
>     u_int8_t res[2];
>     u_int16_t seq_num;
>     u_int16_t offset;
>     u_int16_t tot_len;
>     u_int8_t data[0];
> } /* __attribute__ ((packed)) */;
>
> static char usb_buffer[66000];
> static unsigned usb_remainder = 0;
>
> int process_usb_msg(uint8_t *buf, int len) {
>     struct simtrace_hdr *sh;
>     uint8_t *payload;
>     int payload_len, sh_offset, i;
>
>     sendto(s, buf, len, 0, (struct sockaddr*)&destUSB, sizeof(struct
> sockaddr_in));
>
>     memcpy(usb_buffer + usb_remainder, buf, len);
>     len += usb_remainder;
>     usb_remainder = len;
>     sh_offset = 0;
>     sh = usb_buffer;
>     payload = (char*)sh + sizeof(struct simtrace_hdr);
>     payload_len = sh->tot_len - sizeof(struct simtrace_hdr);
>     if (usb_remainder < sh->tot_len)
>         sh = NULL;
>
>     while (sh) {
>         switch (sh->cmd) {
>             case SIMTRACE_MSGT_DATA:
>                 /* special treatment for ATR */
>                 if (sh->flags & SIMTRACE_FLAG_ATR) {
>                     apdu_split_reset(as);
>                     break;
>                 }
>                 if (sh->flags & SIMTRACE_FLAG_PPS_FIDI) {
>                     module_out("PPS(Fi=%u/Di=%u)\n",
>                         sh->res[0], sh->res[1]);
>                 }
>                 /* everything else goes into APDU splitter */
>                 apdu_split_in(as, payload, payload_len);
>         #if 0
>                 /* If waiting time has expired, signal explicit boundary */
>                 if (sh->flags & SIMTRACE_FLAG_WTIME_EXP)
>                     apdu_split_boundary(as);
>         #endif
>                 break;
>             case SIMTRACE_MSGT_RESET:
>             default:
>                 module_out("unknown simtrace msg type 0x%02x\n", sh->cmd);
>                 break;
>         }
>         sh_offset += sh->tot_len;
>         sh = usb_buffer + sh_offset;
>         if (sh_offset < len) {
>             if (sh_offset + sh->tot_len > len) {
>                 memcpy(usb_buffer, sh, len - sh_offset);
>                 usb_remainder = len - sh_offset;
>                 sh = NULL;
>             } else {
>                 payload = (char*)sh + sizeof(struct simtrace_hdr);
>                 payload_len = sh->tot_len - sizeof(struct simtrace_hdr);
>             }
>         } else {
>             sh = NULL;
>             usb_remainder = 0;
>         }
>     }
>     return 0;
> }
>
>
>
>
>
> On Fri, Jan 24, 2014 at 11:06 AM, Dean Chester <dean.g.chester at gmail.com>wrote:
>
>> Hi min,
>>
>> How do you mean that they would be incompatible with early firmware based
>> tracers?
>>
>> I'd be interested in using your firmware if you were to make it
>> available, I'm also working on a more user friendly host software.
>>
>> Kind regards
>>
>> Dean Chester.
>>
>> Sent from my iPad
>>
>> On 24 Jan 2014, at 19:41, Min Xu <mxu at sanjole.com> wrote:
>>
>> Hi Holger
>>
>> To give you an update.  As far as I can tell, the changes I have
>> submitted, with the addition of a change (on my local system) where the USB
>> protocol has been modified to remove USB overrun, is stable so I can
>> monitor iPhone 5S as well as Galaxy S4 continuously.  If anyone asks, I
>> would be happy to send them the USB protocol changes.  However, it will be
>> INCOMPATIBLE with earlier firmware based SIMTrace boards.
>>
>> I am making changes to the host program (on Windows) to monitor multiple
>> SIMTrace devices, and am interested to know if anyone else is working on
>> something similar.
>>
>> Best Regards
>>
>>
>> Date: Wed, 22 Jan 2014 21:27:31 +0100
>>> From: Holger Hans Peter Freyther <holger at freyther.de>
>>> To: Dean Chester <dean.g.chester at gmail.com>
>>> Cc: "simtrace at lists.osmocom.org" <simtrace at lists.osmocom.org>
>>> Subject: Re: Fast SIM cards loosing bytes
>>> Message-ID: <20140122202731.GL15138 at xiaoyu.lan>
>>> Content-Type: text/plain; charset=us-ascii
>>>
>>> On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
>>> > Hi,
>>> >
>>> > I see earlier in the archives that reducing the buffer size in the
>>> firmware will resolve this. Hasn't this already been done though in new
>>> releases of the firmware? As I have pulled the latest version from the
>>> openpcd git repo and built it. If not what have people reduced the buffer
>>> size to? And has it had any fixes?
>>>
>>> The closest are the patches that Min Xu sent. They require some
>>> clean-ups and review (specially the code changing the interrupt
>>> handling).
>>>
>>> holger
>>>
>>>
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> simtrace mailing list
>>> simtrace at lists.osmocom.org
>>> https://lists.osmocom.org/mailman/listinfo/simtrace
>>>
>>>
>>> End of simtrace Digest, Vol 33, Issue 4
>>> ***************************************
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/simtrace/attachments/20140124/7b0d1afe/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main_simtrace.bin
Type: application/octet-stream
Size: 22860 bytes
Desc: not available
URL: <http://lists.osmocom.org/pipermail/simtrace/attachments/20140124/7b0d1afe/attachment.bin>


More information about the simtrace mailing list