Hi,
I see earlier in the archives that reducing the buffer size in the firmware will resolve this. Hasn't this already been done though in new releases of the firmware? As I have pulled the latest version from the openpcd git repo and built it. If not what have people reduced the buffer size to? And has it had any fixes?
Sent from my iPad
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the firmware will resolve this. Hasn't this already been done though in new releases of the firmware? As I have pulled the latest version from the openpcd git repo and built it. If not what have people reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some clean-ups and review (specially the code changing the interrupt handling).
holger
Hi Holger
To give you an update. As far as I can tell, the changes I have submitted, with the addition of a change (on my local system) where the USB protocol has been modified to remove USB overrun, is stable so I can monitor iPhone 5S as well as Galaxy S4 continuously. If anyone asks, I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple SIMTrace devices, and am interested to know if anyone else is working on something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther holger@freyther.de To: Dean Chester dean.g.chester@gmail.com Cc: "simtrace@lists.osmocom.org" simtrace@lists.osmocom.org Subject: Re: Fast SIM cards loosing bytes Message-ID: 20140122202731.GL15138@xiaoyu.lan Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the
firmware will resolve this. Hasn't this already been done though in new releases of the firmware? As I have pulled the latest version from the openpcd git repo and built it. If not what have people reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some clean-ups and review (specially the code changing the interrupt handling).
holger
simtrace mailing list simtrace@lists.osmocom.org https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
Hi min,
How do you mean that they would be incompatible with early firmware based tracers?
I'd be interested in using your firmware if you were to make it available, I'm also working on a more user friendly host software.
Kind regards
Dean Chester.
Sent from my iPad
On 24 Jan 2014, at 19:41, Min Xu mxu@sanjole.com wrote:
Hi Holger
To give you an update. As far as I can tell, the changes I have submitted, with the addition of a change (on my local system) where the USB protocol has been modified to remove USB overrun, is stable so I can monitor iPhone 5S as well as Galaxy S4 continuously. If anyone asks, I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple SIMTrace devices, and am interested to know if anyone else is working on something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100 From: Holger Hans Peter Freyther holger@freyther.de To: Dean Chester dean.g.chester@gmail.com Cc: "simtrace@lists.osmocom.org" simtrace@lists.osmocom.org Subject: Re: Fast SIM cards loosing bytes Message-ID: 20140122202731.GL15138@xiaoyu.lan Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the firmware will resolve this. Hasn't this already been done though in new releases of the firmware? As I have pulled the latest version from the openpcd git repo and built it. If not what have people reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some clean-ups and review (specially the code changing the interrupt handling).
holger
simtrace mailing list simtrace@lists.osmocom.org https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
Min Xu wrote:
I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB; change either the bDeviceProtocol field in the device descriptor or the bInterfaceProtocol field in the interface descriptor, and make host software do the appropriate thing based on the descriptors of the connected device.
Of course only new host software will work with the new protocol, but this way new host software still continues to work with the old protocol.
//Peter
Thanks Min it works a treat it also fixes the issues running in a virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote:
Min Xu wrote:
I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB; change either the bDeviceProtocol field in the device descriptor or the bInterfaceProtocol field in the interface descriptor, and make host software do the appropriate thing based on the descriptors of the connected device.
Of course only new host software will work with the new protocol, but this way new host software still continues to work with the old protocol.
//Peter
It's incompatible because due to the way the USB peripheral transfer the data out to the host, there is NO mechanism to indicate exactly where the next USB transaction should start. Therefore, when the data is flowing very fast, every 64 byte (the bus width) write to the USB peripheral will be concatenated together, resulting in the host read to combine multiple req_ctx buffers.
The way I solved it is by adding fields to the simtrace_hdr structure: (the + indicates the new fields). This allows me on the host to keep track if / where a new req_ctx begins in the received usb data.
struct simtrace_hdr { u_int8_t cmd; u_int8_t flags; u_int8_t res[2]; + u_int16_t seq_num; + u_int16_t offset; + u_int16_t tot_len; u_int8_t data[0]; } __attribute__ ((packed));
Since the laptop that I was using to create multiple commits is not available at the moment, I am attaching the git diff from the repository (instead of from my last commits in previous email). Meaning this is a single diff file.
For the host software, the basic change in the process_usb_msg function. My implementation is a naive implementation using a much larger buffer to copy data to, where if a req_ctx is split across multiple usb transfers, it will join those fragments then process:
struct simtrace_hdr { u_int8_t cmd; u_int8_t flags; u_int8_t res[2]; u_int16_t seq_num; u_int16_t offset; u_int16_t tot_len; u_int8_t data[0]; } /* __attribute__ ((packed)) */;
static char usb_buffer[66000]; static unsigned usb_remainder = 0;
int process_usb_msg(uint8_t *buf, int len) { struct simtrace_hdr *sh; uint8_t *payload; int payload_len, sh_offset, i;
sendto(s, buf, len, 0, (struct sockaddr*)&destUSB, sizeof(struct sockaddr_in));
memcpy(usb_buffer + usb_remainder, buf, len); len += usb_remainder; usb_remainder = len; sh_offset = 0; sh = usb_buffer; payload = (char*)sh + sizeof(struct simtrace_hdr); payload_len = sh->tot_len - sizeof(struct simtrace_hdr); if (usb_remainder < sh->tot_len) sh = NULL;
while (sh) { switch (sh->cmd) { case SIMTRACE_MSGT_DATA: /* special treatment for ATR */ if (sh->flags & SIMTRACE_FLAG_ATR) { apdu_split_reset(as); break; } if (sh->flags & SIMTRACE_FLAG_PPS_FIDI) { module_out("PPS(Fi=%u/Di=%u)\n", sh->res[0], sh->res[1]); } /* everything else goes into APDU splitter */ apdu_split_in(as, payload, payload_len); #if 0 /* If waiting time has expired, signal explicit boundary */ if (sh->flags & SIMTRACE_FLAG_WTIME_EXP) apdu_split_boundary(as); #endif break; case SIMTRACE_MSGT_RESET: default: module_out("unknown simtrace msg type 0x%02x\n", sh->cmd); break; } sh_offset += sh->tot_len; sh = usb_buffer + sh_offset; if (sh_offset < len) { if (sh_offset + sh->tot_len > len) { memcpy(usb_buffer, sh, len - sh_offset); usb_remainder = len - sh_offset; sh = NULL; } else { payload = (char*)sh + sizeof(struct simtrace_hdr); payload_len = sh->tot_len - sizeof(struct simtrace_hdr); } } else { sh = NULL; usb_remainder = 0; } } return 0; }
On Fri, Jan 24, 2014 at 11:06 AM, Dean Chester dean.g.chester@gmail.comwrote:
Hi min,
How do you mean that they would be incompatible with early firmware based tracers?
I'd be interested in using your firmware if you were to make it available, I'm also working on a more user friendly host software.
Kind regards
Dean Chester.
Sent from my iPad
On 24 Jan 2014, at 19:41, Min Xu mxu@sanjole.com wrote:
Hi Holger
To give you an update. As far as I can tell, the changes I have submitted, with the addition of a change (on my local system) where the USB protocol has been modified to remove USB overrun, is stable so I can monitor iPhone 5S as well as Galaxy S4 continuously. If anyone asks, I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple SIMTrace devices, and am interested to know if anyone else is working on something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther holger@freyther.de To: Dean Chester dean.g.chester@gmail.com Cc: "simtrace@lists.osmocom.org" simtrace@lists.osmocom.org Subject: Re: Fast SIM cards loosing bytes Message-ID: 20140122202731.GL15138@xiaoyu.lan Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the
firmware will resolve this. Hasn't this already been done though in new releases of the firmware? As I have pulled the latest version from the openpcd git repo and built it. If not what have people reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some clean-ups and review (specially the code changing the interrupt handling).
holger
simtrace mailing list simtrace@lists.osmocom.org https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
Dean
Just noticed that you wanted the firmware. So I am including the pre-compiled image in case you didn't want to recompile yourself.
On Fri, Jan 24, 2014 at 11:24 AM, Min Xu mxu@sanjole.com wrote:
It's incompatible because due to the way the USB peripheral transfer the data out to the host, there is NO mechanism to indicate exactly where the next USB transaction should start. Therefore, when the data is flowing very fast, every 64 byte (the bus width) write to the USB peripheral will be concatenated together, resulting in the host read to combine multiple req_ctx buffers.
The way I solved it is by adding fields to the simtrace_hdr structure: (the + indicates the new fields). This allows me on the host to keep track if / where a new req_ctx begins in the received usb data.
struct simtrace_hdr { u_int8_t cmd; u_int8_t flags; u_int8_t res[2];
u_int16_t seq_num;u_int16_t offset;u_int16_t tot_len; u_int8_t data[0];} __attribute__ ((packed));
Since the laptop that I was using to create multiple commits is not available at the moment, I am attaching the git diff from the repository (instead of from my last commits in previous email). Meaning this is a single diff file.
For the host software, the basic change in the process_usb_msg function. My implementation is a naive implementation using a much larger buffer to copy data to, where if a req_ctx is split across multiple usb transfers, it will join those fragments then process:
struct simtrace_hdr { u_int8_t cmd; u_int8_t flags; u_int8_t res[2]; u_int16_t seq_num; u_int16_t offset; u_int16_t tot_len; u_int8_t data[0]; } /* __attribute__ ((packed)) */;
static char usb_buffer[66000]; static unsigned usb_remainder = 0;
int process_usb_msg(uint8_t *buf, int len) { struct simtrace_hdr *sh; uint8_t *payload; int payload_len, sh_offset, i;
sendto(s, buf, len, 0, (struct sockaddr*)&destUSB, sizeof(structsockaddr_in));
memcpy(usb_buffer + usb_remainder, buf, len); len += usb_remainder; usb_remainder = len; sh_offset = 0; sh = usb_buffer; payload = (char*)sh + sizeof(struct simtrace_hdr); payload_len = sh->tot_len - sizeof(struct simtrace_hdr); if (usb_remainder < sh->tot_len) sh = NULL; while (sh) { switch (sh->cmd) { case SIMTRACE_MSGT_DATA: /* special treatment for ATR */ if (sh->flags & SIMTRACE_FLAG_ATR) { apdu_split_reset(as); break; } if (sh->flags & SIMTRACE_FLAG_PPS_FIDI) { module_out("PPS(Fi=%u/Di=%u)\n", sh->res[0], sh->res[1]); } /* everything else goes into APDU splitter */ apdu_split_in(as, payload, payload_len); #if 0 /* If waiting time has expired, signal explicit boundary */ if (sh->flags & SIMTRACE_FLAG_WTIME_EXP) apdu_split_boundary(as); #endif break; case SIMTRACE_MSGT_RESET: default: module_out("unknown simtrace msg type 0x%02x\n", sh->cmd); break; } sh_offset += sh->tot_len; sh = usb_buffer + sh_offset; if (sh_offset < len) { if (sh_offset + sh->tot_len > len) { memcpy(usb_buffer, sh, len - sh_offset); usb_remainder = len - sh_offset; sh = NULL; } else { payload = (char*)sh + sizeof(struct simtrace_hdr); payload_len = sh->tot_len - sizeof(struct simtrace_hdr); } } else { sh = NULL; usb_remainder = 0; } } return 0;}
On Fri, Jan 24, 2014 at 11:06 AM, Dean Chester dean.g.chester@gmail.comwrote:
Hi min,
How do you mean that they would be incompatible with early firmware based tracers?
I'd be interested in using your firmware if you were to make it available, I'm also working on a more user friendly host software.
Kind regards
Dean Chester.
Sent from my iPad
On 24 Jan 2014, at 19:41, Min Xu mxu@sanjole.com wrote:
Hi Holger
To give you an update. As far as I can tell, the changes I have submitted, with the addition of a change (on my local system) where the USB protocol has been modified to remove USB overrun, is stable so I can monitor iPhone 5S as well as Galaxy S4 continuously. If anyone asks, I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple SIMTrace devices, and am interested to know if anyone else is working on something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther holger@freyther.de To: Dean Chester dean.g.chester@gmail.com Cc: "simtrace@lists.osmocom.org" simtrace@lists.osmocom.org Subject: Re: Fast SIM cards loosing bytes Message-ID: 20140122202731.GL15138@xiaoyu.lan Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the
firmware will resolve this. Hasn't this already been done though in new releases of the firmware? As I have pulled the latest version from the openpcd git repo and built it. If not what have people reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some clean-ups and review (specially the code changing the interrupt handling).
holger
simtrace mailing list simtrace@lists.osmocom.org https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
I am contributing it to the project. Once I incorporate Peter Stuge's suggestion, hopefully within the next few weeks I will submit another commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester dean.g.chester@gmail.comwrote:
Thanks Min it works a treat it also fixes the issues running in a virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote:
Min Xu wrote:
I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB; change either the bDeviceProtocol field in the device descriptor or the bInterfaceProtocol field in the interface descriptor, and make host software do the appropriate thing based on the descriptors of the connected device.
Of course only new host software will work with the new protocol, but this way new host software still continues to work with the old protocol.
//Peter
P.S. What problems were you experiencing of it running in a virtualized system, can you elaborate?
Thanks
On Mon, Jan 27, 2014 at 8:36 AM, Min Xu mxu@sanjole.com wrote:
I am contributing it to the project. Once I incorporate Peter Stuge's suggestion, hopefully within the next few weeks I will submit another commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester dean.g.chester@gmail.comwrote:
Thanks Min it works a treat it also fixes the issues running in a virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote:
Min Xu wrote:
I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB; change either the bDeviceProtocol field in the device descriptor or the bInterfaceProtocol field in the interface descriptor, and make host software do the appropriate thing based on the descriptors of the connected device.
Of course only new host software will work with the new protocol, but this way new host software still continues to work with the old protocol.
//Peter
That does make sense to me since I had issues where before I changed the usb header, the host was sometimes merging multiple req_ctx into a single usb read.
I am attaching a tar.gz file containing all my changes as separate git commits / format-patches and hopefully it will get committed to the repository. This includes the latest change to the usb header as well as a change (as Peter suggested) to the bDeviceProtocol in usb_descriptors_openpcd.h, setting the new value to 0.
Best Regards
On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester dean.g.chester@gmail.comwrote:
It was mostly the lost bytes/scrambled data with a variety of handsets that i'd tested with, compared with ubuntu running natively.
Dean
On 27 January 2014 18:37, Min Xu mxu@sanjole.com wrote:
P.S. What problems were you experiencing of it running in a virtualized system, can you elaborate?
Thanks
On Mon, Jan 27, 2014 at 8:36 AM, Min Xu mxu@sanjole.com wrote:
I am contributing it to the project. Once I incorporate Peter Stuge's suggestion, hopefully within the next few weeks I will submit another commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester dean.g.chester@gmail.comwrote:
Thanks Min it works a treat it also fixes the issues running in a virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote:
Min Xu wrote:
I would be happy to send them the USB protocol changes. However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB; change either the bDeviceProtocol field in the device descriptor or the bInterfaceProtocol field in the interface descriptor, and make host software do the appropriate thing based on the descriptors of the connected device.
Of course only new host software will work with the new protocol, but this way new host software still continues to work with the old protocol.
//Peter
Hi Dean
I believe that you are seeing incomplete or partial (maybe even) ATRs. I believe I experienced something similar when I see parity error in the debug messages (from the debug port). The first byte of the ATR must be 3F or 3B. I believe the firmware code I currently have should NOT break and ATR into multiple segments. Besides, ATRs are transmitted in a single frame, so there would not be an idle time to push out the partial ATR into the USB transmit buffer.
So I believe you are having similar problem that I experienced when the ribbon cable is twisted, or bent close together like a U (which might have caused interference... )
Try get a serial cable plugged into the simtrace board, and capture the debug output to see if parity error were seen.
Best regards
On Sun, Feb 2, 2014 at 4:40 AM, Dean Chester dean.g.chester@gmail.comwrote:
Hi Min,
I've just been using the new firmware with a different type of sim card and noticed that the ATR is scrambled, at least for the header of the ATR. The ATR is the same length as the one I had great success with. The cards are running at the same speed according to the ATRs however the new card seems to send it's ATR over 3 commands not just one. Whats also interesting is the ATR scrambling is different. Here is an example with the Simtrace Header attached:
01 01 01 01 01 00 00 00 1a 00 e7 9a 03 1f c7 80 31 00 00 00 00 00 00 00 00 00 01 00 01 01 02 00 00 00 0f 00 00 00 00 00 00 01 00 01 01 03 00 00 00 12 00 00 00 00 00 00 00 00 00
And the ATR should look like:
3B 9F 96 80 1F C7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And with the same card from the same company I got:
01 01 01 01 01 00 00 00 16 00 9f 96 80 1f c7 80 31 00 00 00 00 00 01 00 01 01 02 00 00 00 13 00 67 42 70 02 10 00 00 01 f9 01 00 01 01 03 00 00 00 12 00 ff 10 96 79 ff 10 96 79
With the card I had success with I got the following input in:
01 01 01 01 01 00 00 00 20 00 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And the ATR looks like: 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Any ideas where these errors are coming from in the firmware. I notice that the lengths in the failed are equal to 3B which is the start of the ATR if thats of any help. The tests were done with the same phone connected to the tracer just swapping the sim cards.
Kind Regards,
Dean Chester
On 30 January 2014 21:47, Min Xu mxu@sanjole.com wrote:
That does make sense to me since I had issues where before I changed the usb header, the host was sometimes merging multiple req_ctx into a single usb read.
I am attaching a tar.gz file containing all my changes as separate git commits / format-patches and hopefully it will get committed to the repository. This includes the latest change to the usb header as well as a change (as Peter suggested) to the bDeviceProtocol in usb_descriptors_openpcd.h, setting the new value to 0.
Best Regards
On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester dean.g.chester@gmail.comwrote:
It was mostly the lost bytes/scrambled data with a variety of handsets that i'd tested with, compared with ubuntu running natively.
Dean
On 27 January 2014 18:37, Min Xu mxu@sanjole.com wrote:
P.S. What problems were you experiencing of it running in a virtualized system, can you elaborate?
Thanks
On Mon, Jan 27, 2014 at 8:36 AM, Min Xu mxu@sanjole.com wrote:
I am contributing it to the project. Once I incorporate Peter Stuge's suggestion, hopefully within the next few weeks I will submit another commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester < dean.g.chester@gmail.com> wrote:
Thanks Min it works a treat it also fixes the issues running in a virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote:
> Min Xu wrote: > > I would be happy to send them the USB protocol changes. However, > > it will be INCOMPATIBLE with earlier firmware based SIMTrace > boards. > > There is a standardised way to deal with protocol changes in USB; > change either the bDeviceProtocol field in the device descriptor or > the bInterfaceProtocol field in the interface descriptor, and make > host software do the appropriate thing based on the descriptors of > the connected device. > > Of course only new host software will work with the new protocol, but > this way new host software still continues to work with the old > protocol. > > > //Peter > >
Hi Dean
As far as I know, I made no changes to modify the usart/iso7816 communication parameters. I have seen parity errors like this, and got improved result when I can make sure the flat ribbon cable is not folded over on itself. The parity error is reported by the usart peripheral, which is supposed to provide 1 character at a time to the main software code. I will check the changes again.
On Mon, Mar 10, 2014 at 5:19 AM, Dean Chester dean.g.chester@gmail.comwrote:
Hi Min,
Finally my debug cable arrived, I have some debug output for the card that fails to successfully read the ATR:
[000033] Heart beat 00000007 [000034] Heart beat 00000008 [000035] VCC_PHONE on [000036] RST [000037] computed Fi(1) Di(1) ratio: 372 [000038] UART parity error: 2 [000039] USBT(D=002011A8, S=0002, L=0026, P=00) H4/T4: E7 9A 03 1F / 67 42 70 02 [00003A] UART parity error: 3 [00003B] Heart beat 00000009 [00003C] UART parity error: 4 [00003D] UART parity error: 5 [00003E] UART parity error: 6 [00003F] UART parity error: 7 [000040] UART parity error: 8 [000041] UART parity error: 9 [000042] UART parity error: 10 [000043] UART parity error: 11 [000044] UART parity error: 12 [000045] UART parity error: 13 [000046] UART parity error: 14 [000047] UART parity error: 15 [000048] UART parity error: 16 [000049] UART parity error: 17 [00004A] UART parity error: 18 [00004B] UART parity error: 19 [00004C] UART parity error: 20 [00004D] UART parity error: 21 [00004E] UART parity error: 22 [00004F] UART parity error: 23 [000050] UART parity error: 24 [000051] Heart beat 0000000A [000052] UART parity error: 25 [000053] UART parity error: 26 [000054] UART parity error: 27 [000055] UART parity error: 28 [000056] UART parity error: 29 [000057] UART parity error: 30 [000058] UART parity error: 31 [000059] UART parity error: 32 [00005A] UART parity error: 33 [00005B] UART parity error: 34 [00005C] UART parity error: 35 [00005D] UART parity error: 36 [00005E] UART parity error: 37 [00005F] UART parity error: 38 [000060] UART parity error: 39 [000061] UART parity error: 40 [000062] UART parity error: 41 [000063] UART parity error: 42 [000064] UART parity error: 43 [000065] UART parity error: 44 [000066] UART parity error: 45 [000067] UART parity error: 46 [000068] UART parity error: 47 [000069] Heart beat 0000000B [00006A] Heart beat 0000000C [00006B] UART parity error: 48 [00006C] UART parity error: 49 [00006D] UART parity error: 50 [00006E] UART parity error: 51 [00006F] UART parity error: 52 [000070] UART parity error: 53 [000071] UART parity error: 54 [000072] UART parity error: 55 [000073] UART parity error: 56 [000074] UART parity error: 57 [000075] UART parity error: 58 [000076] UART parity error: 59 [000077] Heart beat 0000000D [000078] UART parity error: 60 [000079] UART parity error: 61 [00007A] UART parity error: 62 [00007B] UART parity error: 63 [00007C] UART parity error: 64 [00007D] UART parity error: 65 [00007E] UART parity error: 66 [00007F] UART parity error: 67 [000080] Heart beat 0000000E
So any ideas where this error is being created with your firmware?
Kind Regards,
Dean Chester
On 22 February 2014 15:28, Dean Chester dean.g.chester@gmail.com wrote:
Hi Min,
Sorry for the slow response had a lot of work on. I do not have a debug cable. However it I have just tried this problematic card with V0.5 firmware off the wiki and it obtains the ATR correctly. The error isn't happening because of a issue with the cable being bent, I know this as I have tried it in all orientations to see if this was the case.
I've also not been able to apply your patches correctly to work out where the error might be occurring in your modifications.
Kind Regards,
Dean Chester
On 4 February 2014 19:10, Min Xu mxu@sanjole.com wrote:
Hi Dean
I believe that you are seeing incomplete or partial (maybe even) ATRs. I believe I experienced something similar when I see parity error in the debug messages (from the debug port). The first byte of the ATR must be 3F or 3B. I believe the firmware code I currently have should NOT break and ATR into multiple segments. Besides, ATRs are transmitted in a single frame, so there would not be an idle time to push out the partial ATR into the USB transmit buffer.
So I believe you are having similar problem that I experienced when the ribbon cable is twisted, or bent close together like a U (which might have caused interference... )
Try get a serial cable plugged into the simtrace board, and capture the debug output to see if parity error were seen.
Best regards
On Sun, Feb 2, 2014 at 4:40 AM, Dean Chester dean.g.chester@gmail.comwrote:
Hi Min,
I've just been using the new firmware with a different type of sim card and noticed that the ATR is scrambled, at least for the header of the ATR. The ATR is the same length as the one I had great success with. The cards are running at the same speed according to the ATRs however the new card seems to send it's ATR over 3 commands not just one. Whats also interesting is the ATR scrambling is different. Here is an example with the Simtrace Header attached:
01 01 01 01 01 00 00 00 1a 00 e7 9a 03 1f c7 80 31 00 00 00 00 00 00 00 00 00 01 00 01 01 02 00 00 00 0f 00 00 00 00 00 00 01 00 01 01 03 00 00 00 12 00 00 00 00 00 00 00 00 00
And the ATR should look like:
3B 9F 96 80 1F C7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And with the same card from the same company I got:
01 01 01 01 01 00 00 00 16 00 9f 96 80 1f c7 80 31 00 00 00 00 00 01 00 01 01 02 00 00 00 13 00 67 42 70 02 10 00 00 01 f9 01 00 01 01 03 00 00 00 12 00 ff 10 96 79 ff 10 96 79
With the card I had success with I got the following input in:
01 01 01 01 01 00 00 00 20 00 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And the ATR looks like: 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Any ideas where these errors are coming from in the firmware. I notice that the lengths in the failed are equal to 3B which is the start of the ATR if thats of any help. The tests were done with the same phone connected to the tracer just swapping the sim cards.
Kind Regards,
Dean Chester
On 30 January 2014 21:47, Min Xu mxu@sanjole.com wrote:
That does make sense to me since I had issues where before I changed the usb header, the host was sometimes merging multiple req_ctx into a single usb read.
I am attaching a tar.gz file containing all my changes as separate git commits / format-patches and hopefully it will get committed to the repository. This includes the latest change to the usb header as well as a change (as Peter suggested) to the bDeviceProtocol in usb_descriptors_openpcd.h, setting the new value to 0.
Best Regards
On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester < dean.g.chester@gmail.com> wrote:
It was mostly the lost bytes/scrambled data with a variety of handsets that i'd tested with, compared with ubuntu running natively.
Dean
On 27 January 2014 18:37, Min Xu mxu@sanjole.com wrote:
> P.S. What problems were you experiencing of it running in a > virtualized system, can you elaborate? > > Thanks > > > On Mon, Jan 27, 2014 at 8:36 AM, Min Xu mxu@sanjole.com wrote: > >> I am contributing it to the project. Once I incorporate Peter >> Stuge's suggestion, hopefully within the next few weeks I will submit >> another commit. >> >> Best Regards >> >> >> On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester < >> dean.g.chester@gmail.com> wrote: >> >>> Thanks Min it works a treat it also fixes the issues running in a >>> virtualised environment which I do for Ubuntu. >>> >>> Is your new firmware under the same licence as the original? >>> >>> Kind Regards, >>> >>> Dean Chester >>> >>> >>> On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote: >>> >>>> Min Xu wrote: >>>> > I would be happy to send them the USB protocol changes. >>>> However, >>>> > it will be INCOMPATIBLE with earlier firmware based SIMTrace >>>> boards. >>>> >>>> There is a standardised way to deal with protocol changes in USB; >>>> change either the bDeviceProtocol field in the device descriptor >>>> or >>>> the bInterfaceProtocol field in the interface descriptor, and make >>>> host software do the appropriate thing based on the descriptors of >>>> the connected device. >>>> >>>> Of course only new host software will work with the new protocol, >>>> but >>>> this way new host software still continues to work with the old >>>> protocol. >>>> >>>> >>>> //Peter >>>> >>>> >>> >> >
Hi,
I was looking for a fix to the problem, I am not technical enough to understand all what you say. But is there a howto or anything i can use, so I can implement your solution whith my limited technical knowledge?
Thanks in advance!
Hi Dean
I recently worked on the firmware again and believe I have fixed the issues with the parity errors (at least for my SGS 2 and SGS 4 with ATT 4G/LTE cards)
Can you try the attached firmware and let me know your results?
Thanks
On Mon, Mar 10, 2014 at 8:23 AM, Min Xu mxu@sanjole.com wrote:
Hi Dean
As far as I know, I made no changes to modify the usart/iso7816 communication parameters. I have seen parity errors like this, and got improved result when I can make sure the flat ribbon cable is not folded over on itself. The parity error is reported by the usart peripheral, which is supposed to provide 1 character at a time to the main software code. I will check the changes again.
On Mon, Mar 10, 2014 at 5:19 AM, Dean Chester dean.g.chester@gmail.com wrote:
Hi Min,
Finally my debug cable arrived, I have some debug output for the card that fails to successfully read the ATR:
[000033] Heart beat 00000007 [000034] Heart beat 00000008 [000035] VCC_PHONE on [000036] RST [000037] computed Fi(1) Di(1) ratio: 372 [000038] UART parity error: 2 [000039] USBT(D=002011A8, S=0002, L=0026, P=00) H4/T4: E7 9A 03 1F / 67 42 70 02 [00003A] UART parity error: 3 [00003B] Heart beat 00000009 [00003C] UART parity error: 4 [00003D] UART parity error: 5 [00003E] UART parity error: 6 [00003F] UART parity error: 7 [000040] UART parity error: 8 [000041] UART parity error: 9 [000042] UART parity error: 10 [000043] UART parity error: 11 [000044] UART parity error: 12 [000045] UART parity error: 13 [000046] UART parity error: 14 [000047] UART parity error: 15 [000048] UART parity error: 16 [000049] UART parity error: 17 [00004A] UART parity error: 18 [00004B] UART parity error: 19 [00004C] UART parity error: 20 [00004D] UART parity error: 21 [00004E] UART parity error: 22 [00004F] UART parity error: 23 [000050] UART parity error: 24 [000051] Heart beat 0000000A [000052] UART parity error: 25 [000053] UART parity error: 26 [000054] UART parity error: 27 [000055] UART parity error: 28 [000056] UART parity error: 29 [000057] UART parity error: 30 [000058] UART parity error: 31 [000059] UART parity error: 32 [00005A] UART parity error: 33 [00005B] UART parity error: 34 [00005C] UART parity error: 35 [00005D] UART parity error: 36 [00005E] UART parity error: 37 [00005F] UART parity error: 38 [000060] UART parity error: 39 [000061] UART parity error: 40 [000062] UART parity error: 41 [000063] UART parity error: 42 [000064] UART parity error: 43 [000065] UART parity error: 44 [000066] UART parity error: 45 [000067] UART parity error: 46 [000068] UART parity error: 47 [000069] Heart beat 0000000B [00006A] Heart beat 0000000C [00006B] UART parity error: 48 [00006C] UART parity error: 49 [00006D] UART parity error: 50 [00006E] UART parity error: 51 [00006F] UART parity error: 52 [000070] UART parity error: 53 [000071] UART parity error: 54 [000072] UART parity error: 55 [000073] UART parity error: 56 [000074] UART parity error: 57 [000075] UART parity error: 58 [000076] UART parity error: 59 [000077] Heart beat 0000000D [000078] UART parity error: 60 [000079] UART parity error: 61 [00007A] UART parity error: 62 [00007B] UART parity error: 63 [00007C] UART parity error: 64 [00007D] UART parity error: 65 [00007E] UART parity error: 66 [00007F] UART parity error: 67 [000080] Heart beat 0000000E
So any ideas where this error is being created with your firmware?
Kind Regards,
Dean Chester
On 22 February 2014 15:28, Dean Chester dean.g.chester@gmail.com wrote:
Hi Min,
Sorry for the slow response had a lot of work on. I do not have a debug cable. However it I have just tried this problematic card with V0.5 firmware off the wiki and it obtains the ATR correctly. The error isn't happening because of a issue with the cable being bent, I know this as I have tried it in all orientations to see if this was the case.
I've also not been able to apply your patches correctly to work out where the error might be occurring in your modifications.
Kind Regards,
Dean Chester
On 4 February 2014 19:10, Min Xu mxu@sanjole.com wrote:
Hi Dean
I believe that you are seeing incomplete or partial (maybe even) ATRs. I believe I experienced something similar when I see parity error in the debug messages (from the debug port). The first byte of the ATR must be 3F or 3B. I believe the firmware code I currently have should NOT break and ATR into multiple segments. Besides, ATRs are transmitted in a single frame, so there would not be an idle time to push out the partial ATR into the USB transmit buffer.
So I believe you are having similar problem that I experienced when the ribbon cable is twisted, or bent close together like a U (which might have caused interference... )
Try get a serial cable plugged into the simtrace board, and capture the debug output to see if parity error were seen.
Best regards
On Sun, Feb 2, 2014 at 4:40 AM, Dean Chester dean.g.chester@gmail.com wrote:
Hi Min,
I've just been using the new firmware with a different type of sim card and noticed that the ATR is scrambled, at least for the header of the ATR. The ATR is the same length as the one I had great success with. The cards are running at the same speed according to the ATRs however the new card seems to send it's ATR over 3 commands not just one. Whats also interesting is the ATR scrambling is different. Here is an example with the Simtrace Header attached:
01 01 01 01 01 00 00 00 1a 00 e7 9a 03 1f c7 80 31 00 00 00 00 00 00 00 00 00 01 00 01 01 02 00 00 00 0f 00 00 00 00 00 00 01 00 01 01 03 00 00 00 12 00 00 00 00 00 00 00 00 00
And the ATR should look like:
3B 9F 96 80 1F C7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And with the same card from the same company I got:
01 01 01 01 01 00 00 00 16 00 9f 96 80 1f c7 80 31 00 00 00 00 00 01 00 01 01 02 00 00 00 13 00 67 42 70 02 10 00 00 01 f9 01 00 01 01 03 00 00 00 12 00 ff 10 96 79 ff 10 96 79
With the card I had success with I got the following input in:
01 01 01 01 01 00 00 00 20 00 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And the ATR looks like: 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Any ideas where these errors are coming from in the firmware. I notice that the lengths in the failed are equal to 3B which is the start of the ATR if thats of any help. The tests were done with the same phone connected to the tracer just swapping the sim cards.
Kind Regards,
Dean Chester
On 30 January 2014 21:47, Min Xu mxu@sanjole.com wrote:
That does make sense to me since I had issues where before I changed the usb header, the host was sometimes merging multiple req_ctx into a single usb read.
I am attaching a tar.gz file containing all my changes as separate git commits / format-patches and hopefully it will get committed to the repository. This includes the latest change to the usb header as well as a change (as Peter suggested) to the bDeviceProtocol in usb_descriptors_openpcd.h, setting the new value to 0.
Best Regards
On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester < dean.g.chester@gmail.com> wrote:
> It was mostly the lost bytes/scrambled data with a variety of > handsets that i'd tested with, compared with ubuntu running natively. > > Dean > > > On 27 January 2014 18:37, Min Xu mxu@sanjole.com wrote: > >> P.S. What problems were you experiencing of it running in a >> virtualized system, can you elaborate? >> >> Thanks >> >> >> On Mon, Jan 27, 2014 at 8:36 AM, Min Xu mxu@sanjole.com wrote: >> >>> I am contributing it to the project. Once I incorporate Peter >>> Stuge's suggestion, hopefully within the next few weeks I will submit >>> another commit. >>> >>> Best Regards >>> >>> >>> On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester < >>> dean.g.chester@gmail.com> wrote: >>> >>>> Thanks Min it works a treat it also fixes the issues running in a >>>> virtualised environment which I do for Ubuntu. >>>> >>>> Is your new firmware under the same licence as the original? >>>> >>>> Kind Regards, >>>> >>>> Dean Chester >>>> >>>> >>>> On 24 January 2014 21:55, Peter Stuge peter@stuge.se wrote: >>>> >>>>> Min Xu wrote: >>>>> > I would be happy to send them the USB protocol changes. >>>>> However, >>>>> > it will be INCOMPATIBLE with earlier firmware based SIMTrace >>>>> boards. >>>>> >>>>> There is a standardised way to deal with protocol changes in USB; >>>>> change either the bDeviceProtocol field in the device descriptor >>>>> or >>>>> the bInterfaceProtocol field in the interface descriptor, and >>>>> make >>>>> host software do the appropriate thing based on the descriptors >>>>> of >>>>> the connected device. >>>>> >>>>> Of course only new host software will work with the new >>>>> protocol, but >>>>> this way new host software still continues to work with the old >>>>> protocol. >>>>> >>>>> >>>>> //Peter >>>>> >>>>> >>>> >>> >> >
-
Dean Chester -
Freddie -
Holger Hans Peter Freyther -
Min Xu -
Peter Stuge