13 Jan
2014
13 Jan
'14
9:35 a.m.
Hi,
I see earlier in the archives that reducing the buffer size in the firmware will resolve
this. Hasn't this already been done though in new releases of the firmware? As I have
pulled the latest version from the openpcd git repo and built it. If not what have people
reduced the buffer size to? And has it had any fixes?
Sent from my iPad
22 Jan
22 Jan
9:27 p.m.
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the firmware will resolve
this. Hasn't this already been done though in new releases of the firmware? As I have
pulled the latest version from the openpcd git repo and built it. If not what have people
reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some
clean-ups and review (specially the code changing the interrupt
handling).
holger
24 Jan
24 Jan
8:41 p.m.
Hi Holger
To give you an update. As far as I can tell, the changes I have submitted,
with the addition of a change (on my local system) where the USB protocol
has been modified to remove USB overrun, is stable so I can monitor iPhone
5S as well as Galaxy S4 continuously. If anyone asks, I would be happy to
send them the USB protocol changes. However, it will be INCOMPATIBLE with
earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple
SIMTrace devices, and am interested to know if anyone else is working on
something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther
<holger(a)freyther.de>
To: Dean Chester <dean.g.chester(a)gmail.com>
Cc: "simtrace(a)lists.osmocom.org" <simtrace(a)lists.osmocom.org>
Subject: Re: Fast SIM cards loosing bytes
Message-ID: <20140122202731.GL15138(a)xiaoyu.lan>
Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the
firmware will
resolve this. Hasn't this already been done though in new
releases of the firmware? As I have pulled the latest version from the
openpcd git repo and built it. If not what have people reduced the buffer
size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some
clean-ups and review (specially the code changing the interrupt
handling).
holger
------------------------------
_______________________________________________
simtrace mailing list
simtrace(a)lists.osmocom.org
https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
***************************************
10:06 p.m.
Hi min,
How do you mean that they would be incompatible with early firmware based tracers?
I'd be interested in using your firmware if you were to make it available, I'm
also working on a more user friendly host software.
Kind regards
Dean Chester.
Sent from my iPad
On 24 Jan 2014, at 19:41, Min Xu
<mxu(a)sanjole.com> wrote:
Hi Holger
To give you an update. As far as I can tell, the changes I have submitted, with the
addition of a change (on my local system) where the USB protocol has been modified to
remove USB overrun, is stable so I can monitor iPhone 5S as well as Galaxy S4
continuously. If anyone asks, I would be happy to send them the USB protocol changes.
However, it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple SIMTrace
devices, and am interested to know if anyone else is working on something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther <holger(a)freyther.de>
To: Dean Chester <dean.g.chester(a)gmail.com>
Cc: "simtrace(a)lists.osmocom.org" <simtrace(a)lists.osmocom.org>
Subject: Re: Fast SIM cards loosing bytes
Message-ID: <20140122202731.GL15138(a)xiaoyu.lan>
Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the firmware will resolve
this. Hasn't this already been done though in new releases of the firmware? As I have
pulled the latest version from the openpcd git repo and built it. If not what have people
reduced the buffer size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some
clean-ups and review (specially the code changing the interrupt
handling).
holger
------------------------------
_______________________________________________
simtrace mailing list
simtrace(a)lists.osmocom.org
https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
*************************************** 
10:55 p.m.
Min Xu wrote:
I would be happy to send them the USB protocol
changes. However,
it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB;
change either the bDeviceProtocol field in the device descriptor or
the bInterfaceProtocol field in the interface descriptor, and make
host software do the appropriate thing based on the descriptors of
the connected device.
Of course only new host software will work with the new protocol, but
this way new host software still continues to work with the old protocol.
//Peter
27 Jan
27 Jan
12:05 p.m.
Thanks Min it works a treat it also fixes the issues running in a
virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se> wrote:
Min Xu wrote:
I would be happy to send them the USB protocol
changes. However,
it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB;
change either the bDeviceProtocol field in the device descriptor or
the bInterfaceProtocol field in the interface descriptor, and make
host software do the appropriate thing based on the descriptors of
the connected device.
Of course only new host software will work with the new protocol, but
this way new host software still continues to work with the old protocol.
//Peter
24 Jan
24 Jan
10:24 p.m.
It's incompatible because due to the way the USB peripheral transfer the
data out to the host, there is NO mechanism to indicate exactly where the
next USB transaction should start. Therefore, when the data is flowing
very fast, every 64 byte (the bus width) write to the USB peripheral will
be concatenated together, resulting in the host read to combine multiple
req_ctx buffers.
The way I solved it is by adding fields to the simtrace_hdr structure:
(the + indicates the new fields). This allows me on the host to keep track
if / where a new req_ctx begins in the received usb data.
struct simtrace_hdr {
u_int8_t cmd;
u_int8_t flags;
u_int8_t res[2];
+ u_int16_t seq_num;
+ u_int16_t offset;
+ u_int16_t tot_len;
u_int8_t data[0];
} __attribute__ ((packed));
Since the laptop that I was using to create multiple commits is not
available at the moment, I am attaching the git diff from the
repository (instead of from my last commits in previous email). Meaning
this is a single diff file.
For the host software, the basic change in the process_usb_msg function.
My implementation is a naive implementation using a much larger buffer to
copy data to, where if a req_ctx is split across multiple usb transfers, it
will join those fragments then process:
struct simtrace_hdr {
u_int8_t cmd;
u_int8_t flags;
u_int8_t res[2];
u_int16_t seq_num;
u_int16_t offset;
u_int16_t tot_len;
u_int8_t data[0];
} /* __attribute__ ((packed)) */;
static char usb_buffer[66000];
static unsigned usb_remainder = 0;
int process_usb_msg(uint8_t *buf, int len) {
struct simtrace_hdr *sh;
uint8_t *payload;
int payload_len, sh_offset, i;
sendto(s, buf, len, 0, (struct sockaddr*)&destUSB, sizeof(struct
sockaddr_in));
memcpy(usb_buffer + usb_remainder, buf, len);
len += usb_remainder;
usb_remainder = len;
sh_offset = 0;
sh = usb_buffer;
payload = (char*)sh + sizeof(struct simtrace_hdr);
payload_len = sh->tot_len - sizeof(struct simtrace_hdr);
if (usb_remainder < sh->tot_len)
sh = NULL;
while (sh) {
switch (sh->cmd) {
case SIMTRACE_MSGT_DATA:
/* special treatment for ATR */
if (sh->flags & SIMTRACE_FLAG_ATR) {
apdu_split_reset(as);
break;
}
if (sh->flags & SIMTRACE_FLAG_PPS_FIDI) {
module_out("PPS(Fi=%u/Di=%u)\n",
sh->res[0], sh->res[1]);
}
/* everything else goes into APDU splitter */
apdu_split_in(as, payload, payload_len);
#if 0
/* If waiting time has expired, signal explicit boundary */
if (sh->flags & SIMTRACE_FLAG_WTIME_EXP)
apdu_split_boundary(as);
#endif
break;
case SIMTRACE_MSGT_RESET:
default:
module_out("unknown simtrace msg type 0x%02x\n", sh->cmd);
break;
}
sh_offset += sh->tot_len;
sh = usb_buffer + sh_offset;
if (sh_offset < len) {
if (sh_offset + sh->tot_len > len) {
memcpy(usb_buffer, sh, len - sh_offset);
usb_remainder = len - sh_offset;
sh = NULL;
} else {
payload = (char*)sh + sizeof(struct simtrace_hdr);
payload_len = sh->tot_len - sizeof(struct simtrace_hdr);
}
} else {
sh = NULL;
usb_remainder = 0;
}
}
return 0;
}
On Fri, Jan 24, 2014 at 11:06 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Hi min,
How do you mean that they would be incompatible with early firmware based
tracers?
I'd be interested in using your firmware if you were to make it available,
I'm also working on a more user friendly host software.
Kind regards
Dean Chester.
Sent from my iPad
On 24 Jan 2014, at 19:41, Min Xu <mxu(a)sanjole.com> wrote:
Hi Holger
To give you an update. As far as I can tell, the changes I have
submitted, with the addition of a change (on my local system) where the USB
protocol has been modified to remove USB overrun, is stable so I can
monitor iPhone 5S as well as Galaxy S4 continuously. If anyone asks, I
would be happy to send them the USB protocol changes. However, it will be
INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple
SIMTrace devices, and am interested to know if anyone else is working on
something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther
<holger(a)freyther.de>
To: Dean Chester <dean.g.chester(a)gmail.com>
Cc: "simtrace(a)lists.osmocom.org" <simtrace(a)lists.osmocom.org>
Subject: Re: Fast SIM cards loosing bytes
Message-ID: <20140122202731.GL15138(a)xiaoyu.lan>
Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the
firmware will
resolve this. Hasn't this already been done though in new
releases of the firmware? As I have pulled the latest version from the
openpcd git repo and built it. If not what have people reduced the buffer
size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some
clean-ups and review (specially the code changing the interrupt
handling).
holger
------------------------------
_______________________________________________
simtrace mailing list
simtrace(a)lists.osmocom.org
https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
***************************************
10:37 p.m.
Dean
Just noticed that you wanted the firmware. So I am including the
pre-compiled image in case you didn't want to recompile yourself.
On Fri, Jan 24, 2014 at 11:24 AM, Min Xu <mxu(a)sanjole.com> wrote:
It's incompatible because due to the way the USB
peripheral transfer the
data out to the host, there is NO mechanism to indicate exactly where the
next USB transaction should start. Therefore, when the data is flowing
very fast, every 64 byte (the bus width) write to the USB peripheral will
be concatenated together, resulting in the host read to combine multiple
req_ctx buffers.
The way I solved it is by adding fields to the simtrace_hdr structure:
(the + indicates the new fields). This allows me on the host to keep track
if / where a new req_ctx begins in the received usb data.
struct simtrace_hdr {
u_int8_t cmd;
u_int8_t flags;
u_int8_t res[2];
+ u_int16_t seq_num;
+ u_int16_t offset;
+ u_int16_t tot_len;
u_int8_t data[0];
} __attribute__ ((packed));
Since the laptop that I was using to create multiple commits is not
available at the moment, I am attaching the git diff from the
repository (instead of from my last commits in previous email). Meaning
this is a single diff file.
For the host software, the basic change in the process_usb_msg function.
My implementation is a naive implementation using a much larger buffer to
copy data to, where if a req_ctx is split across multiple usb transfers, it
will join those fragments then process:
struct simtrace_hdr {
u_int8_t cmd;
u_int8_t flags;
u_int8_t res[2];
u_int16_t seq_num;
u_int16_t offset;
u_int16_t tot_len;
u_int8_t data[0];
} /* __attribute__ ((packed)) */;
static char usb_buffer[66000];
static unsigned usb_remainder = 0;
int process_usb_msg(uint8_t *buf, int len) {
struct simtrace_hdr *sh;
uint8_t *payload;
int payload_len, sh_offset, i;
sendto(s, buf, len, 0, (struct sockaddr*)&destUSB, sizeof(struct
sockaddr_in));
memcpy(usb_buffer + usb_remainder, buf, len);
len += usb_remainder;
usb_remainder = len;
sh_offset = 0;
sh = usb_buffer;
payload = (char*)sh + sizeof(struct simtrace_hdr);
payload_len = sh->tot_len - sizeof(struct simtrace_hdr);
if (usb_remainder < sh->tot_len)
sh = NULL;
while (sh) {
switch (sh->cmd) {
case SIMTRACE_MSGT_DATA:
/* special treatment for ATR */
if (sh->flags & SIMTRACE_FLAG_ATR) {
apdu_split_reset(as);
break;
}
if (sh->flags & SIMTRACE_FLAG_PPS_FIDI) {
module_out("PPS(Fi=%u/Di=%u)\n",
sh->res[0], sh->res[1]);
}
/* everything else goes into APDU splitter */
apdu_split_in(as, payload, payload_len);
#if 0
/* If waiting time has expired, signal explicit boundary */
if (sh->flags & SIMTRACE_FLAG_WTIME_EXP)
apdu_split_boundary(as);
#endif
break;
case SIMTRACE_MSGT_RESET:
default:
module_out("unknown simtrace msg type 0x%02x\n", sh->cmd);
break;
}
sh_offset += sh->tot_len;
sh = usb_buffer + sh_offset;
if (sh_offset < len) {
if (sh_offset + sh->tot_len > len) {
memcpy(usb_buffer, sh, len - sh_offset);
usb_remainder = len - sh_offset;
sh = NULL;
} else {
payload = (char*)sh + sizeof(struct simtrace_hdr);
payload_len = sh->tot_len - sizeof(struct simtrace_hdr);
}
} else {
sh = NULL;
usb_remainder = 0;
}
}
return 0;
}
On Fri, Jan 24, 2014 at 11:06 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Hi min,
How do you mean that they would be incompatible with early firmware based
tracers?
I'd be interested in using your firmware if you were to make it
available, I'm also working on a more user friendly host software.
Kind regards
Dean Chester.
Sent from my iPad
On 24 Jan 2014, at 19:41, Min Xu <mxu(a)sanjole.com> wrote:
Hi Holger
To give you an update. As far as I can tell, the changes I have
submitted, with the addition of a change (on my local system) where the USB
protocol has been modified to remove USB overrun, is stable so I can
monitor iPhone 5S as well as Galaxy S4 continuously. If anyone asks, I
would be happy to send them the USB protocol changes. However, it will be
INCOMPATIBLE with earlier firmware based SIMTrace boards.
I am making changes to the host program (on Windows) to monitor multiple
SIMTrace devices, and am interested to know if anyone else is working on
something similar.
Best Regards
Date: Wed, 22 Jan 2014 21:27:31 +0100
From: Holger Hans Peter Freyther
<holger(a)freyther.de>
To: Dean Chester <dean.g.chester(a)gmail.com>
Cc: "simtrace(a)lists.osmocom.org" <simtrace(a)lists.osmocom.org>
Subject: Re: Fast SIM cards loosing bytes
Message-ID: <20140122202731.GL15138(a)xiaoyu.lan>
Content-Type: text/plain; charset=us-ascii
On Mon, Jan 13, 2014 at 08:35:05AM +0000, Dean Chester wrote:
Hi,
I see earlier in the archives that reducing the buffer size in the
firmware will
resolve this. Hasn't this already been done though in new
releases of the firmware? As I have pulled the latest version from the
openpcd git repo and built it. If not what have people reduced the buffer
size to? And has it had any fixes?
The closest are the patches that Min Xu sent. They require some
clean-ups and review (specially the code changing the interrupt
handling).
holger
------------------------------
_______________________________________________
simtrace mailing list
simtrace(a)lists.osmocom.org
https://lists.osmocom.org/mailman/listinfo/simtrace
End of simtrace Digest, Vol 33, Issue 4
***************************************
27 Jan
27 Jan
7:36 p.m.
I am contributing it to the project. Once I incorporate Peter Stuge's
suggestion, hopefully within the next few weeks I will submit another
commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Thanks Min it works a treat it also fixes the issues
running in a
virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se> wrote:
Min Xu wrote:
I would be happy to send them the USB protocol
changes. However,
it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB;
change either the bDeviceProtocol field in the device descriptor or
the bInterfaceProtocol field in the interface descriptor, and make
host software do the appropriate thing based on the descriptors of
the connected device.
Of course only new host software will work with the new protocol, but
this way new host software still continues to work with the old protocol.
//Peter
7:37 p.m.
P.S. What problems were you experiencing of it running in a virtualized
system, can you elaborate?
Thanks
On Mon, Jan 27, 2014 at 8:36 AM, Min Xu <mxu(a)sanjole.com> wrote:
I am contributing it to the project. Once I
incorporate Peter Stuge's
suggestion, hopefully within the next few weeks I will submit another
commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Thanks Min it works a treat it also fixes the
issues running in a
virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se> wrote:
Min Xu wrote:
I would be happy to send them the USB protocol
changes. However,
it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
There is a standardised way to deal with protocol changes in USB;
change either the bDeviceProtocol field in the device descriptor or
the bInterfaceProtocol field in the interface descriptor, and make
host software do the appropriate thing based on the descriptors of
the connected device.
Of course only new host software will work with the new protocol, but
this way new host software still continues to work with the old protocol.
//Peter
30 Jan
30 Jan
10:47 p.m.
That does make sense to me since I had issues where before I changed the
usb header, the host was sometimes merging multiple req_ctx into a single
usb read.
I am attaching a tar.gz file containing all my changes as separate git
commits / format-patches and hopefully it will get committed to the
repository. This includes the latest change to the usb header as well as a
change (as Peter suggested) to the bDeviceProtocol in
usb_descriptors_openpcd.h, setting the new value to 0.
Best Regards
On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
It was mostly the lost bytes/scrambled data with a
variety of handsets
that i'd tested with, compared with ubuntu running natively.
Dean
On 27 January 2014 18:37, Min Xu <mxu(a)sanjole.com> wrote:
P.S. What problems were you experiencing of it
running in a virtualized
system, can you elaborate?
Thanks
On Mon, Jan 27, 2014 at 8:36 AM, Min Xu <mxu(a)sanjole.com> wrote:
I am contributing it to the project. Once I
incorporate Peter Stuge's
suggestion, hopefully within the next few weeks I will submit another
commit.
Best Regards
On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Thanks Min it works a treat it also fixes the
issues running in a
virtualised environment which I do for Ubuntu.
Is your new firmware under the same licence as the original?
Kind Regards,
Dean Chester
On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se> wrote:
> Min Xu wrote:
> > I would be happy to send them the USB protocol changes. However,
> > it will be INCOMPATIBLE with earlier firmware based SIMTrace boards.
>
> There is a standardised way to deal with protocol changes in USB;
> change either the bDeviceProtocol field in the device descriptor or
> the bInterfaceProtocol field in the interface descriptor, and make
> host software do the appropriate thing based on the descriptors of
> the connected device.
>
> Of course only new host software will work with the new protocol, but
> this way new host software still continues to work with the old
> protocol.
>
>
> //Peter
>
>
4 Feb
4 Feb
8:10 p.m.
Hi Dean
I believe that you are seeing incomplete or partial (maybe even) ATRs. I
believe I experienced something similar when I see parity error in the
debug messages (from the debug port). The first byte of the ATR must be 3F
or 3B. I believe the firmware code I currently have should NOT break and
ATR into multiple segments. Besides, ATRs are transmitted in a single
frame, so there would not be an idle time to push out the partial ATR into
the USB transmit buffer.
So I believe you are having similar problem that I experienced when the
ribbon cable is twisted, or bent close together like a U (which might have
caused interference... )
Try get a serial cable plugged into the simtrace board, and capture the
debug output to see if parity error were seen.
Best regards
On Sun, Feb 2, 2014 at 4:40 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Hi Min,
I've just been using the new firmware with a different type of sim card
and noticed that the ATR is scrambled, at least for the header of the ATR.
The ATR is the same length as the one I had great success with. The cards
are running at the same speed according to the ATRs however the new card
seems to send it's ATR over 3 commands not just one. Whats also interesting
is the ATR scrambling is different. Here is an example with the Simtrace
Header attached:
01 01 01 01 01 00 00 00 1a 00 e7 9a 03 1f c7 80 31 00 00 00 00 00 00 00 00
00
01 00 01 01 02 00 00 00 0f 00 00 00 00 00 00
01 00 01 01 03 00 00 00 12 00 00 00 00 00 00 00 00 00
And the ATR should look like:
3B 9F 96 80 1F C7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And with the same card from the same company I got:
01 01 01 01 01 00 00 00 16 00 9f 96 80 1f c7 80 31 00 00 00 00 00
01 00 01 01 02 00 00 00 13 00 67 42 70 02 10 00 00 01 f9
01 00 01 01 03 00 00 00 12 00 ff 10 96 79 ff 10 96 79
With the card I had success with I got the following input in:
01 01 01 01 01 00 00 00 20 00 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00
00 00 00 00 00 00 00
And the ATR looks like:
3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Any ideas where these errors are coming from in the firmware. I notice
that the lengths in the failed are equal to 3B which is the start of the
ATR if thats of any help. The tests were done with the same phone connected
to the tracer just swapping the sim cards.
Kind Regards,
Dean Chester
On 30 January 2014 21:47, Min Xu <mxu(a)sanjole.com> wrote:
That does make sense to me since I had issues
where before I changed the
usb header, the host was sometimes merging multiple req_ctx into a single
usb read.
I am attaching a tar.gz file containing all my changes as separate git
commits / format-patches and hopefully it will get committed to the
repository. This includes the latest change to the usb header as well as a
change (as Peter suggested) to the bDeviceProtocol in
usb_descriptors_openpcd.h, setting the new value to 0.
Best Regards
On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
It was mostly the lost bytes/scrambled data with
a variety of handsets
that i'd tested with, compared with ubuntu running natively.
Dean
On 27 January 2014 18:37, Min Xu <mxu(a)sanjole.com> wrote:
P.S. What problems were you experiencing of it
running in a
virtualized system, can you elaborate?
Thanks
On Mon, Jan 27, 2014 at 8:36 AM, Min Xu <mxu(a)sanjole.com> wrote:
> I am contributing it to the project. Once I incorporate Peter Stuge's
> suggestion, hopefully within the next few weeks I will submit another
> commit.
>
> Best Regards
>
>
> On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester <
> dean.g.chester(a)gmail.com> wrote:
>
>> Thanks Min it works a treat it also fixes the issues running in a
>> virtualised environment which I do for Ubuntu.
>>
>> Is your new firmware under the same licence as the original?
>>
>> Kind Regards,
>>
>> Dean Chester
>>
>>
>> On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se> wrote:
>>
>>> Min Xu wrote:
>>> > I would be happy to send them the USB protocol changes. However,
>>> > it will be INCOMPATIBLE with earlier firmware based SIMTrace
>>> boards.
>>>
>>> There is a standardised way to deal with protocol changes in USB;
>>> change either the bDeviceProtocol field in the device descriptor or
>>> the bInterfaceProtocol field in the interface descriptor, and make
>>> host software do the appropriate thing based on the descriptors of
>>> the connected device.
>>>
>>> Of course only new host software will work with the new protocol, but
>>> this way new host software still continues to work with the old
>>> protocol.
>>>
>>>
>>> //Peter
>>>
>>>
>>
>
10 Mar
10 Mar
7:23 p.m.
Hi Dean
As far as I know, I made no changes to modify the usart/iso7816
communication parameters. I have seen parity errors like this, and got
improved result when I can make sure the flat ribbon cable is not folded
over on itself. The parity error is reported by the usart peripheral,
which is supposed to provide 1 character at a time to the main software
code. I will check the changes again.
On Mon, Mar 10, 2014 at 5:19 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Hi Min,
Finally my debug cable arrived, I have some debug output for the card that
fails to successfully read the ATR:
[000033] Heart beat 00000007
[000034] Heart beat 00000008
[000035] VCC_PHONE on
[000036] RST
[000037] computed Fi(1) Di(1) ratio: 372
[000038] UART parity error: 2
[000039] USBT(D=002011A8, S=0002, L=0026, P=00) H4/T4: E7 9A 03 1F / 67 42
70 02
[00003A] UART parity error: 3
[00003B] Heart beat 00000009
[00003C] UART parity error: 4
[00003D] UART parity error: 5
[00003E] UART parity error: 6
[00003F] UART parity error: 7
[000040] UART parity error: 8
[000041] UART parity error: 9
[000042] UART parity error: 10
[000043] UART parity error: 11
[000044] UART parity error: 12
[000045] UART parity error: 13
[000046] UART parity error: 14
[000047] UART parity error: 15
[000048] UART parity error: 16
[000049] UART parity error: 17
[00004A] UART parity error: 18
[00004B] UART parity error: 19
[00004C] UART parity error: 20
[00004D] UART parity error: 21
[00004E] UART parity error: 22
[00004F] UART parity error: 23
[000050] UART parity error: 24
[000051] Heart beat 0000000A
[000052] UART parity error: 25
[000053] UART parity error: 26
[000054] UART parity error: 27
[000055] UART parity error: 28
[000056] UART parity error: 29
[000057] UART parity error: 30
[000058] UART parity error: 31
[000059] UART parity error: 32
[00005A] UART parity error: 33
[00005B] UART parity error: 34
[00005C] UART parity error: 35
[00005D] UART parity error: 36
[00005E] UART parity error: 37
[00005F] UART parity error: 38
[000060] UART parity error: 39
[000061] UART parity error: 40
[000062] UART parity error: 41
[000063] UART parity error: 42
[000064] UART parity error: 43
[000065] UART parity error: 44
[000066] UART parity error: 45
[000067] UART parity error: 46
[000068] UART parity error: 47
[000069] Heart beat 0000000B
[00006A] Heart beat 0000000C
[00006B] UART parity error: 48
[00006C] UART parity error: 49
[00006D] UART parity error: 50
[00006E] UART parity error: 51
[00006F] UART parity error: 52
[000070] UART parity error: 53
[000071] UART parity error: 54
[000072] UART parity error: 55
[000073] UART parity error: 56
[000074] UART parity error: 57
[000075] UART parity error: 58
[000076] UART parity error: 59
[000077] Heart beat 0000000D
[000078] UART parity error: 60
[000079] UART parity error: 61
[00007A] UART parity error: 62
[00007B] UART parity error: 63
[00007C] UART parity error: 64
[00007D] UART parity error: 65
[00007E] UART parity error: 66
[00007F] UART parity error: 67
[000080] Heart beat 0000000E
So any ideas where this error is being created with your firmware?
Kind Regards,
Dean Chester
On 22 February 2014 15:28, Dean Chester <dean.g.chester(a)gmail.com> wrote:
Hi Min,
Sorry for the slow response had a lot of work on. I do not have a debug
cable. However it I have just tried this problematic card with V0.5
firmware off the wiki and it obtains the ATR correctly. The error isn't
happening because of a issue with the cable being bent, I know this as I
have tried it in all orientations to see if this was the case.
I've also not been able to apply your patches correctly to work out where
the error might be occurring in your modifications.
Kind Regards,
Dean Chester
On 4 February 2014 19:10, Min Xu <mxu(a)sanjole.com> wrote:
Hi Dean
I believe that you are seeing incomplete or partial (maybe even) ATRs.
I believe I experienced something similar when I see parity error in the
debug messages (from the debug port). The first byte of the ATR must be 3F
or 3B. I believe the firmware code I currently have should NOT break and
ATR into multiple segments. Besides, ATRs are transmitted in a single
frame, so there would not be an idle time to push out the partial ATR into
the USB transmit buffer.
So I believe you are having similar problem that I experienced when the
ribbon cable is twisted, or bent close together like a U (which might have
caused interference... )
Try get a serial cable plugged into the simtrace board, and capture the
debug output to see if parity error were seen.
Best regards
On Sun, Feb 2, 2014 at 4:40 AM, Dean Chester <dean.g.chester(a)gmail.com>wrote;wrote:
Hi Min,
I've just been using the new firmware with a different type of sim card
and noticed that the ATR is scrambled, at least for the header of the ATR.
The ATR is the same length as the one I had great success with. The cards
are running at the same speed according to the ATRs however the new card
seems to send it's ATR over 3 commands not just one. Whats also interesting
is the ATR scrambling is different. Here is an example with the Simtrace
Header attached:
01 01 01 01 01 00 00 00 1a 00 e7 9a 03 1f c7 80 31 00 00 00 00 00 00 00
00 00
01 00 01 01 02 00 00 00 0f 00 00 00 00 00 00
01 00 01 01 03 00 00 00 12 00 00 00 00 00 00 00 00 00
And the ATR should look like:
3B 9F 96 80 1F C7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
And with the same card from the same company I got:
01 01 01 01 01 00 00 00 16 00 9f 96 80 1f c7 80 31 00 00 00 00 00
01 00 01 01 02 00 00 00 13 00 67 42 70 02 10 00 00 01 f9
01 00 01 01 03 00 00 00 12 00 ff 10 96 79 ff 10 96 79
With the card I had success with I got the following input in:
01 01 01 01 01 00 00 00 20 00 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00
00 00 00 00 00 00 00 00
And the ATR looks like:
3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Any ideas where these errors are coming from in the firmware. I notice
that the lengths in the failed are equal to 3B which is the start of the
ATR if thats of any help. The tests were done with the same phone connected
to the tracer just swapping the sim cards.
Kind Regards,
Dean Chester
On 30 January 2014 21:47, Min Xu <mxu(a)sanjole.com> wrote:
> That does make sense to me since I had issues where before I changed
> the usb header, the host was sometimes merging multiple req_ctx into a
> single usb read.
>
> I am attaching a tar.gz file containing all my changes as separate git
> commits / format-patches and hopefully it will get committed to the
> repository. This includes the latest change to the usb header as well as a
> change (as Peter suggested) to the bDeviceProtocol in
> usb_descriptors_openpcd.h, setting the new value to 0.
>
> Best Regards
>
>
> On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester <
> dean.g.chester(a)gmail.com> wrote:
>
>> It was mostly the lost bytes/scrambled data with a variety of
>> handsets that i'd tested with, compared with ubuntu running natively.
>>
>> Dean
>>
>>
>> On 27 January 2014 18:37, Min Xu <mxu(a)sanjole.com> wrote:
>>
>>> P.S. What problems were you experiencing of it running in a
>>> virtualized system, can you elaborate?
>>>
>>> Thanks
>>>
>>>
>>> On Mon, Jan 27, 2014 at 8:36 AM, Min Xu <mxu(a)sanjole.com> wrote:
>>>
>>>> I am contributing it to the project. Once I incorporate Peter
>>>> Stuge's suggestion, hopefully within the next few weeks I will
submit
>>>> another commit.
>>>>
>>>> Best Regards
>>>>
>>>>
>>>> On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester <
>>>> dean.g.chester(a)gmail.com> wrote:
>>>>
>>>>> Thanks Min it works a treat it also fixes the issues running in a
>>>>> virtualised environment which I do for Ubuntu.
>>>>>
>>>>> Is your new firmware under the same licence as the original?
>>>>>
>>>>> Kind Regards,
>>>>>
>>>>> Dean Chester
>>>>>
>>>>>
>>>>> On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se> wrote:
>>>>>
>>>>>> Min Xu wrote:
>>>>>> > I would be happy to send them the USB protocol changes.
>>>>>> However,
>>>>>> > it will be INCOMPATIBLE with earlier firmware based
SIMTrace
>>>>>> boards.
>>>>>>
>>>>>> There is a standardised way to deal with protocol changes in
USB;
>>>>>> change either the bDeviceProtocol field in the device descriptor
>>>>>> or
>>>>>> the bInterfaceProtocol field in the interface descriptor, and
make
>>>>>> host software do the appropriate thing based on the descriptors
of
>>>>>> the connected device.
>>>>>>
>>>>>> Of course only new host software will work with the new
protocol,
>>>>>> but
>>>>>> this way new host software still continues to work with the old
>>>>>> protocol.
>>>>>>
>>>>>>
>>>>>> //Peter
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
12 Jun
12 Jun
9:42 a.m.
Hi,
I was looking for a fix to the problem, I am not technical enough to
understand all what you say. But is there a howto or anything i can use, so
I can implement your solution whith my limited technical knowledge?
Thanks in advance!
24 Jun
24 Jun
11:26 p.m.
Hi Dean
I recently worked on the firmware again and believe I have fixed the issues
with the parity errors (at least for my SGS 2 and SGS 4 with ATT 4G/LTE
cards)
Can you try the attached firmware and let me know your results?
Thanks
On Mon, Mar 10, 2014 at 8:23 AM, Min Xu <mxu(a)sanjole.com> wrote:
Hi Dean
As far as I know, I made no changes to modify the usart/iso7816
communication parameters. I have seen parity errors like this, and got
improved result when I can make sure the flat ribbon cable is not folded
over on itself. The parity error is reported by the usart peripheral,
which is supposed to provide 1 character at a time to the main software
code. I will check the changes again.
On Mon, Mar 10, 2014 at 5:19 AM, Dean Chester <dean.g.chester(a)gmail.com>
wrote:
Hi Min,
Finally my debug cable arrived, I have some debug output for the card
that fails to successfully read the ATR:
[000033] Heart beat 00000007
[000034] Heart beat 00000008
[000035] VCC_PHONE on
[000036] RST
[000037] computed Fi(1) Di(1) ratio: 372
[000038] UART parity error: 2
[000039] USBT(D=002011A8, S=0002, L=0026, P=00) H4/T4: E7 9A 03 1F / 67
42 70 02
[00003A] UART parity error: 3
[00003B] Heart beat 00000009
[00003C] UART parity error: 4
[00003D] UART parity error: 5
[00003E] UART parity error: 6
[00003F] UART parity error: 7
[000040] UART parity error: 8
[000041] UART parity error: 9
[000042] UART parity error: 10
[000043] UART parity error: 11
[000044] UART parity error: 12
[000045] UART parity error: 13
[000046] UART parity error: 14
[000047] UART parity error: 15
[000048] UART parity error: 16
[000049] UART parity error: 17
[00004A] UART parity error: 18
[00004B] UART parity error: 19
[00004C] UART parity error: 20
[00004D] UART parity error: 21
[00004E] UART parity error: 22
[00004F] UART parity error: 23
[000050] UART parity error: 24
[000051] Heart beat 0000000A
[000052] UART parity error: 25
[000053] UART parity error: 26
[000054] UART parity error: 27
[000055] UART parity error: 28
[000056] UART parity error: 29
[000057] UART parity error: 30
[000058] UART parity error: 31
[000059] UART parity error: 32
[00005A] UART parity error: 33
[00005B] UART parity error: 34
[00005C] UART parity error: 35
[00005D] UART parity error: 36
[00005E] UART parity error: 37
[00005F] UART parity error: 38
[000060] UART parity error: 39
[000061] UART parity error: 40
[000062] UART parity error: 41
[000063] UART parity error: 42
[000064] UART parity error: 43
[000065] UART parity error: 44
[000066] UART parity error: 45
[000067] UART parity error: 46
[000068] UART parity error: 47
[000069] Heart beat 0000000B
[00006A] Heart beat 0000000C
[00006B] UART parity error: 48
[00006C] UART parity error: 49
[00006D] UART parity error: 50
[00006E] UART parity error: 51
[00006F] UART parity error: 52
[000070] UART parity error: 53
[000071] UART parity error: 54
[000072] UART parity error: 55
[000073] UART parity error: 56
[000074] UART parity error: 57
[000075] UART parity error: 58
[000076] UART parity error: 59
[000077] Heart beat 0000000D
[000078] UART parity error: 60
[000079] UART parity error: 61
[00007A] UART parity error: 62
[00007B] UART parity error: 63
[00007C] UART parity error: 64
[00007D] UART parity error: 65
[00007E] UART parity error: 66
[00007F] UART parity error: 67
[000080] Heart beat 0000000E
So any ideas where this error is being created with your firmware?
Kind Regards,
Dean Chester
On 22 February 2014 15:28, Dean Chester <dean.g.chester(a)gmail.com> wrote:
Hi Min,
Sorry for the slow response had a lot of work on. I do not have a debug
cable. However it I have just tried this problematic card with V0.5
firmware off the wiki and it obtains the ATR correctly. The error isn't
happening because of a issue with the cable being bent, I know this as I
have tried it in all orientations to see if this was the case.
I've also not been able to apply your patches correctly to work out
where the error might be occurring in your modifications.
Kind Regards,
Dean Chester
On 4 February 2014 19:10, Min Xu <mxu(a)sanjole.com> wrote:
Hi Dean
I believe that you are seeing incomplete or partial (maybe even) ATRs.
I believe I experienced something similar when I see parity error in the
debug messages (from the debug port). The first byte of the ATR must be 3F
or 3B. I believe the firmware code I currently have should NOT break and
ATR into multiple segments. Besides, ATRs are transmitted in a single
frame, so there would not be an idle time to push out the partial ATR into
the USB transmit buffer.
So I believe you are having similar problem that I experienced when the
ribbon cable is twisted, or bent close together like a U (which might have
caused interference... )
Try get a serial cable plugged into the simtrace board, and capture the
debug output to see if parity error were seen.
Best regards
On Sun, Feb 2, 2014 at 4:40 AM, Dean Chester <dean.g.chester(a)gmail.com>
wrote:
> Hi Min,
>
> I've just been using the new firmware with a different type of sim
> card and noticed that the ATR is scrambled, at least for the header of the
> ATR. The ATR is the same length as the one I had great success with. The
> cards are running at the same speed according to the ATRs however the new
> card seems to send it's ATR over 3 commands not just one. Whats also
> interesting is the ATR scrambling is different. Here is an example with the
> Simtrace Header attached:
>
> 01 01 01 01 01 00 00 00 1a 00 e7 9a 03 1f c7 80 31 00 00 00 00 00 00
> 00 00 00
> 01 00 01 01 02 00 00 00 0f 00 00 00 00 00 00
> 01 00 01 01 03 00 00 00 12 00 00 00 00 00 00 00 00 00
>
> And the ATR should look like:
>
> 3B 9F 96 80 1F C7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>
> And with the same card from the same company I got:
>
> 01 01 01 01 01 00 00 00 16 00 9f 96 80 1f c7 80 31 00 00 00 00 00
> 01 00 01 01 02 00 00 00 13 00 67 42 70 02 10 00 00 01 f9
> 01 00 01 01 03 00 00 00 12 00 ff 10 96 79 ff 10 96 79
>
> With the card I had success with I got the following input in:
>
> 01 01 01 01 01 00 00 00 20 00 3b 9f 96 80 1f c7 80 31 00 00 00 00 00
> 00 00 00 00 00 00 00 00 00
>
> And the ATR looks like:
> 3b 9f 96 80 1f c7 80 31 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>
> Any ideas where these errors are coming from in the firmware. I notice
> that the lengths in the failed are equal to 3B which is the start of the
> ATR if thats of any help. The tests were done with the same phone connected
> to the tracer just swapping the sim cards.
>
> Kind Regards,
>
> Dean Chester
>
>
> On 30 January 2014 21:47, Min Xu <mxu(a)sanjole.com> wrote:
>
>> That does make sense to me since I had issues where before I changed
>> the usb header, the host was sometimes merging multiple req_ctx into a
>> single usb read.
>>
>> I am attaching a tar.gz file containing all my changes as separate
>> git commits / format-patches and hopefully it will get committed to the
>> repository. This includes the latest change to the usb header as well as a
>> change (as Peter suggested) to the bDeviceProtocol in
>> usb_descriptors_openpcd.h, setting the new value to 0.
>>
>> Best Regards
>>
>>
>> On Thu, Jan 30, 2014 at 6:21 AM, Dean Chester <
>> dean.g.chester(a)gmail.com> wrote:
>>
>>> It was mostly the lost bytes/scrambled data with a variety of
>>> handsets that i'd tested with, compared with ubuntu running natively.
>>>
>>> Dean
>>>
>>>
>>> On 27 January 2014 18:37, Min Xu <mxu(a)sanjole.com> wrote:
>>>
>>>> P.S. What problems were you experiencing of it running in a
>>>> virtualized system, can you elaborate?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On Mon, Jan 27, 2014 at 8:36 AM, Min Xu <mxu(a)sanjole.com> wrote:
>>>>
>>>>> I am contributing it to the project. Once I incorporate Peter
>>>>> Stuge's suggestion, hopefully within the next few weeks I will
submit
>>>>> another commit.
>>>>>
>>>>> Best Regards
>>>>>
>>>>>
>>>>> On Mon, Jan 27, 2014 at 1:05 AM, Dean Chester <
>>>>> dean.g.chester(a)gmail.com> wrote:
>>>>>
>>>>>> Thanks Min it works a treat it also fixes the issues running in
a
>>>>>> virtualised environment which I do for Ubuntu.
>>>>>>
>>>>>> Is your new firmware under the same licence as the original?
>>>>>>
>>>>>> Kind Regards,
>>>>>>
>>>>>> Dean Chester
>>>>>>
>>>>>>
>>>>>> On 24 January 2014 21:55, Peter Stuge <peter(a)stuge.se>
wrote:
>>>>>>
>>>>>>> Min Xu wrote:
>>>>>>> > I would be happy to send them the USB protocol changes.
>>>>>>> However,
>>>>>>> > it will be INCOMPATIBLE with earlier firmware based
SIMTrace
>>>>>>> boards.
>>>>>>>
>>>>>>> There is a standardised way to deal with protocol changes in
USB;
>>>>>>> change either the bDeviceProtocol field in the device
descriptor
>>>>>>> or
>>>>>>> the bInterfaceProtocol field in the interface descriptor,
and
>>>>>>> make
>>>>>>> host software do the appropriate thing based on the
descriptors
>>>>>>> of
>>>>>>> the connected device.
>>>>>>>
>>>>>>> Of course only new host software will work with the new
>>>>>>> protocol, but
>>>>>>> this way new host software still continues to work with the
old
>>>>>>> protocol.
>>>>>>>
>>>>>>>
>>>>>>> //Peter
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
3886
days inactive
4048
days old
14 comments
5 participants
participants (5)
-
Dean Chester -
Freddie -
Holger Hans Peter Freyther -
Min Xu -
Peter Stuge