From tom at zwizwa.be Thu Aug 1 19:58:20 2013 From: tom at zwizwa.be (Tom Schouten) Date: Thu, 01 Aug 2013 15:58:20 -0400 Subject: MITM Message-ID: <51FABDDC.2060108@zwizwa.be> Hi List, Here are some bare-minimum code contributions for iso7816-3 slave side protocol. https://github.com/zwizwa/at91work/tree/apdu_phone git at github.com:zwizwa/at91work.git branch apdu_phone Relevant files are in: iso7816-slave-cdc-project/ src/ I spent a bit more time on this than expected on low-level platform stuff, leaving me with no time to smooth the rough edges at the PC interface side. It's a start, though. Features: - non-blocking iso7816-3 state machine for slave-side protocol (reusable in COS?) - low-level AT91 USART is abstracted as iso7816_port.h - line-oriented ASCII HEX protocol over CDC USB serial, easy to use in (python) scripts - transport is separate from state machine, easy to replace with other discussed mechanisms - works for me on Nexus One and BLU SAMBA JR plus Non-features: - hardcoded ATR - hardcoded power cycle skipping to select correct operating voltage - not using SIMtrace SIM slot (see mim.py for using a PC card reader to complete MITM) - did not test with DFU but should work (I used Atmel SAM-ICE) - standalone firmware, not integrated with CCID / sniffer - git history is messy. i'd just flatten it for further work From mxu at sanjole.com Wed Aug 7 19:52:04 2013 From: mxu at sanjole.com (Min Xu) Date: Wed, 7 Aug 2013 09:52:04 -1000 Subject: Incomplete trace (due to high-speed SIM?) Message-ID: *Holger Hans Peter Freyther* holger at freyther.de *Wed May 22 12:23:01 CEST 2013* | On Tue, May 21, 2013 at 03:53:45PM +0200, Arthur L?na wrote: | >* I tried booting the iPhone using 2 different SIMs (this one from * | >* Free Mobile and another one) and the trace seems very short using | *>* the Free Mobile in comparison to the other one. By parsing the ATRs | *>* of both SIMs, I found out that the communication is done at 312500 | *>* bits/s with the Free Mobile SIM and only 78125 bits/s (for a clock | *>* running at 5MHz). Could this explain the loss of entire ADPUs by the * | >* tracer? If yes, is there a way to remedy to this problem (firmware * | >* update for example). *| Some people reported success by reducing the size of buffered bytes in the | firmware. | holger Holger There appears to be numerous references to "decrease the buffer in the firmware" to workaround the problem of losing bytes... but can you clarify which buffer to decrease, in which source file? I am guessing it's the req_ctx.c / req_ctx.h? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom at zwizwa.be Thu Aug 8 21:38:54 2013 From: tom at zwizwa.be (Tom Schouten) Date: Thu, 08 Aug 2013 17:38:54 -0400 Subject: MITM In-Reply-To: <51FABDDC.2060108@zwizwa.be> References: <51FABDDC.2060108@zwizwa.be> Message-ID: <52040FEE.6040705@zwizwa.be> > Non-features: > - hardcoded ATR > - hardcoded power cycle skipping to select correct operating voltage fixed: these are now implemented with commands that are encapsulated in the usual APDU traffic, to allow for different packet bearers later. currently I'm sticking with standalone ASCII HEX lines over CDC as that seems to work well. i'll update bug fixes to the github page. there are plenty of bugs still. if you'd like to have a look at the code, best to drop me a line to make sure you have the latest version. i'll stop spamming here for now :) From peter at stuge.se Thu Aug 8 23:46:25 2013 From: peter at stuge.se (Peter Stuge) Date: Fri, 9 Aug 2013 01:46:25 +0200 Subject: MITM In-Reply-To: <52040FEE.6040705@zwizwa.be> References: <51FABDDC.2060108@zwizwa.be> <52040FEE.6040705@zwizwa.be> Message-ID: <20130808234625.28483.qmail@stuge.se> Tom Schouten wrote: > commands that are encapsulated in the usual APDU traffic I would really like if there was less APDU crap rather than more. Sad face. > currently I'm sticking with standalone ASCII HEX lines over CDC What about control requests? //Peter From tom at zwizwa.be Fri Aug 9 00:51:12 2013 From: tom at zwizwa.be (Tom Schouten) Date: Thu, 08 Aug 2013 20:51:12 -0400 Subject: MITM In-Reply-To: <20130808234625.28483.qmail@stuge.se> References: <51FABDDC.2060108@zwizwa.be> <52040FEE.6040705@zwizwa.be> <20130808234625.28483.qmail@stuge.se> Message-ID: <52043D00.6050909@zwizwa.be> On 08/08/2013 07:46 PM, Peter Stuge wrote: > Tom Schouten wrote: >> commands that are encapsulated in the usual APDU traffic > I would really like if there was less APDU crap rather than more. Sad face. I agree. To be removed at some point. I kept the separation between this 3-line hack and the functional code clean. It's easy to add a proper transport later. > >> currently I'm sticking with standalone ASCII HEX lines over CDC > What about control requests? It makes sense to do this once the CCID device works properly. I wonder though, control requests are PC->USB only, right? It wouldn't work well for incoming asynchronous C-APDU from the phone, as the PC needs to poll constantly in software. For the CDC serial at least it's on an IN endpoint. > > > //Peter > > From peter at stuge.se Fri Aug 9 01:16:59 2013 From: peter at stuge.se (Peter Stuge) Date: Fri, 9 Aug 2013 03:16:59 +0200 Subject: MITM In-Reply-To: <52043D00.6050909@zwizwa.be> References: <51FABDDC.2060108@zwizwa.be> <52040FEE.6040705@zwizwa.be> <20130808234625.28483.qmail@stuge.se> <52043D00.6050909@zwizwa.be> Message-ID: <20130809011659.2791.qmail@stuge.se> Tom Schouten wrote: > I wonder though, control requests are PC->USB only, right? Wrong. > It wouldn't work well for incoming asynchronous C-APDU from the phone It would work fine. > as the PC needs to poll constantly in software All USB communication is polled. Study chapters 5 and 8 of the USB 2.0 specification. http://www.usb.org/developers/docs/usb_20_110512.zip //Peter From tom at zwizwa.be Fri Aug 9 01:43:19 2013 From: tom at zwizwa.be (Tom Schouten) Date: Thu, 08 Aug 2013 21:43:19 -0400 Subject: MITM In-Reply-To: <20130809011659.2791.qmail@stuge.se> References: <51FABDDC.2060108@zwizwa.be> <52040FEE.6040705@zwizwa.be> <20130808234625.28483.qmail@stuge.se> <52043D00.6050909@zwizwa.be> <20130809011659.2791.qmail@stuge.se> Message-ID: <52044937.8080706@zwizwa.be> On 08/08/2013 09:16 PM, Peter Stuge wrote: > Tom Schouten wrote: >> I wonder though, control requests are PC->USB only, right? > Wrong. > > >> It wouldn't work well for incoming asynchronous C-APDU from the phone > It would work fine. > > >> as the PC needs to poll constantly in software > All USB communication is polled. > > Study chapters 5 and 8 of the USB 2.0 specification. > > http://www.usb.org/developers/docs/usb_20_110512.zip Yes I understand USB is polled. Though there's a difference between polling by the usb controller and polling at the software level, meaning having to send multiple control requests until there is data ready vs just waiting until there is data on an IN endpoint. I don't believe a control request is meant to block for a long time. I.e. a if there is an IN stage after the SETUP stage, it's supposed to answer promptly to the control packet sent by the host. > > //Peter > > From tom at zwizwa.be Fri Aug 9 02:25:30 2013 From: tom at zwizwa.be (Tom Schouten) Date: Thu, 08 Aug 2013 22:25:30 -0400 Subject: SIMtrace MITM/emulator In-Reply-To: <20130717123710.4804.qmail@stuge.se> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> Message-ID: <5204531A.10002@zwizwa.be> On 07/17/2013 08:37 AM, Peter Stuge wrote: > Tom Schouten wrote: >>> Another method would be type=vendor recipient=device control requests >>> over the default endpoint. This works well everywhere except on Windows. >> Does this work if the device is already bound by another driver - in >> this case libccid? > I'll repeat myself: This works well everywhere except on Windows. Well... I tried this for the CDC and I get "Device or resource busy". It doesn't work if the device is already opened by another process or a kernel driver. It seems plausible a USB device can only be opened once, so you're can you elaborate on what you are referring to by "it works well everywhere" wrt "already bound by another driver" ? From peter at stuge.se Fri Aug 9 06:53:24 2013 From: peter at stuge.se (Peter Stuge) Date: Fri, 9 Aug 2013 08:53:24 +0200 Subject: MITM In-Reply-To: <52044937.8080706@zwizwa.be> References: <51FABDDC.2060108@zwizwa.be> <52040FEE.6040705@zwizwa.be> <20130808234625.28483.qmail@stuge.se> <52043D00.6050909@zwizwa.be> <20130809011659.2791.qmail@stuge.se> <52044937.8080706@zwizwa.be> Message-ID: <20130809065324.27525.qmail@stuge.se> Tom Schouten wrote: > Yes I understand USB is polled. Good, except.. > Though there's a difference between polling by the usb controller > and polling at the software level ..no, not really. > meaning having to send multiple control requests until there is > data ready vs just waiting until there is data on an IN endpoint. The difference is actually very small, but of course it is more convenient to have a dedicated endpoint. I guess the point is that there isn't one. //Peter From peter at stuge.se Fri Aug 9 06:59:15 2013 From: peter at stuge.se (Peter Stuge) Date: Fri, 9 Aug 2013 08:59:15 +0200 Subject: SIMtrace MITM/emulator In-Reply-To: <5204531A.10002@zwizwa.be> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> Message-ID: <20130809065915.28019.qmail@stuge.se> Tom Schouten wrote: >>>> Another method would be type=vendor recipient=device control requests >>>> over the default endpoint. > > Well... I tried this for the CDC and I get "Device or resource busy". What did you try exactly? > It doesn't work if the device is already opened by another process or a > kernel driver. There is always a kernel driver for every single interface, also when using libusb. It is just a different kernel driver. It is not possible to communicate with a USB device without a kernel driver, also using libusb. "open the device" is not really a significant concept with USB, it is much more important what happens with individual interfaces. libccid uses libusb to claim the CCID interface, but that is distinct from the default endpoint. > It seems plausible a USB device can only be opened once, That's not the case however. > so you're can you elaborate on what you are referring to by > "it works well everywhere" wrt "already bound by another driver" ? Please describe what you've done in more detail instead. //Peter From holger at freyther.de Fri Aug 9 07:58:47 2013 From: holger at freyther.de (Holger Hans Peter Freyther) Date: Fri, 9 Aug 2013 09:58:47 +0200 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: References: Message-ID: <20130809075847.GA4254@xiaoyu.lan> On Wed, Aug 07, 2013 at 09:52:04AM -1000, Min Xu wrote: Dear Min, > There appears to be numerous references to "decrease the buffer in the > firmware" to workaround the problem of losing bytes... but src/simtrace/iso7816_uart.c:process_byte if (rctx->tot_len >= rctx->size || ih->rctx_must_be_sent) { ih->rctx_must_be_sent = 0; send_rctx(ih); } You could change this part. > can you clarify which buffer to decrease, in which source file? > > I am guessing it's the req_ctx.c / req_ctx.h? yes, or make sure the ->size is smaller. From tom at zwizwa.be Fri Aug 9 12:08:37 2013 From: tom at zwizwa.be (Tom Schouten) Date: Fri, 09 Aug 2013 08:08:37 -0400 Subject: SIMtrace MITM/emulator In-Reply-To: <20130809065915.28019.qmail@stuge.se> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> <20130809065915.28019.qmail@stuge.se> Message-ID: <5204DBC5.3060700@zwizwa.be> On 08/09/2013 02:59 AM, Peter Stuge wrote: > Tom Schouten wrote: >>>>> Another method would be type=vendor recipient=device control requests >>>>> over the default endpoint. >> Well... I tried this for the CDC and I get "Device or resource busy". > What did you try exactly? # using pyusb 0.x # this is running while CDC is claimed by usb-serial import usb def find(idVendor, idProduct): busses = usb.busses() for bus in busses: devices = bus.devices for dev in devices: if ((dev.idVendor == idVendor) and (dev.idProduct == idProduct)): return dev return 'wrong' dev = find(idVendor=0x03eb, idProduct=0x6119) dh = dev.open() # dh.detachKernelDriver(0) dh.claimInterface(0) # -> usb.USBError: could not claim interface 0: Device or resource busy dh.controlMsg(1,1,[0,0,0,0]) > >> It doesn't work if the device is already opened by another process or a >> kernel driver. > There is always a kernel driver for every single interface, also when > using libusb. It is just a different kernel driver. It is not possible > to communicate with a USB device without a kernel driver, also using > libusb. > > "open the device" is not really a significant concept with USB, it is > much more important what happens with individual interfaces. libccid > uses libusb to claim the CCID interface, but that is distinct from > the default endpoint. > > >> It seems plausible a USB device can only be opened once, > That's not the case however. > I see. I'm confusing device and interface. So basically, I would add another interface to the USB firmware, which then can be claimed by a different process. Since all control transfers use the default endpoint, this won't need extra endpoints which we don't have. Neat. Thanks! From peter at stuge.se Fri Aug 9 13:06:56 2013 From: peter at stuge.se (Peter Stuge) Date: Fri, 9 Aug 2013 15:06:56 +0200 Subject: SIMtrace MITM/emulator In-Reply-To: <5204DBC5.3060700@zwizwa.be> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> <20130809065915.28019.qmail@stuge.se> <5204DBC5.3060700@zwizwa.be> Message-ID: <20130809130656.24645.qmail@stuge.se> Tom Schouten wrote: > dh = dev.open() This opens the device. > dh.claimInterface(0) # -> usb.USBError: could not claim interface 0: Device or resource busy This tries to claim the interface which is already claimed by usb-serial. > dh.controlMsg(1,1,[0,0,0,0]) This performs a control request over the default endpoint, which doesn't belong to any interface at all. >>> It seems plausible a USB device can only be opened once, >> >> That's not the case however. > > I see. I'm confusing device and interface. Yes. > So basically, I would add another interface to the USB firmware, > which then can be claimed by a different process. You can do control requests without having claimed any interface. If you add an interface then it might actually work on Windows too, but I'm not sure. //Peter From tom at zwizwa.be Fri Aug 9 13:59:26 2013 From: tom at zwizwa.be (Tom Schouten) Date: Fri, 09 Aug 2013 09:59:26 -0400 Subject: SIMtrace MITM/emulator In-Reply-To: <20130809130656.24645.qmail@stuge.se> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> <20130809065915.28019.qmail@stuge.se> <5204DBC5.3060700@zwizwa.be> <20130809130656.24645.qmail@stuge.se> Message-ID: <5204F5BE.8010706@zwizwa.be> On 08/09/2013 09:06 AM, Peter Stuge wrote: > >> So basically, I would add another interface to the USB firmware, >> which then can be claimed by a different process. > You can do control requests without having claimed any interface. This seems to be the case only if an interface on the device is not claimed already. dev = find(idVendor=0x03eb, idProduct=0x6129) dh = dev.open() dh.controlMsg(1,1,[0,0,0,0]) # no error when CCID interface is not claimed, busy otherwise > If > you add an interface then it might actually work on Windows too, but > I'm not sure. Harald had already added extra DFU interfaces to the CCID code, so I can have a look at how that works. > //Peter > > Tom From tom at zwizwa.be Fri Aug 9 14:42:15 2013 From: tom at zwizwa.be (Tom Schouten) Date: Fri, 09 Aug 2013 10:42:15 -0400 Subject: SIMtrace MITM/emulator In-Reply-To: <5204F5BE.8010706@zwizwa.be> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> <20130809065915.28019.qmail@stuge.se> <5204DBC5.3060700@zwizwa.be> <20130809130656.24645.qmail@stuge.se> <5204F5BE.8010706@zwizwa.be> Message-ID: <5204FFC7.9090905@zwizwa.be> On 08/09/2013 09:59 AM, Tom Schouten wrote: > On 08/09/2013 09:06 AM, Peter Stuge wrote: > >> >>> So basically, I would add another interface to the USB firmware, >>> which then can be claimed by a different process. >> You can do control requests without having claimed any interface. > > This seems to be the case only if an interface on the device is not > claimed already. > > dev = find(idVendor=0x03eb, idProduct=0x6129) > dh = dev.open() > dh.controlMsg(1,1,[0,0,0,0]) # no error when CCID interface is not > claimed, busy otherwise > > ha! it depends on bmRequestType (9.3.1 in usb11.pdf) a VENDOR request (bmRequestType = 0xC0) does make it through to the firmware, while STANDARD requests seem to be limited. so to make it work on windows, I guess these then should be INTERFACE instead of DEVICE requests? From mxu at sanjole.com Fri Aug 9 20:54:28 2013 From: mxu at sanjole.com (Min Xu) Date: Fri, 9 Aug 2013 10:54:28 -1000 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: <20130809075847.GA4254@xiaoyu.lan> References: <20130809075847.GA4254@xiaoyu.lan> Message-ID: Hi Holger Thanks for getting back to me. After analyzing the issue, I believe the issue is the overrun error the USART reports. The issue appear to stem from the fact that each byte received by the USART is handled in the interrupt handler, but the USB sending code locks out the interrupt for the entirety of its call. So I change the code in the following manner and it appears to solve the issue for me: 1) Increase the small buffer to be large enough to hold 1 typical apdu: 270 bytes: 4 byte SIMTrace header, 4 byte command, 2 byte status, 1 byte instruction that appears to be repeated?? 256 bytes payload 2) Disable USB interrupt before deciding to Continue to send stuff to USB until returning from function. 3) lock out interrupt ONLY when retrieving a new req_ctx to transmit to USB 4) Some additional code change to detect if a req_ctx is completely transmitted .. 5) An additional function call in the main loop to detect (and output to debug serial) if an overrun occured Attached is my diff file based on what's in git. Let me know if it works for you On Thu, Aug 8, 2013 at 9:58 PM, Holger Hans Peter Freyther < holger at freyther.de> wrote: > On Wed, Aug 07, 2013 at 09:52:04AM -1000, Min Xu wrote: > > Dear Min, > > > > There appears to be numerous references to "decrease the buffer in the > > firmware" to workaround the problem of losing bytes... but > > > src/simtrace/iso7816_uart.c:process_byte > > if (rctx->tot_len >= rctx->size || ih->rctx_must_be_sent) { > ih->rctx_must_be_sent = 0; > send_rctx(ih); > } > > You could change this part. > > > can you clarify which buffer to decrease, in which source file? > > > > I am guessing it's the req_ctx.c / req_ctx.h? > > yes, or make sure the ->size is smaller. > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: simtrace.diff Type: application/octet-stream Size: 5392 bytes Desc: not available URL: From holger at freyther.de Sat Aug 10 07:16:36 2013 From: holger at freyther.de (Holger Hans Peter Freyther) Date: Sat, 10 Aug 2013 09:16:36 +0200 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: References: <20130809075847.GA4254@xiaoyu.lan> Message-ID: <20130810071636.GB10447@xiaoyu.lan> On Fri, Aug 09, 2013 at 10:54:28AM -1000, Min Xu wrote: > Hi Holger Dear Min, > > Thanks for getting back to me. After analyzing the issue, I believe the > issue is the overrun error the USART reports. The issue appear to stem > from the fact that each byte received by the USART is handled in the > interrupt handler, but the USB sending code locks out the interrupt for the > entirety of its call. So I change the code in the following manner and it > appears to solve the issue for me: thank you for the analysis and the change! Would you like to create entire git commits for the various things and follow the kernel coding style? > 5) An additional function call in the main loop to detect (and output to > debug serial) if an overrun occured great! We have a slightly different coding style but the change itself looks good and I would support it to be included. > 1) Increase the small buffer to be large enough to hold 1 typical apdu: > 270 bytes: 4 byte SIMTrace header, 4 byte command, 2 byte status, 1 byte > instruction that appears to be repeated?? 256 bytes payload this seems reasonable. Using git gui blame on the req_ctx.h I notice that this has already been increased for the SIM, increasing it further can make sense. > > 2) Disable USB interrupt before deciding to Continue to send stuff to USB > until returning from function. > > 3) lock out interrupt ONLY when retrieving a new req_ctx to transmit to USB > > 4) Some additional code change to detect if a req_ctx is completely > transmitted .. I can't comment on these three as I have not looked into the inner workings of the USB code yet. I would propose we start with creating three git commits for the first three things and then look into locking/disabling irqs. thanks a lot holger From tom at zwizwa.be Mon Aug 12 18:06:46 2013 From: tom at zwizwa.be (Tom Schouten) Date: Mon, 12 Aug 2013 14:06:46 -0400 Subject: SIMtrace MITM/emulator In-Reply-To: <5204FFC7.9090905@zwizwa.be> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> <20130809065915.28019.qmail@stuge.se> <5204DBC5.3060700@zwizwa.be> <20130809130656.24645.qmail@stuge.se> <5204F5BE.8010706@zwizwa.be> <5204FFC7.9090905@zwizwa.be> Message-ID: <52092436.4070603@zwizwa.be> USB vendor control requests draft works - previous was all PEBKAC. Peter, thanks for the idea! From peter at stuge.se Mon Aug 12 18:22:43 2013 From: peter at stuge.se (Peter Stuge) Date: Mon, 12 Aug 2013 20:22:43 +0200 Subject: SIMtrace MITM/emulator In-Reply-To: <52092436.4070603@zwizwa.be> References: <20130715162331.9328.qmail@stuge.se> <51E5A35C.4070008@zwizwa.be> <20130717123710.4804.qmail@stuge.se> <5204531A.10002@zwizwa.be> <20130809065915.28019.qmail@stuge.se> <5204DBC5.3060700@zwizwa.be> <20130809130656.24645.qmail@stuge.se> <5204F5BE.8010706@zwizwa.be> <5204FFC7.9090905@zwizwa.be> <52092436.4070603@zwizwa.be> Message-ID: <20130812182243.32387.qmail@stuge.se> Tom Schouten wrote: > USB vendor control requests draft works - previous was all PEBKAC. Glad to hear that! > Peter, thanks for the idea! You're welcome. Thanks for working on it! //Peter From rupert.lloyd at gmail.com Wed Aug 14 01:36:20 2013 From: rupert.lloyd at gmail.com (Rupert Lloyd) Date: Tue, 13 Aug 2013 21:36:20 -0400 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: References: <20130809075847.GA4254@xiaoyu.lan> Message-ID: I applied this patch and it seems to have fixed my dropped byte issue. Before the patch I could not capture more than ~10 APDUs without dropping bytes, now I am getting through the entire startup sequence (600+ APDUS) Thanks for the change! Rupert On Fri, Aug 9, 2013 at 4:54 PM, Min Xu wrote: > Hi Holger > > Thanks for getting back to me. After analyzing the issue, I believe the > issue is the overrun error the USART reports. The issue appear to stem > from the fact that each byte received by the USART is handled in the > interrupt handler, but the USB sending code locks out the interrupt for the > entirety of its call. So I change the code in the following manner and it > appears to solve the issue for me: > > 1) Increase the small buffer to be large enough to hold 1 typical apdu: > 270 bytes: 4 byte SIMTrace header, 4 byte command, 2 byte status, 1 byte > instruction that appears to be repeated?? 256 bytes payload > > 2) Disable USB interrupt before deciding to Continue to send stuff to USB > until returning from function. > > 3) lock out interrupt ONLY when retrieving a new req_ctx to transmit to > USB > > 4) Some additional code change to detect if a req_ctx is completely > transmitted .. > > 5) An additional function call in the main loop to detect (and output to > debug serial) if an overrun occured > > Attached is my diff file based on what's in git. > > Let me know if it works for you > > > On Thu, Aug 8, 2013 at 9:58 PM, Holger Hans Peter Freyther < > holger at freyther.de> wrote: > >> On Wed, Aug 07, 2013 at 09:52:04AM -1000, Min Xu wrote: >> >> Dear Min, >> >> >> > There appears to be numerous references to "decrease the buffer in the >> > firmware" to workaround the problem of losing bytes... but >> >> >> src/simtrace/iso7816_uart.c:process_byte >> >> if (rctx->tot_len >= rctx->size || ih->rctx_must_be_sent) { >> ih->rctx_must_be_sent = 0; >> send_rctx(ih); >> } >> >> You could change this part. >> >> > can you clarify which buffer to decrease, in which source file? >> > >> > I am guessing it's the req_ctx.c / req_ctx.h? >> >> yes, or make sure the ->size is smaller. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mxu at sanjole.com Wed Aug 14 01:42:06 2013 From: mxu at sanjole.com (Min Xu) Date: Tue, 13 Aug 2013 15:42:06 -1000 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: References: <20130809075847.GA4254@xiaoyu.lan> Message-ID: Thanks for the feedback Rupert. Holger I am not familiar with git... but isn't there access restrictions on commit to the git similar to svn? If not, then I would try commit it later this week. Thanks On Tue, Aug 13, 2013 at 3:36 PM, Rupert Lloyd wrote: > I applied this patch and it seems to have fixed my dropped byte issue. > Before the patch I could not capture more than ~10 APDUs without dropping > bytes, now I am getting through the entire startup sequence (600+ APDUS) > > Thanks for the change! > > Rupert > > > On Fri, Aug 9, 2013 at 4:54 PM, Min Xu wrote: > >> Hi Holger >> >> Thanks for getting back to me. After analyzing the issue, I believe the >> issue is the overrun error the USART reports. The issue appear to stem >> from the fact that each byte received by the USART is handled in the >> interrupt handler, but the USB sending code locks out the interrupt for the >> entirety of its call. So I change the code in the following manner and it >> appears to solve the issue for me: >> >> 1) Increase the small buffer to be large enough to hold 1 typical apdu: >> 270 bytes: 4 byte SIMTrace header, 4 byte command, 2 byte status, 1 byte >> instruction that appears to be repeated?? 256 bytes payload >> >> 2) Disable USB interrupt before deciding to Continue to send stuff to >> USB until returning from function. >> >> 3) lock out interrupt ONLY when retrieving a new req_ctx to transmit to >> USB >> >> 4) Some additional code change to detect if a req_ctx is completely >> transmitted .. >> >> 5) An additional function call in the main loop to detect (and output to >> debug serial) if an overrun occured >> >> Attached is my diff file based on what's in git. >> >> Let me know if it works for you >> >> >> On Thu, Aug 8, 2013 at 9:58 PM, Holger Hans Peter Freyther < >> holger at freyther.de> wrote: >> >>> On Wed, Aug 07, 2013 at 09:52:04AM -1000, Min Xu wrote: >>> >>> Dear Min, >>> >>> >>> > There appears to be numerous references to "decrease the buffer in the >>> > firmware" to workaround the problem of losing bytes... but >>> >>> >>> src/simtrace/iso7816_uart.c:process_byte >>> >>> if (rctx->tot_len >= rctx->size || ih->rctx_must_be_sent) { >>> ih->rctx_must_be_sent = 0; >>> send_rctx(ih); >>> } >>> >>> You could change this part. >>> >>> > can you clarify which buffer to decrease, in which source file? >>> > >>> > I am guessing it's the req_ctx.c / req_ctx.h? >>> >>> yes, or make sure the ->size is smaller. >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at stuge.se Wed Aug 14 10:40:28 2013 From: peter at stuge.se (Peter Stuge) Date: Wed, 14 Aug 2013 12:40:28 +0200 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: References: <20130809075847.GA4254@xiaoyu.lan> Message-ID: <20130814104028.8971.qmail@stuge.se> Hi, Thanks for working on this. Min Xu wrote: > I am not familiar with git... but isn't there access restrictions > on commit to the git similar to svn? An important difference between git and svn is that git is distributed instead of centralized. This means that every developer starts with an identical copy "a clone" of the osmocom.org repository in their worktree, and everyone can create commits locally. Complete commits can then be transfered between developers by a few different means, one of which is email. This significantly reduces workload of project maintainers, since they can include well-formed commits from other developers very quickly. A git commit includes both author and commiter information, and the commit message, so one effect of using git is that new developers can easily contribute directly to the main repository, by sending a commit they have created locally via email to the project. If you can send email on the workstation where you run git, you can use the git send-email command (with appropriate parameters). If you need a separate patch file to attach in a standalone email program you can run git format-patch (with appropriate parameters, such as -1 in order to generate a patch file for the last 1 commit). Please configure git with your name, and please follow the common practise for commit messages; on line 1 a short (<=72 chars) summary, line 2 blank, line 3 and onwards a longer description and rationale for the change. Try to keep lines 3 and later <=74 chars. I recommend the book "Pro Git" at http://git-scm.com/book as well as the more lightweight Git Reference at http://gitref.org/ and the #git IRC channel on irc.freenode.net for community help. //Peter From alexander.huemer at xx.vu Wed Aug 14 11:19:53 2013 From: alexander.huemer at xx.vu (Alexander Huemer) Date: Wed, 14 Aug 2013 13:19:53 +0200 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: <20130814104028.8971.qmail@stuge.se> References: <20130809075847.GA4254@xiaoyu.lan> <20130814104028.8971.qmail@stuge.se> Message-ID: <20130814111953.GP16935@yade.xx.vu> On Wed, Aug 14, 2013 at 12:40:28PM +0200, Peter Stuge wrote: > If you can send email on the workstation where you run git, you can > use the git send-email command (with appropriate parameters). If you > need a separate patch file to attach in a standalone email program > you can run git format-patch (with appropriate parameters, such as -1 > in order to generate a patch file for the last 1 commit). There is also git imap-send, which can be easier to set up when no MTA is running on the machine where git is used. Kind regards, -Alexander Huemer From tom at zwizwa.be Fri Aug 16 15:22:28 2013 From: tom at zwizwa.be (Tom Schouten) Date: Fri, 16 Aug 2013 11:22:28 -0400 Subject: SIMtrace MITM/emulator In-Reply-To: <20130710215236.GH6004@nataraja.gnumonks.org> References: <51DB4096.8050508@zwizwa.be> <1373332541-sup-497@dennou> <51DC252A.5000902@zwizwa.be> <20130710152609.GX6004@nataraja.gnumonks.org> <20130710193558.963578E6@centrum.cz> <51DDCE37.9000406@zwizwa.be> <20130710215236.GH6004@nataraja.gnumonks.org> Message-ID: <520E43B4.6040603@zwizwa.be> Hi Harald, On 07/10/2013 05:52 PM, Harald Welte wrote: > Hi Tom, > > On Wed, Jul 10, 2013 at 05:12:23PM -0400, Tom Schouten wrote: >>>> git://git.gnumonks.org/at91work.git >> Great. I'll check it out. Is there anything essential that is missing? > for CCID (card reader operation) not, it is basically the Atmel > reference code merged with my DFU loader work and configured for the > SIMtrace hardware. Just to make sure: when you last left the CCID firmware, was it working? Were you able to read a SIM card with pcsclite / libccid ? I found at least one bug that makes me think something changed in libccid after you had it work (see other post about two kinds of status messages being confused: CCID spec 6.3.1 and 4.2.1) Cheers, Tom From tom at zwizwa.be Fri Aug 16 15:38:52 2013 From: tom at zwizwa.be (Tom Schouten) Date: Fri, 16 Aug 2013 11:38:52 -0400 Subject: board questions regarding MitM Message-ID: <520E478C.1090907@zwizwa.be> Hi List, When the SIMtrace ciruit operates in MitM mode, the RST and VCC lines coming from the phone are very noisy from I/O line crosstalk in the flex cable. I see about 2V peak-to-peak on the scope! I am expecting it leads to spurious reset/poweroff detections. It might be helpful to add a filter on the AT91 side for RST and VCC. Maye an R/C filter that can be enabled by an I/O pin using 0/HI-Z switching? Another idea is to do enable the internal AT91 pin pullups. Just loading RST and VCC a bit might be enough. ( Another alternative is to filter RST and VCC measurements in software. I tried this but results are not very meaningful at this point.. ) Cheers, Tom From tom at zwizwa.be Fri Aug 16 15:43:12 2013 From: tom at zwizwa.be (Tom Schouten) Date: Fri, 16 Aug 2013 11:43:12 -0400 Subject: simtrace arduino due? Message-ID: <520E4890.2040600@zwizwa.be> Just an idea.. What about building a next version based on the Arduino Due? It's based on a SAM3X8E http://arduino.cc/en/Main/arduinoBoardDue From ml at mail.tsaitgaist.info Fri Aug 16 18:01:08 2013 From: ml at mail.tsaitgaist.info (Kevin Redon) Date: Fri, 16 Aug 2013 20:01:08 +0200 Subject: board questions regarding MitM In-Reply-To: <520E478C.1090907@zwizwa.be> References: <520E478C.1090907@zwizwa.be> Message-ID: <1376675870-sup-8808@dennou> Thanks for the info. I did not looked into it yet, but will either try to fix it in software (the pullup is a good start) before designing the second version of SIMtrace, or add a filter, cap, or pull-down if this is not possible. Kevin Excerpts from Tom Schouten's message of 2013-08-16 17:38:52 +0200: > Hi List, > > When the SIMtrace ciruit operates in MitM mode, the RST and VCC lines > coming from the phone are very noisy from I/O line crosstalk in the flex > cable. > I see about 2V peak-to-peak on the scope! > > I am expecting it leads to spurious reset/poweroff detections. > > It might be helpful to add a filter on the AT91 side for RST and VCC. > Maye an R/C filter that can be enabled by an I/O pin using 0/HI-Z switching? > > Another idea is to do enable the internal AT91 pin pullups. > Just loading RST and VCC a bit might be enough. > > ( Another alternative is to filter RST and VCC measurements in > software. I tried this but results are not very meaningful at this > point.. ) > > Cheers, > Tom From ml at mail.tsaitgaist.info Fri Aug 16 18:09:13 2013 From: ml at mail.tsaitgaist.info (Kevin Redon) Date: Fri, 16 Aug 2013 20:09:13 +0200 Subject: simtrace arduino due? In-Reply-To: <520E4890.2040600@zwizwa.be> References: <520E4890.2040600@zwizwa.be> Message-ID: <1376676134-sup-7191@dennou> for SIMtrace v2 we thought about using a SAM3S because it has the same functionality core as the SAM7S, which by setting the registers right, allows you to sniff the traffic. We should test if the SAM3X8E also supports that first (AFAIR it's an undocumented feature). kevin Excerpts from Tom Schouten's message of 2013-08-16 17:43:12 +0200: > Just an idea.. > What about building a next version based on the Arduino Due? > It's based on a SAM3X8E > http://arduino.cc/en/Main/arduinoBoardDue From laforge at gnumonks.org Sat Aug 24 14:05:53 2013 From: laforge at gnumonks.org (Harald Welte) Date: Sat, 24 Aug 2013 16:05:53 +0200 Subject: Incomplete trace (due to high-speed SIM?) In-Reply-To: References: <20130809075847.GA4254@xiaoyu.lan> Message-ID: <20130824140553.GF16941@nataraja.gnumonks.org> Hi Min Xu, On Fri, Aug 09, 2013 at 10:54:28AM -1000, Min Xu wrote: > Thanks for getting back to me. After analyzing the issue, I believe the > issue is the overrun error the USART reports. The issue appear to stem > from the fact that each byte received by the USART is handled in the > interrupt handler, but the USB sending code locks out the interrupt for the > entirety of its call. Thanks again for analyzing the problem. I was pondering to apply your patch, but think a different approach might be better: Use the USART DMA mode. Thiss way we don't get one interrupt per character, but one interrupt every 'dma buffer size' characters. Using double buffering we should be able to always process one particular dma buffer, while the other buffer is being filled by the hardware without interrupts. Processing of the DMA buffer could happen either in an interrupt handler of the PDC (dma controller) or even in the main loop. Unfortunately the SAM7S USB controller does not support DMA, so it is impossible to improve on that side. I will give this some more thought. If I'm unable to implement this within short time, I might merge your patch anyway as a interim fix. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From tom at zwizwa.be Wed Aug 28 21:12:00 2013 From: tom at zwizwa.be (Tom Schouten) Date: Wed, 28 Aug 2013 17:12:00 -0400 Subject: board questions regarding MitM In-Reply-To: <1376675870-sup-8808@dennou> References: <520E478C.1090907@zwizwa.be> <1376675870-sup-8808@dennou> Message-ID: <521E67A0.4080003@zwizwa.be> Please disregard previous report on noise. The signals are fine. My scope probe handling and soldering skills are not :) I had the probe ground connected to a 100mil header I put on BAT1 which was badly soldered. Taking it at or near the connector makes it all disappear. The reset/poweroff I'm experiencing is probably still a protocol bug, since it happens fairly predictably. On 08/16/2013 02:01 PM, Kevin Redon wrote: > Thanks for the info. > I did not looked into it yet, but will either try to fix it in software (the pullup is a good start) before designing the second version of SIMtrace, > or add a filter, cap, or pull-down if this is not possible. > > Kevin > > Excerpts from Tom Schouten's message of 2013-08-16 17:38:52 +0200: >> Hi List, >> >> When the SIMtrace ciruit operates in MitM mode, the RST and VCC lines >> coming from the phone are very noisy from I/O line crosstalk in the flex >> cable. >> I see about 2V peak-to-peak on the scope! >> >> I am expecting it leads to spurious reset/poweroff detections. >> >> It might be helpful to add a filter on the AT91 side for RST and VCC. >> Maye an R/C filter that can be enabled by an I/O pin using 0/HI-Z switching? >> >> Another idea is to do enable the internal AT91 pin pullups. >> Just loading RST and VCC a bit might be enough. >> >> ( Another alternative is to filter RST and VCC measurements in >> software. I tried this but results are not very meaningful at this >> point.. ) >> >> Cheers, >> Tom >