I've been thought-experimenting with capturing data from two devices and measuring their correlation. For example, if two rtl-sdr processes are launched to read one sample block from two devices, how closely do the two devices' samples line up with each other in time?
I'm expecting that there will be no reliable correlation. My working theory is that each process calls into libusb_handle_events_timeout according to its own process schedule and that there is no way to coordinate these two calls to happen closely enough in time for the transfers to happen in a more synchronized fashion. Threads do not appear to be any better solution to the problem. I'm not an expert in usb transfers, or libusb, or rtlsdr for that matter, so I'm going only on what I can glean from reading the source code of librtlsdr.c and libusb documentation.
libusb does say however that it can specifically handle multiple devices in one async loop. Multiple bulk transfers can be set up to multiple devices in one process, and one call to libusb_handle_events_* can be used to handle those transfers. My thinking is that because the triggering event for both samples is now the same function call in the same process, the samples will be more synchronized, ideally down to some reliably offset of say a few microseconds. Juha Vierinen's hard-hack to make two devices use a coherent sample clock makes this idea even more interesting to me.
But, it doesn't look like librtlsdr can handle multiple devices. I'm game to take a stab at a patch that can extend its functionality. But I wanted to run this idea by the community first before spending code time on figuring this out. Do the experts here think that there is any potential benefit to be gained by taking this approach?
-Michel
Michel Pelletier wrote:
I'm expecting that there will be no reliable correlation.
TL;DR: That's correct. You'll always have some tens of ms of jitter.
My working theory is that each process calls into libusb_handle_events_timeout according to its own process schedule and that there is no way to coordinate these two calls to happen closely enough in time for the transfers to happen in a more synchronized fashion.
That's also correct. This is not a function of host software.
Threads do not appear to be any better solution to the problem.
There isn't.
I'm not an expert in usb transfers, or libusb
I do know USB and libusb and the key in this case is how bulk transfers are scheduled on the bus - namely optimized for throughput. Bulk transfers fill whatever bus time is free up to the maximum throughput the device can handle.
And even if we imagine that you could have two host controllers with a synchronized schedule (not possible without rewriting the kernel USB stack) you still don't know what happens inside the USB device, there is no guarantee that it doesn't introduce jitter too.
USB is not a low latency synchronized interconnect, it is optimized for throughput.
Reliable scheduling over USB *is* possible, but not using bulk and not with too high throughput.
//Peter
Hi Peter, thanks so much for your reply,
On Mon, Nov 11, 2013 at 6:00 AM, Peter Stuge peter@stuge.se wrote:
Michel Pelletier wrote:
I'm expecting that there will be no reliable correlation.
I do know USB and libusb and the key in this case is how bulk transfers are scheduled on the bus - namely optimized for throughput. Bulk transfers fill whatever bus time is free up to the maximum throughput the device can handle.
And even if we imagine that you could have two host controllers with a synchronized schedule (not possible without rewriting the kernel USB stack) you still don't know what happens inside the USB device, there is no guarantee that it doesn't introduce jitter too.
USB is not a low latency synchronized interconnect, it is optimized for throughput.
Reliable scheduling over USB *is* possible, but not using bulk and not with too high throughput.
Rasz has pointed out in a follow up message that two coherently clocked dongles will buffer data internally and the actual arrival time of the data in-process is irrelevant. I think the important point is the event time that the sampling is triggered, and getting the two triggering events to be as closely synchronized as possible. It seems to me that the jitter would be more predictable, and minimized, if it were the case that rtlsdr could setup two bulk transfers in the same process, and wait on those two devices in the same libusb_handle_events_* call.
You have a point in general that you don't know what is going on inside the usb device, and for things like hard drives or network devices that seek or wait on external physical processes (the movement of a head, the arrival of a packet) would have no predictable timing. But it seems to me the rtl dongles have no external physical dependency like that, when you tell them to sample it happens as soon as the dongle receives the command. The data is buffered, and then transferred to the host at some later point.
What I want to achieve is the minimizing of the time between the two triggering events and then measuring that jitter. I'd like to setup the experiment where two coherently clocked dongles both sample from a good timing source, say the GPS L1 frequency, and then compare the two sample blocks against each other to see how they are offset against each other in time and if that offset is reasonably bound across many samples. It seems like I will need to do a little hacking to setup a process capable of this functionality, and will report back with any information I have.
-Michel
Hello Michel,
The problem of correlation between two dongles (or any two independent radio receivers) has nothing to do with the USB transfer. If you have two dongles that run independently and connect them to a common signal source you can simply inject a pulse and see it come out at as sample number xxxx from one dongle and sample number yyyy from the other. Then you can simply provide a buffer that delayes one or the other signal so they appear to be syncronized.
If you send in a second pulse perhaps one minute later you would find that the pulses no longer appear synchronized. Then, you need to insert a fractional re-sampler that changes the sampling rate for one of the channels. (Better to feed both dongles from the same sampling clock.)
Once time is synchronized you would find a strong correlation between the amplitudes on the two signals, but the frequencies and phases will not agree. There will be a strong correlation but only over very short periods of time.
To fix that you have to find the frequency difference and shift the frequency of one or the other signal for them to agree. That is just a complex multiplication and it would lead to longer coherence times in the correlation.
Once you have reached this far the correlation will give precise info on how to adjust the fractional resampler and the frequency shift for the coherence time to become longer (or infinite if you use a singal generator as the signal source.)
The problem is that the fractional resampler will read a little slower from the dongle than the direct read from the other dongle. That means that the delay in the buffer will grow continously reflecting how the two clocks drift in time vs eachother.
- Leif
I've been thought-experimenting with capturing data from two devices and measuring their correlation. For example, if two rtl-sdr processes are launched to read one sample block from two devices, how closely do the two devices' samples line up with each other in time?
I'm expecting that there will be no reliable correlation. My working theory is that each process calls into libusb_handle_events_timeout according to its own process schedule and that there is no way to coordinate these two calls to happen closely enough in time for the transfers to happen in a more synchronized fashion. Threads do not appear to be any better solution to the problem. I'm not an expert in usb transfers, or libusb, or rtlsdr for that matter, so I'm going only on what I can glean from reading the source code of librtlsdr.c and libusb documentation.
libusb does say however that it can specifically handle multiple devices in one async loop. Multiple bulk transfers can be set up to multiple devices in one process, and one call to libusb_handle_events_* can be used to handle those transfers. My thinking is that because the triggering event for both samples is now the same function call in the same process, the samples will be more synchronized, ideally down to some reliably offset of say a few microseconds. Juha Vierinen's hard-hack to make two devices use a coherent sample clock makes this idea even more interesting to me.
But, it doesn't look like librtlsdr can handle multiple devices. I'm game to take a stab at a patch that can extend its functionality. But I wanted to run this idea by the community first before spending code time on figuring this out. Do the experts here think that there is any potential benefit to be gained by taking this approach?
-Michel
USB 2.0 has a built-in hardware latency. If you are unwilling to go through the trouble of special coding, be sure to use the USB Dongles alone on their own PCI USB Card, this will minimize latency.
Hardware may be forced to respond at will if Assembly Code is used to exercise the interfaces, utilizing acquisition calls controlled by processor interrupts. A Real Time Clock (RTC) in internal Hardware or an external RTC can be used to trigger Processor Interrupts initiating any function in the (Assembly) code and attached hardware.
Using a patch will not work with any reasonable accuracy, depending on the Hardware, (an educated guess) of a many microseconds. Patching indicates high level control (high level language like C++, Python, etc), only low level control will call the hardware anywhere near the timing desired. If you are to access two different devices, their data should be both buffered together in parallel through a first-in-first-out buffer, perhaps even with the RTC time stamp to quantize the packets therefore allowing subsequent coherence in post processing. In high speed audio and video interfaces, very high speed is used, like Firewire and Thunderbolt to stream the data into memory, eliminating most of the hardware latency you are experiencing. The maximum acquisition rate for USB 2.0 is about 48,000 samples per second, times 24 bit depth, times 8, and that is for a fast computer doing nothing else but recording the data stream.
From: osmocom-sdr-bounces@lists.osmocom.org [mailto:osmocom-sdr-bounces@lists.osmocom.org] On Behalf Of Michel Pelletier Sent: Sunday, November 10, 2013 11:14 PM To: osmocom-sdr@lists.osmocom.org Subject: multiple devices in one process?
I've been thought-experimenting with capturing data from two devices and measuring their correlation. For example, if two rtl-sdr processes are launched to read one sample block from two devices, how closely do the two devices' samples line up with each other in time?
I'm expecting that there will be no reliable correlation. My working theory is that each process calls into libusb_handle_events_timeout according to its own process schedule and that there is no way to coordinate these two calls to happen closely enough in time for the transfers to happen in a more synchronized fashion. Threads do not appear to be any better solution to the problem. I'm not an expert in usb transfers, or libusb, or rtlsdr for that matter, so I'm going only on what I can glean from reading the source code of librtlsdr.c and libusb documentation.
libusb does say however that it can specifically handle multiple devices in one async loop. Multiple bulk transfers can be set up to multiple devices in one process, and one call to libusb_handle_events_* can be used to handle those transfers. My thinking is that because the triggering event for both samples is now the same function call in the same process, the samples will be more synchronized, ideally down to some reliably offset of say a few microseconds. Juha Vierinen's hard-hack to make two devices use a coherent sample clock makes this idea even more interesting to me.
But, it doesn't look like librtlsdr can handle multiple devices. I'm game to take a stab at a patch that can extend its functionality. But I wanted to run this idea by the community first before spending code time on figuring this out. Do the experts here think that there is any potential benefit to be gained by taking this approach?
-Michel
_____
No virus found in this message. Checked by AVG - www.avg.com Version: 2013.0.3426 / Virus Database: 3222/6830 - Release Date: 11/12/13
On 12/11/2013, Jay Salsburg jsalsburg@bellsouth.net wrote:
USB 2.0 has a built-in hardware latency. If you are unwilling to go through the trouble of special coding, be sure to use the USB Dongles alone on their own PCI USB Card, this will minimize latency.
As Leif said earlier USB latency/jitter is not a problem. Dongles sample into internal buffer. You get this buffer over USB, you are fine as long as it doesnt overflow.
Sync dongle clocks (feed them from one clock source) and there will be no jitter over time.
-
Jay Salsburg -
Leif Asbrink -
Michel Pelletier -
Peter Stuge -
Rasz