Hi,
I encountered this strange behavior when running fl2k_test on Windows 7:
basically after a while the sample rate drifts and speeds up considerably.
Sometimes it even starts up directly like that.
In the command prompt I get:
C:\Users\Francesco
Gugliuzza\Downloads\Sviluppo\FL2000fl2k_win_2018-08-09>fl2k_test.exe -s
138e6
Reporting PPM error measurement every 10 seconds...
Press ^C after a few minutes.
real sample rate: 146656386 current PPM: 62727 cumulative PPM: 62727
real sample rate: 150803676 current PPM: 92780 cumulative PPM: 77858
real sample rate: 149716182 current PPM: 84900 cumulative PPM: 80218
real sample rate: 150107883 current PPM: 87738 cumulative PPM: 82104
real sample rate: 150526750 current PPM: 90774 cumulative PPM: 83843
The same thing happens if I compile the latest osmo-fl2k myself using
Cygwin.
I'm running everything on a laptop with an i7 4710HQ CPU and 8 GB of ram,
and I'm using a RayCue HDMI+VGA box.
Do you have an idea why the real sample rate drifts so much?
Thanks!
Francesco
--
Francesco Gugliuzza
M.Sc. in Computer Engineering
Ph.D. Student at DIID, Università degli Studi di Palermo
Qualified to practice as a Professional Engineer
E-mail: fgugliuzza.mail(a)gmail.com
PEC: francesco.gugliuzza(a)pec.it
Reproducible segfault apparently from race condition on my system in librtlsdr.
---
$ uname -r
4.14.67-1.pvops.qubes.x86_64
rtl-sdr and libusb are both current git master.
---
1. run librtlsdr under gdb
$ gdb --args rtl_sdr -f 43M -n 1000 /dev/null
2. place a breakpoint when the cancellation condition is hit,
currently line 1915
1914 if (RTLSDR_CANCELING == dev->async_status) {
1915
next_status = RTLSDR_INACTIVE;
1916
(gdb) break ./src/librtlsdr.c:1915
3. run and continue; segfault after second breakpoint
(gdb) run
(gdb) cont
(gdb) cont
Thread 1 "rtl_sdr" received signal SIGSEGV, Segmentation fault.
0x00007ffff7d76a15 in add_to_flying_list (transfer=0x426590) at
../../libusb/io.c:1396
1396 if (!timerisset(cur_tv) || (cur_tv->tv_sec >
timeout->tv_sec) ||
--
The crash appears to be because the transfers are deallocated while
they are still in flight, and then later referenced by libusb.
I think this happens because the pause gives the transfer time to
complete before it is canceled. The cancel then fails because the
transfer was completed, and the current code assumes this means it is
not in flight, when it actually hasn't been handled yet and will be
resubmitted.
The documentation for libusb_transfer::status at
http://libusb.sourceforge.net/api-1.0/structlibusb__transfer.html#a64b2e70e…
states that it is only correct to read the field from within the
callback handler, and the documentation for libusb_cancel_transfer at
http://libusb.sourceforge.net/api-1.0/group__asyncio.html#ga685eb7731f9a059…
states that the transfer
cancellation is only complete when the callback handler is called with
such status.
Although I imagine there are simpler solutions, I think the correct
solution would be to move cancellation of the transfers into the
callback handler entirely, to eliminate race conditions like this and
respect the libusb documentation. I would enjoy crafting a patch to
make such a change, if that would be helpful.
Karl Semich
1. _running is guarded with a lock in the rtl event loop epilogue.
this prevents a possible hang that could arise from _buf_cond
being notified prior to work() waiting on _buf_cond, but after
work() checks _running.
2. _buf_used is guarded with a lock in the work function to prevent
it from spuriously reading zero if it is nonatomically written
from the rtl event callback.
---
lib/rtl/rtl_source_c.cc | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/rtl/rtl_source_c.cc b/lib/rtl/rtl_source_c.cc
index a371464..bbd0140 100644
--- a/lib/rtl/rtl_source_c.cc
+++ b/lib/rtl/rtl_source_c.cc
@@ -323,7 +323,11 @@ void rtl_source_c::rtlsdr_wait()
{
int ret = rtlsdr_read_async( _dev, _rtlsdr_callback, (void *)this, _buf_num, _buf_len );
- _running = false;
+ {
+ boost::mutex::scoped_lock lock( _buf_mutex );
+
+ _running = false;
+ }
if ( ret != 0 )
std::cerr << "rtlsdr_read_async returned with " << ret << std::endl;
@@ -347,7 +351,7 @@ int rtl_source_c::work( int noutput_items,
if (!_running)
return WORK_DONE;
- while (noutput_items && _buf_used) {
+ while (noutput_items) {
const int nout = std::min(noutput_items, _samp_avail);
const unsigned char *buf = _buf[_buf_head] + _buf_offset * 2;
@@ -363,6 +367,9 @@ int rtl_source_c::work( int noutput_items,
_buf_head = (_buf_head + 1) % _buf_num;
_buf_used--;
+
+ if (_buf_used == 0)
+ break;
}
_samp_avail = _buf_len / BYTES_PER_SAMPLE;
_buf_offset = 0;
--
2.11.0
Small change to work function to finish draining buffers
before closing when device disappears. Provides a little
more data when device is unexpectedly lost.
---
lib/rtl/rtl_source_c.cc | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/lib/rtl/rtl_source_c.cc b/lib/rtl/rtl_source_c.cc
index a371464..c0ba0b3 100644
--- a/lib/rtl/rtl_source_c.cc
+++ b/lib/rtl/rtl_source_c.cc
@@ -340,13 +340,18 @@ int rtl_source_c::work( int noutput_items,
{
boost::mutex::scoped_lock lock( _buf_mutex );
- while (_buf_used < 3 && _running) // collect at least 3 buffers
+ while (_buf_used < 3) // collect at least 3 buffers
+ {
+ if (!_running) {
+ // finish when remaining samples are drained
+ if (!_buf_used)
+ return WORK_DONE;
+ break;
+ }
_buf_cond.wait( lock );
+ }
}
- if (!_running)
- return WORK_DONE;
-
while (noutput_items && _buf_used) {
const int nout = std::min(noutput_items, _samp_avail);
const unsigned char *buf = _buf[_buf_head] + _buf_offset * 2;
--
2.11.0
From: Karl Semich <0xloem(a)gmail.com>
Previously overflow was handled by outputting "O" on stderr and
placing the extra data at the start of the buffer queue.
This had a number of issues:
- as only 1 buffer advanced on overflow, many overflows could occur in
sequence, as only small amounts of data was allowed to drain
- if the work function was partway through reading the head of the
queue, it would continue reading at the same offset from the new
head rather than from the start, causing extra loss of data
- buffer processing in work function is not guarded with a lock,
so it was possible for overflow to overwrite data while it was
being copied, resulting in garbage passed on
This patch rewrites overflow handling such that streaming is paused
until the work function can catch up. This resolves all 3 issues.
---
lib/rtl/rtl_source_c.cc | 32 ++++++++++++++++++++++++++------
lib/rtl/rtl_source_c.h | 2 ++
2 files changed, 28 insertions(+), 6 deletions(-)
diff --git a/lib/rtl/rtl_source_c.cc b/lib/rtl/rtl_source_c.cc
index a371464..0d26831 100644
--- a/lib/rtl/rtl_source_c.cc
+++ b/lib/rtl/rtl_source_c.cc
@@ -48,6 +48,7 @@ using namespace boost::assign;
#define BUF_LEN (16 * 32 * 512) /* must be multiple of 512 */
#define BUF_NUM 15
#define BUF_SKIP 1 // buffers to skip due to initial garbage
+#define BUF_MIN 3 /* minimum buffers to run work function */
#define BYTES_PER_SAMPLE 2 // rtl device delivers 8 bit unsigned IQ data
@@ -87,8 +88,7 @@ rtl_source_c::rtl_source_c (const std::string &args)
_running(false),
_no_tuner(false),
_auto_gain(false),
- _if_gain(0),
- _skipped(0)
+ _if_gain(0)
{
int ret;
int index;
@@ -297,6 +297,9 @@ void rtl_source_c::rtlsdr_callback(unsigned char *buf, uint32_t len)
return;
}
+ if (_overflow)
+ return;
+
{
boost::mutex::scoped_lock lock( _buf_mutex );
@@ -304,8 +307,9 @@ void rtl_source_c::rtlsdr_callback(unsigned char *buf, uint32_t len)
memcpy(_buf[buf_tail], buf, len);
if (_buf_used == _buf_num) {
- std::cerr << "O" << std::flush;
- _buf_head = (_buf_head + 1) % _buf_num;
+ std::cerr << "OVERFLOW: rtl-sdr stream restarting after draining unread buffers" << std::endl;
+ _overflow = true;
+ rtlsdr_cancel_async( _dev );
} else {
_buf_used++;
}
@@ -321,7 +325,20 @@ void rtl_source_c::_rtlsdr_wait(rtl_source_c *obj)
void rtl_source_c::rtlsdr_wait()
{
- int ret = rtlsdr_read_async( _dev, _rtlsdr_callback, (void *)this, _buf_num, _buf_len );
+ int ret;
+
+ do {
+ {
+ boost::mutex::scoped_lock lock( _buf_mutex );
+ // let unread buffers from last run drain
+ while ( _buf_used >= BUF_MIN )
+ _work_cond.wait( lock );
+ }
+
+ _overflow = false;
+ _skipped = 0;
+ ret = rtlsdr_read_async( _dev, _rtlsdr_callback, (void *)this, _buf_num, _buf_len );
+ } while ( _overflow );
_running = false;
@@ -340,7 +357,7 @@ int rtl_source_c::work( int noutput_items,
{
boost::mutex::scoped_lock lock( _buf_mutex );
- while (_buf_used < 3 && _running) // collect at least 3 buffers
+ while (_buf_used < BUF_MIN && _running) // collect at least BUF_MIN buffers
_buf_cond.wait( lock );
}
@@ -364,6 +381,9 @@ int rtl_source_c::work( int noutput_items,
_buf_head = (_buf_head + 1) % _buf_num;
_buf_used--;
}
+
+ _work_cond.notify_one();
+
_samp_avail = _buf_len / BYTES_PER_SAMPLE;
_buf_offset = 0;
} else {
diff --git a/lib/rtl/rtl_source_c.h b/lib/rtl/rtl_source_c.h
index 902b386..aafea5f 100644
--- a/lib/rtl/rtl_source_c.h
+++ b/lib/rtl/rtl_source_c.h
@@ -133,6 +133,8 @@ private:
unsigned int _buf_used;
boost::mutex _buf_mutex;
boost::condition_variable _buf_cond;
+ boost::condition_variable _work_cond;
+ bool _overflow;
bool _running;
unsigned int _buf_offset;
--
2.11.0
Small API added to allow sample buffers to be used after the
lifetime of the callback, and manually returned to librtlsdr.
Allows multithreaded apps to track USB buffer buildup without
requiring memcpy() be called in the callback.
This update includes a fix to handle unsubmitted buffers on
cancel.
---
include/rtl-sdr.h | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
src/librtlsdr.c | 46 +++++++++++++++++++++++++++++++++++++++++++---
2 files changed, 93 insertions(+), 3 deletions(-)
diff --git a/include/rtl-sdr.h b/include/rtl-sdr.h
index 3ed13ae..1ef508d 100644
--- a/include/rtl-sdr.h
+++ b/include/rtl-sdr.h
@@ -331,6 +331,7 @@ RTLSDR_API int rtlsdr_set_offset_tuning(rtlsdr_dev_t *dev, int on);
*/
RTLSDR_API int rtlsdr_get_offset_tuning(rtlsdr_dev_t *dev);
+
/* streaming functions */
RTLSDR_API int rtlsdr_reset_buffer(rtlsdr_dev_t *dev);
@@ -381,6 +382,55 @@ RTLSDR_API int rtlsdr_read_async(rtlsdr_dev_t *dev,
RTLSDR_API int rtlsdr_cancel_async(rtlsdr_dev_t *dev);
/*!
+ * Enable or disable automatic resubmission of async transfer buffers.
+ * By default this is enabled, but if disabled then buffers will not
+ * be refilled with data until passed to rtlsdr_release_manual().
+ *
+ * This allows applications to track buildup of streaming buffers
+ * by returning from the callback quickly, without requiring an
+ * extra memcpy to take the data out of the callback for processing.
+ *
+ * \param dev the device handle given by rtlsdr_open()
+ * \param on 0 means disabled, 1 enabled
+ * \return 0 on success
+ */
+RTLSDR_API int rtlsdr_set_automatic_release(rtlsdr_dev_t *dev, int on);
+
+/*!
+* When automatic releases are disabled, release a long-lived buffer that
+* was passed to an async callback. Such buffers will not be reused until
+* released.
+*
+* There must be enough released buffers submitted to libusb at all times
+* to accommodate driver and hardware communication latency. The total
+* number of buffers filling this role, and the total number of buffers
+* unreleased, can be calculated:
+*
+* buf_num = value passed to rtlsdr_read_async at stream start
+* callback_calls = number of times a buf has been passed to callback
+* buf_releases = number of successful calls to rtlsdr_release_manual()
+*
+* bufs_reserved = callback_calls - buf_releases
+* bufs_in_flight = buf_num - bufs_reserved
+*
+* Note that this calculation assumes that all reserved buffers have been
+* passed to the callback already. It is only useful if the callback
+* returns very quickly every time. If the callback waits, buffers
+* may accumulate in the waiting time, and only be found when it returns.
+*
+* Once bufs_in_flight falls too low, the integrity of the stream is
+* compromised as the device rewrites over its full buffer. The cutoff
+* for this depends on latency in the operating system and hardware.
+*
+* Once bufs_in_flight hits zero, streaming will stop completely.
+*
+* \param dev the device handle given by rtlsdr_open()
+* \param buf a pointer to the start of the buffer
+* \return 0 on success
+*/
+RTLSDR_API int rtlsdr_release_manual(rtlsdr_dev_t *dev, unsigned char *buf);
+
+/*!
* Enable or disable the bias tee on GPIO PIN 0.
*
* \param dev the device handle given by rtlsdr_open()
diff --git a/src/librtlsdr.c b/src/librtlsdr.c
index 433ed5b..e093ed7 100644
--- a/src/librtlsdr.c
+++ b/src/librtlsdr.c
@@ -103,6 +103,7 @@ struct rtlsdr_dev {
void *cb_ctx;
enum rtlsdr_async_status async_status;
int async_cancel;
+ int manual_release;
int use_zerocopy;
/* rtl demod context */
uint32_t rate; /* Hz */
@@ -1702,7 +1703,8 @@ static void LIBUSB_CALL _libusb_callback(struct libusb_transfer *xfer)
if (dev->cb)
dev->cb(xfer->buffer, xfer->actual_length, dev->cb_ctx);
- libusb_submit_transfer(xfer); /* resubmit transfer */
+ if (!dev->manual_release)
+ libusb_submit_transfer(xfer); /* resubmit transfer */
dev->xfer_errors = 0;
} else if (LIBUSB_TRANSFER_CANCELLED != xfer->status) {
#ifndef _WIN32
@@ -1753,7 +1755,8 @@ static int _rtlsdr_alloc_async_buffers(rtlsdr_dev_t *dev)
dev->use_zerocopy = 1;
for (i = 0; i < dev->xfer_buf_num; ++i) {
- dev->xfer_buf[i] = libusb_dev_mem_alloc(dev->devh, dev->xfer_buf_len);
+ dev->xfer_buf[i] = libusb_dev_mem_alloc(dev->devh, dev->xfer_buf_len
+ + sizeof(struct libusb_transfer *));
if (!dev->xfer_buf[i]) {
fprintf(stderr, "Failed to allocate zero-copy "
@@ -1780,7 +1783,8 @@ static int _rtlsdr_alloc_async_buffers(rtlsdr_dev_t *dev)
/* no zero-copy available, allocate buffers in userspace */
if (!dev->use_zerocopy) {
for (i = 0; i < dev->xfer_buf_num; ++i) {
- dev->xfer_buf[i] = malloc(dev->xfer_buf_len);
+ dev->xfer_buf[i] = malloc(dev->xfer_buf_len +
+ sizeof(struct libusb_transfer *));
if (!dev->xfer_buf[i])
return -ENOMEM;
@@ -1873,6 +1877,9 @@ int rtlsdr_read_async(rtlsdr_dev_t *dev, rtlsdr_read_async_cb_t cb, void *ctx,
(void *)dev,
BULK_TIMEOUT);
+ *(struct libusb_transfer **)(dev->xfer_buf[i] + dev->xfer_buf_len) =
+ dev->xfer[i];
+
r = libusb_submit_transfer(dev->xfer[i]);
if (r < 0) {
fprintf(stderr, "Failed to submit transfer %i\n"
@@ -1909,6 +1916,14 @@ int rtlsdr_read_async(rtlsdr_dev_t *dev, rtlsdr_read_async_cb_t cb, void *ctx,
if (LIBUSB_TRANSFER_CANCELLED !=
dev->xfer[i]->status) {
r = libusb_cancel_transfer(dev->xfer[i]);
+ if (r == LIBUSB_ERROR_NOT_FOUND) {
+ /* transfer was not in flight */
+ r = 0;
+ dev->xfer[i]->status =
+ LIBUSB_TRANSFER_CANCELLED;
+ continue;
+ }
+
/* handle events after canceling
* to allow transfer status to
* propagate */
@@ -1961,6 +1976,31 @@ int rtlsdr_cancel_async(rtlsdr_dev_t *dev)
return -2;
}
+int rtlsdr_set_automatic_release(rtlsdr_dev_t *dev, int on)
+{
+ if (!dev)
+ return -1;
+
+ if (RTLSDR_INACTIVE != dev->async_status)
+ return -2;
+
+ dev->manual_release = !on;
+
+ return 0;
+}
+
+int rtlsdr_release_manual(rtlsdr_dev_t *dev, unsigned char *buf)
+{
+ struct libusb_transfer * xfer;
+
+ if (!dev)
+ return -1;
+
+ xfer = *(struct libusb_transfer **)(buf + dev->xfer_buf_len);
+
+ return libusb_submit_transfer(xfer);
+}
+
uint32_t rtlsdr_get_tuner_clock(void *dev)
{
uint32_t tuner_freq;
--
2.11.0
New rtl-sdr manual buffer resubmission patch after discussion on IRC.
This implementation uses the existing rtlsdr_read_async API and is
more compartmentalized.