7 May
2015
7 May
'15
4:30 a.m.
Hi list
I'm having some trouble with gr-osmosdr and rtl_tcp. I'm trying to set up a
Raspberry Pi as a remote receive to provide I/Q samples over the network to a machine
running gnss-sdr (for realtime GPS processing.) The Pi simply runs "rtl_tcp -f
1575420000 -a 0.0.0.0", and then on gnss-sdr the gr-osmosdr block is created with
osmosdr::source::make("rtl_tcp=<address of pi>");
This connects as expected, but the generated data from rtl_tcp is not being consumed fast
enough; the linked list quickly fills up to 502.
I thought maybe the WiFi was the bottleneck, so I switched to Ethernet. No luck, same
behaviour.
I decided to take it all the way back and plug the dongle straight into the machine that
is running gnss-sdr, and then run rtl_tcp on the same machine: "rtl_tcp -f
1575420000" and change the arguments for the gnuradio block to
?
osmosdr::source::make("rtl_tcp");?
I still get this poor performance. Note that on the same machine, with the GrOsmoSdr block
connecting straight to the dongle via USB (i.e. creating the block with empty arguments)
the performance is fine and I'm able to track GPS satellites in realtime. I understand
that having the network layer in between is going to cause some delay, but I didn't
expect this much.
I thought that maybe the actual processing of gnss-sdr was holding up the reads in some
way, so I tried a different program which post-processes the data; the code which does
this is in the listing below (pulled from the function "frontend_capture" from
https://raw.githubusercontent.com/gnss-sdr/gnss-sdr/next/src/utils/front-en…).
The source block is just a wrapper around an osmosdr::source("rtl_tcp"). The
conditioner is a pass-through block. Unfortunately, the reads *still* aren't keeping
up with the writes from rtl_tcp. I'd like to understand what's causing this. Is it
because of the frequency tuning? Is it the sample rate? I have tried various rates. I
noticed rtl_tcp_source_c::get_sample_rates? returns a few "known to work" rates.
Should I be setting the sample rate to exactly one of these?
gr::top_block_sptr top_block;
GNSSBlockFactory block_factory;
boost::shared_ptr<gr::msg_queue> queue;
queue = gr::msg_queue::make(0);
top_block = gr::make_top_block("Acquisition test");
std::shared_ptr<GNSSBlockInterface> source;
source = block_factory.GetSignalSource(configuration, queue);
std::shared_ptr<GNSSBlockInterface> conditioner =
block_factory.GetSignalConditioner(configuration,queue);
gr::block_sptr sink;
sink = gr::blocks::file_sink::make(sizeof(gr_complex), "tmp_capture.dat");
//--- Find number of samples per spreading code ---
long fs_in_ = configuration->property("GNSS-SDR.internal_fs_hz",
2048000);
int samples_per_code = round(fs_in_
/ (GPS_L1_CA_CODE_RATE_HZ / GPS_L1_CA_CODE_LENGTH_CHIPS));
int nsamples = samples_per_code * 50;
int skip_samples = fs_in_ * 5; // skip 5 seconds
gr::block_sptr head = gr::blocks::head::make(sizeof(gr_complex), nsamples);
gr::block_sptr skiphead = gr::blocks::skiphead::make(sizeof(gr_complex),
skip_samples);
try
{
source->connect(top_block);
conditioner->connect(top_block);
top_block->connect(source->get_right_block(), 0,
conditioner->get_left_block(), 0);
top_block->connect(conditioner->get_right_block(), 0, skiphead, 0);
top_block->connect(skiphead, 0, head, 0);
top_block->connect(head, 0, sink, 0);
top_block->run();
}
catch(std::exception& e)
{
std::cout << "Failure connecting the GNU Radio blocks "
<< e.what() << std::endl;
return false;
}
//delete conditioner;
//delete source;
return true;
Anthony Arnold | Programmer
b: http://anthony-arnold.com/
t: http://twitter.com/anthonyarnold_/
e: anthony.arnold(a)uqconnect.edu.au
8 May
8 May
8:19 p.m.
By the looks of it I'd say the gnss-sdr source is the problem, does the
count keep increasing or does it stop at a certain number?
The explanation for the latter would be that the Application connects,
but does not read, rtl-tcp has no starting or stopping. As soon as a
client connects it tries to pump out samples until the client
disconnects, so buffers are going to pile up if there is a delay between
connecting and reading and the count will stay the same if the
application does not read the data faster than real time to empty the list.
If it keeps increasing the most probable explanation would be a sample
rate mismatch or some other reason why the client does not read fast enough.
The "known to work" rates are just that - values that are known to work
without any side effects, but most other rates should work, too.
Frequent tuning, gain adjustments, or other commands might cause those
commands to pile up, but neither of those would cause the dongle to
produce more samples than anticipated unless your app accidentally sends
out a command to set a higher sample rate...
- Hoernchen
9 May
9 May
2:12 a.m.
The count on rtl_tcp keeps increasing until it reaches the default limit for the
"-n" options (500).
When the gr-osmosdr source block is created, the first thing gnss-sdr does is to set up
the sample rate, frequency, and gain parameters. It then connects the block up to the
flowgraph and the flowgraph is run.
I tried a little experiment by writing my own block for consuming rtl_tcp. It's mostly
the same as the gr-osmosdr one, except that it runs a separate thread for TCP reads. The
producer thread continually reads from the socket, translates the incoming bytes to the
float values via the LUT, then stuffs the floats into a circular buffer. The work function
on the gnuradio thread just pulls data from the circular buffer. This gave a
barely-noticeable improvement.
I did some tracing, and found the the producer thread gets held up; the buffer is almost
constantly full. The hints that the network stack is not the issue, because the producer
thread is reading faster than gnuradio can consume it.
This is all very confusing, because when I'm reading from the USB dongle directly on
the same machine, gnss-sdr handles 1.2Msps in realtime without trouble and no overflows.
It's only when it's reading from the network that it can't keep up, but the
network itself doesn't seem to be the bottleneck.
Is it possible that the deinterleave and/or float_to_complex stages are what's holding
this up?
Anthony Arnold | Programmer
b: http://anthony-arnold.com/
t: http://twitter.com/anthonyarnold_/
e: anthony.arnold(a)uqconnect.edu.au
________________________________________
From: osmocom-sdr <osmocom-sdr-bounces(a)lists.osmocom.org> on behalf of ew
<la(a)tfc-server.de>
Sent: Saturday, 9 May 2015 4:19 AM
To: osmocom-sdr(a)lists.osmocom.org
Subject: Re: Consumption of rtl_tcp too slow?
By the looks of it I'd say the gnss-sdr source is the problem, does the
count keep increasing or does it stop at a certain number?
The explanation for the latter would be that the Application connects,
but does not read, rtl-tcp has no starting or stopping. As soon as a
client connects it tries to pump out samples until the client
disconnects, so buffers are going to pile up if there is a delay between
connecting and reading and the count will stay the same if the
application does not read the data faster than real time to empty the list.
If it keeps increasing the most probable explanation would be a sample
rate mismatch or some other reason why the client does not read fast enough.
The "known to work" rates are just that - values that are known to work
without any side effects, but most other rates should work, too.
Frequent tuning, gain adjustments, or other commands might cause those
commands to pile up, but neither of those would cause the dongle to
produce more samples than anticipated unless your app accidentally sends
out a command to set a higher sample rate...
- Hoernchen
3:39 a.m.
OK, update!
I'm pretty confident that it's the deinterleave stage that's causing the
drama. I ran callgrind and found that around 7% of processing time was spent in the
deinterleave block.
So the rtl_tcp_source_f block is outputting a stream of interleaved floating point values,
and then the rtl_tcp_source_c block is deinterleaving them and then converting each pair
into a complex number.
The problem with this is that the stream is being deinterleaved and then effectively
reinterleaved! The entire deinterleave/float_to_complex process is redundant, because the
original stream is already in the order required.
Anthony Arnold | Programmer
b: http://anthony-arnold.com/
t: http://twitter.com/anthonyarnold_/
e: anthony.arnold(a)uqconnect.edu.au
________________________________________
From: osmocom-sdr <osmocom-sdr-bounces(a)lists.osmocom.org> on behalf of Mr Anthony
Arnold <anthony.arnold(a)uqconnect.edu.au>
Sent: Saturday, 9 May 2015 10:12 AM
To: ew; osmocom-sdr(a)lists.osmocom.org
Subject: Re: Consumption of rtl_tcp too slow?
The count on rtl_tcp keeps increasing until it reaches the default limit for the
"-n" options (500).
When the gr-osmosdr source block is created, the first thing gnss-sdr does is to set up
the sample rate, frequency, and gain parameters. It then connects the block up to the
flowgraph and the flowgraph is run.
I tried a little experiment by writing my own block for consuming rtl_tcp. It's mostly
the same as the gr-osmosdr one, except that it runs a separate thread for TCP reads. The
producer thread continually reads from the socket, translates the incoming bytes to the
float values via the LUT, then stuffs the floats into a circular buffer. The work function
on the gnuradio thread just pulls data from the circular buffer. This gave a
barely-noticeable improvement.
I did some tracing, and found the the producer thread gets held up; the buffer is almost
constantly full. The hints that the network stack is not the issue, because the producer
thread is reading faster than gnuradio can consume it.
This is all very confusing, because when I'm reading from the USB dongle directly on
the same machine, gnss-sdr handles 1.2Msps in realtime without trouble and no overflows.
It's only when it's reading from the network that it can't keep up, but the
network itself doesn't seem to be the bottleneck.
Is it possible that the deinterleave and/or float_to_complex stages are what's holding
this up?
Anthony Arnold | Programmer
b: http://anthony-arnold.com/
t: http://twitter.com/anthonyarnold_/
e: anthony.arnold(a)uqconnect.edu.au
________________________________________
From: osmocom-sdr <osmocom-sdr-bounces(a)lists.osmocom.org> on behalf of ew
<la(a)tfc-server.de>
Sent: Saturday, 9 May 2015 4:19 AM
To: osmocom-sdr(a)lists.osmocom.org
Subject: Re: Consumption of rtl_tcp too slow?
By the looks of it I'd say the gnss-sdr source is the problem, does the
count keep increasing or does it stop at a certain number?
The explanation for the latter would be that the Application connects,
but does not read, rtl-tcp has no starting or stopping. As soon as a
client connects it tries to pump out samples until the client
disconnects, so buffers are going to pile up if there is a delay between
connecting and reading and the count will stay the same if the
application does not read the data faster than real time to empty the list.
If it keeps increasing the most probable explanation would be a sample
rate mismatch or some other reason why the client does not read fast enough.
The "known to work" rates are just that - values that are known to work
without any side effects, but most other rates should work, too.
Frequent tuning, gain adjustments, or other commands might cause those
commands to pile up, but neither of those would cause the dongle to
produce more samples than anticipated unless your app accidentally sends
out a command to set a higher sample rate...
- Hoernchen
1:36 p.m.
Hello,
i have same problem with gr-osmosdr when using gqrx and rtl_tcp over ethernet.
It eats too much cpu on client side with gnuradio 3.7.5.1 and it is
unusable.
I tried to patch it and remove that deinterleave/reinterleave code. It
works for me. But sometimes when i start dsp in gqrx too late it looks
little laggy. I must restart gqrx and start dsp asap, probably some
problem with buffering.
Anyway, you can test my gr-osmosdr, if it works for you i can send
it upstream probably.
https://github.com/stanojr/gr-osmosdr/tree/rtl_tcp_refactor
On Sat, 9 May 2015 01:39:01 +0000 Mr Anthony Arnold
<anthony.arnold(a)uqconnect.edu.au> wrote:
OK, update!
I'm pretty confident that it's the deinterleave stage that's causing
the drama. I ran callgrind and found that around 7% of processing
time was spent in the deinterleave block.
So the rtl_tcp_source_f block is outputting a stream of interleaved
floating point values, and then the rtl_tcp_source_c block is
deinterleaving them and then converting each pair into a complex
number.
The problem with this is that the stream is being deinterleaved and
then effectively reinterleaved! The entire
deinterleave/float_to_complex process is redundant, because the
original stream is already in the order required.
Anthony Arnold | Programmer
b: http://anthony-arnold.com/
t: http://twitter.com/anthonyarnold_/
e: anthony.arnold(a)uqconnect.edu.au
________________________________________
From: osmocom-sdr <osmocom-sdr-bounces(a)lists.osmocom.org> on behalf
of Mr Anthony Arnold <anthony.arnold(a)uqconnect.edu.au> Sent:
Saturday, 9 May 2015 10:12 AM To: ew; osmocom-sdr(a)lists.osmocom.org
Subject: Re: Consumption of rtl_tcp too slow?
The count on rtl_tcp keeps increasing until it reaches the default
limit for the "-n" options (500).
When the gr-osmosdr source block is created, the first thing gnss-sdr
does is to set up the sample rate, frequency, and gain parameters. It
then connects the block up to the flowgraph and the flowgraph is run.
I tried a little experiment by writing my own block for consuming
rtl_tcp. It's mostly the same as the gr-osmosdr one, except that it
runs a separate thread for TCP reads. The producer thread continually
reads from the socket, translates the incoming bytes to the float
values via the LUT, then stuffs the floats into a circular buffer.
The work function on the gnuradio thread just pulls data from the
circular buffer. This gave a barely-noticeable improvement.
I did some tracing, and found the the producer thread gets held up;
the buffer is almost constantly full. The hints that the network
stack is not the issue, because the producer thread is reading faster
than gnuradio can consume it.
This is all very confusing, because when I'm reading from the USB
dongle directly on the same machine, gnss-sdr handles 1.2Msps in
realtime without trouble and no overflows. It's only when it's
reading from the network that it can't keep up, but the network
itself doesn't seem to be the bottleneck.
Is it possible that the deinterleave and/or float_to_complex stages
are what's holding this up?
Anthony Arnold | Programmer
b: http://anthony-arnold.com/
t: http://twitter.com/anthonyarnold_/
e: anthony.arnold(a)uqconnect.edu.au
________________________________________
From: osmocom-sdr <osmocom-sdr-bounces(a)lists.osmocom.org> on behalf
of ew <la(a)tfc-server.de> Sent: Saturday, 9 May 2015 4:19 AM
To: osmocom-sdr(a)lists.osmocom.org
Subject: Re: Consumption of rtl_tcp too slow?
By the looks of it I'd say the gnss-sdr source is the problem, does
the count keep increasing or does it stop at a certain number?
The explanation for the latter would be that the Application connects,
but does not read, rtl-tcp has no starting or stopping. As soon as a
client connects it tries to pump out samples until the client
disconnects, so buffers are going to pile up if there is a delay
between connecting and reading and the count will stay the same if the
application does not read the data faster than real time to empty the
list. If it keeps increasing the most probable explanation would be a
sample rate mismatch or some other reason why the client does not
read fast enough.
The "known to work" rates are just that - values that are known to
work without any side effects, but most other rates should work, too.
Frequent tuning, gain adjustments, or other commands might cause
those commands to pile up, but neither of those would cause the
dongle to produce more samples than anticipated unless your app
accidentally sends out a command to set a higher sample rate...
- Hoernchen
--
[ Ohodnotte kvalitu mailu:
https://www.nicereply.com/websupport/4afafd34 ]
Pavel Stano | Troubleshooter
http://WebSupport.sk
*** BERTE A VYCHUTNAVAJTE ***
3298
days inactive
3300
days old
4 comments
3 participants
participants (3)
-
ew -
Mr Anthony Arnold -
Pavel Stano