<p>Piotr Krysik <strong>merged</strong> this change.</p><p><a href="https://gerrit.osmocom.org/10881">View Change</a></p><div style="white-space:pre-wrap">Approvals:
Piotr Krysik: Looks good to me, approved; Verified
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">burst_file_source: Fix reading longer bursts<br><br>- Read bursts with pmt::deserialize directly from the std::filebuf<br>- Remove the unused unserialized variable<br>- Add tests<br><br>Since df978693 when the rx_time tags are present in the incomming stream<br>the gsm receiver adds fm_time to the burst's PMT and the bursts that<br>burst file sink writes becomes longer because of the additional field.<br><br>The burst file source block was expecting all burst to be 147 bytes long<br>and reading files with longer bursts was failing with an unhandled exception.<br><br>terminate called after throwing an instance of 'pmt::exception'<br>thread[thread-per-block[5]: <block dummy_burst_filter (2)>]: pmt_cdr: wrong_type : #f<br> what(): pmt::deserialize: malformed input stream, tag value = : 115<br><br>Change-Id: I989b0d6a6b214088b7880e5cbf7bb6725492dbfc<br>---<br>M lib/misc_utils/burst_file_source_impl.cc<br>M lib/qa_utils/burst_source_impl.cc<br>M python/CMakeLists.txt<br>A python/qa_burst_file_source.py<br>4 files changed, 109 insertions(+), 13 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/lib/misc_utils/burst_file_source_impl.cc b/lib/misc_utils/burst_file_source_impl.cc</span><br><span>index 4367150..85d5ad9 100644</span><br><span>--- a/lib/misc_utils/burst_file_source_impl.cc</span><br><span>+++ b/lib/misc_utils/burst_file_source_impl.cc</span><br><span>@@ -28,8 +28,6 @@</span><br><span> #include "burst_file_source_impl.h"</span><br><span> #include "stdio.h"</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-#define PMT_SIZE 174</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> namespace gr {</span><br><span> namespace gsm {</span><br><span> </span><br><span>@@ -86,16 +84,13 @@</span><br><span> </span><br><span> void burst_file_source_impl::run()</span><br><span> {</span><br><span style="color: hsl(0, 100%, 40%);">- char *unserialized = (char*)malloc(sizeof(char) * PMT_SIZE);</span><br><span style="color: hsl(0, 100%, 40%);">- while (d_input_file.read(unserialized, PMT_SIZE) && !d_finished)</span><br><span style="color: hsl(120, 100%, 40%);">+ std::filebuf* pbuf = d_input_file.rdbuf();</span><br><span style="color: hsl(120, 100%, 40%);">+ while (!d_finished)</span><br><span> {</span><br><span style="color: hsl(0, 100%, 40%);">- if (d_input_file.bad())</span><br><span style="color: hsl(0, 100%, 40%);">- {</span><br><span style="color: hsl(120, 100%, 40%);">+ pmt::pmt_t burst = pmt::deserialize(*pbuf);</span><br><span style="color: hsl(120, 100%, 40%);">+ if (pmt::is_eof_object(burst)) {</span><br><span> break;</span><br><span> }</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- std::string s(unserialized, PMT_SIZE);</span><br><span style="color: hsl(0, 100%, 40%);">- pmt::pmt_t burst = pmt::deserialize_str(s);</span><br><span> message_port_pub(pmt::mp("out"), burst);</span><br><span> }</span><br><span> d_input_file.close();</span><br><span>diff --git a/lib/qa_utils/burst_source_impl.cc b/lib/qa_utils/burst_source_impl.cc</span><br><span>index db3d85a..f415eaf 100644</span><br><span>--- a/lib/qa_utils/burst_source_impl.cc</span><br><span>+++ b/lib/qa_utils/burst_source_impl.cc</span><br><span>@@ -31,8 +31,6 @@</span><br><span> #include <grgsm/gsmtap.h></span><br><span> #include <grgsm/endian.h></span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-#define PMT_SIZE 174</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> namespace gr {</span><br><span> namespace gsm {</span><br><span> </span><br><span>@@ -110,8 +108,6 @@</span><br><span> </span><br><span> void burst_source_impl::run()</span><br><span> {</span><br><span style="color: hsl(0, 100%, 40%);">- char *unserialized = (char*)malloc(sizeof(char) * PMT_SIZE);</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> for (int i=0; i<d_burst_data.size(); i++)</span><br><span> {</span><br><span> if (d_burst_data[i].length() == BURST_SIZE &&</span><br><span>diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt</span><br><span>index 6f49e71..418605e 100644</span><br><span>--- a/python/CMakeLists.txt</span><br><span>+++ b/python/CMakeLists.txt</span><br><span>@@ -50,6 +50,7 @@</span><br><span> GR_ADD_TEST(qa_decryption ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_decryption.py)</span><br><span> GR_ADD_TEST(qa_burst_printer ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_burst_printer.py)</span><br><span> GR_ADD_TEST(qa_message_printer ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_message_printer.py)</span><br><span style="color: hsl(120, 100%, 40%);">+GR_ADD_TEST(qa_burst_file_source ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_burst_file_source.py)</span><br><span> GR_ADD_TEST(qa_burst_timeslot_splitter ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_burst_timeslot_splitter.py)</span><br><span> GR_ADD_TEST(qa_burst_sdcch_subslot_splitter ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_burst_sdcch_subslot_splitter.py)</span><br><span> GR_ADD_TEST(qa_burst_timeslot_filter ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/qa_burst_timeslot_filter.py)</span><br><span>diff --git a/python/qa_burst_file_source.py b/python/qa_burst_file_source.py</span><br><span>new file mode 100644</span><br><span>index 0000000..cf8835e</span><br><span>--- /dev/null</span><br><span>+++ b/python/qa_burst_file_source.py</span><br><span>@@ -0,0 +1,104 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+# -*- coding: utf-8 -*-</span><br><span style="color: hsl(120, 100%, 40%);">+# @file</span><br><span style="color: hsl(120, 100%, 40%);">+# @author (C) 2018 by Vasil Velichkov <vvvelichkov@gmail.com></span><br><span style="color: hsl(120, 100%, 40%);">+# @section LICENSE</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Gr-gsm is free software; you can redistribute it and/or modify</span><br><span style="color: hsl(120, 100%, 40%);">+# it under the terms of the GNU General Public License as published by</span><br><span style="color: hsl(120, 100%, 40%);">+# the Free Software Foundation; either version 3, or (at your option)</span><br><span style="color: hsl(120, 100%, 40%);">+# any later version.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# Gr-gsm is distributed in the hope that it will be useful,</span><br><span style="color: hsl(120, 100%, 40%);">+# but WITHOUT ANY WARRANTY; without even the implied warranty of</span><br><span style="color: hsl(120, 100%, 40%);">+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span><br><span style="color: hsl(120, 100%, 40%);">+# GNU General Public License for more details.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+# You should have received a copy of the GNU General Public License</span><br><span style="color: hsl(120, 100%, 40%);">+# along with gr-gsm; see the file COPYING. If not, write to</span><br><span style="color: hsl(120, 100%, 40%);">+# the Free Software Foundation, Inc., 51 Franklin Street,</span><br><span style="color: hsl(120, 100%, 40%);">+# Boston, MA 02110-1301, USA.</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+#</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from gnuradio import gr, gr_unittest, blocks</span><br><span style="color: hsl(120, 100%, 40%);">+import grgsm_swig as grgsm</span><br><span style="color: hsl(120, 100%, 40%);">+import tempfile</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+class qa_burst_file_sink (gr_unittest.TestCase):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def setUp (self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tb = gr.top_block ()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def tearDown (self):</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tb = None</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def test_blob_only (self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # prepare the input burst file</span><br><span style="color: hsl(120, 100%, 40%);">+ temp = tempfile.NamedTemporaryFile()</span><br><span style="color: hsl(120, 100%, 40%);">+ handle = open(temp.name, "wb")</span><br><span style="color: hsl(120, 100%, 40%);">+ handle.write(bytearray([</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x07, 0x06, 0x0a, 0x00, 0x00, 0x00, 0x00, 0xa4, 0x01, 0x00, 0x02, 0x04, 0x03, 0x01, 0x00, 0x6f,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0xcd, 0x00, 0x00, 0x25, 0xc9, 0x82, 0x06, 0x1c, 0xf5, 0x38, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x01, 0x01, 0x01, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x01, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x01, 0x00, 0x01, 0x01, 0x00, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x00, 0x01, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x01, 0x00, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x01, 0x01, 0x01, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x01</span><br><span style="color: hsl(120, 100%, 40%);">+ ]))</span><br><span style="color: hsl(120, 100%, 40%);">+ handle.flush();</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ src = grgsm.burst_file_source(temp.name);</span><br><span style="color: hsl(120, 100%, 40%);">+ dst = grgsm.burst_sink();</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tb.msg_connect(src, "out", dst, "in")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tb.run ()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.assertEqual([2476418], list(dst.get_framenumbers()))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.assertEqual([1], list(dst.get_timeslots()))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.assertEqual([</span><br><span style="color: hsl(120, 100%, 40%);">+ "0000001000000101010111111110101000000000010101010111101010101001011011101111000101101111100000000001010101111010101010000000010101011101111010101001"],</span><br><span style="color: hsl(120, 100%, 40%);">+ list(dst.get_burst_data()))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ def test_fn_time (self):</span><br><span style="color: hsl(120, 100%, 40%);">+ # prepare the input burst file</span><br><span style="color: hsl(120, 100%, 40%);">+ temp = tempfile.NamedTemporaryFile()</span><br><span style="color: hsl(120, 100%, 40%);">+ handle = open(temp.name, "wb")</span><br><span style="color: hsl(120, 100%, 40%);">+ handle.write(bytearray([</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x07,</span><br><span style="color: hsl(120, 100%, 40%);">+ # the additional fn_time field - 51 bytes</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x07, 0x07, 0x02, 0x00, 0x07, 0x66, 0x6e, 0x5f, 0x74, 0x69, 0x6d, 0x65, 0x07, 0x07, 0x0b,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x0a, 0x3b, 0x73, 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x07, 0x0b, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x3f, 0xe4, 0x99, 0x45,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0xbe, 0x81, 0xc0, 0xf4,</span><br><span style="color: hsl(120, 100%, 40%);">+ # the 173 original 173 bytes</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x06, 0x0a, 0x00, 0x00, 0x00, 0x00, 0xa4, 0x01, 0x00, 0x02, 0x04, 0x03, 0x01, 0x00, 0x6f,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0xcd, 0x00, 0x00, 0x25, 0xc9, 0x82, 0x06, 0x1c, 0xf5, 0x38, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x01, 0x01, 0x01, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x01, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x01, 0x00, 0x01, 0x01, 0x00, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x00, 0x01, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x01, 0x00, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x01, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01,</span><br><span style="color: hsl(120, 100%, 40%);">+ 0x00, 0x01, 0x01, 0x01, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x01</span><br><span style="color: hsl(120, 100%, 40%);">+ ]))</span><br><span style="color: hsl(120, 100%, 40%);">+ handle.flush();</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ src = grgsm.burst_file_source(temp.name);</span><br><span style="color: hsl(120, 100%, 40%);">+ dst = grgsm.burst_sink();</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tb.msg_connect(src, "out", dst, "in")</span><br><span style="color: hsl(120, 100%, 40%);">+ self.tb.run ()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ self.assertEqual([2476418], list(dst.get_framenumbers()))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.assertEqual([1], list(dst.get_timeslots()))</span><br><span style="color: hsl(120, 100%, 40%);">+ self.assertEqual([</span><br><span style="color: hsl(120, 100%, 40%);">+ "0000001000000101010111111110101000000000010101010111101010101001011011101111000101101111100000000001010101111010101010000000010101011101111010101001"],</span><br><span style="color: hsl(120, 100%, 40%);">+ list(dst.get_burst_data()))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if __name__ == '__main__':</span><br><span style="color: hsl(120, 100%, 40%);">+ gr_unittest.run(qa_burst_file_sink, "qa_burst_file_sink.xml")</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/10881">change 10881</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/10881"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: gr-gsm </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: merged </div>
<div style="display:none"> Gerrit-Change-Id: I989b0d6a6b214088b7880e5cbf7bb6725492dbfc </div>
<div style="display:none"> Gerrit-Change-Number: 10881 </div>
<div style="display:none"> Gerrit-PatchSet: 3 </div>
<div style="display:none"> Gerrit-Owner: Vasil Velichkov <vvvelichkov@gmail.com> </div>
<div style="display:none"> Gerrit-Reviewer: Piotr Krysik <ptrkrysik@gmail.com> </div>
<div style="display:none"> Gerrit-Reviewer: Vasil Velichkov <vvvelichkov@gmail.com> </div>
<div style="display:none"> Gerrit-CC: Vadim Yanitskiy <axilirator@gmail.com> </div>