<p>osmith <strong>submitted</strong> this change.</p><p><a href="https://gerrit.osmocom.org/c/python/osmo-python-tests/+/16540">View Change</a></p><div style="white-space:pre-wrap">Approvals:
laforge: Looks good to me, approved
Jenkins Builder: Verified
</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">Drop python2 support (again)<br><br>Re-apply reverted commit Iabda95073faa2191fd117e9637e0858c589e9d9e<br>("Drop python2 support"), but with additional changes to make the<br>scripts actually work with python3 and to make it build without python2.<br><br>I have verified, that the contrib/jenkins.sh scripts of all Osmocom<br>repositories (with their python3 patches on top) are working with this<br>patch and that all Osmocom repositories with the python3 patches build<br>in OBS (tested in own namespace).<br><br>All related patches for changing from python2 to 3 in other repositories<br>must be merged shortly after this one, as soon as the build slaves were<br>(automatically) updated to have the new osmo-python-tests installed:<br>https://gerrit.osmocom.org/q/topic:drop-py2<br><br>New fixes:<br>* osmopy/obscvty.py: verify: fix compare<br> Comparing maps in python3 does not work the same as in python2. Convert<br> them to lists first, so the compare works as intended again.<br> Fix error:<br> File "/home/user/code/osmo-dev/src/osmo-python-tests/scripts/osmotestvty.py", line 57, in test_history<br> assert(self.vty.w_verify(test_str, [t1]))<br> AssertionError<br>* osmopy/obscvty.py: use enc/dec with send/recv<br> Fix error:<br> self.socket.send("%s\r" % request)<br> TypeError: a bytes-like object is required, not 'str'<br>* scripts/osmotestconfig.py: use encode() before writing to file<br> Fix error:<br> File "/home/user/code/osmo-dev/src/osmo-python-tests/scripts/osmotestconfig.py", line 91, in copy_config<br> tmpfile.write(open(config).read())<br> File "/usr/lib/python3.5/tempfile.py", line 622, in func_wrapper<br> return func(*args, **kwargs)<br> TypeError: a bytes-like object is required, not 'str'<br>* debian/control: add --buildsystem=pybuild. Otherwise "--with python3"<br> is ignored and the build fails if python2 is not installed, with:<br> Can't exec "pyversions": No such file or directory at /usr/[...]/python_distutils.pm line 120.<br><br>Related: OS#2819<br>Change-Id: I3ffc3519bf6c22536a49dad7a966188ddad351a7<br>---<br>M contrib/jenkins.sh<br>M debian/control<br>M debian/rules<br>M osmopy/__init__.py<br>M osmopy/obscvty.py<br>M osmopy/osmo_interact/__init__.py<br>M osmopy/osmo_interact/common.py<br>M osmopy/osmoutil.py<br>M scripts/osmodumpdoc.py<br>M scripts/osmotestconfig.py<br>M scripts/osmotestvty.py<br>M setup.py<br>D tests/test_py2.py<br>13 files changed, 28 insertions(+), 77 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/contrib/jenkins.sh b/contrib/jenkins.sh</span><br><span>index 08908a4..9b219f9 100755</span><br><span>--- a/contrib/jenkins.sh</span><br><span>+++ b/contrib/jenkins.sh</span><br><span>@@ -4,16 +4,6 @@</span><br><span> </span><br><span> COM_FLAGS='-m compileall'</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-# FIXME: remove once python 2 support is deprecated</span><br><span style="color: hsl(0, 100%, 40%);">-PY2=python2</span><br><span style="color: hsl(0, 100%, 40%);">-PY2_LIST="osmopy scripts/osmodumpdoc.py scripts/osmotestvty.py scripts/osmotestconfig.py"</span><br><span style="color: hsl(0, 100%, 40%);">-$PY2 ./setup.py install</span><br><span style="color: hsl(0, 100%, 40%);">-$PY2 tests/test_py2.py</span><br><span style="color: hsl(0, 100%, 40%);">-for f in $PY2_LIST</span><br><span style="color: hsl(0, 100%, 40%);">-do</span><br><span style="color: hsl(0, 100%, 40%);">- $PY2 $COM_FLAGS $f</span><br><span style="color: hsl(0, 100%, 40%);">-done</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> rm -rf ./build</span><br><span> PY3=python3</span><br><span> PY3_LIST="osmopy scripts/osmo_ctrl.py scripts/osmo_rate_ctr2csv.py scripts/osmodumpdoc.py scripts/osmotestvty.py scripts/osmotestconfig.py scripts/osmo_interact_ctrl.py scripts/osmo_interact_vty.py scripts/osmo_verify_transcript_ctrl.py scripts/osmo_verify_transcript_vty.py scripts/soap.py scripts/twisted_ipa.py"</span><br><span>diff --git a/debian/control b/debian/control</span><br><span>index 52b614c..1ad871d 100644</span><br><span>--- a/debian/control</span><br><span>+++ b/debian/control</span><br><span>@@ -2,26 +2,12 @@</span><br><span> Section: python</span><br><span> Priority: optional</span><br><span> Maintainer: Harald Welte <laforge@gnumonks.org></span><br><span style="color: hsl(0, 100%, 40%);">-Build-Depends: debhelper (>= 9), python, dh-python, python-setuptools, python3, python3-setuptools</span><br><span style="color: hsl(120, 100%, 40%);">+Build-Depends: debhelper (>= 9), dh-python, python3, python3-setuptools</span><br><span> Standards-Version: 3.9.8</span><br><span> Homepage: http://git.osmocom.org/python/osmo-python-tests/</span><br><span> Vcs-Git: git://git.osmocom.org/python/osmo-python-tests</span><br><span> Vcs-Browser: http://git.osmocom.org/python/osmo-python-tests/</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-Package: python2-osmopy-libs</span><br><span style="color: hsl(0, 100%, 40%);">-Architecture: all</span><br><span style="color: hsl(0, 100%, 40%);">-Depends: ${python:Depends}, ${misc:Depends}</span><br><span style="color: hsl(0, 100%, 40%);">-Description: Python code (not only) for testing of Osmocom programs</span><br><span style="color: hsl(0, 100%, 40%);">- .</span><br><span style="color: hsl(0, 100%, 40%);">- This package contains the Python 2 version of osmopy libraries.</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-Package: python2-osmopy-utils</span><br><span style="color: hsl(0, 100%, 40%);">-Architecture: all</span><br><span style="color: hsl(0, 100%, 40%);">-Depends: ${python:Depends}, ${misc:Depends}, python2-osmopy-libs</span><br><span style="color: hsl(0, 100%, 40%);">-Description: Python code (not only) for testing of Osmocom programs</span><br><span style="color: hsl(0, 100%, 40%);">- .</span><br><span style="color: hsl(0, 100%, 40%);">- This package contains the Python 2 version of osmopy utils.</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> Package: python3-osmopy-libs</span><br><span> Architecture: all</span><br><span> Depends: ${python3:Depends}, ${misc:Depends}</span><br><span>diff --git a/debian/rules b/debian/rules</span><br><span>index 04b59f6..ff16a03 100755</span><br><span>--- a/debian/rules</span><br><span>+++ b/debian/rules</span><br><span>@@ -1,13 +1,9 @@</span><br><span> #!/usr/bin/make -f</span><br><span> </span><br><span> %:</span><br><span style="color: hsl(0, 100%, 40%);">- dh $@ --with python2,python3</span><br><span style="color: hsl(120, 100%, 40%);">+ dh $@ --with python3 --buildsystem=pybuild</span><br><span> </span><br><span> override_dh_auto_install:</span><br><span style="color: hsl(0, 100%, 40%);">- python2 setup.py install --install-layout=deb --root=$(CURDIR)/debian/python2-osmopy-libs</span><br><span style="color: hsl(0, 100%, 40%);">- rm -rf $(CURDIR)/debian/python2-osmopy-libs/usr/bin</span><br><span style="color: hsl(0, 100%, 40%);">- python2 setup.py install --install-layout=deb --root=$(CURDIR)/debian/python2-osmopy-utils</span><br><span style="color: hsl(0, 100%, 40%);">- rm -rf $(CURDIR)/debian/python2-osmopy-utils/usr/lib</span><br><span> python3 setup.py install --install-layout=deb --root=$(CURDIR)/debian/python3-osmopy-libs</span><br><span> rm -rf $(CURDIR)/debian/python3-osmopy-libs/usr/bin</span><br><span> python3 setup.py install --install-layout=deb --root=$(CURDIR)/debian/python3-osmopy-utils</span><br><span>diff --git a/osmopy/__init__.py b/osmopy/__init__.py</span><br><span>index ce78caf..18362ec 100644</span><br><span>--- a/osmopy/__init__.py</span><br><span>+++ b/osmopy/__init__.py</span><br><span>@@ -1,4 +1,4 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span> __version__ = '0.1.0'</span><br><span> </span><br><span> __all__ = ['obscvty', 'osmoutil', 'osmo_ipa', 'osmo_interact', 'trap_helper', 'twisted_ipa']</span><br><span>diff --git a/osmopy/obscvty.py b/osmopy/obscvty.py</span><br><span>index d39d3da..6b7d56c 100755</span><br><span>--- a/osmopy/obscvty.py</span><br><span>+++ b/osmopy/obscvty.py</span><br><span>@@ -16,7 +16,6 @@</span><br><span> #</span><br><span> # VTY helper code for OpenBSC</span><br><span> #</span><br><span style="color: hsl(0, 100%, 40%);">-from __future__ import print_function</span><br><span> import re</span><br><span> import socket</span><br><span> import sys, subprocess</span><br><span>@@ -176,13 +175,13 @@</span><br><span> self._connect_socket()</span><br><span> </span><br><span> # Now send the command</span><br><span style="color: hsl(0, 100%, 40%);">- self.socket.send("%s\r" % request)</span><br><span style="color: hsl(120, 100%, 40%);">+ self.socket.send(("%s\r" % request).encode())</span><br><span> res = ""</span><br><span> end = ""</span><br><span> </span><br><span> # Unfortunately, timeout and recv don't always play nicely</span><br><span> while True:</span><br><span style="color: hsl(0, 100%, 40%);">- data = self.socket.recv(4096)</span><br><span style="color: hsl(120, 100%, 40%);">+ data = self.socket.recv(4096).decode()</span><br><span> res = "%s%s" % (res, data)</span><br><span> if not res: # yes, this is ugly</span><br><span> raise IOError("Failed to read data (did the app crash?)")</span><br><span>@@ -205,7 +204,7 @@</span><br><span> buffer = ''</span><br><span> data = True</span><br><span> while data:</span><br><span style="color: hsl(0, 100%, 40%);">- data = self.socket.recv(recv_buffer)</span><br><span style="color: hsl(120, 100%, 40%);">+ data = self.socket.recv(recv_buffer).decode()</span><br><span> buffer += data</span><br><span> </span><br><span> while buffer.find(delim) != -1:</span><br><span>@@ -244,8 +243,8 @@</span><br><span> def verify(self, command, results, close=False, loud=True, f=None):</span><br><span> res = self.command(command, close).split('\r\n')</span><br><span> if f:</span><br><span style="color: hsl(0, 100%, 40%);">- res = map(f, res)</span><br><span style="color: hsl(0, 100%, 40%);">- results = map(f, results)</span><br><span style="color: hsl(120, 100%, 40%);">+ res = list(map(f, res))</span><br><span style="color: hsl(120, 100%, 40%);">+ results = list(map(f, results))</span><br><span> </span><br><span> if loud:</span><br><span> if res != results:</span><br><span>diff --git a/osmopy/osmo_interact/__init__.py b/osmopy/osmo_interact/__init__.py</span><br><span>index 4fc4fac..036238a 100644</span><br><span>--- a/osmopy/osmo_interact/__init__.py</span><br><span>+++ b/osmopy/osmo_interact/__init__.py</span><br><span>@@ -1,2 +1,2 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span> __all__ = ['common', 'vty', 'ctrl']</span><br><span>diff --git a/osmopy/osmo_interact/common.py b/osmopy/osmo_interact/common.py</span><br><span>index cc7e190..87eca6a 100644</span><br><span>--- a/osmopy/osmo_interact/common.py</span><br><span>+++ b/osmopy/osmo_interact/common.py</span><br><span>@@ -24,11 +24,6 @@</span><br><span> vty.py and ctrl.py plug VTY and CTRL interface specific bits.</span><br><span> '''</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-# Our setup.py currently wants everything to be parsable by both py2 and py3.</span><br><span style="color: hsl(0, 100%, 40%);">-# IMHO that is not a good idea, but until that changes, let's just keep this</span><br><span style="color: hsl(0, 100%, 40%);">-# py2 legacy shim in here so we can syntax-check this py3 module with py2.</span><br><span style="color: hsl(0, 100%, 40%);">-from __future__ import print_function</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> import argparse</span><br><span> import sys</span><br><span> import os</span><br><span>diff --git a/osmopy/osmoutil.py b/osmopy/osmoutil.py</span><br><span>index 54a3456..ec9c8c3 100755</span><br><span>--- a/osmopy/osmoutil.py</span><br><span>+++ b/osmopy/osmoutil.py</span><br><span>@@ -1,4 +1,4 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span> </span><br><span> # (C) 2013 by Katerina Barone-Adesi <kat.obsc@gmail.com></span><br><span> # This program is free software: you can redistribute it and/or modify</span><br><span>@@ -14,7 +14,6 @@</span><br><span> # You should have received a copy of the GNU General Public License</span><br><span> # along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-from __future__ import print_function</span><br><span> import subprocess</span><br><span> import os</span><br><span> import sys</span><br><span>diff --git a/scripts/osmodumpdoc.py b/scripts/osmodumpdoc.py</span><br><span>index d71edc8..64b088f 100755</span><br><span>--- a/scripts/osmodumpdoc.py</span><br><span>+++ b/scripts/osmodumpdoc.py</span><br><span>@@ -1,10 +1,9 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span> </span><br><span> # Make sure this code is in sync with the BTS directory.</span><br><span> # Fixes may need to be applied to both.</span><br><span> </span><br><span> """Start the process and dump the documentation to the doc dir."""</span><br><span style="color: hsl(0, 100%, 40%);">-from __future__ import print_function</span><br><span> import subprocess</span><br><span> import time</span><br><span> import os</span><br><span>diff --git a/scripts/osmotestconfig.py b/scripts/osmotestconfig.py</span><br><span>index f227504..3c96ca6 100755</span><br><span>--- a/scripts/osmotestconfig.py</span><br><span>+++ b/scripts/osmotestconfig.py</span><br><span>@@ -1,4 +1,4 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span> </span><br><span> # (C) 2013 by Katerina Barone-Adesi <kat.obsc@gmail.com></span><br><span> # This program is free software: you can redistribute it and/or modify</span><br><span>@@ -13,7 +13,6 @@</span><br><span> </span><br><span> # You should have received a copy of the GNU General Public License</span><br><span> # along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(0, 100%, 40%);">-from __future__ import print_function</span><br><span> import os</span><br><span> import os.path</span><br><span> import time</span><br><span>@@ -89,7 +88,7 @@</span><br><span> prefix = os.path.basename(config)</span><br><span> tmpfile = tempfile.NamedTemporaryFile(</span><br><span> dir=dirname, prefix=prefix, delete=False)</span><br><span style="color: hsl(0, 100%, 40%);">- tmpfile.write(open(config).read())</span><br><span style="color: hsl(120, 100%, 40%);">+ tmpfile.write(open(config).read().encode())</span><br><span> tmpfile.close()</span><br><span> # This works around the precautions NamedTemporaryFile is made for...</span><br><span> return tmpfile.name</span><br><span>diff --git a/scripts/osmotestvty.py b/scripts/osmotestvty.py</span><br><span>index 55017a5..600860f 100755</span><br><span>--- a/scripts/osmotestvty.py</span><br><span>+++ b/scripts/osmotestvty.py</span><br><span>@@ -1,4 +1,4 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span> </span><br><span> # (C) 2013 by Katerina Barone-Adesi <kat.obsc@gmail.com></span><br><span> # This program is free software: you can redistribute it and/or modify</span><br><span>@@ -13,7 +13,6 @@</span><br><span> </span><br><span> # You should have received a copy of the GNU General Public License</span><br><span> # along with this program. If not, see <http://www.gnu.org/licenses/>.</span><br><span style="color: hsl(0, 100%, 40%);">-from __future__ import print_function</span><br><span> import os</span><br><span> import time</span><br><span> import unittest</span><br><span>diff --git a/setup.py b/setup.py</span><br><span>index 69e18b0..ab604ce 100755</span><br><span>--- a/setup.py</span><br><span>+++ b/setup.py</span><br><span>@@ -19,24 +19,20 @@</span><br><span> from osmopy import __version__</span><br><span> import sys</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-if sys.version_info.major == 2:</span><br><span style="color: hsl(0, 100%, 40%);">- scripts = [</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmodumpdoc.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmotestconfig.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmotestvty.py",</span><br><span style="color: hsl(0, 100%, 40%);">- ]</span><br><span style="color: hsl(0, 100%, 40%);">-elif sys.version_info.major == 3:</span><br><span style="color: hsl(0, 100%, 40%);">- scripts = [</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_ctrl.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_rate_ctr2csv.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/soap.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/ctrl2cgi.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_trap2cgi.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_interact_vty.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_interact_ctrl.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_verify_transcript_vty.py",</span><br><span style="color: hsl(0, 100%, 40%);">- "scripts/osmo_verify_transcript_ctrl.py",</span><br><span style="color: hsl(0, 100%, 40%);">- ]</span><br><span style="color: hsl(120, 100%, 40%);">+scripts = [</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmodumpdoc.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmotestvty.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmotestconfig.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_ctrl.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_rate_ctr2csv.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/soap.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/ctrl2cgi.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_trap2cgi.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_interact_vty.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_interact_ctrl.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_verify_transcript_vty.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ "scripts/osmo_verify_transcript_ctrl.py",</span><br><span style="color: hsl(120, 100%, 40%);">+ ]</span><br><span> </span><br><span> setup(</span><br><span> name = 'osmopython',</span><br><span>diff --git a/tests/test_py2.py b/tests/test_py2.py</span><br><span>deleted file mode 100644</span><br><span>index cac5261..0000000</span><br><span>--- a/tests/test_py2.py</span><br><span>+++ /dev/null</span><br><span>@@ -1,7 +0,0 @@</span><br><span style="color: hsl(0, 100%, 40%);">-#!/usr/bin/env python2</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-# just import a smoke test for osmopy</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-import osmopy</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">-print '[Python2] Smoke test PASSED.'</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/c/python/osmo-python-tests/+/16540">change 16540</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/c/python/osmo-python-tests/+/16540"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: python/osmo-python-tests </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-Change-Id: I3ffc3519bf6c22536a49dad7a966188ddad351a7 </div>
<div style="display:none"> Gerrit-Change-Number: 16540 </div>
<div style="display:none"> Gerrit-PatchSet: 2 </div>
<div style="display:none"> Gerrit-Owner: osmith <osmith@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins Builder </div>
<div style="display:none"> Gerrit-Reviewer: daniel <dwillmann@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: fixeria <axilirator@gmail.com> </div>
<div style="display:none"> Gerrit-Reviewer: laforge <laforge@osmocom.org> </div>
<div style="display:none"> Gerrit-Reviewer: neels <nhofmeyr@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: osmith <osmith@sysmocom.de> </div>
<div style="display:none"> Gerrit-Reviewer: pespin <pespin@sysmocom.de> </div>
<div style="display:none"> Gerrit-MessageType: merged </div>