osmith has uploaded this change for review.

View Change

jenkins-gerrit: binpkgs: ignore decode errors

The output of recent osmo-s1gw jobs for building binary packages
contains invalid utf-8 characters, even though it should be utf-8
according to the HTTP header:
  content-type: text/plain;charset=utf-8

We only need to get the distro name from the output here, so simply
ignore any decode errors.

Fix for:
  $ ./comment_generate.py https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-s1gw/417/ -o gerrit_report.json -t result
  Reading https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-s1gw/417//consoleText
  Reading https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs/17255/consoleText
  Reading https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs/17260/consoleText
  Traceback (most recent call last):
    File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 270, in <module>
      main()
    File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 257, in main
      comment = get_comment_result(args.build_url, args.notify_on_success)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 198, in get_comment_result
      pipeline = parse_pipeline(build_url)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 89, in parse_pipeline
      stage = stage_from_job_name(job_name, job_url)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 59, in stage_from_job_name
      return stage_binpkgs_from_url(job_url)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 46, in stage_binpkgs_from_url
      content = response.read().decode("utf-8")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xef in position 237446: invalid continuation byte

Change-Id: If398037272c2de54c359ea8d907b35f6fb1e2b8b
---
M scripts/jenkins-gerrit/comment_generate.py
1 file changed, 1 insertion(+), 1 deletion(-)

git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/12/40912/1

diff --git a/scripts/jenkins-gerrit/comment_generate.py b/scripts/jenkins-gerrit/comment_generate.py
index c81011f..ff5a92a 100755
--- a/scripts/jenkins-gerrit/comment_generate.py
+++ b/scripts/jenkins-gerrit/comment_generate.py
@@ -43,7 +43,7 @@
     url = f"{job_url}/consoleText"
     print(f"Reading {url}")
     with urllib.request.urlopen(url) as response:
-        content = response.read().decode("utf-8")
+        content = response.read().decode("utf-8", errors="ignore")
         match = re_distro.search(content)
         assert match, f"couldn't find distro name in log: {url}"
         return match.group(1)

To view, visit change 40912. To unsubscribe, or for help writing mail filters, visit settings.