Change in osmo-ci[master]: verify_*.py: Ignore UTF-8 decoding errors

[laforge]

laforge has submitted this change. ( https://gerrit.osmocom.org/c/osmo-ci/+/16638 )

Change subject: verify_*.py: Ignore UTF-8 decoding errors
......................................................................

verify_*.py: Ignore UTF-8 decoding errors

Some of our source files are inherited from other sources, particularly
for microcontroller firmware projects.  We cannot assume they're all
clean UTF-8.  Let's ignore any decoder errors when verifying log
statements and value_string arrays.

Closes: OS#4334
Change-Id: I1e19f4bc6bee46481c6ea743e8334bd4485909be
---
M scripts/verify_log_statements.py
M scripts/verify_value_string_arrays_are_terminated.py
2 files changed, 2 insertions(+), 2 deletions(-)

Approvals:
  laforge: Looks good to me, approved; Verified



diff --git a/scripts/verify_log_statements.py b/scripts/verify_log_statements.py
index 9a9cd1b..064df73 100755
--- a/scripts/verify_log_statements.py
+++ b/scripts/verify_log_statements.py
@@ -66,7 +66,7 @@
   try:
     errors_found = []
 
-    file_content = codecs.open(f, "r", "utf-8").read()
+    file_content = codecs.open(f, "r", "utf-8", errors='ignore').read()
 
     for log in log_statement_re.finditer(file_content):
       quoted = log.group(2)
diff --git a/scripts/verify_value_string_arrays_are_terminated.py b/scripts/verify_value_string_arrays_are_terminated.py
index 9f0ad82..f6dc545 100755
--- a/scripts/verify_value_string_arrays_are_terminated.py
+++ b/scripts/verify_value_string_arrays_are_terminated.py
@@ -27,7 +27,7 @@
   global errors_found
   if not (f.endswith('.h') or f.endswith('.c') or f.endswith('.cpp')):
     return
-  arrays = value_string_array_re.findall(codecs.open(f, "r", "utf-8").read())
+  arrays = value_string_array_re.findall(codecs.open(f, "r", "utf-8", errors='ignore').read())
   for array_def, name in arrays:
     if not terminator_re.search(array_def):
       print('ERROR: file contains unterminated value_string %r: %r'

-- 
To view, visit https://gerrit.osmocom.org/c/osmo-ci/+/16638
To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings

Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Change-Id: I1e19f4bc6bee46481c6ea743e8334bd4485909be
Gerrit-Change-Number: 16638
Gerrit-PatchSet: 1
Gerrit-Owner: laforge <laforge at osmocom.org>
Gerrit-Reviewer: laforge <laforge at osmocom.org>
Gerrit-MessageType: merged
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osmocom.org/pipermail/gerrit-log/attachments/20191217/ca96aca4/attachment.htm>