<p>Neels Hofmeyr has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/10012">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">add scripts/verify_log_statements.py<br><br>This came up in<br>https://gerrit.osmocom.org/#/c/osmo-bsc/+/9671/6//COMMIT_MSG@36<br><br>The errors it finds in the current code base are numerous, and many are<br>intended LOGP .. LOGPC calls. It doesn't make sense to enforce this, but so far<br>this can be used manually.<br><br>Change-Id: Id79389f090a2fded7ff01dc7e3fe9774e7f22ca0<br>---<br>A scripts/verify_log_statements.py<br>1 file changed, 87 insertions(+), 0 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-ci refs/changes/12/10012/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/scripts/verify_log_statements.py b/scripts/verify_log_statements.py</span><br><span>new file mode 100755</span><br><span>index 0000000..e7752e1</span><br><span>--- /dev/null</span><br><span>+++ b/scripts/verify_log_statements.py</span><br><span>@@ -0,0 +1,87 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python3</span><br><span style="color: hsl(120, 100%, 40%);">+__doc__ = '''</span><br><span style="color: hsl(120, 100%, 40%);">+With regex magic, try to pinpoint all LOG* macro calls that lack a final newline.</span><br><span style="color: hsl(120, 100%, 40%);">+Also find those that have non-printable characters or extra newlines.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+Usage:</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ ./verify_log_statements.py [-d|--debug] [dir] [file] [...]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+Without args, default to '.'</span><br><span style="color: hsl(120, 100%, 40%);">+'''</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+import re</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+import codecs</span><br><span style="color: hsl(120, 100%, 40%);">+import os.path</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+# This regex matches the entire LOGxx(...) statement over multiple lines.</span><br><span style="color: hsl(120, 100%, 40%);">+# It pinpoints the format string by looking for the first arg that contains quotes.</span><br><span style="color: hsl(120, 100%, 40%);">+# It then matches any number of separate quoted strings, and accepts 0 or more args after that.</span><br><span style="color: hsl(120, 100%, 40%);">+log_statement_re = re.compile(r'^[ \t]*LOG[_A-Z]+\(([^";,]*,)* *(("[^"]*"[^";,]*)*)(,[^;]*|)\);',</span><br><span style="color: hsl(120, 100%, 40%);">+ re.MULTILINE | re.DOTALL)</span><br><span style="color: hsl(120, 100%, 40%);">+fmt_re = re.compile(r'("[^"]*".*)*fmt')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+errors_found = 0</span><br><span style="color: hsl(120, 100%, 40%);">+debug = ('-d' in sys.argv) or ('--debug' in sys.argv)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+args = [x for x in sys.argv[1:] if not (x == '-d' or x == '--debug')]</span><br><span style="color: hsl(120, 100%, 40%);">+if not args:</span><br><span style="color: hsl(120, 100%, 40%);">+ args = ['.']</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def check_file(f):</span><br><span style="color: hsl(120, 100%, 40%);">+ global errors_found</span><br><span style="color: hsl(120, 100%, 40%);">+ if not (f.endswith('.h') or f.endswith('.c') or f.endswith('.cpp')):</span><br><span style="color: hsl(120, 100%, 40%);">+ return</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for log in log_statement_re.finditer(codecs.open(f, "r", "utf-8").read()):</span><br><span style="color: hsl(120, 100%, 40%);">+ quoted = log.group(2)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Skip 'LOG("bla" fmt )' strings that typically appear as #defines.</span><br><span style="color: hsl(120, 100%, 40%);">+ if fmt_re.match(quoted):</span><br><span style="color: hsl(120, 100%, 40%);">+ if debug:</span><br><span style="color: hsl(120, 100%, 40%);">+ print('Skipping define:', f, '\n'+log.group(0))</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Drop PRI* parts of 'LOG("bla %"PRIu64" foo")'</span><br><span style="color: hsl(120, 100%, 40%);">+ for n in (16,32,64):</span><br><span style="color: hsl(120, 100%, 40%);">+ quoted = quoted.replace('PRIu' + str(n), '')</span><br><span style="color: hsl(120, 100%, 40%);">+ quoted = quoted.replace('PRId' + str(n), '')</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # Use py eval to join separate string constants: drop any tabs/newlines</span><br><span style="color: hsl(120, 100%, 40%);">+ # that are not in quotes, between separate string constants.</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ quoted = eval('(' + quoted + '\n)' )</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ # hopefully eval broke because of some '## args' macro def</span><br><span style="color: hsl(120, 100%, 40%);">+ if debug:</span><br><span style="color: hsl(120, 100%, 40%);">+ print('Ignoring:', f, '\n'+log.group(0)) </span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # check for errors...</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # final newline</span><br><span style="color: hsl(120, 100%, 40%);">+ if not quoted.endswith('\n'):</span><br><span style="color: hsl(120, 100%, 40%);">+ print('Missing final newline:', f, '\n'+log.group(0))</span><br><span style="color: hsl(120, 100%, 40%);">+ errors_found += 1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # disallowed chars and extra newlines</span><br><span style="color: hsl(120, 100%, 40%);">+ for c in quoted[:-1]:</span><br><span style="color: hsl(120, 100%, 40%);">+ if not c.isprintable() and not c == '\t':</span><br><span style="color: hsl(120, 100%, 40%);">+ if c == '\n':</span><br><span style="color: hsl(120, 100%, 40%);">+ msg = 'Extraneous newline'</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ msg = 'Illegal char'</span><br><span style="color: hsl(120, 100%, 40%);">+ print('%s %r in' % (msg, c), f, '\n' + log.group(0))</span><br><span style="color: hsl(120, 100%, 40%);">+ errors_found += 1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+for f in args:</span><br><span style="color: hsl(120, 100%, 40%);">+ if os.path.isdir(f):</span><br><span style="color: hsl(120, 100%, 40%);">+ for parent_path, subdirs, files in os.walk(f, None, None):</span><br><span style="color: hsl(120, 100%, 40%);">+ for ff in files:</span><br><span style="color: hsl(120, 100%, 40%);">+ check_file(os.path.join(parent_path, ff))</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ check_file(f)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+sys.exit(errors_found)</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/10012">change 10012</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/10012"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-ci </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: Id79389f090a2fded7ff01dc7e3fe9774e7f22ca0 </div>
<div style="display:none"> Gerrit-Change-Number: 10012 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Neels Hofmeyr <nhofmeyr@sysmocom.de> </div>