<p>Neels Hofmeyr has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.osmocom.org/11810">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">fill_config.py: add ${foreach(FOO)}<br><br>This patch indicates that we should rather use a proper templating python<br>library instead of re-inventing this stuff. But now that we're here...<br><br>Add a construct<br><br> ${foreach(BTS)}<br> bts ${BTSn}<br> location_area_code ${BTSn_LAC}<br> ${foreach_end}<br><br>that repeats for each BTS<nr> variable found, e.g.<br><br> BTS0_LAC=23<br> BTS1_LAC=24<br> BTS2_LAC=25<br><br>would result in three blocks of the above.<br><br>I am using this to avoid copy-pasting for configuring N BTSes for 35c3.<br><br>Change-Id: Ie1139a017f42cea5bf7ebbbe457bbc3bfe06944c<br>---<br>M net/fill_config.py<br>1 file changed, 100 insertions(+), 32 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.osmocom.org:29418/osmo-dev refs/changes/10/11810/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/net/fill_config.py b/net/fill_config.py</span><br><span>index c33e6b7..6aa3829 100755</span><br><span>--- a/net/fill_config.py</span><br><span>+++ b/net/fill_config.py</span><br><span>@@ -99,30 +99,109 @@</span><br><span> print('Stale: %r is newer than %r' % (src_path, target_path))</span><br><span> exit(1)</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-def insert_includes(tmpl, tmpl_dir, tmpl_src):</span><br><span style="color: hsl(120, 100%, 40%);">+def replace_vars(tmpl, tmpl_dir, tmpl_src, local_config, strict=True):</span><br><span style="color: hsl(120, 100%, 40%);">+ used_vars = set()</span><br><span style="color: hsl(120, 100%, 40%);">+ for m in replace_re.finditer(tmpl):</span><br><span style="color: hsl(120, 100%, 40%);">+ name = m.group(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not name in local_config:</span><br><span style="color: hsl(120, 100%, 40%);">+ if strict:</span><br><span style="color: hsl(120, 100%, 40%);">+ print('Error: undefined var %r in %r' % (name, tmpl_src))</span><br><span style="color: hsl(120, 100%, 40%);">+ exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ else:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ used_vars.add(name)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ for var in used_vars:</span><br><span style="color: hsl(120, 100%, 40%);">+ tmpl = tmpl.replace('${%s}' % var, local_config.get(var))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ return tmpl</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def insert_includes(tmpl, tmpl_dir, tmpl_src, local_config, arg):</span><br><span style="color: hsl(120, 100%, 40%);">+ include_path = os.path.join(tmpl_dir, arg)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not os.path.isfile(include_path):</span><br><span style="color: hsl(120, 100%, 40%);">+ print('Error: included file does not exist: %r in %r' % (include_path, tmpl_src))</span><br><span style="color: hsl(120, 100%, 40%);">+ exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+ try:</span><br><span style="color: hsl(120, 100%, 40%);">+ incl = open(include_path).read()</span><br><span style="color: hsl(120, 100%, 40%);">+ except:</span><br><span style="color: hsl(120, 100%, 40%);">+ print('Cannot read %r for %r' % (include_path, tmpl_src))</span><br><span style="color: hsl(120, 100%, 40%);">+ raise</span><br><span style="color: hsl(120, 100%, 40%);">+ if args.check_stale:</span><br><span style="color: hsl(120, 100%, 40%);">+ check_stale(include_path, dst)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # recurse, to follow the paths that the included bits come from</span><br><span style="color: hsl(120, 100%, 40%);">+ incl = handle_commands(incl, os.path.dirname(include_path), include_path, local_config)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ return tmpl.replace('${include(%s)}' % arg, incl)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def insert_foreach(tmpl, tmpl_dir, tmpl_src, match, local_config, arg):</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # figure out section to handle</span><br><span style="color: hsl(120, 100%, 40%);">+ start_span = match.span()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if tmpl[start_span[1]] == '\n':</span><br><span style="color: hsl(120, 100%, 40%);">+ start_span = (start_span[0], start_span[1] + 1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ end_str = '${foreach_end}\n'</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ end_at = tmpl.find(end_str, start_span[1])</span><br><span style="color: hsl(120, 100%, 40%);">+ if end_at < 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ end_str = end_str[:-1]</span><br><span style="color: hsl(120, 100%, 40%);">+ end_at = tmpl.find(end_str, start_span[1])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if end_at < 0:</span><br><span style="color: hsl(120, 100%, 40%);">+ raise Exception('%r: unmatched %r' % (tmpl_src, match.string))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ end_span = (end_at, end_at + len(end_str))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ before_block = tmpl[:start_span[0]]</span><br><span style="color: hsl(120, 100%, 40%);">+ foreach_block = tmpl[start_span[1]:end_span[0]]</span><br><span style="color: hsl(120, 100%, 40%);">+ after_block = tmpl[end_span[1]:]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ # figure out what items matching the foreach(FOO<number>) there are</span><br><span style="color: hsl(120, 100%, 40%);">+ item_re = re.compile('(^%s([0-9]+))_.*' % arg)</span><br><span style="color: hsl(120, 100%, 40%);">+ items = set()</span><br><span style="color: hsl(120, 100%, 40%);">+ for item in local_config.keys():</span><br><span style="color: hsl(120, 100%, 40%);">+ item_m = item_re.match(item)</span><br><span style="color: hsl(120, 100%, 40%);">+ if not item_m:</span><br><span style="color: hsl(120, 100%, 40%);">+ continue</span><br><span style="color: hsl(120, 100%, 40%);">+ items.add((item_m.group(1), item_m.group(2)))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ items = sorted(list(items))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded = [before_block]</span><br><span style="color: hsl(120, 100%, 40%);">+ for item, nr in items:</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded_block = foreach_block</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ while True:</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded_block_was = expanded_block</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded_block = expanded_block.replace('${%sn_' % arg, '${%s_' % item)</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded_block = expanded_block.replace('${%sn}' % arg, nr)</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded_block = replace_vars(expanded_block, tmpl_dir, tmpl_src, local_config)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ if expanded_block_was == expanded_block:</span><br><span style="color: hsl(120, 100%, 40%);">+ break</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded.append(expanded_block)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ expanded.extend(after_block)</span><br><span style="color: hsl(120, 100%, 40%);">+ return ''.join(expanded)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def handle_commands(tmpl, tmpl_dir, tmpl_src, local_config):</span><br><span style="color: hsl(120, 100%, 40%);">+ handled = 0</span><br><span> for m in command_re.finditer(tmpl):</span><br><span style="color: hsl(120, 100%, 40%);">+ handled += 1</span><br><span> cmd = m.group(1)</span><br><span> arg = m.group(2)</span><br><span> if cmd == 'include':</span><br><span style="color: hsl(0, 100%, 40%);">- include_path = os.path.join(tmpl_dir, arg)</span><br><span style="color: hsl(0, 100%, 40%);">- if not os.path.isfile(include_path):</span><br><span style="color: hsl(0, 100%, 40%);">- print('Error: included file does not exist: %r in %r' % (include_path, tmpl_src))</span><br><span style="color: hsl(0, 100%, 40%);">- exit(1)</span><br><span style="color: hsl(0, 100%, 40%);">- try:</span><br><span style="color: hsl(0, 100%, 40%);">- incl = open(include_path).read()</span><br><span style="color: hsl(0, 100%, 40%);">- except:</span><br><span style="color: hsl(0, 100%, 40%);">- print('Cannot read %r for %r' % (include_path, tmpl_src))</span><br><span style="color: hsl(0, 100%, 40%);">- raise</span><br><span style="color: hsl(0, 100%, 40%);">- if args.check_stale:</span><br><span style="color: hsl(0, 100%, 40%);">- check_stale(include_path, dst)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- # recurse, to follow the paths that the included bits come from</span><br><span style="color: hsl(0, 100%, 40%);">- incl = insert_includes(incl, os.path.dirname(include_path), include_path)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- tmpl = tmpl.replace('${%s(%s)}' % (cmd, arg), incl)</span><br><span style="color: hsl(120, 100%, 40%);">+ tmpl = insert_includes(tmpl, tmpl_dir, tmpl_src, local_config, arg)</span><br><span style="color: hsl(120, 100%, 40%);">+ elif cmd == 'foreach':</span><br><span style="color: hsl(120, 100%, 40%);">+ tmpl = insert_foreach(tmpl, tmpl_dir, tmpl_src, m, local_config, arg)</span><br><span> else:</span><br><span> print('Error: unknown command: %r in %r' % (cmd, tmpl_src))</span><br><span> exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span> return tmpl</span><br><span> </span><br><span> for tmpl_name in sorted(os.listdir(tmpl_dir)):</span><br><span>@@ -151,23 +230,12 @@</span><br><span> raise</span><br><span> </span><br><span> while True:</span><br><span style="color: hsl(0, 100%, 40%);">- used_vars = set()</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- result = insert_includes(result, tmpl_dir, tmpl_src)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- for m in replace_re.finditer(result):</span><br><span style="color: hsl(0, 100%, 40%);">- name = m.group(1)</span><br><span style="color: hsl(0, 100%, 40%);">- if not name in local_config:</span><br><span style="color: hsl(0, 100%, 40%);">- print('Error: undefined var %r in %r' % (name, tmpl_src))</span><br><span style="color: hsl(0, 100%, 40%);">- exit(1)</span><br><span style="color: hsl(0, 100%, 40%);">- used_vars.add(name)</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span style="color: hsl(0, 100%, 40%);">- if not used_vars:</span><br><span style="color: hsl(120, 100%, 40%);">+ result_was = result</span><br><span style="color: hsl(120, 100%, 40%);">+ result = handle_commands(result, tmpl_dir, tmpl_src, local_config)</span><br><span style="color: hsl(120, 100%, 40%);">+ result = replace_vars(result, tmpl_dir, tmpl_src, local_config)</span><br><span style="color: hsl(120, 100%, 40%);">+ if result_was == result:</span><br><span> break</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">- for var in used_vars:</span><br><span style="color: hsl(0, 100%, 40%);">- result = result.replace('${%s}' % var, local_config.get(var))</span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span> if not args.check_stale:</span><br><span> with open(dst, 'w') as dst_file:</span><br><span> dst_file.write(result)</span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.osmocom.org/11810">change 11810</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.osmocom.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.osmocom.org/11810"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: osmo-dev </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: Ie1139a017f42cea5bf7ebbbe457bbc3bfe06944c </div>
<div style="display:none"> Gerrit-Change-Number: 11810 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Neels Hofmeyr <nhofmeyr@sysmocom.de> </div>