November 2023 - gerrit-log - lists.osmocom.org

[S] Change in osmo-dev[master]: ttcn3: pgw: depend on osmo-uecups
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35121?usp=email ) Change subject: ttcn3: pgw: depend on osmo-uecups ...................................................................... ttcn3: pgw: depend on osmo-uecups Change-Id: I95b77ea1a9f4c31c8886dc3f613e545135da320d --- M all.deps M ttcn3/ttcn3.sh 2 files changed, 16 insertions(+), 1 deletion(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/all.deps b/all.deps index 7857f16..5a1191a 100644 --- a/all.deps +++ b/all.deps @@ -35,6 +35,7 @@ osmo-hnbgw osmo-iuh osmo-mgw libosmo-pfcp osmo-e1d libosmocore osmo-remsim simtrace2_host libosmo-netif +osmo-uecups libosmocore open5gs diff --git a/ttcn3/ttcn3.sh b/ttcn3/ttcn3.sh index 85cbb6b..d665e84 100755 --- a/ttcn3/ttcn3.sh +++ b/ttcn3/ttcn3.sh @@ -187,9 +187,13 @@ # Programs that need to be built get_programs() { case "$PROJECT" in + # Open5gs + *-ogs) echo "open5gs" ;; + pgw) echo "open5gs osmo-uecups-daemon" ;; + + # Osmocom bsc|bsc-*) echo "osmo-stp osmo-bsc osmo-bts-omldummy" ;; bts) echo "osmo-bsc osmo-bts-trx" ;; - *-ogs|pgw) echo "open5gs" ;; msc) echo "osmo-stp osmo-msc" ;; pcu-sns) echo "osmo-pcu" ;; pcu) echo "osmo-pcu osmo-bsc osmo-bts-virtual" ;; @@ -206,6 +210,7 @@ osmo-bts-*) echo "osmo-bts" ;; osmo-pcap-*) echo "osmo-pcap" ;; osmo-stp) echo "libosmo-sccp" ;; + osmo-uecups-*) echo "osmo-uecups" ;; *) echo "$1" ;; esac } -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35121?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: I95b77ea1a9f4c31c8886dc3f613e545135da320d Gerrit-Change-Number: 35121 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-dev[master]: werror.opts: add libgtpnl
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35109?usp=email ) Change subject: werror.opts: add libgtpnl ...................................................................... werror.opts: add libgtpnl Depends: libgtpnl Ife3716c667d2e9b3fcd94d3cdc044d594e7dab5a Change-Id: Ia5f6f6d2641318c00608471256257b4e06627c7c --- M werror.opts 1 file changed, 11 insertions(+), 0 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/werror.opts b/werror.opts index 312b97b..3b27b74 100644 --- a/werror.opts +++ b/werror.opts @@ -6,6 +6,7 @@ libosmo-sccp --enable-werror libosmo-pfcp --enable-werror libsmpp34 --enable-werror +libgtpnl --enable-werror osmo-bsc --enable-werror osmo-ggsn --enable-werror osmo-hlr --enable-werror -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35109?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: Ia5f6f6d2641318c00608471256257b4e06627c7c Gerrit-Change-Number: 35109 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-dev[master]: ttcn3.sh: enable color output from gcc
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35110?usp=email ) Change subject: ttcn3.sh: enable color output from gcc ...................................................................... ttcn3.sh: enable color output from gcc Change-Id: Idde8084ca981333be2fc2b369ea8f25881470654 --- M ttcn3/scripts/docker_configure_make.sh 1 file changed, 10 insertions(+), 0 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/ttcn3/scripts/docker_configure_make.sh b/ttcn3/scripts/docker_configure_make.sh index 56fee71..7a79ff1 100755 --- a/ttcn3/scripts/docker_configure_make.sh +++ b/ttcn3/scripts/docker_configure_make.sh @@ -26,6 +26,7 @@ docker run \ --rm \ + -t \ -e "LD_LIBRARY_PATH=/usr/local/lib" \ -v "$DIR_OSMODEV:$DIR_OSMODEV" \ -v "$DIR_USR_LOCAL:/usr/local" \ -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35110?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: Idde8084ca981333be2fc2b369ea8f25881470654 Gerrit-Change-Number: 35110 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-dev[master]: gen_makefile: update URL code for netfilter urls
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35111?usp=email ) Change subject: gen_makefile: update URL code for netfilter urls ...................................................................... gen_makefile: update URL code for netfilter urls * libgtnpnl is now developed at gerrit.osmocom.org * use https:// instead of git:// for libnftnl, nftables * read the urls from a new all.urls file * don't set a push URL pointing to gerrit for projects that have different URLs Change-Id: I5522c9fcc594d296c581ba59c0beb4882e2d976e --- A all.urls M gen_makefile.py 2 files changed, 43 insertions(+), 3 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/all.urls b/all.urls new file mode 100644 index 0000000..27aaf6e --- /dev/null +++ b/all.urls @@ -0,0 +1,4 @@ +# Git clone URLs for projects not developed at gerrit.osmocom.org +# project url +libnftnl https://git.netfilter.org/libnftnl/ +nftables https://git.netfilter.org/nftables/ diff --git a/gen_makefile.py b/gen_makefile.py index a00234f..9a8a5cf 100755 --- a/gen_makefile.py +++ b/gen_makefile.py @@ -52,6 +52,7 @@ topdir = os.path.dirname(os.path.realpath(__file__)) all_deps_file = os.path.join(topdir, "all.deps") +all_urls_file = os.path.join(topdir, "all.urls") parser = argparse.ArgumentParser(epilog=__doc__, formatter_class=argparse.RawTextHelpFormatter) parser.add_argument('configure_opts_files', @@ -148,6 +149,18 @@ l.append((tokens[0], tokens[1:])) return l +def read_projects_urls(path): + 'Read urls config and return dict {project_name: url, …}.' + ret = {} + for line in open(path): + line = line.strip() + if not line or line.startswith('#'): + continue + project, url = line.split() + assert project not in ret, f"project '{project} found twice in {path}" + ret[project] = url + return ret + def read_configure_opts(path): 'Read config opts file and return tuples of (project_name, config-opts).' if not path: @@ -179,14 +192,21 @@ touch $@ ''' - if proj in ("libgtpnl", "libnftnl", "nftables"): - url = "git://git.netfilter.org" + if proj in projects_urls: + url = projects_urls[proj] + cmd_set_push_url = "true" + else: + url = f"{url}/{proj}" + push_url = f"{push_url}/{proj}" + cmd_set_push_url = f'git -C "{src}/{proj}" remote set-url --push origin "{push_url}"' + + cmd_clone = f'git -C {src} clone --recurse-submodules "{url}" "{proj}"' return f''' .make.{proj}.clone: @echo -e "\\n\\n\\n===== $@\\n" test -d {src} || mkdir -p {src} - test -d {src_proj} || ( git -C {src} clone --recurse-submodules "{url}/{proj}" "{proj}" && git -C "{src}/{proj}" remote set-url --push origin "{push_url}/{proj}" ) + test -d {src_proj} || ( {cmd_clone} && {cmd_set_push_url} ) sync touch $@ ''' @@ -303,6 +323,7 @@ projects_deps = read_projects_deps(all_deps_file) +projects_urls = read_projects_urls(all_urls_file) configure_opts = listdict() configure_opts_files = sorted(args.configure_opts_files or []) for configure_opts_file in configure_opts_files: -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35111?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: I5522c9fcc594d296c581ba59c0beb4882e2d976e Gerrit-Change-Number: 35111 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-dev[master]: gen_makefile: don't shell out to nproc
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35112?usp=email ) Change subject: gen_makefile: don't shell out to nproc ...................................................................... gen_makefile: don't shell out to nproc Use python's multiprocessing.cpu_count() instead of "$(nproc)". The latter didn't work properly in Makefiles, the right syntax would have been "$(shell nproc)". Make didn't complain about it and assumed that we want to use all CPUs with an empty argument "-j ", but meson doesn't accept this syntax. Change-Id: I58ca082339f3aff813f587f4c2be9c0951b9b2dd --- M gen_makefile.py 1 file changed, 19 insertions(+), 1 deletion(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/gen_makefile.py b/gen_makefile.py index 9a8a5cf..7fc2bd2 100755 --- a/gen_makefile.py +++ b/gen_makefile.py @@ -49,6 +49,7 @@ import sys import os import argparse +import multiprocessing topdir = os.path.dirname(os.path.realpath(__file__)) all_deps_file = os.path.join(topdir, "all.deps") @@ -85,7 +86,9 @@ parser.add_argument('-o', '--output', dest='output', default='Makefile', help='''Makefile filename (default: 'Makefile').''') -parser.add_argument('-j', '--jobs', dest='jobs', default='$(nproc)', nargs='?', const='$(nproc)', +parser.add_argument('-j', '--jobs', dest='jobs', type=int, + default=multiprocessing.cpu_count(), nargs='?', + const=multiprocessing.cpu_count(), help='''-j option to pass to 'make'.''') parser.add_argument('-I', '--sudo-make-install', dest='sudo_make_install', -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35112?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: I58ca082339f3aff813f587f4c2be9c0951b9b2dd Gerrit-Change-Number: 35112 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[M] Change in osmo-dev[master]: Support building open5gs
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35114?usp=email ) Change subject: Support building open5gs ...................................................................... Support building open5gs Change-Id: Id145443ff53f13740167e9dde6417b7e915b8399 --- A all.buildsystems M all.deps M all.urls M gen_makefile.py 4 files changed, 82 insertions(+), 11 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/all.buildsystems b/all.buildsystems new file mode 100644 index 0000000..fc1784b --- /dev/null +++ b/all.buildsystems @@ -0,0 +1,3 @@ +# Projects with buildsystems other than autotools +# project buildsystem +open5gs meson diff --git a/all.deps b/all.deps index 4769d02..7857f16 100644 --- a/all.deps +++ b/all.deps @@ -36,6 +36,8 @@ osmo-e1d libosmocore osmo-remsim simtrace2_host libosmo-netif +open5gs + # can only clone these docker-playground osmo-ttcn3-hacks diff --git a/all.urls b/all.urls index 27aaf6e..dfc8ae7 100644 --- a/all.urls +++ b/all.urls @@ -2,3 +2,4 @@ # project url libnftnl https://git.netfilter.org/libnftnl/ nftables https://git.netfilter.org/nftables/ +open5gs https://github.com/open5gs/open5gs diff --git a/gen_makefile.py b/gen_makefile.py index 82ebef0..d306364 100755 --- a/gen_makefile.py +++ b/gen_makefile.py @@ -54,6 +54,7 @@ topdir = os.path.dirname(os.path.realpath(__file__)) all_deps_file = os.path.join(topdir, "all.deps") all_urls_file = os.path.join(topdir, "all.urls") +all_buildsystems_file = os.path.join(topdir, "all.buildsystems") parser = argparse.ArgumentParser(epilog=__doc__, formatter_class=argparse.RawTextHelpFormatter) parser.add_argument('configure_opts_files', @@ -152,8 +153,8 @@ l.append((tokens[0], tokens[1:])) return l -def read_projects_urls(path): - 'Read urls config and return dict {project_name: url, …}.' +def read_projects_dict(path): + 'Read urls/buildsystems config and return dict {project_name: url, …}.' ret = {} for line in open(path): line = line.strip() @@ -215,7 +216,9 @@ ''' def gen_makefile_autoconf(proj, src_proj, distclean_cond): - return f''' + buildsystem = projects_buildsystems.get(proj, "autotools") + if buildsystem == "autotools": + return f''' .make.{proj}.autoconf: .make.{proj}.clone {src_proj}/configure.ac if {distclean_cond}; then $(MAKE) {proj}-distclean; fi @echo -e "\\n\\n\\n===== $@\\n" @@ -223,11 +226,18 @@ cd {src_proj}; autoreconf -fi sync touch $@ - ''' + ''' + elif buildsystem == "meson": + return "" + else: + assert False, f"unknown buildsystem: {buildsystem}" + def gen_makefile_configure(proj, deps_installed, distclean_cond, build_proj, cflags, docker_cmd, build_to_src, configure_opts): - return f''' + buildsystem = projects_buildsystems.get(proj, "autotools") + if buildsystem == "autotools": + return f''' .make.{proj}.configure: .make.{proj}.autoconf {deps_installed} $({proj}_configure_files) if {distclean_cond}; then $(MAKE) {proj}-distclean .make.{proj}.autoconf; fi @echo -e "\\n\\n\\n===== $@\\n" @@ -237,29 +247,74 @@ cd {build_proj}; {cflags}{docker_cmd}{build_to_src}/configure {configure_opts} sync touch $@ - ''' + ''' + elif buildsystem == "meson": + return f''' +.make.{proj}.configure: .make.{proj}.clone {deps_installed} $({proj}_configure_files) + @echo -e "\\n\\n\\n===== $@\\n" + -chmod -R ug+w {build_proj} + -rm -rf {build_proj} + mkdir -p {build_proj} + cd {build_proj}; {cflags}{docker_cmd}meson {build_to_src} . + sync + touch $@ + ''' + else: + assert False, f"unknown buildsystem: {buildsystem}" def gen_makefile_build(proj, distclean_cond, build_proj, docker_cmd, jobs, check): - return f''' + buildsystem = projects_buildsystems.get(proj, "autotools") + + if buildsystem == "autotools": + return f''' .make.{proj}.build: .make.{proj}.configure $({proj}_files) if {distclean_cond}; then $(MAKE) {proj}-distclean .make.{proj}.configure; fi @echo -e "\\n\\n\\n===== $@\\n" {docker_cmd}$(MAKE) -C {build_proj} -j {jobs} {check} sync touch $@ - ''' + ''' + elif buildsystem == "meson": + target = "test" if check else "compile" + test_line = "" + # TODO: currently tests don't pass in this env + # if check: + # test_line = f"{docker_cmd}meson test -C {build_proj} -v" + return f''' +.make.{proj}.build: .make.{proj}.configure $({proj}_files) + @echo -e "\\n\\n\\n===== $@\\n" + {docker_cmd}meson compile -C {build_proj} -j {jobs} + {test_line} + sync + touch $@ + ''' + else: + assert False, f"unknown buildsystem: {buildsystem}" def gen_makefile_install(proj, docker_cmd, sudo_make_install, build_proj, no_ldconfig, sudo_ldconfig): - return f''' + buildsystem = projects_buildsystems.get(proj, "autotools") + if buildsystem == "autotools": + return f''' .make.{proj}.install: .make.{proj}.build @echo -e "\\n\\n\\n===== $@\\n" {docker_cmd}{sudo_make_install}$(MAKE) -C {build_proj} install {no_ldconfig}{sudo_ldconfig}ldconfig sync touch $@ - ''' + ''' + elif buildsystem == "meson": + return f''' +.make.{proj}.install: .make.{proj}.build + @echo -e "\\n\\n\\n===== $@\\n" + {docker_cmd}{sudo_make_install}ninja -C {build_proj} install + {no_ldconfig}{sudo_ldconfig}ldconfig + sync + touch $@ + ''' + else: + assert False, f"unknown buildsystem: {buildsystem}" def gen_makefile_reinstall(proj, deps_reinstall, sudo_make_install, build_proj): @@ -372,7 +427,8 @@ projects_deps = read_projects_deps(all_deps_file) -projects_urls = read_projects_urls(all_urls_file) +projects_urls = read_projects_dict(all_urls_file) +projects_buildsystems = read_projects_dict(all_buildsystems_file) configure_opts = listdict() configure_opts_files = sorted(args.configure_opts_files or []) for configure_opts_file in configure_opts_files: -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35114?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: Id145443ff53f13740167e9dde6417b7e915b8399 Gerrit-Change-Number: 35114 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[M] Change in osmo-dev[master]: gen_makefile: split gen_make into more functions
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35113?usp=email ) Change subject: gen_makefile: split gen_make into more functions ...................................................................... gen_makefile: split gen_make into more functions Add separate functions for each rule (autoconf, configure, build, install, reinstall, clean, distclean). This is in preparation for supporting open5gs (meson). Change-Id: Id7acfec7f5dd0d3d1a080b3c9d15005e51863e66 --- M gen_makefile.py 1 file changed, 122 insertions(+), 63 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/gen_makefile.py b/gen_makefile.py index 7fc2bd2..82ebef0 100755 --- a/gen_makefile.py +++ b/gen_makefile.py @@ -214,14 +214,88 @@ touch $@ ''' +def gen_makefile_autoconf(proj, src_proj, distclean_cond): + return f''' +.make.{proj}.autoconf: .make.{proj}.clone {src_proj}/configure.ac + if {distclean_cond}; then $(MAKE) {proj}-distclean; fi + @echo -e "\\n\\n\\n===== $@\\n" + -rm -f {src_proj}/.version + cd {src_proj}; autoreconf -fi + sync + touch $@ + ''' + +def gen_makefile_configure(proj, deps_installed, distclean_cond, build_proj, + cflags, docker_cmd, build_to_src, configure_opts): + return f''' +.make.{proj}.configure: .make.{proj}.autoconf {deps_installed} $({proj}_configure_files) + if {distclean_cond}; then $(MAKE) {proj}-distclean .make.{proj}.autoconf; fi + @echo -e "\\n\\n\\n===== $@\\n" + -chmod -R ug+w {build_proj} + -rm -rf {build_proj} + mkdir -p {build_proj} + cd {build_proj}; {cflags}{docker_cmd}{build_to_src}/configure {configure_opts} + sync + touch $@ + ''' + +def gen_makefile_build(proj, distclean_cond, build_proj, docker_cmd, jobs, + check): + return f''' +.make.{proj}.build: .make.{proj}.configure $({proj}_files) + if {distclean_cond}; then $(MAKE) {proj}-distclean .make.{proj}.configure; fi + @echo -e "\\n\\n\\n===== $@\\n" + {docker_cmd}$(MAKE) -C {build_proj} -j {jobs} {check} + sync + touch $@ + ''' + +def gen_makefile_install(proj, docker_cmd, sudo_make_install, build_proj, + no_ldconfig, sudo_ldconfig): + return f''' +.make.{proj}.install: .make.{proj}.build + @echo -e "\\n\\n\\n===== $@\\n" + {docker_cmd}{sudo_make_install}$(MAKE) -C {build_proj} install + {no_ldconfig}{sudo_ldconfig}ldconfig + sync + touch $@ + ''' + +def gen_makefile_reinstall(proj, deps_reinstall, sudo_make_install, + build_proj): + return f''' +.PHONY: {proj}-reinstall +{proj}-reinstall: {deps_reinstall} + {sudo_make_install}$(MAKE) -C {build_proj} install + ''' + +def gen_makefile_clean(proj, build_proj): + return f''' +.PHONY: {proj}-clean +{proj}-clean: + @echo -e "\\n\\n\\n===== $@\\n" + -chmod -R ug+w {build_proj} + -rm -rf {build_proj} + -rm -rf .make.{proj}.* + ''' + +def gen_makefile_distclean(proj, src_proj): + return f''' +.PHONY: {proj}-distclean +{proj}-distclean: {proj}-clean + @echo -e "\\n\\n\\n===== $@\\n" + $(MAKE) -C {src_proj} distclean + ''' + def gen_make(proj, deps, configure_opts, jobs, make_dir, src_dir, build_dir, url, push_url, sudo_make_install, no_ldconfig, ldconfig_without_sudo, make_check): src_proj = os.path.join(src_dir, proj) if proj == 'openbsc': src_proj = os.path.join(src_proj, 'openbsc') - build_proj = os.path.join(build_dir, proj) - make_to_build_proj = os.path.relpath(build_proj, make_dir) + build_proj = os.path.join(build_dir, proj) build_to_src = os.path.relpath(src_proj, build_proj) + build_proj = os.path.relpath(build_proj, make_dir) + src = os.path.relpath(src_dir, make_dir) src_proj = os.path.relpath(src_proj, make_dir) push_url = push_url or url @@ -231,6 +305,16 @@ else: configure_opts_str = '' + distclean_cond = f'[ -e {src_proj}/config.status ]' if args.auto_distclean else 'false' + deps_installed = ' '.join(['.make.%s.install' % d for d in deps]) + deps_reinstall = ' '.join(['%s-reinstall' %d for d in deps]) + cflags = 'CFLAGS=-g ' if args.build_debug else '' + docker_cmd = f'{args.docker_cmd} ' if args.docker_cmd else '' + check = 'check' if make_check else '' + no_ldconfig = '#' if no_ldconfig else '' + sudo_ldconfig = '' if ldconfig_without_sudo else 'sudo ' + sudo_make_install = 'sudo ' if sudo_make_install else '' + return r''' ### {proj} ### @@ -251,77 +335,39 @@ {clone_rule} -.make.{proj}.autoconf: .make.{proj}.clone {src_proj}/configure.ac - if {distclean_cond}; then $(MAKE) {proj}-distclean; fi - @echo -e "\n\n\n===== $@\n" - -rm -f {src_proj}/.version - cd {src_proj}; autoreconf -fi - sync - touch $@ - -.make.{proj}.configure: .make.{proj}.autoconf {deps_installed} $({proj}_configure_files) - if {distclean_cond}; then $(MAKE) {proj}-distclean .make.{proj}.autoconf; fi - @echo -e "\n\n\n===== $@\n" - -chmod -R ug+w {build_proj} - -rm -rf {build_proj} - mkdir -p {build_proj} - cd {build_proj}; {cflags}{docker_cmd}{build_to_src}/configure {configure_opts} - sync - touch $@ +{autoconf_rule} -.make.{proj}.build: .make.{proj}.configure $({proj}_files) - if {distclean_cond}; then $(MAKE) {proj}-distclean .make.{proj}.configure; fi - @echo -e "\n\n\n===== $@\n" - {docker_cmd}$(MAKE) -C {build_proj} -j {jobs} {check} - sync - touch $@ +{configure_rule} -.make.{proj}.install: .make.{proj}.build - @echo -e "\n\n\n===== $@\n" - {docker_cmd}{sudo_make_install}$(MAKE) -C {build_proj} install - {no_ldconfig}{sudo_ldconfig}ldconfig - sync - touch $@ +{build_rule} + +{install_rule} + +{reinstall_rule} + +{clean_rule} + +{distclean_rule} .PHONY: {proj} {proj}: .make.{proj}.install -.PHONY: {proj}-reinstall -{proj}-reinstall: {deps_reinstall} - {sudo_make_install}$(MAKE) -C {build_proj} install - -.PHONY: {proj}-clean -{proj}-clean: - @echo -e "\n\n\n===== $@\n" - -chmod -R ug+w {build_proj} - -rm -rf {build_proj} - -rm -rf .make.{proj}.* - -.PHONY: {proj}-distclean -{proj}-distclean: {proj}-clean - @echo -e "\n\n\n===== $@\n" - $(MAKE) -C {src_proj} distclean - '''.format( - url=url, - push_url=push_url, proj=proj, - jobs=jobs, - src=src, src_proj=src_proj, - build_proj=make_to_build_proj, - build_to_src=build_to_src, clone_rule=gen_makefile_clone(proj, src, src_proj, url, push_url), - deps_installed=' '.join(['.make.%s.install' % d for d in deps]), - deps_reinstall=' '.join(['%s-reinstall' %d for d in deps]), - configure_opts=configure_opts_str, - sudo_make_install='sudo ' if sudo_make_install else '', - no_ldconfig='#' if no_ldconfig else '', - sudo_ldconfig='' if ldconfig_without_sudo else 'sudo ', - check='check' if make_check else '', - docker_cmd=f'{args.docker_cmd} ' if args.docker_cmd else '', - cflags='CFLAGS=-g ' if args.build_debug else '', - distclean_cond=f'[ -e {src_proj}/config.status ]' if args.auto_distclean else 'false' + autoconf_rule=gen_makefile_autoconf(proj, src_proj, distclean_cond), + configure_rule=gen_makefile_configure(proj, deps_installed, distclean_cond, + build_proj, cflags, docker_cmd, + build_to_src, configure_opts_str), + build_rule=gen_makefile_build(proj, distclean_cond, build_proj, docker_cmd, + jobs, check), + install_rule=gen_makefile_install(proj, docker_cmd, sudo_make_install, + build_proj, no_ldconfig, sudo_ldconfig), + reinstall_rule=gen_makefile_reinstall(proj, deps_reinstall, + sudo_make_install, build_proj), + clean_rule=gen_makefile_clean(proj, build_proj), + distclean_rule=gen_makefile_distclean(proj, src_proj), ) -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35113?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: Id7acfec7f5dd0d3d1a080b3c9d15005e51863e66 Gerrit-Change-Number: 35113 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[M] Change in osmo-dev[master]: ttcn3: add ggsn-ogs
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35115?usp=email ) Change subject: ttcn3: add ggsn-ogs ...................................................................... ttcn3: add ggsn-ogs Change-Id: Id84047e56a6c07fbf0984eb6aedcecd4d7f3ae97 --- M .gitignore M gen_makefile.py M ttcn3/scripts/docker_configure_make.sh M ttcn3/ttcn3.sh 4 files changed, 82 insertions(+), 25 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/.gitignore b/.gitignore index 2e88651..406a33b 100644 --- a/.gitignore +++ b/.gitignore @@ -20,6 +20,7 @@ ttcn3/make/ ttcn3/.run.sh ttcn3/usr_local +ttcn3/var_local current_log lastlog autolog diff --git a/gen_makefile.py b/gen_makefile.py index d306364..3f6a994 100755 --- a/gen_makefile.py +++ b/gen_makefile.py @@ -364,7 +364,7 @@ deps_installed = ' '.join(['.make.%s.install' % d for d in deps]) deps_reinstall = ' '.join(['%s-reinstall' %d for d in deps]) cflags = 'CFLAGS=-g ' if args.build_debug else '' - docker_cmd = f'{args.docker_cmd} ' if args.docker_cmd else '' + docker_cmd = f'OSMODEV_PROJECT={proj} {args.docker_cmd} ' if args.docker_cmd else '' check = 'check' if make_check else '' no_ldconfig = '#' if no_ldconfig else '' sudo_ldconfig = '' if ldconfig_without_sudo else 'sudo ' diff --git a/ttcn3/scripts/docker_configure_make.sh b/ttcn3/scripts/docker_configure_make.sh index 7a79ff1..e6857fd 100755 --- a/ttcn3/scripts/docker_configure_make.sh +++ b/ttcn3/scripts/docker_configure_make.sh @@ -4,16 +4,60 @@ # location inside the docker container. A usr_local dir is mounted to # /usr/local, so 'make install' can put all files there and following builds # have the files available. +# env vars: OSMODEV_PROJECT DIR_OSMODEV="$(readlink -f "$(dirname $0)/../..")" DIR_MAKE="$DIR_OSMODEV/ttcn3/make" DIR_USR_LOCAL="$DIR_OSMODEV/ttcn3/usr_local" +DIR_VAR_LOCAL="$DIR_OSMODEV/ttcn3/var_local" DIR_CCACHE="$DIR_OSMODEV/ttcn3/ccache/osmocom-programs" RUN_SCRIPT="$DIR_OSMODEV/ttcn3/.run.sh" -DOCKER_IMG="$1" UID="$(id -u)" -shift -mkdir -p "$DIR_MAKE" "$DIR_CCACHE" +# Osmocom libraries and programs relevant for the current testsuite will be +# built in this container. It must have all build dependencies available and +# be based on the same distribution that master-* containers are based on, so +# there are no incompatibilities with shared libraries. +DOCKER_IMG_BUILD="debian-bookworm-build" +DOCKER_IMG_BUILD_OGS="open5gs-master" + +docker_image_exists() { + test -n "$(docker images -q "$1")" +} + +build_docker_img() { + local img="$1" + local dp="${DIR_OSMODEV}/src/docker-playground" + + if ! docker_image_exists "$USER/$img"; then + echo "Building docker image: $USER/$img" + make -C "$dp/$img" + fi +} + +build_docker_imgs() { + build_docker_img "$DOCKER_IMG_BUILD" + + if [ "$OSMODEV_PROJECT" = "open5gs" ]; then + build_docker_img "$DOCKER_IMG_BUILD_OGS" + fi +} + +set_docker_img_var() { + case "$OSMODEV_PROJECT" in + open5gs) + DOCKER_IMG="$USER/$DOCKER_IMG_BUILD_OGS" + ;; + *) + DOCKER_IMG="$USER/$DOCKER_IMG_BUILD" + ;; + esac +} + +mkdir -p \ + "$DIR_MAKE" \ + "$DIR_CCACHE" \ + "$DIR_USR_LOCAL" \ + "$DIR_VAR_LOCAL" # Script running as user inside docker echo "#!/bin/sh -ex" > "$RUN_SCRIPT" @@ -24,12 +68,16 @@ echo >> "$RUN_SCRIPT" chmod +x "$RUN_SCRIPT" +build_docker_imgs +set_docker_img_var + docker run \ --rm \ -t \ -e "LD_LIBRARY_PATH=/usr/local/lib" \ -v "$DIR_OSMODEV:$DIR_OSMODEV" \ -v "$DIR_USR_LOCAL:/usr/local" \ + -v "$DIR_VAR_LOCAL:/var/local" \ -v "$RUN_SCRIPT:/tmp/run.sh:ro" \ -v "$DIR_CCACHE:/home/build/.ccache" \ "$DOCKER_IMG" \ diff --git a/ttcn3/ttcn3.sh b/ttcn3/ttcn3.sh index 3426e7b..5c3fae6 100755 --- a/ttcn3/ttcn3.sh +++ b/ttcn3/ttcn3.sh @@ -5,16 +5,11 @@ DIR_MAKE="${DIR_MAKE:-${DIR_OSMODEV}/ttcn3/make}" DIR_OUTPUT="${DIR_OUTPUT:-${DIR_OSMODEV}/ttcn3/out}" DIR_USR_LOCAL="$DIR_OSMODEV/ttcn3/usr_local" +DIR_VAR_LOCAL="$DIR_OSMODEV/ttcn3/var_local" JOBS="$(nproc)" KERNEL_DIR="" KERNEL_SKIP_MARKER="$DIR_MAKE/.kernel_built_from_source" ARG_TEST_NAME="" - -# Osmocom libraries and programs relevant for the current testsuite will be -# built in this container. It must have all build dependencies available and -# be based on the same distribution that master-* containers are based on, so -# there are no incompatibilities with shared libraries. -DOCKER_IMG_BUILD="debian-bookworm-build" DOCKER_IMG_TITAN="debian-bookworm-titan" clean() { @@ -29,7 +24,8 @@ rm -rf \ "$DIR_MAKE" \ "$DIR_OUTPUT" \ - "$DIR_USR_LOCAL" + "$DIR_USR_LOCAL" \ + "$DIR_VAR_LOCAL" if [ -d "$DIR_OSMODEV/src/osmo-ttcn3-hacks" ]; then make -C "$DIR_OSMODEV/src/osmo-ttcn3-hacks" clean @@ -98,6 +94,7 @@ echo " $name -d ggsn" echo " $name -k ggsn" echo " $name -k -f ggsn" + echo " $name ggsn-ogs" exit 1 ;; esac @@ -134,7 +131,7 @@ case "$PROJECT" in bsc-*) echo "$hacks/bsc" ;; bts-*) echo "$hacks/bts" ;; - ggsn) echo "$hacks/ggsn_tests" ;; + ggsn|ggsn-ogs) echo "$hacks/ggsn_tests" ;; pcu-sns) echo "$hacks/pcu" ;; *) echo "$hacks/$PROJECT" ;; esac @@ -145,6 +142,7 @@ bts-gprs) echo "BTS_Tests_GPRS.cfg" ;; bts-oml) echo "BTS_Tests_OML.cfg" ;; pcu-sns) echo "PCU_Tests_SNS.cfg" ;; + ggsn-ogs) echo "GGSN_Tests.cfg" ;; *) echo "$(get_testsuite_name).cfg" ;; esac } @@ -156,6 +154,9 @@ bsc-*) echo "$dp/ttcn3-bsc-test-$(echo "$PROJECT" | cut -d - -f 2-)" ;; + ggsn-ogs) + echo "$dp/ttcn3-ggsn-test-ogs" + ;; *) echo "$dp/ttcn3-$PROJECT-test" ;; @@ -167,6 +168,9 @@ bsc-*) echo "$USER/ttcn3-bsc-test" ;; + ggsn-ogs) + echo "$USER/ttcn3-ggsn-test" + ;; *) echo "$USER/ttcn3-$PROJECT-test" ;; @@ -185,6 +189,7 @@ case "$PROJECT" in bsc|bsc-*) echo "osmo-stp osmo-bsc osmo-bts-omldummy" ;; bts) echo "osmo-bsc osmo-bts-trx" ;; + ggsn-ogs) echo "open5gs" ;; msc) echo "osmo-stp osmo-msc" ;; pcu-sns) echo "osmo-pcu" ;; pcu) echo "osmo-pcu osmo-bsc osmo-bts-virtual" ;; @@ -218,7 +223,6 @@ cd "$DIR_OSMODEV" local docker_cmd="$DIR_OSMODEV/ttcn3/scripts/docker_configure_make.sh" - docker_cmd="$docker_cmd $USER/$DOCKER_IMG_BUILD" ./gen_makefile.py \ default.opts \ @@ -294,17 +298,6 @@ done } -prepare_docker_build_container() { - local dp="${DIR_OSMODEV}/src/docker-playground" - - if docker_image_exists "$USER/$DOCKER_IMG_BUILD"; then - return - fi - - echo "Building docker image: $USER/$DOCKER_IMG_BUILD" - make -C "$dp/$DOCKER_IMG_BUILD" -} - prepare_docker_testsuite_container() { local testsuite_image="$(get_testsuite_image)" @@ -352,6 +345,12 @@ set +x for program in $programs; do + if [ "$program" = "open5gs" ]; then + # open5gs binaries have different names (open5gs-mmed, + # etc.) + continue + fi + local repo="$(get_program_repo "$program")" local usr_local_bin="$DIR_USR_LOCAL/bin" @@ -472,6 +471,7 @@ DOCKER_ARGS="\ -e LD_LIBRARY_PATH=/usr/local/lib \ -v "$DIR_USR_LOCAL":/usr/local:ro \ + -v "$DIR_VAR_LOCAL":/var/local:ro \ -v $hacks:/osmo-ttcn3-hacks:ro \ " if [ -n "$ARG_TEST_NAME" ]; then @@ -513,7 +513,6 @@ clone_repo "docker-playground" check_dir_testsuite prepare_local_bin -prepare_docker_build_container build_extra_libraries build_osmo_programs build_kernel -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35115?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: Id84047e56a6c07fbf0984eb6aedcecd4d7f3ae97 Gerrit-Change-Number: 35115 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-dev[master]: ttcn3.sh: add arg to only run one test
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35108?usp=email ) Change subject: ttcn3.sh: add arg to only run one test ...................................................................... ttcn3.sh: add arg to only run one test Depends: docker-playground I1f45a05e089b429918096fcd64f7c02033d5a9dd Change-Id: I07a76f6c58d379608ecadfc3ba845651929f668c --- M ttcn3/ttcn3.sh 1 file changed, 23 insertions(+), 4 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/ttcn3/ttcn3.sh b/ttcn3/ttcn3.sh index c315eda..3426e7b 100755 --- a/ttcn3/ttcn3.sh +++ b/ttcn3/ttcn3.sh @@ -8,6 +8,7 @@ JOBS="$(nproc)" KERNEL_DIR="" KERNEL_SKIP_MARKER="$DIR_MAKE/.kernel_built_from_source" +ARG_TEST_NAME="" # Osmocom libraries and programs relevant for the current testsuite will be # built in this container. It must have all build dependencies available and @@ -38,8 +39,11 @@ } parse_args() { - while getopts 'hdkf' OPTION; do + while getopts 'ht:dkf' OPTION; do case "$OPTION" in + t) + ARG_TEST_NAME="$OPTARG" + ;; d) if [ -n "$KERNEL_TEST" ]; then echo "ERROR: use either -d or -k" @@ -73,11 +77,12 @@ ;; h|*) local name="$(basename $0)" - echo "usage: $name [-h] [-d|-k [-f]] PROJECT" + echo "usage: $name [-h] [-t] [-d|-k [-f]] PROJECT" echo " or: $name clean" echo echo "arguments:" echo " -h show help" + echo " -t only run the test with this name" echo echo "arguments for kernel tests:" echo " -d run kernel tests with debian kernel" @@ -89,7 +94,7 @@ echo "examples:" echo " $name bsc" echo " $name bsc-sccplite" - echo " $name hlr" + echo " $name -t TC_gsup_check_imei hlr" echo " $name -d ggsn" echo " $name -k ggsn" echo " $name -k -f ggsn" @@ -464,11 +469,15 @@ fi cd "$(get_testsuite_dir_docker)" - export DOCKER_ARGS="\ + DOCKER_ARGS="\ -e LD_LIBRARY_PATH=/usr/local/lib \ -v "$DIR_USR_LOCAL":/usr/local:ro \ -v $hacks:/osmo-ttcn3-hacks:ro \ " + if [ -n "$ARG_TEST_NAME" ]; then + DOCKER_ARGS="$DOCKER_ARGS -e TEST_NAME=$ARG_TEST_NAME" + fi + export DOCKER_ARGS export NO_LIST_OSMO_PACKAGES=1 ./jenkins.sh -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35108?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: I07a76f6c58d379608ecadfc3ba845651929f668c Gerrit-Change-Number: 35108 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0

[S] Change in osmo-dev[master]: ttcn3.sh: clean osmo-ttcn3-hacks too
by osmith 23 Nov '23

23 Nov '23

osmith has submitted this change. ( https://gerrit.osmocom.org/c/osmo-dev/+/35106?usp=email ) Change subject: ttcn3.sh: clean osmo-ttcn3-hacks too ...................................................................... ttcn3.sh: clean osmo-ttcn3-hacks too Change-Id: Id0fdf5e3f0917e56e3c6abff54a2f886367252f5 --- M ttcn3/ttcn3.sh 1 file changed, 14 insertions(+), 0 deletions(-) Approvals: osmith: Looks good to me, approved; Verified diff --git a/ttcn3/ttcn3.sh b/ttcn3/ttcn3.sh index 54b4328..007fcc7 100755 --- a/ttcn3/ttcn3.sh +++ b/ttcn3/ttcn3.sh @@ -27,6 +27,11 @@ "$DIR_MAKE" \ "$DIR_OUTPUT" \ "$DIR_USR_LOCAL" + + if [ -d "$DIR_OSMODEV/src/osmo-ttcn3-hacks" ]; then + make -C "$DIR_OSMODEV/src/osmo-ttcn3-hacks" clean + fi + exit 0 } -- To view, visit https://gerrit.osmocom.org/c/osmo-dev/+/35106?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: osmo-dev Gerrit-Branch: master Gerrit-Change-Id: Id0fdf5e3f0917e56e3c6abff54a2f886367252f5 Gerrit-Change-Number: 35106 Gerrit-PatchSet: 1 Gerrit-Owner: osmith <osmith(a)sysmocom.de> Gerrit-Reviewer: osmith <osmith(a)sysmocom.de> Gerrit-MessageType: merged

1 0