dependency artifacts for faster builds/gerrit-verifications

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

André Boddenberg dr.blobb at gmail.com
Fri May 5 15:19:35 UTC 2017


Hi Neels,

thanks for adding missing details for those who haven't participated
in our f2f conversation!

> I notice there are hashes of various dependent builds in the tar names:
>
> libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652_libsmpp34.master.cc0bcd6_openggsn.master.19e19e3.tar.gz
>
> I remember that we said to instead have each on its own -- but not saying
> we have to, just curious.

That would be great, but I don't know whether we could simply fetch
and inject e.g. "libosmo-netif.master.b4aed1e.tar.gz" to update an
artifact which is up-to-date except libosmo-netif, because
libosmo-netif depends on libosmo-abis, which depends itself on
libosmocore, correct?

If correct an artifact for libosmo-netif would need to hold its
dependencies in its name as well, because libosmo-netif may didn't
change, but its dependencies may did!?

Ive created a diagram [1] showing openBSC dependencies and the
dependencies of them to illustrate above-stated.

As I see, libsmpp34 would be the only openBSC dependency that could be
simply injected?

Additionally, I created a second diagram [2] that shows a bit of
brainstorming. In this thought experiment openBSC depends on the
following three "sub-artifacts":

  - libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652
  - libosmocore.master.33e0306_openggsn.master.19e19e3.tar.gz
  - libsmpp34.master.cc0bcd6

, which then can be extracted before triggering actual openBSC build
and/or merging all "sub-artifacts" archives to:

  - libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652_libsmpp34.master.cc0bcd6_openggsn.master.19e19e3.tar.gz


In general, I'd say such improvement is definitely worth to
investigate. Furthermore, it might be a good approach to start with
libosmocore as a first project that exposes artifact(s) for depending
projects e.g. libosmo-abis and openggsn. Such mechanism would be much
more elegant as the current, in fact right now all projects are only
carrying about their own necessary dependencies and that's probably a
bit too selfish ;)

Note: *.dot files of diagrams can be accessed by simply replacing
'png' with 'dot' in URLs.


> So the semantics are: figure out all of the hashes of all dependencies, as
> soon as one of them mismatches rebuild all and make a new single tar with
> all included. Right?

Yes.

> Also curious about how a matrix build would do it: would every matrix
> rebuild the same dependency? (We said that that's fine IIRC)

Yes, every matrix/axes-combination will build its own artifact for the
sake of a simple GC.

But at the same time it's probably a good place for further
improvements à la axes-combination share same ARTIFACT_STORE folder,
which could be easily achieved by changing line 86 in osmo-build.sh
[3] to:

$ jobName=$(echo "$JOB_NAME" | cut -d '/' -f1

...just brainstorming. =)


[1] https://blobb.me/osmocom/openBSC_allDepsAndTheirDeps.png
[2] https://blobb.me/osmocom/openBSC_smartDeps.png
[3] https://gerrit.osmocom.org/#/c/2465/6/scripts/osmo-build.sh



More information about the OpenBSC mailing list