Hi Neels,
thanks for adding missing details for those who haven't participated in our f2f conversation!
I notice there are hashes of various dependent builds in the tar names:
libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652_libsmpp34.master.cc0bcd6_openggsn.master.19e19e3.tar.gz
I remember that we said to instead have each on its own -- but not saying we have to, just curious.
That would be great, but I don't know whether we could simply fetch and inject e.g. "libosmo-netif.master.b4aed1e.tar.gz" to update an artifact which is up-to-date except libosmo-netif, because libosmo-netif depends on libosmo-abis, which depends itself on libosmocore, correct?
If correct an artifact for libosmo-netif would need to hold its dependencies in its name as well, because libosmo-netif may didn't change, but its dependencies may did!?
Ive created a diagram [1] showing openBSC dependencies and the dependencies of them to illustrate above-stated.
As I see, libsmpp34 would be the only openBSC dependency that could be simply injected?
Additionally, I created a second diagram [2] that shows a bit of brainstorming. In this thought experiment openBSC depends on the following three "sub-artifacts":
- libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652 - libosmocore.master.33e0306_openggsn.master.19e19e3.tar.gz - libsmpp34.master.cc0bcd6
, which then can be extracted before triggering actual openBSC build and/or merging all "sub-artifacts" archives to:
- libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652_libsmpp34.master.cc0bcd6_openggsn.master.19e19e3.tar.gz
In general, I'd say such improvement is definitely worth to investigate. Furthermore, it might be a good approach to start with libosmocore as a first project that exposes artifact(s) for depending projects e.g. libosmo-abis and openggsn. Such mechanism would be much more elegant as the current, in fact right now all projects are only carrying about their own necessary dependencies and that's probably a bit too selfish ;)
Note: *.dot files of diagrams can be accessed by simply replacing 'png' with 'dot' in URLs.
So the semantics are: figure out all of the hashes of all dependencies, as soon as one of them mismatches rebuild all and make a new single tar with all included. Right?
Yes.
Also curious about how a matrix build would do it: would every matrix rebuild the same dependency? (We said that that's fine IIRC)
Yes, every matrix/axes-combination will build its own artifact for the sake of a simple GC.
But at the same time it's probably a good place for further improvements à la axes-combination share same ARTIFACT_STORE folder, which could be easily achieved by changing line 86 in osmo-build.sh [3] to:
$ jobName=$(echo "$JOB_NAME" | cut -d '/' -f1
...just brainstorming. =)
[1] https://blobb.me/osmocom/openBSC_allDepsAndTheirDeps.png [2] https://blobb.me/osmocom/openBSC_smartDeps.png [3] https://gerrit.osmocom.org/#/c/2465/6/scripts/osmo-build.sh