osmo-ci[master]: Introduce artifacts holding dependencies to speed up builds.

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/gerrit-log@lists.osmocom.org/.

blobb gerrit-no-reply at lists.osmocom.org
Fri May 19 13:30:33 UTC 2017


Patch Set 7:

(6 comments)

Thanks a lot for your review!

https://gerrit.osmocom.org/#/c/2465/7/scripts/osmo-build.sh
File scripts/osmo-build.sh:

Line 49: #		# in jenkins' $WORKSPACE per default.
> like what. cd foo? ./configure && make install?
yes. Maybe a short real example of such script with comments will be more descriptive? I'll try to more descriptive.


Line 86: 	job_name="${JOB_NAME//\//#}"
> what does this do?
It replaces '/' with '#' to avoid slashes in the artifact filename. Will add comment.


Line 130: 
> locally? remotely?
remotely - getBranchAndRevRemotely() => git ls-remote ......

locally - getBranchAndRevLocally() => git rev-parse HEAD

Thanks, I'll document it and change the names to get_artifact_name_by_remote_repos and get_artifact_name_by_local_repos to be more descriptive.


Line 163: 	# be on a different partition or build runs in docker.
> you're lacking the root reason: we want to atomically move an artifact into
True, thanks! Will be added/documented.


Line 171: 		rm -f "$job_store/*"
> hmm, we might still have a GC problem here. If another job tries to use thi
True, I somehow relied on the assumption that job B, which has been triggered directly after job A (concurrent) will archive its artifact after building all deps, thus giving job A enough time to fetch the artifact. (Of course a change in one of the dependencies have been introduced between triggering job A and B.) 

Thanks, for your feedback!


Line 187: 	generateArtifactHashes "$1"
> add comment:
>> And about the GC problem above: the most waterproof 
>> way of avoiding is to probe whether the artifact 
>> exists *by* doing a copy/untar. If it succeeds, 
>> we have it. If not, rebuild.

Thanks for your input. Script will be changed to do so!



>> How likely is this to happen? What would it take? 
>> concurrent jenkins job builds of different versions of 
>> the same repos? (so far we have concurrent builds 
>> disabled mostly for cosmetic reasons)

Yes, it would take concurrent jenkins jobs of the same repo with different versions of one of its dependencies (e.g. change has been introduced to libosmocore). Furthermore the following "race condition" must be won:

     t_buildDeps < t_fetchDeps (+ t_triggerDelay)

In other words the time it takes to build all dependencies is the delay between the indication that an artifact is outdated and its deletion.


-- 
To view, visit https://gerrit.osmocom.org/2465
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifee0a2f837d23b19aa5326f810234d5452e47484
Gerrit-PatchSet: 7
Gerrit-Project: osmo-ci
Gerrit-Branch: master
Gerrit-Owner: blobb <dr.blobb at gmail.com>
Gerrit-Reviewer: Harald Welte <laforge at gnumonks.org>
Gerrit-Reviewer: Holger Freyther <holger at freyther.de>
Gerrit-Reviewer: Neels Hofmeyr <nhofmeyr at sysmocom.de>
Gerrit-Reviewer: blobb <dr.blobb at gmail.com>
Gerrit-Reviewer: neels <nhofmeyr at sysmocom.de>
Gerrit-HasComments: Yes



More information about the gerrit-log mailing list