Hi Neels,
Patchset for openBSC's jenkins.sh build script has been uploaded [1].
But a small change [2] of osmo-build.sh is necessarily pending,
because https://git.osmocom.org is down atm. Furthermore, the
osmo-build.sh script should probably not depend on cgit's
availability. :)
After [2] has been submitted following steps are necessary to verify
[1] via gerrit:
- trigger "update-osmo-ci-on-slaves"
- add ARTIFACT_STORE environment variable to all slaves/nodes e.g. [3]
- add following arguments to docker invocations of openBSC jobs [4]:
-e JOB_NAME="$JOB_NAME"
-e ARTIFACT_STORE="/ARTIFACT_STORE"
-v "$ARTIFACT_STORE:/ARTIFACT_STORE"
Afterwards, re-triggering [1] should result in a '+1 Jenkins Builder'.
I have sufficient permission to apply above stated steps, except
+2'ing [2] and you may want to suggest the absolute path for
ARTIFACT_STORE variable?
Regards,
André
[1] https://gerrit.osmocom.org/#/c/3823/
[2] https://gerrit.osmocom.org/#/c/3822/
[3] https://jenkins.osmocom.org/jenkins/computer/OsmocomBuild1/configure
[4] https://jenkins.osmocom.org/jenkins/view/Jenkins-Gerrit/job/OpenBSC-gerrit/
Just now I only got long waits and then "502 Bad Gateway" on
git.osmocom.org ... the cgit ezjail hit its root partition limit. I had to
re-figure out what I did last time, tried to find ezjail-admin options until
finally I ended up back in zfs and started remembering... I increased the zfs
quota for the cgit jail from 10G to 15G. Now it's working again.
For the record, resizing an ezjail goes like
# zfs list
# zfs get quota tank/jails/cgit
# zfs set quota=15G tank/jails/cgit
(so next time I can find it in my mail archive, and I guess it should go in a
wiki somewhere. On osmocom.org?)
Last time it was the jenkins jail hitting disc limits. Then I said
"We should probably set a refquota so that the live file system's quota is
somewhat independent from the quota used for snapshots."
I see the jenkins jail now has a 40G refquota, but the others seem not to.
How does it work, set a huge quota to allow space for snapshots, and limit the
fs itself by a refquota?
~N
Some time back we implemented OAP to add an authentication layer to the IPA,
IIRC aimed at osmo-bsc_nat <-> MAP proxy to do milenage mutual auth.
Now the oap protocol code is in libosmocore, but the oap_client code is dup'ed
in osmo-{bsc,sgsn}.git. I would move the oap_client to libosmocore as well.
But all the while I'm not really certain that it is indeed being used.
osmo-sgsn has vty config for it, but I'm certain osmo-hlr doesn't bother
to implement the OAP server part (there's code but in #if 0).
osmo-bsc_nat remains, and possibly a GSUP counter part to the osmo-sgsn that
I'm not using (presumably the MAP proxy)?
Is anyone using OAP and the oap_client and is it worth the effort to keep it?
Can we / should we just drop OAP instead?
Thanks,
~N
I'm reading through https://osmocom.org/projects/cellular-infrastructure/wiki/Make_a_new_release
and noting things as I go...
> library vs. non-library projects
This is not a thing and should disappear from the release process. We have many
projects that build programs as well as install libraries.
I see this is about the *_LIBVERSION, right? ... maybe we can say something
like "adjust *_LIBVERSION (not required if the project does not install any
libraries)" ?
> "adhere to RERO better"
RERO -- had to look it up, Eric's quote of "Release early, release often" :)
And lol, the PDF link there refers to papers written basically by the "private
club" of my former colleagues at the Subversion project :) Eric, Justin, Hyrum,
all Subversion folks.
Interesting side fact is that Subversion has never had time-based releases,
despite one hackathon vote accepting that as policy.
One detail though is that technically we do radical Release Early, because all
of our code is on public git immediately. I'm not sure that Eric's quote refers
to actual glorified "stable" releases, I understood as "make available early".
Anyhow, this paragraph seems unrelated to the tagging process, could be related
to the time-based package feed releases (see below).
> "Proposed policy:"
We already have what I'd call a loose bunch of accepted policies after
OsmoDevCon 2017 and mails following it.
It is first of all important to note a profound difference between two things
we're aming for: providing a package feed with a global glorified state
(time based), and the individual version tags in each git (feature based).
With this in mind, recall the thread starting with:
https://lists.osmocom.org/pipermail/openbsc/2017-April/010569.html
Let me summarize:
- "Global" package feeds
We are planning to provide package feeds in a time-based fashion, by pinning
code states and providing built packages. This is mostly a service to our user
base, so that they can easily communicate and update to versions.
They are to be called osmocom-cell-net / OsmocomCellNet and will have
year+month based version numbers, YY.MM like 17.03 17.06 17.09 17.12
and, quote:
> > I suggest that we follow a regular release schedule of: at the beginning
> > of March, June, September and December. (so that we don't need to roll a
> > release during/right after the festive season, i.e. not on 1st January)
At this point we might aim at an OsmocomCellNet-17.12 package feed to be out
on 1st of December.
These feeds will not wait for features or code-freeze ahead. We will simply pin
a certain point in time. If serious bugs arise we might choose to adjust, but
the advice is to then just use the previous / wait three months for the next.
- Individual git tree version numbers
In contrast to the global package feeds, each git tree has its own local
version with a git version tag. This does not follow the year+month scheme but
rather "1.2.3" major.minor.patch.
Compatibility is reflected in these version numbers, i.e. big incompatibilities
require a major version bump, mere internal fixes can bump the patch version.
(So far I understood the wiki page "Make a new release" referring to only these
individual git repository versions, but we must clarify the two different
concepts on that page and at least refer to a package feed wiki page.)
How do the global package feeds correspond to individual major-minor-patch
versions? For each of the global feeds, we should make sure that the precise
git trees that are packaged correspond 1:1 to local git version tags. I.e.
before we roll out a package feed, we make local version number tags to glorify
the current master branch in each repository (bumping versions according to
compatibility considerations) and actually use those precise version tags to
mark the state that will go out in the OsmocomCellNet-YY.MM feed.
So, while the time-based package feeds will be one driving force behind bumping
individual version numbers, we are technically still able to mark new local
version tags here and there independently from the time-based feeds.
To avoid confusion, rather than talking of "release" it could help to talk of
"package feed" versus "git version tag".
> master branch is expected to depend on latest master branches of depended-upon projects
The master branch HEADs are in constant flux, the requirement that each master
works with the other latest masters is more of a daily development requirement
enforced by our gerrit +V builds. For feeds and tags, we should not even
mention "master branch" anywhere. Rather, the entire process *must* refer to
specific version tags *everywhere*.
But I'm not entirely sure how to put it. It is possible that we already have
libosmo-foo 1.2.3 out, but osmo-bar also still builds fine with libosmo-foo
1.2.1.
I guess all we need is that configure.ac version requirements are accurate. We
will not always have to depend on the latest version number that's out, and if
older ones work, ./configure should *not* require the most recent version.
How to verify? I guess a version tag of osmo-X must verify
- that building with the precise minimal versions requested in configure.ac
works
AND
- that building with the currently latest tagged version of each depended-upon
project also works
AND
- that building with each intermittent version of each dependency also works???
That's turning out rather complex ... maybe verifying with only the oldest
required and the latest tagged versions is enough?
> make release of depended-upon projects if necessary before making non-library project release
So, are you saying, if foo needs bar which needs baz, releasing foo requires
also releasing bar, in turn needing a release of baz first?
... "if necessary" :)
To spell it out: this is not necessarily required; only when foo is using new
API of bar that is not available in a tagged version of bar yet are we forced
to make a version tag of bar and depend on it.
BTW, this is implicit by "it must build with the version requested by
configure.ac".
The text on that wiki page before your edits did say exactly that:
>> As soon as a feature is added to one Osmocom project that is needed for
>> another dependent project to compile, we should tag at least a minor-revision
>> bump in the depended-upon project and require it in the depend[ent] project's
>> configure.ac.
IMHO that's still exactly what it should say there.
For the global package feed we choose to tag a new version everywhere so that
we can pinpoint the current state that we want to glorify, but it is not
strictly required for a local version bump to also tag depended-upon gits.
> make sure that we have correct version dependencies before making non-library project release
IOW: make sure configure.ac reflects the version number of depended-upon gits
that are the minimum requirement to build. (scratch the "non-library")
Ok great, I've reviewed the first paragraph :)
Well, there was a lot of ground to cover, I hope I can do the rest less
verbosely...
> modify *_LIBVERSION in Makefile.am as necessary according to TODO-RELEASE file
would be good to clarify what you mean, e.g. by an example.
BTW, I find RANAP_LIBVERSION in osmo-iuh, ABIS_LIBVERSION and TRAU_LIBVERSION
in libosmo-abis and LEGACY_MGCP_LIBVERSION in osmo-mgw; that's all. Do we need
a lot more *_LIBVERSION everywhere, starting with libosmocore?
Also it would be good to embed this in the list of steps under "common steps"
... i.e. have just one list of steps standing out clearly; Ideally I can
start reading at the top and follow the steps one by one.
> The release helper is trying to be smart about it and prevent making new
> library release with non-empty TODO-RELEASE file if *_LIBVERSION is not
> adjusted beforehand.
Not sure I understand. I can either empty out TODO-RELEASE or change
*_LIBVERSION? If I modify *_LIBVERSION, I can make a release despite
TODO-RELEASE being nonempty?
> Use following guidelines when choosing release type:
I'd guess at
major: Previous API is no longer available, ABI has changed incompatibly.
minor: Additions to public API/ABI are present.
patch: API/ABI unchanged, internal changes only (like bugfixes).
> Deprecation policy
I think we should never remove deprecated functions unless it would cause us
inhumane pain to still include them. Dropping them would require a major
version bump.
> TODO-RELEASE file format and maintenance
would be excellent to provide an actual example there
> How to (re)tag a new release
Is tagging and re-tagging the same? Do I need to delete a tag first? A word to
clarify that would be good. Also, above it says re-sign, is that the same
thing?
Still unclear to me is, do I first tag a version, then do 'make release', and
then I need to re-tag / re-sign again? A clear sequential list of steps and/or
a complete screen dump example would be very helpful to grok it.
[I remember OpenBSD CD sets having a complete listing of the default
installation in the CD booklet :) ]
Finally, I tried to call 'make release' but face these problems:
▶ make release
/bin/bash: bumpversion: command not found
▶ apt-cache search bumpversion
(no results)
Though I find 'bumpversion' on packages.debian.org, I am unable to install it
with 'apt-get'.
https://packages.debian.org/buster/bumpversion
It says "Packages / buster (testing) / devel / bumpversion"
...do I need to add "devel" to my apt sources somehow?
Do we really need bumpversion? Can I instead bump the version manually?
Also:
▶ make release
/bin/bash: bumpversion: command not found
osmo-release.mk:13: *** Please fix versioning to match http://semver.org/ spec (current is 1.0.1.115-f7251) before proceeding.. Stop.
The intended version should be 1.0.2.
Is that because bumpversion is not available?
(Shouldn't it abort right away when bumpversion is missing?)
I tried to also tag a version first so that git-version-gen produces a clean
semver version:
▶ cat .version
1.0.2
but still get
▶ make release
/bin/bash: bumpversion: command not found
osmo-release.mk:13: *** Please fix versioning to match http://semver.org/ spec (current is 1.0.2) before proceeding.. Stop.
(i.e. now it complains about "1.0.2" not being semver, which is inaccurate)
Can you document these pitfalls on the wiki as well, please?
~N
(P.S.: "first .. then" and "if .. then" ... it's almost always "then" with "e".)
On some systems the ALSA output buffer is pretty big, and
if the audio samples are not being passed into the buffer
quickly enough, it becomes starved for data, resulting
in an error called underrun.
Previously, when it happenned, GAPK used to stop processing
with the following message (where X is a random number):
[+] PQ: Adding ALSA output (dev='default', blk_len=320)
[!] pq_execute(): abort, item returned -1
[+] Processed X frames
According to the ALSA documentation, the pcm_handle
changes its state when the problem happens, and should
be recovered using the snd_pcm_prepare() call. This change
actually does that.
---
src/pq_alsa.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/pq_alsa.c b/src/pq_alsa.c
index 9cee426..a3435dd 100644
--- a/src/pq_alsa.c
+++ b/src/pq_alsa.c
@@ -57,7 +57,15 @@ pq_cb_alsa_output(void *_state, uint8_t *out, const uint8_t *in, unsigned int in
struct pq_state_alsa *state = _state;
unsigned int num_samples = in_len/2;
int rv;
+
rv = snd_pcm_writei(state->pcm_handle, in, num_samples);
+ if (rv == -EPIPE) {
+ /* Recover from buffer underrun */
+ snd_pcm_prepare(state->pcm_handle);
+ /* Send a new sample again */
+ rv = snd_pcm_writei(state->pcm_handle, in, num_samples);
+ }
+
return rv == num_samples ? 0 : -1;
}
--
2.14.1
I would have welcomed more patience in merging https://gerrit.osmocom.org/3685,
"Use value string check from osmo-ci" in libosmocore.
(the commit log of which fails to indicate removal of the script, btw)
Moving to the new repositories, I have enough loose ends flapping in the wind,
and dropping the script from libosmocore has added yet another distraction,
while not being necessary to start using the one from osmo-ci.
Max clearly posted "N. B: this should be merged last, after all the repos
converted to check script from osmo-ci." -- which is not the case for the new
repositores split from openbsc.git.
In osmo-msc I have >50 patches waiting to be merged. The timing could hardly
have been worse, luckily only a few gerrit builds are affected. Will try to
sort out merging of the patches without needing a rebase and >50 rebuilds.
Thanks for you attention :)
~N
Hi.
I'd like to propose to switch from local (unmaintained?) copy of git-version-gen
script and use it directly from upstream gnulib package. It's available pretty-much
everywhere (including ancient Debian which we support for nightly). It's also
available in meta-oe: https://layers.openembedded.org/layerindex/recipe/47365/
It doesn't feel right to use copy-pasted code when we could offload the maintenance
work to packaging/dependency tools.
What do you think? Please contribute to https://osmocom.org/issues/2467 which I've
created to track this RFC.
--
Max Suraev <msuraev(a)sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte
