release process review

I'm reading through https://osmocom.org/projects/cellular-infrastructure/wiki/Make_a_new_release and noting things as I go... This is not a thing and should disappear from the release process. We have many projects that build programs as well as install libraries. I see this is about the *_LIBVERSION, right? ... maybe we can say something like "adjust *_LIBVERSION (not required if the project does not install any libraries)" ? RERO -- had to look it up, Eric's quote of "Release early, release often" :) And lol, the PDF link there refers to papers written basically by the "private club" of my former colleagues at the Subversion project :) Eric, Justin, Hyrum, all Subversion folks. Interesting side fact is that Subversion has never had time-based releases, despite one hackathon vote accepting that as policy. One detail though is that technically we do radical Release Early, because all of our code is on public git immediately. I'm not sure that Eric's quote refers to actual glorified "stable" releases, I understood as "make available early". Anyhow, this paragraph seems unrelated to the tagging process, could be related to the time-based package feed releases (see below). We already have what I'd call a loose bunch of accepted policies after OsmoDevCon 2017 and mails following it. It is first of all important to note a profound difference between two things we're aming for: providing a package feed with a global glorified state (time based), and the individual version tags in each git (feature based). With this in mind, recall the thread starting with: https://lists.osmocom.org/pipermail/openbsc/2017-April/010569.html Let me summarize: - "Global" package feeds We are planning to provide package feeds in a time-based fashion, by pinning code states and providing built packages. This is mostly a service to our user base, so that they can easily communicate and update to versions. They are to be called osmocom-cell-net / OsmocomCellNet and will have year+month based version numbers, YY.MM like 17.03 17.06 17.09 17.12 and, quote: At this point we might aim at an OsmocomCellNet-17.12 package feed to be out on 1st of December. These feeds will not wait for features or code-freeze ahead. We will simply pin a certain point in time. If serious bugs arise we might choose to adjust, but the advice is to then just use the previous / wait three months for the next. - Individual git tree version numbers In contrast to the global package feeds, each git tree has its own local version with a git version tag. This does not follow the year+month scheme but rather "1.2.3" major.minor.patch. Compatibility is reflected in these version numbers, i.e. big incompatibilities require a major version bump, mere internal fixes can bump the patch version. (So far I understood the wiki page "Make a new release" referring to only these individual git repository versions, but we must clarify the two different concepts on that page and at least refer to a package feed wiki page.) How do the global package feeds correspond to individual major-minor-patch versions? For each of the global feeds, we should make sure that the precise git trees that are packaged correspond 1:1 to local git version tags. I.e. before we roll out a package feed, we make local version number tags to glorify the current master branch in each repository (bumping versions according to compatibility considerations) and actually use those precise version tags to mark the state that will go out in the OsmocomCellNet-YY.MM feed. So, while the time-based package feeds will be one driving force behind bumping individual version numbers, we are technically still able to mark new local version tags here and there independently from the time-based feeds. To avoid confusion, rather than talking of "release" it could help to talk of "package feed" versus "git version tag". The master branch HEADs are in constant flux, the requirement that each master works with the other latest masters is more of a daily development requirement enforced by our gerrit +V builds. For feeds and tags, we should not even mention "master branch" anywhere. Rather, the entire process *must* refer to specific version tags *everywhere*. But I'm not entirely sure how to put it. It is possible that we already have libosmo-foo 1.2.3 out, but osmo-bar also still builds fine with libosmo-foo 1.2.1. I guess all we need is that configure.ac version requirements are accurate. We will not always have to depend on the latest version number that's out, and if older ones work, ./configure should *not* require the most recent version. How to verify? I guess a version tag of osmo-X must verify - that building with the precise minimal versions requested in configure.ac works AND - that building with the currently latest tagged version of each depended-upon project also works AND - that building with each intermittent version of each dependency also works??? That's turning out rather complex ... maybe verifying with only the oldest required and the latest tagged versions is enough? So, are you saying, if foo needs bar which needs baz, releasing foo requires also releasing bar, in turn needing a release of baz first? ... "if necessary" :) To spell it out: this is not necessarily required; only when foo is using new API of bar that is not available in a tagged version of bar yet are we forced to make a version tag of bar and depend on it. BTW, this is implicit by "it must build with the version requested by configure.ac". The text on that wiki page before your edits did say exactly that: IMHO that's still exactly what it should say there. For the global package feed we choose to tag a new version everywhere so that we can pinpoint the current state that we want to glorify, but it is not strictly required for a local version bump to also tag depended-upon gits. IOW: make sure configure.ac reflects the version number of depended-upon gits that are the minimum requirement to build. (scratch the "non-library") Ok great, I've reviewed the first paragraph :) Well, there was a lot of ground to cover, I hope I can do the rest less verbosely... would be good to clarify what you mean, e.g. by an example. BTW, I find RANAP_LIBVERSION in osmo-iuh, ABIS_LIBVERSION and TRAU_LIBVERSION in libosmo-abis and LEGACY_MGCP_LIBVERSION in osmo-mgw; that's all. Do we need a lot more *_LIBVERSION everywhere, starting with libosmocore? Also it would be good to embed this in the list of steps under "common steps" ... i.e. have just one list of steps standing out clearly; Ideally I can start reading at the top and follow the steps one by one. Not sure I understand. I can either empty out TODO-RELEASE or change *_LIBVERSION? If I modify *_LIBVERSION, I can make a release despite TODO-RELEASE being nonempty? I'd guess at major: Previous API is no longer available, ABI has changed incompatibly. minor: Additions to public API/ABI are present. patch: API/ABI unchanged, internal changes only (like bugfixes). I think we should never remove deprecated functions unless it would cause us inhumane pain to still include them. Dropping them would require a major version bump. would be excellent to provide an actual example there Is tagging and re-tagging the same? Do I need to delete a tag first? A word to clarify that would be good. Also, above it says re-sign, is that the same thing? Still unclear to me is, do I first tag a version, then do 'make release', and then I need to re-tag / re-sign again? A clear sequential list of steps and/or a complete screen dump example would be very helpful to grok it. [I remember OpenBSD CD sets having a complete listing of the default installation in the CD booklet :) ] Finally, I tried to call 'make release' but face these problems: ▶ make release /bin/bash: bumpversion: command not found ▶ apt-cache search bumpversion (no results) Though I find 'bumpversion' on packages.debian.org, I am unable to install it with 'apt-get'. https://packages.debian.org/buster/bumpversion It says "Packages / buster (testing) / devel / bumpversion" ...do I need to add "devel" to my apt sources somehow? Do we really need bumpversion? Can I instead bump the version manually? Also: ▶ make release /bin/bash: bumpversion: command not found osmo-release.mk:13: *** Please fix versioning to match http://semver.org/ spec (current is 1.0.1.115-f7251) before proceeding.. Stop. The intended version should be 1.0.2. Is that because bumpversion is not available? (Shouldn't it abort right away when bumpversion is missing?) I tried to also tag a version first so that git-version-gen produces a clean semver version: ▶ cat .version 1.0.2 but still get ▶ make release /bin/bash: bumpversion: command not found osmo-release.mk:13: *** Please fix versioning to match http://semver.org/ spec (current is 1.0.2) before proceeding.. Stop. (i.e. now it complains about "1.0.2" not being semver, which is inaccurate) Can you document these pitfalls on the wiki as well, please? ~N (P.S.: "first .. then" and "if .. then" ... it's almost always "then" with "e".)