Hi all,
as some of you may know, I am working on the MS side implementation of GPRS protocol stack. The end goal is to have something similar to srsUE (part of srsRAN, former srsLTE), but for GPRS.
In order to reduce code duplication it was decided to move some stuff from both osmo-pcu.git and osmo-sgsn.git into shared libraries, so that we could re-use existing code in the upcoming MS side implementation.
== What is already done
* Redmine: https://osmocom.org/projects/libosmo-gprs * Gerrit: https://gerrit.osmocom.org/q/libosmo-gprs * Gitea: https://gitea.osmocom.org/osmocom/libosmo-gprs * osmo-ci.git: https://gerrit.osmocom.org/c/osmo-ci/+/29019
== What is missing
* GitHub: https://github.com/osmocom/libosmo-gprs * cgit: https://cgit.osmocom.org/libosmo-gprs/ ** Coverity pulls from this mirror ** Jenkins build verification uses this mirror
I will need some help with both GitHub and cgit, as I don't have admin permissions. Please also let me know if I missed something.
Best regards, Vadim.
Hi Vadim,
On Fri, Aug 12, 2022 at 03:32:48AM +0700, Vadim Yanitskiy wrote:
now at https://github.com/osmocom/libosmo-gprs
now at https://cgit.osmocom.org/libosmo-gprs/
** Coverity pulls from this mirror
this should be fixed. Coverity should use gerrit directly, or gitea.
** Jenkins build verification uses this mirror
this should be fixed. Jenkins jobs should use gerrit directly, or gitea. I think I did start a patch to address this some time ago.
Hi Harald,
thank you!
On 8/12/22 15:12, Harald Welte wrote:
Cloning via https:// works, but doesn't via git://
$ git clone git://git.osmocom.org/libosmo-abis Cloning into 'libosmo-gprs'... fatal: remote error: access denied or repository not exported: /libosmo-gprs.git
$ git clone https://git.osmocom.org/libosmo-gprs Cloning into 'libosmo-gprs'... Fetching objects: 193, done.
Best regards, Vadim.
Hi Vadim,
On Fri, Aug 12, 2022 at 04:38:42PM +0700, Vadim Yanitskiy wrote:
Cloning via https:// works, but doesn't via git://
some people would consider that a feature... In fact we've even had a bug report years ago considering clone possibility via git:// a bug.
I think a good first step would be to not enable it for new projects, to avoid people from creating clones that would break if we eventually disable it.
Regards, Harald
Hi Harald,
On 8/12/22 18:26, Harald Welte wrote:
On Fri, Aug 12, 2022 at 04:38:42PM +0700, Vadim Yanitskiy wrote:
Cloning via https:// works, but doesn't via git://
some people would consider that a feature... In fact we've even had a bug report years ago considering clone possibility via git:// a bug.
I think a good first step would be to not enable it for new projects, to avoid people from creating clones that would break if we eventually disable it.
this way we need to invest some additional time into git-to-https migration, because, as I mentioned earlier, both Coverity [1] and Jenkins [2] are still using git://. And currently both are failing when trying to clone libosmo-gprs.git, see:
[1] https://jenkins.osmocom.org/jenkins/job/osmocom-coverity/1698/ [2] https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-pcu/2430/
The main problem with using Gitea is that all projects are grouped: we have 'osmocom', 'cellular-infrastructure', 'phone-side', and more. This is why we cannot simply do s#git://git#https://gitea#.
To overcome this problem, we could clone/pull from Gerrit instead. But this is not going to work for projects not present in Gerrit... :/
Best regards, Vadim.
On 8/13/22 02:17, Vadim Yanitskiy wrote:
The main problem with using Gitea is that all projects are grouped: we have 'osmocom', 'cellular-infrastructure', 'phone-side', and more. This is why we cannot simply do s#git://git#https://gitea#.
To overcome this problem, we could clone/pull from Gerrit instead. But this is not going to work for projects not present in Gerrit... :/
Oh, wait. I am complicating things for no reason. Of course we can still clone/pull from git.osmocom.org via https://, so I can simply do s#git://git.osmocom#https://git.osmocom#. I'll submit patches soon.
Best regards, Vadim.
Hi Harald,
On 8/13/22 02:53, Vadim Yanitskiy wrote:
Oh, wait. I am complicating things for no reason. Of course we can still clone/pull from git.osmocom.org via https://, so I can simply do s#git://git.osmocom#https://git.osmocom#.%C2%A0 I'll submit patches soon.
I found your https://gerrit.osmocom.org/c/osmo-ci/+/28349 and fixed problems found during code review. Oliver gave CR+2, so I merged it.
Now osmo-ci.git is using https:// instead of git://, so libosmo-gprs.git is now accessible by Jenkins jobs. But we're not doing good yet.
== cgit is out of sync
Today I noticed that cgit mirror of libosmo-gprs.git is out of sync with Gerrit/Gitea:
$ git ls-remote https://git.osmocom.org/libosmo-gprs HEAD 11cbba9d9dd3423959adc6cd9200fe1ce9afd227 HEAD
$ git ls-remote https://gerrit.osmocom.org/libosmo-gprs HEAD 147c95f09f5c222d8e1384d131781d8ae4cbd3d8 HEAD
git ls-remote https://gitea.osmocom.org/osmocom/libosmo-gprs HEAD 147c95f09f5c222d8e1384d131781d8ae4cbd3d8 HEAD
The sync worked until recently: I merged several changes via Gerrit, and they do appear at https://cgit.osmocom.org/libosmo-gprs/log/.
Is there anything I could do to troubleshoot this?
== Sporadic git-clone failures on Jenkins
Another problem is that after the git-to-https migration, I started to observe sporadic git-clone failures in master Jenkins jobs like:
+ git clone https://git.osmocom.org/libosmo-abis libosmo-abis Cloning into 'libosmo-abis'... error: (curl_result = 56, http_code = 200, sha1 = 6498826eae07a02be1afc8e4f161bd1caa626e7d) error: Unable to find 6498826eae07a02be1afc8e4f161bd1caa626e7d under https://git.osmocom.org/libosmo-abis Cannot obtain needed tree 6498826eae07a02be1afc8e4f161bd1caa626e7d while processing commit e58d33153dd2bed3629b9a09fd6add58f296bd6a. error: fetch failed.
+ git clone https://git.osmocom.org/libosmo-sccp libosmo-sccp Cloning into 'libosmo-sccp'... error: HTTP/2 stream 0 was closed cleanly, but before getting all response header fields, treated as error (curl_result = 92, http_code = 0, sha1 = 80f1e4bfb9c161c76f04443bd03b8a2c3ce7dbd5) error: HTTP/2 stream 0 was closed cleanly, but before getting all response header fields, treated as error (curl_result = 92, http_code = 0, sha1 = 7be9bdd1632efe63b9af2d42dd355eda7883747c) error: Unable to find 7be9bdd1632efe63b9af2d42dd355eda7883747c under https://git.osmocom.org/libosmo-sccp Cannot obtain needed blob 7be9bdd1632efe63b9af2d42dd355eda7883747c while processing commit e7c8067df026fd294af2c95fbd2ceb16eeab7d74. error: fetch failed.
Looks like our cgit instance is unreliable when accessed via https://.
Best regards, Vadim.
I propose we clone from gerrit for now: https://gerrit.osmocom.org/c/osmo-ci/+/29153
On 8/18/22 09:39, Vadim Yanitskiy wrote:
Hi Harald,
On 8/13/22 02:53, Vadim Yanitskiy wrote:
Oh, wait. I am complicating things for no reason. Of course we can still clone/pull from git.osmocom.org via https://, so I can simply do s#git://git.osmocom#https://git.osmocom#.%C2%A0 I'll submit patches soon.
I found your https://gerrit.osmocom.org/c/osmo-ci/+/28349 and fixed problems found during code review. Oliver gave CR+2, so I merged it.
Now osmo-ci.git is using https:// instead of git://, so libosmo-gprs.git is now accessible by Jenkins jobs. But we're not doing good yet.
== cgit is out of sync
Today I noticed that cgit mirror of libosmo-gprs.git is out of sync with Gerrit/Gitea:
$ git ls-remote https://git.osmocom.org/libosmo-gprs HEAD 11cbba9d9dd3423959adc6cd9200fe1ce9afd227 HEAD
$ git ls-remote https://gerrit.osmocom.org/libosmo-gprs HEAD 147c95f09f5c222d8e1384d131781d8ae4cbd3d8 HEAD
git ls-remote https://gitea.osmocom.org/osmocom/libosmo-gprs HEAD 147c95f09f5c222d8e1384d131781d8ae4cbd3d8 HEAD
The sync worked until recently: I merged several changes via Gerrit, and they do appear at https://cgit.osmocom.org/libosmo-gprs/log/.
Is there anything I could do to troubleshoot this?
== Sporadic git-clone failures on Jenkins
Another problem is that after the git-to-https migration, I started to observe sporadic git-clone failures in master Jenkins jobs like:
- git clone https://git.osmocom.org/libosmo-abis libosmo-abis
Cloning into 'libosmo-abis'... error: (curl_result = 56, http_code = 200, sha1 = 6498826eae07a02be1afc8e4f161bd1caa626e7d) error: Unable to find 6498826eae07a02be1afc8e4f161bd1caa626e7d under https://git.osmocom.org/libosmo-abis Cannot obtain needed tree 6498826eae07a02be1afc8e4f161bd1caa626e7d while processing commit e58d33153dd2bed3629b9a09fd6add58f296bd6a. error: fetch failed.
- git clone https://git.osmocom.org/libosmo-sccp libosmo-sccp
Cloning into 'libosmo-sccp'... error: HTTP/2 stream 0 was closed cleanly, but before getting all response header fields, treated as error (curl_result = 92, http_code = 0, sha1 = 80f1e4bfb9c161c76f04443bd03b8a2c3ce7dbd5) error: HTTP/2 stream 0 was closed cleanly, but before getting all response header fields, treated as error (curl_result = 92, http_code = 0, sha1 = 7be9bdd1632efe63b9af2d42dd355eda7883747c) error: Unable to find 7be9bdd1632efe63b9af2d42dd355eda7883747c under https://git.osmocom.org/libosmo-sccp Cannot obtain needed blob 7be9bdd1632efe63b9af2d42dd355eda7883747c while processing commit e7c8067df026fd294af2c95fbd2ceb16eeab7d74. error: fetch failed.
Looks like our cgit instance is unreliable when accessed via https://.
Best regards, Vadim.
On Thu, Aug 18, 2022 at 11:48:05AM +0200, Oliver Smith wrote:
I propose we clone from gerrit for now: https://gerrit.osmocom.org/c/osmo-ci/+/29153
Cloning from gerrit also makes sense because there is no delay between pushing/merging and the repos being updated.
But, I understand the idea behind gitea was that users can create their own repositories. If we use gerrit as the git source for CI, we deny all user created repositories the possibility of using osmo-ci infrastructire.
So I'm not sure that we want gerrit to be regarded as the primary repository; though it de-facto already is for most active projects. (I have nostalgia for the times when git.osmocom.org was the only place for repositories.)
I'm not sure what the intention is at the moment, i.e. which is the one definitive primary git:
Is it gitea.osmocom.org? Those are mostly mirrors of gerrit. There is often a delay between merging patches (gerrit) and seeing them here.
Is it git.osmocom.org? Served by cgit and labeled "Legacy mirror" There is often a delay between merging patches (gerrit) and seeing them here.
Is it gerrit.osmocom.org? Not all projects are there. Weird Java implementation of git, though works for all intents and purposes.
Definitely not github/gitlab.
~N
-
Harald Welte -
Neels Hofmeyr -
Oliver Smith -
Vadim Yanitskiy