I guess it's a bit late, but do we really need categories on gitea?
The simple world I would have expected:
git clone https://gitea.osmocom.org/libosmo-pfcp git clone https://gitea.osmocom.org/osmo-upf
The world we live in:
git clone https://gitea.osmocom.org/celullar_infarsturcure/libosmo-pfcp git clone https://gitea.osmocom.org/osmocom/osmo-upf git clone https://gerrit.osmocom.org/libosmo-dsp
and
works: git clone git://git.osmocom.org/osmo-upf fails: git clone https://git.osmocom.org/osmo-upf works: git clone https://git.osmocom.org/libosmo-pfcp
While I'm at it: do we have a designated primary git place for Osmocom? De facto the answer seems to be: gerrit, except when it's not on gerrit.
Because of the above issues, currently there is a wave of patches rolling out which edit build scripts and READMEs to change git URLs -- it feels wrong that this is necessary.
It seems the topic needs some clarification, and maybe it is still possible to simplify the git structures?
- remove the layer of categories from gitea? - move gitea/cellular-infrastraucute/* to gitea/osmocom/* ? - point git.osmocom.org at gitea.osmocom.org/osmocom/ ? (cgit.osmocom.org still works) - keep cgit as the primary git place, gitea as users' playgrounds?
(not sure if possible or good ideas)
~N
Hi Neels,
On Fri, Aug 19, 2022 at 04:23:01PM +0200, Neels Hofmeyr wrote:
I guess it's a bit late, but do we really need categories on gitea?
I guess you're familiar with the work ongoing for multiple months in https://osmocom.org/issues/5397 ?
To clarify, we're referring to "organizations" in the gitea/github/gitlab context. At least to us in Osmocom, they serve multiple purposes:
a) logical grouping of repositories that belong together. Makes it easy to find related projects, and helps people who are just interested in one aspect, e.g. sdr or op25, for example
b) permission management. It's much easier to manage permissions of an "organization" rather than for each and every repository. Think again of e.g. the "sdr" organization: People like steve-m or tnt can autonomously create new repositories in that organization, give new developers permissions, etc. - this is what always used to require my manual intervention in the pre-gitea past.
Please note that even before gitea we had two orthogonal ways of grouping:
1) logical grouping in the cgit UI (based on a config file) 2) sub-directory grouping, like the erlang / smalltalk / cc / sub-direcotries
The new namespace in gitea cleans that up to some extent.
The simple world I would have expected:
git clone https://gitea.osmocom.org/libosmo-pfcp git clone https://gitea.osmocom.org/osmo-upf
that flat namespace only works via the old git.osmocom.org mirror, and only for those repositories that have been configured respectively.
and
works: git clone git://git.osmocom.org/osmo-upf
that's because this is not set-up/configured. It's questionable whether we still want to care about git://
fails: git clone https://git.osmocom.org/osmo-upf
this is due to the fact that https://git.osmocom.org/ are in fact HTTP redirects to the respective gitea URLs. And of course if somebody creates a new repository without adding such a redirect to the nginx config, it won't work.
While I'm at it: do we have a designated primary git place for Osmocom? De facto the answer seems to be: gerrit, except when it's not on gerrit.
The Osmocom project is more than the subset of projects/repos any single developer is working on. Some projects (around CNI) have chosen to use gerrit because it makes sense for them. Other projects have chosen not to, and I am certain we don't want to force every sub-project to use a certain tool.
Also, gerrit makes sense for projects with large number of changes, where the group of contributors is relatively static. I certainly understand why people don't want that for a project that receives <= 1 patch per month.
Because of the above issues, currently there is a wave of patches rolling out which edit build scripts and READMEs to change git URLs -- it feels wrong that this is necessary.
There are multiple overlapping topics:
* the change away from git:// to https:// should have happened a long time ago
* automatic build verification, etc. makes sense to operate on the "root" repo of a given project (gerrit in most, gitea in other cases), rather than a mirror which might lag "whatever the mirroring interval is" behind in time.
It seems the topic needs some clarification, and maybe it is still possible to simplify the git structures?
What exactly is the problem? How often do any of us have to create a new clone of a repo? I have my 100+ cloned repos lying around. Those that used gerrit before needed no update, and only those that used gitolite before *and* to which I push needed to update the remote URL. pulling/reading the old URLs continues to work, which is what was most important for me, given that this is by far > 99% of the git utilization.
- remove the layer of categories from gitea?
This is not how (gitea,gitlab,github) work.
- move gitea/cellular-infrastraucute/* to gitea/osmocom/* ?
What would be the point of that?
* osmocom is the umbrella project of *all* the projects * CNI is a subset of certain projects within osmocom
By renaming CNI to osmocom we'd introduce wrong terminology.
The osmocom/* "organization" should IMHO only be used for stuff like libosmocore which is used by multiple projects.
- point git.osmocom.org at gitea.osmocom.org/osmocom/ ? (cgit.osmocom.org still works)
Can you please be more specific what you are referring to? As stated, git://git.osmocom.org is a read-only mirror and https://git.osmocom.org/foo/bar are HTTP 302 redirects to the respective gitea counterpart
- keep cgit as the primary git place, gitea as users' playgrounds?
What is a "primary git place"? The developers working on the related projects inevitably will have to use gerrit for projects hosted there (to push pactches for review) and gitea for projects hosted there (to generate PR)
And for regular users, as I stated, nothing should change: * if they had a git:// clone, it continues to work (they can 'git pull' without any changes) * if thyey had a https:// clone, it contiues to work, likewise * if they do a first-time clone, they just copy+paste the URL from redmine or gitea. it doesn't really matter what's inside the copy+paste buffer
btw: By coincidence, today a person showed up on IRC expressing their enthusiasm that now with gitea finally they can send pull-requests to projects like rtl-sdr.
- automatic build verification, etc. makes sense to operate on the "root" repo of a given project (gerrit in most, gitea in other cases)
There is no trivial way to always get the "root" repo of a project (the one guaranteed to have no lag) -- a script now needs prior knowledge about what repositories live in gerrit. (Merely stating facts.)
- point git.osmocom.org at gitea.osmocom.org/osmocom/ ? (cgit.osmocom.org still works)
https://git.osmocom.org/foo/bar are HTTP 302 redirects to the respective gitea counterpart
ok, that's nice. Noting though that git://git.o/foo and https://git.o/foo end up in two different repositories.
https://git.osmocom.org/ are in fact HTTP redirects to the respective gitea URLs
This is interesting: if i use a https://git.o.o/ url i reach the gitea repos without knowing the gitea subdir. or, in other words, the nginx config for HTTPS git URLs is a mapping from a flat dir structure to the various subdirs.
If I follow that tangent, the https://git.o URLs to gerrit projects could redirect to gerrit, not gitea. Then the https://git.o nginx redirections would provide the lag-free "root" repos for each project, in a flat subdir structure.
- move gitea/cellular-infrastraucute/* to gitea/osmocom/* ?
What would be the point of that?
The reason to bring it up is that libosmo-abis, -sccp, -pfcp,... might also be considered cellular infrastructure. They seem to live in osmocom/ only because they start with "lib"?
The osmocom/* "organization" should IMHO only be used for stuff like libosmocore which is used by multiple projects.
So as soon as some other project starts using one of the cell-infra projects, the cell-infra project must be moved to "osmocom"? I'm just teasing to test the reasoning...
~N
Hi Neels,
On Mon, Aug 29, 2022 at 01:16:35AM +0200, Neels Hofmeyr wrote:
- automatic build verification, etc. makes sense to operate on the "root" repo of a given project (gerrit in most, gitea in other cases)
There is no trivial way to always get the "root" repo of a project (the one guaranteed to have no lag) -- a script now needs prior knowledge about what repositories live in gerrit. (Merely stating facts.)
If we'd write (or rather generate) a gerrit replication config file for all the gerrit-hosted repositories, we could switch the gitea mirrors of those repositories from pull mirrors to push mirrors.
That way gitea.osmocom.org should be in sync all the time and could be treated as 'root' for both gerrit-root and gitea-root repositories.
- point git.osmocom.org at gitea.osmocom.org/osmocom/ ? (cgit.osmocom.org still works)
https://git.osmocom.org/foo/bar are HTTP 302 redirects to the respective gitea counterpart
ok, that's nice. Noting though that git://git.o/foo and https://git.o/foo end up in two different repositories.
that's correct, as git:// doesn't have redirects, and is probably not even supported by gitea. So if we want that backwards compatibility, it's yet another mirror.
This is interesting: if i use a https://git.o.o/ url i reach the gitea repos without knowing the gitea subdir. or, in other words, the nginx config for HTTPS git URLs is a mapping from a flat dir structure to the various subdirs.
for legacy repositories, yes.
If I follow that tangent, the https://git.o URLs to gerrit projects could redirect to gerrit, not gitea. Then the https://git.o nginx redirections would provide the lag-free "root" repos for each project, in a flat subdir structure.
I don't think we generally want non-developers to use gerrit to avoid it overloading at some point. I think it makes sense to separate the "publicly advertised repo" from the "internal repo used by active developers"
- move gitea/cellular-infrastraucute/* to gitea/osmocom/* ?
What would be the point of that?
The reason to bring it up is that libosmo-abis, -sccp, -pfcp,... might also be considered cellular infrastructure. They seem to live in osmocom/ only because they start with "lib"?
Unfortunately libosmo-abis is needed by several projects that have nothing to do with abis (and hence CNI), like osmo-remsim, for example. The reason is our complex way of spreading around IPA multiplex bits over 3 different git repositories :(
SCCP/sigtran is a regular ITU SS7 protocol not strictly tied to cellular protocols. There are other use cases, primarily in wired telepony.
So the only one that's wrong is libosmo-pfcp. I'm sorry if I created it in the wrong organization :(
The osmocom/* "organization" should IMHO only be used for stuff like libosmocore which is used by multiple projects.
So as soon as some other project starts using one of the cell-infra projects, the cell-infra project must be moved to "osmocom"? I'm just teasing to test the reasoning...
I think it's more whether or not the code inside the project implements something that is specific to cellular infrastructure or not. And that should be known from the onset and unlikely change at a later point?
On Tue, Aug 30, 2022 at 07:26:02PM +0200, Harald Welte wrote:
So the only one that's wrong is libosmo-pfcp. I'm sorry if I created it in the wrong organization :(
I think I did that, and you indicated the wrong place, and I responded "but it starts with 'lib' like the others". So that was a misunderstanding then.
Should we move it now, while we are still used to seeing build failures of PFCP related stuff?
~N
Hi,
I've added a new wiki page with an overview of the git infrastructure we now have, to help developers choose the right one when e.g. working on the jenkins scripts.
https://osmocom.org/projects/cellular-infrastructure/wiki/Git_infrastructure
I saw that the Gitea wiki page also explains the mirrors, but IMHO it's useful to mention it in this overview as well.
Best, Oliver
On 30.08.22 19:26, Harald Welte wrote:
Hi Neels,
On Mon, Aug 29, 2022 at 01:16:35AM +0200, Neels Hofmeyr wrote:
- automatic build verification, etc. makes sense to operate on the "root" repo of a given project (gerrit in most, gitea in other cases)
There is no trivial way to always get the "root" repo of a project (the one guaranteed to have no lag) -- a script now needs prior knowledge about what repositories live in gerrit. (Merely stating facts.)
If we'd write (or rather generate) a gerrit replication config file for all the gerrit-hosted repositories, we could switch the gitea mirrors of those repositories from pull mirrors to push mirrors.
That way gitea.osmocom.org should be in sync all the time and could be treated as 'root' for both gerrit-root and gitea-root repositories.
- point git.osmocom.org at gitea.osmocom.org/osmocom/ ? (cgit.osmocom.org still works)
https://git.osmocom.org/foo/bar are HTTP 302 redirects to the respective gitea counterpart
ok, that's nice. Noting though that git://git.o/foo and https://git.o/foo end up in two different repositories.
that's correct, as git:// doesn't have redirects, and is probably not even supported by gitea. So if we want that backwards compatibility, it's yet another mirror.
This is interesting: if i use a https://git.o.o/ url i reach the gitea repos without knowing the gitea subdir. or, in other words, the nginx config for HTTPS git URLs is a mapping from a flat dir structure to the various subdirs.
for legacy repositories, yes.
If I follow that tangent, the https://git.o URLs to gerrit projects could redirect to gerrit, not gitea. Then the https://git.o nginx redirections would provide the lag-free "root" repos for each project, in a flat subdir structure.
I don't think we generally want non-developers to use gerrit to avoid it overloading at some point. I think it makes sense to separate the "publicly advertised repo" from the "internal repo used by active developers"
- move gitea/cellular-infrastraucute/* to gitea/osmocom/* ?
What would be the point of that?
The reason to bring it up is that libosmo-abis, -sccp, -pfcp,... might also be considered cellular infrastructure. They seem to live in osmocom/ only because they start with "lib"?
Unfortunately libosmo-abis is needed by several projects that have nothing to do with abis (and hence CNI), like osmo-remsim, for example. The reason is our complex way of spreading around IPA multiplex bits over 3 different git repositories :(
SCCP/sigtran is a regular ITU SS7 protocol not strictly tied to cellular protocols. There are other use cases, primarily in wired telepony.
So the only one that's wrong is libosmo-pfcp. I'm sorry if I created it in the wrong organization :(
The osmocom/* "organization" should IMHO only be used for stuff like libosmocore which is used by multiple projects.
So as soon as some other project starts using one of the cell-infra projects, the cell-infra project must be moved to "osmocom"? I'm just teasing to test the reasoning...
I think it's more whether or not the code inside the project implements something that is specific to cellular infrastructure or not. And that should be known from the onset and unlikely change at a later point?
-
Harald Welte -
Neels Hofmeyr -
Oliver Smith