Moving from trac to a single redmine
Hi guys, I think some of us would like to move to redmine and start using public tickets more frequently. So in case we move there are some topics to be discussed and I would like to start with a couple of them right now. Tickets: Redmine has a global linear sequence of ticket numbers. If we move from many tracs to a single redmine we can either: * not import tickets * only import from one project * deal with changing ticket numbers In terms of installations the GMR trac is broken in regard to tickets, there are some for SDR that are probably not being fixed anytime soon, baseband might be relevant and OpenBSC is unlikely to be relevant. I don't think we have ever used ticket reference in OpenBSC commit messages so in terms of OpenBSC having changing ticket numbers would not be a big deal. E.g. we could add a custom field with the old trac number? Wiki: We have external references that should be redirected to the new place. Is there any way besides maintaining a list in the apache2/nginx configuration and making redirects as we find broken references? Can we proactively manage this? Is anybody willing to come up with a script and nginx configuration for doing this? kind regards holger
Hi Holger, On Thu, Feb 04, 2016 at 08:12:19AM +0100, Holger Freyther wrote:
I think some of us would like to move to redmine and start using public tickets more frequently. So in case we move there are some topics to be discussed and I would like to start with a couple of them right now.
Thanks for getting the public discussion about this started. To give some more background to the mailing list: * The trac installations at osmocom.org are pretty much underused/dormant. The only part that's used is the Wiki. * Osmocom started when a single project (OpenBSC) got a sister project (OsmocomBB) and has grown into many projects. Running a single trac instance for each project on a separate dns hostname is overkill. Also, as code is shifted around between libraries and programs, we'd appreciate some flexibility. * at sysmocom internally we have successfully used redmine for dozens of different projects. The project hierarchy can be changed as needed on the fly, and issues can relate to issues of other projects, shifted from project to project, etc. * Quite a bit of the work we do at sysmocom on the Osmocom software should have the issue tracker for bugs and features in the public, but as our internal redmine is so much easier than the public trac setup, we kept using the internal redmine. So my plan moving forward is to migrate all Osmocom projects (initially those related to GSM) to a public redmine, and then keep all issues updated there. This would give more visibility into the work we're doing, such as the EDGE PCU, the 3G NITB + SGSN, the HNB-GW, etc.
Redmine has a global linear sequence of ticket numbers. If we move from many tracs to a single redmine we can either:
* not import tickets * only import from one project * deal with changing ticket numbers
I think not importing tickets or dealing wih changing numbers is the way to go.
In terms of installations the GMR trac is broken in regard to tickets, there are some for SDR that are probably not being fixed anytime soon, baseband might be relevant and OpenBSC is unlikely to be relevant. I don't think we have ever used ticket reference in OpenBSC commit messages so in terms of OpenBSC having changing ticket numbers would not be a big deal. E.g. we could add a custom field with the old trac number?
If there is automatic import/conversion available, I'd prefer to import the OsmocomBB, SIMtrace, (non-spam) Security and OpenBSC tickets, even though most of them are probably stale and outdated for years. They're still part of the history. Changing the numbers doesn't matter, as we don't refer to them.
We have external references that should be redirected to the new place. Is there any way besides maintaining a list in the apache2/nginx configuration and making redirects as we find broken references? Can we proactively manage this? Is anybody willing to come up with a script and nginx configuration for doing this?
I'm not aware of any tools that might be able to help here. Indeed, it would be great if anyone would volunteer to generate a script to generate the redirects. I guess the old format is e.g. http://openbsc.osmocom.org/trac/wiki/nanoBTS/Internals and the new URL would be something like http://projects.osmocom.org/redmine/openbsc/wiki/nanoBTS/Internals Or should we strip even the redmine from the URL? And should we have a rewrite for http://openbsc.osmocom.org/redmine to http://projects.osmocom.org/redmine/openbsc ? Any ideas? Regards, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
Hi,
I think some of us would like to move to redmine and start using public tickets more frequently. So in case we move there are some topics to be discussed and I would like to start with a couple of them right now.
+1 for redmine.
In terms of installations the GMR trac is broken in regard to tickets, there are some for SDR that are probably not being fixed anytime soon, baseband might be relevant and OpenBSC is unlikely to be relevant. I don't think we have ever used ticket reference in OpenBSC commit messages so in terms of OpenBSC having changing ticket numbers would not be a big deal. E.g. we could add a custom field with the old trac number?
If there is automatic import/conversion available, I'd prefer to import the OsmocomBB, SIMtrace, (non-spam) Security and OpenBSC tickets, even though most of them are probably stale and outdated for years. They're still part of the history. Changing the numbers doesn't matter, as we don't refer to them.
- OsmocomBB never used tickets AFAIK. - OsmocomGMR definitely never used them - OsmocomSDR ... don't think it's worth importing. - Security ... yeah maybe, although not really sure anyone is really using this.
We have external references that should be redirected to the new place. Is there any way besides maintaining a list in the apache2/nginx configuration and making redirects as we find broken references? Can we proactively manage this? Is anybody willing to come up with a script and nginx configuration for doing this?
I'm not aware of any tools that might be able to help here.
Indeed, it would be great if anyone would volunteer to generate a script to generate the redirects.
script ? The page names should be the same, so it should pretty much be just a few redirects per project with regexp to redirect properly. Shouldn't be too hard once we settled for a URL scheme. Any reason not to go for just http://osmocom.org/ as a base ? We have nothing really in the front page and we could just let redmine handle it too. Then just http://osmocom.org/projectname/wiki/page/name looks good to me if doable. Cheers, Sylvain
Hi Holger, let's move ahead and start with the migration, I can't wait to put all the various tickets into a public redmine. Thanks! -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
On 18 Feb 2016, at 19:43, Harald Welte <laforge@gnumonks.org> wrote:
Hi Holger,
Hi all!
let's move ahead and start with the migration, I can't wait to put all the various tickets into a public redmine.
last weekend I used the "stock" redmine trac->redmine converter and the result is on projects.osmocom.org. It is not perfect but I think good enough to start moving. We see that certain formating needs a post-import correction but that should be doable by all of us quickly. My plan/timeline (on short notice but I don't expect a major resistence so I hope it is fine): * Patch the redmine importer to not import tickets. This hopefully allows me to import other tracs at the same time (and re-add the security tickets by hand) * Friday evening set the tracs to read-only (I probably move the account file away) * Saturday/Sunday do the migration of the trac. * Make it available on osmocom.org (currently it is projects.osmocom.org) * Keep the trac as read-only version for now In the mid-term: * We need to post-process some pages and move them around to have a better structure * Add permanent redirects from the trac to osmocom.org for "approved" content. This way normal traffic will automatically move to the new page and then we can look at the access.log for which pages still need a re-direct. any objections? holger
On Thu, Feb 18, 2016 at 07:52:55PM +0100, Holger Freyther wrote:
last weekend I used the "stock" redmine trac->redmine converter and the result is on projects.osmocom.org. It is not perfect but I think good enough to start moving. We see that certain formating needs a post-import correction but that should be doable by all of us quickly.
I swiftly registered a 'neels' account and as soon as it's granted project memberships, I'd start off by making the overview png visible inline on the OpenBSC wiki page (unless we can automate those somehow). nitpick: I notice that in redmine, the project name "Openbsc" should probably be capitalized ("OpenBSC")?
* Saturday/Sunday do the migration of the trac.
was I too quick then? I'll recreate my account again if necessary.
any objections?
no problem, since we're all idling around all the time anyway ;) More seriously though, the wiki is on my todo list to explain gtphub and adjust that overview png, also to include Iu. Some day not too far I hope... ~Neels -- - Neels Hofmeyr <nhofmeyr@sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschäftsführer / Managing Directors: Holger Freyther, Harald Welte
On 19 Feb 2016, at 00:16, Neels Hofmeyr <nhofmeyr@sysmocom.de> wrote:
nitpick: I notice that in redmine, the project name "Openbsc" should probably be capitalized ("OpenBSC")?
The redmine importer insists only works if I enter "openbsc" and then it apparently does "Openbsc" with it. But we can fix that after the import.
* Saturday/Sunday do the migration of the trac.
was I too quick then? I'll recreate my account again if necessary.
I think we do not need to remove accounts. We might have to fix the email address of contributors that got their account automatically created as part of the import.
any objections?
no problem, since we're all idling around all the time anyway ;) More seriously though, the wiki is on my todo list to explain gtphub and adjust that overview png, also to include Iu. Some day not too far I hope...
The wiki conversion is not that great. We really have to do some work in post processing and making sure that all content has been migrated. E.g. "Software/Overview" and "SoftwareOverview" seem to confuse the importer: http://projects.osmocom.org/projects/baseband/wiki/SoftwareOverview vs. http://bb.osmocom.org/trac/wiki/Software/Overview vs. http://bb.osmocom.org/trac/wiki/SoftwareOverview So there will be some fallout but I think it still makes sense to move forward now. holger
On 19 Feb 2016, at 08:09, Holger Freyther <holger@freyther.de> wrote:
The wiki conversion is not that great. We really have to do some work in post processing and making sure that all content has been migrated. E.g. "Software/Overview" and "SoftwareOverview" seem to confuse the importer:
http://projects.osmocom.org/projects/baseband/wiki/SoftwareOverview vs. http://bb.osmocom.org/trac/wiki/Software/Overview vs. http://bb.osmocom.org/trac/wiki/SoftwareOverview
I am fixing this but if somebody wants to help with some code/regexp it would be nice too. In trac we have: [[Image(motorola_filter_replacement_step_1_low.jpg, 250px)]] For redmine we need: {{thumbnail(motorola_filter_replacement_step_1_low.jpg, size=250)}} Anyone wants to try his luck with a regexp to do this automatically? Or a bit of ruby code to split and manipulate if it is matching [[Image?
According to Google, this should do the trick perl -p -i -e 's/\[\[Image\((.+?)\)\]\]/\{\{thumbnail\($1\)\}\}/g' filename.txt (note that it replaces [[Image...]] with {{thumbnail...}} in the file without creating a backup)). /Jens E 2016-02-19 21:00 GMT+01:00 Holger Freyther <holger@freyther.de>:
On 19 Feb 2016, at 08:09, Holger Freyther <holger@freyther.de> wrote:
The wiki conversion is not that great. We really have to do some work in post processing and making sure that all content has been migrated. E.g. "Software/Overview" and "SoftwareOverview" seem to confuse the importer:
http://projects.osmocom.org/projects/baseband/wiki/SoftwareOverview vs. http://bb.osmocom.org/trac/wiki/Software/Overview vs. http://bb.osmocom.org/trac/wiki/SoftwareOverview
I am fixing this but if somebody wants to help with some code/regexp it would be nice too.
In trac we have:
[[Image(motorola_filter_replacement_step_1_low.jpg, 250px)]]
For redmine we need:
{{thumbnail(motorola_filter_replacement_step_1_low.jpg, size=250)}}
Anyone wants to try his luck with a regexp to do this automatically? Or a bit of ruby code to split and manipulate if it is matching [[Image?
On 19 Feb 2016, at 08:09, Holger Freyther <holger@freyther.de> wrote:
The wiki conversion is not that great. We really have to do some work in post processing and making sure that all content has been migrated. E.g. "Software/Overview" and "SoftwareOverview" seem to confuse the importer:
http://projects.osmocom.org/projects/baseband/wiki/SoftwareOverview vs. http://bb.osmocom.org/trac/wiki/Software/Overview vs. http://bb.osmocom.org/trac/wiki/SoftwareOverview
So there will be some fallout but I think it still makes sense to move forward now.
This post is Sponsored by the Ministry of Stuff you Didn't want to know. Trac has a "flat" wiki model. The hierarchy is added by the notion of adding / into the path. This means that SoftwareOverview is a top-level page and Software/Overview is a child of software. In Redmine there is the concept of "parent" pages but the name is a flat namespace. So by convention the page would be called "Overview" but be a child of "Software". Redmine will normalize a pagetitle like Software/Overview to SoftwareOverview. To avoid having "SoftwareOverview" and "Software/Overview" clash I am expanding the later to SoftwareSoftwareOverview and this gives us funny names and broken links (*sigh*) but with reasonable effort this seems to be as good as I can get it. This means our manual fix-up step will so far include: * Fix general format issue (e.g. <pre> but no </pre>) * Remove custom html I created to have a multi-column view. * Fix the [[Image unless I get the regexp to work in ruby * Rename SoftwareSoftwareOverview and other pages (and delete the old one) kind regards holger
Dear all, It's been more than half a year that we transitioned the Osmocom projects from the various trac instances to redmine. Still, a lot of people end up on the old redmine pages. How do I know? Because of the amount of personal e-mail I receive from people asking for accounts or for modification of the content there. [This is of course just the indicator, I don't mind those mails]. But what worries me is that people are looking at outdated information in the old trac without even knowing. How do we proceed about that? I'm not quite clear. The first step could be to add some kind of banner to the old trac installations, indicating at the header / footer of every page (or even as a floating block visible at all times) that this is old, outdated content that is scheduled to be removed at a certain date, and indicating the top-level page for the specific project in the osmocom.org redmine as an updated source. After that grace period has passed, we should probably make all the old trac pages redirect to the entry-page of the project in redmine. Trying to link to the respective new page in each individual wiki would be overly complicated and take a lot of time, unless somebody masocistic enough would volunteer for creating such a list. It can probably be auto-generated to some extent, but then each individual item would need to be manually verified and fixed up, if needed. Do you have other ideas on how to proceed in a better way? Do we have any volunters about any of the above? I'm quite sure it wouldbe easy to provide volunteers witha copy of the existing trac databases or any other information that might be needed. Thanks in advance for any assistance, Harald -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
On 24 Sep 2016, at 14:31, Harald Welte <laforge@gnumonks.org> wrote:
Dear all,
Still, a lot of people end up on the old redmine pages. How do I know? Because of the amount of personal e-mail I receive from people asking for accounts or for modification of the content there.
eek.
How do we proceed about that? I'm not quite clear. The first step could be to add some kind of banner to the old trac installations, indicating at the header / footer of every page (or even as a floating block visible at all times) that this is old, outdated content that is scheduled to be removed at a certain date, and indicating the top-level page for the specific project in the osmocom.org redmine as an updated source. After that grace period has passed, we should probably make all the old trac pages redirect to the entry-page of the project in redmine.
Trying to link to the respective new page in each individual wiki would be overly complicated and take a lot of time, unless somebody masocistic enough would volunteer for creating such a list. It can probably be auto-generated to some extent, but then each individual item would need to be manually verified and fixed up, if needed.
Do you have other ideas on how to proceed in a better way?
* banner to say to look at our redmine * disallow search engines to index trac (if that is still allowed) * not modify trac sites but use redirect of nginx
Do we have any volunters about any of the above? I'm quite sure it wouldbe easy to provide volunteers witha copy of the existing trac databases or any other information that might be needed.
We can provide the access logs (all IP addresses point to the frontend server so we only show user agent that might be private). holger
Hi,
* not modify trac sites but use redirect of nginx
I've just put this in place for 'openbsc.osmocom.org' : rewrite ^/trac/wiki/(.*)/(.*)$ $scheme://osmocom.org/projects/openbsc/wiki/$1$2 redirect; rewrite ^/trac/wiki(.*)$ $scheme://osmocom.org/projects/openbsc/wiki$1 redirect; This seems to work fairly well. I'm sure there are some exceptions to those regexps, but I couldn't find any immediately. If we uncover them we can try to tweak them or just handle them manually. I've set them to temporary (302) redirects for now. Once we're a little more confident that this works well, this can be changed to 301s and extended to all the other old domains. If you find any case where those fails, please report. Cheers, Sylvain
I am working on fernvale-nuttx-bb (fernvale-nuttx + gnutoo's nuttx-bb) and wanted to check the osmocom bb on nuttx-bb. It seems all the links are broken in the redmine version of things and the trac URLs redirect to redmine so the content is "gone"? Can you help me resurrect pages like http://osmocom.org/projects/baseband/wiki/Nuttx-bbcompile?parent=Nuttx-bb Thanks,Craig From: Sylvain Munaut <246tnt@gmail.com> To: Holger Freyther <holger@freyther.de> Cc: "osmocom-sdr@lists.osmocom.org" <osmocom-sdr@lists.osmocom.org>; baseband-devel <baseband-devel@lists.osmocom.org>; OpenBSC Mailing List <openbsc@lists.osmocom.org>; gmr@lists.osmocom.org Sent: Saturday, September 24, 2016 9:01 AM Subject: Re: Moving from trac to a single redmine Hi,
* not modify trac sites but use redirect of nginx
I've just put this in place for 'openbsc.osmocom.org' : rewrite ^/trac/wiki/(.*)/(.*)$ $scheme://osmocom.org/projects/openbsc/wiki/$1$2 redirect; rewrite ^/trac/wiki(.*)$ $scheme://osmocom.org/projects/openbsc/wiki$1 redirect; This seems to work fairly well. I'm sure there are some exceptions to those regexps, but I couldn't find any immediately. If we uncover them we can try to tweak them or just handle them manually. I've set them to temporary (302) redirects for now. Once we're a little more confident that this works well, this can be changed to 301s and extended to all the other old domains. If you find any case where those fails, please report. Cheers, Sylvain
On Sat, Sep 24, 2016 at 08:01:52AM -0600, Sylvain Munaut wrote:
I've just put this in place for 'openbsc.osmocom.org' :
rewrite ^/trac/wiki/(.*)/(.*)$ $scheme://osmocom.org/projects/openbsc/wiki/$1$2 redirect; rewrite ^/trac/wiki(.*)$ $scheme://osmocom.org/projects/openbsc/wiki$1 redirect;
But so far the idea is to keep the trac online and browsable, right? It didn't sound like the grace period before switching off trac has already passed. And I think we already have one complaint of missing content :) ~Neels -- - Neels Hofmeyr <nhofmeyr@sysmocom.de> http://www.sysmocom.de/ ======================================================================= * sysmocom - systems for mobile communications GmbH * Alt-Moabit 93 * 10559 Berlin, Germany * Sitz / Registered office: Berlin, HRB 134158 B * Geschäftsführer / Managing Directors: Harald Welte
On Sun, Sep 25, 2016 at 05:54:10PM +0200, Neels Hofmeyr wrote:
On Sat, Sep 24, 2016 at 08:01:52AM -0600, Sylvain Munaut wrote:
I've just put this in place for 'openbsc.osmocom.org' :
rewrite ^/trac/wiki/(.*)/(.*)$ $scheme://osmocom.org/projects/openbsc/wiki/$1$2 redirect; rewrite ^/trac/wiki(.*)$ $scheme://osmocom.org/projects/openbsc/wiki$1 redirect;
But so far the idea is to keep the trac online and browsable, right?
the main problem is that search engines still primarily seem to index + link to the old trac instances. So we were keeping that alive for some time. However, as the content is going out of date, and people still show an interest in editing the old content, I think we need to switch the trac off soon and replace it with links to the redmine, even if it just links to the redmine entry page of that project. -- - Harald Welte <laforge@gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)
participants (6)
-
Craig Comstock -
Harald Welte -
Holger Freyther -
Jens Elofsson -
Neels Hofmeyr -
Sylvain Munaut