VTY tests still causing build failures

Dear all, for probably about a year (or longer) we have been putting up with VTY tests which cause builds to break under unclear circumstances. I personally believe the probability of a VTY test failing has recently increased again, and this is barely tolerable anymore. Often, rebasing/cherry-picking the given patch one or two times also doesn't work. Yet, the given patch-under-test is not even touching anything related to VTY, like In https://gerrit.osmocom.org/3899 which has failed in https://jenkins.osmocom.org/jenkins/job/OpenBSC-gerrit/2451/ and https://jenkins.osmocom.org/jenkins/job/OpenBSC-gerrit/2454/ I know Neels and others have spend already significant time in the past trying to resolve this - unsuccessfully. So I think the situation has reached a point where we should disable the vty tests, or at least the specific part of the vty tests that is known to break most frequently. I definitely want us to have *more* testing, not less. However, when the test itself is not stable yet - particularly after that much time - we cannot have that buggy test delay our development. I would vote for running those tests regularly (daily, every few hours, you name it), but not as part of the mandatory build verification for gerrit V+1. What do others think? -- - Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6)