New features without tests :/

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

Harald Welte laforge at gnumonks.org
Fri May 24 09:36:33 UTC 2019


Dear fellow Osmocom developers,

I just would like to use this as a gentle reminder that we shouldn't be
merging new features without having automatic tests for them in place.

What comes to my mind immediately right now (but I'm sure there are other
examples) is the "subscriber create on demand" feature.  I couldn't find
any tests in HLR_Tests.ttcn about this feature.

What this means is, that it will *eventually* break unnoticed, particularly
until somebody is using that feature and updating frequently to master,
which is unlikely to happen.  Either you run a production system and then
you don't follow master, or you're just playing around and then that
feature is probably not something you'd be using in many cases.

What actually bugs me most about it, is that the tests should have been
written first.  For most of our development [1], the existing infrastructure
in terms of TTCN3 Modules is very strong, and adding related tests is
rather quick.  This means it's *very* feasible to write the tests first
and then the actual code.  In fact, my own experience shows that development
is much faster this way, as maual testing of the entire stack with phones
is sloooow.

This is not to complain to Oliver or any single person, but just a general
reminder to all of us.  That includes first and foremost myself as I'm
merging most of the development, but is addressed to all of the developers
and reviewers:  IF something doesn't have a test, but could reasonably be
tested without spending a multitude of the development time on tests, we
should mandate that tests are available at the time of merge.

We of course cannot mandate or enforce that developers write the tests first
and follow test-driven development methodology.  But I would at least strongly
encourage everyone to try that.  If you first spend tons of time with manual
testing and then write a test case, it's much less efficient as you spend time
for both.  OTOTH, if you first invest the time into writing the test, then
the development can make very quick progress and you save a lot of time
that would otherwise be wasted with manual testing.

Regards,
	Harald

[1] I'm referring to rather self-cntained, small features.  For sure,
e.g. testing inter-MSC-HO requires lots of new tsting infrastructure to be
developed, as does testing of the PCU.  So yes, there are exception.

-- 
- Harald Welte <hwelte at sysmocom.de>             http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Alt-Moabit 93
* 10559 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Director: Harald Welte



More information about the OpenBSC mailing list