[PATCH] osmo-ttcn3-hacks[master]: support ccache to speed up ttcn3 test compilation

[Stefan Sperling]

Review at  https://gerrit.osmocom.org/7601

support ccache to speed up ttcn3 test compilation

If ccache exists in PATH, use it to cache compiled C++ object files.

Rebuilding the test suite from scratch with a populated cache
is an order of magnitude faster than doing an initial build.
In my case, compile time for BSC_Tests goes down from 3 minutes
and 12 seconds to just 3 seconds, after 'make clean'.

There is a small downside: We need to tweak the generated C++ files
to prevent cache misses due to timestemps inserted into the code
by ttcn3_compiler. This is done by regen-makefile.sh if ccache
is used. This mechanism could break in case the output of
ttcn3_makefilegen changes. However, the only consequence would
be a slow build due to cache misses.

Another thing to note is that builds using objects from ccache
will contain timestamp strings from the build which populated
the cache. ccache provides a way to repopulate the cache if
needed though (set CCACHE_RECACHE=1 in the environment).

These tradeoffs seem acceptable to me since in return we can
achieve much faster turnaround, both during local test
development and during test runs on jenkins.

Change-Id: Ibb538f602206535c06980f88191c1dabe3c4cd82
---
M regen-makefile.sh
1 file changed, 40 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.osmocom.org:29418/osmo-ttcn3-hacks refs/changes/01/7601/1

diff --git a/regen-makefile.sh b/regen-makefile.sh
index 57eebff..35e2e51 100755
--- a/regen-makefile.sh
+++ b/regen-makefile.sh
@@ -12,7 +12,22 @@
 
 test -x "$(which ttcn3_makefilegen 2>/dev/null)" || { echo "ERROR: ttcn3_makefilegen not in PATH"; exit 1; }
 
-ttcn3_makefilegen -p -l -f $*
+# Enable ccache if it can be found in path.
+# This speeds up repeated builds of the TTCN3 tests by an order of magnitude
+# since most of the generated C++ source files don't change very often.
+# Roughly, for an initial build which takes N minutes, a complete rebuild
+# after 'make clean' will only take N seconds with ccache.
+# Note that ccache cannot cache compilation of .o files to .so files, so do
+# not pass the -l option to ttcn3_makefilegen if ccache is used. The time
+# savings promised by the -l option are dwarfed by use of ccache anyway.
+if which ccache 2>/dev/null; then
+	USE_CCACHE=1
+	MAKEFILEGEN_FLAGS="-p -f"
+else
+	MAKEFILEGEN_FLAGS="-p -l -f"
+fi
+
+ttcn3_makefilegen $MAKEFILEGEN_FLAGS $*
 sed -i -e 's/# TTCN3_DIR = /TTCN3_DIR = \/usr/' Makefile
 sed -i -e 's/LDFLAGS = /LDFLAGS = -L \/usr\/lib\/titan /' Makefile
 #sed -i -e 's/TTCN3_LIB = ttcn3-parallel/TTCN3_LIB = ttcn3/' Makefile
@@ -32,3 +47,27 @@
         sed -i -e 's/TTCN3_DIR = $/TTCN3_DIR = \/usr/' Makefile
 fi
 sed -i -e 's/\/bin\/compiler/\/bin\/ttcn3_compiler/' Makefile
+
+if [ -n "$USE_CCACHE" ]; then
+	# enable ccache
+	sed -i -e 's/^CXX = g++ $/CXX = env CCACHE_SLOPPINESS=time_macros ccache g++/' Makefile
+
+	# After 'make compile', remove timestamp comments from generated C++ files.
+	#
+	# The ttcn3_compiler will generate C++ comments with a timestamp, such as:
+	#   // for Stefan Sperling (stsp at fintan) on Sat Mar 31 14:34:30 2018
+	# We must remove such comments since they cause cache misses.
+	#
+	# A related problem is the use of __DATE__ and __TIME__ in generated C++ code,
+	# which is worked around by "CCACHE_SLOPPINESS=time_macros" above. This
+	# workaround implies that we must get our cache hits from ccache's "direct"
+	# cache, since ccache's "preprocessor" caching method won't work.
+	#
+	# XXX FIXME This should really be addressed in ttcn3_compiler itself!
+	# It should provide an option to suppress use of timestamps in generated code.
+	#
+	# Now the magic `sed` command which makes ccache effective for us:
+	# The generated Makefile's compile target contains 'touch $@', and there we
+	# append a sed command that strips the offending C++ comment from the code.
+	sed -i -e '/^compile:/ { n; n; n; s!touch $@!& \&\& sed -i -e "/\\/\\/ for .* on .*/d" $(GENERATED_HEADERS) $(GENERATED_SOURCES)! }' Makefile
+fi

-- 
To view, visit https://gerrit.osmocom.org/7601
To unsubscribe, visit https://gerrit.osmocom.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibb538f602206535c06980f88191c1dabe3c4cd82
Gerrit-PatchSet: 1
Gerrit-Project: osmo-ttcn3-hacks
Gerrit-Branch: master
Gerrit-Owner: Stefan Sperling <ssperling at sysmocom.de>