commitlog ML improvements

This is merely a historical archive of years 2008-2021, before the migration to mailman3.

A maintained and still updated list archive can be found at https://lists.osmocom.org/hyperkitty/list/OpenBSC@lists.osmocom.org/.

Alexander Huemer alexander.huemer at xx.vu
Sat Nov 12 09:41:11 UTC 2011


Hi all,

I personally would like to keep track of the changes that happen in most
of the osmocom git repositories. Generally there are currently two ways
to do that.

a)
The atom feed that cgit offers, e.g.
http://cgit.osmocom.org/cgit/osmocom-bb/atom/?h=jolly/testing
cgit does not offer this feature in a way firefox would recognize, but
it works. Atom feeds are quite convenient, but in the way this works ATM
one has to opt-in explicitly to every branch one wants to "follow".
Newly created branches are not advertised, ...

b)
The commitlog ML, which does not have any of the drawbacks of the cgit
atom feed, but does not show the actual patch, in case there is one,
only the revision hash.

There are several thinkable ways to bring together the best of both
systems, IMHO the easiest thing to do is modify the way the commitlog ML
composes its emails.
The example post-receive-email scripts of git (1.7.8rc1) contains the
following comment:

# hooks.showrev
#   The shell command used to format each revision in the email, with
#   "%s" replaced with the commit id.  Defaults to "git rev-list -1
#   --pretty %s", displaying the commit id, author, date and log
#   message.  To list full patches separated by a blank line, you
#   could set this to "git show -C %s; echo".
#   To list a gitweb/cgit URL *and* a full patch for each change set,
#   use this:
#     "t=%s; printf 'http://.../?id=%%s' \$t; echo;echo; git show -C
#     \$t; echo"
#   Be careful if "..." contains things that will be expanded by shell
#   "eval"
#   or printf.

The whole file is attached.
As stated on IRC, I think pasting the patches in the emails would be
most convenient, must takes up quite some space in mailboxes over time.
Of course that problem would be addressable with rm and there is
gmane/tin, ...
In any case, an added cgit link does not harm and also provides an easy
way to read the patches.

Any comments and ideas how to do a similar RSS/atom feed welcome!

Kind regards
-Alex

-------------- next part --------------
#!/bin/sh
#
# Copyright (c) 2007 Andy Parkins
#
# An example hook script to mail out commit update information.  This hook
# sends emails listing new revisions to the repository introduced by the
# change being reported.  The rule is that (for branch updates) each commit
# will appear on one email and one email only.
#
# This hook is stored in the contrib/hooks directory.  Your distribution
# will have put this somewhere standard.  You should make this script
# executable then link to it in the repository you would like to use it in.
# For example, on debian the hook is stored in
# /usr/share/git-core/contrib/hooks/post-receive-email:
#
#  chmod a+x post-receive-email
#  cd /path/to/your/repository.git
#  ln -sf /usr/share/git-core/contrib/hooks/post-receive-email hooks/post-receive
#
# This hook script assumes it is enabled on the central repository of a
# project, with all users pushing only to it and not between each other.  It
# will still work if you don't operate in that style, but it would become
# possible for the email to be from someone other than the person doing the
# push.
#
# To help with debugging and use on pre-v1.5.1 git servers, this script will
# also obey the interface of hooks/update, taking its arguments on the
# command line.  Unfortunately, hooks/update is called once for each ref.
# To avoid firing one email per ref, this script just prints its output to
# the screen when used in this mode.  The output can then be redirected if
# wanted.
#
# Config
# ------
# hooks.mailinglist
#   This is the list that all pushes will go to; leave it blank to not send
#   emails for every ref update.
# hooks.announcelist
#   This is the list that all pushes of annotated tags will go to.  Leave it
#   blank to default to the mailinglist field.  The announce emails lists
#   the short log summary of the changes since the last annotated tag.
# hooks.envelopesender
#   If set then the -f option is passed to sendmail to allow the envelope
#   sender address to be set
# hooks.emailprefix
#   All emails have their subjects prefixed with this prefix, or "[SCM]"
#   if emailprefix is unset, to aid filtering
# hooks.showrev
#   The shell command used to format each revision in the email, with
#   "%s" replaced with the commit id.  Defaults to "git rev-list -1
#   --pretty %s", displaying the commit id, author, date and log
#   message.  To list full patches separated by a blank line, you
#   could set this to "git show -C %s; echo".
#   To list a gitweb/cgit URL *and* a full patch for each change set, use this:
#     "t=%s; printf 'http://.../?id=%%s' \$t; echo;echo; git show -C \$t; echo"
#   Be careful if "..." contains things that will be expanded by shell "eval"
#   or printf.
# hooks.emailmaxlines
#   The maximum number of lines that should be included in the generated
#   email body. If not specified, there is no limit.
#   Lines beyond the limit are suppressed and counted, and a final
#   line is added indicating the number of suppressed lines.
# hooks.diffopts
#   Alternate options for the git diff-tree invocation that shows changes.
#   Default is "--stat --summary --find-copies-harder". Add -p to those
#   options to include a unified diff of changes in addition to the usual
#   summary output.
#
# Notes
# -----
# All emails include the headers "X-Git-Refname", "X-Git-Oldrev",
# "X-Git-Newrev", and "X-Git-Reftype" to enable fine tuned filtering and
# give information for debugging.
#

# ---------------------------- Functions

#
# Function to prepare for email generation. This decides what type
# of update this is and whether an email should even be generated.
#
prep_for_email()
{
	# --- Arguments
	oldrev=$(git rev-parse $1)
	newrev=$(git rev-parse $2)
	refname="$3"
	maxlines=$4

	# --- Interpret
	# 0000->1234 (create)
	# 1234->2345 (update)
	# 2345->0000 (delete)
	if expr "$oldrev" : '0*$' >/dev/null
	then
		change_type="create"
	else
		if expr "$newrev" : '0*$' >/dev/null
		then
			change_type="delete"
		else
			change_type="update"
		fi
	fi

	# --- Get the revision types
	newrev_type=$(git cat-file -t $newrev 2> /dev/null)
	oldrev_type=$(git cat-file -t "$oldrev" 2> /dev/null)
	case "$change_type" in
	create|update)
		rev="$newrev"
		rev_type="$newrev_type"
		;;
	delete)
		rev="$oldrev"
		rev_type="$oldrev_type"
		;;
	esac

	# The revision type tells us what type the commit is, combined with
	# the location of the ref we can decide between
	#  - working branch
	#  - tracking branch
	#  - unannoted tag
	#  - annotated tag
	case "$refname","$rev_type" in
		refs/tags/*,commit)
			# un-annotated tag
			refname_type="tag"
			short_refname=${refname##refs/tags/}
			;;
		refs/tags/*,tag)
			# annotated tag
			refname_type="annotated tag"
			short_refname=${refname##refs/tags/}
			# change recipients
			if [ -n "$announcerecipients" ]; then
				recipients="$announcerecipients"
			fi
			;;
		refs/heads/*,commit)
			# branch
			refname_type="branch"
			short_refname=${refname##refs/heads/}
			;;
		refs/remotes/*,commit)
			# tracking branch
			refname_type="tracking branch"
			short_refname=${refname##refs/remotes/}
			echo >&2 "*** Push-update of tracking branch, $refname"
			echo >&2 "***  - no email generated."
			return 1
			;;
		*)
			# Anything else (is there anything else?)
			echo >&2 "*** Unknown type of update to $refname ($rev_type)"
			echo >&2 "***  - no email generated"
			return 1
			;;
	esac

	# Check if we've got anyone to send to
	if [ -z "$recipients" ]; then
		case "$refname_type" in
			"annotated tag")
				config_name="hooks.announcelist"
				;;
			*)
				config_name="hooks.mailinglist"
				;;
		esac
		echo >&2 "*** $config_name is not set so no email will be sent"
		echo >&2 "*** for $refname update $oldrev->$newrev"
		return 1
	fi

	return 0
}

#
# Top level email generation function.  This calls the appropriate
# body-generation routine after outputting the common header.
#
# Note this function doesn't actually generate any email output, that is
# taken care of by the functions it calls:
#  - generate_email_header
#  - generate_create_XXXX_email
#  - generate_update_XXXX_email
#  - generate_delete_XXXX_email
#  - generate_email_footer
#
# Note also that this function cannot 'exit' from the script; when this
# function is running (in hook script mode), the send_mail() function
# is already executing in another process, connected via a pipe, and
# if this function exits without, whatever has been generated to that
# point will be sent as an email... even if nothing has been generated.
#
generate_email()
{
	# Email parameters
	# The email subject will contain the best description of the ref
	# that we can build from the parameters
	describe=$(git describe $rev 2>/dev/null)
	if [ -z "$describe" ]; then
		describe=$rev
	fi

	generate_email_header

	# Call the correct body generation function
	fn_name=general
	case "$refname_type" in
	"tracking branch"|branch)
		fn_name=branch
		;;
	"annotated tag")
		fn_name=atag
		;;
	esac

	if [ -z "$maxlines" ]; then
		generate_${change_type}_${fn_name}_email
	else
		generate_${change_type}_${fn_name}_email | limit_lines $maxlines
	fi

	generate_email_footer
}

generate_email_header()
{
	# --- Email (all stdout will be the email)
	# Generate header
	cat <<-EOF
	To: $recipients
	Subject: ${emailprefix}$projectdesc $refname_type $short_refname ${change_type}d. $describe
	X-Git-Refname: $refname
	X-Git-Reftype: $refname_type
	X-Git-Oldrev: $oldrev
	X-Git-Newrev: $newrev

	This is an automated email from the git hooks/post-receive script. It was
	generated because a ref change was pushed to the repository containing
	the project "$projectdesc".

	The $refname_type, $short_refname has been ${change_type}d
	EOF
}

generate_email_footer()
{
	SPACE=" "
	cat <<-EOF


	hooks/post-receive
	--${SPACE}
	$projectdesc
	EOF
}

# --------------- Branches

#
# Called for the creation of a branch
#
generate_create_branch_email()
{
	# This is a new branch and so oldrev is not valid
	echo "        at  $newrev ($newrev_type)"
	echo ""

	echo $LOGBEGIN
	show_new_revisions
	echo $LOGEND
}

#
# Called for the change of a pre-existing branch
#
generate_update_branch_email()
{
	# Consider this:
	#   1 --- 2 --- O --- X --- 3 --- 4 --- N
	#
	# O is $oldrev for $refname
	# N is $newrev for $refname
	# X is a revision pointed to by some other ref, for which we may
	#   assume that an email has already been generated.
	# In this case we want to issue an email containing only revisions
	# 3, 4, and N.  Given (almost) by
	#
	#  git rev-list N ^O --not --all
	#
	# The reason for the "almost", is that the "--not --all" will take
	# precedence over the "N", and effectively will translate to
	#
	#  git rev-list N ^O ^X ^N
	#
	# So, we need to build up the list more carefully.  git rev-parse
	# will generate a list of revs that may be fed into git rev-list.
	# We can get it to make the "--not --all" part and then filter out
	# the "^N" with:
	#
	#  git rev-parse --not --all | grep -v N
	#
	# Then, using the --stdin switch to git rev-list we have effectively
	# manufactured
	#
	#  git rev-list N ^O ^X
	#
	# This leaves a problem when someone else updates the repository
	# while this script is running.  Their new value of the ref we're
	# working on would be included in the "--not --all" output; and as
	# our $newrev would be an ancestor of that commit, it would exclude
	# all of our commits.  What we really want is to exclude the current
	# value of $refname from the --not list, rather than N itself.  So:
	#
	#  git rev-parse --not --all | grep -v $(git rev-parse $refname)
	#
	# Get's us to something pretty safe (apart from the small time
	# between refname being read, and git rev-parse running - for that,
	# I give up)
	#
	#
	# Next problem, consider this:
	#   * --- B --- * --- O ($oldrev)
	#          \
	#           * --- X --- * --- N ($newrev)
	#
	# That is to say, there is no guarantee that oldrev is a strict
	# subset of newrev (it would have required a --force, but that's
	# allowed).  So, we can't simply say rev-list $oldrev..$newrev.
	# Instead we find the common base of the two revs and list from
	# there.
	#
	# As above, we need to take into account the presence of X; if
	# another branch is already in the repository and points at some of
	# the revisions that we are about to output - we don't want them.
	# The solution is as before: git rev-parse output filtered.
	#
	# Finally, tags: 1 --- 2 --- O --- T --- 3 --- 4 --- N
	#
	# Tags pushed into the repository generate nice shortlog emails that
	# summarise the commits between them and the previous tag.  However,
	# those emails don't include the full commit messages that we output
	# for a branch update.  Therefore we still want to output revisions
	# that have been output on a tag email.
	#
	# Luckily, git rev-parse includes just the tool.  Instead of using
	# "--all" we use "--branches"; this has the added benefit that
	# "remotes/" will be ignored as well.

	# List all of the revisions that were removed by this update, in a
	# fast-forward update, this list will be empty, because rev-list O
	# ^N is empty.  For a non-fast-forward, O ^N is the list of removed
	# revisions
	fast_forward=""
	rev=""
	for rev in $(git rev-list $newrev..$oldrev)
	do
		revtype=$(git cat-file -t "$rev")
		echo "  discards  $rev ($revtype)"
	done
	if [ -z "$rev" ]; then
		fast_forward=1
	fi

	# List all the revisions from baserev to newrev in a kind of
	# "table-of-contents"; note this list can include revisions that
	# have already had notification emails and is present to show the
	# full detail of the change from rolling back the old revision to
	# the base revision and then forward to the new revision
	for rev in $(git rev-list $oldrev..$newrev)
	do
		revtype=$(git cat-file -t "$rev")
		echo "       via  $rev ($revtype)"
	done

	if [ "$fast_forward" ]; then
		echo "      from  $oldrev ($oldrev_type)"
	else
		#  1. Existing revisions were removed.  In this case newrev
		#     is a subset of oldrev - this is the reverse of a
		#     fast-forward, a rewind
		#  2. New revisions were added on top of an old revision,
		#     this is a rewind and addition.

		# (1) certainly happened, (2) possibly.  When (2) hasn't
		# happened, we set a flag to indicate that no log printout
		# is required.

		echo ""

		# Find the common ancestor of the old and new revisions and
		# compare it with newrev
		baserev=$(git merge-base $oldrev $newrev)
		rewind_only=""
		if [ "$baserev" = "$newrev" ]; then
			echo "This update discarded existing revisions and left the branch pointing at"
			echo "a previous point in the repository history."
			echo ""
			echo " * -- * -- N ($newrev)"
			echo "            \\"
			echo "             O -- O -- O ($oldrev)"
			echo ""
			echo "The removed revisions are not necessarilly gone - if another reference"
			echo "still refers to them they will stay in the repository."
			rewind_only=1
		else
			echo "This update added new revisions after undoing existing revisions.  That is"
			echo "to say, the old revision is not a strict subset of the new revision.  This"
			echo "situation occurs when you --force push a change and generate a repository"
			echo "containing something like this:"
			echo ""
			echo " * -- * -- B -- O -- O -- O ($oldrev)"
			echo "            \\"
			echo "             N -- N -- N ($newrev)"
			echo ""
			echo "When this happens we assume that you've already had alert emails for all"
			echo "of the O revisions, and so we here report only the revisions in the N"
			echo "branch from the common base, B."
		fi
	fi

	echo ""
	if [ -z "$rewind_only" ]; then
		echo "Those revisions listed above that are new to this repository have"
		echo "not appeared on any other notification email; so we list those"
		echo "revisions in full, below."

		echo ""
		echo $LOGBEGIN
		show_new_revisions

		# XXX: Need a way of detecting whether git rev-list actually
		# outputted anything, so that we can issue a "no new
		# revisions added by this update" message

		echo $LOGEND
	else
		echo "No new revisions were added by this update."
	fi

	# The diffstat is shown from the old revision to the new revision.
	# This is to show the truth of what happened in this change.
	# There's no point showing the stat from the base to the new
	# revision because the base is effectively a random revision at this
	# point - the user will be interested in what this revision changed
	# - including the undoing of previous revisions in the case of
	# non-fast-forward updates.
	echo ""
	echo "Summary of changes:"
	git diff-tree $diffopts $oldrev..$newrev
}

#
# Called for the deletion of a branch
#
generate_delete_branch_email()
{
	echo "       was  $oldrev"
	echo ""
	echo $LOGEND
	git show -s --pretty=oneline $oldrev
	echo $LOGEND
}

# --------------- Annotated tags

#
# Called for the creation of an annotated tag
#
generate_create_atag_email()
{
	echo "        at  $newrev ($newrev_type)"

	generate_atag_email
}

#
# Called for the update of an annotated tag (this is probably a rare event
# and may not even be allowed)
#
generate_update_atag_email()
{
	echo "        to  $newrev ($newrev_type)"
	echo "      from  $oldrev (which is now obsolete)"

	generate_atag_email
}

#
# Called when an annotated tag is created or changed
#
generate_atag_email()
{
	# Use git for-each-ref to pull out the individual fields from the
	# tag
	eval $(git for-each-ref --shell --format='
	tagobject=%(*objectname)
	tagtype=%(*objecttype)
	tagger=%(taggername)
	tagged=%(taggerdate)' $refname
	)

	echo "   tagging  $tagobject ($tagtype)"
	case "$tagtype" in
	commit)

		# If the tagged object is a commit, then we assume this is a
		# release, and so we calculate which tag this tag is
		# replacing
		prevtag=$(git describe --abbrev=0 $newrev^ 2>/dev/null)

		if [ -n "$prevtag" ]; then
			echo "  replaces  $prevtag"
		fi
		;;
	*)
		echo "    length  $(git cat-file -s $tagobject) bytes"
		;;
	esac
	echo " tagged by  $tagger"
	echo "        on  $tagged"

	echo ""
	echo $LOGBEGIN

	# Show the content of the tag message; this might contain a change
	# log or release notes so is worth displaying.
	git cat-file tag $newrev | sed -e '1,/^$/d'

	echo ""
	case "$tagtype" in
	commit)
		# Only commit tags make sense to have rev-list operations
		# performed on them
		if [ -n "$prevtag" ]; then
			# Show changes since the previous release
			git rev-list --pretty=short "$prevtag..$newrev" | git shortlog
		else
			# No previous tag, show all the changes since time
			# began
			git rev-list --pretty=short $newrev | git shortlog
		fi
		;;
	*)
		# XXX: Is there anything useful we can do for non-commit
		# objects?
		;;
	esac

	echo $LOGEND
}

#
# Called for the deletion of an annotated tag
#
generate_delete_atag_email()
{
	echo "       was  $oldrev"
	echo ""
	echo $LOGEND
	git show -s --pretty=oneline $oldrev
	echo $LOGEND
}

# --------------- General references

#
# Called when any other type of reference is created (most likely a
# non-annotated tag)
#
generate_create_general_email()
{
	echo "        at  $newrev ($newrev_type)"

	generate_general_email
}

#
# Called when any other type of reference is updated (most likely a
# non-annotated tag)
#
generate_update_general_email()
{
	echo "        to  $newrev ($newrev_type)"
	echo "      from  $oldrev"

	generate_general_email
}

#
# Called for creation or update of any other type of reference
#
generate_general_email()
{
	# Unannotated tags are more about marking a point than releasing a
	# version; therefore we don't do the shortlog summary that we do for
	# annotated tags above - we simply show that the point has been
	# marked, and print the log message for the marked point for
	# reference purposes
	#
	# Note this section also catches any other reference type (although
	# there aren't any) and deals with them in the same way.

	echo ""
	if [ "$newrev_type" = "commit" ]; then
		echo $LOGBEGIN
		git show --no-color --root -s --pretty=medium $newrev
		echo $LOGEND
	else
		# What can we do here?  The tag marks an object that is not
		# a commit, so there is no log for us to display.  It's
		# probably not wise to output git cat-file as it could be a
		# binary blob.  We'll just say how big it is
		echo "$newrev is a $newrev_type, and is $(git cat-file -s $newrev) bytes long."
	fi
}

#
# Called for the deletion of any other type of reference
#
generate_delete_general_email()
{
	echo "       was  $oldrev"
	echo ""
	echo $LOGEND
	git show -s --pretty=oneline $oldrev
	echo $LOGEND
}


# --------------- Miscellaneous utilities

#
# Show new revisions as the user would like to see them in the email.
#
show_new_revisions()
{
	# This shows all log entries that are not already covered by
	# another ref - i.e. commits that are now accessible from this
	# ref that were previously not accessible
	# (see generate_update_branch_email for the explanation of this
	# command)

	# Revision range passed to rev-list differs for new vs. updated
	# branches.
	if [ "$change_type" = create ]
	then
		# Show all revisions exclusive to this (new) branch.
		revspec=$newrev
	else
		# Branch update; show revisions not part of $oldrev.
		revspec=$oldrev..$newrev
	fi

	other_branches=$(git for-each-ref --format='%(refname)' refs/heads/ |
	    grep -F -v $refname)
	git rev-parse --not $other_branches |
	if [ -z "$custom_showrev" ]
	then
		git rev-list --pretty --stdin $revspec
	else
		git rev-list --stdin $revspec |
		while read onerev
		do
			eval $(printf "$custom_showrev" $onerev)
		done
	fi
}


limit_lines()
{
	lines=0
	skipped=0
	while IFS="" read -r line; do
		lines=$((lines + 1))
		if [ $lines -gt $1 ]; then
			skipped=$((skipped + 1))
		else
			printf "%s\n" "$line"
		fi
	done
	if [ $skipped -ne 0 ]; then
		echo "... $skipped lines suppressed ..."
	fi
}


send_mail()
{
	if [ -n "$envelopesender" ]; then
		/usr/sbin/sendmail -t -f "$envelopesender"
	else
		/usr/sbin/sendmail -t
	fi
}

# ---------------------------- main()

# --- Constants
LOGBEGIN="- Log -----------------------------------------------------------------"
LOGEND="-----------------------------------------------------------------------"

# --- Config
# Set GIT_DIR either from the working directory, or from the environment
# variable.
GIT_DIR=$(git rev-parse --git-dir 2>/dev/null)
if [ -z "$GIT_DIR" ]; then
	echo >&2 "fatal: post-receive: GIT_DIR not set"
	exit 1
fi

projectdesc=$(sed -ne '1p' "$GIT_DIR/description" 2>/dev/null)
# Check if the description is unchanged from it's default, and shorten it to
# a more manageable length if it is
if expr "$projectdesc" : "Unnamed repository.*$" >/dev/null
then
	projectdesc="UNNAMED PROJECT"
fi

recipients=$(git config hooks.mailinglist)
announcerecipients=$(git config hooks.announcelist)
envelopesender=$(git config hooks.envelopesender)
emailprefix=$(git config hooks.emailprefix || echo '[SCM] ')
custom_showrev=$(git config hooks.showrev)
maxlines=$(git config hooks.emailmaxlines)
diffopts=$(git config hooks.diffopts)
: ${diffopts:="--stat --summary --find-copies-harder"}

# --- Main loop
# Allow dual mode: run from the command line just like the update hook, or
# if no arguments are given then run as a hook script
if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
	# Output to the terminal in command line mode - if someone wanted to
	# resend an email; they could redirect the output to sendmail
	# themselves
	prep_for_email $2 $3 $1 && PAGER= generate_email
else
	while read oldrev newrev refname
	do
		prep_for_email $oldrev $newrev $refname || continue
		generate_email $maxlines | send_mail
	done
fi


More information about the OpenBSC mailing list