I would suggest something like http://i.imgur.com/F20Q7.png
Where basically you have: - The osmo-bts core and all common BTS logic / A-bis interface and such - It talks to the lower L1 layer via the primitives defined in the spec for creation of channels and stuff. AFAIK it's "pretty" similar to the current DSP interface but some specific stuff might need to be made generic.
Then you have two possible L1 implementation speaking thos primitives - the sysmobts one which basically includes all the specific stuff to convert the 'generic' primitves into the specific call to the DSP.
- The TRX one which includes scheduler / channel coding internally and would use the UDP protocol used by OpenBTS This impl can itself offer a generic interface internally so that the OpenBTS UDP protocol is only one of the possible user of the scheduler / channel coding ...
Cheers,
Sylvain