24 Nov
2013
24 Nov
'13
7:05 p.m.
Hi,
I have added rate counters to the PCU and noticed that a lot of frames
in the DL are re-sent without there actually being any NACK or
transmission errors. I have also re-produced a "stall" by just using
ping with a big enough data size and a low enough interval (the PCU
appears to transmit the ICMP PING but the MS never gets to send a PONG
packet).
In both cases it could be a scheduling issue (e.g. we don't schedule
the UL Packet ACK/NACK). Is any of you aware of it? I had already
highlighted that the scheduling is not fair. I wonder if we are seeing
starvation (and will re-factor to be able to unit test this).
In terms of scheduling. For the SBA we are already having a reservation
that we try to honor in multiple places. Maybe it is time to extend it
and have more reservations?
holger
--
- Holger Freyther <hfreyther(a)sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Schivelbeiner Str. 5
* 10439 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Holger Freyther, Harald Welte
25 Nov
25 Nov
8:24 a.m.
Holger Hans Peter Freyther wrote:
Hi,
I have added rate counters to the PCU and noticed that a lot of frames
in the DL are re-sent without there actually being any NACK or
transmission errors.
dear holger,
i can only guess what causes the issues you describe. so i suggest that
we should do a debug session at the congress, if you attend to it and
find the time.
generally there is always a re-send due to delay between pcu and the
radio interface. if all rlc/mac data blocks of an llc frame have been
sent to the phone, a packet downlink ack control block is requested and
scheduled. the pcu repeats all unacknowledged data blocks in a loop
until it receives that requested control block. this way the pcu starts
re-sending data blocks that might got lost, before it actually knows if
and which blocks still need to be resend. (this procedure is described
in TS 04.60.)
I have also re-produced a "stall" by just
using
ping with a big enough data size and a low enough interval (the PCU
appears to transmit the ICMP PING but the MS never gets to send a PONG
packet).
could this be an mtu issue at the phone? does the phone requests an
uplink tbf to send a pong?
In both cases it could be a scheduling issue (e.g. we
don't schedule
the UL Packet ACK/NACK).
in case of an ongoing uplink tbf, we ack/nack received or
timed out
uplink rlc/mac blocks by using a control block. even if there is an
ongoing downlink tbf, all control blocks have priority over data blocks.
Is any of you aware of it? I had already
highlighted that the scheduling is not fair. I wonder if we are seeing
starvation (and will re-factor to be able to unit test this).
In terms of scheduling. For the SBA we are already having a reservation
that we try to honor in multiple places. Maybe it is time to extend it
and have more reservations?
uplink control messages are reserved in the tbf object (poll_fn). the
sba control messages are reserved in a separate structure, since sba are
not (or not yet) related to a tbf.
regards,
andreas
9:21 a.m.
On Mon, Nov 25, 2013 at 08:24:27AM +0100, Andreas Eversberg wrote:
As I have already pointed out. Some control blocks can starve. I don't
know if it does but it is where I will continue to have a look.
well. In the tbf you have:
else if (bts->sba()->find(trx->trx_no, ts, (fn + 13) % 2715648))
LOGP(DRLCMAC, LOGL_DEBUG, "Polling cannot be "
"sheduled, because single block alllocation "
"already exists\n");
But there is nothing that checks if another TBF is setting poll_fn to
a frame that another tbf is already polling.
--
- Holger Freyther <hfreyther(a)sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Schivelbeiner Str. 5
* 10439 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Holger Freyther, Harald Welte
could this be an mtu issue at the phone? does the
phone requests an
uplink tbf to send a pong?
the first 10 pings going through and then nothing coming back? I don't
think it is a MTU issue. I don't know what the root cause is but the
structure of the code doesn't make it easy to find it.
In both cases
it could be a scheduling issue (e.g. we don't schedule
the UL Packet ACK/NACK).
in case of an ongoing uplink tbf, we ack/nack received or
timed out
uplink rlc/mac blocks by using a control block. even if there is an
ongoing downlink tbf, all control blocks have priority over data blocks. Is any of you
aware of it? I had already
highlighted that the scheduling is not fair. I wonder if we are seeing
starvation (and will re-factor to be able to unit test this).
In terms of scheduling. For the SBA we are already having a reservation
that we try to honor in multiple places. Maybe it is time to extend it
and have more reservations?
uplink control messages are reserved in the tbf object (poll_fn). the
sba control messages are reserved in a separate structure, since sba are
not (or not yet) related to a tbf.
26 Nov
26 Nov
12:18 a.m.
On Mon, Nov 25, 2013 at 09:21:08AM +0100, Holger Hans Peter Freyther wrote:
As I have already pointed out. Some control blocks can
starve. I don't
know if it does but it is where I will continue to have a look.
Here is the starvation theory for the ping:
We have a LLC frame (or many queued up)... we schedule the polls and
at the final_ack indicate we decide to re-use the TBF. This means that
we will schedule another PACKET DOWNLINK ASSIGNMENT. But at the same
time we either want to honor the "rh->si" or want to schedule the ACK
due SEND_ACK_AFTER_FRAMES.
So we more or less want to send "PACKET DOWNLINK ASSIGNMENT" and the
"PACKET UPLINK ACK" at the same time (with more DL tbfs/traffic this
is getting more likely) but currently we will always prefer the PACKET
DOWNLINK ASSIGNMENT. This means that the uplink will starve (e.g. the
window stalled, rh->si means that the uplink will starve (the window
will stall, rh->si being set, etc).
Fairness improvements:
* put ul_ass_tbf, dl_ass_tbf, ul_ack_tbf in an array and store the
last index in the PDCH and iterate over it.
* move the TBF to the front of the ul_tbfs and dl_tbfs list so that
it is not selected again.
--
- Holger Freyther <hfreyther(a)sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Schivelbeiner Str. 5
* 10439 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Holger Freyther, Harald Welte
27 Nov
27 Nov
8:49 a.m.
Holger Hans Peter Freyther wrote:
Here is the starvation theory for the ping:
We have a LLC frame (or many queued up)... we schedule the polls and
at the final_ack indicate we decide to re-use the TBF. This means that
we will schedule another PACKET DOWNLINK ASSIGNMENT. But at the same
time we either want to honor the "rh->si" or want to schedule the ACK
due SEND_ACK_AFTER_FRAMES.
the idea behind priority of PACKET DOWNLINK ASSIGNMENT is that i do not
want the phone to switch back to idle mode. if i would ACK all uplink
blocks, the MS might switch back to idle mode immediately and will never
receive the PACKET DOWNLINK ASSIGNMENT.
when there was an ongoing downlink tbf, the phone keeps in transfer mode
until T3193 fires, so in case of a tbf re-use we can safely schedule a
PACKET UPLINK ACK/NACK prior PACKET DOWNLINK ASSIGNMENT.
So we more or less want to send "PACKET DOWNLINK
ASSIGNMENT" and the
"PACKET UPLINK ACK" at the same time (with more DL tbfs/traffic this
is getting more likely) but currently we will always prefer the PACKET
DOWNLINK ASSIGNMENT. This means that the uplink will starve (e.g. the
window stalled, rh->si means that the uplink will starve (the window
will stall, rh->si being set, etc).
once we sent PACKET DOWNLINK ASSIGNMENT we can ACK the uplink blocks
right afterwards. this should resolve the stalled window just a bit
later. this is my theory, but your tests showed that this does not work
as i would expect.
26 Nov
26 Nov
9:22 a.m.
On Mon, Nov 25, 2013 at 08:24:27AM +0100, Andreas Eversberg wrote:
generally there is always a re-send due to delay
between pcu and the
radio interface. if all rlc/mac data blocks of an llc frame have been
sent to the phone, a packet downlink ack control block is requested and
scheduled. the pcu repeats all unacknowledged data blocks in a loop
until it receives that requested control block. this way the pcu starts
re-sending data blocks that might got lost, before it actually knows if
and which blocks still need to be resend. (this procedure is described
in TS 04.60.)
Are you referring to 9.1.3.1 "Acknowledge state array V(B) for GPRS
TBF Mode"?
"
If there are no further RLC data blocks available for transmission
(i.e. the RLC data block with BSN= V(S) does not exist), the sending
side shall transmit the oldest RLC data block whose corresponding
element in V(B) has the value PENDING_ACK, then the next oldest block
whose corresponding element in V(B) has the value PENDING_ACK, etc.
"
With statistics in place it appears that for CS4 close to 50% of the
sent RLC blocks are re-sends. Is that what you expected when implementing
it?
holger
--
- Holger Freyther <hfreyther(a)sysmocom.de> http://www.sysmocom.de/
=======================================================================
* sysmocom - systems for mobile communications GmbH
* Schivelbeiner Str. 5
* 10439 Berlin, Germany
* Sitz / Registered office: Berlin, HRB 134158 B
* Geschaeftsfuehrer / Managing Directors: Holger Freyther, Harald Welte
27 Nov
27 Nov
9:06 a.m.
Holger Hans Peter Freyther wrote:
Are you referring to 9.1.3.1 "Acknowledge state
array V(B) for GPRS
TBF Mode"?
yes, the protocol sends all unacknowledged blocks in a loop until all
blocks are acknowledged and the tbf is complete.
With statistics in place it appears that for CS4 close
to 50% of the
sent RLC blocks are re-sends. Is that what you expected when implementing
it?
if there is a continuous download (opening a web page), there should be
almost no re-sends unless the window stalls or the download is complete.
in case of a ping with several small tbfs, there are re-sends due to the
delay between PCU and the radio interface: the PCU sends the final block
of a tbf (polling bit set) and then waits for PACKET DOWNLINK ACK/NACK.
higher delay will cause more re-sends for each tbf at the end, so
smaller tbf will cause more percentage of re-sends.
3826
days inactive
3829
days old
osmocom-net-gprs@lists.osmocom.org
6 comments
2 participants
participants (2)
-
Andreas Eversberg -
Holger Hans Peter Freyther