22 Jun
2013
22 Jun
'13
2:05 a.m.
Hello,
I just updated my machine with the latest OpenBSC code from git. In the past
under heavy data load (EDGE), the BTS would crash. Now, after updating the
code, the BTS stays up, but the SGSN crashes. Below are a few details about
my setup and debugging output.
BTS: NanoBTS 165BU 1900
PC: Atom Z530 1GB ram, Running Debian (Wheezy)
Output from gdb:
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xc3123cCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xac6064CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xdc7cc7CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x18af79CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x84caf1CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xf1f6b6CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xd9e63bCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xbadc11CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=64
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=197
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xb7e07cCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x987e00CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xf386aaCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x38700eCMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x62743eCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x1cba00CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x6e4346CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xb20578CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x49f5d0CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x760e46CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xc839b5CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xacd5a3CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xdaf317CMD=UI DATA
<0011> gprs_bssgp.c:747 BSSGP BVCI=3 Rx Flow Control BVC
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x7d7a91CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x1eef1aCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xd83b65CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xf6a775CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x855a58CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x18aa6fCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x167b76CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xce3dde6f Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x55d812CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
Program received signal SIGFPE, Arithmetic exception.
fc_queue_timer_cfg (fc=fc@entry=0x808def0) at gprs_bssgp.c:596
596 msecs = (fcqe->llc_pdu_len * 1000) / fc->bucket_leak_rate;
(gdb)
OpenBSC Console Output:
Fri Jun 21 16:36:19 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Fri Jun 21 16:36:20 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:315 OC=GPRS-NSVC(f2)
INST=(00,00,ff) STATE CHG: OP_STATE=Disabled AVAIL=Failed(01)
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:315 OC=GPRS-NSE(f0)
INST=(00,ff,ff) STATE CHG: OP_STATE=Disabled AVAIL=Dependency(05)
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:1757 OC=GPRS-NSE(f0)
INST=(00,ff,ff) Sending OPSTART
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:315 OC=GPRS-CELL(f1)
INST=(00,00,ff) STATE CHG: OP_STATE=Disabled AVAIL=Dependency(05)
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:1757 OC=GPRS-CELL(f1)
INST=(00,00,ff) Sending OPSTART
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:315 OC=GPRS-NSE(f0)
INST=(00,ff,ff) STATE CHG: OP_STATE=Disabled AVAIL=Dependency(05)
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:315 OC=GPRS-CELL(f1)
INST=(00,00,ff) STATE CHG: OP_STATE=Disabled AVAIL=Dependency(05)
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:2418 OC=GPRS-NSE(f0)
INST=(00,ff,ff) IPACCESS(0xf6): SET ATTR ACK
Fri Jun 21 16:38:00 2013 <0005> abis_nm.c:2418 OC=GPRS-CELL(f1)
INST=(00,00,ff) IPACCESS(0xf6): SET ATTR ACK
Fri Jun 21 16:38:42 2013 <0005> abis_nm.c:315 OC=GPRS-NSVC(f2)
INST=(00,00,ff) Failure Event Report Type=communication failure
Severity=critical failure Probable cause= 03 03 11 Additional Text=
Let me know if you need any further information.
Regards,
Caleb
22 Jun
22 Jun
2:32 a.m.
On Fri, Jun 21, 2013 at 05:05:22PM -0700, Caleb Pal wrote:
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN,
length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
Program received signal SIGFPE, Arithmetic exception.
fc_queue_timer_cfg (fc=fc@entry=0x808def0) at gprs_bssgp.c:596
596 msecs = (fcqe->llc_pdu_len * 1000) / fc->bucket_leak_rate;
(gdb)
(gdb) p fc->bucket_leak_rate
It seems to be 0.
Kind regards,
-Alexander Huemer
9:50 a.m.
Hi Caleb,
On Fri, Jun 21, 2013 at 05:05:22PM -0700, Caleb Pal wrote:
Program received signal SIGFPE, Arithmetic exception.
fc_queue_timer_cfg (fc=fc@entry=0x808def0) at gprs_bssgp.c:596
596 msecs = (fcqe->llc_pdu_len * 1000) / fc->bucket_leak_rate;
(gdb)
What you should always do when reporting crashes is to include a full
backtrace (bt full) as well as 'list' to show some surrounding lines of
code.
However, in this case it is quite obvious. Please try the attached
(untested) patch against libosmocore.
What happens is basically that the BTS wants the SGSN to stop all
downlink GPRS transmission.
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
23 Jun
23 Jun
5:17 a.m.
Harald,
Thanks for the patch to test. I applied it, and now I am back to the BTS
crashing under heavy load, although it did survive a bit longer. Shortly
after the BTS crashed, the sgsn crashed, and the relevant output from the
OpenBSC console and gdb is below.
OpenBSC console:
Sat Jun 22 20:00:41 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=processing failure
Severity=warning level failure Probable cause= 03 00 01 Additional
Text=38137:WARN:GBHSS_UDPE:udp_entity.c#503:S: 736 1712 5592 1000 1416 672
888 1536 2440 848 5416 1052 816 1112 584 1488
Sat Jun 22 20:00:41 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=processing failure
Severity=warning level failure Probable cause= 03 00 01 Additional
Text=38140:WARN:GBHSS_UDPE:udp_entity.c#503: 496 672 424 968 784 952 1072
904 520 496 512 3176 504 888 1048 2832 1072 864
Sat Jun 22 20:00:41 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=processing failure
Severity=warning level failure Probable cause= 03 00 01 Additional
Text=38142:WARN:GBHSS_UDPE:udp_entity.c#503: 1168 1824 1128 496 512 424
Sat Jun 22 20:00:52 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Sat Jun 22 20:00:54 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (5 occurences)
Sat Jun 22 20:02:04 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Sat Jun 22 20:02:07 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Sat Jun 22 20:03:14 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Sat Jun 22 20:03:16 2013 <0005> abis_nm.c:315 OC=BASEBAND-TRANSCEIVER(04)
INST=(00,00,ff) Failure Event Report Type=quality of service failure
Severity=warning level failure Probable cause= 03 05 01 Additional Text=UDP
overflow alarm on port 23000 (1 occurences)
Sgsn/gdb
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x717dcfCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x7e714bCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x6d27e3CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=105
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xb00b8bCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x4fcc88CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=151
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=131
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xa0115bCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xbc7635CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x690bd2CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x24fc1cCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xf06ef5CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x65d78cCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x4c0b92CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xe9acbeCMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x239284CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xf6d681CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xf4b3e6CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x92d6faCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xea0d6dCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xe78cddCMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x1dd741CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x757059CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x4d6e75CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=83
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xcb0b55CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xbb0405CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0x3c3339CMD=UI DATA
<0011> gprs_bssgp.c:376 BSSGP TLLI=0xc50a3bee Rx UPLINK-UNITDATA
<0012> gprs_llc.c:502 LLC SAPI=3 C FCS=0xfdcb47CMD=UI DATA
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1470
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1470
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=377
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=105
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=105
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=105
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=140
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=55
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=55
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=131
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=55
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=55
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=116
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=115
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=151
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=131
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=83
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=109
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1470
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=81
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=52
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=120
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=60
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1470
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=377
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1470
<0010> gprs_ns.c:489 NSEI=102 Tns-alive expired more then 10 times, blocking
NS-VC
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=55
<0010> gprs_ns.c:573 NSEI=102 is not alive, cannot send
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=55
<0010> gprs_ns.c:573 NSEI=102 is not alive, cannot send
<000f> sgsn_libgtp.c:425 GTP DATA IND from GGSN, length=1500
<0010> gprs_ns.c:573 NSEI=102 is not alive, cannot send
Program received signal SIGABRT, Aborted.
0xb7fe1424 in __kernel_vsyscall ()
(gdb)
(gdb) bt full
#0 0xb7fe1424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb79eb941 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
No symbol table info available.
#2 0xb79eed72 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
No symbol table info available.
#3 0xb7bb4738 in talloc_abort (reason=<optimized out>) at talloc.c:167
No locals.
#4 0xb7bb8d2a in talloc_chunk_from_ptr (ptr=0x808f0b0) at talloc.c:190
pp = 0x808f0b0 ""
tc = 0x808f080
#5 _talloc_free (ptr=0x808f0b0) at talloc.c:517
No locals.
#6 talloc_free (ptr=ptr@entry=0x808f0b0) at talloc.c:990
No locals.
#7 0xb7baeaab in msgb_free (m=m@entry=0x808f0b0) at msgb.c:72
No locals.
#8 0x0804df54 in sndcp_send_ud_frag (fs=0xbfffd7cc) at gprs_sndcp.c:423
lle = 0x808e164
sne = 0x808f020
fmsg = 0x808f0b0
max_payload_len = <optimized out>
len = <optimized out>
rc = <optimized out>
more = 1
#9 sndcp_unitdata_req (msg=msg@entry=0x809a420, lle=0x808e164, nsapi=5
'\005', mmcontext=mmcontext@entry=0x808e950) at gprs_sndcp.c:471
rc = 0
fs = {frag_nr = 1 '\001', msg = 0x809a420,
next_byte = 0x809a6d4 "h=/\r\nSet-Cookie:
bb_thread_lastview=bea63447ee9fe14d112a986a3c0e3923c0d88807a-17-%7Bi-3751_i-
1371319030_i-3686_i-1371306998_i-3620_i-1370565014_i-3675_i-1370754429_i-370
1_i-1371277053_i-3511_i-1369170"..., sne = 0x808f020, mmcontext = 0x808e950}
#10 0x0804fbf3 in cb_data_ind (lib=lib@entry=0xb7bd1a20,
packet=packet@entry=0xbfffd928, len=1500) at sgsn_libgtp.c:477
pinfo = {mode = BSSGP_PAGING_PS, scope = BSSGP_PAGING_BSS_AREA, raid
= {mnc = 0, mcc = 0, lac = 0, rac = 0 '\000'}, bvci = 0, imsi = 0x0, ptmsi =
0x0,
drx_params = 7680, qos = "\274\267X"}
pdp = 0x808edf8
mm = 0x808e950
msg = 0x809a420
#11 0xb7bc8ea9 in gtp_gpdu_ind (gsn=gsn@entry=0x808ccb0,
version=version@entry=1, peer=peer@entry=0xbfffd90c, fd=fd@entry=11,
pack=pack@entry=0xbfffd91c, len=1512)
at gtp.c:2714
hlen = <optimized out>
pdp = 0xb7bd1a20
#12 0xb7bc957a in gtp_decaps1u (gsn=0x808ccb0) at gtp.c:3162
buffer = "2\377\005\340\000\000\000\001\017\342\000\000E
\005\334\001\332@\000\062\006ÓC\347\030\020\n#\a\202\000P\350\321|\a\031\251
\335\024\233,P\020>0\357^\000\000HTTP/1.1 200 OK\r\nDate: Sun, 23 Jun 2013
03:03:48 GMT\r\nServer: Apache/2.2.23 (Unix) mod_ssl/2.2.23
OpenSSL/1.0.0-fips mod_auth_passthrough/2.1 mod_bwlimited/1.4 Fr"...
peer = {sin_family = 2, sin_port = 26632, sin_addr = {s_addr =
50799370}, sin_zero = "\000\000\000\000\000\000\000"}
---Type <return> to continue, or q <return> to quit---
peerlen = 16
status = <optimized out>
pheader = 0xbfffd91c
fd = 11
#13 0xb7bae7e2 in osmo_select_main (polling=polling@entry=0) at select.c:158
flags = <optimized out>
ufd = 0x80593d4
tmp = 0x807c3f0
readset = {__fds_bits = {0 <repeats 32 times>}}
writeset = {__fds_bits = {0 <repeats 32 times>}}
exceptset = {__fds_bits = {0 <repeats 32 times>}}
work = 1
rc = <optimized out>
no_time = {tv_sec = 0, tv_usec = 0}
#14 0x0804a3c1 in main (argc=1, argv=0xbffffda4) at sgsn_main.c:368
dummy_network = {country_code = 61428, network_code = 47103,
name_long = 0x0,
name_short = 0xbffffc04
"\f\375\377\277\004N\234\267\370\255\377\267\060\374\377\277\260\374\377\277
\016", auth_policy = 3086928783,
reject_cause = 3087006400, a5_encryption = -1214510736, neci =
-1212686476, send_mm_info = -1207980552, handover = {active = -1073742960,
win_rxlev_avg = 0, win_rxqual_avg = 24, win_rxlev_avg_neigh =
3082283604, pwr_interval = 3080455104, pwr_hysteresis = 3628221240,
max_distance = 3086973856}, stats = {chreq = {total = 0xa,
no_channel = 0x0}, handover = {attempted = 0x0, no_channel = 0x1, timeout =
0xae,
completed = 0xb79c0540, failed = 0xb79cd824}, loc_upd_type =
{attach = 0xd696910, normal = 0xbffffbe4, periodic = 0xb7feada6, detach =
0xb79d3a30},
loc_upd_resp = {reject = 0x804946a, accept = 0xbffffcb0}, paging
= {attempted = 0x8049355, detached = 0xb79c0400, completed = 0xbfff0002,
expired = 0xb7ff0ae0}, sms = {submitted = 0x8049355,
no_receiver = 0xb7ba8aa0, delivered = 0xb7ffeff4, rp_err_mem = 0xb79c4bf8,
rp_err_other = 0x7},
call = {mo_setup = 0xbffffc64, mo_connect_ack = 0xb7feb425,
mt_setup = 0x1, mt_connect = 0xb7ba8000}, chan = {rf_fail = 0xb7fea562,
rll_err = 0xb7fd7000}, bts = {oml_fail = 0x7e10, rsl_fail =
0xb7ffeff4}}, mncc_state = 0xbffffd0c, mncc_recv = 0xb79c4e04, upqueue = {
next = 0xb7ffadf8, prev = 0xbffffc30}, trans_list = {next =
0xbffffcb0, prev = 0xe}, bsc_api = 0xb79cd824, num_bts = 3080455120,
bts_list = {
next = 0xf63d4e2e, prev = 0x0}, T3101 = 10, T3103 = 0, T3105 =
0, T3107 = 1, T3109 = 2210, T3111 = -1214512128, T3113 = -1212511480, T3115
= 134517786,
T3117 = -1214455724, T3119 = 134514968, T3122 = 1, T3141 =
-1207963660, subscr_expire_timer = {node = {rb_parent_color = 3221224784,
rb_right = 0xb7fffac0,
rb_left = 0xbffffd24}, list = {next = 0xb7feb662, prev =
0xbffffd14}, timeout = {tv_sec = 134514968, tv_usec = -1073742584}, active =
0, cb = 0,
data = 0xb79c0400}, rrlp = {mode = RRLP_MODE_MS_BASED},
dtx_enabled = 0, ctype_by_chreq = {GSM_LCHAN_SDCCH, 3087005960,
GSM_LCHAN_SDCCH, GSM_LCHAN_SDCCH,
GSM_LCHAN_NONE, GSM_LCHAN_SDCCH, 3080652277, 168972, 3082276864,
GSM_LCHAN_NONE, 3221224784, 3081899232, 134519124, 3086943632, 134582576,
3221224696},
pag_any_tch = 134556299, bsc_data = 0x1, keep_subscr =
-1073742428, sms_queue = 0xbffffdac, ctrl = 0xbffffcf8}
rc = <optimized out>
(gdb)
(gdb) list
280
281 static const struct log_info gprs_log_info = {
282 .filter_fn = gprs_log_filter_fn,
283 .cat = gprs_categories,
284 .num_cat = ARRAY_SIZE(gprs_categories),
285 };
286
287
288 int main(int argc, char **argv)
289 {
(gdb)
Let me know if you need any further information.
Regards,
Caleb
-----Original Message-----
From: openbsc-bounces(a)lists.osmocom.org
[mailto:openbsc-bounces@lists.osmocom.org] On Behalf Of Harald Welte
Sent: Saturday, June 22, 2013 0051
To: Caleb Pal
Cc: openbsc(a)lists.gnumonks.org
Subject: Re: SGSN Crash Report
Hi Caleb,
On Fri, Jun 21, 2013 at 05:05:22PM -0700, Caleb Pal wrote:
Program received signal SIGFPE, Arithmetic exception.
fc_queue_timer_cfg (fc=fc@entry=0x808def0) at gprs_bssgp.c:596
596 msecs = (fcqe->llc_pdu_len * 1000) / fc->bucket_leak_rate;
(gdb)
What you should always do when reporting crashes is to include a full
backtrace (bt full) as well as 'list' to show some surrounding lines of
code.
However, in this case it is quite obvious. Please try the attached
(untested) patch against libosmocore.
What happens is basically that the BTS wants the SGSN to stop all downlink
GPRS transmission.
--
- Harald Welte <laforge(a)gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
24 Jun
24 Jun
7:44 a.m.
On Sat, Jun 22, 2013 at 08:17:20PM -0700, Caleb Pal wrote:
Program received signal SIGABRT, Aborted.
0xb7fe1424 in __kernel_vsyscall ()
(gdb)
(gdb) bt full
#0 0xb7fe1424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb79eb941 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
No symbol table info available.
#2 0xb79eed72 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
No symbol table info available.
#3 0xb7bb4738 in talloc_abort (reason=<optimized out>) at talloc.c:167
No locals.
#4 0xb7bb8d2a in talloc_chunk_from_ptr (ptr=0x808f0b0) at talloc.c:190
pp = 0x808f0b0 ""
tc = 0x808f080
#5 _talloc_free (ptr=0x808f0b0) at talloc.c:517
No locals.
#6 talloc_free (ptr=ptr@entry=0x808f0b0) at talloc.c:990
No locals.
#7 0xb7baeaab in msgb_free (m=m@entry=0x808f0b0) at msgb.c:72
No locals.
#8 0x0804df54 in sndcp_send_ud_frag (fs=0xbfffd7cc) at gprs_sndcp.c:423
This looks like the known double free. My workaround/solution has been
described here[1]. Could you please generate a PCAP file, I will then
look in what it takes to re-produce the issue..
thanks
holger
[1] http://lists.osmocom.org/pipermail/openbsc/2013-March/004492.html
3982
days inactive
3984
days old
4 comments
4 participants
participants (4)
-
Alexander Huemer -
Caleb Pal -
Harald Welte -
Holger Hans Peter Freyther