Cellular Network Infrastructure » Radio Access Network » OsmoPCU

I am running osmo-pcu master (c46b6f29f9c534ec07c3e9bed81561cd479c90b7) and saw this segfault today:

Started Osmocom OsmoPCU.
DLGLOBAL NOTICE pcu_main.cpp:298 Setting up GSMTAP Um forwarding to '127.0.1.1:4729'
DLGLOBAL NOTICE telnet_interface.c:88 Available via telnet 127.0.0.1 4240
DL1IF NOTICE osmobts_sock.c:237 osmo-bts PCU socket /tmp/pcu_bts has been connected
DL1IF NOTICE pcu_l1_if.cpp:1244 Received message for new BTS0
DL1IF NOTICE pcu_l1_if.cpp:778 PCUIF version 10 is deprecated. OS#5927
DLNS NOTICE gprs_ns2.c:575 NSE(00101) NS-STATUS.ind(bvci=00000): cause=NSE recovery, transfer=100, first=1, mtu=65523
DPCU NOTICE gprs_bssgp_pcu.c:681 NS-NSE 101 became available
DLBSSGP NOTICE gprs_bssgp.c:124 BSSGP (BVCI=0) Tx BVC-RESET CAUSE=O&M intervention
DLNS NOTICE gprs_ns2.c:571 NSE(00101)-NSVC(00101) NS-STATUS.ind(bvci=00000): cause=NSVC recovery, transfer=100, first=0, mtu=65523
DLBSSGP NOTICE gprs_bssgp_pcu.c:421 Rx BSSGP BVCI=0 (SIGN) BVC_RESET_ACK
DLBSSGP NOTICE gprs_bssgp.c:124 BSSGP (BVCI=2) Tx BVC-RESET CAUSE=O&M intervention
DLBSSGP NOTICE gprs_bssgp_pcu.c:421 Rx BSSGP BVCI=0 (SIGN) BVC_RESET_ACK
DLBSSGP NOTICE gprs_bssgp_bss.c:281 BSSGP (BVCI=2) Tx BVC-UNBLOCK
DLBSSGP NOTICE gprs_bssgp_pcu.c:435 Rx BSSGP BVCI=0 (SIGN) BVC_UNBLOCK_ACK
DRLCMAC NOTICE pdch_ul_controller.c:327 PDCH(bts=0,trx=0,ts=7) Timeout for registered POLL (FN=5213, reason=UL_ASS): TBF(DL:TFI-0-0-0:G:IMSI-001010000000000:TLLI-0x80000000){FLOW}
DTBF NOTICE tbf.cpp:487 TBF(DL:TFI-0-0-0:G:IMSI-001010000000000:TLLI-0x80000000){FLOW} poll timeout for FN=5213, TS=7 (curr FN 5213)
DTBF NOTICE tbf_ul_ass_fsm.c:270 TBF(DL:TFI-0-0-0:G:IMSI-001010000000000:TLLI-0x80000000){FLOW} Timeout for polling PACKET CONTROL ACK for PACKET UPLINK ASSIGNMENT: |Assignment was on PACCH|Downlink ACK was received|
DRLCMAC NOTICE pdch_ul_controller.c:327 PDCH(bts=0,trx=0,ts=7) Timeout for registered POLL (FN=6045, reason=UL_ASS): TBF(DL:TFI-0-0-0:G:IMSI-001010000000000:TLLI-0x80000000){WAIT_RELEASE}
DTBF NOTICE tbf.cpp:487 TBF(DL:TFI-0-0-0:G:IMSI-001010000000000:TLLI-0x80000000){WAIT_RELEASE} poll timeout for FN=6045, TS=7 (curr FN 6045)
DTBF NOTICE tbf_ul_ass_fsm.c:270 TBF(DL:TFI-0-0-0:G:IMSI-001010000000000:TLLI-0x80000000){WAIT_RELEASE} Timeout for polling PACKET CONTROL ACK for PACKET UPLINK ASSIGNMENT: |Assignment was on PACCH|Downlink ACK was received|
Assert failed !ms_dl_tbf(ms) ../../../src/osmo-pcu/src/gprs_ms.c:1099
Signal 6 received.
...
Process 536300 (osmo-pcu) of user 1000 dumped core.

Stack trace of thread 536300:
#0  0x00007fc3739b226c n/a (libc.so.6 + 0x8926c)
#1  0x00007fc373962a08 raise (libc.so.6 + 0x39a08)
#2  0x00007fc37394b538 abort (libc.so.6 + 0x22538)
#3  0x00007fc374074b63 osmo_panic_default (libosmocore.so.20 + 0x24b63)
#4  0x000055b305c51f81 ms_is_reachable_for_dl_ass (osmo-pcu + 0x29f81)
#5  0x000055b305c67826 dl_tbf_handle (osmo-pcu + 0x3f826)
#6  0x000055b305c4a9bd gprs_bssgp_pcu_rx_dl_ud (osmo-pcu + 0x229bd)
#7  0x000055b305c4bb2a gprs_ns_prim_cb (osmo-pcu + 0x23b2a)
#8  0x00007fc3743a2081 ns2_recv_unitdata (libosmogb.so.14 + 0x2c081)
#9  0x00007fc374068e4e _osmo_fsm_inst_dispatch (libosmocore.so.20 + 0x18e4e)
#10 0x00007fc3743a2e68 ns2_vc_rx (libosmogb.so.14 + 0x2ce68)
#11 0x00007fc37439ac2a ns2_recv_vc (libosmogb.so.14 + 0x24c2a)
#12 0x00007fc37439d5c7 handle_nsip_recvfrom (libosmogb.so.14 + 0x275c7)
#13 0x00007fc3740748c7 iofd_poll_ofd_cb_recvmsg_sendmsg (libosmocore.so.20 + 0x248c7)
#14 0x00007fc374076c2f poll_disp_fds (libosmocore.so.20 + 0x26c2f)
#15 0x00007fc374076d0e osmo_select_main (libosmocore.so.20 + 0x26d0e)
#16 0x000055b305c4535d main (osmo-pcu + 0x1d35d)
#17 0x00007fc37394c850 n/a (libc.so.6 + 0x23850)
#18 0x00007fc37394c90a __libc_start_main (libc.so.6 + 0x2390a)
#19 0x000055b305c45865 _start (osmo-pcu + 0x1d865)
ELF object binary architecture: AMD x86-64

This happened just once when I started the modem app as usual, obviously it failed to complete the ATTACH procedure.

Here is a backtrace:

(gdb) bt
#0  0x00007fc3739b226c in ?? () from /usr/lib/libc.so.6
#1  0x00007fc373962a08 in raise () from /usr/lib/libc.so.6
#2  0x00007fc37394b538 in abort () from /usr/lib/libc.so.6
#3  0x00007fc374074b63 in osmo_panic_default (args=0x7ffda9ff2c60, fmt=0x55b305c921fb "Assert failed %s %s:%d\n") at ../../../../src/libosmocore/src/core/panic.c:45
#4  osmo_panic (fmt=fmt@entry=0x55b305c921fb "Assert failed %s %s:%d\n") at ../../../../src/libosmocore/src/core/panic.c:80
#5  0x000055b305c51f81 in ms_is_reachable_for_dl_ass (ms=0x55b306e50b50) at ../../../src/osmo-pcu/src/gprs_ms.c:1099
#6  ms_append_llc_dl_data (ms=0x55b306e50b50, pdu_delay_csec=1000, data=0x55b306e959e8 "\001\300\r\b\002\001*D\t\361\a", len=26)
    at ../../../src/osmo-pcu/src/gprs_ms.c:1400
#7  0x000055b305c67826 in dl_tbf_handle (bts=0x55b306e442b0, tlli=tlli@entry=2147483648, tlli_old=tlli_old@entry=4294967295, 
    imsi=imsi@entry=0x7ffda9ff2e64 "001010000000000", ms_class=ms_class@entry=0 '\000', egprs_ms_class=egprs_ms_class@entry=0 '\000', delay_csec=1000, 
    data=0x55b306e959e8 "\001\300\r\b\002\001*D\t\361\a", len=26) at ../../../src/osmo-pcu/src/tbf_dl.cpp:230
#8  0x000055b305c4a9bd in gprs_bssgp_pcu_rx_dl_ud (tp=0x7ffda9ff3180, msg=<optimized out>) at ../../../src/osmo-pcu/src/gprs_bssgp_pcu.c:180
#9  gprs_bssgp_pcu_rx_ptp (bctx=<optimized out>, tp=0x7ffda9ff3180, msg=<optimized out>) at ../../../src/osmo-pcu/src/gprs_bssgp_pcu.c:354
#10 gprs_bssgp_pcu_rcvmsg (msg=<optimized out>) at ../../../src/osmo-pcu/src/gprs_bssgp_pcu.c:573
#11 0x000055b305c4bb2a in gprs_ns_prim_cb (oph=0x7ffda9ff4200, ctx=<optimized out>) at ../../../src/osmo-pcu/src/gprs_bssgp_pcu.c:735
#12 0x00007fc3743a2081 in ns2_recv_unitdata (msg=<optimized out>, fi=<optimized out>) at ../../../../src/libosmocore/src/gb/gprs_ns2_vc_fsm.c:627
#13 0x00007fc374068e4e in _osmo_fsm_inst_dispatch (fi=fi@entry=0x55b306d67f10, event=event@entry=10, data=data@entry=0x55b306e958c0, 
    file=file@entry=0x7fc3743c29e0 "../../../../src/libosmocore/src/gb/gprs_ns2_vc_fsm.c", line=line@entry=964) at ../../../../src/libosmocore/src/core/fsm.c:863
#14 0x00007fc3743a2e68 in ns2_vc_rx (nsvc=nsvc@entry=0x55b306e23280, msg=msg@entry=0x55b306e958c0, tp=tp@entry=0x7ffda9ff4340)
    at ../../../../src/libosmocore/src/gb/gprs_ns2_vc_fsm.c:964
#15 0x00007fc37439ac2a in ns2_recv_vc (nsvc=0x55b306e23280, msg=msg@entry=0x55b306e958c0) at ../../../../src/libosmocore/src/gb/gprs_ns2.c:1362
#16 0x00007fc37439d5c7 in handle_nsip_recvfrom (iofd=<optimized out>, error=<optimized out>, msg=0x55b306e958c0, saddr=0x7ffda9ff5430)
    at ../../../../src/libosmocore/src/gb/gprs_ns2_udp.c:218
#17 0x00007fc3740748c7 in iofd_poll_ofd_cb_recvmsg_sendmsg (what=<optimized out>, ofd=0x55b306d88ad0) at ../../../../src/libosmocore/src/core/osmo_io_poll.c:75
#18 iofd_poll_ofd_cb_dispatch (ofd=0x55b306d88ad0, what=1) at ../../../../src/libosmocore/src/core/osmo_io_poll.c:130
#19 0x00007fc374076c2f in poll_disp_fds (n_fd=<optimized out>) at ../../../../src/libosmocore/src/core/select.c:419
#20 _osmo_select_main (polling=<optimized out>) at ../../../../src/libosmocore/src/core/select.c:457
#21 0x00007fc374076d0e in osmo_select_main (polling=<optimized out>) at ../../../../src/libosmocore/src/core/select.c:496
#22 0x000055b305c4535d in main (argc=3, argv=<optimized out>) at ../../../src/osmo-pcu/src/pcu_main.cpp:353

Ping me if you need more info from the coredump file.