Bug #3444
closedmgw hits assertion in mgcp_network.c:489
100%
Description
osmo-mgw hits an assertion in mgcp_network.c for some reason. Here is a backtrace and the config:
root@test123:/etc/osmocom# cat osmo-mgw.cfg ! ! MGCP configuration example ! mgcp bind ip 172.16.23.1 rtp port-range 4002 16000 rtp ip-probing rtp ip-tos 184 bind port 2427 sdp audio payload number 98 sdp audio payload name GSM number endpoints 31 loop 0 force-realloc 1 rtcp-omit rtp-patch ssrc rtp-patch timestamp ---------------------------------------- <0000> mgcp_network.c:879 endpoint:0x0 data from wrong address: 172.16.23.1, expected: 0.0.0.0 <0000> mgcp_network.c:883 endpoint:0x0 packet tossed <0000> mgcp_network.c:879 endpoint:0x2 data from wrong address: 172.16.23.1, expected: 0.0.0.0 <0000> mgcp_network.c:883 endpoint:0x2 packet tossed <0011> mgcp_protocol.c:942 MDCX: modifying existing connection ... <0011> mgcp_sdp.c:275 Got media info via SDP: port:4004, addr:172.16.23.1, duration:20, payload-types:none <0011> mgcp_protocol.c:1104 MDCX: endpoint:0x2 connection successfully modified <0011> mgcp_protocol.c:942 MDCX: modifying existing connection ... <0011> mgcp_sdp.c:275 Got media info via SDP: port:4012, addr:172.16.23.1, duration:20, payload-types:none <0011> mgcp_protocol.c:1104 MDCX: endpoint:0x0 connection successfully modified <0000> mgcp_network.c:979 endpoint:0x0 dummy message received <0000> mgcp_network.c:981 endpoint:0x0 packet tossed <0011> mgcp_protocol.c:942 MDCX: modifying existing connection ... <0011> mgcp_sdp.c:275 Got media info via SDP: port:4016, addr:172.16.23.1, duration:20, payload-types:none <0011> mgcp_protocol.c:1104 MDCX: endpoint:0x2 connection successfully modified <0000> mgcp_network.c:979 endpoint:0x2 dummy message received <0000> mgcp_network.c:981 endpoint:0x2 packet tossed Assert failed len >= sizeof(struct rtp_hdr) mgcp_network.c:489 backtrace() returned 9 addresses /usr/lib/x86_64-linux-gnu/libosmocore.so.11(osmo_panic+0xcb) [0x7ffff75386fb] /usr/bin/osmo-mgw(+0x9f26) [0x55555555df26] /usr/bin/osmo-mgw(+0xa0ce) [0x55555555e0ce] /usr/bin/osmo-mgw(+0xa4c3) [0x55555555e4c3] /usr/lib/x86_64-linux-gnu/libosmocore.so.11(osmo_select_main+0x222) [0x7ffff752f022] /usr/bin/osmo-mgw(+0x4547) [0x555555558547] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7ffff6f962e1] /usr/bin/osmo-mgw(+0x462a) [0x55555555862a] Program received signal SIGABRT, Aborted. __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) (gdb) (gdb) (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007ffff6faa42a in __GI_abort () at abort.c:89 #2 0x00007ffff7538700 in osmo_panic_default (args=0x7fffffffd680, fmt=0x555555567a3d "Assert failed %s %s:%d\n") at panic.c:49 #3 osmo_panic (fmt=fmt@entry=0x555555567a3d "Assert failed %s %s:%d\n") at panic.c:84 #4 0x000055555555df26 in mgcp_patch_pt (len=0, data=0x7fffffffd8b0 "#\003\377\264\033\353y6{\272\261j\330 \242\341ZP", conn_dst=0x55555581fe18, conn_src=0x555555821958) at mgcp_network.c:489 #5 mgcp_send (endp=endp@entry=0x55555581af20, is_rtp=1, addr=addr@entry=0x7fffffffd8a0, buf=buf@entry=0x7fffffffd8b0 "#\003\377\264\033\353y6{\272\261j\330 \242\341ZP", len=len@entry=0, conn_src=conn_src@entry=0x555555821958, conn_dst=0x55555581fe18) at mgcp_network.c:720 #6 0x000055555555e0ce in mgcp_send_rtp (proto=0, addr=0x7fffffffd8a0, buf=0x7fffffffd8b0 "#\003\377\264\033\353y6{\272\261j\330 \242\341ZP", buf_size=0, conn_src=0x555555821958, conn_dst=0x55555581fe18) at mgcp_network.c:1022 #7 0x000055555555e4c3 in rtp_data_net (fd=0x555555821b38, what=<optimized out>) at mgcp_network.c:1169 #8 0x00007ffff752f022 in osmo_fd_disp_fds (_eset=0x7fffffffea20, _wset=0x7fffffffe9a0, _rset=0x7fffffffe920) at select.c:217 #9 osmo_select_main (polling=0) at select.c:257 #10 0x0000555555558547 in main (argc=4, argv=0x7fffffffec08) at mgw_main.c:333 (gdb)
Updated by dexter over 5 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 90
I was not able to reproduce the problem, but I am pretty sure that a short packet made it into mgcp_patch_pt(), which must not happen. I found out that we also feed RTCP packets into mgcp_patch_pt, and one of those might have been shorter than the normal RTP header length. I have now added a check to be sure we do not feed RTCP packets into mgcp_patch_pt()
https://gerrit.osmocom.org/#/c/osmo-mgw/+/10329 network: do not patch PT of RTCP packets
Also I had a look at our receiving function and added some basic checks to avoid packets that are either too short or obviously wrong.
https://gerrit.osmocom.org/#/c/osmo-mgw/+/10330 network: check packets before further processing
(we should consider adding RTCP to our TTCN3 tests)
Updated by dexter over 5 years ago
- Status changed from In Progress to Resolved
- % Done changed from 90 to 100