Bug #6379
closedttcn3-{msc,sgsn}-test regressions (IUT SIGSEGV)
100%
Description
Both testsuites exhibit massive regressions since a few days ago:
https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-msc-test/2308/ +213 failures
https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-sgsn-test/2264/ +70 failures
The artifacts generated while running those testsuites contain core dump files, so the IUT is crashing.
I managed to reproduce the problem by running ttcn3-msc-test against the most recent version of osmo-msc:
20240228153930783 DLGSUP NOTICE GSUP connecting to 127.0.0.1:4222 (gsup_client.c:74) 20240228153930783 DLINP NOTICE 127.0.0.1:4222 connection done (ipa.c:143) Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7c02274 in ipaccess_bts_handle_ccm (link=link@entry=0x55555584bef0, dev=0x55555584a920, msg=msg@entry=0x555555889b10) at ../../../src/libosmo-abis/src/input/ipaccess.c:897 897 LOGPIL(line, DLINP, LOGL_NOTICE, "received ID_GET for unit ID %u/%u/%u\n", (gdb) bt #0 0x00007ffff7c02274 in ipaccess_bts_handle_ccm (link=link@entry=0x55555584bef0, dev=0x55555584a920, msg=msg@entry=0x555555889b10) at ../../../src/libosmo-abis/src/input/ipaccess.c:897 #1 0x00007ffff7c1aa7a in gsup_client_read_cb (link=0x55555584bef0, msg=0x555555889b10) at ../../../../src/osmo-hlr/src/gsupclient/gsup_client.c:209 #2 0x00007ffff7bfd0df in ipa_client_read (link=0x55555584bef0) at ../../../src/libosmo-abis/src/input/ipa.c:77 #3 ipa_client_fd_cb (ofd=<optimized out>, what=1) at ../../../src/libosmo-abis/src/input/ipa.c:151 #4 0x00007ffff7aefc2f in poll_disp_fds (n_fd=<optimized out>) at ../../../../src/libosmocore/src/core/select.c:419 #5 _osmo_select_main (polling=polling@entry=0) at ../../../../src/libosmocore/src/core/select.c:457 #6 0x00007ffff7aefd5e in osmo_select_main_ctx (polling=polling@entry=0) at ../../../../src/libosmocore/src/core/select.c:513 #7 0x000055555556971d in main (argc=<optimized out>, argv=<optimized out>) at ../../../../src/osmo-msc/src/osmo-msc/msc_main.c:846
Updated by fixeria 2 months ago
- Project changed from Cellular Network Infrastructure to libosmo-abis
- % Done changed from 0 to 10
This appears to be a regression of recent libosmo-abis.git commit:
commit a87de3cd60d447c5199194ae8fa44335198de9c9 Author: Harald Welte <laforge@osmocom.org> Date: Wed Feb 21 17:41:12 2024 +0100 input/ipaccess: Always log with context, if we have any
specifically, the following part of this commit:
@@ -920,13 +894,13 @@ int ipaccess_bts_handle_ccm(struct ipa_client_conn *link,
if (link->ofd->priv_nr >= E1INP_SIGN_RSL)
trx_nr = link->ofd->priv_nr - E1INP_SIGN_RSL;
- LOGP(DLINP, LOGL_NOTICE, "received ID_GET for unit ID %u/%u/%u\n",
- dev->site_id, dev->bts_id, trx_nr);
+ LOGPIL(line, DLINP, LOGL_NOTICE, "received ID_GET for unit ID %u/%u/%u\n",
+ dev->site_id, dev->bts_id, trx_nr);
rmsg = ipa_bts_id_resp(dev, data + 1, len - 1, trx_nr);
ret = ipa_send(link->ofd->fd, rmsg->data, rmsg->len);
if (ret != rmsg->len) {
- LOGP(DLINP, LOGL_ERROR, "cannot send ID_RESP "
- "message. Reason: %s\n", strerror(errno));
+ LOGPIL(line, DLINP, LOGL_ERROR, "cannot send ID_RESP message. Reason: %s\n",
+ strerror(errno));
goto err_rmsg;
}
msgb_free(rmsg);
@@ -935,8 +909,8 @@ int ipaccess_bts_handle_ccm(struct ipa_client_conn *link,
rmsg = ipa_bts_id_ack();
ret = ipa_send(link->ofd->fd, rmsg->data, rmsg->len);
if (ret != rmsg->len) {
- LOGP(DLINP, LOGL_ERROR, "cannot send ID_ACK "
- "message. Reason: %s\n", strerror(errno));
+ LOGPIL(line, DLINP, LOGL_ERROR, "cannot send ID_ACK message. Reason: %s\n",
+ strerror(errno));
goto err_rmsg;
}
msgb_free(rmsg);
The problem is that line
is NULL
and LOGPIL
is dereferencing it unconditionally:
Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7c02274 in ipaccess_bts_handle_ccm (link=link@entry=0x55555584bef0, dev=0x55555584a920, msg=msg@entry=0x55555588f7f0) at ../../../src/libosmo-abis/src/input/ipaccess.c:897 897 LOGPIL(line, DLINP, LOGL_NOTICE, "received ID_GET for unit ID %u/%u/%u\n", (gdb) p line $1 = (struct e1inp_line *) 0x0
Updated by fixeria 2 months ago
- % Done changed from 10 to 40
Here is a partial revert, which fixes ttcn3-msc-test:
https://gerrit.osmocom.org/c/libosmo-abis/+/36100 input/ipaccess: fix regression in ipaccess_bts_handle_ccm() [NEW]
I am currently testing ttcn3-sgsn-test.
Updated by fixeria 2 months ago
- Status changed from Feedback to Resolved
- % Done changed from 80 to 100
The patch has been merged.
- ttcn3-msc-test is back to normal: https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-msc-test/2313/
- ttcn3-sgsn-test is back to normal: https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-sgsn-test/2269/
- ttcn3-remsim-test was also affected and recovered now: https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-remsim-test/1305/
- likewise, https://jenkins.osmocom.org/jenkins/view/osmo-gsm-tester/job/osmo-gsm-tester_virtual/7512/ is back to normal