Cellular Network Infrastructure

fixeria wrote:

Looks like osmo-bsc is not happy about the BSSAP Assignment Request sent by the testsuite?

Nope. As far as I can see from GSMTAP logging, it's parsed correctly and osmo-bsc does not complain. I compared two BSSAP Assignment Request messages: one from a PCAP before build 1911 (tests passing) with another from a PCAP after build 1911 (tests failing). There is no difference between them. The problem must be somewhere else.

There appear to be some MGCP related failures:

MGW(mgw) Tx MGCP: r=172.18.2.203:2428<->l=172.18.2.20:45150: len=77 'CRCX 1 rtpbridge/*@mgw MGCP 1.0\r\nC: 123\r\nL'...
E1TS(0:2) TX 2: 08 21 01 0a 03 01 06 04 03 01 08 01 05 06 64 0a 43 67 72 00 04 00 0d 07 18 00 1f 00 
MGW(mgw) Failed to read: r=172.18.2.203:2428<->l=172.18.2.20:45150: 111='Connection refused'

Adding pespin as a watcher, he submitted some MGW/MGCP related patches recently.

Assignment failure actually happens because T3101 expires:

OsmoBSC CHAN    fsm.c:317                   lchan(0-0-2-TCH_F-0)[0x557513cd2f50]{WAIT_RLL_RTP_ESTABLISH}: Timeout of T3101 
OsmoBSC CHAN    lchan_fsm.c:1822            lchan(0-0-2-TCH_F-0)[0x557513cd2f50]{WAIT_RLL_RTP_ESTABLISH}: (type=TCH_F) Handling failure, will then transition to state WAIT_RF_RELEASE_ACK 
OsmoBSC CHAN    lchan_fsm.c:1822            lchan(0-0-2-TCH_F-0)[0x557513cd2f50]{WAIT_RLL_RTP_ESTABLISH}: (type=TCH_F) lchan allocation failed in state WAIT_RLL_RTP_ESTABLISH: Timeout (rll_ready=yes,voice_require=yes,voice_ready=no) 
OsmoBSC CHAN    lchan_fsm.c:150             lchan(0-0-2-TCH_F-0)[0x557513cd2f50]{WAIT_RLL_RTP_ESTABLISH}: (type=TCH_F) Signalling Assignment FSM of error (lchan allocation failed in state WAIT_RLL_RTP_ESTABLISH: Timeout (rll_ready=yes,voice_require=yes,voice_ready=no)) 
OsmoBSC AS      lchan_fsm.c:1822            assignment(msc0-conn291_subscr-IMSI-002029876543210_0-0-2-TCH_F-0)[0x557513d24e00]{WAIT_LCHAN_ESTABLISHED}: Received Event ASSIGNMENT_EV_LCHAN_ERROR 
OsmoBSC AS      assignment_fsm.c:971        assignment(msc0-conn291_subscr-IMSI-002029876543210_0-0-2-TCH_F-0)[0x557513d24e00]{WAIT_LCHAN_ESTABLISHED}: (bts=0,trx=0,ts=2,ss=0) Assignment failed in state WAIT_LCHAN_ESTABLISHED, cause RADIO INTERFACE MESSAGE FAILURE: Failed to activate lchan (bts=0,trx=0,ts=2,ss=0) 
OsmoBSC AS      assignment_fsm.c:971        assignment(msc0-conn291_subscr-IMSI-002029876543210_0-0-2-TCH_F-0)[0x557513d24e00]{WAIT_LCHAN_ESTABLISHED}: (bts=0,trx=0,ts=2,ss=0) incrementing rate counter: assignment:error Assignment failed for other reason 
OsmoBSC AS      assignment_fsm.c:971        assignment(msc0-conn291_subscr-IMSI-002029876543210_0-0-2-TCH_F-0)[0x557513d24e00]{WAIT_LCHAN_ESTABLISHED}: (bts=0,trx=0,ts=2,ss=0) incrementing rate counter: bts0 assignment:error_speech Assignment of speech lchan failed for other reason 
OsmoBSC MSC     osmo_bsc_sigtran.c:379      Tx MSC: BSSMAP: ASSIGNMENT FAIL 

I am guessing because osmo-bsc never gets CRCX ACK from the virtual MGW?

Another thing that might be related:

(passing)   MGW(mgw) Tx MGCP: r=172.18.2.203:2427<->l=172.18.2.20:38794: len=79 'CRCX 451 rtpbridge/*@mgw MGCP 1.0\r\nC: 11c\r'...
(failing)   MGW(mgw) Tx MGCP: r=172.18.2.203:2428<->l=172.18.2.20:45150: len=77 'CRCX 1 rtpbridge/*@mgw MGCP 1.0\r\nC: 123\r\nL'...

Note that the remote port is different: 2427 vs 2428. We even get an ICMP Destination port unreachable:

MGW(mgw) Tx MGCP: r=172.18.2.203:2428<->l=172.18.2.20:45150: len=77 'CRCX 1 rtpbridge/*@mgw MGCP 1.0\r\nC: 123\r\nL'...
ICMP Destination port unreachable
MGW(mgw) Failed to read: r=172.18.2.203:2428<->l=172.18.2.20:45150: 111='Connection refused'

fixeria wrote in #note-4:

Note that the remote port is different: 2427 vs 2428. We even get an ICMP Destination port unreachable:

Relevant commit:

commit 3c63053fe87a7455d2fa569a3ee0e034df62e50e
Author: Pau Espin Pedrol <pespin@sysmocom.de>
Date:   Thu Oct 20 19:00:11 2022 +0200

    bsc: Support running several MGW node emulations

    Change-Id: Ic84726c09ad3c6bea3ce6f0198db862d7dbb6dd3

modifies f_init_mgcp() in osmo-ttcn3-hacks.git as follows:

diff --git a/bsc/BSC_Tests.ttcn b/bsc/BSC_Tests.ttcn
index 0f3c75c1..f00cc88b 100644
--- a/bsc/BSC_Tests.ttcn
+++ b/bsc/BSC_Tests.ttcn
@@ -1053,8 +1055,8 @@ altstep no_bssmap_reset() runs on test_CT {
        }
 }

-function f_init_mgcp(charstring id) runs on test_CT {
-       id := id & "-MGCP";
+function f_init_mgcp(integer mgw_nr, charstring id) runs on test_CT {
+       id := id & "-MGCP-" & int2str(mgw_nr);

        var MGCPOps ops := {
                create_cb := refers(MGCP_Emulation.ExpectedCreateCallback),
@@ -1064,14 +1066,14 @@ function f_init_mgcp(charstring id) runs on test_CT {
                callagent_ip := mp_bsc_ip,
                callagent_udp_port := -1,
                mgw_ip := mp_test_ip,
-               mgw_udp_port := 2427,
+               mgw_udp_port := 2427 + mgw_nr,
                /* Enable it for SCCPlite, since we have 2 MGCP sockets towards MGW (UDP one +
                   the one with MGCP over IPA forwarded from MSC one) */
                multi_conn_mode := (mp_bssap_cfg[0].transport == BSSAP_TRANSPORT_SCCPlite_SERVER)
        };