Cellular Network Infrastructure » OsmoMGW

Some notes/ideas regarding what needs to be implemented.

That's the required steps (in order) to be added during an MOMT call over AoIP (some information may be missing, it's just RFC).

MO part (1st group of steps):

• BSC => CM Service Request (Speech Codec Element List) => MSC <---- Here we want to announce we support osmux based on osmo-bsc config [NOT REALLY NEEDED]
• MSC => CM Service Accept / Reject => BSC <---- Here we may want to accept or reject based on osmo-msc cfg policy (ie, reject if we accept osmux only and BSC doesn't advertise osmux) [NOT REALLY NEEDED].
• BSC => Setup (Supported Codec List) => MSC <--- Here we want to advertise Osmux on an extra TLV after/inside Supported Codec List
• MSC => CRCX, MDCX => MGW_msc <--- Since BSC already announced use of Osmux, we manage MGW to use Osmux and retrieve Osmux endp information
• MSC => Assig Request (Speech Codec List MSC Preferred) => BSC <--- Here MSC has to confirm use of Osmux, providing Osmux endp information it got from MGW_msc
• BSC => CRCX, MDCX => MGW_bsc <----- Since BSC was already confirmed by MSC to use Osmux and it has remote Osmux endp information, configure the local endpoint to use Osmux and retrieve local Osmux endp information
• BSC => Assig Complete (Speech Codec Chosen) => MSC <--- Here BSC confirms it uses Osmux, providing Osmux endp information it got from MGW_bsc

MT part (2nd group of steps):

• MSC => Setup => BSC <-- Here MSC announces it wants to use Osmux (based on policy per BSC it can decide to use it or not)
• BSC => Call Confirmed (Supportd Codec List) => MSC <-- Here BSC announces it wants to use Osmux. Based on policy (osmux only) it can decide to drop the call.
• MSC => CRCX, MDCX => MGW_msc <--- Since BSC already announced use of Osmux, we manage MGW to use Osmux and retrieve Osmux endp information
• MSC => Assig Request (Speech Codec List MSC Preferred) => BSC <--- Here MSC has to confirm use of Osmux, providing Osmux endp information it got from MGW_msc
• BSC => CRCX, MDCX => MGW_bsc <----- Since BSC was already confirmed by MSC to use Osmux and it has remote Osmux endp information, configure the local endpoint to use Osmux and retrieve local Osmux endp information
• BSC => Assig Complete (Speech Codec Chosen) => MSC <--- Here BSC confirms it uses Osmux, providing Osmux endp information it got from MGW_bsc

TODO / some dangling sutff:

• What to do with LCLS?
• Format of TLVs to announce/confirm use of Osmux and pass its endp info through AoIP
• Format of messages in MGCP to configure stuff
• SCCPLite

Regarding Osmux confirmation during Assignment Request (MSC->BSC), I see two options:

1)

• IE AoIP Transport Layer Address (MGW) [48.008 3.2.2.102] contains IP addr + port of Osmux endp of MGW_msc
• IE Codec List (MSC Preferred) [48.008 3.2.2.103] contains only one "Speech Codec Element", defined as us by Osmux and containing the CID as one of the up-to-3 Configuration octets. To define Osmux Codec Type, we use "Codec Type" == 0xf and then use the entire next octet "Extended Codec Type" with some value defined by us, for instance 0xf0.
• This is fine in the case we announce Osmux by means of an extra T field in BSC->MSC CC Setup, because BSC already announced it supports it.
• BSC will answer with Assigment Complete with AoIP Transport layer Address IE containing its Osmux Ipv4/v6+port, and IE "Codec Chosen" with the Osmux one sent by MSC to it (patching CID to local one if needed?).

2)

• We add a new TLV to Assignment Request specific to handle Osmux case, containing IPv4/v6 + port + CID
• If BSC allows Osmux, it will take this new TLV into account instead of the Speech Codec List ones.
• Drawback: We still need to allocate a regular RTP endp in MGW_msc and announce it in IE AoIP Transport Layer Address (MGW) [48.008 3.2.2.102].
• If BSC wants RTP, it will answer with an RTP addr in IE AoIP Transport Layer Address and some RTP codec in "Chosen Codec", and no Osmux
• If Osmux is wanted, then we don't care about values in IE AoIP Transport Layer Address and IE "Chosen Codec" and we put everything (ipv4+port+CID) into the new Osmux TLV we add.

So far option 1 seems the more elegant one.

So after discussing some bits with laforge :

• It's better to avoid overloading/extending existing IEs/TLVs and add custom ones for osmux (that means not adding a new type in "Extended Codec Type" as mentioned before.
• Layer3 / DTAP messages shouldn't be patched, those belong to the MS

As a result, best option is, in Assign Request to:

• Fill IE AoIP Transport Layer Address (MGW) [48.008 3.2.2.102] with osmux MGW_msc endp.
• Leave "Extended Codec Type" empty, or add only one elem with support for AMR 5.9 (since Osmux uses that AMR type).
• Add an extra TV Osmux IE containing CID to be used.

Something important to keep in mind: Since MSC announces the AoIP MGW endp addr during Assign Request, it means the MSC must know beforehand whether that BSC wants to use Osmux or wants to use RTP, because those endp are different and MSC must request one to MGW through MGCP. That means BSC needs to "announce" it supports Osmux at some point before doing first call, otherwise MSC doesn't know which type of endpoint to allocate. Harald suggested patching BSSMAP RESET or COMPL_L3_INFO instead to announce Osmux support.

If we patch BSSMAP RESET or COMPL_L3_INFO, we need to be careful with "hotplug changes" in osmux policy in the BSC, for instance disabling osmux support in the VTY while BSC is running. If we want to support this kind of hotplug disabling, we need to announce that to the MSC somehow so next time MSC sends an Assig Request it allocates and announces the proper MGW_msc endpoint.

I pushed some changes I did to osmo-bsc, osmo-mgw and libosmocore, all in branch pespin/osmux. This should help as a good proof of concept.

Current state:

• osmo-bsc got a new VTY command under "msc" node: osmux (off|on|only)
• osmo-bsc adds our extension IE GSM0808_IE_OSMO_OSMUX_SUPPORT (0xf0) during BSSMAP RESET (ACK) to announce osmux support to MSC
• osmo-bsc checks for same extension IE during BSSMAP RESET (ACK) receive (this may actually not be needed because MSC is always the first confirmiming osmux through Assig req)
• upon receive of Assig Request with our extension IE GSM0808_IE_OSMO_OSMUX_CID (0xf1), which contains the MSC_mgw CID, osmo-bsc instructs its MGW to use Osmux by asking "CRXC osmuxbridge/*@mgw". Answered "*" is going to be the local CID.
• osmo-mgw still lacks supporting osmuxbrdige and currentlty returns "FAIL": "The transaction could not be executed, because the endpoint is unknown. (500)"
• osmo-msc still unimplemented, I'm testing by hardcoding use_osmux in osmo-bsc Assig Request after parsing for now. I leave this one for later specially since code bomb there is in process of being merged.

Next step: implement osmuxbridge using libosmo-netif osmux API in osmo-mgw.

After looking with a bit more detail on how all the endpoint/trunk/conn stuff is handled, I don't see a point to have a different osmuxbridge endpoint, because anyway one of the 2 conns in the endpoint (the one towards the BTS) is still going to be RTP. So it's better to simply to:

• Both osmux and rtp use rtpbridge (on MGCP_TRUNK_VIRTUAL).
• On MSC-side CRCX towards MGW_bsc, osmo-bsc must add X-Osmo-Osmux (or X-Osmux) instead of using osmuxbridge. CRCX ACK will provide the CID in X-Osmux (adapt it in my osmo-bsc changes).
• Mode in handle_create_con already allocates next CID if X-Osmux is set in CRCX.
• Make sure to have set conn->u.rtp.type = MGCP_OSMUX_* around conn creation time (handle_create_con).
• Then based on conn->u.rtp.type we decide how to allocate socket (rtp port or osmux CID on osmux socket) and send rtp frames (usual rtp, or osmux)

I first thought about adding a new MGCP_CONN_TYPE_OSMUX to "enum mgcp_conn_type", but actually it seems there's a rtp_conn subclass type in "enum mgcp_conn_rtp_type" (struct mgcp_conn_rtp) for that.

Some ideas regarding osmux CID allocation and use:

According to osmux spec, a CID (or CIC, Circuit ID Code) is defined/referenced by following tuple: "{srcip, srcport, dstip, dstport, ssrc}". I think actually we never used ssrc to identify them afaik.
That means Osmux CID for a stream in one direction doesn't necessarily be the same on the other direction for the same call. So a Call for given MS in BSC<->MSC could have uplink CID 1 and downlink CID 2 (so CID are unique and referenced per direction).

When assigning numbering to an osmux CID (0-255), we have to think, in the case of MSC, whether we want the 0-255 range to be unique across all BSCs or unique per BSC conn. Having unique across all BSCs means we can only have around 256 (or 256/2) concurrent calls on-going per MSC, and next call willing to be established will fail due to lack of free CIDs. If we have a 0-255 range for each BSC, then we increase considerably the amount of calls we can place (unlimited as long as we scale with more BSC), so that's desirable.

That means, on the MSC side the CID must be used together with socket conn information to really identify a stream (at least "{srcip, srcport, dstip, dstport}"). That's fine, because for receiving frames we know {dstip, dstport} (same for all us, it's our local addr), and we also know {srcip, srcport} Because MSC passed it to MGW_msc during CRCX/MDCX.
WARNING: what if BSC is under a NAT? then {srcip,dstip} may not the one configured during CRCX/MDCX, what do we do?!
POSSIBLE ANSWER (see below and end too): Add a config mode to use unique incrementing CID across all BSC conns (I think that's what osmo-bsc-nat used to do). Another option would be to split 8 bits of CID to be something like 4 bits BSC id and 4 bits CID, but let's drop this possibility (too limiting in numbers of streams).

As a result, implementation wise, there should be 1 osmux socket conn per BSC<->MSC link on the MGW_msc. So we can actually have only 1 socket locally at the MGW_msc, and differentiate based on {srcip, srcport} (recvmsg) to know to which endpoint should the payload be forwarded to.

If we have more than 1 BSC then it could be that both BSCs select to send using same CID, and then MSC would be confused. That means, as a conclusion, that MSC must decide which CID it will receive on, and thus the BSC is going to use to send osmux frames to it.

So the idea to allocate CIDs would be:

• X-Osmux value provided during MGCP Request (cli->MGW, CRCX, MDCX) defines the "sendCID", that is, the CID the local endpoint uses to send OSmux frames and that the remote endpoint has to look for upon osmux socket recv. In case we use 256 range per BSC, it's fine because we provide the other side endp addr and port (or actually in MDCX sometimes), so MGW knows the {srcip, srcport, CID} to look for.
• X-Osmux value provided during Response (MGW->cli), define the "recvCID", that is, the CID the remote endpoint is going to use to send osmux frames and that the local endp will to listent to. In case we use 256 range per BSC, in here MGW already provides us with {srcip, srcport} to provide together with the CID to the other endpoint (over AoIP).

Thus we can use it this way, steps happen in chronological order:

• MSC requests CID "X-Osmux: *" (wildcard) during CRCX to MGW_msc. This means we don't yet know the "sendCID" towards the MGW_bsc, but still we ask the endpoint to use osmux and provide us with its recvCID.
• MGW_msc acks it with "X-Osmux: 2" towards MSC with {srcip+srcport}A, so we now know this endpoint is going to use CID 2 on {srcip+srcport}A to recv osmux frames.
• MSC provides CID 2 + {srcip+srcport}A (IE AoIP Transport Layer Address) during Assign Request on AoIP to BSC.
• BSC issues CRCX to MGW_bsc with "X-Osmux: 2" + {srcip+srcport}A (received in previous step). This means MGW_bsc endp is configured to send the osmux frames received on CID 2 to {srcip+srcport}A.
• MGW_bsc issues ack with "X-Osmux: 30" + {srcip+srcport}B (the one in MGW_bsc), BSC receives it. This means MGW_bsc endp will use CID 30 to recv osmux frames on {srcip+srcport}B from MGW_msc ({srcip+srcport}A).
• BSC answers Assign Resquest with Assign Response providing CID 30 + {srcip+srcport}B received above.
• MSC receives Assign Response with CID 30 + {srcip+srcport}B and issues MDCX "X-Osmux: 30" on MGW_msc. Now MGW_msc knowns it recives using CID 2 on {srcip+srcport}A and sends to CID 30 on {srcip+srcport}B.

^ Following setup verifies that same code of osmo-mgw works for both MSC and BSC without any specific code (only the "*" feature to leave dstip+dstport for later).

So coming back to the BSC-behind NAT issue:
Since MSC allocates it's own recvCID for each conn, if we enable the "NAT-proof" config in MGW_msc then we can easily have incrementing-for-all-bsc CID allocator value instead of having it per BSC, and we disable checking {srcip+srcport} passed during call setup time. Since CIDs are unique we can always match the corresponding endpoint, and in this mode once we receive some packet, we get the UDP header and overwrite the MGW_msc {dstip+dstport} for that osmux socket (since the BSC did the hole in the NAT for us and we now know how to reach it back).

Comments above are the generalized way. So far, while re-using current osmux code base, we'll start with a simplification and keep growing towards what was described in above comment. Current status is explained inside brackets.
All work is done in pespin/osmux branch of libosmocore, osmo-mgw and osmo-bsc, with some preparatory patches already in gerrit.

Simplified way:

• MSC Requests Osmux Cid during CRCX to MGW_msc with "X-Osmux: *" and MGW_msc answers with "X-Osmux: 30" (whatever next CID is avail). <-- [osmo-mgw, libosmo-mgcp-cli side: DONE. MSC side: TODO]
• MSC sends the osmux CID on Assignment Request to BSC <-- [MSC: TODO]
• BSC receives osmux CID and submits CRCX to MGW_bsc with "X-Osmux: 30". MGW responds with "X-Osmux: 30" (so same CID is used everywhere). <- [osmo-mgw, libosmo-mgcp-cli side: DONE. BSC side: almost DONE, need to verify CRCX response is parsed correctly on BSC]
• BSC submits Assign Response with the osmux CID to MSC <-- [BSC: WIP for some reason it sends Assign Failure]
• MSC parses CID from Assign Response and does MDCX to update the MGW_msc <-- [MSC: TODO. MGW: TODO, but not really needed if we use same CID]
• Make sure the osmux data plane works fine <-- [MGW: WIP, some stuff will require updating to current osmo-mgw code base]

So first implementation will have these premises:

• MSC always allocates first the CID, and uses same CID to send and receive for both directions of a call leg.
• MSC allocates CIDs globally, not per BSC, meaning 2 BSCs cannot collide on CIDs.
• BSC uses the CID told by the MSC through AoIP both for sending and receiving on that leg call.

From that point, basically 2 different things are required to go to more general scenario:

• change osmo-mgw code to have separate receiveCID and sendCID, so a call can send on one CID and receive on another one.
• Add code to osmo-mgw to handle CID pool per BSC (actually per remote endp detected or passed as dst addr from MSC).

• libosmocore patch to allow Osmux CID passing in AoIP:
  https://gerrit.osmocom.org/#/c/libosmocore/+/14017/ gsm0808: Introduce Osmocom extensions to announce Osmux support

• osmo-mgw patches submitted with osmux fixes (others have been already been merged previously):
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14019 cosmetic: mgcp_udp_send: Document port param is in network byte order
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14020 cosmetic: osmux: Document network byte order in port variable
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14021 vty: Allow enabling Osmux
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14022 osmux: Allocate CID during CRCX
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14023 osmux: Introduce mgcp_conn_rtp_is_osmux() helper
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14024 osmux: Mark conn_rtp->type as osmux during CRCX
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14025 mgcp-cli: Allow submitting X-Osmux on CRCX request
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14026 mgcp-cli: Parse X-Osmux on CRCX response
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14027 osmux: Handle Osmux MGCP extension in MDCX messages
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14028 mgcp-cli: Allow submitting X-Osmux on MDCX request
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14029 mgcp-cli: Parse X-Osmux on MDCX response
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14030 osmux: Drop unneeded OSMUX_STATE_NEGOTIATING
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14031 mgcp-cli: endpoint_fsm: Add API to retrieve Osmux CID from MGW
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14032 osmux: Provide correct local port during mgcp resp
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14033 osmux: Use remote port to send osmux frames
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14034 osmux: Improve logging around osmux enabling events
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14035 osmux: Delay osmux enable of conn until remote addr is configured by MDCX
  remote: https://gerrit.osmocom.org/#/c/osmo-mgw/+/14036 osmux: Redo read/write osmux glue code to have data routed correctly

• osmo-bsc:
  remote: https://gerrit.osmocom.org/#/c/osmo-bsc/+/13922 vty: Add option to enable osmux towards MSC
  remote: https://gerrit.osmocom.org/#/c/osmo-bsc/+/13923 bssap: Detect MSC Osmux support on RESET (ACK) recv
  remote: https://gerrit.osmocom.org/#/c/osmo-bsc/+/13924 bssap: Announce Osmux support on RESET (ACK) send
  remote: https://gerrit.osmocom.org/#/c/osmo-bsc/+/13925 bssap: Parse Osmux CID on BSSAP Assign Req recv and use it in MGCP
  remote: https://gerrit.osmocom.org/#/c/osmo-bsc/+/13927 bssap: Feed Assign Complete with Osmux CID retrieved from MGW

• osmo-msc:
  remote: https://gerrit.osmocom.org/#/c/osmo-msc/+/14037 ran_peer: Move rx_reset_ack logic into its own func
  remote: https://gerrit.osmocom.org/#/c/osmo-msc/+/14038 vty: Add option to enable osmux towards BSCs
  remote: https://gerrit.osmocom.org/#/c/osmo-msc/+/14039 bssap: Detect BSC Osmux support on RESET (ACK) recv
  remote: https://gerrit.osmocom.org/#/c/osmo-msc/+/14040 a_iface: Announce Osmux support on RESET (ACK) send
  remote: https://gerrit.osmocom.org/#/c/osmo-msc/+/14041 Request Osmux CID and forward it in Assign Req and Assign Compl

Some of them can be considered as RFC and may require some changes/refactoring, specially the osmo-msc ones.

Initial Osmux support for TTCN3 tests:
https://gerrit.osmocom.org/#/c/osmo-ttcn3-hacks/+/14046 mgw: Add 2 CRCX tests with Osmux enabled
https://gerrit.osmocom.org/#/c/osmo-ttcn3-hacks/+/14048 WIP: Introduce Osmux support in TTCN3 and use it in MGCP_Test [WIP]
https://gerrit.osmocom.org/#/c/docker-playground/+/14047/ mgw: Enable osmux in osmo-mgw.cfg

Also found out issues with our osmo-mgw checks not being case insensitive, so I opened a ticket (#4001) and provided a patch to have Osmux handled correctly that way.

I have TTCN3 infrastructure for Osmux working fine with a bunch of tests in mgw ttcn3 tests.

TODO:

• Make "osmux on" in osmo-mgw.cfg happen dynamically by calling VTY cmd on every test during vty_init().
• Add a TTCN3 parameter to enable/disable Osmux related jobs, since we don't want to run it on osmo-mgw latest (osmux not supported there until next release).
• Test the TTCN3 tests in docker container (been running them in my PC directly).
• Add TTCN3 tests in BSC and MSC to check osmux support + osmux cid features work fine (and catch regresions)
• Write documentation
• Check status of BSc-NAT TTCN3 tests and add Osmux support there.
• Add osmo-gsm-tester test with Osmux
• Test new osmo-bsc + osmo-mgw with BSC-NAT setup (I'll need the other missing feature about encapsulating MGCP in IPA afair).

TODO:

• Add osmo-gsm-tester test with Osmux and AoIP.
• Send patch to wireshark for new AoIP IEs.
• Write documentation
• Test new osmo-bsc + osmo-mgw with BSC-NAT setup (requires MGCP-over-IPA #2536 and OAP #2013).
• Once done, create a ticket to implemented more generalized Osmux CID support (sendCID!=recvCID) and close this one.

Submitted patches for wireshark to support the introduced BSSMAP IEs for AoIP:

remote: https://code.wireshark.org/review/33667 BSSMAP: Use correct IE number for Selected PLMN ID
remote: https://code.wireshark.org/review/33668 BSSMAP: Introduce Osmocom Osmux Support and CID extension IE

Osmux test was added to osmo-gsm-tester.
wireshark patches were merged.
Documentation is being handled in #4090.
Testing with bsc-nat handled in #2554.
Generalization of CID allocating handled in #4090.

Initial support is there, some specifics/improvements still need work and are being handled in separate issues. Closing this ticket.