Project

General

Profile

Actions

Feature #6448

open

Support Themyscira RTP extensions in Osmocom BSS

Added by falconia 22 days ago. Updated 4 days ago.

Status:
Stalled
Priority:
Normal
Assignee:
Target version:
-
Start date:
05/08/2024
Due date:
% Done:

20%

Spec Reference:

Description

As discussed in OsmoDevCall 2023-06-21, the industry transition from TDM transport per TS 48.060 & 48.061 to RTP transport in the standard formats of RFC 3551 or TS 101 318 (same format defined in both specs) has brought on a functional regression: there are metadata bits present in 48.060/48.061 UL frame formats that have been dropped in the standard RTP formats. Themyscira Wireless is seeking to recreate the classic GSM architecture in its full glory, but do so under the constraint of economic realities that only allow IP transport between physically distant network elements. In order to implement a network edge transcoder that logically and semantically functions like a classic GSM TRAU (of MSC-colocated variety) despite physically being nothing more than ordinary Unix/Linux processes running on a rented server with only IP connectivity, we had to invent our own non-standard extensions to RTP payload formats for transport within a GSM RAN. TW-TS-001 and TW-TS-002 are formal specifications for Themyscira RTP extensions, corresponding to TS 48.060 and 48.061, respectively:

https://www.freecalypso.org/specs/tw-ts-001-v010100.txt
https://www.freecalypso.org/specs/tw-ts-002-v010100.txt

However, 3GPP TS 48.103 (the spec for AoIP user plane) says indirectly (by referencing 26.102) that RTP payload formats at the AoIP interface shall be those of RFC 3551 (FR & EFR) and RFC 5993 (HR). Therefore, if a given network needs to pass TW-TS-001 & TW-TS-002 payloads instead across that interface, it will be in direct violation of TS 48.103. But what if a given CN implementation really needs these Themyscira RTP extensions? Themyscira government's official answer is an extension to BSSMAP by which the MSC asks the BSS to use the RTP formats in question and thereby deviate from 48.103, codified here:

https://www.freecalypso.org/specs/tw-ts-003-v010001.txt

Now bringing this matter to Osmocom: my current vision (as Themyscira HPS of telecom) is that the AoIP interface shall serve as the boundary between Osmocom-controlled and Themyscira-controlled network elements. On the CN side we'll be implementing our own MSC (based on OsmoMSC, but with changes so big that a merge doesn't seem likely), but for the 3 Osmocom BSS sw components (OsmoBSC, OsmoBSC-associated OsmoMGW and OsmoBTS), I am hoping to be able to continue using Osmocom mainline, if we can merge the small changes required for Themyscira RTP extensions.

In order to add support for Themyscira RTP extensions to Osmocom BSS, there will need to be signaling at both AoIP and Abis-IP interfaces negotiating the use of these non-standard extensions, with the requestor at each interface asking for the extension to be used and the respondent acknowledging or failing to acknowledge that request. At the AoIP interface the signaling extension is codified in TW-TS-003, but its counterpart for Abis-IP will need to be defined entirely in Osmocom, given that Abis-IP as it currently exists is inherently an Osmocom-specific interface altogether.

Here are current Gerrit patches implementing the initial parts of this feature proposal, namely signaling interface definitions:

https://gerrit.osmocom.org/c/libosmocore/+/36669
https://gerrit.osmocom.org/c/libosmocore/+/36704
https://gerrit.osmocom.org/c/libosmocore/+/36705
https://gerrit.osmocom.org/c/osmo-bts/+/36706


Checklist

  • add TW-TS-003 definition to libosmocore gsm0808
  • define Osmocom Abis-IP extensions for signaling the use of Themyscira RTP formats
  • implement the necessary support in OsmoBSC
  • implement the necessary support in OsmoBTS
Actions #1

Updated by falconia 22 days ago

As noted in the initial ticket, implementation of this feature will require defining a mechanism whereby OsmoBSC can test whether or not a given IP BTS supports Themyscira RTP extensions, and can ask the BTS to use these enhanced RTP payload formats if the CN asked for them via TW-TS-003 BSSMAP extension. In furtherance of this goal, I propose the following extensions to Osmocom-controlled Abis-IP interface:

  1. BTS feature flags indicating support for TW-TS-001 and TW-TS-002, one flag for each.
  2. An Osmocom-specific IE in Abis RSL, to be added to CRCX and MDCX command messages, directing the BTS to use specific RTP extensions.

Here are libosmocore patches implementing parts 1 and 2 of the above proposal:

https://gerrit.osmocom.org/c/libosmocore/+/36704
https://gerrit.osmocom.org/c/libosmocore/+/36705

plus this patch to the Abis document:

https://gerrit.osmocom.org/c/osmo-bts/+/36706

I have marked them as WIP in Gerrit so that the principal idea can be discussed in the present ticket, before delving down into line-by-line code review of the final implementation.

One question has come up in Gerrit review of the Abis document patch, before it was changed to WIP: should this new IE be defined as having TV or TLV format? From my PoV, I say that TV would be more efficient: the ultimate origin of the request to use Themyscira RTP formats is the CN, which asks the BSS to use these RTP formats via TW-TS-003 BSSMAP extension. OsmoBSC will do little more than pass this special request along from the A interface to the BTS (Abis RSL), hence it makes logical sense for the new IE on Abis RSL to mimic the one on the A interface - and that one is defined as a TV-format IE in TW-TS-003. However, it would be entirely reasonable for other Osmocom developers to argue that the new IE on Abis RSL should be a TLV so that it can be extended later to do other functions that aren't related to TW-TS-003 per se. Hence the present discussion - which should it be?

Actions #2

Updated by pespin 21 days ago

falconia thanks. I'd like to have the full patchset in gerrit implementing all the bits here and there before merging stuff (even if some patches depending on libosmocore changes still fail when running in jenkins gerrit verification).

Then once we are fine with the full patchset, merge them together.

Actions #3

Updated by falconia 21 days ago

In Gerrit review of the existing introductory patches (adding definitions of new IEs on A and Abis interfaces), laforge and pespin indicated that new IEs shall be in TLV coding format, not TV. This ruling conflicts with the original instruction of TW-TS-003 spec - however, because that spec is still very new and open to revision, I have issued a new revision of the spec that conforms to the will of Osmocom in this regard:

https://www.freecalypso.org/specs/tw-ts-003-v010002.txt

Actions #4

Updated by falconia 21 days ago

pespin wrote in #note-2:

falconia thanks. I'd like to have the full patchset in gerrit implementing all the bits here and there before merging stuff (even if some patches depending on libosmocore changes still fail when running in jenkins gerrit verification).

Then once we are fine with the full patchset, merge them together.

Ack. Looking at my current work queue, for the next few days I will probably be busy putting together the next release of ThemWi GSM codec libraries and utilities package, but then I'll switch gears to putting together the necessary test rig that will enable me to develop and test the patches to OsmoBSC and OsmoBTS for the present issue. In the meantime, this branch will facilitate builds of components that depend on libosmocore patches:

https://cgit.osmocom.org/libosmocore/?h=falconia/os6448

Actions #5

Updated by falconia 8 days ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 20

I got one more patch to libosmocore, beyond those that got a preliminary review earlier:

https://gerrit.osmocom.org/c/libosmocore/+/36895

and I also implemented RSL-controlled TW-TS-001 support in OsmoBTS:

https://gerrit.osmocom.org/c/osmo-bts/+/36896

(fails Jenkins build because it needs libosmocore changes)

However, implementing the necessary support in OsmoBSC will take me longer - I am much less familiar with that code, and the logic is more complex. Hence I am making this update so that any interested parties can see where I am with it.

Actions #6

Updated by falconia 7 days ago

neels made a general (non-specific) comment in Gerrit under this patch:

https://gerrit.osmocom.org/c/libosmocore/+/36669

I tried writing the reply in the same place, but for some reason Gerrit web interface is not letting me compose a reply there. Hence I am replying in this ticket, which is probably a better place for the general discussion anyway.

this seems to be a single person effort.

Just because I am the sole developer currently working on this project does not mean that others can't benefit from it - on the contrary, the idea (like with most FOSS) is to produce a result that others would appreciate. The software suite which I am producing under my choice of branding (yes, named after the fictional home of Wonder Woman, FOSS project founders get to choose the name) is free to the world (public domain), and when used together with Osmocom CNI components, this combination of Osmocom+ThemWi allows any GSM/2G lover anywhere in the world to run a GSM network of the same quality as the major commercial ones that are now being wrongfully shut down in a lot of places.

but on the other hand there are bounds to what amount of API we can hope to maintain.

If you are not willing to merge these changes and therefore I will have to maintain my own forks of all affected Osmocom CNI components (the functionality in question is absolutely required for ThemWi to work) until the end of time, would you at least be willing to merge a teensy-tiny patch that merely reserves the IEI code points I am using on BSSMAP and RSL, so someone else in Osmocom won't grab the same IEI code points for something entirely different?

I find this curious description of apparently a single person, I suspect the author of this patch =)

"under the authority of the Presiding Sisterhood (government) of the Women's Republic of Themyscira

The Presiding Sisterhood currently consists of 3 members; I am the only one working on telecom-related stuff (as High Priestess of telecommunications), the other 2 sisters work on stuff that has no relevance to Osmocom - yet the authority is still that of the Presiding Sisterhood. Not sure if we should continue this subthread here - probably not...

Adding custom IEs to BSSMAP is not done in osmocom yet,

I beg to differ: please look at GSM0808_IE_OSMO_OSMUX_SUPPORT and GSM0808_IE_OSMO_OSMUX_CID. When assigning the IEI code for TW-TS-003, I took the code point right after those two.

and we have no way of indicating the BSS' custom capabilities to osmo-msc. How do you solve that?

The next version of ThemWi software suite, the one I am beginning to work on, will not use OsmoMSC - it will use its own ThemWi-MSC instead. Because the functionality in question is absolutely required for ThemWi to work, ThemWi-MSC will always include the RTPext IE unconditionally in the ASSIGNMENT REQUEST message, and if the BSS does not support it, then that BSS can't work with ThemWi. The intent is to use Osmocom BSS, preferably with the needed patches mainlined, or else a fork thereof if my good-faith efforts are rejected with prejudice.

BTW, if we have custom BSSMAP IEs, then I might want to add an IE to use an array of octets to describe all permitted AMR bitrates

No objection from my side, as long as you pick an IEI code point for it other than 0xF2. But it should probably go into a separate feature ticket...

But there are ample reasons to not deviate from 3GPP if at all possible.

The problem is that 3GPP broke things in the transition from TDM to AoIP: the RTP payload formats stipulated by TS 48.103 (the industry standard ones that originated from non-3GPP, non-GSM world of VoIP) exhibit functional regressions relative to 3GPP TS 48.060 and 48.061 in the TDM world. Themyscira Wireless requires functionality just like TDM (08.60 & 08.61), but under economic constraints that allow only IP connectivity between physically distant network elements - hence the present work as the proposed solution.

I would need a good TL;DR to start comprehending.

TL;DR: the standard RTP formats make it impossible to implement a network edge transcoder to PSTN that correctly implements the in-band protocol of TS 28.062 inside the G.711 PCM sample stream; this snafu is likely the reason why this TFO protocol is implemented only in T1/E1 TRAUs and nowhere else apparently. I seek to be the first free sw developer to implement TS 28.062 in the flavor of RTP (PCMU/PCMA, no GSM or AMR codecs) used by USA PSTN, and do it in FOSS, with zero proprietary components. To boldly go where no free sw developer has gone before - isn't that the mission of Osmocom?

Perhaps we can discuss these issues live in the next OsmoDevCall? I can keep the slides portion short (much shorter than my previous presentations), to leave more time for interactive discussion.

Actions #7

Updated by falconia 4 days ago

  • Checklist item add TW-TS-003 definition to libosmocore gsm0808 set to Done
  • Checklist item define Osmocom Abis-IP extensions for signaling the use of Themyscira RTP formats set to Done
  • Checklist item implement the necessary support in OsmoBTS set to Done
  • Status changed from In Progress to Stalled

I am now blocked in terms of further progress on this task - I am not able to make further progress until libosmocore patches are merged. The two remaining tasks are:

  1. Test my OsmoBTS patch (marked as WIP in Gerrit) on real hardware;
  2. Develop the patch for OsmoBSC, using a test network with patch-applied OsmoBTS as the necessary test environment.

I tried doing these tasks with libosmocore patches unmerged (like pespin asked earlier in this ticket), but I ran into this roadblock: the only suitable BTS hw I have for testing is sysmoBTS, and while I have no problem building custom (unmerged patches) versions of osmo-bts-sysmo binary using mainline libs (as they appear in sysmobts-v2/201705-nightly), when I tried to build custom libosmocore for sysmoBTS ARM world, I got deeper into the weeds than I can currently untangle.

(A clarification regarding testing: the RTP parts of the OsmoBTS patch under consideration have already been extensively tested, they are already deployed in production with a different patch that enables TW-TS-001 with a local vty option, without involving BSSMAP or RSL signaling. Instead the part that needs to be tested is control via RSL, and that part is where the dependency on libosmocore patches bites.)

I am OK with waiting until June OsmoDevCall in which we can discuss this issue and hopefully come to an agreement on the principal idea, before delving into the details of individual patches.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)