https://projects.osmocom.org/https://projects.osmocom.org/favicon.ico?16647414092022-10-25T09:47:24ZOpen Source Mobile CommunicationsOsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=252102022-10-25T09:47:24Zdexter
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=252182022-10-25T11:17:19Zdexter
<ul></ul><p>Note: The same could be done for AMR BWE aligned packets which. Those are dropped in l1sap.c. We could also just convert them.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=257712022-12-08T16:08:08Zdexter
<ul><li><strong>% Done</strong> changed from <i>0</i> to <i>10</i></li></ul><p>I have now submitted a patch that lets osmo-bts accept any of the two HR GSM formats:</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/30523">https://gerrit.osmocom.org/c/osmo-bts/+/30523</a> l1sap: Accept RFC5993 and TS 101.318 HR GSM payload</p>
<p>The format conversion is done with minimal effort, so that no additional memcpy() or buffers are introduced. However, even if we now accept both formats we still may emit the wrong format. One idea to get around this would be to make the TX format adaptive so that we always send RTP packets in the same format as we receive, but that will be in a follow up patch.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=258192022-12-16T10:09:27Zdexter
<ul><li><strong>% Done</strong> changed from <i>10</i> to <i>80</i></li></ul><p>The problem on the receiving end of the BTS should now be solved since the BTS will now accept both formats. However, the receiving side is still a problem. We already have a various extra flags in the ASSIGNMENT COMMAND. I added one to signal the type of the HR GSM RTP audio. So we now can configure which format the BTS should use.</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/30629">https://gerrit.osmocom.org/c/osmo-bts/+/30629</a> rsl: allow configuration of HR GSM RTP format via CHANNEL ACTIVATE [NEW]<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bsc/+/30630">https://gerrit.osmocom.org/c/osmo-bsc/+/30630</a> abis_rsl: signal HR GSM RTP format to BTS via RSL [NEW]</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=259282023-01-16T10:33:07Zdexter
<ul></ul><p>We are currently discussing if it is such a good idea to configure the format via RSL, it may be better to have it directly in the osmo-bts.cfg: <a class="external" href="https://gerrit.osmocom.org/c/libosmocore/+/30724">https://gerrit.osmocom.org/c/libosmocore/+/30724</a></p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=261792023-02-13T09:37:02Zdexter
<ul><li><strong>% Done</strong> changed from <i>80</i> to <i>50</i></li></ul><p>As discussed we now follow a more generic approach. The MGW will get the information about which of the two GSM HR format is used via SDP from the BSC. The BSC is fully aware of the BTS type and therefore also has the information which format is used.</p>
<p>There is a patch for the MGW and a related patch for the TTCN3 testsuite in gerrit:<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-mgw/+/31214">https://gerrit.osmocom.org/c/osmo-mgw/+/31214</a><br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/31205">https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/31205</a></p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=262212023-02-20T16:15:27Zdexter
<ul></ul><p>I have updated the patch for osmo-mgw so that it now also generates the FMTP string using the osmo-mgcp-client.</p>
<p>The patch for osmo-bts has also received an update. We still accept both formats but now we have to use BTS feature flag so that the info about the preferred Format is available to the BSC.<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/31417">https://gerrit.osmocom.org/c/osmo-bts/+/31417</a></p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=264052023-03-20T10:42:28Zdexter
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Stalled</i></li></ul><p>This task is currently blocked by <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet format (Resolved)" href="https://projects.osmocom.org/issues/5688">#5688</a> due to a conflict in the osmo-mgcp-client API.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=265732023-03-30T16:28:16Zneelsnhofmeyr@sysmocom.de
<ul></ul><p>dexter wrote in <a href="#note-9">#note-9</a>:</p>
<blockquote>
<p>This task is currently blocked by <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet format (Resolved)" href="https://projects.osmocom.org/issues/5688">#5688</a> due to a conflict in the osmo-mgcp-client API.</p>
</blockquote>
<p>This here is <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet format (Resolved)" href="https://projects.osmocom.org/issues/5688">#5688</a> -- it is blocked by #5723</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269182023-05-17T21:50:20Zfalconia
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-1 priority-2 priority-default" href="/issues/6036">Feature #6036</a>: Implement ternary SID classification for HR1 uplink</i> added</li></ul> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269202023-05-17T22:54:12Zfalconia
<ul></ul><p>The way in which this bug has been handled so far has been rather disappointing. In my opinion, this bug ticket conflates two separate problems:</p>
<p>Issue 1: in the realm of interoperable interfaces between components from different vendors, there exist two different RTP encoding formats for HR1.</p>
<p>Issue 2: internally to OsmoBTS, libosmocoding functions used by osmo-bts-trx are poorly designed in that they natively implement a pseudo-RFC5993 format, causing osmo-bts-trx to treat RFC 5993 as its native.</p>
<p>These issues need to be addressed separately. For Issue 1 (interoperability), the best principle is "be conservative in what you send and liberal in what you accept": have OsmoBTS accept both formats in RTP input, and produce one format or the other on output per vty option, consistent across all models.</p>
<p>In previous comments on this ticket and related patch proposals, there is a recurring but misguided notion that different PHY implementations may have either TS 101 318 or RFC 5993 as their "native" format for HR1, and that higher-level design of OsmoBTS (and even supporting library functions like SID checks, ECUs etc) needs to dance around these two different HR1 formats percolating up and down the stack. I contest this idea:</p>
<ul>
<li>Thinking in philosophical terms, the "true" HR1 frame payload, as in a Platonic ideal or a Kantian thing-in-itself, is 112 bits or 14 octets, whereas TS 101 318 and RFC 5993 are two different external representations of this Platonic ideal. It just so happens that the "true internal" Platonic ideal form of HR1 and the TS 101 318 representation are bitwise identical - but philosophically they are separate.</li>
</ul>
<ul>
<li>By the very nature of what a PHY needs to do (GSM 05.03 channel encoding and decoding), BTS PHYs should operate on the "Platonic ideal" form of HR1 frame payload (which just happens to be the same as TS 101 318), whereas external representation may be that or RFC 5993.</li>
</ul>
<ul>
<li>gsm0503_tch_hr_decode() does not actually emit RFC 5993 output, instead it emits pseudo-5993 or bogo-5993: it emits one constant 0x00 octet followed by 14 octets of "Platonic ideal" HR1 payload. This output is NOT valid RFC 5993: in true RFC 5993 the FT field in the ToC octet needs to be set to the result of SID classification decision (see <a class="issue tracker-2 status-1 priority-2 priority-default" title="Feature: Implement ternary SID classification for HR1 uplink (New)" href="https://projects.osmocom.org/issues/6036">#6036</a>), but gsm0503_tch_hr_decode() always writes 0. gsm0503_tch_hr_encode() should likewise be seen as accepting payloads consisting of one totally ignored octet (no ToC checks are done) followed by 14 octets of "true" HR1 payload.</li>
</ul>
<ul>
<li>I propose this fix to libosmocoding: extend gsm0503_tch_hr_encode() so it will accept both 14-byte and 15-byte formats (the latter being only for backward compat with current bad design, with the extra octet ignored), and add gsm0503_tch_hr_decode2() function that will be exactly like current gsm0503_tch_hr_decode(), but without the extra bogon octet in the output.</li>
</ul>
<ul>
<li>In the extremely unlikely case that we run into a third-party proprietary PHY that takes and/or emits HR1 payload in a form that is claimed to be RFC 5993, we'll need to experiment with it (behavioral reverse eng) to see what it actually does with the extra octet in the input, and what it puts into that extra octet in the output - and then decide from there on the most appropriate course of action.</li>
</ul>
<p>The takeaway is that we should streamline OsmoBTS code to treat the 14-octet form of HR1 as canonical internally, and implement RFC 5993 at the point of external interfaces. Ironically, however, despite the internal form being the 14-octet one, the preferred external form should be RFC 5993, with TS 101 318 as the non-preferred alternative. Why? See <a class="issue tracker-2 status-1 priority-2 priority-default" title="Feature: Implement ternary SID classification for HR1 uplink (New)" href="https://projects.osmocom.org/issues/6036">#6036</a> for the rationale.</p>
<p>If the community is agreeable to the ideas I just outlined, I am willing to implement them in code, i.e., prepare and submit the necessary patches to libosmocodec, libosmocoding and osmo-bts, covering both the present ticket and <a class="issue tracker-2 status-1 priority-2 priority-default" title="Feature: Implement ternary SID classification for HR1 uplink (New)" href="https://projects.osmocom.org/issues/6036">#6036</a>. However, I will only be able to begin this HR1 codec work after all of my currently outstanding patches for FR and EFR codecs have been merged.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269212023-05-18T04:41:55Zfalconia
<ul></ul><p>I forgot to add: if people like the ideas I outlined in my previous comment and would like me to follow through with actually implementing them, please feel free to assign this ticket to me.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269672023-05-23T18:30:33Zfalconia
<ul><li><strong>Status</strong> changed from <i>Stalled</i> to <i>In Progress</i></li><li><strong>Assignee</strong> changed from <i>dexter</i> to <i>falconia</i></li></ul><p>The libosmocoding portion of my proposed solution has already been merged:</p>
<p><a class="external" href="https://cgit.osmocom.org/libosmocore/commit/?id=8d8d5af957dd0abecc9f77d311f59a9076c2585a">https://cgit.osmocom.org/libosmocore/commit/?id=8d8d5af957dd0abecc9f77d311f59a9076c2585a</a><br /><a class="external" href="https://cgit.osmocom.org/libosmocore/commit/?id=57a3b3a51fff40cff46c632285cd91e7458bf84e">https://cgit.osmocom.org/libosmocore/commit/?id=57a3b3a51fff40cff46c632285cd91e7458bf84e</a></p>
<p>In just a few hours from now I will have a clean patch to osmo-bts that makes all models accept both formats in RTP output and emit one or the other format per vty configuration, thereby fully solving the present bug ticket.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269682023-05-23T19:11:07Zfalconia
<ul></ul><p>Brown paper bag moment:</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/libosmocore/+/32967">https://gerrit.osmocom.org/c/libosmocore/+/32967</a></p>
<p>Please forgive your old 1980s UNIX gal, this whole DSO business is rather foreign to me and smells too much like Windows DLLs.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269692023-05-23T19:23:00Zlaforge
<ul></ul><p>On Tue, May 23, 2023 at 07:11:07PM +0000, falconia wrote:</p>
<blockquote>
<p>Please forgive your old 1980s UNIX gal, this whole DSO business is rather foreign to me and smells too much like Windows DLLs.</p>
</blockquote>
<p>note that symbol visibility in ELF .so files is a 90s topic, I'm quite sure :P</p>
<p>I find it rather useful to be able to have symbols shared between files within a shared library,<br />but not expose those to the global symbol namespace of the application. Yes, usability with the .map<br />file sort-of sucks, but the goal justifies the means.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269702023-05-23T21:41:10Zfalconia
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li><li><strong>% Done</strong> changed from <i>50</i> to <i>100</i></li></ul><p>The following patch series implements all needed functionality:</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/32968">https://gerrit.osmocom.org/c/osmo-bts/+/32968</a><br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/32969">https://gerrit.osmocom.org/c/osmo-bts/+/32969</a><br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/32970">https://gerrit.osmocom.org/c/osmo-bts/+/32970</a></p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269752023-05-25T09:04:04Zdexter
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>In Progress</i></li></ul><p>Thank you very much for contributing. You are welcome to continue this ticket. I was approaching the problem from two sides:</p>
<p>1) The MGW currently converts HR audio by just converting it in the opposite format. This of course may lead to confusion. It works most setups and as far as I know we never had problems with this. But it definitely will become a problem when the BSC has to manage BTSs that have opposite formats.</p>
<p>There is a patch for osmo-mgw (<a class="external" href="https://gerrit.osmocom.org/c/osmo-mgw/+/31214">https://gerrit.osmocom.org/c/osmo-mgw/+/31214</a>) that introduces an fmtp string to signal the HR format to osmo-mgw. When this feature is used, the HR format is explicitly negotiated.</p>
<p>Unfortunately the patch is blocked because it does modifications to the osmo-mgcp-client API that has a design problem. We have decided to fix that design problem first (see <a class="external" href="https://osmocom.org/issues/5723">https://osmocom.org/issues/5723</a>, we also have decided how this has to be fixed, I will start doing this as soon there is time to do it).</p>
<p>Technically this is an entirely different location, but I wanted to make you aware of this part. (The progress is set to 100% now but since the MGW part is not yet done I was setting it to 50%)</p>
<p>2) Different osmo-bts implementation may use a different HR format. This is of course a problem which needs to be resolved. I was trying to solve this in two steps: First I wanted to make sure that no incompatible payload enters the BTS code, so my Idea was to filter non supported formats. In order to know which format is supported I intended to use BTS feature flags.</p>
<p>For convenience I intended to convert arriving RTP payloads that do not match the supported HR format. This is not strictly needed and one might to argue about this. After all payloads in the wrong format are the result of another problem and maybe it is better to have a completely silent call rather then a call that is silent in one direction only.</p>
<p>The following patches are currently in review, maybe you can re-use parts of it:<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/31417">https://gerrit.osmocom.org/c/osmo-bts/+/31417</a> l1sap: Drop invalid HR GSM payload<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/32630">https://gerrit.osmocom.org/c/osmo-bts/+/32630</a> l1sap: Accept RFC5993 and TS 101.318 HR GSM payload<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-bts/+/32752">https://gerrit.osmocom.org/c/osmo-bts/+/32752</a> l1sap: cosmetic: rename payload_len to rtp_pl_len</p>
<p>The next step would have been to add a way to configure the HR format that the BTS should transmit in the core network direction. We already discussed this and opted for a simple VTY option in osmo-bts. As I can see you already implemented this part.</p>
<p>From what I can see we are basically done on the BTS side, we only have to get those patches through review. The ticket is currently assigned to you. You can assign it back to me when the BTS side is done.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269822023-05-25T20:16:56Zfalconia
<ul><li><strong>% Done</strong> changed from <i>100</i> to <i>80</i></li></ul><p><a class="user active" href="https://projects.osmocom.org/users/15">dexter</a> - thanks for explaining the background, and what happened before I jumped in.</p>
<blockquote>
<p>The MGW currently converts HR audio by just converting it in the opposite format. This of course may lead to confusion. It works most setups and as far as I know we never had problems with this.</p>
</blockquote>
<p>I wonder about that "works most setups" part - does anyone among "real" network deployments (beyond Osmocom developers testing everything) actually use HRv1 codec, and if so, why? I thought I was the only person in the Osmocom community who is interested in deploying GSM networks specifically for providing service to vintage mobile phones, with all other "real" Osmocom network operators assuming that user handsets are reasonably "modern" - and AMR support in handsets goes back to at least 2005, or probably a few years earlier with manufs who didn't drag their feet. And if you have AMR support, why would a real operator run with HRv1 rather than AMR-HR?</p>
<p>I personally am not really interested in deploying this HR codec either - although I am doing full retro, I seek to mimic the practices of the existing commercial GSM network in USA that is about to be shut down - and as far as I know, they never used HR. I've experimented with sending different lists of supported speech codec versions in the Bearer cap IE from a test MS, and I can never get the network to give me TCH/HS, even if I construct a weird Bearer cap IE that says that the MS prefers HR over everything else. Instead this network always goes for AMR-HR if the MS supports it; if the MS lacks AMR support, then the network allocates TCH/F instead of TCH/H and selects EFR if the MS supports it, or FR otherwise. The specs require every MS to support FRv1, with all others being optional in any combination.</p>
<p>Back to the present Osmocom ticket, I took this issue up and I am pursuing my preferred solution (OsmoBTS part) in Gerrit because I have other patches that I care about for FR & EFR, the two non-AMR codecs I actually use in my network deployment - and because DTX specs are mostly consistent between FR/EFR and HR (the only major diff is the lack of fixed bit-counting rules for HR, see <a class="issue tracker-2 status-1 priority-2 priority-default" title="Feature: Implement ternary SID classification for HR1 uplink (New)" href="https://projects.osmocom.org/issues/6036">#6036</a>), the work I am doing for FR/EFR also needs to extend to HR1. And if your "competing" osmo-bts patches for the present issue were to be merged, that solution would completely preclude any possibility of me extending my FR/EFR work to also cover HR the same way - hence me jumping in passionately with a counter-proposal of different design.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269922023-05-27T11:12:37Zfalconia
<ul><li><strong>Assignee</strong> changed from <i>falconia</i> to <i>dexter</i></li></ul><p>The OsmoBTS portion of this task is done. I am reassigning this ticket back to <a class="user active" href="https://projects.osmocom.org/users/15">dexter</a> for the OsmoMGW part as instructed.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=269952023-05-27T14:35:49Zfalconia
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>% Done</strong> changed from <i>80</i> to <i>100</i></li></ul><p>Applied in changeset <a class="changeset" title="all models, HR1 codec: select RTP output format via vty option The new vty option "rtp hr-format..." href="https://projects.osmocom.org/projects/osmobts/repository/osmo-bts/revisions/c18f9f256c905d845c0060f19f3cd46cc40a2fe5">osmo-bts|c18f9f256c905d845c0060f19f3cd46cc40a2fe5</a>.</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=270792023-06-13T10:40:44Zdexter
<ul></ul><p>(I have added a new ticket to deal with the conversion problem on the BSC/MGW side: <a class="issue tracker-1 status-1 priority-2 priority-default" title="Bug: negotiate GSM-HR RTP packet format explicitly (New)" href="https://projects.osmocom.org/issues/6059">#6059</a>)</p> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=283422023-10-27T00:06:31Zneelsnhofmeyr@sysmocom.de
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-2 priority-2 priority-default" href="/issues/6171">Feature #6171</a>: mgcp-client API: there should be a separate mgcp_codec_param for each codec in mgcp_msg.codecs[]</i> added</li></ul> OsmoBTS - Bug #5688: osmo-bts-sysmo and osmo-bts-trx use different GSM-HR RTP packet formathttps://projects.osmocom.org/issues/5688?journal_id=283442023-10-27T00:06:54Zneelsnhofmeyr@sysmocom.de
<ul></ul><p>possible confusion: this ticket is resolved but related patches are still in heavy review?<br /><a class="external" href="https://gerrit.osmocom.org/c/osmo-mgw/+/31214">https://gerrit.osmocom.org/c/osmo-mgw/+/31214</a></p>