Open Source Mobile Communications: Issueshttps://projects.osmocom.org/https://projects.osmocom.org/favicon.ico?16647414092024-03-06T12:08:24ZOpen Source Mobile Communications
Redmine Osmocom Conferences (OsmoDevCon, OsmoCon, OsmoDevCall) - Feature #6392 (New): video recording / c...https://projects.osmocom.org/issues/63922024-03-06T12:08:24Zlaforge
<p>We need to check how we can get c3voc video recording equipment on-site and who will operate it.</p> OsmoMGW - Feature #6387 (In Progress): osmo_io / io_uring support for RTP/RTCPhttps://projects.osmocom.org/issues/63872024-03-02T16:22:02Zlaforge
<p>The RTP/RTCP sockets of osmo-mgw should be prime candidates for migration to osmo_io and hence benefit from the optional io_uring backend.</p>
<p>Given the many small recvfrom/sendto syscalls on those sockets, performance should be enhanced in a significant way.</p> pySim - Bug #6317 (Stalled): data driven tests for TLV decodershttps://projects.osmocom.org/issues/63172023-12-23T15:12:32Zlaforge
<p>While we do have the <code>_test_de_encode</code> data driven tests for file definitions, we don't yet have something similar for derived classes of BER_TLV_IE. This means that TLVs used outside of the filesystem context (for example, decoding the SELECT/STATUS response, but also eUICC and other stuff) do not yet have test coverage.</p> Osmocom Conferences (OsmoDevCon, OsmoCon, OsmoDevCall) - Bug #6291 (New): 2024 event bringing tog...https://projects.osmocom.org/issues/62912023-12-06T14:31:01Zlaforge
<p>I've been considering this for a number of years, but never actually got aroun to acting on it:</p>
<p>Have an event where the various FOSS mobile communications projects meet, get to know each other and exchange status, plans and ideas.</p>
<p>Matthias Kirschner of the FSFE suggested to <em>attach</em> that event to sfscon (<a class="external" href="https://www.sfscon.it/">https://www.sfscon.it/</a>) and do it a few days ahead (or after) the 2024 incarnation, which is likely happning again in November 2024. The FSFE could help with funding of the related expenses from a donation made by sysmocom to FSFE earmarked to help FOSS in mobile communications.</p>
<p>Let's use this issue to collect a list of projects we'd want to invite, and to generally keep track on status.</p>
<p>For now, I am thinking of the following potential projects:</p>
<a name="cellular-infrastructureprotocol-stacks"></a>
<h2 >cellular infrastructure/protocol stacks<a href="#cellular-infrastructureprotocol-stacks" class="wiki-anchor">¶</a></h2>
<ul>
<li><a href="https://osmocom.org/" class="external">Osmocom</a></li>
<li><a href="https://open5gs.org/" class="external">open5gs</a></li>
<li><a href="https://www.srslte.com/" class="external">srsRAN/srsUE</a></li>
<li><a href="https://free5gc.org/" class="external">free5gc</a></li>
<li><a href="https://github.com/edgecomllc/eupf" class="external">eupf</a></li>
<li><a href="https://github.com/travelping/ergw" class="external">ergw</a></li>
<li><a href="https://magmacore.org/" class="external">Magma</a> + <a href="https://github.com/magma/S1APTester" class="external">S1APTester</a></li>
<li><a href="https://github.com/omec-project" class="external">OMEC</a></li>
<li><a href="https://github.com/wmnsk" class="external">wmnsk</a> and his various go-{gtp,pfcp,m3ua,sccp,tcap} projects</li>
</ul>
<a name="SIM-card-related-projects"></a>
<h2 >SIM card related projects<a href="#SIM-card-related-projects" class="wiki-anchor">¶</a></h2>
<ul>
<li><a href="https://github.com/tomasz-lisowski/swsim" class="external">swsim</a> + <a href="https://github.com/tomasz-lisowski/swicc" class="external">swicc</a></li>
</ul>
<a name="Development-Debugging"></a>
<h2 >Development + Debugging<a href="#Development-Debugging" class="wiki-anchor">¶</a></h2>
<ul>
<li><a href="https://github.com/fgsect/scat" class="external">SCAT</a></li>
<li><a href="https://github.com/P1sec/QCSuper" class="external">QCsuper</a></li>
<li><a href="https://github.com/PentHertz/Modmobmap" class="external">Modmobmap</a></li>
<li><a href="https://github.com/SysSec-KAIST/LTESniffer" class="external">LTESniffer</a></li>
<li><a href="https://github.com/falkenber9/falcon" class="external">FALCON</a></li>
<li><a href="https://github.com/aligungr/UERANSIM" class="external">UERANSIM</a></li>
<li><a href="https://github.com/HewlettPackard/PacketRusher" class="external">PacketRusher</a></li>
<li>the various projects by <a href="https://github.com/fasferraz" class="external">fasferraz</a> (Swu-IKEv2, ...)</li>
<li><a href="https://github.com/P1sec/pycrate" class="external">pycrate</a></li>
</ul>
<a name="FOSS-software-stacks-on-mobile-devices"></a>
<h2 >FOSS software stacks on mobile devices<a href="#FOSS-software-stacks-on-mobile-devices" class="wiki-anchor">¶</a></h2>
<ul>
<li><a href="https://postmarketos.org/" class="external">postmarketos</a></li>
<li><a href="https://replicant.us/" class="external">replicant</a></li>
<li><a href="https://ubuntu-touch.io/" class="external">Ubuntu touch</a></li>
</ul>
<p>I'm very happy to accept further suggestions for projects/people to add to the invite list.</p> Osmocom Conferences (OsmoDevCon, OsmoCon, OsmoDevCall) - Feature #6290 (In Progress): Organize Os...https://projects.osmocom.org/issues/62902023-12-06T14:17:47Zlaforge
<p>We already wanted to re-start 2023 but somehow I didn't manage to ever find the time and energy to do it. But let's make sure we definitely have an OsmoDevCon in 2024 again.</p>
<p>The date and venue are still TBD, as is pretty much anything else.</p> Retronetworking - Feature #6219 (New): Publish pics/videos for EWSD rescue/movehttps://projects.osmocom.org/issues/62192023-10-17T14:10:34Zlaforge
<p>Before publishing more pictures and/or videos, have to clarify</p>
<ul>
<li>individual permission (rights to one's own picture) or in the worst case pixelize some individuals</li>
<li>corporate permission (other equipment which might be visible around the EWSD)</li>
<li>find a good way to publish an entire gallery of pictures + videos, possibly with captions</li>
<li>cut out the lunch breaks from the timelapse videos</li>
<li>provide down-scaled (HD?) versions of the videos for people who don't want to download or stream gigabytes...</li>
<li>some kind of intro / title slide for the videos (just to give context)</li>
</ul> Retronetworking - Feature #6218 (New): write article/report on EWSD rescue for IGHFT / Fernmeldem...https://projects.osmocom.org/issues/62182023-10-17T14:05:27Zlaforge
<p>I'm a member there and it would be nice to get a report into their member magazine "Tele-Kurier" (not publicly available). Still it might be there are some people with interest not only for electromechanical switching.</p>
<p><a class="external" href="https://fernmeldemuseum-dresden.de/">https://fernmeldemuseum-dresden.de/</a></p> osmo-e1d - Bug #6169 (New): Frame masking against network frame ordering, not frame numbershttps://projects.osmocom.org/issues/61692023-09-06T08:33:34Zmanawyrm
<p>The osmo-e1d code includes functionality to save bandwidth by not transmitting timeslots, which haven't changed since the last frame.</p>
<p><a class="external" href="https://gitea.osmocom.org/Manawyrm/osmo-e1d/src/branch/master/src/octoi/e1oip.c#L263">https://gitea.osmocom.org/Manawyrm/osmo-e1d/src/branch/master/src/octoi/e1oip.c#L263</a></p>
<p>Later in development, the frame_rifo was introduced to combat packet reordering in real-world networks (like DOCSIS).<br />The code unpacking the timeslots into full E1 frames is currently unpacking against the last frame in the incoming buffer, not the last frame in the RIFO (random in, first out) buffer.<br />This means that frames will get filled with random data from other frames in the event of a re-ordering.</p>
<p>Unpacking/Unmasking should be done after the RIFO mechanism.</p> pySim - Bug #6119 (In Progress): aram_get_config runs into an exception on a sysmocomSJA2https://projects.osmocom.org/issues/61192023-07-28T22:48:14Zlynxis
<ul>
<li>Use a SysmocomSJA2</li>
<li>pysim master / 791f80a44f8110f478a632cd2bcde5944ad4bd96</li>
</ul>
<pre>
pySIM-shell (MF)> select ADF.ARA-M
pySIM-shell (MF/ADF.ARA-M)> aram_get_config
EXCEPTION of type 'ValueError' occurred with message: auto_collection_DeviceConfigDO(<class 'pySim.ara_m.DeviceInterfaceVersionDO'>): Unknown TLV Class DeviceInterfaceVersionDO in [{'DeviceInterfaceVersionDO': {'major': 0, 'minor': 0, 'patch': 1}}]; expected dict_keys(['device_interface_version_do'])
To enable full traceback, run the following command: 'set debug true'
pySIM-shell (MF/ADF.ARA-M)>
pySIM-shell (MF/ADF.ARA-M)> set debug true
debug - was: False
now: True
pySIM-shell (MF/ADF.ARA-M)> aram_get_config
Traceback (most recent call last):
File "/home/lynxis/.local/share/virtualenvs/pysim-C216Z-pe/lib/python3.11/site-packages/cmd2/cmd2.py", line 2399, in onecmd_plus_hooks
stop = self.onecmd(statement, add_to_history=add_to_history)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lynxis/.local/share/virtualenvs/pysim-C216Z-pe/lib/python3.11/site-packages/cmd2/cmd2.py", line 2852, in onecmd
stop = func(statement)
^^^^^^^^^^^^^^^
File "/home/lynxis/projects/osmocom/repos/pysim/pySim/ara_m.py", line 315, in do_aram_get_config
res_do = ADF_ARAM.get_config(self._cmd.card._scc._tp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lynxis/projects/osmocom/repos/pysim/pySim/ara_m.py", line 298, in get_config
cmd_do.from_dict([{'DeviceInterfaceVersionDO': {
File "/home/lynxis/projects/osmocom/repos/pysim/pySim/tlv.py", line 161, in from_dict
self.children = self.nested_collection.from_dict(decoded)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lynxis/projects/osmocom/repos/pysim/pySim/tlv.py", line 406, in from_dict
raise ValueError('%s: Unknown TLV Class %s in %s; expected %s' %
ValueError: auto_collection_DeviceConfigDO(<class 'pySim.ara_m.DeviceInterfaceVersionDO'>): Unknown TLV Class DeviceInterfaceVersionDO in [{'DeviceInterfaceVersionDO': {'major': 0, 'minor': 0, 'patch': 1}}]; expected dict_keys(['device_interface_version_do'])
EXCEPTION of type 'ValueError' occurred with message: auto_collection_DeviceConfigDO(<class 'pySim.ara_m.DeviceInterfaceVersionDO'>): Unknown TLV Class DeviceInterfaceVersionDO in [{'DeviceInterfaceVersionDO': {'major': 0, 'minor': 0, 'patch': 1}}]; expected dict_keys(['device_interface_version_do'])
pySIM-shell (MF/ADF.ARA-M)>
pySIM-shell (MF/ADF.ARA-M)> aram_get_all
[
{
"response_all_ref_ar_do": [
{
"ref_ar_do": [
{
"ref_do": [
{
"aid_ref_do": "ffffffffffff"
},
{
"dev_app_id_ref_do": "550f1a164ccd48d27a5ea3b765957493cd830171"
}
]
},
{
"ar_do": [
{
"perm_ar_do": {
"permissions": "0000000000000001"
}
}
]
}
]
}
]
}
]
</pre> pySim - Feature #6089 (Feedback): log the current file / operation during execeution of a scripthttps://projects.osmocom.org/issues/60892023-07-10T07:42:17Zlaforge
<p>When we execute a script (e.g. one generated by <code>export</code>), there is no output that helps us understand where in the script we currently are. This is particularly problematic in case something goes wrong (error/exception), where the backtrace might not give us an indication of the EF that was currently selected.</p>
<p>One can enable <code>set apdu_trace true</code> and then check back from the FID of the last SELECT (A4) to the filename manually, but that's cumbersome and just a hack.</p>
<p>I think when executing a script, we should give a brief log of what we're doing, such as simply echoing the script line to stdout.</p>
<p>Generating the <code>export</code> with explicit <code>echo</code> statements not a solution, as that wouldn't work for other scripts (not generated by <code>export</code>)</p> Core testing infrastructure - Feature #6023 (Stalled): test rack/lab power management software wi...https://projects.osmocom.org/issues/60232023-05-04T06:38:46Zlaforge
<a name="Problem-Statement"></a>
<h2 >Problem Statement<a href="#Problem-Statement" class="wiki-anchor">¶</a></h2>
<p>In our test/lab setups, we do have a number of systems that run 24/7 but which are really used only very few hours per day. This has become very visible after we started (a few months ago) to deploy tasmota/influxdb/grafana for plotting many different power rails.</p>
<p>Direct on/off switching from within a given test job only works if that test job is the only user of the given resource (such as e.g. a BTS in osmo-gsm-tester).</p>
<p>For jenkins builders, OBS workers and similar machines, there could be any number of concurrent users. So there's no single job that can power on the resourec before using it, and power it off after it terminates.</p>
<p>What we need is a system that maintains a usage count, similar to how we do usage/reference counting in data structures in software development.</p>
<p>I've started to prototype a modular python daemon which offers a REST API over which users can obtain <em>usage tokens</em> for named resources. The daemon then keeps track of the current use count and switches resources on/off as needed.</p>
<p>Jenkins jobs would then (e.g. in a pipeline) first obtain a usage token (which would implicitly power up the resource if it is not aleady powwered), and release the usage token after they're gone. This way we can power up build machines only when needed, saving significant electrical power, reducing noise and minimizing heat dissipation.</p>
<a name="Architecture-Class-model"></a>
<h2 >Architecture / Class model<a href="#Architecture-Class-model" class="wiki-anchor">¶</a></h2>
<a name="Resource"></a>
<h3 >Resource<a href="#Resource" class="wiki-anchor">¶</a></h3>
<p>A <em>Resource</em> is typically some kind of physical equipment (server, build host, network equipment, ...) which one or multiple users may be using concurrently.</p>
<p>The Resource has a state, such as</p>
<table>
<tr>
<th>State</th>
<th>Description</th>
</tr>
<tr>
<td>off</td>
<td>Powered down</td>
</tr>
<tr>
<td>powered</td>
<td>Powered up, but not reachable yet</td>
</tr>
<tr>
<td>available</td>
<td>Powered up and reachable (typically via network)</td>
</tr>
</table>
<p>A Resource refers to a <em>Switcher</em> and an <em>AvailabilityChecker</em></p>
<p>A Resource keeps a list of <em>UsageToken</em>; one for each concurrent user.</p>
<a name="Switcher"></a>
<h3 >Switcher<a href="#Switcher" class="wiki-anchor">¶</a></h3>
A <em>Switcher</em> is something that can switch power. Possible implementations include
<ul>
<li>sispm compatible USB-switchable power sockets</li>
<li>Intellinet rackmount PDUs with IP/HTTP interface</li>
<li>Tasmota switchable power sockets</li>
<li>ethernet wake-on-lan (on) + ssh-based shutdown (off)</li>
</ul>
A <em>Switcher</em> implementation (inheriting from the abstract Switcher base class) provides methods for
<ul>
<li>changing the power state (on/off)</li>
<li>determining the current actual power state (if supported by hardware). This is important to get the state right at start-up time.</li>
</ul>
<a name="SwitcherGroup"></a>
<h3 >SwitcherGroup<a href="#SwitcherGroup" class="wiki-anchor">¶</a></h3>
<p>A <em>SwitcherGroup</em> is a logical group of multiple <em>Switcher</em>, for example the set of four switchable sispm sockets in one power strip, or the set of 8 switchable power ports in an Intellinet PDU.</p>
<a name="AvailabilityChecker"></a>
<h3 >AvailabilityChecker<a href="#AvailabilityChecker" class="wiki-anchor">¶</a></h3>
<p>An <em>AvailabilityChecker</em> is something that can check the logical availability of a (powered-up) resource. One common example for an IP-attached resources is an ICMP echo request/response based check.</p>
<a name="UsageToken"></a>
<h3 >UsageToken<a href="#UsageToken" class="wiki-anchor">¶</a></h3>
<p>An <em>UsageToken</em> for a <em>Resource</em> must be obtained by any user intending to use the Resource. The UsageToken has a validity time (in seconds), after which it automatically expires.</p>
<p>Release of the UsageToken can hence be either explicit (via REST API after the user is done) or implicit (timeout of the validity period).</p> Retrocomputing - Feature #5938 (In Progress): assemble "fake parallel printer"https://projects.osmocom.org/issues/59382023-03-06T16:33:11Zlaforge
<p>This will be useful for generating digital captures of e.g. the traces generated by <a class="wiki-page" href="https://projects.osmocom.org/projects/retronetworking/wiki/Wandel_Goltermann_DTX-1">Wandel_Goltermann_DTX-1</a>.</p>
<p><a class="external" href="https://github.com/tomverbeure/fake_parallel_printer">https://github.com/tomverbeure/fake_parallel_printer</a></p> SIMtrace 2 - Bug #5921 (In Progress): simtrace2 cardem vs. Linux kernel USB autosuspendhttps://projects.osmocom.org/issues/59212023-02-23T17:40:01Zlaforge
<p>(at least) after the following patch was merged, simtrace2-cardem doesn't work with Linux kernels' USB autosuspend anymore:<br /><pre>
commit a5d537973db9359804e82a506057f3dd6d53fab9
Author: Harald Welte <laforge@osmocom.org>
Date: Mon Jul 25 19:59:08 2022 +0200
cardem: reset the uC in case of USB disconnect
</pre></p>
<p>The problem is that the USB kernel notices the simtrace2 device is not in use (no application using it), which in turn means it will power-down the device after 5s. The device then recognizes this as USB disconnect, and we use that to reset the firmware:</p>
<pre>
=============================================================================
SIMtrace2 firmware 0.8.1.58-773d, BOARD=simtrace, APP=cardem
(C) 2010-2019 by Harald Welte, 2018-2019 by Kevin Redon
=============================================================================
-I- Chip ID: 0x299b0a60 (Ext 0x00000000)
-I- Serial Nr. 44203020-48574336-30303931-32323032
-I- Reset Cause: software reset (processor reset required by the software)
-I- USB init...
USBD_Init
SetAddr(22) -W- Sta 0x888A8 [0] -W- _ -W- Sta 0x888A8 [0] -W- _ -W- Sta 0x888A8 [0] -W- _ SetCfg(1) cfgChanged1 -I- calling configure of all configurations...
-I- calling init of config 1...
-I- Modem 0: physical SIM
-I- 0: Use local/physical SIM
-I- entering main loop...
-I- USB is now configured
-I- Resetting uC on USB disconnect
=============================================================================
SIMtrace2 firmware 0.8.1.58-773d, BOARD=simtrace, APP=cardem
(C) 2010-2019 by Harald Welte, 2018-2019 by Kevin Redon
=============================================================================
-I- Chip ID: 0x299b0a60 (Ext 0x00000000)
-I- Serial Nr. 44203020-48574336-30303931-32323032
-I- Reset Cause: software reset (processor reset required by the software)
-I- USB init...
USBD_Init
SetAddr(23) -W- Sta 0x888A8 [0] -W- _ -W- Sta 0x888A8 [0] -W- _ -W- Sta 0x888A8 [0] -W- _ SetCfg(1) cfgChanged1 -I- calling configure of all configurations...
-I- calling init of config 1...
-I- Modem 0: physical SIM
-I- 0: Use local/physical SIM
-I- entering main loop...
-I- USB is now configured
-I- Resetting uC on USB disconnect
</pre>
<p>This in turn will make the device enumerate and re-enumerate in 5s cycles:</p>
<pre>
[585591.174222] usb 1-1: new full-speed USB device number 84 using xhci_hcd
[585591.330180] usb 1-1: New USB device found, idVendor=1d50, idProduct=60e3, bcdDevice= 0.02
[585591.330216] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=11
[585591.330231] usb 1-1: Product: SIMtrace 2
[585591.330242] usb 1-1: Manufacturer: sysmocom - s.f.m.c. GmbH
[585591.330253] usb 1-1: SerialNumber: 44203020485743363030393132323032
[585594.759881] usb 1-1: USB disconnect, device number 84
[585595.530214] usb 1-1: new full-speed USB device number 85 using xhci_hcd
[585595.682690] usb 1-1: New USB device found, idVendor=1d50, idProduct=60e3, bcdDevice= 0.02
[585595.682697] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=11
[585595.682701] usb 1-1: Product: SIMtrace 2
[585595.682704] usb 1-1: Manufacturer: sysmocom - s.f.m.c. GmbH
[585595.682706] usb 1-1: SerialNumber: 44203020485743363030393132323032
[585598.802549] usb 1-1: USB disconnect, device number 85
[585602.158170] usb 1-1: new full-speed USB device number 86 using xhci_hcd
[585602.313720] usb 1-1: New USB device found, idVendor=1d50, idProduct=60e3, bcdDevice= 0.02
[585602.313757] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=11
[585602.313772] usb 1-1: Product: SIMtrace 2
[585602.313784] usb 1-1: Manufacturer: sysmocom - s.f.m.c. GmbH
[585602.313795] usb 1-1: SerialNumber: 44203020485743363030393132323032
[585606.602416] usb 1-1: USB disconnect, device number 86
</pre> osmo-e1d - Bug #5824 (Feedback): Line dead message in the log after starthttps://projects.osmocom.org/issues/58242022-12-12T20:57:41Zperformer
<p>In half cases I have next message in the log:<br /><0000> intf_line.c:95 (I0:L0) Received Only 0 bytes/s (expected: 262144): Line dead?</p>
<p>Also no traffic on client socket. Restart of e1d helps.</p> Retronetworking - Bug #5681 (New): Figure out suitable connectors for EKSOShttps://projects.osmocom.org/issues/56812022-09-15T15:47:06Zlaforge
<p>While we did receive a few cables for EKSOS, many of them are missing, so I am looking for suitable alternative cables.</p>
<ul>
<li>EKSOS shelf DC power connector is confirmed to be a TE 350766-1 (contacts for 2.5mm2 wire: 640310-3)</li>
<li>EKSOS NCU E1 connector is <em>almost</em> a Harting 09232326824. Almost in that the clip-on frame has some notches, so the clip-on guide would have to be removed.</li>
<li>EKSOS POTS/ISDN line card subscriber line conncetor is <strong>not</strong> a typical 32pin 3-row DIN "C" connector. While both the EKSOS and this connector have pins every second position in the outer two rows, the order is exactly reversed: The standard DIN Connector has pins where EKSOS has none, and vice versa. A 64pos DIN C connector should work, assuming one then simply doesn't use every second pin present.</li>
</ul>
<p>The original manfacturer of those connectors (Perlos corporation of Joensuu, Finland) <a href="https://www.thefreelibrary.com/Finland%27s+Perlos+Corporation+sells+Special+Products+and+Connectors...-a0163748830" class="external">sold the connector business to Valuukumpu/M2 LTD in 2007</a></p>
<p><a href="https://www.cambridgeelectronics.com/DIN-41612-connectors" class="external">Cambridge Electronics</a> seems to be selling compatible connectors for at least some configurations, using seemingly the same retention lock / frame construction as featured on the EKSOS units.</p> OsmoMSC - Bug #5564 (Stalled): blocking database I/O by SMS databasehttps://projects.osmocom.org/issues/55642022-05-15T14:18:42Zlaforge
<p>when OsmoMSC was split from OsmoNITB, we externalized the HLR database and removed the database-stored counters. This leaves the internal SMS queue / database code as the only remaining part of code which performs potentailly blocking disk I/O.</p>
<p>As seen in <a class="issue tracker-1 status-7 priority-3 priority-high3" title="Bug: OsmoMSC sometimes stalls for dozens of seconds in a production deployment (Stalled)" href="https://projects.osmocom.org/issues/5563">#5563</a> this is a real issue.</p>
I spent half a day on reviewing the code in detail and playing with different ideas, including
<ol>
<li>ripping out the sms_queue.c / db.c code completely into an external osmo-smsc which then uses GSUP</li>
<li>just moving db.c into a separate thread; make DB operations asynchronous</li>
<li>move sms_queue + db.c into a separate thread</li>
</ol>
<a name="moving-sms_queue-DB-code-to-new-osmo-smsc-intrfaced-via-GSUP"></a>
<h2 >moving sms_queue + DB code to new osmo-smsc, intrfaced via GSUP<a href="#moving-sms_queue-DB-code-to-new-osmo-smsc-intrfaced-via-GSUP" class="wiki-anchor">¶</a></h2>
<p>osmo-msc already contains code to do SMS via GSUP, so there's no mandatory modification to osm-msc expected in this approach.</p>
the major disadvantages of this appraoch are:
<ul>
<li>SMPP code would have to move to SMSC, and it is more tied into the MSC/VLR codebase -> extra effort</li>
<li>GSUP SMS interface is at a lower level than current sms_queue intrface -> extra effort of migrating/reimplementing that stuff in SMSC</li>
</ul>
<a name="SMS-related-VTY-commands-not-an-issue-SMSC-would-have-its-own-VTY"></a>
<h3 >SMS related VTY commands (not an issue, SMSC would have its own VTY)<a href="#SMS-related-VTY-commands-not-an-issue-SMSC-would-have-its-own-VTY" class="wiki-anchor">¶</a></h3>
<p>this would cover the following API parts</p>
<ul>
<li>sms_queue_clear</li>
<li>sms_queue_set_max_failure</li>
<li>sms_queue_set_max_pending</li>
<li>sms_queue_stats</li>
<li>sms_queue_sms_is_pending</li>
<li>sms_queue_trigger</li>
<li>vty_out</li>
</ul>
<a name="incoming-signals-into-sms_queue"></a>
<h3 >incoming signals into sms_queue<a href="#incoming-signals-into-sms_queue" class="wiki-anchor">¶</a></h3>
<ul>
<li>SS_SUBSCR / S_SUBSCR_ATTACHED
<ul>
<li>FIXME: unclear how this is handled in the GSUP case?</li>
</ul>
</li>
<li>SS_SMS / S_SMS_DELIVERED
<ul>
<li>-> gsm411_gsup_mt_fwd_sm_res()</li>
</ul>
</li>
<li>SS_SMS / S_SMS_MEM_EXCEEDED
<ul>
<li>-> gsm411_gsup_mt_fwd_sm_err()</li>
</ul>
</li>
<li>SS_SMS / S_SMS_UNKNOWN_ERROR
<ul>
<li>-> gsm411_gsup_mt_fwd_sm_err()</li>
</ul>
</li>
<li>SS_SMS / S_SMS_SUBMITTED
<ul>
<li>-> gsm411_gsup_mo_fwd_sm_req()</li>
</ul>
</li>
<li>SS_SMS / S_SMS_SMMA
<ul>
<li>-> gsm411_gsup_mo_ready_for_sm_req()</li>
</ul></li>
</ul>
<a name="DB-not-an-issue-DB-code-would-then-run-in-SMSC"></a>
<h3 >DB (not an issue, DB code would then run in SMSC)<a href="#DB-not-an-issue-DB-code-would-then-run-in-SMSC" class="wiki-anchor">¶</a></h3>
<ul>
<li>db_sms_delete_oldest_expired_message</li>
<li>db_sms_delete_sent_message_by_id</li>
<li>db_sms_get</li>
<li>db_sms_get_next_unsent_rr_msisdn</li>
<li>db_sms_get_unsent_for_subscr</li>
<li>db_sms_inc_deliver_attempts</li>
</ul>
<a name="SMS-transmission"></a>
<h3 >SMS transmission<a href="#SMS-transmission" class="wiki-anchor">¶</a></h3>
<ul>
<li>gsm411_send_sms calls by sms_queue
<ul>
<li>would have to be mapped to OSMO_GSUP_MSGT_MT_FORWARD_SM_REQUEST</li>
</ul>
</li>
<li>sms_free
<ul>
<li>FIXME: what about vsub pointer/references?</li>
</ul>
</li>
<li>vlr_subscr_msisdn_or_name
<ul>
<li>just for logging, can be avoided</li>
</ul></li>
</ul>
<a name="making-just-the-DB-code-async-run-in-separate-thread"></a>
<h2 > making just the DB code async / run in separate thread<a href="#making-just-the-DB-code-async-run-in-separate-thread" class="wiki-anchor">¶</a></h2>
Is not easy as all of the call sites are assuming synchronous return/results<br />db_sms_get
<ul>
<li>sms_resend_pending
<ul>
<li>resend_pending timer
<ul>
<li>sms_queue_start
<ul>
<li>=> can be executed from separate thread</li>
</ul></li>
</ul></li>
</ul></li>
</ul>
db_sms_get_next_unsent_rr_msisdn
<ul>
<li>smsq_take_next_sms
<ul>
<li>sms_submit_pending
<ul>
<li>sms_send_next
<ul>
<li>sms_sms_cb / S_SMS_DELIVERED
<ul>
<li>=> happens from the send_next it_Q completion handler</li>
</ul>
</li>
</ul>
</li>
<li>push_queue_timer
<ul>
<li>sms_queue_start
<ul>
<li>=> can be executed from separate thread</li>
</ul></li>
</ul></li>
</ul></li>
</ul></li>
</ul>
db_sms_get_unsent_for_subscr
<ul>
<li>sms_send_next
<ul>
<li>sms_sms_cb / S_SMS_DELIVERED
<ul>
<li>=> request to it_Q; completion then might add SMS to pending + gsm411_send_sms</li>
</ul>
</li>
</ul>
</li>
<li>sub_ready_for_sm
<ul>
<li>sms_subscr_cb / S_SUBSCR_ATTACHED
<ul>
<li>=> request to it_Q; completion then might add SMS to pending + gsm411_send_sms</li>
</ul></li>
</ul></li>
</ul>
db_sms_delete_sent_message_by_id
<ul>
<li>sms_sms_cb / S_SMS_DELIVERED
<ul>
<li>=> no return value, no success check: async it_Q</li>
</ul></li>
</ul>
db_sms_inc_deliver_attempts
<ul>
<li>sms_sms_cb / S_SMS_UNKNOWN_ERROR
<ul>
<li>=> no return value, no success check: async it_Q</li>
</ul></li>
</ul>
db_sms_delete_oldest_expired_message
<ul>
<li>sms_sms_cb / any signal
<ul>
<li>=> no return value, no success check: async it_Q</li>
</ul></li>
</ul>
<a name="moving-sms_queue-DB-code-to-separate-thread"></a>
<h2 >moving sms_queue + DB code to separate thread<a href="#moving-sms_queue-DB-code-to-separate-thread" class="wiki-anchor">¶</a></h2>
<a name="access-to-pending_sms-linked-list"></a>
<h3 >access to pending_sms linked list<a href="#access-to-pending_sms-linked-list" class="wiki-anchor">¶</a></h3>
<p>There are quite a number of accesses to the pending_sms linked list. Given most ar read, and only some are write, we might use a rwlock?</p>
<ul>
<li>sms_find_pending [R]
<ul>
<li>sms_sms_cb</li>
<li>sms_queue_sms_is_pending</li>
</ul></li>
</ul>
<ul>
<li>sms_queue_sms_is_pending [R]
<ul>
<li>sms_submit_pending
<ul>
<li>timer</li>
</ul>
</li>
<li>vty</li>
</ul></li>
</ul>
<ul>
<li>sms_subscriber_find_pending [R]
<ul>
<li>sub_ready_for_sm
<ul>
<li>SS_SUBSCR / S_SUBSCR_ATTACHED</li>
</ul>
</li>
<li>sms_subscriber_is_pending
<ul>
<li>sms_submit_pending
<ul>
<li>timer</li>
</ul>
</li>
<li>sms_send_next
<ul>
<li>sms_sms_cb / S_SMS_DELIVERED</li>
</ul></li>
</ul></li>
</ul></li>
</ul>
<ul>
<li>sms_pending_from [R]
<ul>
<li>sms_submit_pending
<ul>
<li>timer</li>
</ul>
</li>
<li>sms_send_next
<ul>
<li>sms_sms_cb / S_SMS_DELIVERED</li>
</ul></li>
</ul></li>
</ul>
<ul>
<li>sms_pending_free [W]
<ul>
<li>sms_pending_failed
<ul>
<li>sms_sms_cb / S_SMS_UNKNOWN_ERROR</li>
</ul>
</li>
<li>sms_resend_pending
<ul>
<li>sms_sms_cb / S_SMS_DELIVERED</li>
<li>sms_sms_cb / S_SMS_MEM_EXCEEDED</li>
</ul>
</li>
<li>sms_queue_clear
<ul>
<li>vty</li>
</ul></li>
</ul></li>
</ul>
<ul>
<li>sms_resend_pending [R]
<ul>
<li>timer</li>
</ul></li>
</ul>
<ul>
<li>sms_queue_stats [R]
<ul>
<li>vty</li>
</ul></li>
</ul>
<ul>
<li>sms_queue_clear [W]
<ul>
<li>vty</li>
</ul></li>
</ul>
<a name="Conclusion"></a>
<h2 >Conclusion<a href="#Conclusion" class="wiki-anchor">¶</a></h2>
I think the following approach is best:
<ul>
<li>have a separate "SMS" thread</li>
<li>all database access happens <strong>from that thread only</strong></li>
<li>inter-thread message queues (libosmocore it_q) between main thread and SMS thread</li>
<li>sms_queue timers (push_queue_timer, resend_pending_timer) run in that thread</li>
<li>other input (mainly signals today) are serialized via it_q in main -> SMS direction</li>
<li>other output (mainly gsm411_send_sms) are serialized via it_q in SMS -> main direction</li>
</ul>
<a name="Serialize-SS_SMS-signals"></a>
<h3 >Serialize SS_SMS signals<a href="#Serialize-SS_SMS-signals" class="wiki-anchor">¶</a></h3>
<ul>
<li>we really only need to serialize paging_result and sms->id</li>
<li>submit them into it_q to SMS thread</li>
</ul>
<a name="serialize-SS_SUBSCR-signal"></a>
<h3 >serialize SS_SUBSCR signal<a href="#serialize-SS_SUBSCR-signal" class="wiki-anchor">¶</a></h3>
<ul>
<li>sms_subscriber_find_pending() can be done in main thread before serialization</li>
<li>check for vsub->lu_complete and zero MSISDN before serialization</li>
<li>we really only need to serialize the MSISDN</li>
<li>db_sms_get_unsent_for_subscr() then happens from SMS thread</li>
</ul>
<a name="move-push_queue_timer-resend_pending_timer-to-SMS-thread"></a>
<h3 >move push_queue_timer + resend_pending_timer to SMS thread<a href="#move-push_queue_timer-resend_pending_timer-to-SMS-thread" class="wiki-anchor">¶</a></h3>
<a name="serialize-db_sms_store-MO-SMS-SMPP"></a>
<h3 >serialize db_sms_store() (MO-SMS, SMPP)<a href="#serialize-db_sms_store-MO-SMS-SMPP" class="wiki-anchor">¶</a></h3>
<ul>
<li>failure to store in database would only be known asynchronously!</li>
<li>we can probably just ignore that.</li>
</ul>
<a name="serialize-db_sms_mark_delivered"></a>
<h3 >serialize db_sms_mark_delivered()<a href="#serialize-db_sms_mark_delivered" class="wiki-anchor">¶</a></h3>
<ul>
<li>we don't care about success right now anyway, so async is no problem</li>
</ul>
<a name="VTY"></a>
<h3 >VTY<a href="#VTY" class="wiki-anchor">¶</a></h3>
<ul>
<li>remove 'sms send pending' or implement different command via it_Q</li>
<li>remove 'sms delete expired' or implement different command via it_Q</li>
<li>serialize 'subscriber ... sms ...' via it_Q</li>
</ul> OsmoMSC - Bug #5563 (Stalled): OsmoMSC sometimes stalls for dozens of seconds in a production dep...https://projects.osmocom.org/issues/55632022-05-14T07:02:28Zlaforge
<p>When we take a long-term (8 hours) bpftrace showing us the delay between subsequent calls to <code>poll()</code> (by libosmocore/src/select.c) in osmo-msc, we get the following histogram (units in milli-seconds):</p>
<pre>
@poll:
[0] 532245 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[1] 13088 |@ |
[2, 4) 5621 | |
[4, 8) 5566 | |
[8, 16) 2746 | |
[16, 32) 5282 | |
[32, 64) 5262 | |
[64, 128) 6139 | |
[128, 256) 14273 |@ |
[256, 512) 18357 |@ |
[512, 1K) 13806 |@ |
[1K, 2K) 4222 | |
[2K, 4K) 1331 | |
[4K, 8K) 450 | |
[8K, 16K) 0 | |
[16K, 32K) 0 | |
[32K, 64K) 5 | |
[64K, 128K) 17 | |
[128K, 256K) 2 | |
</pre>
So as we can see
<ul>
<li>the majority is very low (sub-second to 128ms)</li>
<li>there is a smaller peak in the order of 128ms to 1s (surprisingly long)</li>
<li>there are still several thousand of instances where the delay isn the 1s..4s. interval (too long!)</li>
<li>there ar rare occasions where we don't return to poll for 32, 64, or evne more than 128 seconds (crazy!)</li>
</ul>
<p>If we contrast this with the amount of time we spent in <code>dbi_conn_queryf</code>, this is clearly not the culprit:</p>
<pre>
@dbi_query:
[0] 37008 |@ |
[1] 1640233 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[2, 4) 1245771 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[4, 8) 21406 | |
[8, 16) 325 | |
[16, 32) 71 | |
[32, 64) 17 | |
</pre>
<p>So the longest duration DB query was in the order of 32..63 ms. Not good, but not a problem either with all the MSC (MM, CC, SMS, BSSAP, SCCP, ...) time-outs being in the multiple-second range.</p>
<p>So now we have to find out if the stalls are</p>
<ol>
<li>due to excessive system load (like I/O) outside of osmo-msc, or</li>
<li>due to something osmo-msc is doing by itself (like calling thousands of database queries of several milli-seconds each) without going through the libosmocore poll main loop.</li>
</ol> OsmoMSC - Bug #5559 (Stalled): OsmoMSC at 100% CPU and unresponsive for up to several minutes!https://projects.osmocom.org/issues/55592022-05-12T23:22:09Zkeith
<p>Not much more to say than the title I'm afraid.</p>
<p>So far, I've actually only noticed it on a system using the RBS and osmo-e1d. But I do not have conclusive proof that it is exclusively happening here.</p>
<p>I'm assuming a culprit might be the sms queue, but I'm not convinced because I'm not seeing it on other systems with more messages in the queue in the sqlite db - and this can be upwards of 1,000 SMS queued.</p> osmo-remsim - Bug #5527 (Stalled): warn on duplicate client (id) connectionshttps://projects.osmocom.org/issues/55272022-04-12T17:37:27Zlaforge
<p>Every client must have its own unique tuple of (client_id/slot_nr).</p>
<p>If a remsim-server receives a duplicate connection, it should pring a clear warning message to the log.</p>
<p>This might not always be a bug, as in csae of network outages / restarts a new connection might arrive before the old one is closed.</p>
<p>The same should apply to remsim-bankd.</p> Open Source IMS Client - Feature #5481 (New): SIM card interface for doubangohttps://projects.osmocom.org/issues/54812022-03-07T10:53:16Zlaforge
<p>The pre-existing <a class="wiki-page" href="https://projects.osmocom.org/projects/foss-ims-client/wiki/Doubango">doubango</a> library code assumes that the IMS client has knowledge of the secret key material (K + OP/OPc) in order to perform the authentication and IPsec key establoshment to the P-CSCF.</p>
<p>This may be the case in <em>some</em> testing/lab setups, but in general this key material is stored on the ISIM or USIM application of a SIM card.</p>
<p>If we want to use doubango with such standard cards, we need some kind of interface how doubango can perform authentication via ISIM/USIM.</p>
The interface should be rather generic, as the detailed interface for SIM access will be highly platform specific:
<ul>
<li>For development on a normal Linux laptop, a pcsc-lite based interface to a smart card reader will be used.</li>
<li>For execution inside a specific phone, phone specific interfaces for SIM card access may be used (QMI, AT+CSIM, ...)</li>
</ul> Open Source IMS Client - Feature #5480 (In Progress): Linux kernel IPsec plugin for doubangohttps://projects.osmocom.org/issues/54802022-03-07T10:49:03Zlaforge
<p>This task is about creating an IPsec plugin for the <a class="wiki-page" href="https://projects.osmocom.org/projects/foss-ims-client/wiki/Doubango">doubango</a> library, allowing doubango to control IPsec policies and security associations in the kernel IPsec stack.</p>
<p>Doubango already has a related plugin architecture and a sample plugin for Windows 10</p> SIMtrace 2 - Bug #5419 (Stalled): cardem errors with higher baud ratehttps://projects.osmocom.org/issues/54192022-01-25T18:27:00Zlaforge
Setup is as follows:
<ul>
<li>sysmoISIM-SJA2 in built-in CCID reader of my Thinkpad x260</li>
<li>SIMtrace2 with cardem firmware 'master' (0.8.1.7-ea9a) hooked up via FPC to</li>
<li>CCID reader "Identive CLOUD 2700 F" </li>
<li><code>simtrace2-cardem-pcsc</code> to forward request from IdentiveCCID -> SIMtrace -> st2-cardem-pcsc -> builtin-CCID</li>
</ul>
<p>This works fine with F/D ratio 372, and also works fine in most cases with F/D ratio 16.</p>
<p>However, sometimes with ratio 16, things break down at some point.</p>
<a name="log-output-of-cardem-firmware"></a>
<h2 >log output of cardem firmware<a href="#log-output-of-cardem-firmware" class="wiki-anchor">¶</a></h2>
<pre>
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 b2 9d 04 22
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 a4 00 04 02
-I- 0: flush_rx_buffer (5)
-I- 0: flush_rx_buffer (2)
-I- 0: send_tpdu_header: 00 c0 00 00 23
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 b2 9e 04 22
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 a4 00 04 02
-I- 0: flush_rx_buffer (5)
-I- 0: flush_rx_buffer (2)
-I- 0: send_tpdu_header: 00 c0 00 00 23
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 b2 9f 04 22
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 a4 00 04 02
-I- 0: flush_rx_buffer (5)
-I- 0: flush_rx_buffer (2)
-I- 0: send_tpdu_header: 00 c0 00 00 23
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 b2 a0 04 22
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 a4 00 04 02
-I- 0: flush_rx_buffer (5)
-I- 0: flush_rx_buffer (2)
-I- 0: send_tpdu_header: 00 c0 00 00 23
-I- 0: flush_rx_buffer (5)
N-I- 0: send_tpdu_header: 00 b2 a1 04 22
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 00 a4 00 04 02
-I- 0: flush_rx_buffer (5)
-I- 0: flush_rx_buffer (2)
N-I- 0: send_tpdu_header: 00 c0 00 00 60
-I- 0: flush_rx_buffer (5)
-I- 0: send_tpdu_header: 02 00 a4 00 04
-I- 0: flush_rx_buffer (5)
</pre>
two things noticable:
<ul>
<li>the 'N' being printed by card_emu as waiting time extension</li>
<li>the last TPDU header <code>02 00 a4 00 04</code> doesn't look like a TPDU header: The 02 seems wrong, the TPDU likely starts with <code>00 a4</code>.</li>
</ul>
<a name="situation-on-Identive-CCID-reader-side"></a>
<h2 >situation on "Identive CCID reader" side<a href="#situation-on-Identive-CCID-reader-side" class="wiki-anchor">¶</a></h2>
<p>pySim-shell "export" shows:<br /><pre>
update_record 159 ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
update_record 160 ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
update_record 161 ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
# bad file: MF/DF.TELECOM/EF.ADN, Failed to transmit with protocol T0. Transaction failed.
EXCEPTION of type 'RuntimeError' occurred with message: '6881: Functions in CLA not supported - Logical channel not supported'
To enable full traceback, run the following command: 'set debug true'
</pre></p>
<a name="simtrace2-cardem-pcsc"></a>
<h2 >simtrace2-cardem-pcsc<a href="#simtrace2-cardem-pcsc" class="wiki-anchor">¶</a></h2>
<pre>
=> DATA: flags=2, 6f 3a : SW=0x6123, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 c0 00 00 23
=> DATA: flags=1, 00 c0 00 00 23 : SW=0x9000, len_rx=35
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 b2 9d 04 22
=> DATA: flags=1, 00 b2 9d 04 22 : SW=0x9000, len_rx=34
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 a4 00 04 02
=> DATA: flags=1, 00 a4 00 04 02 : -> 01 06 00 00 00 00 10 00 02 00 00 00 02 00 6f 3a
=> DATA: flags=2, 6f 3a : SW=0x6123, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 c0 00 00 23
=> DATA: flags=1, 00 c0 00 00 23 : SW=0x9000, len_rx=35
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 b2 9e 04 22
=> DATA: flags=1, 00 b2 9e 04 22 : SW=0x9000, len_rx=34
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 a4 00 04 02
=> DATA: flags=1, 00 a4 00 04 02 : -> 01 06 00 00 00 00 10 00 02 00 00 00 02 00 6f 3a
=> DATA: flags=2, 6f 3a : SW=0x6123, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 c0 00 00 23
=> DATA: flags=1, 00 c0 00 00 23 : SW=0x9000, len_rx=35
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 b2 9f 04 22
=> DATA: flags=1, 00 b2 9f 04 22 : SW=0x9000, len_rx=34
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 a4 00 04 02
=> DATA: flags=1, 00 a4 00 04 02 : -> 01 06 00 00 00 00 10 00 02 00 00 00 02 00 6f 3a
=> DATA: flags=2, 6f 3a : SW=0x6123, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 c0 00 00 23
=> DATA: flags=1, 00 c0 00 00 23 : SW=0x9000, len_rx=35
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 b2 a0 04 22
=> DATA: flags=1, 00 b2 a0 04 22 : SW=0x9000, len_rx=34
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 a4 00 04 02
=> DATA: flags=1, 00 a4 00 04 02 : -> 01 06 00 00 00 00 10 00 02 00 00 00 02 00 6f 3a
=> DATA: flags=2, 6f 3a : SW=0x6123, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 c0 00 00 23
=> DATA: flags=1, 00 c0 00 00 23 : SW=0x9000, len_rx=35
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 b2 a1 04 22
=> DATA: flags=1, 00 b2 a1 04 22 : SW=0x9000, len_rx=34
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 a4 00 04 02
=> DATA: flags=1, 00 a4 00 04 02 : -> 01 06 00 00 00 00 10 00 02 00 00 00 02 00 6f 3a
=> DATA: flags=2, 6f 3a : SW=0x6123, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 00 c0 00 00 60
=> DATA: flags=1, 00 c0 00 00 60 : SW=0x6c23, len_rx=0
-> 01 06 00 00 00 00 13 00 01 00 00 00 05 00 02 00 a4 00 04
<0000> apdu_dispatch.c:112 Unknown APDU case 0
=> DATA: flags=1, 02 00 a4 00 04 : SW=0x6881, len_rx=0
</pre>
<p>it also agrees that this last APDU is somehow wrong and cannot determine the APDU case.</p>
<a name="USB-communication"></a>
<h2 >USB communication<a href="#USB-communication" class="wiki-anchor">¶</a></h2>
<p>last message from SIMtrace to host is "RX DATA" with header flag set and 0200a40004. The card still responds with SW 6881 to that, as obviously the APDU header is invalid.</p>
<p><img src="https://projects.osmocom.org/attachments/download/4852/wireshark.png" alt="" /></p> osmo-e1d - Bug #4916 (Stalled): USB unplug / replug renders e1d unusablehttps://projects.osmocom.org/issues/49162020-12-18T10:28:45Zlaforge
right now the behavior on USB unplug (or - god forbid - a firmware crash) is not very user friendly:
<ul>
<li>e1d keeps running</li>
<li>e1d does not re-open the device when it comes back</li>
</ul>
IMHO, we have the following options
<ol>
<li>fail fast - simply exit when the device is lost, assume systemd or some other management instance will keep respawning us until the device is back
<ul>
<li>but what about client programs like osmo-bsc / osmo-mgw ?</li>
</ul>
</li>
<li>implement re-opening of a single icE1usb device, knowing our blocking control transfers would corrupt any other ongoing communication
<ul>
<li>is it worth the effort, assuming this is only an interim solution</li>
</ul>
</li>
<li>go for a full-blown hot-plug capable architecture lined out in <a class="issue tracker-2 status-1 priority-1 priority-lowest" title="Feature: consider a one-thread-per-line architecture (New)" href="https://projects.osmocom.org/issues/4915">#4915</a>
<ul>
<li>will probably take significant effort</li>
</ul></li>
</ol>
<p>I think right now we mostly have to worry about situations with a single icE1usb, so I'm tempted to go for the fail-fast approach, assuming osmo-bsc/osmo-mgw recover in some way.</p> OsmocomBB - Bug #4829 (Stalled): OsmocomBB Rx bit errors in dedicated modehttps://projects.osmocom.org/issues/48292020-10-24T03:09:38Zlaforge
<p>I'm observing quite a number of bit errors when receiving in dedicated mode.</p>
<p>Those bit errors are not present while in receive-only mode (CCCH/BCCH camping.</p>
<p>The bit errors can be observed on SDCCH/4, SDCCH/8 and TCH/F (didn't try TCH/H).</p>
<p>I've tried to roll back to old OsmocomBB firmware versions as old as 2012 (using old gnuarm-4.0.x toolchain) - the problem even exists in those old versions, so it doesn't look like a regression.</p>
<p>I've tested with perfect RF signals (coaxial connection via attenuators) to exclude any real-world impact.</p>
<p>I've tested both with sysmoBTS 1002 as well as with a commercial cellular network.</p>
<p>I've tested with C118, C121, C123 and C140.</p>
<p>In all situations, the problem persists and looks like this:</p>
<p>Camping with good signal level 0 BER / fire_crc = 2<br /><pre>
<000c> l1ctl.c:237 BCCH on TS0 (0301/13/02) -56 dBm 0/0: 59 06 1a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff e5 04 00
<000c> l1ctl.c:237 PCH/AGCH on TS0 (0301/17/06) -56 dBm 0/0: 03 03 01 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b
<000c> l1ctl.c:237 PCH/AGCH on TS0 (0301/23/12) -56 dBm 0/0: 15 06 21 00 01 f0 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b
<000c> l1ctl.c:237 PCH/AGCH on TS0 (0301/01/16) -56 dBm 0/0: 2d 06 3f 03 41 e3 67 00 68 8f 00 00 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b
</pre></p>
<p>Once switched to dedicated channel:<br /><pre>
<000c> l1ctl.c:237 SDCCH/8(0) on TS1 (0301/10/00) -56 dBm 0/0: 03 03 01 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b 2b
<000c> l1ctl.c:237 SDCCH/8(0) on TS1 (0301/16/32) -56 dBm 66/2: a3 4b 33 0a 38 66 2a a4 57 cc 2e db 60 f7 e2 7e 9e cd ac d8 ee dc bd
<000c> l1ctl.c:300 Dropping frame with 66 bit errors
<000c> l1ctl.c:237 SDCCH/8(0) on TS1 (0301/09/00) -56 dBm 74/2: a3 35 0f 0a b8 79 13 2a 6b 5e c2 da 60 f7 e2 7e 9e cd ac d8 ee dc fd
<000c> l1ctl.c:300 Dropping frame with 74 bit errors
<000c> l1ctl.c:237 SDCCH/8(0) on TS1 (0301/08/00) -56 dBm 82/2: a3 35 0f 14 f1 f7 17 aa 57 cc 2e 12 67 f7 e2 7e 9e cd ac d8 7e 29 85
<000c> l1ctl.c:300 Dropping frame with 82 bit errors
<000c> l1ctl.c:237 SDCCH/8(0) on TS1 (0301/14/32) -56 dBm 77/2: a3 4b 33 0a 38 66 2a a4 57 cc 2e 92 44 f8 e2 7e 9e cd ac d8 fe 85 2f
<000c> l1ctl.c:300 Dropping frame with 77 bit errors
<000c> l1ctl.c:237 SDCCH/8(0) on TS1 (0301/07/00) -56 dBm 74/2: a3 4b 33 0a 38 66 2a 24 6b 5e c2 da 60 f7 e2 7e 9e cd ac d8 fe 85 2f
<000c> l1ctl.c:300 Dropping frame with 74 bit errors
</pre></p>
<p>so as we can see, he very first block is still received well, all other blocks have massive bit errors (typically in he 70..95 erroneous bit range)</p>
<p>Playing with the source code I could narrow it down to whether or not we are enabling the PA in the <code>rffe_mode()</code> function by means of <code>tspact | PA_ENABLE</code></p>
<p>If I enable the PA only for RACH / access burst, but not for any normal bursts, the SDCCH bit errors are not reported anymore.</p>
<p>I've played a lot with l1s_tx_win_ctrl in terms of ordering, etc. but the problem persists.</p>
<p>It cannot be a general scheduling problem, as the TX window duration is not affected by whether or not we enable the PA. There are just as many TPU instructions etc. even without that one bit.</p> Cellular Network Infrastructure - Bug #4578 (New): Updated slides / presentations / videos on set...https://projects.osmocom.org/issues/45782020-06-03T14:56:52Zlaforge
<p>I just realized that since OsmoCon was discontinued after 2018 didn't result in sufficient turn-out, and only the 2017 incarnation contained some entry-level talks, we actually only have introductory level presentations / videos about a setup that still covers OsmoNITB. (<a class="external" href="https://media.ccc.de/v/osmocon17-4002-running_a_basic_circuit_switched_osmocom_gsm_network">https://media.ccc.de/v/osmocon17-4002-running_a_basic_circuit_switched_osmocom_gsm_network</a>)</p>
<p>This may or may not be a reason why some people still want to set up OsmoNITB in 2020. While our wiki and user manuals all have deprecated OsmoNITB years ago, people who only look for videos might be mislead.</p>
<p>Also, I'm sure that several other presentation (like the one about 3G / osmo-iuh before the CNI split) would similarly benefit from an update.</p>
<p>So I think we should try to create/update some slide decks and record related lectures / tutorials at some point this year. I wouldn't bother setting up a virtual conference or some kind of remote interaction at this point. Recording some talks and/or screencasts allows us to create and release them one-by-one.</p>
<p>Let's use this issue to first collect a list of topics that we think either need an update, or topics that never have been covered so far.</p> OsmoCBC - Feature #3977 (New): SABP stream delineation routineshttps://projects.osmocom.org/issues/39772019-05-06T13:52:08Zlaforge
<p>SABP is not only specified as ASN.1 with APER encoding, but 3GPP in their infinite wisdom specified that it will run directly inside a TCP stream.</p>
<p>As TCP, like any stream, doesn't preserve message boudaries, there's no length field or other framing that would tell us once a given SABP PDU is fully received. Instead, we have to implement something like an "incremental APER length determinant parser" which will parse the outer length of the APER encoded data, and then use the result of that to determine the length of one binary/encoded SABP PDU.</p>
<p>See <code>dissect_per_length_determinant()</code> from wireshark <code>packet-per.c</code> which is actually used by the wireshark SABP dissector, facing the same problem.</p> OsmoBTS - Bug #3568 (Stalled): CBCH on SDCCH/8 not working for osmo-bts-sysmohttps://projects.osmocom.org/issues/35682018-09-18T20:54:03Zlaforge
<p>During manual testing I observed that while sms arrived on my test phone when using CBCH on SDCCH/4, they were not arriving when using CBCH on SDCCH/8.</p>
<p>SI4 CBCH location was updated properly, so that was not the cause...</p> SIMtrace 2 - Bug #3379 (Stalled): documentation on how to use SIMtrace2https://projects.osmocom.org/issues/33792018-07-04T16:10:36Zlaforge
<p>the wiki in the SIMtrace2 redmine project currently only documents flashing, but there should of course be good information on how to use the host tools in order to run the complete system.</p> Cellular Network Infrastructure - Feature #2604 (Stalled): GSUP-to-DIAMETER converter / IWFhttps://projects.osmocom.org/issues/26042017-10-29T23:01:06Zlaforge
<p>In order to support a single subscriber database for 2G/3G and LTE, operators normally use a single HSS. HSS is the EPC successor of the LTE.</p>
<p>Instead of MAP over SS7, HSS's speak DIAMETER over SCTP or TCP (with optional TLS). The types of procedures / transactions are pretty much the same as before, but the encoding and the protocol changed completely.</p>
<p>There are a couple of (at least minimal) HSS implementations out there, from freeDiameter to nextepc, to name some examples.</p>
<p>If we were to implement a converter between GSUP and DIAMETER, then the OsmoMSC and OsmoSGSN could access an external HSS - and that same HSS could be accessed from nextepc or even openair-cn for LTE access.</p>
<p>3GPP TS 29.305 specifies the MAP-to-DIAMETeR InterWorkingFunction (IWF), which is pretty much the same device, with the exception that we'd use GSUP instead of MAP.</p> SIMtrace 2 - Bug #1705 (Stalled): re-integrate tracing + card reader modes into SIMtrace2 firmwar...https://projects.osmocom.org/issues/17052016-05-09T19:46:30Zlaforge
<p>the current laforge/cardemu branch breaks sim card tracing + card reader modes. Re-integrate and test those to have one firmware image working in all modes</p>