Project

General

Profile

Actions

Feature #4945

closed

SBc/LTE support in OsmoCBC

Added by laforge about 3 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Low
Assignee:
Target version:
-
Start date:
01/13/2021
Due date:
% Done:

100%

Spec Reference:
3GPP TS 29.168

Description

This is about adding the SBc interface to OsmoCBC. SBc is used between the MME and the CBC in the LTE network, in order to provide PWS messages from the CBC to the MME.

  • libfftranscode already has SBcAP in master
  • osmo-ttcn3-hacks laforge/sbcap branch contains a CdoecPort and the encoding/decoding

Files


Checklist

  • SBcAP encoding/decoding in TTCN3
  • SBcAP transcoding in libfftranscode
  • SBcAP adapter/prot for TTCN3
  • SBcAP teste cases in TTCN3
  • SBcAP message handling / logic in OsmoCBC
  • manual tests with a MME supporting SBcAP
Actions #1

Updated by laforge almost 2 years ago

See also https://github.com/open5gs/open5gs/issues/824 for a discussion regarding SBcAP in open5gs

Actions #2

Updated by pespin almost 2 years ago

  • Assignee changed from laforge to pespin
Actions #3

Updated by pespin almost 2 years ago

AFAICT laforge/cbps several commits where squashed and merged to master in osmo-ttcn3-hacks.git 08332307eac13ae2049d3e8135b34401ad69334e.

Actions #4

Updated by pespin almost 2 years ago

Last commit containing SBCAP support (c679d6aed0b50b481b56f49fd5a8836b8fe42417) in laforge/sbcap was not merged at the time. I cherry-picked it on top of current master in pespin/sbcap.

Actions #5

Updated by pespin almost 2 years ago

  • Spec Reference set to 3GPP TS 29.169
Actions #6

Updated by pespin almost 2 years ago

I continued adding SBcAP support in TTCN3 in branch osmo-ttcn3-hacks.git "pespin/sbcap".

Similarly, I started working on osmo-cbc SBcAP support in osmo-cbc.git branch "pespin/sbcap".

Actions #7

Updated by pespin almost 2 years ago

laforge what is this "ECBE" stuff I see in osmo-cbc?

Actions #8

Updated by pespin almost 2 years ago

Is that ECBE the naming you use for the REST interface? I wonder whether I can totally reuse the existing one, I have to modify/extend it or create a whole new one.

Actions #9

Updated by laforge almost 2 years ago

External CBSP Entity (like our naming ESME for SMS or EUSE for USSD)

Actions #10

Updated by laforge almost 2 years ago

Same REST interface, no needfor anything new IMHO

Actions #11

Updated by pespin almost 2 years ago

SBc-AP uses ASN.1 APER encoding, which current asn1c we use in other projects such as libosmo-sccp doesn't support.
However, that encoding is probably supported in a recent version of as1nc, so the idea is to try to use that one.
There's no need to put SBc-AP in some separate lib since we don't plan to use it anywhere else, so it can be put directly in osmo-cbc.git.

Actions #12

Updated by pespin almost 2 years ago

  • Status changed from New to In Progress

I started adding an internal libosmo-sbcap in osmo-cbc.git (similar to the ones in osmo-iuh.git).

Parsing the SBc-AP spec ASN files fails with asn1c latest release (v0.9.28), but works with asn1c master (9925dbbda86b436896108439ea3e0a31280a6065).

However, it seems asn1c master in upstream (https://github.com/vlm/asn1c) still doesn't support APER, which we need for SBc-AP. Fortunately there's a fork of asn1c from one of the big contributors which does support APER:
https://github.com/vlm/asn1c/issues/452
https://github.com/mouse07410/asn1c
That fork should be rebased on top of current master and it seems to receive fixes even in 2022.

So far I'm doing the build with usptream master asn1c with --gen-PER, but we should move to that fork and user APER.

Also another issue:
It seems for our old build of asn1c we use for osmo-iuh.git we have a split libasn1c.git containing a runtime library. AFAICT that's done by ourselves and asn1c master doesn't contain that library. Hence, however wants to build osmo-cbc will require asn1c master available, since the skeleton files found in /usr/share/asn1c are required during compilation time of the generated code (asn_ioc.h, asn_application.h, etc.)
Not sure what do we want to do here.

WIP available in osmo-cbc.git branch "pespin/sbcap".

Actions #13

Updated by laforge almost 2 years ago

On Thu, Jun 23, 2022 at 05:25:54PM +0000, pespin wrote:

https://github.com/mouse07410/asn1c
That fork should be rebased on top of current master and it seems to receive fixes even in 2022.

I think the mouse07410 only misses 2-3 commits, not sure if it's worth to rebase. I'd try it as-is

So far I'm doing the build with usptream master asn1c with --gen-PER, but we should move to that fork and user APER.

Also another issue:
It seems for our old build of asn1c we use for osmo-iuh.git we have a split libasn1c.git containing a runtime library. AFAICT that's done by ourselves and asn1c master doesn't contain that library. Hence, however wants to build osmo-cbc will require asn1c master available, since the skeleton files found in /usr/share/asn1c are required during compilation time of the generated code (asn_ioc.h, asn_application.h, etc.)
Not sure what do we want to do here.

Have the newly generated skeleton files for SBcAP inside a shared library that hides all the symbol to avoid
clashes? The shlib should be inside some omso-cbc subdirectory, IMHO.

Actions #14

Updated by pespin over 1 year ago

Status update: In osmo-cbc.git "pespin/sbcap" branch I have a running osmo-cbc linked properly towards SBcAP library with code generated from asn1c master. The asn1c code generation into C code and the C compilation steps are decoupled.
The first step requires asn1c and can be re-run using "make -C src/ regen", which copies over all the required C autogenerated files and headers, towards with asn1c support/skeleton files.
All those files are included in to the git repository, so that second step (regular "./configure && make") doesn't require asn1c at all.

Next thing to do now is to regen the C files using "make -C src/ regen" with an asn1c compiler built from mouse07410 repo instead of upstream, in order to get APER generation code.

Once that's done, I can continue with the TTCN3 tests by validating RESET/RESTART stuff is being sent in tests.

laforge wrote in #note-13:

https://github.com/mouse07410/asn1c
That fork should be rebased on top of current master and it seems to receive fixes even in 2022.

I think the mouse07410 only misses 2-3 commits, not sure if it's worth to rebase. I'd try it as-is

Agree, I meant something more like "should already be rebased on top of current master" in here :)

Actions #15

Updated by pespin over 1 year ago

  • Spec Reference changed from 3GPP TS 29.169 to 3GPP TS 29.168
Actions #16

Updated by pespin over 1 year ago

I already have generated code from mouse07410 GH repo imported into osmo-cbc. I started doing some tests encoding a SBcAP message but the encoder crashes the process. After debugging for a while I found a bug in the asn1c code generator and submitted a PR here, which fixes the crash:
https://github.com/mouse07410/asn1c/pull/92

Actions #17

Updated by pespin over 1 year ago

  • % Done changed from 20 to 50

Status update:
I have everything in osmo-ttcn3-hacks.git, osmo-cbc.git branch "pespin/sbcap". With what I have there, I already have some TTCN3 CBC_Tests tests passing. I still need to check some IEs in Write-replace-Request

I also submitted a fix for osmo-cbc which I detected while working on SBc-AP with TTCN3, but I think it should also affect CBSP:
https://gerrit.osmocom.org/c/osmo-cbc/+/28586

I investigated a couple more bugs in the asn1c APER encoder and submitted here:
https://github.com/mouse07410/asn1c/pull/96
https://github.com/mouse07410/asn1c/pull/97

Actions #18

Updated by pespin over 1 year ago

  • % Done changed from 50 to 60

I submitted a bunch of patches for osmo-cbc, osmo-ttcn3-hacks and docker-playground, adding similar level of support for 4G to what we currently have for 2G.

Further requested features can be implemented on top of that work separately.

https://gerrit.osmocom.org/c/osmo-cbc/+/28600
https://gerrit.osmocom.org/c/osmo-cbc/+/28601
https://gerrit.osmocom.org/c/osmo-cbc/+/28602
https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/28603
https://gerrit.osmocom.org/c/docker-playground/+/28604

Actions #19

Updated by osmith over 1 year ago

Packaging related patches for libosmo-sbcap: https://gerrit.osmocom.org/q/topic:sbcap-packaging

Actions #20

Updated by osmith over 1 year ago

Note that the testsuite is currently failing on arm:

[ 509s] 1. testsuite.at:4: 1. sbcap (testsuite.at:4): FAILED (testsuite.at:7)

https://obs.osmocom.org/package/live_build_log/osmocom:nightly/osmo-cbc/Raspbian_10/armv7l

Actions #21

Updated by pespin over 1 year ago

osmith wrote in #note-20:

Note that the testsuite is currently failing on arm:

[ 509s] 1. testsuite.at:4: 1. sbcap (testsuite.at:4): FAILED (testsuite.at:7)

https://obs.osmocom.org/package/live_build_log/osmocom:nightly/osmo-cbc/Raspbian_10/armv7l

I'm currently trying to reproduce this building and running manually in rpi4-3 (10.9.25.70), but I couldn't so far.

I submitted a PR fixing tons of formatting errors in 32bit archs (armv7):
https://github.com/mouse07410/asn1c/pull/101

I also fixed a small issue with printf formats in osmo-cbc itself:
https://gerrit.osmocom.org/c/osmo-cbc/+/28645 Fix printf format in 32bit arch

Actions #22

Updated by pespin over 1 year ago

I fixed a regression just introduced by the APER maintainer, which makes the osmo-cbc unit test fail:
https://github.com/mouse07410/asn1c/pull/104

Once it gets merged, I'll update the asn1c skeleton files in osmo-cbc.git, which should then also contain the 32bit printf format errors. Having these in place will hopefully fix the crashes we see running the unit test in raspbian 10 in OBS.

Actions #23

Updated by pespin over 1 year ago

  • Checklist item SBcAP teste cases in TTCN3 set to Done
  • Checklist item SBcAP message handling / logic in OsmoCBC set to Done
Actions #24

Updated by pespin over 1 year ago

Initial support for SBc-AP has just been merged in osmo-cbc, as well as related TTCN3 improvements to test it.

Next step: Testing osmo-cbc SBc-AP against Amarisoft network.

Actions #25

Updated by pespin over 1 year ago

  • Status changed from In Progress to Feedback
  • Assignee changed from pespin to laforge

I have been working on running amarisoft network + enodeB (b200) on a debian9 LXC container. I documented the steps here:
https://projects.sysmocom.de/projects/miscprojects/wiki/Amarisoft_LTE#section-6

I tried using the newest Amarisoft release available under software-lab shared volume (2021-12-17), but apparently the dongle available contains a license restricting to older versions:

Your license is too old for this version.
Support and software update available until 2020-03-19.

So I went on with the newest version supported by the license (2020-01-23). With that, I am at a point where I have the network running and a phone attaching and using 4G data successfully.

Unfortunately, it seems the CBC functionality in amarisoft is not available in 2020-01-23 I'm using. It seems to be available in the latest 2021-12-17 though.

I see "CBC"/"sbc" text in pdf documentation of 2021-12-17 (2021-12-17/ltemme-linux-2021-12-17/doc/ltemme.pdf), it also states there's a "sbc" command in ltemme terminal. But in documentation of 2020-01-23 there's nothing "CBC"/"sbc" related, no "sbc" command in the terminal (just tried), and "ss -lnp" shows nothing on default Sbc-AP port 29168:

sbc
Get list of CBC connections.
Response definition:
cbc_list
Array of object. Each object represents a CBC connection:
address
lcs
String. CBC address and port.
(mme) sbc
Unknown command: sbc
# ss -lnp | grep ltemme
udp    UNCONN  0        0                              127.0.1.100:2152                                              0.0.0.0:*                                   users:(("ltemme",pid=1750,fd=7))
tcp    LISTEN  0        5                                  0.0.0.0:9000                                              0.0.0.0:*                                   users:(("ltemme",pid=1750,fd=13))
sctp   LISTEN  0        5                              127.0.1.100:36412                                             0.0.0.0:*                                   users:(("ltemme",pid=1750,fd=6))
sctp   LISTEN  0        5                              127.0.1.100:29118                                             0.0.0.0:*                                   users:(("ltemme",pid=1750,fd=11))
sctp   LISTEN  0        5                              127.0.1.100:3868                                              0.0.0.0:*                                   users:(("ltemme",pid=1750,fd=8))

BTW, according to config it seems amarisoft sets up SBc-AP as an SCTP server, which means we'll need to configure osmo-cbc as an SCTP client. That should be supported in current code but probably not yet tested.

Assigning to laforge for him to generate a new dongle with an up to date license.

Actions #26

Updated by pespin over 1 year ago

I have CBSP and SBc-AP TCP/SCTP client mode implemented in osmo-cbc.git branch "pespin/sbcap".

I am currently testing it against an Amarisoft 2022-06-18 ltemme+lteenb.

I used this command to generate a sample CBS message:

./cbc-apitool.py -H 10.0.3.40 -p 12345 -v create-cbs --msg-id 3 --payload-data-utf8 12345

I attach the resulting pcap file containing SBc-AP between osmo-cbc and ltemme, and S1AP between ltemme and lteenb.
According to ltemme and lteenb everything seems to look good (cause=accepted), but I don't get any notification in the UE attached to the network, I still need to investigate why.

PS: amarisoft ltemme seems to crash everytime I kill the SCTP SBc-AP connection (kill osmo-cbc).

Actions #27

Updated by pespin over 1 year ago

Here's attached an SBc-AP Stop-Warning-Request and its counterpart accepted response osmo-cbc<->ltemme from an ETWS message which was previously generated (not in the pcap).

The Stop-Warning-Request was generated with:

./cbc-apitool.py -H 10.0.3.40 -p 12345 -v delete --msg-id 4
DELETE http://10.0.3.40:12345/api/ecbe/v1/message/4 (None)
-> <Response [200]>

Actions #28

Updated by pespin over 1 year ago

I got the first ETWS Earthquake alert on my phone after some encoding fixes to the Warning-Type IE.

 $ ./cbc-apitool.py -H 10.0.3.40 -p 12345 -v create-etws --msg-id 4352
POST http://10.0.3.40:12345/api/ecbe/v1/message ({'cbe_name': 'cbc_apitool', 'category': 'normal', 'repetition_period': 5, 'num_of_bcast': 999, 'scope': {'scope_plmn': {}}, 'smscb_message': {'message_id': 4352, 'serial_nr': {'serial_nr_decoded': {'geo_scope': 'plmn_wide', 'msg_code': 768, 'update_nr': 0}}, 'payload': {'payload_etws': {'warning_type': {'warning_type_decoded': 'earthquake'}, 'emergency_user_alert': True, 'popup_on_display': True}}}})
-> <Response [201]>

$ ./cbc-apitool.py -H 10.0.3.40 -p 12345 -v delete --msg-id 4352
DELETE http://10.0.3.40:12345/api/ecbe/v1/message/4352 (None)
-> <Response [200]>

TODO: It seems the messages are not deleted properly after I do cbc-apitool delete. It seems to delete properly towards the MME, but I can see them still showing up in vty "show messages" afterwards:

OsmoCBC# show messages etws
|MsgId|SerNo|      CBE Name      |  Category   |Period|E|Warning Type|
|-----|-----|--------------------|-------------|------|-|------------|
| 1100| 7000|cbc_apitool         |Normal       |  5   |N|      0000|

Actions #29

Updated by pespin over 1 year ago

I submitted some patches to gerrit to improve a bit the managament of smscb messages in osmo-cbc VTY.

I did some testing against ltemme+ltemme + android phone and I'd say minimal ETWS functionality to have alerts transmitted is working fine.
I'm not sure what's the expected behavior for CBS messages though, I still need to investigate that. Some insight on what/how to test those would be welcome, since I didn't play much with CBS and CBSP yet.

Nexts steps I'd propose would be:
  • Implement the indication flags in Write-Replace Request so that the MME sends us back indications of which ENBs/cells received successfully the message, and handle that in osmo-cbc so that it can be seen in the rows already present in VTY "show message id". ltemme seems to have some configuration options regarding the indication list which may need tweaking, they are listed in the user manhual pdf.
  • Implement the minimal subset in open5gs-mme to handle SBc-AP and fill in the open5gs structures. From then on, the S1AP part is already implemented AFAICT.
  • Figure out how the Replace stuff is supposed to work and find out whether it's properly implemented. AFAIU using "--update-nr 1" in cbc-apitool would trigger a different serial_nr which would cause replace of the existing mesage? But it's not working right now:
    20220725173933432 DREST DEBUG /message POST: {"cbe_name": "cbc_apitool", "category": "normal", "repetition_period": 5, "num_of_bcast": 999, "scope": {"scope_plmn": {}}, "smscb_message": {"message_id": 4352, "serial_nr": {"serial_nr_decoded": {"geo_scope": "plmn_wide", "msg_code": 768, "update_nr": 1}}, "payload": {"payload_etws": {"warning_type": {"warning_type_decoded": "earthquake"}, "emergency_user_alert": true, "popup_on_display": true}}}} (/osmo-cbc/src/rest_api.c:571)
    ...
    20220725173933432 DCBSP ERROR Cannot create message_id 4352 (already exists) (/osmo-cbc/src/smscb_message_fsm.c:353)
    20220725173933432 DREST DEBUG rest_it_op_complete(op=0x7fb090001730) complete (/osmo-cbc/src/rest_it_op.c:131)
    20220725173933432 DREST DEBUG rest_it_op completed with 0 (HTTP 409) (/osmo-cbc/src/rest_it_op.c:71)
    20220725173933432 DREST DEBUG /message POST -> 409 (Could not allocate) (/osmo-cbc/src/rest_api.c:589)
    
Actions #30

Updated by pespin over 1 year ago

The crash in ltemme when SBcAP ScTP conn is closed has been fixed in amarisoft release 2022-07-26.

I'm currently testing the Write-Replace-Warning-Indication feature against Amarisoft. I attach a pcap showing the procedure.

The reported cell shows up correctly in VTY:

OsmoCBC# show messages etws
|MsgId|SerNo|      CBE Name      |  Category   |Period|E|Warning Type|       Created      |       Expired      |
|-----|-----|--------------------|-------------|------|-|------------|--------------------|--------------------|
| 1100| 7001|cbc_apitool         |Normal       |  5   |N|        0000|2022-07-27T15:07:46Z|2022-07-27T15:07:57Z|
OsmoCBC# show message id 4352
Message ID 1100, Serial Number 7001, State: DELETED
 Created by CBE 'cbc_apitool' at Wed Jul 27 15:07:46 2022
 Repetition Period: 5 ( 9.41s), Number of broadcasts: 999
 ETWS Warning Type Value: 0x00, User Alert: On, Popup: On
 Security: 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
 Peer: 'mme-amarisoft', State: DELETED
  Cells Installed:
  Cells Failed:
  Number of Broadcasts Completed:
   ECGI 234-70-1715456 (1/1)

Only the CI is wrong, I decoded it as being 24 bits instead of 28, I'll fix that soon.

Actions #31

Updated by pespin over 1 year ago

I submitted a bunch of patches adding several improvements and fixes, and adding support for Write-Replace-Warning-Indication, so now we see from VTY which cells successfully received or which failed.

TODO:
  • Split smscb_peer_fsm.c into 2 different FSM implementations with same interface: smscb_peer_cbsp_fsm.c and smscb_peer_sbcap_fsm.c
  • Figure out testing of CBS messages (non-ETWS), how to run test and what to expect as a user (laforge may be able to provide some feedback here since he probably tested that for CBSP at the time).
  • Implement the minimal subset in open5gs-mme to handle SBc-AP
  • Test against open5gs+srsenb (verify first that srsenb actually supports it).
Actions #32

Updated by pespin over 1 year ago

  • Checklist item manual tests with a MME supporting SBcAP set to Done
  • Assignee changed from laforge to pespin
  • % Done changed from 60 to 70

pespin wrote in #note-31:

TODO:
  • Split smscb_peer_fsm.c into 2 different FSM implementations with same interface: smscb_peer_cbsp_fsm.c and smscb_peer_sbcap_fsm.c

Done: https://gerrit.osmocom.org/c/osmo-cbc/+/28828 Split smscb_peer_fsm into CBSP and SBcAP specific FSMs

  • Figure out testing of CBS messages (non-ETWS), how to run test and what to expect as a user (laforge may be able to provide some feedback here since he probably tested that for CBSP at the time).

I was able to test this successfully by using msg id "Presidential Alert" (4370). When using it I successfully see the message "Hellooo" displayed at the phone.

./cbc-apitool.py -H 10.0.3.40 -p 12345 -v create-cbs --msg-id 4370 --update-nr 1 --payload-data-utf8 "Hellooo" 
./cbc-apitool.py -H 10.0.3.40 -p 12345 -v delete --msg-id 4370

Actions #33

Updated by pespin over 1 year ago

I'm currently improving the TTCN3 tests.

while doing so I found out that I may be missing to build up a target list in Stop-Warning-Request, based on what it was received in Write-Replace-Warning-Indication.
This is similarly done by CBSP code, where the KILL uses whatever list was received during Write-Replace Complete message. SBc-AP doesn't receive a list in Write-Replace-Response, instead it has to rely on Write-Replace-Warning-Indication.

Actions #34

Updated by pespin over 1 year ago

pespin wrote in #note-33:

while doing so I found out that I may be missing to build up a target list in Stop-Warning-Request, based on what it was received in Write-Replace-Warning-Indication.
This is similarly done by CBSP code, where the KILL uses whatever list was received during Write-Replace Complete message. SBc-AP doesn't receive a list in Write-Replace-Response, instead it has to rely on Write-Replace-Warning-Indication.

Actually that may not be a good behavior, since there could be a race condition between the Stop-Warning-Request being sent and new Write-Replace-Indication containing new cells arriving.
Furthermore, I think if the initial scope was "plmn" (aka empty list), then it makes sense to also keep sending an empty list when stopping it.

Actions #35

Updated by pespin over 1 year ago

  • % Done changed from 70 to 90

I have been considerably improving the TTCN3 CBC_Tests testsuite with new tests and a new infrastructure which also lets make tests with different scenarios rather easy.
During the process, I also improved osmo-cbc support in several places as well as fixed bugs in both CBSP and SBc-AP.

I also spotted a CBC feature which is not yet implemented in CBSP nor in SBc-AP: Reload peers with active messages on cells being reported as "restarted". I created a ticket here to track this topic, since it alone can take some time to think about and implement: #5641

Right now I consider SBc-AP support to be on par with existing CBSP one, which allows using osmo-cbc as a basic CBC towards both BSC and MMEs. Specific improvements or bugs can be tracked on newly created specific tickets.

I plan to do the following on Monday to wrap up and finish with this task:
  • Get all pending patches in gerrit merged.
  • Manual test against amarisoft MME to make sure some of my last improvements/fixes didn't break anything there.
  • Make a new release of osmo-cbc.
Actions #36

Updated by pespin over 1 year ago

  • % Done changed from 90 to 100

I did a last round of successful interop testing against amarisoft ltemme+lteenb.
I made a new release of osmo-cbc which contains all the improvements and fixes to support SBc-AP:
https://gerrit.osmocom.org/c/osmo-cbc/+/28975

Once merged and the tag is pushed, the ticket can be closed.

Actions #37

Updated by pespin over 1 year ago

  • Status changed from Feedback to Resolved

Tag 0.4.0 pushed, closing ticket.

Actions #38

Updated by laforge about 1 year ago

Note: Development of this feature was provided by NLnet foundation, see https://nlnet.nl/project/OsmoCBC/

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)