Project

General

Profile

Actions

Feature #4472

closed

Intra-domain connection of OsmoGBPROXY to multiple SGSNs (pooling)

Added by laforge almost 4 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
Start date:
03/29/2020
Due date:
% Done:

100%

Spec Reference:
TS 23.236 Section 5.3.2

Description

SGSN pooling is based on splitting the P-TMSI into two parts: A few bits that identify the SGSN in the pool (the so called NRI), and the remainder, which identifies the subscriber within the SGSN.

At time of first contact (random/foreign TLLI), there is a node selection function which chooses the SGSN to use for this request. All follow-up messages are then handled via this SGSN.

This feature would have to be implemented in OsmoPCU, alongside with corresponding test suite in TTCN-3.

Each PCU then has multiple Gb links; at least one to each of the SGSNs in the pool.


Files

sgsn-pool.png View sgsn-pool.png 153 KB laforge, 11/04/2020 11:48 AM
bssgp-pdu-overview.gnumeric bssgp-pdu-overview.gnumeric 57.2 KB laforge, 12/06/2020 11:21 AM
bssgp-pdu-overview.png View bssgp-pdu-overview.png 278 KB laforge, 12/06/2020 11:22 AM

Checklist

  • per-BVC FSM for RESET/BLOCK/UNBLOCK
  • ability to configure multiple SGSN-side NSE in gbproxy
  • TTCN3 test case for RESET/BLOCK/UNBLOCK with two SGSNs in pool
  • NRI configuration in VTY
  • uplink message routing based on NRI extracted from TLLI
  • RADIO-STATUS
  • BVC FLOW CONTROL
  • RIM
  • MS-REGISTRATION-ENQUIRY
  • PS-PAGING-REJECT (requires ISMI cache)

Related issues

Related to OsmoBSC - Feature #3682: Intra-domain connection of OsmoBSC to multiple MSCsResolvedneels11/06/2018

Actions
Related to osmo-gbproxy - Bug #4897: gbproxy2: Re-introduce handling of NS_AFF_CAUSE_FAILURENewdaniel12/08/2020

Actions
Related to osmo-gbproxy - Feature #4896: gbproxy2: Routing of RIM messagesResolved12/08/2020

Actions
Related to osmo-gbproxy - Bug #4895: gbproxy2: Implement small TLLI cache for SUSPEND/RESUME procedureResolveddaniel12/08/2020

Actions
Related to osmo-gbproxy - Bug #4894: gbproxy2: Revisit information storageResolveddaniel12/08/2020

Actions
Related to osmo-gbproxy - Feature #4893: gbproxy2: Make feature bitmap configurableNew12/08/2020

Actions
Related to osmo-gbproxy - Bug #4892: gbproxy2: Route BSSGP-STATUS based on "Erroneous PDU IE"Resolveddaniel12/08/2020

Actions
Related to osmo-gbproxy - Bug #4891: gbproxy2: Implement processing of BVC-Flow-Control via BVC FSMStalleddaniel12/08/2020

Actions
Related to osmo-gbproxy - Bug #4890: gbproxy2: VTY configuration of NRI / pool related bitsResolveddaniel12/08/2020

Actions
Related to osmo-gbproxy - Bug #4889: implement truncating of BSSGP STATUS when exceeding the FR MTUResolveddaniel12/06/2020

Actions
Related to osmo-gbproxy - Feature #4951: more TTCN3 tests for SGSN poolingIn Progressdaniel01/15/2021

Actions
Related to osmo-gbproxy - Bug #4954: Fix routing of RADIO-STATUS by TMSIResolveddaniel01/17/2021

Actions
Actions #1

Updated by laforge almost 4 years ago

  • Related to Feature #3682: Intra-domain connection of OsmoBSC to multiple MSCs added
Actions #2

Updated by ipse almost 4 years ago

It would be really nice if this feature could be implemented both in OsmoPCU and OsmoGbProxy at the same time - OsmoGbProxy is very helpful in practical installations.

Actions #3

Updated by laforge almost 4 years ago

On Sun, Mar 29, 2020 at 07:15:37PM +0000, ipse [REDMINE] wrote:

It would be really nice if this feature could be implemented both in OsmoPCU and OsmoGbProxy at the same time - OsmoGbProxy is very helpful in practical installations.

I understand that, and obviously I want that more than anyone else.
The problem with pretty much all 'feature' tickets here is: Who is going to contribute related code /
developer time or funds :) It's not like we're seeing a lot of resources thrown at Osmocom...

Actions #4

Updated by laforge over 3 years ago

  • Subject changed from Intra-domain connection of OsmoPCU to multiple SGSNs to Intra-domain connection of OsmoPCU to multiple SGSNs (pooling)
Actions #5

Updated by laforge over 3 years ago

  • Priority changed from Low to High
Actions #6

Updated by laforge over 3 years ago

It has become clear that this feature will be implemented in a generic nature so it can be used from both osmo-pcu and osmo-gbproxy.

What we need to make this work is:
  • NAS Node Selection Function inside BSSGP code on BSS side
  • multiple NSE inside the BSS (one for each SGSN), each with it's own set of NS-VCs
  • signaling BVC from each SGSN
  • separate PTP BVC from each cell to each SGSN

This diagram from TS 48.016 serves as a good illustration:

Actions #7

Updated by laforge over 3 years ago

  • Status changed from New to In Progress
  • Assignee set to laforge

WE could implement this "only" in OsmoPCU, but we also need it in osmo-gbproxy. Unfortuantely the proxy as a "MITM" has quite some different implementation requirements as the PCU, so there would be relatively separate implementation in both PCU and gbproxy, using some common library/infrastructure.

The easier approach is probably to focus on gbproxy for now. If somebody needs this for a single osmo-pcu, they can always run osmo-gbproxy in front in order to get the pooling feature. And for people running multiple omso-pcu, they will want osmo-gbproxy anyway for aggregation needs.

osmo-gbproxy

I did an analysis for osmo-gbproxy in terms of what kind of processing we need to introduce at each BSSGP message.

The general assumption is that we will have
  • multiple PCU/BSS side Gb interfaces, each
    • with one NSE/NS-VCG
    • one or multiple NS-VC
    • one or multiple PTP BVC
    • no support for pooling in the PCU required or enabled
  • multiple SSN side Gb interfaces, each
    • with one NSE/NS-VCG
    • one or multiple NS-VC
    • support for pooling in the SGSN enabled

For each PTP BVC, we need to "duplicate" it, i.e. establish one per-SGSN BVC for each BSS side BVC. This is achieved by "terminating" the BVC-RESET locally in the Gbproxy on both sides, and having related interconnects, i.e. a BVC-RESET for a PTP BVC on the BSS side will trigger Nx BVC-RESET on the SGSN side, one for each SGSN/NSE.

As for the individual messages:

Downlink-Only messages with no routing requirements

Downlink-only messages are accepted from any SGSN and routed baesd on BVCI, just like it is done prior to this feature.

  • RA-CAPABILITY
  • DL-UNITDATA
  • PAGING-PS
  • PAGING-CS
  • RA-CAPABILITY-UPDATE-ACK
  • SUSPEND-ACK
  • SUSPEND-NACK
  • RESUME-ACK
  • RESUME-NACK
  • FLUSH-LL
  • FLOW-CONTROL-MS-ACK
  • FLOW-CONTROL-PFC-ACK
  • CREATE-BSS-PFC
  • MODIFY-BSS-PFC
  • DELETE-BSS-PFC
  • PS-HANDOVER-REQUIRED-ACK
  • PS-HANDOVER-REQUIRED-NACK
  • PS-HANDOVER-REQUEST
  • PS-HANDOVER-COMPLETE-ACK
  • PERFORM-LOCATION-REQUEST
  • PERFORM-LOCATION-ABORT
  • POSITION-RESPOSNE

Messages routed by NRI extracted from TLLI

  • UL-UNITDATA
  • RA-CAPABILITY-UPDATE
  • SUSPEND
  • RESUME
  • FLUSH-LL-ACK
  • LLC-DISCARDED
  • FLOW-CONTROL-MS
  • FLOW-CONTROL-PFC
  • CREATE-BSS-PFC-ACK/NACK
  • MODIFY-BSS-PFC-ACK
  • DELETE-BSS-PFC-ACK
  • DELETE-BSS-PFC-REQ
  • PS-HANDOVER-REQUIRED
  • PS-HANDOVER-REQUEST-ACK
  • PS-HANDOVER-REQUEST-NACK
  • PS-HANDOVER-COMPLETE
  • PS-HANDOVER-CANCEL
  • PERFORM-LOCATION-RESPONSE
  • POSITION_COMMAND

Special cases

RADIO-STATUS

  • if TLLI or P-TMSI are included, route based on NRI
  • if IMSI is included, route to "randomly distributed" SGSN in pool

BVC flow control

The capacity advertised by the BSS/PCU side has to be split between all SGSNs. Separate BVC flow control instances need to be operated in gbproxy for each BVC, with coordination between them. Probably implemented by osmo_fsm.

STATUS

The STATUS message doesn't really tell us where it belongs to, unless the contained PDU would have TLLI or P-TMSI information. We could consider broadcasting it?

BVC BLOCK/UNBLOCK/RESET

Those have to be terminated for each BVC inside osmo-gbproxy: One each on the BSS/PCU side, and one per-SGSN-per-BVC on the SGSN side. Coordination must happen between them. Probably implemented by osmo_fsm.

SGSN-INVOKE-TRACE

This downlink-only message must be broadcast to all BSS, like it is done in the non-pooling case.

OVERLOAD

This downlink-only message must be broadcast to all BSS

RIM

It is assumed any SGSN can route RIM messages to any other PCU, so we can simply chose any of them. In the case of responses it might be a good idea to use the same thorugh which the request was received.

MS-REGISTRATION-ENQUIRY

The ENQUIRY is in uplink direction and contains no TlLI but only IMSI. We can route it to any SGSN. The ENQUIRY-RESPONSE is in downlink direction and must be sent back to whichever PCU/NSE requesting it. We therefore need to track some state here.

Actions #8

Updated by laforge over 3 years ago

  • Checklist item per-BVC FSM for RESET/BLOCK/UNBLOCK added
  • Checklist item ability to configure multiple SGSN-side NSE in gbproxy added
  • Checklist item TTCN3 test case for RESET/BLOCK/UNBLOCK with two SGSNs in pool added
  • Checklist item NRI configuration in VTY added
  • Checklist item uplink message routing based on NRI extracted from TLLI added
  • Checklist item RADIO-STATUS added
  • Checklist item BVC FLOW CONTROL added
  • Checklist item RIM added
  • Checklist item MS-REGISTRATION-ENQUIRY added

I've started with implementing the slightly contrived BVC-RESET handling, as this will be the logical first step we need in order to bring up a GbProxy with a pool of several SGSNs.

There will be a per-BVC osmo_fsm that takes care of reset/block/unblock. This FSM can behave in either SGSN or BSSGP role, and gb-proxy will use this FSM both towards each BSS and towards each SGSN.

state-change notification call-backs and reset-indication-callbacks will be used by gbproxy to "stitch" those FSMs together. So for example, if there's a reset-indication from a BSS-side FSM, then that will trigger "reset request FSM events" on each SGSNs BVC-FSM for the same BVCI, which iwll then in turn trigger BVC-RESET procedures towards each of those SGSNs.

The plan is to get this implemented together with a minimal "bring up all BVCs with two SGSN" TTCN3 testcase, and then hand it over to daniel for implementation of the handling of various PDU types as outlined here in this ticket.

Actions #9

Updated by laforge over 3 years ago

I've created a table that shows all BSSGP PDUs, their direction UL/DL, SIG/PTP BVC, SAP as well as the Prosy handling in the context of multiple SGSN / pool support.

bssgp-pdu-overview.gnumeric

Actions #10

Updated by laforge over 3 years ago

  • Project changed from OsmoPCU to osmo-gbproxy
  • Subject changed from Intra-domain connection of OsmoPCU to multiple SGSNs (pooling) to Intra-domain connection of OsmoGBPROXY to multiple SGSNs (pooling)
Actions #11

Updated by laforge over 3 years ago

  • Checklist item per-BVC FSM for RESET/BLOCK/UNBLOCK set to Done
  • Checklist item RADIO-STATUS set to Done
  • % Done changed from 0 to 20
In the laforge/gbproxy branch, there is now the foundation for this feature.
  • all data structures have been adjusted to accomodate the pooling situation
  • BVC-RESET/BLOCK/UNBLOCK is now terminated inside the proxy for all BVCs on all NSEs (BSS, SGSN)
  • RADIO STATUS routing based on either TLLI, TMSI or IMSI is implemented
Actions #12

Updated by laforge over 3 years ago

  • Related to Bug #4897: gbproxy2: Re-introduce handling of NS_AFF_CAUSE_FAILURE added
Actions #13

Updated by laforge over 3 years ago

  • Related to Feature #4896: gbproxy2: Routing of RIM messages added
Actions #14

Updated by laforge over 3 years ago

  • Related to Bug #4895: gbproxy2: Implement small TLLI cache for SUSPEND/RESUME procedure added
Actions #15

Updated by laforge over 3 years ago

  • Related to Bug #4894: gbproxy2: Revisit information storage added
Actions #16

Updated by laforge over 3 years ago

  • Related to Feature #4893: gbproxy2: Make feature bitmap configurable added
Actions #17

Updated by laforge over 3 years ago

  • Related to Bug #4892: gbproxy2: Route BSSGP-STATUS based on "Erroneous PDU IE" added
Actions #18

Updated by laforge over 3 years ago

  • Related to Bug #4891: gbproxy2: Implement processing of BVC-Flow-Control via BVC FSM added
Actions #19

Updated by laforge over 3 years ago

  • Related to Bug #4890: gbproxy2: VTY configuration of NRI / pool related bits added
Actions #20

Updated by laforge over 3 years ago

  • Related to Bug #4889: implement truncating of BSSGP STATUS when exceeding the FR MTU added
Actions #21

Updated by laforge over 3 years ago

  • Checklist item BVC FLOW CONTROL set to Done
Actions #22

Updated by laforge over 3 years ago

  • % Done changed from 20 to 40
Actions #23

Updated by laforge over 3 years ago

https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/21698 contains a WIP patch that adds support for checking the BVC-bringup to two SGSNs. It is working fine in local tests. Patch cannot be merged yet as it is waiting for the NS2 VTY related changes to be completed by lynxis

Actions #24

Updated by daniel over 3 years ago

In addition to NRI configuration in VTY issue #4890 will also implement the ability to configure multiple SGSN-side NSE in gbproxy

Actions #25

Updated by daniel about 3 years ago

  • Checklist item ability to configure multiple SGSN-side NSE in gbproxy set to Done
  • Checklist item NRI configuration in VTY set to Done
  • Checklist item uplink message routing based on NRI extracted from TLLI set to Done
  • Assignee changed from laforge to daniel
  • % Done changed from 40 to 60

Patches in https://gerrit.osmocom.org/q/topic:%22sgsn-pooling%22+(status:open%20OR%20status:merged) add pooling support to gbproxy, but still untested (apart from the simple case of only having one SGSN in the pool that is active).

I'm now implementing a TLLI cache to forward SUSPEND/RESUME properly.

Actions #26

Updated by laforge about 3 years ago

  • Checklist item TTCN3 test case for RESET/BLOCK/UNBLOCK with two SGSNs in pool set to Done
Actions #27

Updated by laforge about 3 years ago

  • % Done changed from 60 to 70

BVC-{RESET,BLOCK,UNBLOCK} for multiple SGSNs is now tested in https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/21698 as part of the normal test case initialization function

Actions #28

Updated by laforge about 3 years ago

Just committed a series of patches with https://gerrit.osmocom.org/c/osmo-ttcn3-hacks/+/22231 on top which expand testing to include SGSN pool related testing. We're already verifying
  • Suspend/Resume is routed by NRI to correct SGSN
  • BVC-RESET is duplicated to every SGSN
  • BVC-FC is duplicated to every SGSN

Most of the tests currently test only with a NRI routed to the first SGSN.

I'm creating a new ticket (#4951) to track the missing TTCN3 tests so we can use this ticket to really keep only track of the actual features (only RIM + MS-REQ-ENQ missing)

Actions #29

Updated by laforge about 3 years ago

  • Related to Feature #4951: more TTCN3 tests for SGSN pooling added
Actions #30

Updated by laforge about 3 years ago

  • Checklist item PS-PAGING-REJECT (requires ISMI cache) added
Actions #31

Updated by daniel about 3 years ago

Interesting that 3GPP TS 48.018 does not specify how PS-PAGING-REJECT or DUMMY-PAGING should be sent. I'm looking at Release 16 and Table 5.4.1 has no mention of those. Table 5.2 and 5.3 also don't. I'm guessing we can assume that it's PTP or signalling just like PAGING-PS.

Actions #32

Updated by daniel about 3 years ago

IMSI cache and support for PAGING_PS_REJECT (at lease on signalling) are in Gerrig:
https://gerrit.osmocom.org/c/osmo-sgsn/+/22251
https://gerrit.osmocom.org/c/osmo-sgsn/+/22252

Actions #33

Updated by daniel about 3 years ago

  • Checklist item PS-PAGING-REJECT (requires ISMI cache) set to Done
Actions #34

Updated by daniel about 3 years ago

  • Related to Bug #4954: Fix routing of RADIO-STATUS by TMSI added
Actions #35

Updated by daniel about 3 years ago

  • Checklist item RIM set to Done

RIM support has been merged, tests are passing

Actions #36

Updated by laforge over 2 years ago

so MS-REQ-ENQ is the only missing bit to get this issue closed?

Actions #37

Updated by daniel over 2 years ago

Apart from BSSGP STATUS routing which is in #4892 yeah. I can implement it, shouldn't take too long. We already have the IMSI cache which we need to route the response.

Actions #39

Updated by daniel over 2 years ago

  • Checklist item MS-REGISTRATION-ENQUIRY set to Done
  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

The MS Reg enquiry patches are now merged

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)