Cellular Network Infrastructure » Distributed GSM

Thoughts about data persistence

What data is stored in the proxy?

• IMSI (key)
• MSISDN (for mslookup)
• last CS LU (for mslookup)
• last CS LU VLR's IPA name (which MSC is the phone attached at)
• auth tuples
• auth tuple state? (which ones used how often?)
• LU "faked" locally or solicited by home HLR?

Auth tuples: see #4305

What do we need to persist?

The scenarios are:
(A) home HLR unreachable
(B) proxy HLR restarted
(C) both

• A subscriber is currently attached at a proxy HLR, and the link to the home HLR goes down or unresponsive.
• We've asked for auth tuples before. For incoming SendAuthInfo-Request from a local VLR (MSC or SGSN),
  we use the cached auth tuples and don't even attempt asking the home HLR.
• Once the VLR has authenticated, it will follow up with an UpdateLocation-Request.
  We attempt to forward, but the home HLR doesn't respond (timeout?) or the GSUP client is unable to connect.
• We send an UpdateLocation-Result (and flag in the proxy data that we have "faked" a LU without the home HLR knowing? may even be unnecessary).

Maybe it would make sense to actually never forward UpdateLocation to the home HLR, and always keep that entirely in the proxy HLR?
Except, when the home HLR has disabled the subscriber (nam_cs/nam_ps), the proxy HLR won't know about that.
IIUC the SendAuthInfo-Request is serviced regardless of nam_cs/nam_ps, so the proxy would still get tuples
and continue to accept LU, even though the home HLR has disabled the subscriber.
Also, if the MSISDN or anything else changes, the proxy needs to obtain an update as soon as possible.
So, it does make sense to at least attempt to contact the home HLR for each periodic LU.

• A subscriber is currently attached at a proxy HLR, and the proxy HLR experiences power outage / program restart.
• After the outage, the subscriber does a Complete-Layer-3 with auth+ciph
• We still have usable tuples in the persistent cache, we use those and don't even do SendAuthInfo-Request to the home HLR.
• Upon UpdateLocation-Request, we reach the home HLR and continue as usual.

• A subscriber is currently attached at a proxy HLR, and the proxy HLR experiences power outage / program restart.
• After the outage, the subscriber does a Complete-Layer-3 with auth+ciph
• We still have usable tuples in the persistent cache, we use those and don't even do SendAuthInfo-Request to the home HLR.
• Upon UpdateLocation-Request, we cannot reach the home HLR and continue as in (A)

thoughts about persistence technology

Which is easier, storing in a CSV file or in the SQLite database?

• we already have SQLite in osmo-hlr
• add separate table for proxy cache (key=IMSI); another table for N auth tuples per IMSI
• future: proxy cache structural changes will require DB upgrade path
• store:
  • UPDATE last cache entry for IMSI
  • INSERT INTO to add auth tuples
  • garbage collection in auth tuples table: DELETE from db table after N re-uses
  • DB writes for every action
• read:
  • select proxy cache entry for IMSI
  • select N auth tuple results, possible to select by previous usage count, etc.
  • DB reads for every action -- could also keep in RAM and only write to SQLite, and read only if a new IMSI shows up
• scale: scalability handled by SQLite -- we can choose to keep all or "no" records in RAM

• need to manage another file location (new config item / default to current dir)
• store:
  • only append to CSV file.
  • write last cache state, one per line
  • write N tuples in separate lines? (I)
  • write all N remaining tuples with usage count in one line along with cache state? (CSV within a CSV column, one with tab sep and one with comma?) (II)
  • filesys write for every action
  • no protection against corrupted writes
• read:
  • read only on program startup.
  • read and parse all previous lines and keep only the last cache entry item per IMSI.
  • guard against parse errors; mangle strings and so forth
  • in running program, no filesys reads, only RAM state used
• garbage collection:
  • Remove the file manually?
  • regularly keep only unique IMSI entries by cron? make sure to not write at the same time as osmo-hlr attempts to append an entry.
  • On program startup, osmo-hlr could read all entries, keep only the last, and then overwrite the file.
• scale: always require all proxy data in RAM (also entries inactive for a long time?)

(I) N tuples in separate lines

IMSI   MSISDN   last-LU ...  USED RAND SRES KC IK CK AUTN RES
123456 123      2019-12-08        
123456                       2 abcd123 abc123 abc123...
123456                       0 def987 def987 def987...
987654 987      2019-12-09  
987654                       ...
123456 123      2019-12-10  
123456                       3 abcd123 abc123 abc123...

(II) all in one line

IMSI   MSISDN   last-LU ...  TUPLES
123456 123      2019-12-08   2,abcd123,abc123,abc123...;0,def987,def987,def987...;...
987654 987      2019-12-09   ...
123456 123      2019-12-10   3,abcd123,abc123,abc123...;0,def987,def987,def987...;...

conclusion

It is hard to tell which solution would benchmark as more efficient.

At first glance, it seems simpler to not hit the database all the time; but we do that for normal HLR operation anyway.

I would personally favor writing a SQL DB API for a proxy cache instead of adding CSV parsing and string mangling to osmo-hlr.
Especially since we already have all the SQLite tooling in osmo-hlr, it seems odd to not use that for an obviously similar dataset.
Storing and reading from SQLite is already solved and known, CSV parsing would be a new wheel invented with possible surprises / debugging needed on a basic level.

As a gut feel, we should benefit from the SQLite implementation in terms of protection against interrupted writes
and similar basic problems we might have to re-invent protection against for CSV.

My choice would be to go for SQLite.

Since the auth tuple cache #4305 itself is stalled, there is no use in making the HLR proxy cache any data persistently.