Cellular Network Infrastructure » Radio Access Network » OsmoBTS

Contrary to my initial expectation, it's unrelated to rach11 patch.
Looking at osmo-bts.log:

last known-good build:
pcu_sock.c:480 Data request received: sapi=AGCH arfcn=0 block=0 data=00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16

failing build:
pcu_sock.c:480 Data request received: sapi=AGCH arfcn=0 block=0 data=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Related diff:

last known-good build:
DL1C <0006> l1sap.c:623 activate confirm chan_nr=UNKNOWN on TS7 trx=0
failing build:
DL1C <0006> l1sap.c:623 activate confirm chan_nr=PDCH on TS7 trx=0

Most likely cause is one of the patches merged to osmo-ttcn3-hacks around Feb. 19 because the issue appeared at the same time for both ttcn3-bts-test-latest and ttcn3-bts-test.

Note to self - handy commands:

Remove first column with timestamp:
cut -d' ' -f2- osmo-bts.log > osmo-bts.log.base

Cleanup logs from random jitter noise:
grep -v 'TRX Clock Ind: elapsed_us' osmo-bts.log.base | grep -v 'Clock indication: fn' | grep -v 'FN timer expire_count' | grep -v 'GSM clock jitter' | grep -v 'FN slower than TRX' > osmo-bts.no-jitter.base

Reminder: this is an Urgent issue and it pre-empts any other takss

msuraev wrote:

Most likely cause is one of the patches merged to osmo-ttcn3-hacks around Feb. 19 because the issue appeared at the same time for both ttcn3-bts-test-latest and ttcn3-bts-test.

That could be:

commit 07e8dde671b0cf22842d1fb1cec2ed78f3f7bf0f
Author:     Harald Welte <laforge@gnumonks.org>
AuthorDate: Mon Feb 18 20:38:45 2019 +0100
Commit:     Harald Welte <laforge@gnumonks.org>
CommitDate: Mon Feb 18 20:42:16 2019 +0100

    PCUIF: Use OCT4 for IP address, not uint32_t

    In TTCN-3, a 4-byte octetstring is the more usual representation for
    IP addresses, not an integer type.  This is also what functions like
    f_inet_addr() etc. are using as types, and we may want to use them
    in combination with the PCUIF.

    Change-Id: Ia08e1bb8a9bfbd5bf5b63922c77bb221ce1a12f5

or

commit e1fd916b2110e0282f3d9320958c27876b7d91e1
Author:     Harald Welte <laforge@gnumonks.org>
AuthorDate: Mon Feb 18 19:47:53 2019 +0100
Commit:     Harald Welte <laforge@gnumonks.org>
CommitDate: Mon Feb 18 20:25:00 2019 +0100

    PCUIF: Prepare for simulating BTS side of PCU Interface

    Our TTCN-3 PCUIF code so far was only used to simulate the PCU side
    of the interface: connecting to the socket as a client.  However,
    it's also useful to emulate the BTS side of the interface: Listening
    for a connection as a server.

    Also, the send/receive templates are prepared for the inverse role.

    Change-Id: I779ff2903cab8c13ffb8fe10a4cacd996bafe69a

or

commit 9fbcf4b01e1e1a5b5316eab348d5023871101f9e
Author:     Harald Welte <laforge@gnumonks.org>
AuthorDate: Fri Dec 7 07:56:52 2018 +0100
Commit:     Harald Welte <laforge@gnumonks.org>
CommitDate: Thu Feb 21 11:53:42 2019 +0100

    PCU: Tests for the BSS/PCU side NS/BSSGP implementations

    The existing (unused) PCU_Tests are operating on top of a NS + BSSGP
    emulation, i.e. they're aimed at testing higher protocol layers.  Also,
    they required BTS+BSC to run next to the PCU.

    The new PCU_Tests_RAW introduced in this patch are the exact opposite:
    * they test the PCU alone (attach to PCUIF and Gb interface)
    * they don't require BTS or BSC to run
    * they don't use NS + BSSGP emulation but raw NS/BSSGP frames to
      test the very NS/BSSGP implementation inside of OsmoPCU.

    Change-Id: I7ad76b96974cf0a686ad0f00ccd09d1a9df8b4d5
    Related: OS#2890

Please see: https://gerrit.osmocom.org/#/c/osmo-ttcn3-hacks/+/14073/.

Hmm, for some reason both TC_pcu_data_req_agch and TC_pcu_data_req_imm_ass_pch test cases are still failing on Jenkins. At the same time, both pass on my local host. Let's investigate why.

I will try to build Docker images ('ttcn3-bts-test') and run both test cases there.

I will try to build Docker images ('ttcn3-bts-test') and run both test cases there.

Both test cases pass in my local replication of 'ttcn3-bts-test' Docker image.
It seems Jenkins is still building / using an old version of osmo-ttcn-hacks?

do you have any evidence pointing to that?

Well, the evidence is already in your message:

[0m [91mAlready on 'master'
[0mYour branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
master
a2d59c6e6efa1fa979aa5e1cd61a9eb21a775cb6

Right now while I am writing this message, 'origin/master' points to c78ea26f219ff4d8b0a8fce08a17168dedf2f157 "library/PCUIF_Types.ttcn: add explicit ALIGN attribute", while a2d59c6e6efa1fa979aa5e1cd61a9eb21a775cb6 from the mentioned log is "add three tests for CIPHER MODE COMPLETE without algo".

The containers should be rebuilt automatically at the start of each test suite execution, if there were any changes in osmo-ttcn3-hacks.git.

As we can see, there was at least one change, but Jenkins is still building an older version. This is not the first time I see this odd behaviour: after fixing both BTS_Tests.TC_rach_content and BTS_Tests.TC_rach_count in Ibb6d27c6589965c8b59a6d2598a7c43fd860f284, it took a few days until they actually become green.

If I am not missing anything and my understanding is correct, we need to create a separate issue for that.

The test case is green since build #507. Closing.