Project

General

Profile

Actions

Bug #5578

open

osmo-remsim-client-st2 hangs after usb-reset without power loss on qmod

Added by roh 17 days ago. Updated 11 days ago.

Status:
New
Priority:
Normal
Assignee:
Category:
remsim-client
Target version:
-
Start date:
06/15/2022
Due date:
% Done:

0%

Spec Reference:

Description

in case of a osmo-remsim-client-st2 session which gets restarted after being stopped by a usb-disconnect
(terminates with "DST2 FATAL user_simtrace2.c:221 USB IN transfer failed, status=1" as expected)

osmo-remsim-client-st2 gets stuck and then terminates after some time:

root@raspberrypi:~# /usr/bin/osmo-remsim-client-st2 -i 10.15.1.135 -V 1d50 -P 4004 -C 1 -I 0 -H 1-1.3.1 -c 1 -n 0
DRSPRO INFO ../rspro_client_fsm.c:307 RSPRO_CLIENT(server){REESTABLISH}: Creating TCP connection to server at 10.15.1.135:9998
DLINP NOTICE simtrace2_api.c:267 [0] <= osmo_st2_cardem_request_config(features=00000001)
DLINP NOTICE input/ipa.c:128 10.15.1.135:9998 connection done
DRSPRO NOTICE ../rspro_client_fsm.c:125 RSPRO_CLIENT(server){REESTABLISH}: RSPRO link to 10.15.1.135:9998 UP
DRSPRO INFO ../rspro_client_fsm.c:307 RSPRO_CLIENT(bankd){REESTABLISH}: Creating TCP connection to server at 10.15.1.135:9999
DLINP NOTICE input/ipa.c:128 10.15.1.135:9999 connection done
DRSPRO NOTICE ../rspro_client_fsm.c:125 RSPRO_CLIENT(bankd){REESTABLISH}: RSPRO link to 10.15.1.135:9999 UP
DLINP NOTICE simtrace2_api.c:168 [0] <= osmo_st2_cardem_request_card_insert(inserted=1)
DLINP NOTICE simtrace2_api.c:317 [0] <= _modem_sim_select(remote_sim=1)
DLINP NOTICE simtrace2_api.c:250 [0] <= osmo_st2_cardem_request_set_atr(3b 00 )
DLINP NOTICE simtrace2_api.c:285 [0] <= _modem_reset(asserted=2, pulse_ms=300)
DLINP NOTICE simtrace2_api.c:250 [0] <= osmo_st2_cardem_request_set_atr(3b 9f 96 80 1f 87 80 31 e0 73 fe 21 1b 67 4a 4c 75 30 34 05 4b a9 )
DST2 INFO user_simtrace2.c:65 SIMtrace => PTS req: ff 10 96 79 00 00 
...
<modem starts, usb enumeration of serials/wwlan>
...
USB OUT transfer failed, status=2

later restarts remain the same in behaviour but a bit different in output:

root@raspberrypi:~# /usr/bin/osmo-remsim-client-st2 -i 10.15.1.135 -V 1d50 -P 4004 -C 1 -I 0 -H 1-1.3.1 -c 1 -n 0
DRSPRO INFO ../rspro_client_fsm.c:307 RSPRO_CLIENT(server){REESTABLISH}: Creating TCP connection to server at 10.15.1.135:9998
DLINP NOTICE simtrace2_api.c:267 [0] <= osmo_st2_cardem_request_config(features=00000001)
DLINP NOTICE input/ipa.c:128 10.15.1.135:9998 connection done
DRSPRO NOTICE ../rspro_client_fsm.c:125 RSPRO_CLIENT(server){REESTABLISH}: RSPRO link to 10.15.1.135:9998 UP
DRSPRO INFO ../rspro_client_fsm.c:307 RSPRO_CLIENT(bankd){REESTABLISH}: Creating TCP connection to server at 10.15.1.135:9999
DLINP NOTICE input/ipa.c:128 10.15.1.135:9999 connection done
DRSPRO NOTICE ../rspro_client_fsm.c:125 RSPRO_CLIENT(bankd){REESTABLISH}: RSPRO link to 10.15.1.135:9999 UP
DLINP NOTICE simtrace2_api.c:168 [0] <= osmo_st2_cardem_request_card_insert(inserted=1)
DLINP NOTICE simtrace2_api.c:317 [0] <= _modem_sim_select(remote_sim=1)
DLINP NOTICE simtrace2_api.c:250 [0] <= osmo_st2_cardem_request_set_atr(3b 00 )
DLINP NOTICE simtrace2_api.c:285 [0] <= _modem_reset(asserted=2, pulse_ms=300)
DLINP NOTICE simtrace2_api.c:250 [0] <= osmo_st2_cardem_request_set_atr(3b 9f 96 80 1f 87 80 31 e0 73 fe 21 1b 67 4a 4c 75 30 34 05 4b a9 )
USB OUT transfer failed, status=2

please note that the modem does not seem to reboot on later restarts of osmo-remsim-client-st2, which is odd.

osmo-remsim-client version 1.0.0.18-f5f5
qmod firmware 0.8.1.33-9088

osmo-remsim-bankd version 1.0.0.18-f5f5
osmo-resmim-server version 1.0.0.18-f5f5

maybe this is a rerun of #4409

for this test the server/bankd were on x86_64 and the client was on armv7l (rpi) but i don't think it matters.
i have also tried a older qmod firmware (0.7.0.57-f46d) for comparison but i can see no difference in behavior.

please note that i could not reproduce this issue with a modem on st34, only on st12. i suspect it has to do with st12 staying powered all the time and not reset/rebooted on usb-connect properly somehow.

this may as well be a firmware issue on the qmod.

Actions #1

Updated by roh 17 days ago

further details:
  • testmodem is a ec20 in slot1 of 4 on the qmod which is powered externally with 12V 2A (enough for one modem)
  • mapping of osmo-remsim-server is slot 0 of bank1 to slot0 on client1
  • ccid reader is sysmooctsim with a card in slot0 - but i don't think this matters
  • packages from -nightly feeds
  • platform does not matter (arm/x86)
  • distribution does not matter (all 3 services can run on the same host)
  • only both slots of st12 show this behavior, st34 is not affected - i have tested all 4 slots individually
Actions #2

Updated by laforge 15 days ago

  • Category set to remsim-client
  • Assignee set to laforge

do we have the serial console output during the time this behavior is observed? I think it's more likely to be a simtrace2/cardem firmware problem, than a osmo-remsim issue...

Actions #3

Updated by roh 11 days ago

just tried to redo the test with a serial cable connected to provide a log, but for some reason i can not replicate the behavior anymore. tried again without the serial cable and its still stable.

i also suspect the issue to be in the firmware, due to the differences in what gets reset when, but i need to get it to show again, first.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)