Project

General

Profile

Actions

Bug #6269

closed

osmo-trx-uhd fails to start on Debian 12 system

Added by laforge 5 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
11/22/2023
Due date:
% Done:

100%

Spec Reference:

Description

I'm currently setting up an osmocom system using the nightly feed for Debian 12 (on what I assume to be a clean Debian 12).

However, the osmo-trx-uhd.service is not starting up:

Nov 22 15:15:10 nuc-osmocom2 systemd[1]: Started osmo-trx-uhd.service - Osmocom SDR BTS L1 Transceiver (UHD Backend).
Nov 22 15:15:10 nuc-osmocom2 osmo-trx-uhd[3503]: Wed Nov 22 15:15:10 2023 DLGLOBAL <0008> cpu_sched_vty.c:471 Setting SCHED_RR priority 18
Nov 22 15:15:10 nuc-osmocom2 osmo-trx-uhd[3503]: Wed Nov 22 15:15:10 2023 DLGLOBAL <0008> telnet_interface.c:88 Available via telnet 127.0.0.1 4237
Nov 22 15:15:10 nuc-osmocom2 osmo-trx-uhd[3503]: Wed Nov 22 15:15:10 2023 DLCTRL <000f> control_if.c:1014 CTRL at 127.0.0.1 4236
Nov 22 15:15:10 nuc-osmocom2 osmo-trx-uhd[3503]: [INFO] [UHD] linux; GNU C++ version 12.2.0; Boost_107400; UHD_4.3.0.0+ds1-5
Nov 22 15:15:12 nuc-osmocom2 osmo-trx-uhd[3503]: Wed Nov 22 15:15:12 2023 DDEV <0005> UHDDevice.cpp:525 UHD make failed, device , exception:
Nov 22 15:15:12 nuc-osmocom2 osmo-trx-uhd[3503]: RuntimeError: get_xdg_config_home(): Unable to find $HOME or $XDG_CONFIG_HOME.
Nov 22 15:15:12 nuc-osmocom2 osmo-trx-uhd[3503]: Wed Nov 22 15:15:12 2023 DMAIN <0000> osmo-trx.cpp:607 Failed to create radio device
Nov 22 15:15:12 nuc-osmocom2 osmo-trx-uhd[3503]: Wed Nov 22 15:15:12 2023 DMAIN <0000> osmo-trx.cpp:588 Shutting down transceiver...
Nov 22 15:15:12 nuc-osmocom2 systemd[1]: osmo-trx-uhd.service: Main process exited, code=exited, status=1/FAILURE
Nov 22 15:15:12 nuc-osmocom2 systemd[1]: osmo-trx-uhd.service: Failed with result 'exit-code'.

I'm very puzzled. Does this really mean that UHD expects HOME or XDG_CONFIG_HOME environment variables being present? Have they ever considered that this is not usual for any program not started by an user at an interactive console? And why on earth would something minor like that result in a fatal failure?

Does anyone know anything about this?


Related issues

Related to Cellular Network Infrastructure - Bug #6272: default config files have different log format config for stderrResolvedjolly11/24/2023

Actions
Actions #1

Updated by Hoernchen 5 months ago

yes. and you will probably have to set a path for the fpga images, too.

Actions #2

Updated by laforge 5 months ago

Hoernchen wrote in #note-1:

yes. and you will probably have to set a path for the fpga images, too.

It's amazing how much Ettus/NI seem to be intent on making sure they loose their brand image as former industry standard of FPGA drivers. What a shame.

Actions #3

Updated by laforge 5 months ago

  • Assignee set to osmith

But to get more on-topic here: If this is required by UHD these days, then we need to adjust our systemd unit accordingly to make sure it has whatever environment variables it deems necessary.

I really cannot get my head around why one would intentionally break downstream applications and end-users alike with those kind of changes. Of what use is it if the API for drivers is stable, if you suddenly have to do all kinds of other settings in order to start your program after linking a new version of UHD.

Actions #4

Updated by laforge 5 months ago

the patch introducing the method uhd::get_xdg_data_home() is:

commit 1383fde3457168ed759d6ed3b913dfae8a6085ef
Author:     Martin Braun <martin.braun@ettus.com>
AuthorDate: Thu Mar 5 16:23:07 2020 -0800
Commit:     Aaron Rossetto <aaron.rossetto@ni.com>
CommitDate: Thu Apr 2 12:36:20 2020 -0500

which was first shipped in uhd-v4.0.0.0

Before the commit UHD_CONFIG_DIR/APPDATA/HOME environment variables were used if present, but not mandatory, as uhd::get_app_path() would simply have returned return uhd::get_tmp_path(); by default.

And of course the commit log only talks about moving around files and/or variable names and doesn't state at all that suddenly there is a significant change in policy, i.e. that those variables are now mandatory and not optional anymore.

Actions #5

Updated by funkylab 5 months ago

bit of insight might help: absolutely sure no one at Ettus intentionally breaks things¹ ;-)
There's been an UHD config file for a long time, there's good reason to put it into XDG directories if possible, and I guess the fact that this would fatally fail if these environment variables aren't set and that this can totally happen in the wild just got overlooked; the docs² contradict this behaviour. Want me to put in an upstream bug report?

If this is required by UHD these days, then we need to adjust our systemd unit accordingly to make sure it has whatever environment variables it deems necessary.

Honestly, setting the firmware image directory explicitly is actually a good idea should testing need to work with different UHD versions; being able to have a test-specific config makes sense to me, as well.

Bit surprised that $HOME isn't set in a systemd service!

¹ There might or might not be stories of people throwing E310 prototypes off roofs or driving over them with heavy cars, but I don't think that's the kind of attempted breakage you're referring to!
² https://uhd.readthedocs.io/en/uhd-4.5/page_configfiles.html#configfiles_location

Actions #6

Updated by laforge 5 months ago

On Wed, Nov 22, 2023 at 03:31:09PM +0000, funkylab wrote:

There's been an UHD config file for a long time, there's good reason to put it into XDG directories if possible,

yes, certainly. I don't doubt that.

and I guess the fact that this would fatally fail if these environment variables aren't set and that this can totally happen in the wild just got overlooked; the docs² contradict this behaviour. Want me to put in an upstream bug report?

I actually did check the upstream github reports and didn't find any mention of it whatsoever. I presume it's not uncommon for people in production deployments to start their osmo-trx-uhd / yatebts / openbts / srsran / amarisoft / ... cellular base station (or other SDR application) via a systemd unit.

If you think it is a bug, feel free to file it upstream. I can also do that, of course, but I'm less familiar with what is intended.

If this is required by UHD these days, then we need to adjust our systemd unit accordingly to make sure it has whatever environment variables it deems necessary.

Honestly, setting the firmware image directory explicitly is actually a good idea should testing need to work with different UHD versions; being able to have a test-specific config makes sense to me, as well.

Note that I did not have to set a firmware path explicitly, but that might have been because I already loaded the firmware with another invoacation before starting osmo-trx-uhd.

In any case, this is not about testing. This is about production use. Its reasonable to expect that you can install a software package via apt, edit the config file as needed and then simply enable the packaged systemd service.

I'd also not understand why setting a firmware path would be required explicitly. Isn't there one valid
firmware image for one valid piece of hardware (auto-detected) and UHD version (determined by the single UHD
version installed from the distribution pacakge manager)?

Bit surprised that $HOME isn't set in a systemd service!

I'm far from being a systemd expert, but I guess it might if you'd run the service as non-root.
Sadly #4107 is still not completed. I'll do some more experiments.

Actions #7

Updated by funkylab 5 months ago

:) Just submitting a report upstream then, since I can locally reproduce, anyways.

I'd also not understand why setting a firmware path would be required explicitly .

That certainly shouldn't be the case. It should (and does, as far as I can reproduce) default to /usr/share/uhd/images/.

Isn't there one valid firmware image for one valid piece of hardware (auto-detected) and UHD version (determined by the single UHD version installed from the distribution pacakge manager)?

Ah, not quite. The X3x0 series (and newer dual Q?SFP models) come with different FPGA image "flavors" depending on which combination of 1G/10G/… interfaces you want to operate.
Also, let's call them "3-series+" models (so everything after the B200, essentially) all support RFNoC, where the user builds their own FPGA image with their own compute (selection of stock, or self-written) elements next to the fixed, necessary infrastructure, into the FPGA; they all get connected to a central crossbar, and then you can use host-side software to say things like "take the samples coming in packets from the radio frontend, push them through the custom preamble detector, …

That requires that aside from the "default" images that basically make the USRP an RF soundcard on speed there's a need to be able to specify your own images.

Actions #8

Updated by laforge 5 months ago

Thanks!

On Wed, Nov 22, 2023 at 05:11:07PM +0000, funkylab wrote:

I'd also not understand why setting a firmware path would be required explicitly .

That certainly shouldn't be the case. It should (and does, as far as I can reproduce) default to /usr/share/uhd/images/.

ok, we're all in agreement. for sure it should always be possible to specify a different image as override,
we can all agree on that.

Actions #9

Updated by laforge 5 months ago

  • Related to Bug #6272: default config files have different log format config for stderr added
Actions #10

Updated by laforge 5 months ago

  • Status changed from New to In Progress
  • Assignee changed from osmith to laforge
  • % Done changed from 0 to 80
Actions #11

Updated by laforge 5 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 80 to 100
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)