Bug #5260
closedbootloader goes beyond partition size in modern gcc
100%
Description
As we noticed several times now, gcc is becoming more and more inefficient in building our DFU loader, resulting in it no longer fitting the partition on modern compilers.
I think switching to a larger partition is not really an option as it would create a compatibility nightmare with older boards and a lot of fall-out in our user community.
So if we really cannot shrink the bootloader further, then we will have to settle on an old gcc version and create our official DFU loader builds from a specific build slave (or maybe rather docker container).
btw: what about Hoernchen's favorite llvm/clang? Does it generate smaller code for this bare-iron bootloader use case?
Hoernchen, please specify which exact environment shall be built.
osmith can then (After his holidays next week) work on changing our jenkins setup so that the existing simtrace2 firmware job just builds the applications, and create a new job with a docker image containing the specific gcc version required to build all the DFU loaders.
Checklist
- report this size regression to upstream gcc (Hoernchen)
- possibly check llvm/clang? (Hoernchen)
- specify which compiler version on which distro to use (Hoernchen)
- jenkins/docker/... stuff to build DFU loaders from simtrace2.git in that specific environment (osmith)
Updated by Hoernchen over 2 years ago
I actually gave clang a try, there is https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm which conveniently builds clang and compiler-rt and newlib. Of course any distro provided clang package would work as well for compiling, but the issue is that we need a stdlib and compiler runtime as well, so just running that script is the most convenient way to get all of that (until oom, pass --parallel=x). Still needs binutils, but i just used my existing arm gcc package for that to give it a try.
The -Os binaries are about as large as the gcc versions
16324 Okt 13 19:45 firmware/bin/ngff_cardem-dfu-flash.bin 15484 Okt 13 19:45 firmware/bin/qmod-dfu-flash.bin 15104 Okt 13 19:45 firmware/bin/simtrace-dfu-flash.bin
but clang does have a -Oz that offers smaller code:
14704 Okt 13 19:50 firmware/bin/ngff_cardem-dfu-flash.bin 13996 Okt 13 19:50 firmware/bin/qmod-dfu-flash.bin 13612 Okt 13 19:50 firmware/bin/simtrace-dfu-flash.bin
This is significantly smaller than what gcc can do.. We could also try bitcode-level LTO.
The gcc last toolchain I used that "just works" was gcc-arm-none-eabi-9-2020-q2-update - tho I do remember having size issues some time last year with a slightly older one..
Updated by Hoernchen over 2 years ago
- Checklist item possibly check llvm/clang? (Hoernchen) set to Done
- Checklist item specify which compiler version on which distro to use (Hoernchen) set to Done
Clang support in https://gerrit.osmocom.org/c/simtrace2/+/25849 I'd propose to just build the firmware with clang, and that's it.
As for reporting, I don't really know what to report - after all at the time having a bootloader that barely was able to fit into 16k was fine, but continuously rebuilding and modifying it and expecting the optimizer performance to keep the size without adding a few bytes here and there was not really a reasonable assumption.
Updated by osmith over 2 years ago
- Status changed from New to Stalled
- % Done changed from 0 to 50
Hoernchen wrote in #note-1:
I actually gave clang a try, there is https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm which conveniently builds clang and compiler-rt and newlib. Of course any distro provided clang package would work as well for compiling, but the issue is that we need a stdlib and compiler runtime as well, so just running that script is the most convenient way to get all of that (until oom, pass --parallel=x). Still needs binutils, but i just used my existing arm gcc package for that to give it a try.
I've attempted to build llvm with the linked scripts, however building took a very long time. I didn't measure the time unfortunately, but I think I let it run for more than an hour before I aborted it. Unless we really want to build it from source, I suggest we use binary packages instead.
Apparently there will be official binary packages from the project available in coming days, so I'll wait for them: https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm/issues/79
Updated by osmith over 1 year ago
- Checklist item jenkins/docker/... stuff to build DFU loaders from simtrace2.git in that specific environment (osmith) set to Done
- Status changed from Stalled to In Progress
- % Done changed from 50 to 90
These patches add the binary release of LLVM-embedded-toolchain-for-Arm to the docker image we use in jenkins, and enables USE_CLANG=1:
Updated by osmith over 1 year ago
- Status changed from In Progress to Resolved
- % Done changed from 90 to 100
Patch is merged, marking as resolved.
The "report this size regression to upstream gcc" checkbox is still unchecked, but this was commented on in #note-2.
