Crossdirect redirects archivers (gz, bzip2, ...) to their native
versions when cross compiling. This works e.g. when extracting sources or
packing the .apks at the end of the build. But it currently fails when
such an archiver gets called during package().
For example with bootchart2:
>>> bootchart2: Entering fakeroot...
...
gzip -c bootchart2.1 > /home/pmos/build/pkg/bootchart2/usr/share/man/man1/bootchart2.1.gz
Error relocating /native/usr/lib/libfakeroot.so: unsupported relocation type 6
Error relocating /native/usr/lib/libfakeroot.so: unsupported relocation type 6
Error relocating /native/usr/lib/libfakeroot.so: unsupported relocation type 1
Error relocating /native/usr/lib/libfakeroot.so: unsupported relocation type 6
Fix this by running the foreign arch version when it gets called in a
fakeroot.
Fixes: 8a7c9ce7 ("cross/crossdirect: run archivers natively too")
Part-of: <https://gitlab.postmarketos.org/postmarketOS/pmaports/-/merge_requests/7120>
Run native binaries for extracting / packing various archives if they
are installed in the native chroot when using the crossdirect
compilation method, instead of having them go through QEMU. The
resulting extracted/packed files should always be the same but this is
significantly faster.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmaports/-/merge_requests/6763
Alpine's abuild runs build() without fakeroot, and package() with
fakeroot. From the APKBUILD reference page in the Alpine wiki:
> Note: Building in fakeroot will reduce performance for parallel
> builds dramatically. It is for this reason that we split the build
> and package process into two separate functions.
Every now and then we see a package that tries to run a compiler (gcc,
g++, clang, ...) during package() in the APKBUILD. This is a bug in the
APKBUILD / build system of the program we are packaging. All compiling
should be done during build().
Let crossdirect print the following when this happens:
============================================================================================
ERROR: crossdirect was called with: LD_PRELOAD=libfakeroot.so
This means your package tried to run a compiler during package().
This is not supported by crossdirect, and usually not a good idea.
* Try to fix your APKBUILD so it does not run the compiler during package(), only in build()
* If you're using meson install, try to add '--no-rebuild'
* If this is not possible, you can work around it by setting options="!pmb:crossdirect"
(compilation will be slower!)
============================================================================================
Instead of:
ERROR: crossdirect: can't handle LD_PRELOAD: libfakeroot.so
Please report this at: https://gitlab.com/postmarketOS/pmaports/issues
As a workaround, you can compile without crossdirect.
In the past I've also tried to add 'strcmp(ldPreload, "libfakeroot.so") == 0'
(pmaports MR 2231), and today I made the same patch. The current code
looks like it would then work by just using libfakeroot.so from the
native chroot, but it does not work. And as mentioned, if we hit this
then we are compiling in package() which is something we should not do
anyway!
Fixes: pmb issue 2039
Related: pma issue 2351, pma MR 5738
Wrap rustc commands with sccache when building native build scripts.
More interesting will be using sscache for compiling for the target
architecture, but sccache currently refuses to cache with --sysroot.
This can be optimized further later on, get some initial support for
sccache in here first.
While at it, tweak the -vV logic to also support -V, start using semver
with this and add myself as maintainer of the package.
[ci:skip-build]: already built successfully in CI
Add a cargo wrapper which appends a --target argument to the command
line. This makes cargo pass the --target argument to rustc for crates
being built for the target architecture, even if the target is the same
as the host. It will omit the --target argument for build scripts and
crates used in macros.
Check for this --target argument in the rustc wrapper and adjust the
library paths depending on its presence. The fallback that runs rustc
under qemu is no longer needed because macros are now built for the
native architecture and can be loaded into the native compiler without
any problems.
Also check if the arguments passed to rustc are "-vV". If this is the
case, we still need to fall back to the target rustc because the native
rustc will return the wrong architecture. If the wrong host architecture
is passed to a build script, it might try to look for a cross-compiler
or do something else that doesn't work.
[ci:skip-build]: already built successfully in CI
Use the native rustc cross compiler where possible, and fall back to the
qemu rustc if necessary. This should improve speed and reliability.
Without this patch, building squeekbox 1.4.0 for aarch64 with the
pmbootstrap stack just hangs forever.
Related: pmbootstrap#1861
Rebuild to make it use the new armhf triplet. Otherwise cross compiling
with crossdirect to armhf fails with:
ERROR: crossdirect: failed to execute /native/usr/lib/ccache/bin/armv6-alpine-linux-muslgnueabihf-gcc: No such file or directory
This finally fixes#295.
Cross linker (e.g. ld from binutils-armhf) does not support any method
of setting additional link paths, and this causes problems when building
various packages. Work around this problem by calling the qemu binary
instead of the cross binary from the native chroot, whenever we are
linking.
I have tested that this allows successfully building hello-world,
coreinfo, libhybris for armhf. So this fixes#227 and fixes#228.
Regarding performance, this is the same way as distcc does it. We would
get a performance gain if we were able to use the cross-linker. But so
far I could not figure out how to patch the binutils source to make it
work like we want to have it. Maybe I'll ask on some binutils mailing
list for advice. In the meantime, this workaround should make
crossdirect work for all use-cases, and it is still faster than the
previous distcc setup, because we can avoid the overhead of setting up
and running a ssh server locally to work around missing authentication
in distcc (see pmbootstrap!1649). Let's test this for some more time,
and then make a pmbootstrap release that rolls out crossdirect for
everyone.
Instead of running compilers directly, let them go through the native
ccache binary. Note that it will still use the foreign arches cache dir
because the native cache dir would be at /native/home/pmos/.ccache.
Use execve instead of setenv and execv, because that didn't work with
LD_LIBRARY_PATH (ccache was complaining that it couldn't load zlib).
Enable -Werror, because we are maintaining this program and it helps
us catching bugs early.
A set of wrapper executables that launch native cross compilers inside
foreign chroots. Speeds up cross compilation a lot, compared to the
distcc-sshd method.
Oliver Smith