Currently, when postmarketOS fails to boot up, retrieving any
information necessary to root cause the issue is non-trivial, requiring
building a custom initramfs with debug-shell enabled and then manually
copying out data.
Let's improve the situation by exporting logs automatically on boot
failure. This is safer than just exposing a root shell but still
provides a whole lot of useful info that should make duplicating and
triaging issues much easier for developers.
This commit implements the functionality and includes some generally
useful data. We then generate a README and expose files for each command/log
as well as an archive that can be easily attached to a GitLab issue.
To help with triaging, also record the version of the postmarketos-initramfs
package and hash the init.sh and init_functions.sh files.
For testing purposes, you can trigger the log recovery mode on-time by
creating an empty file named ".pmos_export_logs" in the /boot partition.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Always write logs to the ramdisk, adjust setup_log() to spawn "tail" to
pipe the log to the console device when PMOS_NO_OUTPUT_REDIRECT is set.
This is necessary to ensure that the initramfs logs can be extracted on
a failed boot.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Logging is important, we want our users to have logging, so make sure we
enable the new logbookd service on existing installation.
We check that we're upgrading from a postmarketos-base version that
predates logbookd and only enable the service in this case. This way we
won't enable it again for folks who disabled it manually.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
[ci:skip-vercheck]
Drop all the non-sdm845 DTBs. This saves about 12MiB
of space in /boot
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
[ci:skip-build]: already built successfully in CI
Add a test for CI that:
* ensures unl0kr runs for 10 seconds without crashing
* ensures that it doesn't print any errors
This sure isn't an "ideal" test, but it at least validates that the
framebuffer interface doesn't die horribly...
With a little more effort we should be able to run unl0kr and then pull
the framebuffer and check it against some known good unl0kr screenshot.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
bootrr is a simple shell script tool for validating that a given board
has booted correctly. It checks that all expected drivers are loaded.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Add a basic test that runs qrtr-lookup and ensures that at least one QMI
service is available. This is mostly a proof of concept and can be
extended in the future.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Enable using the postmarketOS initramfs for boot-testing devices, in
this scenario we don't care about some components like the splash, mdev,
or subpartitions, instead we want to run full udev (to load all devices)
and then run hooks. The ci hook will deadloop after running tests.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Add a helper to generate a post-install script for device pmtest
subpackages. This script configures the console and enables logging in
the initramfs for images running in CI.
[ci:ignore-count]
This package provides a helper for writing a pmtest subpackage, as well
as being a place to write generic tests that aren't specific to a device
(e.g. a suspend test).
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
[ci:ignore-count]
Add a new initramfs hook for running tests in a CI environment.
This hook automatically runs any scripts from /usr/libexec/ci-tests.
Tests should be installed as a subpackage of some other package (e.g.
qrtr) with an install_if clause to install them when this hook is
installed.
This allows platform specific packages like device packages to define
tests that will automatically be installed and executed when building
the initramfs for that device. See
device/community/device-oneplus-enchilada for an example.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Explicitly pull in the blkid package (we already added it to
00-initramfs-base.files), and replace uses of busybox findfs. Full fat
blkid supports PARTLABEL (so the /dev/disk/by-partlabel symlinks will
work consistently now), and has some nice optimisations that make the
lookup times a lot faster than busybox findfs.
[ci:skip-build]: already built successfully in CI
The "pmos_[br]oot(_uuid)?" kernel cmdline args can be used to specify
how to mount the root and boot partitions. During a previous rework, the
behaviour of these was made inconsistent (becoming dependent on the
order they were specified on the cmdline).
Undo the previous "optimisation" by splitting them back out into two
FOR loops, where the _uuid variants take precedence.
Some change (I think some tools switching from the coreutils to busybox
versions?) subtly broke debug-shell behaviour. Try to fix it.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
[ci:skip-build]: already built successfully in CI
With full-fat mdev supporting by-partlabel lookups, there is a small
delay during startup. As mdev and dynamic partitions aren't actually
needed for the framebuffer device, move show_splash earlier.
Furthermore, slightly rework setup_framebuffer to not dump the verbose
message about waiting for the framebuffer unless the framebuffer isn't
found.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
findfs might produce a block device path like /dev/dm-3, these can
change across boots and devices. Make things more readable and
consistent by mapping from the /dev/dm-* path to the equivalent
/dev/mapper/xyz path. Combined with invoking kpartx from a
/dev/disk/by-partlabel/abc symlink, this results in the final block
device having a name like /dev/mapper/userdata2 on an Android device.
Whilst this is just nicer to work with, this will be especially useful for
the upcoming ondev2 postmarketOS installer to make device-specific
configuration and detection easier.
Save the root/boot partition paths once found, and optimise a few loops.
Additionally, fall-back to other search methods when the pmos_boot or
pmos_uuid_boot cmdline args are set but fail.
Co-authored-by: Clayton Craft <clayton@craftyguy.net>
Drop in mdev.conf and the /lib/mdev/persistent-storage script, these
cause /dev/disk/by-* to be populated in the ramdisk, making it
possible to do look up partition by-label and by-partlabel for free,
compared to findfs which can take some time.
As a first optimisation based on this, check for some partitions using
these paths and prioritise them when looking for subpartitions. Drop the
first find_boot_partition call as it doesn't really save time over the
call in the while loop.
The mdev config also handles setting up /dev/null, /dev/random, etc, so
these don't have to be done manually.
Fix a bug where find_dtb() would fail if find encounters any errors,
even if it correctly found the dtb.
Signed-off-by: Caleb Connolly <caleb@connolly.tech>
Since linux 6.4.0 the SDM845 kernel uses compressed kernel modules, we
need to kmod version of modprobe to handle loading these, so depend on
it.
[ci:skip-build] already built
Add support for the Qualcomm Robotics (RB3) dev kit, the Thundercomm
DB845c. This is a successor to the arrow 410c board already supported in
postmarketOS. It is based on the SDA845 SoC, almost identical to SDM845
but without a fully featured modem.
All the things you'd expect to work out of the box:
* UART
* USB host and peripheral
* HDMI display out
* Wifi, Bluetooth
The upcoming db845c doesn't have cellular modem capabilities, but still
has wifi. Avoid installing modem related configs and services by default
and only install it for devices that need it.
The UART console makes the boot process a whole lot slower due to printk
being synchronous. These devices are well enough supported that we can
disable UART logging by default to improve the boot speed. This
constitutes a ~30 second speedup on the SHIFT6mq at 115200 baud.
If pbsplash is running when a display manager starts it often causes
issues as display managers want to run on a particular VT, but are
unable to switch the active VT since pbsplash has exclusive control of
the framebuffer.
Making DMs not start until after local would solve the issue but
it would also needlessly delay the boot process.
Until we have a better way to detect display managers starting and react
to it, this will at least get devices working again. Though due to how
openrc resolves service start order it may mean the splash is killed
earlier than necessary.
This modifies to the postmarketOS ramdisk init to use the new pbsplash
tool for displaying splash messages, replacing the old fbsplash.
This also moves the show_splash call to run as soon as possible rather
than waiting for mount_subpartitions to run (which can take a while).
mesa-egl is usually installed by dependencies of ui packages, but when
choosing the "none" ui option in pmbootstrap, it isn't selected, leading
to libEGL.so.1 not being found.
Add it as an explicit dependency.
[ci:skip-build]: already built successfully in CI
When moving the code to it's own repo I changed the license to GPL-2.0,
I forgot to update the APKBUILD.
[ci:skip-build] already built successfully in CI
This upgrades ttyescape to use buffyboard, a new fbkeyboard replacement
by @Cherrypicker. It adds a lot of missing keys and uses the same layout
as Phosh's Squeekboard, so should feel much more like home for Phosh
users.
This also switches to the new HKDM daemon for hotkey handling, replacing
triggerhappy.
[ci:skip-build] already passed
Finally a new upgrade! This brings along some minor driver changes and
improvements, more stuff slowly landing upstream.
We also get a few new splats during boot, these are inevitable at this
point due to fw_devlink changes and large reworking of the drm/msm code,
they are caused by the common clock framework not knowing that some
clocks are left enabled by the bootloader, it panics when they aren't in
a state it expects. They are harmless as far as I know.
[ci:skip-build]
This fix was pending since before charging-sdl was deprecated and still
provides value should anyone wish to pickup and improve charging-sdl
again.
charging-sdl currently does not check for devices using mesa, even
though it supports it in the same way as osk-sdl. Check for msa support
and set SDL_VIDEODRIVER in that case.
Also fix the font path, as it was reading both keyboard-font and
keyboard-font-size from osk.conf.
The delay to wait for the SIM card to show up appears to be unneeded on
our modems, remove it so that we don't delay boot by 10 seconds if your
device doesn't have a SIM card installed.
[ci:skip-build] already built successfully in CI
The flavor string was naively parsed using sed by removing everything
before the first hyphen, this breaks for -rc kernels. Instead lets glob
/usr/share/kernel for the installed kernel flavor. This will also
prevent the kernel being flashed if the running kernel has a different
localversion than the updated one.
[ci:skip-build] already built successfully in CI
Add ttyescape, a script and triggerhappy configuration to allow mobile device
users to access and use a shell without having to plug in to a computer.
One of the largest limitations with a mobile device is the lack of keyboard, for
mainstream OSs like Android and iOS, this is a non-issue as the whole OS stack
is built to automatically recover in case of a crash / hang, hiding the internal
state of affairs from users and making use of careful design to minimise the
impact. When bringing Linux to mobile, we carry not only the benefits of the
Linux desktop but also it's limitations. In the event that your desktop manager
goes haywire or hangs completely, or your graphics drivers get unhappy, the
ability to quickly jump to a tty and start killing bad behaving programs or
reset your display manager is one that most of us take for granted. But when
hit by similar errors on a mobile device there is no such recourse available,
users either have to reboot and hope that the issue doesn't occur again, or pull
out a laptop and pull up a shell (assuming ssh is enabled and the rndis
interface comes up).
ttyescape proposes to solve this issues by pieceing together several already
available tools, notably:
- triggerhappy, a tool used to perform actions when
certain buttons or key combinations are pressed with no dependencies on the
display manager in use.
- fbkeyboard, a framebuffer keyboard for tty's, it
renders on top of the current tty and uses the device touchscreen as input.
To prevent accidental bricks, postmarketos-update-kernel is not
installed by default. If it works for your device and you've added the
deviceinfo variable, you also need to depend on the package for the
updates to actually happen.
[ci:skip-build] already built successfully in CI
On Android devices that support kernel updates (marked by the deviceinfo
variable "deviceinfo_flash_kernel_on_update") lets flash the kernel and
initramfs automatically!
Caleb Connolly