Refactor the install code to be generic, so we only need to add the new
path in one place when adding a new file. Put the full path into the
filename to make this possible.
I've tried to mimic the final directory structure in the aport first
(e.g. $pkgdir/sbin/swapfile -> main/postmarketos-base/sbin/swapfile),
but that leads to conflicts as abuild only takes the filename for
checksums (conflict with etc/conf.d/swapfile):
https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10013
We don't really need checksum verification for the files shipped in the
same directory (not downloaded), but nevertheless this is a bug. It
leads to confusing behavior and might be fixed by letting abuild demand
that there is only one source file with the same name, as I suggested in
the issue linked above. So let's avoid this altogether with the flat
file name.
postmarketos-base will soon build its source="" variable with a shell
function. Support this use case by not trying to parse source="" with
pmbootstrap anymore, but rather parsing the trivial sha512sums
variable.
Don't print a message for each APKBUILD that is getting parsed. pytest
will display the stdout below the raised exception, so with the print
one had to scroll up a lot to find the actual error message. Now it is
printed at the bottom.
When using '[ci:skip-build]' without any changed packages, do not
attempt to run "pmbootstrap checksum" without any package arguments.
This may happen, because CI in MRs runs across all patches in the MR,
but CI on branches (master, v20.05) only run on the last commit.
Prevents CI failures like this one:
https://gitlab.com/postmarketOS/pmaports/-/commit/9a48b41a2362ba0dd2b0901cae966011b37d1856
Split the aports-build job into arch-specific build-x86_64,
build-aarch64, ... jobs. Extend build_changed_aports.py to accept the
architecture as argument, and to build all packages for that arch where
possible.
Import and use pmbootstrap code for APKBUILD parsing etc, as we do this
already in various testcases running in this repository.
Co-Authored-By: Bart Ribbers <bribbers@disroot.org>
Verify _pmb_recommends for architectures listed in the APKBUILD of
postmarketos-ui-* packages, not from their binary packages. Otherwise
disabling an architecture in a merge request will not be respected, as
the binary package still has the arch enabled.
Libhybris is failing to build for armhf too. Instead of adding !armhf
everywhere, let's just remove it: as it became clear in #703, nobody is
willing to put in the effort to maintain this properly.
If you would like to continue with libhybris on postmarketOS, please
create your own aports repository and put the aports there.
Related: https://wiki.postmarketos.org/wiki/HybrisCloses: #703, #700, #678
Instead of encouraging the user to add the directory to
test_directory_structure.py, mention that we just moved the firmware to
device/{main,community,testing}/. This will be more helpful for users
who get the message after rebasing on master.
In the rare case that we need to change the directory structure, we will
figure out which file to edit.
[ci skip]: various firmware packages fail when built at the same time
(they use generic source files like "bcmdhd.cal" with different checksums)
Add common.get_upstream_branch() and use it instead of assuming "master"
as upstream branch.
Example: user forks v20.05 to v20.05_some-fix, then creates a merge
request. CI must use v20.05 as upstream branch, when comparing for
modified files etc.
This script is used by multiple repositories by now, let's make sure we
only have one place where we need to maintain it. The version in
ci-common is shellchecked and displays less output for 'pmbootstrap
init' again (unless it fails, then it shows the whole log).
Read kernel module names from files in
/etc/postmarketos-mkinitfs/modules and print out which modules are
required by which of these files as the initramfs gets generated. Put
the default modules into a new 00-default.modules. This allows mkinitfs
hooks to add modules to the initramfs.
Parse the modules from a file similar to modules-load.d, with commented
lines and empty lines ignored. Add a simple test script for that and run
it in check().
Split mkinitfs.sh into two files, so the functions (now in
mkinitfs_functions.sh) can be sourced in a test. Such a test will be
added in the next commit.
Move $BINARIES_EXTRA into the get_binaries_extra() function, so its
$(find...) does not get executed when sourcing the functions file. Move
$BINARIES into get_binaries() too for consistency.
Make it pass shellcheck by fixing obvious mistakes like '==' instead of
'='. Add "shellcheck disable" lines for things that are intentionally done
or would be a bit more complicated to rewrite. It would be nice to
refactor some things like '${outfile/initramfs-/uImage-}', but this should
be done in a separate change (patches welcome).
Remove unused variable "modules_path". While at it, add two new fatal
error messages instead of just "exit 1" without any comment and remove
redundant mkdir in APKBUILD.
At the moment the function does not handle all cases correctly.
For example, it cannot say which package was changed when a file
in a subdirectory of a package was changed.
This is used e.g. in device-samsung-golden:
device-samsung-golden
├── APKBUILD
├── deviceinfo
├── downstream
│ ├── init-usb-hook.sh
│ ├── module-config.conf
│ └── modules-load.conf
└── kwin.sh
At the moment, changing a file under "downstream" would attempt
building a "downstream" package, instead of device-samsung-golden.
Refactor the function a bit to walk up the directory hierarchy,
until we find an APKBUILD.
For APKBUILD linting we care only about APKBUILD files that were changed,
not to which package they belong or whatever.
The "with_directory" option of get_changed_packages() really meant
"just give me changed files", which means that we can just use
get_changed_files() instead.
All users of get_changed_files() check if the file was removed,
so it seems like most of them do not need files that were removed.
Also, there is no need to have files listed twice,
so we should return a set instead of a list.
It seems that the .gitlab-ci.yml did not trigger properly when the file
was changed last time, and now it does in this MR. Fix the file to make
it pass.
Do not run upstream compatibility checks whenever pushing to master.
This is confusing, because if the upstream compatibility check fails,
then it appears that the last patch was broken in some way although it
isn't related.
I've moved the upstream compatibility checks to a separate repository
already, and added a badge to the pmaports gitlab project that indicates
whether they are currently succeding or not. The checks run hourly now.
Related: https://gitlab.com/postmarketOS/monitoringFixes: #457
pmbootstrap now allows APKBUILDs in arbitrarily nested directories
within pmaports. This means that we really need some checks to ensure
APKBUILDs don't end up all over the place.
Add a test case with a whitelist of allowed directories.
Having this list in pmaports allows changing it without a new pmbootstrap
release.
At the moment we assume that all files except in /.* belong to
a package. Now that .shared-patches was moved to device/.shared-patches
that does not work correctly anymore.
This method should really check to which package the files belong
(e.g. walk up directories until it finds an APKBUILD) instead of
assuming they are directly in the same directory as the APKBUILD.
For now just ignore **/.* (i.e. all files in dot folders),
to unblock the MR since it's a critical fix.
GitLab disabled downloads of tarballs currently, because it was being
exploited for DoS agains their infrastructure. Work around this by using
git clone for now. I've thought about using --depth=1, but that would
break the ability to choose different branches.
Related: https://status.gitlab.com
At the moment we check that the kernel variants for a device package
match the suffix of an existing linux-postmarketos-<suffix> package.
However, in most cases the name of the kernel subpackage has nothing
to do with the actual kernel that is used:
- the kernel subpackages usually have generic names like "mainline" or
"downstream", which have nothing to do with the actual suffixes like
"qcom-msm8916", "allwinner" or "novathor"
- pine64-dontbeevil uses kernel subpackages to switch between
"longts" and "shortts" device tree variants. Both are actually
using linux-postmarketos-allwinner.
So far we have been whitelisting names like "longts" or "shortts",
but the check does not seem very useful overall.
Removing it fixes the following CI failure for MSM8916:
RuntimeError: Invalid kernel subpackage name 'mainline-modem',
valid: ['downstream', 'rpi', 'rpi2', 'longts', 'shortts', 'stable', 'mainline',
'allwinner', 'qcom-msm8916', 'qcom-msm8974', 'novathor']
Since we copy these aports directly from Alpine and try to keep them in
sync, we don't want to differ from upstream because our CI doesn't pass
on linting. If the APKBUILD should be improved, it should be done
upstream which is then synced back to us.
Oliver Smith