Instead of encouraging the user to add the directory to
test_directory_structure.py, mention that we just moved the firmware to
device/{main,community,testing}/. This will be more helpful for users
who get the message after rebasing on master.
In the rare case that we need to change the directory structure, we will
figure out which file to edit.
[ci skip]: various firmware packages fail when built at the same time
(they use generic source files like "bcmdhd.cal" with different checksums)
Add common.get_upstream_branch() and use it instead of assuming "master"
as upstream branch.
Example: user forks v20.05 to v20.05_some-fix, then creates a merge
request. CI must use v20.05 as upstream branch, when comparing for
modified files etc.
This script is used by multiple repositories by now, let's make sure we
only have one place where we need to maintain it. The version in
ci-common is shellchecked and displays less output for 'pmbootstrap
init' again (unless it fails, then it shows the whole log).
Read kernel module names from files in
/etc/postmarketos-mkinitfs/modules and print out which modules are
required by which of these files as the initramfs gets generated. Put
the default modules into a new 00-default.modules. This allows mkinitfs
hooks to add modules to the initramfs.
Parse the modules from a file similar to modules-load.d, with commented
lines and empty lines ignored. Add a simple test script for that and run
it in check().
Split mkinitfs.sh into two files, so the functions (now in
mkinitfs_functions.sh) can be sourced in a test. Such a test will be
added in the next commit.
Move $BINARIES_EXTRA into the get_binaries_extra() function, so its
$(find...) does not get executed when sourcing the functions file. Move
$BINARIES into get_binaries() too for consistency.
Make it pass shellcheck by fixing obvious mistakes like '==' instead of
'='. Add "shellcheck disable" lines for things that are intentionally done
or would be a bit more complicated to rewrite. It would be nice to
refactor some things like '${outfile/initramfs-/uImage-}', but this should
be done in a separate change (patches welcome).
Remove unused variable "modules_path". While at it, add two new fatal
error messages instead of just "exit 1" without any comment and remove
redundant mkdir in APKBUILD.
At the moment the function does not handle all cases correctly.
For example, it cannot say which package was changed when a file
in a subdirectory of a package was changed.
This is used e.g. in device-samsung-golden:
device-samsung-golden
├── APKBUILD
├── deviceinfo
├── downstream
│ ├── init-usb-hook.sh
│ ├── module-config.conf
│ └── modules-load.conf
└── kwin.sh
At the moment, changing a file under "downstream" would attempt
building a "downstream" package, instead of device-samsung-golden.
Refactor the function a bit to walk up the directory hierarchy,
until we find an APKBUILD.
For APKBUILD linting we care only about APKBUILD files that were changed,
not to which package they belong or whatever.
The "with_directory" option of get_changed_packages() really meant
"just give me changed files", which means that we can just use
get_changed_files() instead.
All users of get_changed_files() check if the file was removed,
so it seems like most of them do not need files that were removed.
Also, there is no need to have files listed twice,
so we should return a set instead of a list.
It seems that the .gitlab-ci.yml did not trigger properly when the file
was changed last time, and now it does in this MR. Fix the file to make
it pass.
Do not run upstream compatibility checks whenever pushing to master.
This is confusing, because if the upstream compatibility check fails,
then it appears that the last patch was broken in some way although it
isn't related.
I've moved the upstream compatibility checks to a separate repository
already, and added a badge to the pmaports gitlab project that indicates
whether they are currently succeding or not. The checks run hourly now.
Related: https://gitlab.com/postmarketOS/monitoringFixes: #457
pmbootstrap now allows APKBUILDs in arbitrarily nested directories
within pmaports. This means that we really need some checks to ensure
APKBUILDs don't end up all over the place.
Add a test case with a whitelist of allowed directories.
Having this list in pmaports allows changing it without a new pmbootstrap
release.
At the moment we assume that all files except in /.* belong to
a package. Now that .shared-patches was moved to device/.shared-patches
that does not work correctly anymore.
This method should really check to which package the files belong
(e.g. walk up directories until it finds an APKBUILD) instead of
assuming they are directly in the same directory as the APKBUILD.
For now just ignore **/.* (i.e. all files in dot folders),
to unblock the MR since it's a critical fix.
GitLab disabled downloads of tarballs currently, because it was being
exploited for DoS agains their infrastructure. Work around this by using
git clone for now. I've thought about using --depth=1, but that would
break the ability to choose different branches.
Related: https://status.gitlab.com
At the moment we check that the kernel variants for a device package
match the suffix of an existing linux-postmarketos-<suffix> package.
However, in most cases the name of the kernel subpackage has nothing
to do with the actual kernel that is used:
- the kernel subpackages usually have generic names like "mainline" or
"downstream", which have nothing to do with the actual suffixes like
"qcom-msm8916", "allwinner" or "novathor"
- pine64-dontbeevil uses kernel subpackages to switch between
"longts" and "shortts" device tree variants. Both are actually
using linux-postmarketos-allwinner.
So far we have been whitelisting names like "longts" or "shortts",
but the check does not seem very useful overall.
Removing it fixes the following CI failure for MSM8916:
RuntimeError: Invalid kernel subpackage name 'mainline-modem',
valid: ['downstream', 'rpi', 'rpi2', 'longts', 'shortts', 'stable', 'mainline',
'allwinner', 'qcom-msm8916', 'qcom-msm8974', 'novathor']
Since we copy these aports directly from Alpine and try to keep them in
sync, we don't want to differ from upstream because our CI doesn't pass
on linting. If the APKBUILD should be improved, it should be done
upstream which is then synced back to us.
Temporarily disabled, because the postmarketOS binary repository has
armhf packages for various kde related packages depending on
qt5-qtwebengine-dev, although qt5-qtwebengine-dev isn't available for
armhf anymore. All related pmaports were disabled for armhf, however
currently the binary repository does not delete binary packages yet,
after the corresponding pmaport was deleted/disabled for a given arch.
I'll work on this with high priority, so we can enable the test again.
Related: https://gitlab.com/postmarketOS/build.postmarketos.org/issues/38
The framework check is supposed to make sure, that we always have the
same versions of multiple packages that belong to the same framework.
We must exclude git versions for this check to have it working properly.
The interesting question is of course: why was this working before?
The check is using the url field to figure out which packages belong to
a certain framework, and by chance, all packages that had a git version,
did also use single quotes around the URL:
url='http://qt-project.org/'
With the old APKINDEX parsing code, this was parsed as:
url: "'http://qt-project.org/'"
And therefore, this was considered a different URL than the URL with
double quotes (again, by chance!) used by the stable qt versions:
url: "http://qt-project.org/"
I have improved the APKBUILD parsing code in pmbootstrap today, and now
it is properly parsing both URLs.
Related: https://gitlab.com/postmarketOS/pmbootstrap/merge_requests/1837
* Only run the wiki test, if anything matching device/device-* was
modified.
* Split testcase linting (flake8) from shellcheck and only run flake8 if
.gitlab-ci/* was modified.
* Rename .gitlab-ci/static_code_analysis.sh to .gitlab-ci/shellcheck.sh
and make it scan all pre/post install/upgrade/deinstall scripts.
[ci:skip-vercheck]: other commits in this merge request have several
intended version downgrades, such as 3 -> 1.0.0
Without this, the device will simply run the broken ACPI DSDT table, so
we might as well update it. If we had put it into a subpackage, it
would not be possible to use the fixed DSDT without any other firmware
packages.
Cut off the hotfix-version (fourth number) from the version strings
before comparing them, to fix invalid errors like:
[14:57:00] Reference pkgver: 5.16.90 (from 'khotkeys')
[14:57:00] ERROR: plasma-workspace has version 5.16.90.1
We need to freeze binutils at 2.31 (Alpine has 2.32 already), until
issue #295 is resolved. Let's disable the test, so the testsuite passes
again if everything else is good, and we don't start to ignore whether
it fails or not.
Do not require ci:ignore-count, when using ci:skip-build in a MR that
has change a lot of packages. If the user already supplies ci:skip-build
then we can safely assume that we don't to check the package count.
Fixes#267
When running on upstream/master, don't compare HEAD against
upstream/master. This is the same commit, so all versions will be the
same. Instead compare against HEAD~1.
Make sure that changed aports always have a higher version than what is
currently in master. This check can be skipped with ci:skip-vercheck (in
square brackets).
Related: #187
Move most code from build_changed_aports.py to common.py. Another patch
will follow soon, which uses common.py to check the versions of changed
aports.
Related: #187
Oliver Smith