The `force_build_tools` config option was added a long time ago to
brute force the _build-tools_ version by trying to replace the value
in `build.gradle` files. This is never something that should be used
in production, since the app's build metadata should specify this kind
of thing. And now that we're moving towards _androguard_ for
everything except fdroid build and fdroid publish, _build-tools_ will
no longer even be used in the other commands.
The buildSrc directory contains custom build logic written in Kotlin.
Before this change we had to 'scandelete' buildSrc/build in the build
recipes becase 'gradle clean' leaves binary artifacts there.
The 'dirs' array contains a single-level listing of a directory, e. g.
['app', 'build', 'build.gradle', 'gradle', '.gradle']. Multi-component
paths like 'build/tmp' could never be found in this array and thus were
never removed.
Call shutil.rmtree() without checking that the argument is in 'dirs'. If
it exists and is a directory, it'll be removed. Otherwise shutil.rmtree()
will do nothing.
The build command has to use some threading stuff to handle the timeout and
locks. This seems to prevent the command from exiting, unless this hack is
used.
This fixes bandit misdetection of hardcoded /tmp dir. posixpath.join() is
good to use anyway, it highlights what is on the remote server, vs what is
local. Local paths should use os.path.join() to support Windows, etc.
posixpath is built in since Python 3.4, maybe earlier
This expands the gradle wrapper shell script used by the buildserver for
usage outside the buildserver environment. It also allows downloading
whitelisted versions of gradle if they are not yet deployed to the
buildserver by simply upsating the copy of fdroidserver (in contrast to
having to reprovision the whole buildserver).
We first move the buildserver/gradle shell script to the repo root
as gradlew-fdroid, as it's an fdroid specific gradle wrapper.
We also now sync it inside the build VM before each build.
We then add a list of whitelisted gradle distributions taken from the
makebuildserver script.
The script additionally now reads two env vars which tell it where to
expect installed versions of gradle and where it might store downloaded
gradle .zip files. Both of those are configurable from config.py. As the
first should normally just be a subdir of the second it's not exposed in
the example config.py but only used by the buildserver config.py.
Default config now uses this internal gradle wrapper but a path to a
custom wrapper or specific gradle distribution can still be set from
config.py.
Closesfdroid/fdroidserver#98
Ref: fdroid/fdroidserver#370
This commit fixes two bugs with reproducible builds:
* Files added by the buildserver to META-INF (fdroidserverid and buildserverid)
were causing signature verification to always fail when --on-server was used.
Since they are not needed anymore, they are no longer added to APKs.
* When showing a diff between both APK files, `jar xf` did not extract
the full APK properly which was causing useless diffs.
Instead of using jar, python's zipfile library is used instead.
Currently f-droid.org has a lot of pending builds and big delays
between "fdroid build --all" runs. Bump overall build time limit from
12 hours to 36 hours to increase buildserver throughput.
There are all sorts of unfiltered user inputs like tag and branch names in
source repos. If those names are fed into popen calls that use shell=True,
that opens up a wide range of exploits. All core operations should never
use shell=True.
Hans-Christoph Steiner