This was done with much help from @uniqx. This is the first level of
supporting APK Signatures v1, v2, and v3. This is enough to include
APKs with any combo of v1/v2/v3 signatures. For this to work at all,
apksigner and androguard 3.3.3+ must be installed.
closes#399
androguard's v2 signature methods return DER-encoded certificates. The DER-
encoded certificates are parsed from JAR Signature's raw "Signature Block
File".
Also, update the documentation comment to be absolutely clear.
The function used to copy images from any locale directory starting with at least two lowercase letters correctly, but only add those to the app metadata that started with two letters plus one big letter or a "-". The same problem did not exist for adding texts to the metadata.
build-tools 28.0.3 added a new field in the end 'compileSdkVersionCodename=', which also accidentally ends with the string 'name='.
The purpose of this regex was to catch the 'packageName' field, which is in ht eaapt ouput the exact ' name=', therefore added whe non-caracter \W prefix match.
sample aapt output (28.0.3):
package: name='com.a.b.app' versionCode='1' versionName='1.0' compileSdkVersion='28' compileSdkVersionCodename='9'
previously, regex was catching second occurence, so '9'
See merge request !582
Fastlane/Triple-T app store metadata files must be in UTF-8. Before this
would crash if they were not. This changes the handling to just replace
the non-UTF8 chars with a �. Here's the stacktrace:
CRITICAL: Unknown exception found!
Traceback (most recent call last):
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 164, in <module>
main()
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 159, in main
raise e
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 138, in main
mod.main()
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2010, in main
insert_localized_app_metadata(apps)
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 885, in insert_localized_app_metadata
os.path.join(root, f))
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 700, in _set_localized_text_entry
text = fp.read()[:limit]
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 94: invalid start byte
699b3e4c69 got it wrong for targetSdkVersion.
Also, one confusing thing is that aapt outputs "sdkVersion: '3'" for
com.politedroid_3.apk but no "sdkVersion:" for no.min.target.sdk_987.apk.
F-Droid never really supported running on android-1 or android-2, so it
seems pointless to debug support for them.
Android Application IDs must be valid Java Package Names. While the build
tools likely validate the Application ID, it is possible to manually create
a malicious APK.
pickle can serialize executable code, while JSON is only ever pure data.
The APK cache is only ever pure data, so no need for the security risks of
pickle. For example, if some malicious thing gets write access on the
`fdroid update` machine, it can write out a custom tmp/apkcache which would
then be executed. That is not possible with JSON.
This does just ignore any existing cache and rebuilds from scratch. That is
so we don't need to maintain pickle anywhere, and to ensure there are no
glitches from a conversion from pickle to JSON.
closes#163
Sometimes androguard returns the XMLNS as entirely empty, which would make
it an invalid APK since normally the 'android' name is mapped to the
'http://schemas.amazon.com/apk/res/android' value. Occasionally, a
different key is used.
closesfdroid/fdroidserver#515
fixes de35f1b05bb864c2d51b278f933cfd9a39a96b16:
Traceback (most recent call last):
File "/home/hans/code/fdroid/server/fdroid", line 164, in <module>
main()
File "/home/hans/code/fdroid/server/fdroid", line 138, in main
mod.main()
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1932, in main
apks, cachechanged = process_apks(apkcache, repodirs[0], knownapks, options.use_date_from_apk)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1459, in process_apks
use_date_from_apk, ada, True)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1332, in process_apk
apk = scan_apk(apkfile)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1051, in scan_apk
scan_apk_androguard(apk, apk_file)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1220, in scan_apk_androguard
res_id = arsc.get_id(apk['packageName'], res_id)[1]
TypeError: 'NoneType' object is not subscriptable
For example https://f-droid.org/archive/com.abitsinc.andr_5.apk:
$ aapt dump badging archive/com.abitsinc.andr_5.apk |head -1
package: name='com.abitsinc.andr' versionCode='5' versionName='5' platformBuildVersionName='2.3.3'
Instead of just crashing, first try to use the versionName as written in the
build metadata, otherwise just let it be blank. A blank versionName will
cause fdroidclient < 1.3 to crash. Blank versionNames are not allowed in
the .txt metadata format, only .yml.
closes#477closes#478
closes fdroidclient#1416
closes fdroidclient#1417
closes fdroidclient#1418
fdroiddata!3061
Fixes the following crash:
```
$ fdroid update --create-metadata --rename-apks
WARNING: Using Java's jarsigner, not recommended for verifying APKs! Use apksigner
CRITICAL: Unknown exception found!
Traceback (most recent call last):
File "/home/jonas/miniconda3/bin/fdroid", line 164, in <module>
main()
File "/home/jonas/miniconda3/bin/fdroid", line 138, in main
mod.main()
File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1927, in main
apks, cachechanged = process_apks(apkcache, repodirs[0], knownapks, options.use_date_from_apk)
File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1454, in process_apks
use_date_from_apk, ada, True)
File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1327, in process_apk
apk = scan_apk(apkfile)
File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1056, in scan_apk
scan_apk_aapt(apk, apk_file)
File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1184, in scan_apk_aapt
apk['icons_src'] = _get_apk_icons_src(apkfile, icon_name)
File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1089, in _get_apk_icons_src
density_re = re.compile('^res/(.*)/' + icon_name + '\.(png|xml)$')
TypeError: must be str, not NoneType
```
aapt --rename-manifest-package changes the applicationId for an app without
changing the packageName listed in AndroidManifest.xml under
<application android:package="">
repo/ch.swift.willi_417101.apk had a C/Java comment in the
AndroidManifest.xml rather than an XML comment:
<uses-sdk android:minSdkVersion="16" android:targetSdkVersion="26">
</uses-sdk>
// Remove permissions introduced by the appsflyer library
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION">
</uses-permission>
Hans-Christoph Steiner