Commit Graph

453 Commits

Author SHA1 Message Date
Hans-Christoph Steiner d98d8c2e52 update: do not crash with androguard when working with apps without icons
For example, SpeedoMeterApp.main_1.apk
2018-03-12 17:55:45 +01:00
Hans-Christoph Steiner 70d9633555 build/checkupdates/update: log current fdroiddata commit to wiki 2018-03-05 21:49:09 +01:00
Hans-Christoph Steiner b2ca49b26c update: make icon extraction less dependent on aapt
For androguard, @thezero already developed a way to get all the icons after
only extracting the icon name.  So this uses that for the aapt-based scans
also, to make them less brittle.

This should fix the problem where `fdroid update` was choosing the XML icon
for apps that include one, like NewPipe.

closes fdroid/fdroid-website#192
2018-02-26 23:43:42 +01:00
Hans-Christoph Steiner 88e24dc4e3 update: switch to improved androguard detection 2018-02-22 15:08:54 +01:00
Hans-Christoph Steiner 52b3436ff6 make is_apk_and_debuggable() default to using androguard before aapt 2018-02-22 15:08:53 +01:00
Hans-Christoph Steiner aa4f54bf18 update: include implied permissions when using androguard
`aapt dump badging` includes these when listing uses-permissions:
https://github.com/androguard/androguard/pull/428
2018-02-21 12:34:54 +01:00
Hans-Christoph Steiner 63d4d46291 update: 'features' list only includes required features
The F-Droid index 'features' list is not the same as what is in the
AndroidManifest.xml.  It only includes "required" features, for example.
2018-02-20 17:08:55 +01:00
Hans-Christoph Steiner 008110889a update: fix crash from missing file extension in extracted icon
I missed this in 40fac10ebc, yay tests!
2018-02-20 16:29:52 +01:00
Hans-Christoph Steiner d1ded7f64d update: use androguard by default if it is available
closes #236
2018-02-15 14:28:48 +01:00
Hans-Christoph Steiner 5281228ea5 update: switch to new androguard v3.1 API based on lxml
apkobject.get_android_manifest_xml() used to return a xml.dom.minidom
object, now it returns an lxml.etree.Element object.
2018-02-15 14:28:45 +01:00
Hans-Christoph Steiner 699b3e4c69 update: fix min/target/max edge case parsing with androguard
In order to test that aapt defaults minSdkVersion to 3, I ran this script
then compared the output with meld:

cd $ANDROID_HOME/build-tools
for d in *.*; do echo $d; $ANDROID_HOME/build-tools/$d/aapt dump badging /home/hans/code/fdroid/server/tests/repo/com.politedroid_3.apk > /tmp/${d}.txt; done
meld /tmp/17.0.0.txt /tmp/26.0.2.txt /tmp/27.0.3.txt
2018-02-15 14:28:45 +01:00
Hans-Christoph Steiner 40fac10ebc update: extract and store XML icons
These can then be used by the client.

#344
#392
2018-02-15 14:28:45 +01:00
Hans-Christoph Steiner 5713b54e0b update: find alternate PNGs for apps that have an XML app icon
Apps can now use an XML icon, but if the app supports older Android
versions, it'll also contain PNG versions of the same icon.  This finds
those PNGs and uses them instead.

#344
closes #392
fdroiddata#913
2018-02-15 14:28:45 +01:00
Hans-Christoph Steiner 1f99a27a9c Revert "wiki: include per-app link to all related activity on gitlab.com"
This reverts commit a57f17b276.

Turns out this was totally useless, this can be fully handled in
https://f-droid.org/wiki/page/Template:App since it already has the
Application ID there.
2018-02-12 12:15:56 +01:00
Hans-Christoph Steiner a57f17b276 wiki: include per-app link to all related activity on gitlab.com 2018-01-26 10:18:41 +01:00
Hans-Christoph Steiner 010f1c5029 log installed android sdk versions for update and checkupdates 2018-01-22 13:49:10 +01:00
Hans-Christoph Steiner 4beb2d52e9 wiki: log update start/stop time and command line 2018-01-22 13:49:10 +01:00
Hans-Christoph Steiner 5b92820ff3 wiki: fix bug updating Repository Maintenance
site.pages doesn't seem to exist anywhere, site.Pages is used throughout.
2018-01-22 13:49:10 +01:00
Izzy 42ac65e8aa simplifying fix for "cannot identify image file" with XML icons 2018-01-17 16:48:08 +01:00
Izzy 6f5b539a54 fix "cannot identify image file" with XML icons 2018-01-12 22:12:27 +01:00
Hans-Christoph Steiner 2659312a7c update: support working with old versions of PIL/Pillow
Image.close() was added in Pillow 2.4 or so.
2017-12-20 23:46:37 +01:00
Michael Pöhn c17aeb5231 fix handling unreadable images in update.extract_apk_icons 2017-12-16 22:06:20 +01:00
Hans-Christoph Steiner 42522c23c9 update: do not crash if AndroidManifest.xml in APK has invalid date
This crash actually blocked a Janus exploit APK from being added to the
repo, but crashing isn't really the appropriate way to do that.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner 8f45796ecb update: close unclosed Image instance 2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner 387eebc4d6 update: strip all metadata from PNGs
This strips metadata and optimizes the compression of all PNGs copied
from the app's source repo as well as all the icons extracted from the
APKs.  There have been exploits delivered via image metadata, and
F-Droid isn't using it all, so its best to just remove it.

This unfortunately uncompresses and recompresses the files.  Luckily,
that's a lossless procedure with PNGs, and we might end up with
smaller files.  The only tool I could find that strips without
changing the image data is exiftool, but that is written in Perl.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner 67b9514c5a update: strip EXIF data from all JPEGs
EXIF data can be abused to exploit systems a lot easier than the JPEG image
data can.  The F-Droid ecosystem does not use the EXIF data, so keep things
safe and strip it all away.  There is a chance that some images might rely
on the rotation to be set by EXIF, but I think having a safe system is more
important.

If needed, only the rotation data could be saved.  But that then makes it
hard to tell which images have been stripped.  This way, if there is no
EXIF, it has been stripped.  And if there is EXIF data, then it is suspect.

https://securityaffairs.co/wordpress/51043/mobile-2/android-cve-2016-3862-flaw.html
https://threatpost.com/google-shuts-down-potentially-massive-android-bug/120393/
https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-exif-headers.html

The big downside of this is that it decompresses and recompresses the
image data.  That should be replaced by a technique from jhead,
exiftool, ObscuraCam, etc. that only strips the metadata.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner bde0558d82 update: reject APKs with invalid file sig, probably Janus exploits
This just checks the first four bytes of the APK file, aka the "file
signature", to make sure it is the ZIP signature and not the DEX signature.
This was checked against the test APK, and I ran it against some known
malware and all of f-droid.org to make sure it works.

All valid ZIP files (therefore APK files) should start with the ZIP
Local File Header of four bytes.

https://www.guardsquare.com/en/blog/new-android-vulnerability-allows-attackers-modify-apps-without-affecting-their-signatures
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner 5ce950e748 update: print warnings for all KnownVulns found
Some baby steps towards making the KnownVuln stuff more visible.
2017-12-14 16:57:22 +01:00
Andrea Scarpino 19a83b6219 Add Liberapay support 2017-12-12 11:53:31 +01:00
Hans-Christoph Steiner 5d54446efc update: do not replace Name/Summary from template unless blank
`fdroid nightly` needs this change so it can set the Summary using the
template.
2017-12-07 22:39:32 +01:00
tobiasKaminsky d5ecb34df7
add flavour to metadata 2017-12-01 15:25:00 +01:00
Hans-Christoph Steiner d46d9574b4 update: use KnownApks dates to check system clock on offline machines
KnownApks provides a reliable source of a relatively recent date.
2017-11-29 21:06:02 +01:00
Hans-Christoph Steiner 9ade09349b update: improve help strings 2017-11-08 14:05:17 +01:00
Hans-Christoph Steiner 7be46d53c7 sort Anti-Features before writing them to the wiki
This avoids silly updates to the wiki where the sort order of the Anti-
Features is the only change:

−{{AntiFeature|KnownVuln}}
 {{AntiFeature|DisabledAlgorithm}}
+{{AntiFeature|KnownVuln}}

https://f-droid.org/wiki/index.php?title=info.guardianproject.browser&curid=517&diff=189655&oldid=188037
2017-10-27 09:15:22 +02:00
Ciaran Gultnieks 505d2c1325 update bombs out when no icons (closes #402) 2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner ac69d5b17d update: make all strings translatable 2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner ffc91e301a make _ always be the gettext function, nothing else
This avoids hard bugs where the _() function gets overidden by a str or
something else.
2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner 927104a4e3 update: make strings translatable 2017-10-19 23:16:58 +02:00
Hans-Christoph Steiner 460c4185bb update: support Fastlane's official location for screenshots
fastlane/android/metadata/locale/images/*Screenshots

https://github.com/fastlane/fastlane/blob/2.61.0/screengrab/lib/screengrab/runner.rb#L324

closes #357
2017-10-13 14:58:15 +02:00
Hans-Christoph Steiner a4169484fd update: fix detection of Triple-T screenshots
* https://github.com/westnordost/StreetComplete/issues/489
* https://forum.f-droid.org/t/screenshots-dont-show-up-in-client-using-gradle-play-publisher/881/2

#357
2017-10-13 14:58:15 +02:00
Michael Pöhn 290b9050e0 convert apk timestamp from filesystem to utc before storing 2017-09-26 14:13:08 +02:00
Michael Pöhn 6fc968f7cd choose best apk version vor index v0 2017-09-26 14:11:09 +02:00
Michael Pöhn 6930edf889 sort index-v1; publish now creates and stores a list of signature fingerprints 2017-09-26 14:11:09 +02:00
Michael Pöhn 6a4a78429f add signer to index v1 2017-09-26 14:11:09 +02:00
Hans-Christoph Steiner 176f539647 allow spaces in filenames
This fixes all the bugs I could find that prevented fdroid from
handling files with spaces in them.  This is more important now that
fdroid supports random media files, and Repomaker
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner 6c7b987239 fix string formats that are ambiguous for translators 2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner 53e4ec47a7 English source string corrections
From @monolifed <monolifed@gmail.com>
2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner 278d67d960 implement gettext localization
This allows all the text to be localized via Weblate.  This is a quick
overview of all the strings, but there are certainly some that were left
out.

closes #342
2017-09-15 11:39:00 +02:00
Hans-Christoph Steiner 7bd171480f update: test metadata creation using internal and external templates
There is a hardcoded template in update.py, and there is also the
possibility for the user to create a template.yml.  This tests both of them
and cleans up the related code a bit.

#352 !310
2017-07-26 15:34:13 -07:00
Izzy cbd420e490 preserve order and formatting on creating Metadata from Yaml template (closes #352) 2017-07-26 09:22:01 +02:00