Commit Graph

40 Commits

Author SHA1 Message Date
Hans-Christoph Steiner d98d8c2e52 update: do not crash with androguard when working with apps without icons
For example, SpeedoMeterApp.main_1.apk
2018-03-12 17:55:45 +01:00
Hans-Christoph Steiner 40fac10ebc update: extract and store XML icons
These can then be used by the client.

#344
#392
2018-02-15 14:28:45 +01:00
Hans-Christoph Steiner 109eb928e8 aapt 26.0.0 is required to properly parse permissions and label
#236

closes #395
aapt 26.0.0 outputs the permissions correctly

closes #306
aapt 26.0.0 now outputs:  application-label:'K-9 Mail'
2017-12-28 23:07:26 +01:00
Hans-Christoph Steiner bde0558d82 update: reject APKs with invalid file sig, probably Janus exploits
This just checks the first four bytes of the APK file, aka the "file
signature", to make sure it is the ZIP signature and not the DEX signature.
This was checked against the test APK, and I ran it against some known
malware and all of f-droid.org to make sure it works.

All valid ZIP files (therefore APK files) should start with the ZIP
Local File Header of four bytes.

https://www.guardsquare.com/en/blog/new-android-vulnerability-allows-attackers-modify-apps-without-affecting-their-signatures
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner ca50adb2e5 update: switch tests to using standardized setUp() method 2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner bec4f7d547 add Conversations as gradle flavor test case 2017-12-14 16:52:02 +01:00
Hans-Christoph Steiner 3f4e93bf86 Merge branch 'metadataFlavours' into 'master'
Add flavour to metadata

See merge request fdroid/fdroidserver!370
2017-12-02 20:51:24 +00:00
tobiasKaminsky d5ecb34df7
add flavour to metadata 2017-12-01 15:25:00 +01:00
Hans-Christoph Steiner 3ff4b656c6 tests: name temp test dir after test function that used it 2017-11-30 17:32:53 +01:00
Hans-Christoph Steiner 17efa13183 tests: pre-set failfast as reminder of a handy time saver 2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner e4ca82d80c tests: support running on platforms with old java
In order to test on travis-ci's OSX setup, its easiest to support the old
version of Java that is installed there by default.  It is pretty old, so
it does not include the April 2017 update that disables MD5:

https://blogs.oracle.com/java-platform-group/oracle-jre-will-no-longer-trust-md5-signed-code-by-default
https://www.bleepingcomputer.com/news/security/oracle-to-block-jar-files-signed-with-md5-starting-with-april-2017
https://support.ca.com/us/knowledge-base-articles.TEC1691042.html

#292 #323
2017-09-15 11:37:24 +02:00
Hans-Christoph Steiner c67f8e349c metadata: handle empty YAML files without crashing
!311
2017-07-26 18:21:28 -07:00
Hans-Christoph Steiner 7bd171480f update: test metadata creation using internal and external templates
There is a hardcoded template in update.py, and there is also the
possibility for the user to create a template.yml.  This tests both of them
and cleans up the related code a bit.

#352 !310
2017-07-26 15:34:13 -07:00
Hans-Christoph Steiner df99c85ca6 support manually adding per-build antiFeatures in metadata
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
2017-07-06 12:25:48 +02:00
Torsten Grote 0bbe0c5db0
Break up the scan_apk() method and rename it to process_apk() 2017-07-03 18:37:01 -03:00
Hans-Christoph Steiner 746d4bd4cf update: allow_disabled_algorithms option to keep MD5 sigs in repo
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.

closes #323
closes #292
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner be2926ffc8 tests: test moving files to and from the archive
#166
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner 8776221988 check signature and OpenSSL after APK has proven valid
If working with a random grabbag of APKs, there can be all sorts of
issues like corrupt entries in the ZIP, bad signatures, signatures that
are invalid since they use MD5, etc.  Moving these two checks later means
that the APKs can be renamed still.

This does change how common.getsig() works.  For years, it returned
None if the signature check failed.  Now that I've started working
with giant APK collections gathered from the wild, I can see that
`fdroid update` needs to be able to first index what's there, then
make decisions based on that information.  So that means separating
the getsig() fingerprint fetching from the APK signature verification.

This is not hugely security sensitive, since the APKs still have to
get past the Android checks, e.g. update signature checks.  Plus the
APK hash is already included in the signed index.
2017-06-01 17:45:29 +02:00
Hans-Christoph Steiner 4053f03d77 update: add --rename-apks to force APK filenames to fdroid standard
uses the standard package.name_123.apk.  If that exists, it appends the
shasum.  If that exists, then its a duplicate, so its deleted. This should
help @SergeWinters with his 12,000 APKs.
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner 0f4cbc7224 allow APKs with same packageName/versionCode but different signer
There are many APKs out in the wild that claim to be the same app and
version and each other, but they are signed by different keys.  fdroid
should be able to index these, and work with them.   This supports having
the developer's signature via reproducible builds, random collections of
APKs like repomaker, etc.
2017-06-01 10:30:41 +02:00
Hans-Christoph Steiner c348186ad6 update: fix fastlane scraping, it uses a subdir called 'images'
https://commons.wikimedia.org/wiki/File:GetChromium_FeatureGraphic_1024x500.png
2017-05-18 16:13:17 +02:00
thez3ro 06598ae406
use androguard if aapt isn't found 2017-05-04 23:35:17 +02:00
Hans-Christoph Steiner cdef5bcd92 downcase all 'localized' key names to match the rest of index-v1
This is a little omission.  keys that are used in metadata/*.yml all start
with an UpperCase letter, but in fdroidserver, index-v1.json, and
fdroidclient, it is all camelCase with lowercase first letter. The keys
from the 'localized' section are currently never in metadata/*.yml, so
these keys never get downcase.  This change will break fdroidclient
versions that do not also have this change, but since we're in alpha, that
should be fine.

If support for a 'localized' section is added to metadata/*.yml, then the
keys there should probably be UpperCase CamelCase to match the other keys.
2017-04-27 21:12:49 +02:00
Hans-Christoph Steiner 82095c7a9a add basic test for Triple-T Gradle Play Publisher scraping 2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner 9589d13ef2 update: include "What's New" texts when they are available
This uses the "What's New" entry for the CurrentVersionCode and includes it
as the current WhatsNew metadata for the App class.

Things like fastlane supply and Google Play support a "What's New" entry
per-APK, but fdroidclient does not current use anything but the current
version of this data.  Right now, it seems we probably only want to have
the latest WhatsNew in the index to save space.

In theory, we could make the WhatsNew data structure follow the structure
of fastlane/Play, but that would quite a bit of complexity for something
that might never be used.

fdroidclient#910
2017-04-19 10:05:24 +02:00
Torsten Grote 8f6fa2955d
Fix bug when scanning invalid APKs and add a test for it 2017-04-03 13:07:49 -03:00
Hans-Christoph Steiner 6c2cf2ccdd convert internal dict keys to match fdroidclient's Apk class
This syncs up the field names between the fdroiddata .yml files, the keys
used in the implementation in fdroidserver, the index data format, and the
final data structures in fdroidclient.  This makes it easier for devs to
follow, and makes the Jackson parsing library automatically handle
converting the data from the index file to Java instances.

This bumps the metadata version since the apkcache will have to be
discarded.

Here are the name changes:
* apkname --> apkName
* id --> packageName
* sha256 --> hash
* version --> versionName
* versioncode --> versionCode

tests/repo/index.xml was changed only to bump the metadata version
from 17 to 18.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner ce3efe4168 remove support for XML app metadata, its broken
JSON and YAML are very closely related, so supporting both of them is
basically almost no extra work.  Both are also closely related to how
Python works with dicts and pickles. XML is a very different beast, and its
not popular for this kind of thing anyway, so just purge it.
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner 47d9fd330d remove unused 'apps' argument from update.scan_apks() 2016-11-02 16:11:52 +01:00
Hans-Christoph Steiner 5f8beaa2c6 tests for finding the proper OBB files for each APK 2016-07-07 22:34:41 +02:00
Hans-Christoph Steiner 1b7a8f85fc parse targetSdkVersion from APKs
The default targetSdkVersion is minSdkVersion, according to the docs:
https://developer.android.com/guide/topics/manifest/uses-sdk-element.html#target

https://gitlab.com/fdroid/fdroidclient/issues/682
2016-06-14 11:43:07 +02:00
Hans-Christoph Steiner 547a57e693 fix "DeprecationWarning: Please use assertEqual instead."
They've been deprecated since python 3.2, which was released a long time
ago.
2016-06-14 10:06:02 +02:00
Hans-Christoph Steiner 3768d7a4d6 refactor env handling for FDroidPopen to support .fdroid.* metadata
The start up sequence of processes that are based on the .fdroid.* metadata
is a bit different, so this ensures that the environment variables get
properly initialized in all cases.

This also creates a single function where the environment is set.  Before
it was being set in multiple places across multiple files.
2016-03-23 17:16:27 +01:00
Daniel Martí a51d849a5b update test: replace decode('hex') with unhexlify 2016-03-11 13:27:00 +00:00
Daniel Martí 24ad0418e3 tests: switch to python3 2016-03-11 13:27:00 +00:00
Daniel Martí ee9a296b64 Make pre-commit hook pass after python3 switch 2016-03-10 16:43:37 +00:00
Hans-Christoph Steiner 69c81c3817 use jarsigner and keytool from same JDK as is being set in JAVA7_HOME
Using the same JDK throughout should prevent weird bugs where a setup might
use Java8's jarsigner and Java7's keytool.  This also allows the user to
set java_paths and have jarsigner and keytool used from that specified JDK.

This incorporates almost all of the patch that is in the Debian package
that forces fdroidserver to use the default JDK on that Debian release.

closes #93 https://gitlab.com/fdroid/fdroidserver/issues/93
2016-02-11 21:17:23 +01:00
Hans-Christoph Steiner 9e5dd19fc8 add test files to pre-commit hook and fix pep8 errors 2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner 2a87ae5127 fix update.TestCase: remove unused SilentPopen import, that class was removed 2015-01-21 10:21:54 +01:00
Hans-Christoph Steiner 18db382c47 include test cases for python getsig replacement
This includes the old getsig.java since that is the canonical implementation
of that algorithm.

fixes #5 https://gitlab.com/fdroid/fdroidserver/issues/5
2014-11-07 09:20:33 +00:00