Commit Graph

4199 Commits

Author SHA1 Message Date
Torsten Grote 8d424f19ec
Support ETag when downloading repository index 2017-05-02 15:37:02 -03:00
Michael Pöhn e7e97654b1 Merge branch 'libvirt-support' into 'master'
Check for libvirt specific shutoff message

See merge request !262
2017-05-02 07:39:51 +00:00
Torsten Grote f758b5c205 Merge branch 'pre-0.103-alpha4-rename' into 'master'
downcase all 'localized' key names to match the rest of index-v1

See merge request !261
2017-05-01 17:59:59 +00:00
Andrew Patrikalakis 9ee96117d8 Check for libvirt specific shutoff message 2017-04-29 18:38:44 -07:00
Hans-Christoph Steiner cdef5bcd92 downcase all 'localized' key names to match the rest of index-v1
This is a little omission.  keys that are used in metadata/*.yml all start
with an UpperCase letter, but in fdroidserver, index-v1.json, and
fdroidclient, it is all camelCase with lowercase first letter. The keys
from the 'localized' section are currently never in metadata/*.yml, so
these keys never get downcase.  This change will break fdroidclient
versions that do not also have this change, but since we're in alpha, that
should be fine.

If support for a 'localized' section is added to metadata/*.yml, then the
keys there should probably be UpperCase CamelCase to match the other keys.
2017-04-27 21:12:49 +02:00
Hans-Christoph Steiner 9f9f0d1a16 Merge branch 'git-identity-file' into 'master'
Make git server mirror upload honor config['identity_file'] option

See merge request !259
2017-04-24 21:24:07 +00:00
Torsten Grote c137231d8b
Make git server mirror upload honor config['identity_file'] option 2017-04-24 16:22:33 -03:00
Hans-Christoph Steiner 591bfc4474 Merge branch '148-no-verify-logs-on-split-build-publish-infrastructure' into 'master'
compare apk with Binaries from metadata right after building

Closes #148

See merge request !247
2017-04-24 18:50:58 +00:00
Michael Pöhn fc4e40255d remove build output when not reproducible 2017-04-23 11:33:51 +02:00
Michael Pöhn 5b7209d882 unified method for naming build output 2017-04-22 12:04:32 +02:00
Michael Pöhn 62af9b8e3d check meld config for manual apk diffs 2017-04-22 10:48:50 +02:00
Michael Pöhn 4044e06e7b use verify_apks for checking build output 2017-04-22 10:48:50 +02:00
Michael Pöhn f4a7c19043 fix java architecture in buildserver config 2017-04-22 10:48:50 +02:00
Michael Pöhn a3e7eacc9b compare apk with Binaries from metadata right after building 2017-04-22 10:48:50 +02:00
Hans-Christoph Steiner 1c25c516aa Merge branch '273-invalid-apk-gets-through-to-publish-stage' into 'master'
throw proper error when download of apks fail

Closes #273

See merge request !258
2017-04-21 07:56:55 +00:00
Michael Pöhn abe04272b1 throw proper error when download of apks fail 2017-04-20 17:48:38 +02:00
Daniel Martí c7ae6bc826 Merge branch 'gradleplugin' into 'master'
bs: use only prefixes for gradle kv pairs

See merge request !257
2017-04-19 09:27:42 +00:00
Hans-Christoph Steiner 650b3c95b4 Merge branch 'full-app-store-metadata-scraping' into 'master'
Full app store metadata scraping

Closes #204 and #143

See merge request !253
2017-04-19 08:25:56 +00:00
Hans-Christoph Steiner 82095c7a9a add basic test for Triple-T Gradle Play Publisher scraping 2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner fd21d68bc1 update: fix glob to actually match all locales
glob != regexp, [A-Z-.@] means one of those chars is required in that
position, so the glob pattern was requiring a least 3 chars.  Locales are
usually just two lower case letters, e.g. vi, de, ar.
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner cb49f57c06 support "Author Web Site" as metadata field
Fastlane Supply, Triple-T Gradle Play Publisher, and many app stores
include the possibility to specify a website for the author, as distinct
from the website for the app.

closes #204
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner 18f949c62b include Author/Contact info from Triple-T Gradle Play Publisher
This is just the bare minimum, it adds it to the index, but AuthorPhone
and AuthorWebsite are not yet supported by fdroidclient.  AuthorName is.

#204
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner 9589d13ef2 update: include "What's New" texts when they are available
This uses the "What's New" entry for the CurrentVersionCode and includes it
as the current WhatsNew metadata for the App class.

Things like fastlane supply and Google Play support a "What's New" entry
per-APK, but fdroidclient does not current use anything but the current
version of this data.  Right now, it seems we probably only want to have
the latest WhatsNew in the index to save space.

In theory, we could make the WhatsNew data structure follow the structure
of fastlane/Play, but that would quite a bit of complexity for something
that might never be used.

fdroidclient#910
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner 822c2d3992 update: app store metadata from Triple-T gradle play publisher
Since f-droid.org has the source repos, `fdroid update` can just grab files
from the source repo.  This makes `fdroid update` use the metadata that is
laid out in the Triple-T gradle play publisher filesystem layout:

https://github.com/Triple-T/gradle-play-publisher#upload-images
https://github.com/Triple-T/gradle-play-publisher#play-store-metadata

closes #143
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner 89c480181c update: include localized app store metadata from fastlane
This includes all metadata from the app's source repo following the path
layout specified for the fastlane supply tool:
https://github.com/fastlane/fastlane/blob/1.109.0/supply/README.md#images-and-screenshots

refs #35
closes #143
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner a7daec7c36 add char limits for text: Name=30, Video=256, WhatsNew=500
Since this info can come from automated sources, this prevents giant blobs
from being mistakenly stuck in these fields.  It also brings it inline with
the standard lengths other popular app stores use.

https://support.google.com/googleplay/android-developer/answer/113469?hl=en
https://stackoverflow.com/a/20276565
2017-04-19 10:05:24 +02:00
Boris Kraut 58900744ea bs: use only prefixes for gradle kv pairs 2017-04-18 23:36:01 +02:00
Hans-Christoph Steiner ac26304bf4 Merge branch 'gradleplugin' into 'master'
bs: update gradle plugin/version pairs

See merge request !256
2017-04-18 19:35:11 +00:00
Boris Kraut ebbea3f0d1 bs: update gradle plugin/version pairs 2017-04-18 19:42:52 +02:00
Hans-Christoph Steiner 0ffbff9aec Merge branch 'patch-1' into 'master'
Properly add gradle3.5

See merge request !255
2017-04-18 07:13:34 +00:00
Ruslan Boitsov a6ec4ba363 Add gradle 3.5 to the list of available versions 2017-04-18 06:41:09 +00:00
Hans-Christoph Steiner 1d44a42774 Merge branch 'gradle35' into 'master'
makebs: add gradle 3.5

See merge request !254
2017-04-17 12:01:01 +00:00
Boris Kraut 10e91d1be2 makebs: add gradle 3.5 2017-04-15 15:08:13 +02:00
TheZ3ro 89b0dea2fa Merge branch 'support-new-features-with-offline' into 'master'
Support new features with offline signing

See merge request !250
2017-04-13 15:21:01 +00:00
Hans-Christoph Steiner 489b129dd8 PEP8 E305 expected 2 blank lines after class or function definition 2017-04-13 10:55:07 +02:00
Hans-Christoph Steiner 5d5e34005b Merge branch 'bs_dependency_pyasn1' into 'master'
provision missing pyasn1 dependency to build server

See merge request !251
2017-04-12 23:02:02 +00:00
Hans-Christoph Steiner 31d92367ed git mirror test requires git >= 2.3 2017-04-13 00:49:36 +02:00
Hans-Christoph Steiner fcb0634eaa server: store virustotal JSON locally as reusable data cache
This helps with uploading faster when hitting the public API rating
limiting since it will not hit the API if there is already a matching
response on the local filesystem.  This also provides a source of data for
things like repomaker to diplay to users.
2017-04-13 00:49:36 +02:00
Michael Pöhn 82bc71fa6f provision missing pyasn1 dependency to build server 2017-04-13 00:44:53 +02:00
Hans-Christoph Steiner d74d6160dd server: check virustotal has APK before uploading it
This restructures the virustotal uploading so that first checks whether
virustotal already has the file, and only if not does it upload it.  This
also handles the public API rate limiting, which returns an HTTP 204. This
will now try again until it succeeds, even when rate limited.   Instead of
just getting the list of files from the filesystem, this reads the index-v1
which also already has the SHA256 in there.  virustotal also uses SHA256 as
a unique ID for files.
2017-04-12 15:04:05 +02:00
Hans-Christoph Steiner 8dd9d63e8f server: show progress bar when pushing git mirrors
The git mirrors can be really slow to upload, so they need a progress bar.
2017-04-12 15:04:05 +02:00
Hans-Christoph Steiner f94ef29271 server: prefer s3cmd over libcloud for syncing
libcloud has been flaky for years, but it is also widely available and is
more pythonic.  So this adds the option for using `s3cmd sync` if it is
available.  `s3cmd sync` acts like rsync for uploading to S3.  So this code
emulates the rsync uploading code in `fdroid server update`.
2017-04-12 15:04:05 +02:00
Hans-Christoph Steiner 17219d56f4 server: support git mirrors with the offline signing setup 2017-04-12 15:04:04 +02:00
Hans-Christoph Steiner d3b9dbece9 server: rely on rsync to delete files from git-mirror
Instead of each time deleting the whole local git-mirror repo, and
recreating it, this just deletes the .git/ dir, then lets the rsync in
_local_sync() handle deleting anything that should no longer be in the repo.
2017-04-12 15:04:04 +02:00
Hans-Christoph Steiner 5cc15d0fa9 support git@gitlab.com: style URLs in servergitmirrors
This converts the git@gitlab.com SSH URLs to the proper HTTPS URLs that
fdroidclient can directly use.
2017-04-12 00:45:22 +02:00
Hans-Christoph Steiner 183ce9541b server: support btlog on offline signing setups
This is necessary to shuttle the binary transparency log from the offline
machine to the thumb drive, then from the thumb drive to the online box.
2017-04-12 00:45:22 +02:00
Hans-Christoph Steiner f75b547795 server: remove unused --sync-from-local-copy-dir
This option was not hooked up at all, and does not make sense as a command
line argument.  It should just be a config.py item.  In that case, the
presence of config.py marks the current dir as a repo, so there is no
longer a need to test for a dir called repo/ as a safety.  This makes the
setup easier, since sync_from_localcopy() now creates repo/ for the user.
2017-04-12 00:45:22 +02:00
Hans-Christoph Steiner 0c19d4566d Merge branch 'elimitate-password-files' into 'master'
Eliminate the need for password files

See merge request !249
2017-04-11 21:37:46 +00:00
Torsten Grote 4d25113fa0
Eliminate the need for password files
The passwords are now passed as private environment variables to the
processes that need them.
2017-04-11 16:34:49 -03:00
Torsten Grote cb942168d5 Merge branch 'nail-down-btlog-and-bugfix' into 'master'
Nail down btlog and key bugfix

See merge request !248
2017-04-10 21:21:52 +00:00