Commit Graph

44 Commits

Author SHA1 Message Date
proletarius101 947217549a feat: add servergitmirrors as a dict support 2024-02-14 16:50:39 +00:00
Hans-Christoph Steiner 6d541e3ef6
establish config/mirrors.yml as a way to set up mirrors
The mirrors: entry in config.yml is great for quick access and shorter
mirror lists.  Now that we are adding a lot more metadata to the mirrors,
including potentially the full text of the privacy policy, having this
in its own file should make these cases easier to manage.
2023-12-07 14:38:26 +01:00
Hans-Christoph Steiner c7070b2799
index.TestCase: manually convert to black code format
* switch to `from fdroidserver import common, index, signindex...`
2023-04-27 11:34:07 +02:00
Hans-Christoph Steiner 98decf8cfa index: use 'fake.url' as test domain when no networking is used 2023-04-27 10:35:09 +02:00
Hans-Christoph Steiner 7c692a4532 index-v2 'mirrors' fully settable from config
This lets mirrors: in config.yml be the same list-of-dicts format as it is
in index-v2.  This also includes a data format conversion to maintain the
right format for the old, unchanging index v0 and v1 formats.

#928
#1107
2023-04-27 07:53:02 +00:00
Hans-Christoph Steiner ceef07d2f2 test of current internal format of mirrors list in config 2023-04-27 07:53:02 +00:00
Jochen Sprickerhof 889b8cb372 Don't copy per version anti features to all versions
make_v0() modified the apps data structure to copy an anti feature for a
specific version to all versions resulting in index-v1 and -v2 to
contain wrong anti feature annotations. This patch fixes this and adds a
test that the data structure is no longer modified.

The bug shadowed bugs in the AF implementation of -v1 and -v2 resulting
in not coping the version specific data. This is corrected as well.
This is also tested now.

For -v2 the AF dict is now sorted to make the result reproducible.

Finally The NoSourceSince AF was added as a per version and overall AF
in -v1 and is now only applied as an overall AF and the test is updated
accordingly.
2023-04-04 14:20:41 +00:00
Jochen Sprickerhof 484f289919 index.TestCase: support more arguments (like -k) 2023-04-04 14:20:41 +00:00
Hans-Christoph Steiner b1f482009a
tame verbose logging in index tests by default 2023-03-09 12:35:31 +01:00
Hans-Christoph Steiner f3e49f4bcb
download_repo_index_v2() for verified downloading of index-v2 2023-03-09 12:35:29 +01:00
Hans-Christoph Steiner a557764b4d
sign tests/repo/index-v1.jar once per index.TestCase run
Signing files is a slow operation, especially with jarsigner. This speeds
up the full test run from 10-12 seconds to 2-3 seconds, which makes it
possible to run the tests interactively again. And it stops signing the
file entirely for tests that do not even touch that file.

In the long run, it would probably make sense to have each test case sign
the file as it needs it, but that's a much bigger change.
2023-03-09 12:34:57 +01:00
Hans-Christoph Steiner b697845bda add test case for index.package_metadata() 2023-02-16 21:45:39 +01:00
Jochen Sprickerhof d29a486e31 tests: use context manager and/or standard setup temp files 2023-02-08 18:12:35 +01:00
Hans-Christoph Steiner 947d94e0a9
deploy: support GitLab Job Artifacts as a mirror 2022-11-16 20:16:32 +01:00
Hans-Christoph Steiner f24613b701
index: fix git-mirror size check for GitLab Pages
The test case had the wrong folder setup, this was confirmed on a production
repo setup.
2022-11-16 20:16:25 +01:00
linsui 5a28f20301 make versionCode/build.timeout an integer
Co-authored-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>
2022-10-14 08:42:26 +00:00
Hans-Christoph Steiner 7544761e86
index: add test for --nosign config and file generation 2022-06-07 13:22:06 +02:00
Jochen Sprickerhof d70e5c2cd9 Index v2 2022-05-23 10:39:17 +00:00
Hans-Christoph Steiner 8ea154c125 fix IndexTest.test_gitlab_get_mirror_service_urls
This makes it actually call the method rather than mocking it.
2021-09-13 13:18:21 +02:00
Hans-Christoph Steiner 7987c746de index: do not include GitLab Pages mirror if it can't be deployed
GitLab Pages sites are limited to 1GB on gitlab.com, so the CI/CD job will
fail if the repo is bigger than that.  It should not be included as a
mirror in that case.

https://docs.gitlab.com/ee/user/gitlab_com/#gitlab-pages
2021-09-11 15:30:55 +00:00
Hans-Christoph Steiner 5267699d81
index: make download_repo_index() handle common URL mistakes 2021-07-01 14:43:58 +02:00
Hans-Christoph Steiner d05ff9db1d
easy changes to black code format in test cases
This does not change areas of code that should be manually reformatted.
2021-06-07 11:53:58 +02:00
Hans-Christoph Steiner dbf80ad771
metadata: remove git repo handling from metadata parsing
This is a vestige of implementing builds from a .fdroid.yml file directly
in the app's source repo.  It was never fully complete and seems to not be
used in any apps in fdroiddata.  This makes `fdroid build --all` runs much
faster since it does not need to do any git handling for apps that do not
have any new builds to run.

4e8e29794
948689281
a4e431080
e37be9b06
e775d330c
2021-03-17 13:44:24 +01:00
Hans-Christoph Steiner 544a45c16a index: raise error rather than crash on bad repo file
If a non-APK is added with the appid/packageName that matches some APKs, it
should through an error.

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/home/hans/code/fdroid/server/fdroidserver/update.py", line 2343, in main
    index.make(apps, sortedids, apks, repodirs[0], False)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 142, in make
    fdroid_signing_key_fingerprints)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 166, in make_v1
    v1_sort_packages(packages, fdroid_signing_key_fingerprints)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 292, in v1_sort_packages
    packages.sort(key=v1_sort_keys)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 288, in v1_sort_keys
    .format(apkfilename=package['apkName']))
fdroidserver.exception.FDroidException: at.roteskreuz.stopcorona_8.jobf does not have a valid signature!
2021-03-09 15:40:30 +01:00
Benedikt Brückmann 9c609aa361 Fix copy-paste error 2021-02-08 12:25:02 +01:00
Benedikt Brückmann b1bde9e407 * Skip tidy check if html5print is not available
* install test dependencies on arch
2021-02-08 12:25:02 +01:00
Benedikt Brückmann 022d73b3b6 Add HTML/CSS tidy test
and tidy up the content
2021-02-08 12:25:02 +01:00
Hans-Christoph Steiner fff59e5197 only use AutoName: in checkupdates
AutoName: is only needed for the commit messages generated by checkupdates,
and it makes the logic for localized names confusing.

closes #654
refs #304
2021-01-14 20:36:09 +01:00
Hans-Christoph Steiner 384922118f index: sanitize fingerprint arg, extract_pubkey() returns with spaces
The key fingerprint should be only hex digits, everything else can be
discarded.  That makes it easy to use this function various fingerprint
formats, including the common, human-readable forms spaces between pairs or
quartets.
2020-12-15 08:30:48 +01:00
Hans-Christoph Steiner ba854cbc0f index: fix GitLab Raw URLs with gitlab.com and recent versions
GitLab seems to be moving to always having "-" as the first path segment in
all the project URLs.  So the URL without a "-" is now a redirect.
2020-12-15 08:30:48 +01:00
Hans-Christoph Steiner 283f10dec1 index: generate repo icon if missing, and add tests 2020-10-07 18:56:17 +02:00
Hans-Christoph Steiner 5fb368916f
index: fix no JAR test case that fails on example.org
http://example.org/index-v1.jar now returns the HTTP header
"Content-Encoding: gzip" but then the reply is plain HTML. That
triggers a ContentDecodingError instead of an HTTPError, so this
changes the test to success on any RequestsException.
2020-02-05 20:33:40 +01:00
Michael Pöhn 37e0dce73b made index.TestCase run independant of cwd 2018-08-14 10:08:03 +02:00
Hans-Christoph Steiner ff90c0246e fix PEP8 W605 invalid escape sequence
Python 3.7 will get a lot stricter with escape sequences.  They must be
valid.

* https://lintlyci.github.io/Flake8Rules/rules/W605.html
* https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior
2018-05-29 13:51:47 +02:00
Hans-Christoph Steiner 43c61aecee fix pylint Unused argument 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner 6228162cbd handle jarsigner/apksigner output cleanly for rational logging
These were both spamming the output with lots of confusing messages, even
when --verbose was not used.  Jarsigner especially has confusing messages,
since it has warnings that do not pertain to APK signatures at all, like
the ones about timestamps and missing Certificate Authority.

closes #405
2017-12-07 17:32:14 +01:00
Hans-Christoph Steiner 5b22ff7dc6 tests: use standard dir setup so all tests start in same dir 2017-11-30 17:32:53 +01:00
Hans-Christoph Steiner 17efa13183 tests: pre-set failfast as reminder of a handy time saver 2017-10-25 23:01:25 +02:00
Michael Pöhn b6b79edf11 store fdroid signing-key-fp-list in stats dir 2017-09-26 14:11:09 +02:00
Michael Pöhn 6930edf889 sort index-v1; publish now creates and stores a list of signature fingerprints 2017-09-26 14:11:09 +02:00
Hans-Christoph Steiner 5dcb48831f index: always use jarsigner for verifying JAR signatures
apksigner v0.7 (build-tools 26.0.1), Google made it require that the
AndroidManifest.xml was present in the archive before it verifies the
signature.  So this needs to stick with the jarsigner hack for JARs.
2017-09-19 20:13:36 +02:00
Torsten Grote 8d424f19ec
Support ETag when downloading repository index 2017-05-02 15:37:02 -03:00
Hans-Christoph Steiner 53b69f507e allow index.TestCase to work when run using any path 2017-04-03 20:24:21 +02:00
Torsten Grote a23da47118
Add method for downloading (and verifying) a repository index
This includes some test cases to test the new code.
2017-04-03 09:02:07 -03:00