Commit graph

116 commits

Author SHA1 Message Date
Drew DeVault
f8a1b753a1 Upgrade to Alpine 3.20 2024-08-05 15:18:50 +02:00
Conrad Hoffmann
17c693c6c1 Makefile: add target for pagessrht-daily 2024-05-14 12:53:42 +02:00
Conrad Hoffmann
f96af1f505 Fix build
This seems to have been an oversight in 2c81652.
2024-05-14 12:48:31 +02:00
Conrad Hoffmann
0957bf1ffc Make cleanup script a daily cron job
The only sensitive way to clean up old pages (given the current
architecture) is currently to schedule a bucket policy. The bucket
policy evaluation happens at a low level API and will delete even huge
amounts of data with little impact on performance.

The program will find the first 1000 obsolete site versions and schedule
a policy with rules to remove those. This is far from perfect, and but a
decent solution for now, and the learning from trying to implement the
cleanup in various ways will be reflected in future updates to the pages
& storage architecture.
2024-04-11 16:39:53 +02:00
Simon Ser
2c81652d14 Use slice to store CSP entries
No change in behavior.
2024-04-03 14:01:06 +02:00
Paper
bd32bcbdcd server.go: add allow-downloads directive to CSP 2024-04-03 13:58:00 +02:00
Drew DeVault
6b0b807f3f .builds: drop arch linux 2024-03-20 14:47:12 +01:00
Drew DeVault
153b78e553 Upgrade to Alpine 3.19 2024-03-20 14:46:36 +01:00
Conrad Hoffmann
48e9988e32 contrib/cleanup: fix wait in wrong place 2024-02-19 16:22:08 +01:00
Conrad Hoffmann
68e414ae5d contrib/cleanup: use lifecycle for efficiency
Deleting individual objects is painfully slow. Instead, use a lifecycle
policy rule to expire a given prefix. The actual deleting will then be
done by internal mechanisms, which are vastly more efficient.
2024-02-19 16:12:06 +01:00
Conrad Hoffmann
ce6df7d02d contrib: add tool to purge old sites from storage 2024-02-14 10:14:33 +01:00
Knut Magnus Aasrud
6660d3d516 server.go: add data: to img-src CSP 2024-02-13 18:15:59 +01:00
Simon Ser
554b986f6a Use core-go/s3.NewClient 2024-01-31 08:54:22 +01:00
Simon Ser
5de8d00b22 Replace auth middleware hack with auth.RequireMiddleware 2024-01-31 08:54:21 +01:00
Simon Ser
3e05c6ea66 Upgrade core-go 2024-01-31 08:54:21 +01:00
Drew DeVault
fef1f3166d server.go: add style-src to CSP 2024-01-29 11:24:51 +01:00
Drew DeVault
27e1f6de84 server.go: loosen Content Security Policy
The following changes apply:

- Media (img, audio, video) can be embedded from any HTTPS origin
- iframes can be embedded from any https origin
- Scripts & workers are still limited to same origin
2024-01-29 10:33:58 +01:00
Simon Ser
aa82be3122 Check StatObject error for 404
We were returning 404s for any kind of error. Instead, check whether
the S3 object exists.
2024-01-26 17:15:57 +01:00
Conrad Hoffmann
d70bfd1812 Assert error is not nil before using it 2024-01-23 10:40:15 +01:00
Simon Ser
824924a54e Drop unnecessary recover calls 2024-01-17 19:33:50 +01:00
Simon Ser
36dd5ae168 go get -u 2024-01-17 12:51:40 +01:00
Logan Garcia
641147e6e2 Add index.xhtml as directory index 2023-12-04 12:11:40 +01:00
Samuel Tardieu
8aa96a9f9b Use index.xml as directory index if index.html is absent
Some projects prefer to use XML files instead of HTML ones.

Discussion: <https://lists.sr.ht/~sircmpwn/sr.ht-discuss/%3CDE58BBD2-2381-491E-878A-525A876DACBA%40jonmsterling.com%3E>
Signed-off-by: Samuel Tardieu <sam@rfc1149.net>
2023-11-30 11:56:16 +01:00
Conrad Hoffmann
5f27cce364 Upgrade core-go and chi to v5 2023-11-30 11:53:50 +01:00
Conrad Hoffmann
d0fceafcb9 Update to latest version of core-go
Mainly to pull in the "bad driver" mitigation:
https://git.sr.ht/~sircmpwn/core-go/commit/a06a624
2023-11-16 12:39:10 +01:00
Adnan Maolood
f50f0f54b8 server: Use hostname to lookup Gemini site objects
The server will no longer return 'File not found' for Gemini requests
that include a port in the URL.
2023-11-16 12:36:48 +01:00
Simon Ser
4dc1b007b5 Add PROXY protocol support to HTTP server
This unlocks HTTP/2 support.
2023-10-24 13:31:43 +02:00
Knut Magnus Aasrud
ab98e8f0e9 server.go: add CORS support 2023-10-24 13:31:33 +02:00
Simon Ser
35131fd2f9 graph: fix invalid @access directive in WebhookEvent enum
Fixes the following schema error:

    Directive access is not applicable on ENUM_VALUE.
2023-10-24 13:30:24 +02:00
Umar Getagazov
ce815c651e Allow for more advanced glob matching
This allows to use recursive patterns like "**/*.txt". Go's path.Match
only allows explicit nesting, so to accomplish the same thing you'd need
to duplicate the same pattern multiple times with different amount of
asterisks ("*.txt", "*/*.txt", "*/*/*.txt"), depending on how deep your
site's directory structure is.
2023-07-07 09:58:39 +02:00
Alexey Yerin
b87d8cace0 publish: Drop files variable
It's not used anymore.
2023-06-22 10:40:24 +02:00
Alexey Yerin
438d3665f1 Add subdir to final path when publishing
This was removed in 56d1db33 and broke subdirectory publishing.
2023-06-22 10:40:22 +02:00
Conrad Hoffmann
56d1db3320 Upload files only once to final destination in S3
Currently, files are first uploaded to a temporary location as the hash
of all content is required to compute their final location.
Unfortunately, the move that occurs afterwards has copy & delete
semantics, so it can take a lot of requests and time for sites with many
files, even causing timeouts.

With this commit, the final location is simply determined by generating
a new UUID, so files can be uploaded right to it. The only downside is
that an update will happen even if you upload the same site twice, but
given that this really has no user-facing effect and the work done
before this gets handled is already significant anyways, the benefits
outweigh it by a margin.

Cuts publish time for sites with many files to roughly a third.
2023-06-01 12:43:57 +02:00
Conrad Hoffmann
3b59767b86 Remove allow-orientation-lock from CSP sandbox
It has been deprecated and is no longer present in CSP levels 2 and 3.

See e.g. https://www.w3.org/TR/CSP2/#directive-sandbox
2023-05-24 08:32:57 +02:00
Drew DeVault
bd296fe6f5 server.go: add allow-popup
This enables target=_blank, though I have my reservations we shouldn't
just uniformly block it.
2023-05-16 13:05:13 +02:00
Drew DeVault
ab345188ab Upgrade to Alpine 3.17 2023-03-22 12:21:26 +01:00
Simon Ser
258b15b56b Add optional toplevel info URL
Similar to sr.ht::site-info, but for pages.sr.ht.
2023-02-22 12:13:25 +01:00
Conrad Hoffmann
aefdf8fd0d Fix local IP lookup
The `Host` part of any URI can contain a port, which does not go down
well with the resolver. Use `Hostname()` instead.

See https://pkg.go.dev/net/url#URL.Hostname
2022-12-19 13:23:01 +01:00
Drew DeVault
1575ebd471 Implement user account deletion 2022-11-01 16:48:22 +01:00
Adnan Maolood
7386926d08 Use canonical user IDs
Add contrib scripts to update user IDs across pages.sr.ht to match those
of meta.sr.ht.
2022-10-26 10:49:15 +02:00
Drew DeVault
d8f92a8c6a Upgrade gqlgen 2022-10-20 15:16:41 +02:00
Adnan Maolood
40956cf754 graph: Move custom function to resolver.go
Custom functions can't be placed in schema.resolvers.go, otherwise
gqlgen will move them to the bottom of the file when regenerating the
graph.
2022-08-15 15:45:43 +02:00
xander johnson
0c43f79de3 copy the default-src content security policy to worker-src and add blob 2022-07-26 13:43:18 +02:00
Simon Ser
231713c04a Check custom domain DNS records
Refuse to publish a site for a custom domain if the DNS records
aren't set up properly. This should make it easier for users to
understand why a site doesn't work.
2022-07-07 13:21:18 +02:00
Drew DeVault
c5ff3a5d29 Update core-go 2022-06-30 12:35:14 +02:00
Adnan Maolood
dcb827b016 graph: Implement GraphQL-native user webhooks
Implement GraphQL-native user webhooks for site publish and unpublish
events.
2022-06-29 17:15:52 +02:00
Drew DeVault
5790f96787 model.Site: fix table name
Is double quoted otherwise
2022-06-17 12:47:46 +02:00
Simon Ser
25fb6086f0 Upgrade dependencies
Brings gqlgen 0.17.6 which has some multipart file upload fixes.
2022-05-30 13:39:13 +02:00
Conrad Hoffmann
57c9c0b31e Fixup for f209b016
Something went wrong when applying f209b016 and it is missing most of
the changes that were intended. This commit is a fixup adding all those
missing changes. See the commit message of f209b016 for details.
2022-04-28 11:15:18 +02:00
Drew DeVault
92f5bb8e98 Fix makefile & gitignore 2022-04-28 11:09:04 +02:00