Commit graph

31 commits

Author SHA1 Message Date
Earl Warren
d1dadf4e06
fix(ci): use 512MB for elasticsearch instead of 32GB
It can be observed that when elasticsearch runs in the CI the process
uses 32GB of resident memory.

Set Xmx and Xms to no more than the threshold that the JVM uses for
compressed object pointers (compressed oops); the exact threshold
varies but is near 32 GB. You can verify that you are under the
threshold by looking for a line in the logs like the following:

Refs: https://www.elastic.co/guide/en/elasticsearch/reference/7.5/docker.html
Refs: https://www.elastic.co/guide/en/elasticsearch/reference/7.5/heap-size.html
2024-07-16 08:10:58 +02:00
Elias Elwyn
f3f6bc5b33
test: add valkey to remote cache test 2024-07-09 14:26:46 +10:00
Earl Warren
ee7f568aed
fix(ci): do not block pull request CI when go-versions lags behind
When a new go version is published, it takes about 24h for
https://github.com/actions/go-versions to be updated (see
https://github.com/actions/go-versions/pull/102 for example).

In the meantime the setup-go action that depends on it will install a
version of go that fails golang.org/x/vuln/cmd/govulncheck.

Move the security check to be the last step of the test job instead of
the first. It will still block the PRs from being merged but it will
allow the PR authors to keep working and look at the test results in
the meantime.

Fixes: https://codeberg.org/forgejo/forgejo/issues/4294
2024-07-03 15:57:35 +02:00
Earl Warren
536efeb26e
test(js): make test-frontend-coverage displays vitest coverage 2024-06-24 10:03:42 +02:00
Elias Elwyn
abde31148f Integration test for redis alternatives - garnet, redict (#4138)
For #4082.

~~Per the discussion in the issue, the current plan will likely involve duplicating the redis library calling code once for each cacher, as neither garnet nor redict guarantee continued compatibility with redis.~~

See discussion below for details.

## Tasklist

- [x] Write workflow to run cache-specific unit test(s) only (cache, session, queue, nosql) for each cacher
- [x] Check whether garnet and redict pass unit tests with no code modification (gauge required work)
  - both passed, but that is because there were very few tests that test the remote cache store

### Out of scope for this PR

- Improve test coverage
  - `modules/cache` against a server
  - `modules/session` against a server (also needs tests in general)
- _(?) Duplicate implementation for each cacher_
  - _Restructure redis usage in `modules/cache` and `modules/settings/cache`_
  - _Restructure `modules/session` and its settings_
  - _Restructure `modules/queue` and its settings_
  - _Restructure `modules/nosql` and its settings_

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4138
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Elias Elwyn <a@jthv.ai>
Co-committed-by: Elias Elwyn <a@jthv.ai>
2024-06-18 12:31:15 +00:00
Renovate Bot
6e6c2aadd7 Update elasticsearch Docker tag to v7.17.22 2024-06-14 00:02:49 +00:00
Renovate Bot
c071bdaf96 Update elasticsearch Docker tag to v7.17.21 2024-06-05 00:05:10 +00:00
Mai-Lapyst
e58ce86264 Update the elasticsearch mappings for issues to use long instead of integer for int64 (#3982)
This updates the mapping definition of the elasticsearch issue indexer backend to use `long` instead of `integer`s wherever the go type is a `int64`. Without it larger instances could run into an issue.

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3982
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mai-Lapyst <mai-lapyst@noreply.codeberg.org>
Co-committed-by: Mai-Lapyst <mai-lapyst@noreply.codeberg.org>
2024-06-04 07:37:59 +00:00
Earl Warren
98be7826a3
chore(dependency): pin redis to 7.2 for testing
There is no need to pin a specific patch version for testing. The
worst that can happen in this context is that the CI fails and it can
be addressed in this context. It will not impact releases.
2024-05-25 08:25:51 +02:00
Renovate Bot
237362ddd3 Update redis Docker tag to v7.2.5 2024-05-25 00:05:17 +00:00
Victoria Nadasdi
df0d1a2134 feat: parse prefix from redis URI for queues (#3836)
For security reasons, scoping access to a redis server via ACL rules is
a good practice. Some parts of the codebase handles prefix like cache[^1]
and session[^2], but the queue module doesn't.

This patch adds this missing functionality to the queue module.

Note about relevant test:
I tried to keep the PR as small as possible (and reasonable), and not
change how the test runs. Updated the existing test to use the same
redis address and basically duplicated the test with the extra flag. It
does NOT test if the keys are correct, it ensures only it works as
expected. To make assertions about the keys, the whole test has to be
updated as the general wrapper doesn't allow the main test to check
anything provider (redis) specific property. That's not something I
wanted to take on now.

[^1]: e4c3c039be/modules/cache/cache_redis.go (L139-L150)
[^2]: e4c3c039be/modules/session/redis.go (L122-L129)

Signed-off-by: Victoria Nadasdi <victoria@efertone.me>

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3836
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Victoria Nadasdi <victoria@efertone.me>
Co-committed-by: Victoria Nadasdi <victoria@efertone.me>
2024-05-20 14:10:54 +00:00
Earl Warren
aa66d0a0eb
test(ldap): add LDAP tests 2024-04-24 22:06:32 +02:00
Earl Warren
dbdab1bc60 Merge pull request 'Update bitnami/minio Docker tag to v2024.3.30' (#2917) from renovate/bitnami-minio-2024.x into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2917
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
2024-04-06 07:20:08 +00:00
Earl Warren
262533570f
Revert "[CI] pin go v1.22.2 version"
The golang action now knows about the latest Go version.

This reverts commit 3509242ee6.
2024-04-05 05:31:00 +02:00
Earl Warren
3509242ee6
[CI] pin go v1.22.2 version
Because setup-go fails to pick it up. It usually happens within 24h
and this commit can be reverted then.
2024-04-04 06:45:48 +02:00
Earl Warren
7aa686d979
[CI] backport: show event information for debug purposes
It will help figure out why two runs of testing happen when a backport
PR is open.

Refs: https://codeberg.org/forgejo/forgejo/pulls/2922
Refs: https://codeberg.org/forgejo/forgejo/issues/2009
2024-03-31 07:27:23 +02:00
Renovate Bot
7ec39aa756 Update bitnami/minio Docker tag to v2024.3.30 2024-03-31 02:07:27 +00:00
Earl Warren
84f28f36b0
go: upgrade to go v1.22 2024-03-25 06:52:23 +01:00
oliverpool
98f1fe7944 Makefile: check git diff exit-code (#2651)
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2651
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-03-14 10:38:58 +00:00
jilen
f4fd81b5d1 Add minio bucket lookup type option 2024-03-06 09:38:02 +00:00
Earl Warren
dc354ae0f7
Revert "[CI] pin go v1.21.8 version"
This reverts commit ac0f58035f.
2024-03-06 12:14:44 +08:00
Earl Warren
ac0f58035f
[CI] pin go v1.21.8 version
Because setup-go fails to pick it up.

Refs: https://github.com/actions/setup-go/issues/462
2024-03-06 11:38:47 +08:00
Earl Warren
5d2c4706d0
[CI] run frontend checks 2024-02-25 23:08:09 +01:00
oliverpool
ffc94d9463 remove debug print 2024-02-09 11:27:58 +01:00
oliverpool
cab17d7a72 DEBUG: -j 2024-02-09 11:20:15 +01:00
oliverpool
01539730c0 CI: merge checks-backend and lint-backend 2024-02-09 11:08:19 +01:00
Earl Warren
a89cb638a0
[CI] pin go 1.21 2024-02-07 12:19:56 +01:00
Earl Warren
3f9b7cb892
[CI] Forgejo Actions based CI for PR & branches (squash) install git >= 2.42
(cherry picked from commit 2e43bd580d)
(cherry picked from commit f4a21a873c)
2024-02-05 13:33:59 +01:00
Earl Warren
82395a1f72
[CI] Forgejo Actions based release process (squash) refactor (#2053)
Refs: https://codeberg.org/forgejo/forgejo/issues/2051
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2053
Reviewed-by: Loïc Dachary <dachary@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
(cherry picked from commit 01abf4d505)

[CI] Forgejo Actions based release process (squash) no ownca

The private Forgejo instance trusted with the release signing keys no
longer requires the installation of the ownca certificate authority.

Refs: https://codeberg.org/forgejo/docs/pulls/338
(cherry picked from commit 72f9ae796d)
(cherry picked from commit b0ca4236d7)
(cherry picked from commit 81619cf8b5)
(cherry picked from commit 2cb32c1a2b)
(cherry picked from commit f817d97f85)
(cherry picked from commit 7499661326)
2024-02-05 13:33:59 +01:00
Gusted
40d852d12c
[CI] Switch to non-coverage unit test (squash)
- Currently the collected coverage is not being used by the CI, so use
the non-coverage variant to not have the overhead of collecting
coverage.

(cherry picked from commit 9b3032c14b)
(cherry picked from commit b0fded409d)
(cherry picked from commit 14b5efb1c4)
(cherry picked from commit 30bd28eed3)
(cherry picked from commit 54bb708ce7)
(cherry picked from commit f1664c360b)
(cherry picked from commit cd8648178d)
(cherry picked from commit 69894fbc82)
(cherry picked from commit 314ae5e183)
2024-02-05 13:33:59 +01:00
Earl Warren
0dbcf3a3e8
[CI] Forgejo Actions based CI for PR & branches
(cherry picked from commit f9d75d4705)
(cherry picked from commit 64f76f4ab2)
(cherry picked from commit 5d02454155)

[CI] Forgejo Actions workflows

(cherry picked from commit 3ff59b5379)
(cherry picked from commit 8af826a6f7)
(cherry picked from commit d7c09d9cc8)

[CI] use the docker label instead of ubuntu-latest

(cherry picked from commit b6a6470db6)

[CI] all tests need compliance before proceeding

(cherry picked from commit b35c496f2c)
(cherry picked from commit 36a4148a8e)
(cherry picked from commit 7ffcffa653)
(cherry picked from commit 8a246d296e)
(cherry picked from commit dd0b6e1826)

[CI] Forgejo Actions based release process (squash) MySQL optimization

Refs: https://codeberg.org/forgejo/forgejo/issues/976
(cherry picked from commit b4b8c489e6)
(cherry picked from commit 1e861db4af)
(cherry picked from commit a6c0e00330)
(cherry picked from commit f97b336465)
(cherry picked from commit 6d65d5f0d6f798556a8f9e547896be03a5ee2f87)
(cherry picked from commit 79bfbadbed)
(cherry picked from commit e86c40a34a)
(cherry picked from commit da0c454adb)
(cherry picked from commit b49d892cda)

[CI] enable minio tests

(cherry picked from commit 4d8f438031)
(cherry picked from commit c4eeb0a61e)

[CI] Forgejo Actions based CI for PR & branches (squash) cleanup

(cherry picked from commit 80eb20e842)
(cherry picked from commit d2ff589858)
(cherry picked from commit f6eedecb67)
(cherry picked from commit cf458091e2)
(cherry picked from commit ddd322cb2d)
(cherry picked from commit f0f5729b64)

Conflicts:
	.github/workflows/pull-db-tests.yml
	https://codeberg.org/forgejo/forgejo/pulls/1573
(cherry picked from commit bb347aedd4)
(cherry picked from commit 0f5ebe8c3e)
(cherry picked from commit 9101cb4715)
(cherry picked from commit ab118fe4ef)

[CI] Forgejo Actions based CI for PR & branches (squash) use node:20-bookworm

No longer use the custom test-env image, it is unecessary technical
debt.

Also upgrade to bitnami/minio:2023.8.31 to align with what Gitea tests

(cherry picked from commit d9b77fd273)
(cherry picked from commit ef8f099f9e)
(cherry picked from commit 6d3c675d20)
(cherry picked from commit fc00ff45f2)
(cherry picked from commit b7f02b9846)
(cherry picked from commit d23d86e567)
(cherry picked from commit e68e65460c)
(cherry picked from commit 96f93a1fdb)
(cherry picked from commit 2ff3080018)
(cherry picked from commit 8ea00bcff4)
(cherry picked from commit 85be961970)
(cherry picked from commit 387b279fc3)
(cherry picked from commit 219be7aa85)
(cherry picked from commit 929b75ee39)
(cherry picked from commit dd42a30d6a)
2024-02-05 13:33:59 +01:00