* Simplify Boost/Pause logic
#18658 has added a check to see if we need to boost because there is still work to do
however the check is slightly complex and not ideal. There's no point boosting if
the queue is paused or can't scale. Therefore merge the two selects into one and add
a check to p.paused.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* And on resume add a zeroboost if necessary
Signed-off-by: Andrew Thornton <art27@cantab.net>
* simplify
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Restart zero worker if there is still work to do
It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.
Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.
Fix#18607
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove unnecessary web context data fields, and unify the i18n/translation related functions to `Locale`
* in development, show an error if a translation key is missing
* remove the unnecessary loops `for _, lang := range translation.AllLangs()` for every request, which improves the performance slightly
* use `ctx.Locale.Language()` instead of `ctx.Data["Lang"].(string)`
* add more comments about how the Locale/LangType fields are used
When a net.OpError occurs during rendering the underlying connection is essentially
dead and therefore attempting to render further data will only cause further errors.
Therefore in serverErrorInternal detect if the passed in error is an OpError and
if so do not attempt any further rendering.
Fix#18629
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Only attempt to flush queue if the underlying worker pool is not finished
There is a possible race whereby a worker pool could be cancelled but yet the
underlying queue is not empty. This will lead to flush-all cycling because it
cannot empty the pool.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
- Switch to use `CryptoRandomBytes` instead of `CryptoRandomString`, OAuth's secrets are copied pasted and don't need to avoid dubious characters etc.
- `CryptoRandomBytes` gives  `CryptoRandomString` gives  possible states.
- Add a prefix, such that code scanners can easily grep these in source code.
- 32 Bytes + prefix
* Collaborator trust model should trust collaborators
There was an unintended regression in #17917 which leads to only
repository admin commits being trusted. This PR restores the old logic.
Fix#18501
Signed-off-by: Andrew Thornton <art27@cantab.net>
* COrrect use `UserID` in `SearchTeams`
- Use `UserID` in the `SearchTeams` function, currently it was useless
to pass such information. Now it does a INNER statement to `team_user`
which obtains UserID -> TeamID data.
- Make OrgID optional.
- Resolves#18484
* Seperate searching specific user
* Add condition back
* Use correct struct type
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* add test coverage for original author conversion during migrations
And create a function to factorize a code snippet that is repeated
five times and would otherwise be more difficult to test and maintain
consistently.
Signed-off-by: Loïc Dachary <loic@dachary.org>
* fix variable scope and int64 formatting
* add missing calls to remapExternalUser and fix misplaced %d
Co-authored-by: Loïc Dachary <loic@dachary.org>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Don't panic & allow shorter sha1
- Don't panic when the full regex isn't matched and allow the usage of a
shorter sha1 being used.
- Resolves#18471
* Update modules/markup/html.go
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Several users run Gitea in situations whereby `bash` is not available.
If the `SCRIPT_TYPE` is not changed this will cause hooks to fail.
A simple test to check if the provided type is on the PATH should be
sufficient to warn them about this problem.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Change some logging levels
* PlainTextWithBytes - 4xx/5xx this should just be TRACE
* notFoundInternal - the "error" here is too noisy and should be DEBUG
* WorkerPool - Worker pool scaling messages are normal and should be DEBUG
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Ensure git tag tests and other create test repos in tmpdir
There are a few places where tests appear to reuse testing repos which
causes random CI failures.
This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.
Fix#18444
* Change log root for integration tests to use the REPO_TEST_DIR
There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.
Fix#18077
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Attempt to prevent the deadlock in the QueueDiskChannel Test again
This time we're going to adjust the pause tests to only test the right
flag.
* Only switch off pushback once we know that we are not pushing anything else
* Ensure full redirection occurs
* More nicely handle a closed datachan
* And handle similar problems in queue_channel_test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent deadlocks in persistable channel pause test
Because of reuse of the old paused/resumed channels in this test there
was a potential for deadlock. This PR ensures that the channels are always
reobtained.
It further adds some control code to detect hangs in future - and it
ensures that the pausing warning is not shown on shutdown.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* do not warn but do pause
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Pass the Global command args into serviceRPC.
- Fixes error with partial cloning.
- Add partial clone test
- Include diff
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Start adding mechanism to return unhandled data
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create pushback interface
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add Pausable interface to WorkerPool and Manager
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Implement Pausable and PushBack for the bytefifos
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Wire in UI for pausing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add testcases and fix a few issues
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix build
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prevent "race" in the test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix jsoniter mismerge
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix conflicts
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix format
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add warnings for no worker configurations and prevent data-loss with redis/levelqueue
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use StopTimer
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
- Disable the browser's function to "sniff" for the content-type on the
provided plain text, this will prevent the possible usage of
user-controlled data being sent, which could be malicious.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Add config option to hide issue events
Adds a config option `HIDE_ISSUE_EVENTS` to hide most issue events (changed labels, milestones, projects...) on the issue detail page.
If this is true, only the following events (comment types) are shown:
* plain comments
* closed/reopned/merged
* reviews
* Make configurable using a list
* Add docs
* Add missing newline
* Fix merge issues
* Allow changes per user settings
* Fix lint
* Rm old docs
* Apply suggestions from code review
* Use bitsets
* Rm comment
* fmt
* Fix lint
* Use variable/constant to provide key
* fmt
* fix lint
* refactor
* Add a prefix for user setting key
* Add license comment
* Add license comment
* Update services/forms/user_form_hidden_comments.go
Co-authored-by: Gusted <williamzijl7@hotmail.com>
* check len == 0
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Make router logger more friendly, show the related function name/file/line.
[BREAKING]
This PR substantially changes the logging format of the router logger. If you use this logging for monitoring e.g. fail2ban you will need to update this to match the new format.
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.
This now means that the if there is a git repo already open in the context it will be used instead of reopening it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
