This PR contains multiple fixes. The most important of which is:
* Prevent hang in git cat-file if the repository is not a valid repository
Unfortunately it appears that if git cat-file is run in an invalid
repository it will hang until stdin is closed. This will result in
deadlocked /pulls pages and dangling git cat-file calls if a broken
repository is tried to be reviewed or pulls exists for a broken
repository.
Fix#14734Fix#9271Fix#16113
Otherwise there are a few small other fixes included which this PR was initially intending to fix:
* Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes
* Fix links on pulls pages due to regression from #17551 - by making most /issues routes match /pulls too - Fix#17983
* Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix#17943
* Add missing locale entries for oauth group claims
* Prevent NPEs if ColorFormat is called on nil users, repos or teams.
Save a bit of bandwidth by only requesting 3-times the rendered avatar
size. Factor 4 is only really beneficial on a handful of mobile phones
and I don't think they are the primary device we design for.
Configurability contributed by zeripath.
Fixes: https://github.com/go-gitea/gitea/pull/17422
Fixes: https://github.com/go-gitea/gitea/issues/16287
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Add missing `X-Total-Count` and fix some related bugs
Adds `X-Total-Count` header to APIs that return a list but doesn't have it yet.
Fixed bugs:
* not returned after reporting error (39eb82446c/routers/api/v1/user/star.go (L70))
* crash with index out of bounds, API issue/issueSubscriptions
I also found various endpoints that return lists but do not apply/support pagination yet:
```
/repos/{owner}/{repo}/issues/{index}/labels
/repos/{owner}/{repo}/issues/comments/{id}/reactions
/repos/{owner}/{repo}/branch_protections
/repos/{owner}/{repo}/contents
/repos/{owner}/{repo}/hooks/git
/repos/{owner}/{repo}/issue_templates
/repos/{owner}/{repo}/releases/{id}/assets
/repos/{owner}/{repo}/reviewers
/repos/{owner}/{repo}/teams
/user/emails
/users/{username}/heatmap
```
If this is not expected, an new issue should be opened.
Closes#13043
* fmt
* Update routers/api/v1/repo/issue_subscription.go
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
* Use FindAndCount
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: 6543 <6543@obermui.de>
* Add setting to OAuth handlers to override local 2FA settings
This PR adds a setting to OAuth and OpenID login sources to allow the source to
override local 2FA requirements.
Fix#13939
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix regression from #16544
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add scopes settings
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix trace logging in auth_openid
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add required claim options
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move UpdateExternalUser to externalaccount
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Allow OAuth2/OIDC to set Admin/Restricted status
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Allow use of the same group claim name for the prohibit login value
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fixup! Move UpdateExternalUser to externalaccount
* as per wxiaoguang
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add label back in
Signed-off-by: Andrew Thornton <art27@cantab.net>
* adjust localisation
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Running `make test-backend` will delete `data/` due to reloading the configuration and resetting the appdatapath.
This PR removes this unnecessary config reload but also adds extra code in to the unittest main to prevent its cleanup from deleting the wrong directory.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move keys to models/keys
* Rename models/keys -> models/asymkey
* change the missed package name
* Fix package alias
* Fix test
* Fix docs
* Fix test
* Fix test
* merge
* Some refactors related repository model
* Move more methods out of repository
* Move repository into models/repo
* Fix test
* Fix test
* some improvements
* Remove unnecessary function
* Fix a panic in NotifyCreateIssueComment (caused by string truncation)
* more unit tests
* refactor
* fix some edge cases
* use SplitStringAtByteN for comment content
* Refactor install page (db type)
* set correct default DB HOST for different DB TYPE
* remove legacy TiDB from documents
* unify the usage of DB TYPE, in code we only use "mysql". "MySQL" is only shown to users for friendly name.
* Gitea can use TiDB via MySQL protocol
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Check if column exist before rename if exist, just return with no error
* Also check if errors column exist
* Add comment for migration
* Fix sqlite test
* Improve install code to avoid low-level mistakes.
If a user tries to do a re-install in a Gitea database, they gets a warning and double check.
When Gitea runs, it never create empty app.ini automatically.
Also some small (related) refactoring:
* Refactor db.InitEngine related logic make it more clean (especially for the install code)
* Move some i18n strings out from setting.go to make the setting.go can be easily maintained.
* Show errors in CLI code if an incorrect app.ini is used.
* APP_DATA_PATH is created when installing, and checked when starting (no empty directory is created any more).
sshd(8) list restrict as a future-proof way to restrict feature
enabled in ssh. It is supported since OpenSSH 7.2, out since
2016-02-29.
OpenSSH will ignore unknown options (see sshauthopt_parse in
auth-options.c), so it should be safe to add the option and
no-user-rc.
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* More pleasantly handle broken or missing git repositories
In #17742 it was noted that there a completely invalid git repository underlying a
repo on gitea.com. This happened due to a problem during a migration however, it
is not beyond the realms of possibility that a corruption could occur to another
user.
This PR adds a check to RepoAssignment that will detect if a repository loading has
failed due to an absent git repository. It will then show a page suggesting the user
contacts the administrator or deletes the repository.
Fix#17742
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update options/locale/locale_en-US.ini
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Remove unnecessary functions of User struct
* Move more database methods out of user struct
* Move more database methods out of user struct
* Fix template failure
* Fix bug
* Remove finished FIXME
* remove unnecessary code
* Improvements to content history
* initialize content history when making an edit to an old item created before the introduction of content history
* show edit history for code comments on pull request files tab
* Fix a flaw in keepLimitedContentHistory
Fix a flaw in keepLimitedContentHistory, the first and the last should never be deleted
* Remove obsolete eager initialization of content history
Use hostmacher to replace matchlist.
And we introduce a better DialContext to do a full host/IP check, otherwise the attackers can still bypass the allow/block list by a 302 redirection.
* Use a standalone struct name for Organization
* recover unnecessary change
* make the code readable
* Fix template failure
* Fix template failure
* Move HasMemberWithUserID to org
* Fix test
* Remove unnecessary user type check
* Fix test
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
- The code will get the first and second character `link[{0,1]]`.
However in a rare case the `link` could have 1 character and thus the
`link[1]` will create a panic.
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
This is an extensive PR attempting to fix these issues.
1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
* feat: Allow multiple tags on comments
- Allow for multiples tags(Currently Poster + {Owner, Writer}).
- Utilize the Poster tag within the commentTag function and remove the
checking from templates.
- Use bitwise on CommentTags to enable specific tags.
- Don't show poster tag(view_content.tmpl) on the initial issue comment.
* Change parameters naming
* Change function name
* refactor variable wording
* Merge 'master' branch into 'tags-comments' branch
* Change naming
* `tag` -> `role`
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix 500 when a comment was deleted which has a notification
* Tolerate missing Comment in other places too
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This change enables the usage of U2F without being forced to enroll an TOTP authenticator.
The `/user/auth/u2f` has been changed to hide the "use TOTP instead" bar if TOTP is not enrolled.
Fixes#5410Fixes#17495
* Fix stat chunks searching
- Fixes a issue whereby the given chunk of issueIDs wasn't respected and
thus the returned results where not the correct results.
* Add tests
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
- This will only allow `GetDeletedBranchByID` to return deletedBranch
which are on the repo, and thus don't return a deletedBranch from
another repo.
- This just should prevent possible bugs in the futher when a code is
passing the wrong ID into this function.
* Simplify Gothic to use our session store instead of creating a different store
We have been using xormstore to provide a separate session store for our OAuth2 logins
however, this relies on using gorilla context and some doubling of our session storing.
We can however, simplify and simply use our own chi-based session store. Thus removing
a cookie and some of the weirdness with missing contexts.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per review
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per review
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle MaxTokenLength
Signed-off-by: Andrew Thornton <art27@cantab.net>
* oops
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
CountOrphanedObjects needs to quote the table it is joining with as this table may
be `user`.
Fix#17485
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Run Migrate in Install rather than just SyncTables
The underlying problem in #17328 appears to be that users are re-running the install
page during upgrades. The function that tests and creates the db did not intend for
this and thus instead the migration scripts being run - a simple sync tables occurs.
This then causes a weird partially migrated DB which causes, in this release cycle,
the duplicate column in task table error. It is likely the cause of some weird
partial migration errors in other cycles too.
This PR simply ensures that the migration scripts are also run at this point too.
Fix#17328
Signed-off-by: Andrew Thornton <art27@cantab.net>
There is a small bug in the way that repo access is checked in
repoAssignment: Accessibility is checked by checking if the user has a
marked access to the repository instead of checking if the user has any
team granted access.
This PR changes this permissions check to use HasAccess() which does the
correct test. There is also a fix in the release api ListReleases where
it should return draft releases if the user is a member of a team with
write access to the releases.
The PR also adds a testcase.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Improve: make diff result better, make the HTML element fit the full height in the content history diff dialog
* Bug fix: when edit the main issue, the poster is wrongly set to the issue poster
We have the `AppState` module now, it can store app related data easily. We do not need to create separate tables for each feature.
So the update checker can use `AppState` instead of a one-row dedicate table.
And the code of update checker is moved from `models` to `modules`.
Gitea writes its own AppPath into git hook scripts. If Gitea's AppPath changes, then the git push will fail.
This PR:
* Introduce an AppState module, it can persist app states into database
* During GlobalInit, Gitea will check if the current AppPath is the same as last one. If they don't match, Gitea will sync git hooks.
* Refactor some code to make them more clear.
* Also, "Detect if gitea binary's name changed" #11341 is related, we call models.RewriteAllPublicKeys to update ssh authorized_keys file
* Don't panic if we fail to parse a U2FRegistration data
Downgrade logging statement from Fatal to Error so that errors parsing
U2FRegistration data does not panic; instead, the invalid key will be
skipped and we will attempt to parse the next one, if available.
Signed-off-by: David Jimenez <dvejmz@sgfault.com>
* Ensure that git daemon export ok is created for mirrors
There is an issue with #16508 where it appears that create repo requires that the
repo does not exist. This causes #17241 where an error is reported because of this.
This PR fixes this and also runs update-server-info for mirrors and generated repos.
Fix#17241
Signed-off-by: Andrew Thornton <art27@cantab.net>
It makes Admin's life easier to filter users by various status.
* introduce window.config.PageData to pass template data to javascript module and small refactor
move legacy window.ActivityTopAuthors to window.config.PageData.ActivityTopAuthors
make HTML structure more IDE-friendly in footer.tmpl and head.tmpl
remove incorrect <style class="list-search-style"></style> in head.tmpl
use log.Error instead of log.Critical in admin user search
* use LEFT JOIN instead of SubQuery when admin filters users by 2fa. revert non-en locale.
* use OptionalBool instead of status map
* refactor SearchUserOptions.toConds to SearchUserOptions.toSearchQueryBase
* add unit test for user search
* only allow admin to use filters to search users
* issue content history
* Use timeutil.TimeStampNow() for content history time instead of issue/comment.UpdatedUnix (which are not updated in time)
* i18n for frontend
* refactor
* clean up
* fix refactor
* re-format
* temp refactor
* follow db refactor
* rename IssueContentHistory to ContentHistory, remove empty model tags
* fix html
* use avatar refactor to generate avatar url
* add unit test, keep at most 20 history revisions.
* re-format
* syntax nit
* Add issue content history table
* Update models/migrations/v197.go
Co-authored-by: 6543 <6543@obermui.de>
* fix merge
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
It is possible that a keyring can contain duplicate keys on a keyring due to jpegs or
other layers. This currently leads to a confusing error for the user - where we report
a duplicate key insertion.
This PR simply coalesces keys into one key if there are duplicates.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Why this refactor
The goal is to move most files from `models` package to `models.xxx` package. Many models depend on avatar model, so just move this first.
And the existing logic is not clear, there are too many function like `AvatarLink`, `RelAvatarLink`, `SizedRelAvatarLink`, `SizedAvatarLink`, `MakeFinalAvatarURL`, `HashedAvatarLink`, etc. This refactor make everything clear:
* user.AvatarLink()
* user.AvatarLinkWithSize(size)
* avatars.GenerateEmailAvatarFastLink(email, size)
* avatars.GenerateEmailAvatarFinalLink(email, size)
And many duplicated code are deleted in route handler, the handler and the model share the same avatar logic now.
* Nicely handle missing user in collaborations
It is possible to have a collaboration in a repository which refers to a no-longer
existing user. This causes the repository transfer to fail with an unusual error.
This PR makes `repo.getCollaborators()` nicely handle the missing user by ghosting
the collaboration but also adds consistency check. It also adds an
Access consistency check.
Fix#17044
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
