6f9c278559
# ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage:  |
||
|---|---|---|
| .. | ||
| migration-test | ||
| schemas | ||
| admin_user_test.go | ||
| api_activitypub_person_test.go | ||
| api_admin_org_test.go | ||
| api_admin_test.go | ||
| api_branch_test.go | ||
| api_comment_attachment_test.go | ||
| api_comment_test.go | ||
| api_feed_user_test.go | ||
| api_fork_test.go | ||
| api_gitignore_templates_test.go | ||
| api_gpg_keys_test.go | ||
| api_helper_for_declarative_test.go | ||
| api_httpsig_test.go | ||
| api_issue_attachment_test.go | ||
| api_issue_config_test.go | ||
| api_issue_label_test.go | ||
| api_issue_milestone_test.go | ||
| api_issue_reaction_test.go | ||
| api_issue_stopwatch_test.go | ||
| api_issue_subscription_test.go | ||
| api_issue_test.go | ||
| api_issue_tracked_time_test.go | ||
| api_keys_test.go | ||
| api_license_templates_test.go | ||
| api_nodeinfo_test.go | ||
| api_notification_test.go | ||
| api_oauth2_apps_test.go | ||
| api_org_test.go | ||
| api_packages_cargo_test.go | ||
| api_packages_chef_test.go | ||
| api_packages_composer_test.go | ||
| api_packages_conan_test.go | ||
| api_packages_conda_test.go | ||
| api_packages_container_test.go | ||
| api_packages_debian_test.go | ||
| api_packages_generic_test.go | ||
| api_packages_helm_test.go | ||
| api_packages_maven_test.go | ||
| api_packages_npm_test.go | ||
| api_packages_nuget_test.go | ||
| api_packages_pub_test.go | ||
| api_packages_pypi_test.go | ||
| api_packages_rpm_test.go | ||
| api_packages_rubygems_test.go | ||
| api_packages_swift_test.go | ||
| api_packages_test.go | ||
| api_packages_vagrant_test.go | ||
| api_private_serv_test.go | ||
| api_pull_commits_test.go | ||
| api_pull_review_test.go | ||
| api_pull_test.go | ||
| api_releases_test.go | ||
| api_repo_archive_test.go | ||
| api_repo_collaborator_test.go | ||
| api_repo_edit_test.go | ||
| api_repo_file_create_test.go | ||
| api_repo_file_delete_test.go | ||
| api_repo_file_get_test.go | ||
| api_repo_file_helpers.go | ||
| api_repo_file_update_test.go | ||
| api_repo_get_contents_list_test.go | ||
| api_repo_get_contents_test.go | ||
| api_repo_git_blobs_test.go | ||
| api_repo_git_commits_test.go | ||
| api_repo_git_hook_test.go | ||
| api_repo_git_notes_test.go | ||
| api_repo_git_ref_test.go | ||
| api_repo_git_tags_test.go | ||
| api_repo_git_trees_test.go | ||
| api_repo_hook_test.go | ||
| api_repo_languages_test.go | ||
| api_repo_lfs_locks_test.go | ||
| api_repo_lfs_migrate_test.go | ||
| api_repo_lfs_test.go | ||
| api_repo_raw_test.go | ||
| api_repo_tags_test.go | ||
| api_repo_teams_test.go | ||
| api_repo_test.go | ||
| api_repo_topic_test.go | ||
| api_settings_test.go | ||
| api_team_test.go | ||
| api_team_user_test.go | ||
| api_token_test.go | ||
| api_user_email_test.go | ||
| api_user_follow_test.go | ||
| api_user_heatmap_test.go | ||
| api_user_info_test.go | ||
| api_user_org_perm_test.go | ||
| api_user_orgs_test.go | ||
| api_user_search_test.go | ||
| api_user_star_test.go | ||
| api_user_watch_test.go | ||
| api_wiki_test.go | ||
| attachment_test.go | ||
| auth_ldap_test.go | ||
| benchmarks_test.go | ||
| branches_test.go | ||
| change_default_branch_test.go | ||
| cmd_keys_test.go | ||
| compare_test.go | ||
| cors_test.go | ||
| create_no_session_test.go | ||
| csrf_test.go | ||
| delete_user_test.go | ||
| download_test.go | ||
| dump_restore_test.go | ||
| editor_test.go | ||
| empty_repo_test.go | ||
| eventsource_test.go | ||
| explore_repos_test.go | ||
| git_clone_wiki_test.go | ||
| git_helper_for_declarative_test.go | ||
| git_smart_http_test.go | ||
| git_test.go | ||
| goget_test.go | ||
| gpg_git_test.go | ||
| html_helper.go | ||
| incoming_email_test.go | ||
| integration_test.go | ||
| issue_test.go | ||
| lfs_getobject_test.go | ||
| lfs_local_endpoint_test.go | ||
| lfs_view_test.go | ||
| links_test.go | ||
| markup_external_test.go | ||
| migrate_test.go | ||
| mirror_pull_test.go | ||
| mirror_push_test.go | ||
| nonascii_branches_test.go | ||
| oauth_test.go | ||
| org_count_test.go | ||
| org_team_invite_test.go | ||
| org_test.go | ||
| private-testing.key | ||
| privateactivity_test.go | ||
| pull_compare_test.go | ||
| pull_create_test.go | ||
| pull_merge_test.go | ||
| pull_review_test.go | ||
| pull_status_test.go | ||
| pull_update_test.go | ||
| README.md | ||
| README_ZH.md | ||
| release_test.go | ||
| rename_branch_test.go | ||
| repo_activity_test.go | ||
| repo_branch_test.go | ||
| repo_commits_search_test.go | ||
| repo_commits_test.go | ||
| repo_fork_test.go | ||
| repo_generate_test.go | ||
| repo_migrate_test.go | ||
| repo_search_test.go | ||
| repo_tag_test.go | ||
| repo_test.go | ||
| repo_topic_test.go | ||
| repo_watch_test.go | ||
| repofiles_delete_test.go | ||
| repofiles_update_test.go | ||
| setting_test.go | ||
| signin_test.go | ||
| signout_test.go | ||
| signup_test.go | ||
| ssh_key_test.go | ||
| timetracking_test.go | ||
| user_avatar_test.go | ||
| user_test.go | ||
| version_test.go | ||
| view_test.go | ||
| webfinger_test.go | ||
| xss_test.go | ||
Integration tests
Integration tests can be run with make commands for the appropriate backends, namely:
make test-sqlite
make test-pgsql
make test-mysql
make test-mysql8
make test-mssql
Make sure to perform a clean build before running tests:
make clean build
Run all tests via local drone
drone exec --local --build-event "pull_request"
Run sqlite integration tests
Start tests
make test-sqlite
Run MySQL integration tests
Setup a MySQL database inside docker
docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql
Run pgsql integration tests
Setup a pgsql database inside docker
docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql
Run mssql integration tests
Setup a mssql database inside docker
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)
Start tests based on the database container
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql
Running individual tests
Example command to run GPG test:
For SQLite:
make test-sqlite#GPG
For other databases(replace mssql to mysql, mysql8 or pgsql):
TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG
Setting timeouts for declaring long-tests and long-flushes
We appreciate that some testing machines may not be very powerful and the default timeouts for declaring a slow test or a slow clean-up flush may not be appropriate.
You can either:
- Within the test ini file set the following section:
[integration-tests]
SLOW_TEST = 10s ; 10s is the default value
SLOW_FLUSH = 5S ; 5s is the default value
- Set the following environment variables:
GITEA_SLOW_TEST_TIME="10s" GITEA_SLOW_FLUSH_TIME="5s" make test-sqlite