conduit/docs/differences.md

list of features, bug fixes, etc that conduwuit does that upstream does not:

• GitLab CI ported to GitHub Actions
• Fixed every single clippy (default lints) and rustc warnings, including some that were performance related or potential safety issues / unsoundness
• Add a lot of other clippy and rustc lints and a rustfmt.toml file
• Has Renovate and significantly updates all dependencies possible
• Uses proper argon2 crate instead of questionable rust-argon2 crate
• Improved and cleaned up logging (less noisy dead server logging, registration attempts, more useful troubleshooting logging, etc)
• Attempts and interest in removing extreme and unnecessary panics/unwraps/expects that can lead to denial of service or such (upstream and upstream contributors want this unusual behaviour for some reason)
• Merged and cleaned up upstream MRs that have been sitting for 6-12 months
• Configurable RocksDB logging (LOG files) with proper defaults (rotate, max size, verbosity, etc) to stop LOG files from accumulating so much
• Federated presence support and configurable local presence (via upstream MR)
• Concurrency support for key fetching for faster remote room joins and room joins that will error less frequently (via upstream MR)
• Room version 11 support (via upstream MR)
• Config option to allow guest registrations
• Explicit startup error/warning if your configuration allows open registration without a token or such like Synapse
• Improved RocksDB defaults to use new features that help with performance significantly, uses settings tailored to SSDs, various ways to tweak RocksDB, and a conduwuit setting to tell RocksDB to use settings that are tailored to HDDs or slow spinning rust storage.
• Updated Ruma to latest commit where possible, and add some unstable MSCs (some still require an implementation though)
• Revamped admin room infrastructure and commands (via upstream MR)
• Admin room commands to delete room aliases and unpublish rooms from our room directory (via upstream MR)
• Make spaces/hierarchy cache use cache_capacity_modifier instead of hardcoded small value
• Add optional feature flag to use SHA256 key names for media instead of base64 to overcome filesystem file name length limitations (OS error file name too long) (via upstream MR)
• Add optional feature flag to enable zstd HTTP body compression
• Add support for querying both Matrix SRV records, the deprecated _matrix record and _matrix-fed record if necessary
• Add config option for device name federation with a privacy-friendly default (disabled)
• Add config option for requiring authentication to the /publicRooms endpoint (room directory) with a default enabled for privacy
• Add config option for federating /publicRooms endpoint (room directory) to other servers with a default disabled for privacy
• Add support for listening on a UNIX socket for performance and host security with proper default permissions (660)
• Add missing destination key to all X-Matrix Authorization requests (spec compliance issue)
• Use aggressive build-time performance optimisations for release builds (1 codegen unit, no debug, fat LTO, etc, and optimise all crates with same)
• Raise various hardcoded timeouts in codebase that were way too short, making some things like room joins and client bugs error less or none at all than they should
• Add debug admin command to force update user device lists (could potentially resolve some E2EE flukes) (ForceDeviceListUpdates)
• Declare various missing Matrix versions and features at /_matrix/client/versions
• Add support for serving server and client well-known files from conduwuit using well_known_client and well_known_server options
• Add non-standard sliding sync proxy health check (?) endpoint at /client/server.json that some clients such as Element Web query using the well_known_client or well_known_server config options
• Send a User-Agent on all of our requests (conduwuit/0.7.0-alpha+conduwuit-0.1.1) which strangely was not done upstream since forever. Some providers consider no User-Agent suspicious and block said requests.
• Safer and cleaner shutdowns on both database side as we run cleanup on shutdown and exits database loop better (no potential hanging issues in database loop), overall cleaner shutdown logic
• Allow HEAD HTTP requests in CORS for clients (despite not being explicity mentioned in Matrix spec, HTTP spec says all HEAD requests need to behave the same as GET requests, Synapse supports HEAD requests)
• Purge unmaintained/irrelevant/broken database backends (heed, sled, persy)
• webp support for images
• Support for suggesting servers to join at /_matrix/client/v3/directory/room/{roomAlias}
• Prevent admin credential commands like reset password and deactivate user from modifying non-local users (https://gitlab.com/famedly/conduit/-/issues/377)
• Fixed spec compliance issue with room version 8 - 11 joins (https://github.com/matrix-org/synapse/issues/16717 / https://github.com/matrix-org/matrix-spec/issues/1708)
• Add basic cache eviction for true destinations when requests fail if we use a cached destination (e.g. a server has modified their well-known and we're still connecting to the old destination)
• Generate passwords with 25 characters instead of 15
• Add missing reason field to user ban events (/ban)
• For all /report requests: check if the reported event ID belongs to the reported room ID, raise report reasoning character limit to 750, fix broken formatting, make a small delayed random response per spec suggestion on privacy, and check if the sender user is in the reported room.
• Support blocking servers from downloading remote media from
• Support sending well_known response to client logins if using config option well_known_client
• Send avatar_url on invite room membership events/changes
• Revamp example config, adding a lot of config options available (still some missing)
• Return joined member count of rooms for push rules/conditions instead of a hardcoded value of 10
• Respect most client parameters for /media/ requests (allow_redirect still needs work)
• Config option ip_range_denylist to support refusing to send requests (typically federation) to specific IP ranges, typically RFC 1918, non-routable, testnet, etc addresses like Synapse for security (note: this is not a guaranteed protection, and you should be using a firewall with zones if you want guaranteed protection as doing this on the application level is prone to bypasses).
• Support for creating rooms with custom room IDs like Maunium Synapse (room_id request body field to /createRoom)
• Assume well-knowns are broken if they exceed past 10000 characters.
• Basic validation/checks on user-specified room aliases and custom room ID creations
• Warn on unknown config options specified
• Add support for preventing certain room alias names and usernames using regex (via upstream MR) and extended to custom room IDs
• URL preview support (via upstream MR) with various improvements
• Increased graceful shutdown timeout from a low 60 seconds to 180 seconds to avoid killing connections and let the remaining ones finish processing, and ask systemd for more time to shutdown if needed to prevent systemd's default TimeoutStopSec= of 90 seconds from killing conduwuit
• Bumped default max_concurrent_requests to 500
• Query parameter ?format=event|content for returning either the room state event's content (default) for the full room state event on /_matrix/client/v3/rooms/{roomId}/state/{eventType}[/{stateKey}] requests (see https://github.com/matrix-org/matrix-spec/issues/1047)
• Add admin commands for banning (blocking) room IDs from our local users joining (admins are always allowed) and evicts all our local users from that room, in addition to bulk room banning support, and blocks room invites (remote and local) to the banned room, as a moderation feature
• Add admin command to delete media via a specific MXC. This deletes the MXC from our database, and the file locally.
• Replace the lightning bolt emoji option with support for setting any arbitrary text (e.g. another emoji) to suffix to all new user registrations
• Add admin command to bulk delete media via a codeblock list of MXC URLs.
• Add admin command to delete both the thumbnail and media MXC URLs from an event ID (e.g. from an abuse report)
• Add !admin as a way to call the Conduit admin bot
• Add support for listening on multiple TCP ports
• Add admin command to list all the rooms a local user is joined in
• Add admin command to delete all remote media in the past X minutes as a form of deleting media that you don't want on your server that a remote user posted in a room
• Config option to block non-admin users from sending room invites or receiving remote room invites. Admin users are still allowed.
• Startup check if conduwuit running in a container and is listening on 127.0.0.1
• Make CONDUIT_CONFIG optional, relevant for container users that configure only by environment variables and no longer need to set CONDUIT_CONFIG to an empty string.
• Config option to change Conduit's behaviour of homeserver key fetching (query_trusted_key_servers_first). This option sets whether conduwuit will query trusted notary key servers first before the individual homeserver(s), or vice versa.
• Implement database flush and cleanup Conduit operations when using RocksDB
• Implement legacy Matrix /v1/ media endpoints that some clients and servers may still call
• Commandline argument to specify the path to a config file
• Admin debug command to fetch a PDU from a remote server and inserts it into our database/timeline
• Update rusqlite/sqlite (not that you should be using it)
• Disable update check by default as it's not useful for conduwuit
• Declare various missing server capabilities at /_matrix/client/v3/capabilities which also fixes FluffyChat password resets, and other clients that expose basic features based on this endpoint's response
• Config option to disable incoming remote read receipts if desired
• Extend clear cache admin command to support clearing DNS and TLS name override caches
• Responsive outgoing read receipt EDU support
• Eliminate all usage of the thread-blocking getaddrinfo(3) call upon DNS queries, significantly improving federation latency/ping and cache DNS results using hickory-dns / hickory-resolver
• Store the sender user with the MXC URL for all media uploads (/upload) (not for thumbnails or media requests which are unauthenticated)
• Perform connection pooling and keepalives where necessary to significantly improve federation performance and latency
• Implement RocksDB online backups via admin command
• Implement RocksDB write buffer corking and coalescing in database write-heavy areas
• Various config options to tweak connection pooling, request timeouts, connection timeouts, DNS timeouts and settings, etc with good defaults
• Implement config option to auto join rooms upon registration
• Overall significant database, Client-Server, and federation performance and latency improvements
• Outgoing read receipt and private read receipt support (EDU)
• Outgoing typing indicator support (EDU)
• Opt-in Sentry.io telemetry and metrics, mainly used for crash reporting
• Add /_conduwuit/server_version route to return the version of Conduwuit without relying on the federation API /_matrix/federation/v1/version
• Add configurable RocksDB recovery modes to aid in recovering corrupte RocksDB database
• Config option to forbid publishing rooms to the room directory (lockdown_public_room_directory) except for admins
• Don't allow m.call.invite events to be sent in public rooms (prevents calling the entire room)
• On new public room creations, only allow moderators to send m.call.invite, org.matrix.msc3401.call, and org.matrix.msc3401.call.member events
• Stop sending make_join requests on room joins if 15 servers respond with M_UNSUPPORTED_ROOM_VERSION or M_INVALID_ROOM_VERSION
• Stop sending make_join requests if 50 servers cannot provide make_join for us
• Admin debug command to send a federation request/ping to a server's /_matrix/federation/v1/version endpoint and measures the latency it took