Handler is overused. Handler ought to mean the end-function handling the
command. The command processor is the central dispatcher to the handler.
Signed-off-by: Jason Volk <jason@zemos.net>
we use ring for hashing state and ruma, and reqwest/rustls defaults
to aws_lc_rs, so we have to manually pick which one. there doesn't
seem to be a way to just use one for some reason, so lets just use
the new aws_lc_rs.
Signed-off-by: strawberry <strawberry@puppygock.gay>
while their code of conduct has very great baseline points, this is not a code of
conduct the foundation upholds themselves, so it would be a disserve to mislead
folks into using this as a baseline.
Signed-off-by: strawberry <strawberry@puppygock.gay>
attic just sucks and every 5 CI runs guarantees at least 2 false
gateway errors that i can't do anything about.
Signed-off-by: strawberry <strawberry@puppygock.gay>
this is already done but we just don't error and always
use the sender user. match synapse behaviour where we check
and error.
Signed-off-by: strawberry <strawberry@puppygock.gay>
this prefix causes you to require setting the environment variable
to `CONDUWUIT_CONDUIT_CACHE_CAPACITY_MODIFIER`
alias this so we dont break any configs
Signed-off-by: strawberry <strawberry@puppygock.gay>
we have the patch already and i dont want to make it nix
exclusive by reverting it in my rocksdb fork
Signed-off-by: strawberry <strawberry@puppygock.gay>
I run a homeserver whose logs show a high number of incoming empty
presence EDUs originating from the user agent "Conduwuit/0.4.4". They
arrive at a rate of about 2 queries per second per Conduwuit server.
The empty EDUs all look the same, only with `origin_server_ts`
increasing:
```
{"origin":"example.com","origin_server_ts":1720266475601,"edus":[{"edu_type":"m.presence","content":{"push":[]}}]}
```
These updates are unnecessary because they don't do anything. They
only increase network traffic and CPU usage on both sides.
After this commit, the empty presence updates are no longer inserted
into the outgoing event queue.
also had to fix a million clippy lints
fix(spaces): deal with hierarchy recursion
fix(spaces): properly handle max_depth
refactor(spaces): token scheme to prevent clients from modifying max_depth and suggested_only
perf(spaces): use tokens to skip to room to start populating results at
feat(spaces): request hierarchy from servers in via field of child event
Co-authored-by: Matthias Ahouansou <matthias@ahouansou.cz>
Signed-off-by: strawberry <strawberry@puppygock.gay>
sorry but this is just non-functional in our build system,
i haven't had time to figure out why nix doesn't like what
i'm doing with it, and i haven't heard anyone using this
(the user who i anticipated would use this has not updated
their conduwuit in many months)
Signed-off-by: strawberry <strawberry@puppygock.gay>
should reduce all but rocksdb from here: 0a48586e8c/pkgs/conduwuit-git/pins.json
Co-authored-by: PedroHLC <root@pedrohlc.com>
Signed-off-by: strawberry <strawberry@puppygock.gay>
Previously, anyone could remove any local alias, meaning that someone could re-route a popular alias elsewhere
Now, only the creator of the alias, users who can set canonical aliases for the room, server admins and the server user can delete aliases
added some additional changes/fixes to adapt to our codebase
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
This ensures that the tokenization algorithm will remain in sync between
querying, indexing, and deindexing. The existing code had slightly
different behavior for querying, because it did not discard words with
>50 bytes. This was inconsequential, because >50 byte tokens are never
present in the index.
Signed-off-by: strawberry <strawberry@puppygock.gay>
The previous code would drop some events entirely if any events between
`skip` and `skip + limit` were not visible to the user. This would cause
the set of events skipped by the `skip(skip)` method to extend past
`skip` in the raw result set, because `skip(skip)` was being called
*after* filtering out invisible events.
This bug will become much more severe with a full filtering
implementation, because it will be more likely for events to be filtered
out. Currently, it is only possible to trigger with rooms that have
history visibility set to "invited" or "joined".
Signed-off-by: strawberry <strawberry@puppygock.gay>
The previous code would fail to return next_batch if any of the events
in the window were not visible to the user. It would also return an
unnecessary next_batch when no more results are available if the total
number of results is exactly `skip + limit`.
This bug will become much more severe with a full filtering
implementation, because we will be more likely to trigger it by
filtering out events in a search call. Currently, it is only possible to
trigger with rooms that have history visibility set to "invited" or
"joined".
Signed-off-by: strawberry <strawberry@puppygock.gay>
for a few months now we accidentally had double 0xFF splits being
inserted into `roomuserid_joined` cf when membership counts and such
are being updated.
this is a conduwuit-specific db migration and does NOT break conduit
compatibility.
`fix_bad_double_separator_in_state_cache`
Signed-off-by: strawberry <strawberry@puppygock.gay>
When enabled, if a file is deemed unremovable, it skips past it and
continues deleting all other files that fit the criteria. Additionally,
fix age comparison under the same command.
Signed-off-by: Lux Aliaga <lux@nixgoat.me>
`roomuserid_joined` cf seems unreliable, so in the mean time we need to check
membership state (or maybe this is a more reliable check anyways)
Signed-off-by: strawberry <strawberry@puppygock.gay>
this `real_users_cache` cache seems weird, and i have no idea what
prompted its creation upstream. perhaps they did this because
sqlite was very slow and their rocksdb setup is very poor, so
a "solution" was to stick member counts in memory.
slow iterators, scanning, etc do not apply to conduwuit where
our rocksdb is extremely tuned, and i seriously doubt something
like this would have any real world net-positive performance impact.
also for some reason, there is suspicious logic where we
overwrite the entire push target collection.
both of these things could be a potential cause for receiving
notifications in rooms we've left.
Signed-off-by: strawberry <strawberry@puppygock.gay>
only return `inline` if the detected content-type is an allowed
inline content-type as defined by MSC2702
Signed-off-by: strawberry <strawberry@puppygock.gay>
so in theory: guest users, peaking over federation,
and world readable rooms should be allowed to send
read receipts even if they're not joined.
relaxing this check to only allow the read receipt if
the server has at least 1 member in the room makes
some of this still work
Signed-off-by: strawberry <strawberry@puppygock.gay>
this is a code of conduct that the moderation team made up,
specific to conduwuit's community spaces such as the matrix rooms.
the matrix foundation and contributor's covenant still apply, but
having conduwuit-specific additional guidelines help
Signed-off-by: strawberry <strawberry@puppygock.gay>
rust:
* ALL lints which rustc defaults to "allow" have been set to "warn".
* NEW "warn" lints which produce a warning as of this commit have been
explicitly identified and commented with a TODO for later review.
clippy:
* ALL categories (sans restriction) now fully enabled to "warn".
* redundant lints set to "warn" from categories now at "warn" are removed.
* previous "allow" sadness moved into respective categories.
* new warnings produced as of this commit have been explicitly identified:
- nursery lints set to "allow" marked with TODO for later review.
- pedantic lints set to "allow"
Signed-off-by: Jason Volk <jason@zemos.net>
test -L $object [object exists and is a symbolic link (same as -h)]
It is not recommended to use -h
[True if file exists and is a symbolic link. This operator is retained for compatibility with previous versions of this program. Do not rely on its existence; use -L instead.]
this is dual-stack by default on linux, resolves
issues with nginx using `localhost` and randomly
choosing between 127.0.0.1 and [::1], causing
intermittent upstream issues
Signed-off-by: strawberry <strawberry@puppygock.gay>
The original implementation of this was really weird, so I restructed it
a lot while debugging, and am just gonna leave the restructured version.
Root cause of the segfault seems to be that upstream nixpkgs liburing
derivation is generating both static and dynamic libraries, causing
rocksdb to statically link liburing in a dynamic build, pulling in some
allocator stuff at the same time. I created a PR[1] to fix this upstream,
but it probably won't be available on nixos-unstable for quite a while,
so we can also patch it locally.
[1]: https://github.com/NixOS/nixpkgs/pull/314945
I talked to somebody yesterday in #conduwuit:puppygock.gay that was using
this output in their system config. The dynamically-linked jemalloc build
is quite fragile, and is not tested by anything else in CI. We want to
make sure we don't break it again in the future.
This failed to inherit the fix from bec507d7390dda9ca2e624ef846521a20e2252c7
because the crane package's buildInputs become propagatedBuildInputs in
a static stdenv, but become normal buildInputs in a dynamic stdenv. Since
we were only pulling propagatedBuildInputs into the devshell, dynamically
linked devshells did not include the rust-jemalloc-sys package. This
causes tikv-jemalloc-sys to build it's own static jemalloc package, and
we end up loading libc before jemalloc at runtime.
It turns out that this was actually fixed by
bec507d7390dda9ca2e624ef846521a20e2252c7 and
857ac42aacf430f14a9b7108591cea91185af8b3, but we didn't identify it at the
time. Notably, the `dynamic` devshell is still broken.
needed for hot reloading but rpath being true by default
causes linker errors on lld because of the sad rpath bug
Signed-off-by: strawberry <strawberry@puppygock.gay>
for a very long time, if a remote server responded to us with
a valid but unsuccessful (HTTP 4xx) response and the caller was the
`send_federation_request` function, we may find ourselves
with a warning message only containing the destination's
server name which was very unhelpful. the true error was
buried away in trace logs. this would primarily be noticed
with server key fetch requests from us.
conduit has been throwing away the ruma request error: https://gitlab.com/famedly/conduit/-/blame/next/src/utils/error.rs#L62
before: 2024-05-23T04:45:02.930224Z WARN router:{path=/_matrix/client/v3/publicRooms}:handle: conduit_api::client_server::directory: Failed to return our /publicRooms: matrix.org
after: 2024-05-23T05:05:02.435272Z WARN router:{path=/_matrix/client/v3/publicRooms}:handle: conduit_api::client_server::directory: Failed to return our /publicRooms: matrix.org: [401 / M_UNAUTHORIZED] Failed to find any key to satisfy: _FetchKeyRequest(server_name='your.server.name', minimum_valid_until_ts=1716440702337, key_ids=['ed25519:RQB3XPQX'])
Signed-off-by: strawberry <strawberry@puppygock.gay>
this matrix-react-sdk PR (and the cited sliding sync MSC)
says that they will intend on checking sliding sync support
from this unstable feature flag at /versions until the CORS
header stuff is specced
https://github.com/matrix-org/matrix-react-sdk/pull/12498
Signed-off-by: strawberry <strawberry@puppygock.gay>
Previously we were relying on NIX_CFLAGS_COMPILE, but this is not being
set in static devshells. A cleaner solution for complement would likely
be to build the tests in their own nix derivation instead of building
them in the devshell, but this change unblocks CI for now.
This is causing build failures on Mac:
> In file included from /tmp/nix-build-rocksdb-static-aarch64-apple-darwin-9.1.1.drv-0/source/memory/memory_allocator.cc:8:
> In file included from /tmp/nix-build-rocksdb-static-aarch64-apple-darwin-9.1.1.drv-0/source/memory/jemalloc_nodump_allocator.h:11:
> /tmp/nix-build-rocksdb-static-aarch64-apple-darwin-9.1.1.drv-0/source/port/jemalloc_helper.h:63:36: warning: unknown attribute '_rjem_malloc' ignored [-Wunknown-attributes]
> mallocx(size_t, int) JEMALLOC_ATTR(malloc) JEMALLOC_ALLOC_SIZE(1)
> ^~~~~~
> /nix/store/3bix0kzy670dyhhizri3dwb1qfj3sdpa-jemalloc-static-aarch64-apple-darwin-5.3.0/include/jemalloc/jemalloc.h:412:18: note: expanded from macro 'malloc'
> # define malloc je_malloc
> ^~~~~~~~~
> /nix/store/3bix0kzy670dyhhizri3dwb1qfj3sdpa-jemalloc-static-aarch64-apple-darwin-5.3.0/include/jemalloc/jemalloc.h:75:21: note: expanded from macro 'je_malloc'
> # define je_malloc _rjem_malloc
> ^~~~~~~~~~~~
> /nix/store/3bix0kzy670dyhhizri3dwb1qfj3sdpa-jemalloc-static-aarch64-apple-darwin-5.3.0/include/jemalloc/jemalloc.h:183:43: note: expanded from macro 'JEMALLOC_ATTR'
> # define JEMALLOC_ATTR(s) __attribute__((s))
Full build log at <https://girlboss.ceo/~strawberry/pb/ygJ3>. This is
likely fixable with patches to rocksdb, but not worth it since darwin is
only a dev platform.
CI is running `cargo build --all-features`, so we should be passing all
the features to nix as well.
The only thing this currently affects is the jemalloc_prof feature, but if
we add any non-default features that affect nix in the future they should
also be handled correctly now.
Dynamically-linked jemalloc doesn't work due to link-order issues, and we
want CI to be testing a static binary anyway since that's what we're
publishing in releases.
Some of the features affect nix dependencies, so we need to have a
full feature list available when constructing the nix derivation. This
incidentally fixes the bug where we weren't enabling jemalloc on rocksdb
in CI/devshells, because jemalloc is now a default feature. It does not
fix the more general class of that issue, where CI is performing an
`--all-features` build in a nix devshell built for default-features.
I am now passing `--no-default-features` to cargo, and having it use our
unified feature list rather than duplicating the unification inside cargo.
Without setting JEMALLOC_OVERRIDE, we end up linking to two different
jemalloc builds. Once dynamically, as a transitive dependency through
rocksdb, and a second time to the static jemalloc that tikv-jemalloc-sys
builds.
Previously, we were returning redundant member count updates or encrypted
device updates from the /sync endpoint in some cases. The extra member
count updates are spec-compliant, but unnecessary, while the extra
encrypted device updates violate the spec.
The refactor necessary to fix this bug is also necessary to support
filtering on state events in sync.
Details:
Joined room incremental sync needs to examine state events for four
purposes:
1. determining whether we need to return an update to room member counts
2. determining the set of left/joined devices for encrypted rooms
(returned in `device_lists`)
3. returning state events to the client (in `rooms.joined.*.state`)
4. tracking which member events we have sent to the client, so they can
be omitted on future requests when lazy-loading is enabled.
The state events that we need to examine for the first two cases is member
events in the delta between `since` and the end of `timeline`. For the
second two cases, we need the delta between `since` and the start of
`timeline`, plus contextual member events for any senders that occur in
`timeline`. The second list is subject to filtering, while the first is
not.
Before this change, we were using the same set of state events that we are
returning to the client (cases 3/4) to do the analysis for cases 1/2.
In a compliant implementation, this would result in us missing some
relevant member events in 1/2 in addition to seeing redundant member
events. In current conduwuit this is not the case because the set of
events that we return to the client is always a superset of the set that
is needed for cases 1/2. This is because we don't support filtering, and
we have an existing bug[1] where we are returning the delta between
`since` and the end of `timeline` rather than the start.
[1]: https://github.com/girlbossceo/conduwuit/issues/361
Fixing this is necessary to implement filtering because otherwise
we would start missing some member events for member count or encrypted
device updates if the relevant member events are rejected by the filter.
This would be much worse than our current behavior.
This cache can serve invalid responses, and has an extremely low hit
rate.
It serves invalid responses because because it's only keyed off
the `since` parameter, but many of the other request parameters also
affect the response or it's side effects. This will become worse once we
implement filtering, because there will be a wider space of parameters
with different responses. This problem is fixable, but not worth it
because of the low hit rate.
The low hit rate is because normal clients will always issue the next
sync request with `since` set to the `prev_batch` value of the previous
response. The only time we expect to see multiple requests with the same
`since` is when the response is empty, but we don't cache empty
responses.
This was confirmed experimentally by logging cache hits and misses over
15 minutes with a wide variety of clients. This test was run on
matrix.computer.surgery, which has only a few active users, but a
large volume of sync traffic from many rooms. Over the test period, we
had 3 hits and 5309 misses. All hits occurred in the first minute, so I
suspect that they had something to do with client recovery from an
offline state. The clients that were connected during the test are:
- element web
- schildichat web
- iamb
- gomuks
- nheko
- fractal
- fluffychat web
- fluffychat android
- cinny web
- element android
- element X android
Fixes: #336
the namespace check on username login is unnecessary, hashes aren't ever
going to match, and axum auth handles this kind of stuff already
Signed-off-by: strawberry <strawberry@puppygock.gay>
debconf support needs to be done in a way that does not duplicate
the config file like upstream does.
Signed-off-by: strawberry <strawberry@puppygock.gay>
This reverts commit 321a6ca0fe.
These checks were not working as intended, resulting in the unban button not working
The join check gets kept since it slightly reduces the amount of sent joins in some cases
This check will probably be replaced soon for a more universal solution to the "made no change" issue
Signed-off-by: morguldir <morguldir@protonmail.com>
protects against a maintainer creating a downgrading version tag, and
uploading artifacts with that version
this check is only ran via workflow dispatch on the tag
Signed-off-by: strawberry <strawberry@puppygock.gay>
Only normal users should be prevented from creating an alias within an
exclusive namespace, not the appservice itself. This mirrors the
behaviour in api/client_server/room.rs on room creation.
this was already enabled by default by rocksdb technically, but
it wasn't building with it properly.
Signed-off-by: strawberry <strawberry@puppygock.gay>
from ffd479d66f
This is primarily useful for replicating the environment from CI so that
the `nix-build-and-cache` script is easier to invoke.
Signed-off-by: strawberry <strawberry@puppygock.gay>
this makes `CONDUWUIT_WELL_KNOWN__CLIENT` a valid env variable config
option as it would normally exist under `[well_known.client]` in toml
Signed-off-by: strawberry <strawberry@puppygock.gay>
after getting the shared rooms with the target user, we actually only
get the presence of ourselves instead of the requested user
Signed-off-by: strawberry <strawberry@puppygock.gay>
from f5bd9bc45e
with changes for GitHub CI and misc
Co-authored-by: Charles Hall <charles@computer.surgery>
Signed-off-by: strawberry <strawberry@puppygock.gay>
this is purely DNSSEC related which we don't use, and DNSSEC on matrix
is unbearable for federation (no one sets it up properly, it's extremely taxing, etc)
Signed-off-by: strawberry <strawberry@puppygock.gay>
there's really no point in trying to stay as low as possible for us,
and this makes development easier. Debian users should just use rustup,
Nix users already get the proper toolchains.
Signed-off-by: strawberry <strawberry@puppygock.gay>
Hardcoding the output path to something in CWD is a pain if you're running
conduwuit through systemd or similar. Also made the error message when
it's unable to create the output file a little more friendly.
Previously, enabling the `tracing_flame` or `allow_jaeger` options would
prevent any logs from being written to stdout. In addition, enabling the
`allow_jaeger` option would inhibit the `tracing_flame` option.
Now that we have a way to use separate tracing filters with different
layers, we can enable all three at the same time without issues.
This commit also prevents the `debug log_level` command from modifying
the `tracing-flame` filter. This was supported previously, but I don't
think it's something that you would ever want to do intentionally. Now
that we have both the normal log filter and the `tracing-flame` filter
enabled at the same time, we want to `debug log_level` to only modify the
normal filter.
imagine this SQL query but in conduwuit:
select * from users_in_public_rooms where user_id like '%user_id%';
Signed-off-by: strawberry <strawberry@puppygock.gay>
This turned out to be quite hairy, mostly because we need to apply the
config's log level filter to the actual logs (stdout and, optionally
sentry), but do not want to filter out the tokio tracing events needed by
the console_subscriber. I hit several edge cases in tracing getting
this to work, and we now depend on a git version of tracing with a
backported patch :(
All images should be generating correctly with parallelism and Docker manifests, and should output the end of the CI testing errors in a job summary box when the test fails.
When the test succeeds you get a big ✅ then at the end of the Docker publish it should include the `docker pull` commands for both Docker Hub and GHCR registries to make those pesky Docker users lives easier!
Should've been done in fe606f4fad12b14de398ebf0319462f432dc336e but the
author didn't realize it.
Flake lock file updates:
• Updated input 'rocksdb':
'github:facebook/rocksdb/bcf88d48ce8aa8b536aee4dd305533b3b83cf435' (2024-04-16)
→ 'github:facebook/rocksdb/6f7cabeac80a3a6150be2c8a8369fcecb107bf43' (2024-04-22)
Apparently github actions VMs ship with it and that's how it was working
before? Cursed. We should control our own supply chain and also ensure
that local development uses the same version as CI.
Some of the improvements here include:
* rocksdb can actually use jemalloc now instead of just pulling in a
second rocksdb for no reason
* "complement-runtime" factored back out into shell file
* complement image no longer uses `mkDerivation` for `copyToRoot`
because that's what `buildEnv` is for
* complement image no longer sets `SERVER_NAME`, complement already does
that
* all packages were factored out into `callPackage`-able files for use
with a custom `lib.makeScope pkgs.newScope`
* new version of `mkPackage` has options that are easier to use and
override such as `features`
Apparently it uses `date -Iseconds` to parse, so we can use @ with a timestamp
Also it doesn't parse `created` in buildImage, only buildLayeredImage
Signed-off-by: morguldir <morguldir@protonmail.com>
Signed-off-by: strawberry <strawberry@puppygock.gay>
resolves X-Matrix signatures being invalid in some edge-cases,
and fixes Complement/Sytest federation tests
Signed-off-by: strawberry <strawberry@puppygock.gay>
this is hard to maintain now, and i think conduwuit is reaching a point that
it's way too different from upstream to simply "list all the differences" out
Signed-off-by: strawberry <strawberry@puppygock.gay>
this does deprecate the original `well_known_` prefixed config options
with a dedicated/proper config sub-block (`[config.well_known]`)
Signed-off-by: strawberry <strawberry@puppygock.gay>
there's nothing unstable about this, and per upstream only unstable
room versions are used if they are complex to support (versions <=5)
Signed-off-by: strawberry <strawberry@puppygock.gay>
this is not helpful informational logging
```
2024-04-02T04:40:52.590444Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking thomcat.rocks for /hierarchy
2024-04-02T04:40:52.628248Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking hackingfor.eu for /hierarchy
2024-04-02T04:40:52.808526Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking matrix.org for /hierarchy
2024-04-02T04:40:52.920936Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking matrix.org for /hierarchy
2024-04-02T04:40:52.959362Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Got response from matrix.org for /hierarchy
Response { children: [], inaccessible_children: [], room: SpaceHierarchyParentSummary { canonical_alias: Some("#cybersec-whonix:matrix.org"), name: Some("Whonix"), num_joined_members: 329, room_id: "!OJFkLJksWastbfdRuf:matrix.org", topic: Some("| Part of the Cybersec matrix.org community (#cyber-space:matrix.org) | RULES: https://cybersec-rules.thomcat.rocks | Whonix-focused room | Off-topic chat is fine. Keep it brief and/or move it to #cybersec-offtopic:matrix.org"), world_readable: true, guest_can_join: false, avatar_url: Some("mxc://matrix.org/DMJtrQdhQHKxeODrSibtrczX"), join_rule: "public", room_type: None, children_state: [], allowed_room_ids: [] } }
2024-04-02T04:40:52.959762Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking matrix.org for /hierarchy
2024-04-02T04:40:53.109611Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Got response from matrix.org for /hierarchy
Response { children: [], inaccessible_children: [], room: SpaceHierarchyParentSummary { canonical_alias: Some("#lockpicking:matrix.org"), name: Some("Lockpicking"), num_joined_members: 284, room_id: "!uEYjSxQOZnHEkiurTP:matrix.org", topic: Some("| Part of the Cybersec matrix.org community (#cyber-space:matrix.org) | RULES: https://cybersec-rules.thomcat.rocks | Off-topic chat is fine. Keep it brief and/or move it to #cybersec-offtopic:matrix.org"), world_readable: false, guest_can_join: false, avatar_url: Some("mxc://thomcat.rocks/cSeFfMLUdjymKdkUmXtIQTjf"), join_rule: "public", room_type: None, children_state: [], allowed_room_ids: [] } }
2024-04-02T04:40:53.109880Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking matrix.org for /hierarchy
2024-04-02T04:40:53.261581Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Got response from matrix.org for /hierarchy
Response { children: [], inaccessible_children: [], room: SpaceHierarchyParentSummary { canonical_alias: Some("#cybersec-rss:matrix.org"), name: Some("Cybersecurity-RSS"), num_joined_members: 347, room_id: "!IVHnAZkhJOhdZxlHRA:matrix.org", topic: Some("| Part of the Cybersec matrix.org community (#cyber-space:matrix.org) | Suggestions to our RSS feed list welcome at #cybersecurity:matrix.org"), world_readable: true, guest_can_join: false, avatar_url: Some("mxc://thomcat.rocks/ugdubBloOkVCYxqFzdVoPwcV"), join_rule: "public", room_type: None, children_state: [], allowed_room_ids: [] } }
2024-04-02T04:40:53.305166Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Got response from matrix.org for /hierarchy
Response { children: [], inaccessible_children: [], room: SpaceHierarchyParentSummary { canonical_alias: Some("#cybersec-whonix:matrix.org"), name: Some("Whonix"), num_joined_members: 329, room_id: "!OJFkLJksWastbfdRuf:matrix.org", topic: Some("| Part of the Cybersec matrix.org community (#cyber-space:matrix.org) | RULES: https://cybersec-rules.thomcat.rocks | Whonix-focused room | Off-topic chat is fine. Keep it brief and/or move it to #cybersec-offtopic:matrix.org"), world_readable: true, guest_can_join: false, avatar_url: Some("mxc://matrix.org/DMJtrQdhQHKxeODrSibtrczX"), join_rule: "public", room_type: None, children_state: [], allowed_room_ids: [] } }
2024-04-02T04:40:53.323257Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking thomcat.rocks for /hierarchy
2024-04-02T04:40:53.323672Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking matrix.org for /hierarchy
2024-04-02T04:40:53.369721Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Asking matrix.org for /hierarchy
2024-04-02T04:40:53.529250Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Got response from matrix.org for /hierarchy
Response { children: [], inaccessible_children: [], room: SpaceHierarchyParentSummary { canonical_alias: None, name: None, num_joined_members: 1463, room_id: "!aBXqGDWIxVYeYxVbRu:matrix.org", topic: Some("| Part of the Cybersec matrix.org community (#cyber-space:matrix.org) | The room needs urgent attention? @ all the mods! | RULES: https://cybersec-rules.thomcat.rocks | \"you seem to be completely in lack of basic knowledge of how a computer or any programming language operates, to the point that any attempt of discussing any security measures is a complete waste of time\""), world_readable: false, guest_can_join: false, avatar_url: Some("mxc://thomcat.rocks/DjRWzuYjLkScQYPySYNedlFr"), join_rule: "public", room_type: None, children_state: [], allowed_room_ids: [] } }
2024-04-02T04:40:53.546581Z INFO http_request{path=/_matrix/client/v1/rooms/:room_id/hierarchy}: conduit::service::rooms::spaces: Got response from matrix.org for /hierarchy
Response { children: [], inaccessible_children: [], room: SpaceHierarchyParentSummary { canonical_alias: None, name: None, num_joined_members: 1463, room_id: "!aBXqGDWIxVYeYxVbRu:matrix.org", topic: Some("| Part of the Cybersec matrix.org community (#cyber-space:matrix.org) | The room needs urgent attention? @ all the mods! | RULES: https://cybersec-rules.thomcat.rocks | \"you seem to be completely in lack of basic knowledge of how a computer or any programming language operates, to the point that any attempt of discussing any security measures is a complete waste of time\""), world_readable: false, guest_can_join: false, avatar_url: Some("mxc://thomcat.rocks/DjRWzuYjLkScQYPySYNedlFr"), join_rule: "public", room_type: None, children_state: [], allowed_room_ids: [] } }
```
Signed-off-by: strawberry <strawberry@puppygock.gay>
this is the url field in the appservice registration file,
this is almost always localhost and the admin should
be vetting the appservice registration yaml file before
registering it anyways.
Signed-off-by: strawberry <strawberry@puppygock.gay>
there seems to be a regression, likely from https://github.com/facebook/rocksdb/issues/12361 / https://github.com/facebook/rocksdb/pull/12309
```
[1/0/2 built] building rocksdb-9.0.0 (configurePhase): -- Detecting CXX compile features - donedirenv: ([/Users/strawberry/.nix-profile/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
error: builder for '/nix/store/9slwgpnardhn2vqzqhn361ic668n38wq-rocksdb-9.0.0.drv' failed with exit code 1;
last 10 log lines:
> -- Found lz4: /nix/store/cafwv4439qbm2ij04mpc7xz5m3f7mfix-lz4-1.9.4/lib/liblz4.dylib
> CMake Error at /nix/store/bin32lqag7lx38994xpf9jvhk1xbd64c-cmake-3.28.2/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
> Could NOT find zstd (missing: ZSTD_INCLUDE_DIRS)
> Call Stack (most recent call first):
> /nix/store/bin32lqag7lx38994xpf9jvhk1xbd64c-cmake-3.28.2/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
> cmake/modules/Findzstd.cmake:17 (find_package_handle_standard_args)
> CMakeLists.txt:167 (find_package)
>
>
> -- Configuring incomplete, errors occurred!
For full logs, run 'nix log /nix/store/9slwgpnardhn2vqzqhn361ic668n38wq-rocksdb-9.0.0.drv'.
error: 1 dependencies of derivation '/nix/store/ir8jf2wic98iymjlk7d2i1kjjsgv15v2-nix-shell-env.drv' failed to build
```
happens in both rust-rocksdb and our fork of rust-rocksdb
Signed-off-by: strawberry <strawberry@puppygock.gay>
There are so many ways to do this we realistically shouldn't bother
describing any of them, especially because people should be learning all
the options and choosing the one that suits them best anyway.
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
The main thing this section is really useful for is explaining how to
configure various reverse proxies, which applies to basically anything.
Also, remove all the language about this being "recommended", because
nothing in this documentation is actually tested in CI.
Signed-off-by: strawberry <strawberry@puppygock.gay>
It is very stale. Please just use Nix. Trying to do it outside of Nix
will be an exercise in frustration, I guarantee it.
Signed-off-by: strawberry <strawberry@puppygock.gay>
this should fix FluffyChat password resets, and
other possible client issues that expose features
based on server capabilities.
Signed-off-by: strawberry <strawberry@puppygock.gay>
squashed from https://gitlab.com/famedly/conduit/-/merge_requests/604
added differences.md
Co-authored-by: Charles Hall <charles@computer.surgery>
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
save the few bytes of bandwidth for something else
also now that we send our User-Agent, it is
technically sending conduwuit versions now
Signed-off-by: strawberry <strawberry@puppygock.gay>
i build/develop on a mac, i have a windows machine,
my servers run linux and i do prod builds on linux,
and BSD support is generally inherent with linux
and mac.
i don't think it hurts or would be difficult for me
to say i support all of these.
upstream conduit only officially supports Linux
Signed-off-by: strawberry <strawberry@puppygock.gay>
yes windows technically supports unix sockets,
but its not as good or the same as actual
nix platform unix sockets
Signed-off-by: strawberry <strawberry@puppygock.gay>
reduces unnecessary crates being compiled. splits them
into features.
i have yet to see anyone use conduit's opentelemetry
stuff, and realistically those people who do
performance benchmarking and measurements will be
building stuff anyways so they can just enable this
feature.
Signed-off-by: strawberry <strawberry@puppygock.gay>
if https://github.com/ruma/ruma/issues/1240 is to
be trusted, there are apparently still servers
and clients that may call these endpoints.
i'm unable to read the history of that matrix.to
link so i don't know the full context, but this
is trivial to implement so..
Signed-off-by: strawberry <strawberry@puppygock.gay>
from https://gitlab.com/famedly/conduit/-/merge_requests/583
and fixed panic from blocking async call in timeline/mod.rs
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
squashed from https://gitlab.com/famedly/conduit/-/merge_requests/596
ported the relevant parts to GitHub Actions
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
retains compatibility for container users
who set it to empty.
if the variable is unspecified, it will
use the CONDUIT_ variables as normal.
Signed-off-by: strawberry <strawberry@puppygock.gay>
at least with Element Web: they still rely on the
"redacts" top level key. it was reported that
federated redactions were not working for v11 rooms
with conduwuit. after adding back the top level
key when appending a PDU to the timeline, they
now work again.
spec below says to continue adding the top level
key for compatibility with older clients.
https://spec.matrix.org/v1.9/rooms/v11/#moving-the-redacts-property-of-mroomredaction-events-to-a-content-property
Signed-off-by: strawberry <strawberry@puppygock.gay>
this can be used as a way to deal with the thumbnail
and the media file at the same time without knowing
the thumbnail MXC URL.
Signed-off-by: strawberry <strawberry@puppygock.gay>
complement sends C-S requests over HTTP, and federation
over HTTPS.
complement without caddy *almost* works. unfortunately
i am now dealing with invalid X-Matrix signatures
due to non-percent encoded URIs and it does not
seem trivial to percent-encode URIs that a
reverse proxy would normally do for you.
Signed-off-by: strawberry <strawberry@puppygock.gay>
this provides not only some future compatibility with MSC4051,
but it just makes sense to not crash/error if we can't get a server_name
from the room ID and should just use the server_name from the sender
user's invite event. there is already code ahead that accounts for
an empty vector so this is safe.
Signed-off-by: strawberry <strawberry@puppygock.gay>
default of 90 seconds can be too low if hardware is not very fast
and rocksdb compaction or shutdown takes too long
Signed-off-by: strawberry <strawberry@puppygock.gay>
takes a full room ID, evicts all our users from that room,
adds room ID to banned room IDs metadata db table, and
forbids any new local users from attempting to join it.
Signed-off-by: strawberry <strawberry@puppygock.gay>
query parameter `?format=event|content`
defaults to normal behaviour which is the event's content.
ruma impl: 788ea6b00f
Signed-off-by: strawberry <strawberry@puppygock.gay>
from upstream MR https://gitlab.com/famedly/conduit/-/merge_requests/347
with the following changes (so far):
- remove hardcoded list of allowed hosts (strongly disagree with this,
even if it is desired, it should not be harcoded)
- add more allow config options for granularity via URL contains,
host contains, and domain is (explicit match) for security
- warn if a user is allowing all URLs to be previewed for security reasons
- replace an expect with proper error handling
- bump webpage to 2.0
- improved code style a tad
Co-authored-by: rooot <hey@rooot.gay>
Signed-off-by: rooot <hey@rooot.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
This uses flakes-compat to read the `flake.nix` and expose it
to non-flake users.
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
Signed-off-by: strawberry <strawberry@puppygock.gay>
`IndexMap::remove` was deprecated in favor of explicitly named methods.
I assume that we actually needed to be using `shift_remove`, otherwise
we probably wouldn't be bothering with `indexmap` here in the first
place. I wonder if this fixes any bugs lol
from: https://gitlab.com/famedly/conduit/-/merge_requests/580
Signed-off-by: strawberry <strawberry@puppygock.gay>
This fixes a bug where the aarch64 OCI image had metadata saying it was
an x86_64 OCI image. On top of that, I think the metadata was actually
right (aside from Conduit's binary): since all other packages were being
pulled from `pkgsHost`, an OCI image cross compiled for aarch64 from a
different architecture would result in unexecutable binaries (e.g. tini)
since they were compiled for the completely wrong architecture.
from: https://gitlab.com/famedly/conduit/-/merge_requests/579
Signed-off-by: strawberry <strawberry@puppygock.gay>
Thanks to the crane maintainer to fixing my issue in a way that doesn't
suck, unlike my attempt in the fork we were briefly using.
from: https://gitlab.com/famedly/conduit/-/merge_requests/576
Signed-off-by: strawberry <strawberry@puppygock.gay>
Well, kinda. It crashed on me after 10 minutes because the tests timed
out like in <https://github.com/matrix-org/complement/issues/394>.
Sounds like this means it's a them problem though.
I want to use Nix to build this image instead in the future but this
will at least make it work for now and give me a reference for while I'm
porting it. I also want to make Conduit natively understand Complement's
requirements instead of `sed`ing a bunch of stuff and needing a reverse
proxy in the container. Should be more reliable that way.
I'm not making this run in CI until the above stuff is addressed and
until I can decide on a way to pin the revision of Complement being
tested against.
from: https://gitlab.com/famedly/conduit/-/merge_requests/575
Signed-off-by: strawberry <strawberry@puppygock.gay>
Without this, checking the authority of TLS certificates fails, making
Conduit (rightly) refuse to connect to anything.
Signed-off-by: strawberry <strawberry@puppygock.gay>
Honestly not sure why it wasn't done like this before. This code is much
less awkward to follow and more compartmentalized.
These changes were mainly motivated by a clippy lint triggering on the
original code, which then made me wonder if I could get rid of some of
the `Box`ing. Turns out I could, and this is the result of that.
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: strawberry <strawberry@puppygock.gay>
a 6+ year old deprecated field that isnt even spelled
right, and no clients use it must still be sent
according to spec
Signed-off-by: strawberry <strawberry@puppygock.gay>
this can most definitely be improved but this is a decent attempt.
the only annoying this is i couldn't just use a Vec<IPAddress> which
would have significantly simplified all of this, but serde can't
deserialise it on the config side i guess.
i may find a better way to do this in the future, but this should cover
most areas anyways.
Signed-off-by: strawberry <strawberry@puppygock.gay>
as far as i can tell, it will return a normal
error in the auth token handling code so this is fine.
we also shouldnt assume all errors from this are
access_token related.
Signed-off-by: strawberry <strawberry@puppygock.gay>
latest rocksdb now has WriteBufferManager support
i hope no one is using sqlite with conduwuit, but if they are let's
bump it to latest git too for the latest sqlite version available.
Signed-off-by: strawberry <strawberry@puppygock.gay>
im not sure what the TODO is trying to say here,
but since it's many years old and conduwuit is
fast, i dont see an issue with this.
Signed-off-by: strawberry <strawberry@puppygock.gay>
these are benign errors that are from things like
conduwuit fetching remote media from dead servers
Signed-off-by: strawberry <strawberry@puppygock.gay>
default for RocksDB is -1 and conduwuit already raises the
soft and hard nofile limits at startup.
Signed-off-by: strawberry <strawberry@puppygock.gay>
in short, the `/report` endpoint now checks if:
- the reporting room in the URI matches the PDU/event reported
- sender user is in the room reported
- raises report reasoning to 750 characters (spec doesn't say to limit
these, but thorough and informative reports for server admins are not
a bad thing)
- (hopefully) fixes some broken formatting
- add a random short delay before sending a successful response to the
client to make it more annoying to enumerate for events on our server
(security by obscurity but spec suggests it)
basically, secure reports better lol
see https://spec.matrix.org/v1.9/client-server-api/#post_matrixclientv3roomsroomidreporteventid
Signed-off-by: strawberry <strawberry@puppygock.gay>
this seems to require some more work to properly ignore
dead server errors without breaking the entire room join
Signed-off-by: strawberry <strawberry@puppygock.gay>
this can help if users change their well-known or such and we don't want
to keep on hitting the old destination.
from 11357d1f1a
Co-authored-by: Jacob Taylor <jacob@explodie.org>
Signed-off-by: strawberry <strawberry@puppygock.gay>
this will be a major pain to work through. for now, let's
just add them and overtime work through these.
Signed-off-by: strawberry <strawberry@puppygock.gay>
these database backends are either unmaintained, broken in conduit, or
incredibly niche for something like conduwuit.
also i want to bump the MSRV.
Signed-off-by: strawberry <strawberry@puppygock.gay>
amazing that this wasn't done in the first place. a lack of a UA is
suspicious and some providers may block our requests just for the sake
of no UA.
Signed-off-by: strawberry <strawberry@puppygock.gay>
the last endpoint is a non-standard health check endpoint used by at
least Element Web as a weird way to determine if syncv3 is available
there can also be some valid use-cases for serving well-knowns from the
application itself
Signed-off-by: strawberry <strawberry@puppygock.gay>
info is still logging useless database statistics and metrics that will
accumulate, less I/O activity.
Signed-off-by: strawberry <strawberry@puppygock.gay>
* feat: replaced flaky argon2 with better argon2 crate
* fix: applied cargo fmt nightly
* docs: added comment specifying what the settings for Argon2 mean
* fix: made hashing error a bit more descriptive
* fix: fixed incorrect value for Kib
this might be an unreachable state with the next commit, but let's be
extra sure here just in case a guest still managed to register before
a real admin was created.
Signed-off-by: strawberry <strawberry@puppygock.gay>
Also removed all instances of `#[command(verbatim_doc_comment)]` because
I'm pretty sure it's not necessary anymore. The `[commandbody]` things
were making rustdoc upset about broken link syntax. I also normalized
"code-block" to "code block" in that file since the latter appears more
often.
even rust-analyzer themselves don't compile with debug info because it
makes cargo checks and builds in general a lot longer. helps speed
things up and we mainly care about panics/stacktraces and compiler
errors.
https://github.com/rust-lang/rust-analyzer/blob/master/Cargo.toml#L12-L15
upstream rust-rocksdb is active again finally
Signed-off-by: strawberry <strawberry@puppygock.gay>
split out the spinning disk focused options into a configurable option, current
conduwuit users are NVMe/SSDs anyways so those options are just hindering performance.
rocksdb logging builds up overtime with no cleanup or anything, adds support for
configuring the amount of logging, size of files, log rotate, etc.
fixes https://gitlab.com/girlbossceo/conduwuit/-/issues/1
misc conduit logging improvements for help debugging issues and maybe a future feature
experimental Room V11 support from https://gitlab.com/famedly/conduit/-/merge_requests/562
Signed-off-by: strawberry <strawberry@puppygock.gay>
room IDs in conduit have a server name attached to them already
outputting the server name again just makes it look incorrect
also add codeblock to the internal room ID portion
Signed-off-by: strawberry <june@girlboss.ceo>
this project's codebase is so horrendous, im shocked that no one has ran
clippy at all. it had ~200 total lint warnings, some with performance
issues and unsoundness, and the rest just very ugly codebase. i have sat
down and fixed as many of these as possible and i am exhausted.
i haven't fixed some extremely complex ones, but i brought it down from
~200 to ~30.
i have also removed io_uring as a default feature due to it falling
under the same category as linux eBPF: major kernel attack surface for
minimal performance gains. this also makes it impossible to cross-compile
from macOS to Linux because io_uring does not exist in Darwin land.
there are far better ways to achieve better performance than io_uring on
the codebase level.
Signed-off-by: strawberry <june@girlboss.ceo>
iouring falls into the same category as eBPF and is a major source of kernel vulnerabilities. the benefits gained here are too minimal to bother keeping this enabled, and makes cross-compiling from macOS impossible
Signed-off-by: strawberry <june@girlboss.ceo>
can't update ruma to very latest commit because of the weird JsOption thing for syncv4 that i can't wrap my head around how to use, not important anyways
Signed-off-by: strawberry <strawberry@pupbrain.dev>
This speeds up handling of /_matrix/federation/v1/send/:transaction_id
when more than one event contains unknown keys.
In particular, when receiving multiple PDUs with dead servers in their
auth chain, timeouts of each server accumulate and can make handling of
incoming requests take several minutes, to the point the client closes
the connection (eg. matrix.org has a 2 minute timeout), causing new
events to be dropped eventually.
rust-rocksdb fork, add optional opt-in zstd response body compression
config, add webp support for images
cant upgrade things like axum, http, tower, hyper, etc due to the 1.0
release of http which are breaking all of these libs
Signed-off-by: strawberry <strawberry@pupbrain.dev>
the ones now used by default in rust-argon2 are extremely aggressive and
resource intensive. owasp's 2nd default is very reasonable with the same
security.
Signed-off-by: girlbossceo <june@girlboss.ceo>
causes tokio runtime worker to panic as we attempt to convert an HTTP
Request from ruma (try_into_http_request) into a reqwest Request so the
reqwest http client on the server can execute said request. error
message is not ideal ("invalid port number") but core issue is fixed.
Co-authored-by: infamous <ehuff007@gmail.com>
Signed-off-by: girlbossceo <june@girlboss.ceo>
Initial implementation done in https://gitlab.com/famedly/conduit/-/merge_requests/507,
*substantially* reworked, corrected, improved by infamous <ehuff007@gmail.com>,
and few parts done by me.
Co-authored-by: infamous <ehuff007@gmail.com>
Signed-off-by: girlbossceo <june@girlboss.ceo>
even though these rooms would be quite broken as they typically would be
super super old rooms, people seem to still complain about conduit not
"supporting" room versions 1 and 2. to make them happy, just make them
as unstable so they can at least join it.
Signed-off-by: girlbossceo <june@girlboss.ceo>
this has been the new default since Matrix 1.6, and V10 has been flagged
as stable for a while in Conduit
Signed-off-by: girlbossceo <june@girlboss.ceo>
2023-09-13 20:36:03 -04:00
497 changed files with 60470 additions and 36958 deletions
bin/nix-build-and-cache just .#static-${{ matrix.target }}-all-features-debug
# > warning: dev profile is not supported and will be a hard error in the future. cargo-deb is for making releases, and it doesn't make sense to use it with dev profiles.
# so we need to coerce cargo-deb into thinking this is a release binary
- if command -v nix > /dev/null; then echo "experimental-features = nix-command flakes" >> /etc/nix/nix.conf; fi
# Add our own binary cache
- if command -v nix > /dev/null; then echo "extra-substituters = https://attic.conduit.rs/conduit" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null; then echo "extra-trusted-public-keys = conduit:ddcaWZiWm0l0IXZlO8FERRdWvEufwmd0Negl1P+c0Ns=" >> /etc/nix/nix.conf; fi
# Add conduwuit binary cache
- if command -v nix > /dev/null; then echo "extra-substituters = https://attic.kennel.juneis.dog/conduwuit" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null; then echo "extra-trusted-public-keys = conduwuit:BbycGUgTISsltcmH0qNjFR9dbrQNYgdIAcmViSGoVTE=" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null; then echo "extra-substituters = https://attic.kennel.juneis.dog/conduit" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null; then echo "extra-trusted-public-keys = conduit:eEKoUwlQGDdYmAI/Q/0slVlegqh/QmAvQd7HBSm21Wk=" >> /etc/nix/nix.conf; fi
# Add alternate binary cache
- if command -v nix > /dev/null && [ -n "$ATTIC_ENDPOINT" ]; then echo "extra-substituters = $ATTIC_ENDPOINT" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null && [ -n "$ATTIC_PUBLIC_KEY" ]; then echo "extra-trusted-public-keys = $ATTIC_PUBLIC_KEY" >> /etc/nix/nix.conf; fi
# Add Lix binary cache
- if command -v nix > /dev/null; then echo "extra-substituters = https://cache.lix.systems" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null; then echo "extra-trusted-public-keys = cache.lix.systems:aBnZUw8zA7H35Cz2RyKFVs3H4PlGTLawyY5KRbvJR8o=" >> /etc/nix/nix.conf; fi
# Add crane binary cache
- if command -v nix > /dev/null; then echo "extra-substituters = https://crane.cachix.org" >> /etc/nix/nix.conf; fi
- if command -v nix > /dev/null; then echo "extra-trusted-public-keys = crane.cachix.org-1:8Scfpmn9w+hGdXH/Q9tTLiYAE/2dnJYRJP7kl80GuRk=" >> /etc/nix/nix.conf; fi
### a very cool, featureful fork of [Conduit](https://conduit.rs/)
<!-- ANCHOR_END: catchphrase -->
Please visit the [Conduit documentation](https://famedly.gitlab.io/conduit) for more information.
Alternatively you can open [docs/introduction.md](docs/introduction.md) in this repository.
Visit the [Conduwuit documentation](https://conduwuit.puppyirl.gay/) for more
information.
<!-- ANCHOR: body -->
#### What is Matrix?
[Matrix](https://matrix.org) is an open network for secure and decentralized
communication. Users from every Matrix homeserver can chat with users from all
other Matrix servers. You can even use bridges (also called Matrix appservices)
other Matrix servers. You can even use bridges (also called Matrix Appservices)
to communicate with users outside of Matrix, like a community on Discord.
#### What is the goal?
An efficient Matrix homeserver that's easy to set up and just works. You can install
it on a mini-computer like the Raspberry Pi to host Matrix for your family,
friends or company.
An efficient Matrix homeserver that's easy to set up and just works. You can
install it on a mini-computer like the Raspberry Pi to host Matrix for your
family, friends or company.
#### Can I try it out?
Yes! You can test our Conduit instance by opening a client that supports registration tokens such as [Element web](https://app.element.io/), [Nheko](https://matrix.org/ecosystem/clients/nheko/) or [SchildiChat web](https://app.schildi.chat/) and registering on the `conduit.rs` homeserver. The registration token is "for_testing_only". Don't share personal information. Once you have registered, you can use any other [Matrix client](https://matrix.org/ecosystem/clients) to login.
An official conduwuit server ran by me is available at transfem.dev
Conduit is Beta, meaning you can join and participate in most
Matrix rooms, but not all features are supported and you might run into bugs
from time to time.
conduwuit is a hard fork of Conduit which is in beta, meaning you can join and
participate in most Matrix rooms, but not all features are supported and you
might run into bugs from time to time.
There are still a few important features missing:
- E2EE emoji comparison over federation (E2EE chat works)
- Outgoing read receipts, typing, presence over federation (incoming works)
<!-- ANCHOR_END: body -->
<!-- ANCHOR: footer -->
#### How can I contribute?
1. Look for an issue you would like to work on and make sure no one else is currently working on it.
2. Tell us that you are working on the issue (comment on the issue or chat in
[#conduit:fachschaften.org](https://matrix.to/#/#conduit:fachschaften.org)). If it is more complicated, please explain your approach and ask questions.
3. Fork the repo, create a new branch and push commits.
4. Submit a MR
#### Contact
If you have any questions, feel free to
- Ask in `#conduit:fachschaften.org` on Matrix
- Write an E-Mail to `conduit@koesters.xyz`
- Send an direct message to `@timokoesters:fachschaften.org` on Matrix
- [Open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new)
If you run into any question, feel free to
#### Security
If you believe you have found a security issue, please send a message to [Timo](https://matrix.to/#/@timo:conduit.rs)
and/or [Matthias](https://matrix.to/#/@matthias:ahouansou.cz) on Matrix, or send an email to
[conduit@koesters.xyz](mailto:conduit@koesters.xyz). Please do not disclose details about the issue to anyone else before
a fix is released publically.
#### Thanks to
Thanks to FUTO, Famedly, Prototype Fund (DLR and German BMBF) and all individuals for financially supporting this project.
Thanks to the contributors to Conduit and all libraries we use, for example:
- Ruma: A clean library for the Matrix Spec in Rust
- axum: A modular web framework
- Ask us in `#conduwuit:puppygock.gay` on Matrix
- [Open an issue on GitHub](https://github.com/girlbossceo/conduwuit/issues/new)
# Enables registration. If set to false, no users can register on this
# server.
# If set to true without a token configured, users can register with no form of 2nd-
# step only if you set
# `yes_i_am_very_very_sure_i_want_an_open_registration_server_prone_to_abuse` to
# true in your config. If you would like
# registration only via token reg, please configure the `registration_token` key.
allow_registration=false
# Please note that an open registration homeserver with no second-step verification
# is highly prone to abuse and potential defederation by homeservers, including
# matrix.org.
# A static registration token that new users will have to provide when creating
# an account. If unset and `allow_registration` is true, registration is open
# without any condition. YOU NEED TO EDIT THIS.
registration_token="change this token for something specific to your server"
# controls whether federation is allowed or not
# defaults to true
# allow_federation = true
# controls whether users are allowed to create rooms.
# appservices and admins are always allowed to create rooms
# defaults to true
# allow_room_creation = true
# controls whether non-admin local users are forbidden from sending room invites (local and remote),
# and if non-admin users can receive remote room invites. admins are always allowed to send and receive all room invites.
# defaults to false
# block_non_admin_invites = false
# List of forbidden username patterns/strings. Values in this list are matched as *contains*.
# This is checked upon username availability check, registration, and startup as warnings if any local users in your database
# have a forbidden username.
# No default.
# forbidden_usernames = []
# List of forbidden room aliases and room IDs as patterns/strings. Values in this list are matched as *contains*.
# This is checked upon room alias creation, custom room ID creation if used, and startup as warnings if any room aliases
# in your database have a forbidden room alias/ID.
# No default.
# forbidden_alias_names = []
# List of forbidden server names that we will block incoming AND outgoing federation with, and block client room joins / remote user invites.
#
# This check is applied on the room ID, room alias, sender server name, sender user's server name, inbound federation X-Matrix origin, and outbound federation handler.
#
# Basically "global" ACLs. No default.
# forbidden_remote_server_names = []
# List of forbidden server names that we will block all outgoing federated room directory requests for. Useful for preventing our users from wandering into bad servers or spaces.
# Set this to true to allow your server's public room directory to be federated.
# Set this to false to protect against /publicRooms spiders, but will forbid external users
# from viewing your server's public room directory. If federation is disabled entirely
# (`allow_federation`), this is inherently false.
allow_public_room_directory_over_federation=false
# Set this to true to allow your server's public room directory to be queried without client
# authentication (access token) through the Client APIs. Set this to false to protect against /publicRooms spiders.
allow_public_room_directory_without_auth=false
# Set this to true to lock down your server's public room directory and only allow admins to publish rooms to the room directory.
# Unpublishing is still allowed by all users with this enabled.
#
# Defaults to false
lockdown_public_room_directory=false
# Set this to true to allow federating device display names / allow external users to see your device display name.
# If federation is disabled entirely (`allow_federation`), this is inherently false. For privacy, this is best disabled.
allow_device_name_federation=false
# Vector list of domains allowed to send requests to for URL previews. Defaults to none.
# Note: this is a *contains* match, not an explicit match. Putting "google.com" will match "https://google.com" and "http://mymaliciousdomainexamplegoogle.com"
# Setting this to "*" will allow all URL previews. Please note that this opens up significant attack surface to your server, you are expected to be aware of the risks by doing so.
url_preview_domain_contains_allowlist=[]
# Vector list of explicit domains allowed to send requests to for URL previews. Defaults to none.
# Note: This is an *explicit* match, not a contains match. Putting "google.com" will match "https://google.com", "http://google.com", but not "https://mymaliciousdomainexamplegoogle.com"
# Setting this to "*" will allow all URL previews. Please note that this opens up significant attack surface to your server, you are expected to be aware of the risks by doing so.
url_preview_domain_explicit_allowlist=[]
# Vector list of URLs allowed to send requests to for URL previews. Defaults to none.
# Note that this is a *contains* match, not an explicit match. Putting "google.com" will match "https://google.com/", "https://google.com/url?q=https://mymaliciousdomainexample.com", and "https://mymaliciousdomainexample.com/hi/google.com"
# Setting this to "*" will allow all URL previews. Please note that this opens up significant attack surface to your server, you are expected to be aware of the risks by doing so.
url_preview_url_contains_allowlist=[]
# Vector list of explicit domains not allowed to send requests to for URL previews. Defaults to none.
# Note: This is an *explicit* match, not a contains match. Putting "google.com" will match "https://google.com", "http://google.com", but not "https://mymaliciousdomainexamplegoogle.com"
# The denylist is checked first before allowlist. Setting this to "*" will not do anything.
url_preview_domain_explicit_denylist=[]
# Maximum amount of bytes allowed in a URL preview body size when spidering. Defaults to 384KB (384_000 bytes)
url_preview_max_spider_size=384_000
# Option to decide whether you would like to run the domain allowlist checks (contains and explicit) on the root domain or not. Does not apply to URL contains allowlist. Defaults to false.
# Example: If this is enabled and you have "wikipedia.org" allowed in the explicit and/or contains domain allowlist, it will allow all subdomains under "wikipedia.org" such as "en.m.wikipedia.org" as the root domain is checked and matched.
# Useful if the domain contains allowlist is still too broad for you but you still want to allow all the subdomains under a root domain.
url_preview_check_root_domain=false
# Config option to allow or disallow incoming federation requests that obtain the profiles
# of our local users from `/_matrix/federation/v1/query/profile`
#
# This is inherently false if `allow_federation` is disabled
#
# Defaults to true
allow_profile_lookup_federation_requests=true
# Config option to automatically deactivate the account of any user who attempts to join a:
# - banned room
# - forbidden room alias
# - room alias or ID with a forbidden server name
#
# This may be useful if all your banned lists consist of toxic rooms or servers that no good faith user would ever attempt to join, and
# to automatically remediate the problem without any admin user intervention.
#
# This will also make the user leave all rooms. Federation (e.g. remote room invites) are ignored here.
#
# Defaults to false as rooms can be banned for non-moderation-related reasons
#auto_deactivate_banned_room_attempts = false
### Admin Room and Console
# Controls whether the conduwuit admin room console / CLI will immediately activate on startup.
# This option can also be enabled with `--console` conduwuit argument
#
# Defaults to false
#admin_console_automatic = false
# Controls what admin commands will be executed on startup. This is a vector list of strings of admin commands to run.
#
# An example of this can be: `admin_execute = ["debug ping puppygock.gay", "debug echo hi"]`
#
# This option can also be configured with the `--execute` conduwuit argument and can take standard shell commands and environment variables
#
# Such example could be: `./conduwuit --execute "server admin-notice conduwuit has started up at $(date)"`
#
# Defaults to nothing.
#admin_execute = [""]
# Controls whether conduwuit should error and fail to start if an admin execute command (`--execute` / `admin_execute`) fails
#
# Defaults to false
#admin_execute_errors_ignore = false
# Controls the max log level for admin command log captures (logs generated from running admin commands)
#
# Defaults to "info" on release builds, else "debug" on debug builds
#admin_log_capture = info
# Allows admins to enter commands in rooms other than #admins by prefixing with \!admin. The reply
# will be publicly visible to the room, originating from the sender.
# defaults to true
#admin_escape_commands = true
### Misc
# max log level for conduwuit. allows debug, info, warn, or error
# see also: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives
# **Caveat**:
# For release builds, the tracing crate is configured to only implement levels higher than error to avoid unnecessary overhead in the compiled binary from trace macros.
# For debug builds, this restriction is not applied.
#
# Defaults to "info"
#log = "info"
# controls whether encrypted rooms and events are allowed (default true)
#allow_encryption = false
# if enabled, conduwuit will send a simple GET request periodically to `https://pupbrain.dev/check-for-updates/stable`
# for any new announcements made. Despite the name, this is not an update check
# endpoint, it is simply an announcement check endpoint.
# Defaults to false.
#allow_check_for_updates = false
# Set to false to disable users from joining or creating room versions that aren't 100% officially supported by conduwuit.
# conduwuit officially supports room versions 6 - 10. conduwuit has experimental/unstable support for 3 - 5, and 11.
# Defaults to true.
#allow_unstable_room_versions = true
# Option to control adding arbitrary text to the end of the user's displayname upon registration with a space before the text.
# This was the lightning bolt emoji option, just replaced with support for adding your own custom text or emojis.
# To disable, set this to "" (an empty string)
# Defaults to "🏳️⚧️" (trans pride flag)
#new_user_displayname_suffix = "🏳️⚧️"
# Option to control whether conduwuit will query your list of trusted notary key servers (`trusted_servers`) for
# remote homeserver signing keys it doesn't know *first*, or query the individual servers first before falling back to the trusted
# key servers.
#
# The former/default behaviour makes federated/remote rooms joins generally faster because we're querying a single (or list of) server
# that we know works, is reasonably fast, and is reliable for just about all the homeserver signing keys in the room. Querying individual
# servers may take longer depending on the general infrastructure of everyone in there, how many dead servers there are, etc.
#
# However, this does create an increased reliance on one single or multiple large entities as `trusted_servers` should generally
# contain long-term and large servers who know a very large number of homeservers.
#
# If you don't know what any of this means, leave this and `trusted_servers` alone to their defaults.
#
# Defaults to true as this is the fastest option for federation.
#query_trusted_key_servers_first = true
# List/vector of room **IDs** that conduwuit will make newly registered users join.
# The room IDs specified must be rooms that you have joined at least once on the server, and must be public.
#
# No default.
#auto_join_rooms = []
# Retry failed and incomplete messages to remote servers immediately upon startup. This is called bursting.
# If this is disabled, said messages may not be delivered until more messages are queued for that server.
# Do not change this option unless server resources are extremely limited or the scale of the server's
# deployment is huge. Do not disable this unless you know what you are doing.
#startup_netburst = true
# Limit the startup netburst to the most recent (default: 50) messages queued for each remote server. All older
# messages are dropped and not reattempted. The `startup_netburst` option must be enabled for this value to have
# any effect. Do not change this value unless you know what you are doing. Set this value to -1 to reattempt
# every message without trimming the queues; this may consume significant disk. Set this value to 0 to drop all
# messages without any attempt at redelivery.
#startup_netburst_keep = 50
# If the 'perf_measurements' feature is enabled, enables collecting folded stack trace profile of tracing spans using
# tracing_flame. The resulting profile can be visualized with inferno[1], speedscope[2], or a number of other tools.
# [1]: https://github.com/jonhoo/inferno
# [2]: www.speedscope.app
# tracing_flame = false
# If 'tracing_flame' is enabled, sets a filter for which events will be included in the profile.
# Supported syntax is documented at https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives
# tracing_flame_filter = "trace,h2=off"
# If 'tracing_flame' is enabled, set the path to write the generated profile.
# tracing_flame_output_path = "./tracing.folded"
# Enable the tokio-console. This option is only relevant to developers.
# See: docs/development.md#debugging-with-tokio-console for more information.
#tokio_console = false
# Enable backward-compatibility with Conduit's media directory by creating symlinks of media. This
# option is only necessary if you plan on using Conduit again. Otherwise setting this to false
# reduces filesystem clutter and overhead for managing these symlinks in the directory. This is now
# disabled by default. You may still return to upstream Conduit but you have to run Conduwuit at
# least once with this set to true and allow the media_startup_check to take place before shutting
# down to return to Conduit.
#
# Disabled by default.
#media_compat_file_link = false
# Prunes missing media from the database as part of the media startup checks. This means if you
# delete files from the media directory the corresponding entries will be removed from the
# database. This is disabled by default because if the media directory is accidentally moved or
# inaccessible the metadata entries in the database will be lost with sadness.
#
# Disabled by default.
#prune_missing_media = false
# Checks consistency of the media directory at startup:
# 1. When `media_compat_file_link` is enbled, this check will upgrade media when switching back
# and forth between Conduit and Conduwuit. Both options must be enabled to handle this.
# 2. When media is deleted from the directory, this check will also delete its database entry.
#
# If none of these checks apply to your use cases, and your media directory is significantly large
# setting this to false may reduce startup time.
#
# Enabled by default.
#media_startup_check = true
# OpenID token expiration/TTL in seconds
#
# These are the OpenID tokens that are primarily used for Matrix account integrations, *not* OIDC/OpenID Connect/etc
#
# Defaults to 3600 (1 hour)
#openid_token_ttl = 3600
# Emergency password feature. This password set here will let you login to the server service account (e.g. `@conduit`)
# and let you run admin commands, invite yourself to the admin room, etc.
#
# no default.
#emergency_password = ""
### Generic database options
# Set this to any float value to multiply conduwuit's in-memory LRU caches with.
# By default, the caches scale automatically with cpu-core-count.
# May be useful if you have significant memory to spare to increase performance.
#
# This was previously called `conduit_cache_capacity_modifier`
#
# Defaults to 1.0.
#cache_capacity_modifier = 1.0
# Set this to any float value in megabytes for conduwuit to tell the database engine that this much memory is available for database-related caches.
# May be useful if you have significant memory to spare to increase performance.
# Defaults to 128.0 + (64.0 * CPU core count).
#db_cache_capacity_mb = 256.0
### RocksDB options
# Set this to true to use RocksDB config options that are tailored to HDDs (slower device storage)
#
# It is worth noting that by default, conduwuit will use RocksDB with Direct IO enabled. *Generally* speaking this improves performance as it bypasses buffered I/O (system page cache).
# However there is a potential chance that Direct IO may cause issues with database operations if your setup is uncommon. This has been observed with FUSE filesystems, and possibly ZFS filesystem.
# RocksDB generally deals/corrects these issues but it cannot account for all setups.
# If you experience any weird RocksDB issues, try enabling this option as it turns off Direct IO and feel free to report in the conduwuit Matrix room if this option fixes your DB issues.
# See https://github.com/facebook/rocksdb/wiki/Direct-IO for more information.
#
# Defaults to false
#rocksdb_optimize_for_spinning_disks = false
# Enables direct-io to increase database performance. This is enabled by default. Set this option to false if the
# database resides on a filesystem which does not support direct-io.
#rocksdb_direct_io = true
# RocksDB log level. This is not the same as conduwuit's log level. This is the log level for the RocksDB engine/library
# which show up in your database folder/path as `LOG` files. Defaults to error. conduwuit will typically log RocksDB errors as normal.
#rocksdb_log_level = "error"
# Max RocksDB `LOG` file size before rotating in bytes. Defaults to 4MB.
#rocksdb_max_log_file_size = 4194304
# Time in seconds before RocksDB will forcibly rotate logs. Defaults to 0.
#rocksdb_log_time_to_roll = 0
# Amount of threads that RocksDB will use for parallelism on database operatons such as cleanup, sync, flush, compaction, etc. Set to 0 to use all your logical threads.
#
# Defaults to your CPU logical thread count.
#rocksdb_parallelism_threads = 0
# Enables idle IO priority for compaction thread. This prevents any unexpected lag in the server's operation and
# is usually a good idea. Enabled by default.
#rocksdb_compaction_ioprio_idle = true
# Enables idle CPU priority for compaction thread. This is not enabled by default to prevent compaction from
# falling too far behind on busy systems.
#rocksdb_compaction_prio_idle = false
# Maximum number of LOG files RocksDB will keep. This must *not* be set to 0. It must be at least 1.
# Defaults to 3 as these are not very useful.
#rocksdb_max_log_files = 3
# Type of RocksDB database compression to use.
# Available options are "zstd", "zlib", "bz2", "lz4", or "none"
# It is best to use ZSTD as an overall good balance between speed/performance, storage, IO amplification, and CPU usage.
# For more performance but less compression (more storage used) and less CPU usage, use LZ4.
# See https://github.com/facebook/rocksdb/wiki/Compression for more details.
#
# "none" will disable compression.
#
# Defaults to "zstd"
#rocksdb_compression_algo = "zstd"
# Level of compression the specified compression algorithm for RocksDB to use.
# Default is 32767, which is internally read by RocksDB as the default magic number and
# translated to the library's default compression level as they all differ.
# See their `kDefaultCompressionLevel`.
#
#rocksdb_compression_level = 32767
# Level of compression the specified compression algorithm for the bottommost level/data for RocksDB to use.
# Default is 32767, which is internally read by RocksDB as the default magic number and
# translated to the library's default compression level as they all differ.
# See their `kDefaultCompressionLevel`.
#
# Since this is the bottommost level (generally old and least used data), it may be desirable to have a very
# high compression level here as it's lesss likely for this data to be used. Research your chosen compression algorithm.
#
#rocksdb_bottommost_compression_level = 32767
# Whether to enable RocksDB "bottommost_compression".
# At the expense of more CPU usage, this will further compress the database to reduce more storage.
# It is recommended to use ZSTD compression with this for best compression results.
# See https://github.com/facebook/rocksdb/wiki/Compression for more details.
#
# Defaults to false as this uses more CPU when compressing.
#rocksdb_bottommost_compression = false
# Level of statistics collection. Some admin commands to display database statistics may require
# this option to be set. Database performance may be impacted by higher settings.
#
# Option is a number ranging from 0 to 6:
# 0 = No statistics.
# 1 = No statistics in release mode (default).
# 2 to 3 = Statistics with no performance impact.
# 3 to 5 = Statistics with possible performance impact.
# 6 = All statistics.
#
# Defaults to 1 (No statistics, except in debug-mode)
# TURN secret to use for generating the HMAC-SHA1 hash apart of username and password generation
#
# this is more secure, but if needed you can use traditional username/password below.
#
# no default
#turn_secret = ""
# TURN username to provide the client
#
# no default
#turn_username = ""
# TURN password to provide the client
#
# no default
#turn_password = ""
# TURN TTL
#
# Default is 86400 seconds
#turn_ttl = 86400
# allow guests/unauthenticated users to access TURN credentials
#
# this is the equivalent of Synapse's `turn_allow_guests` config option. this allows
# any unauthenticated user to call `/_matrix/client/v3/voip/turnServer`.
#
# defaults to false
#turn_allow_guests = false
# Other options not in [global]:
#
#
# Enables running conduwuit with direct TLS support
# It is strongly recommended you use a reverse proxy instead. This is primarily relevant for test suites like complement that require a private CA setup.
# [global.tls]
# certs = "/path/to/my/certificate.crt"
# key = "/path/to/my/private_key.key"
#
# Whether to listen and allow for HTTP and HTTPS connections (insecure!)
# This config option is only available if conduwuit was built with `axum_dual_protocol` feature (not default feature)
# Defaults to false
#dual_protocol = false
# If you are using delegation via well-known files and you cannot serve them from your reverse proxy, you can
# uncomment these to serve them directly from conduwuit. This requires proxying all requests to conduwuit, not just `/_matrix` to work.
#
#[global.well_known]
#server = "matrix.example.com:443"
#client = "https://matrix.example.com"
#
# A single contact and/or support page for /.well-known/matrix/support
# All options here are strings. Currently only supports 1 single contact.
Information about downloading and deploying the Debian package. This may also be referenced for other `apt`-based distros such as Ubuntu.
Information about downloading, building and deploying the Debian package, see
the "Installing Conduit" section in the Deploying docs.
All following sections until "Setting up the Reverse Proxy" be ignored because
this is handled automatically by the packaging.
### Installation
Configuration
-------------
It is recommended to see the [generic deployment guide](../deploying/generic.md) for further information if needed as usage of the Debian package is generally related.
When installed, Debconf generates the configuration of the homeserver
(host)name, the address and port it listens on. This configuration ends up in
`/etc/matrix-conduit/conduit.toml`.
### Configuration
You can tweak more detailed settings by uncommenting and setting the variables
in `/etc/matrix-conduit/conduit.toml`. This involves settings such as the maximum
file size for download/upload, enabling federation, etc.
When installed, the example config is placed at `/etc/conduwuit/conduwuit.toml` as the default config. At the minimum, you will need to change your `server_name` here.
Running
-------
You can tweak more detailed settings by uncommenting and setting the config options
in `/etc/conduwuit/conduwuit.toml`.
The package uses the `matrix-conduit.service` systemd unit file to start and
stop Conduit. It loads the configuration file mentioned above to set up the
environment before running the server.
### Running
This package assumes by default that Conduit will be placed behind a reverse
proxy such as Apache or nginx. This default deployment entails just listening
on `127.0.0.1` and the free port `6167` and is reachable via a client using the URL
<http://localhost:6167>.
The package uses the [`conduwuit.service`](../configuration/examples.md#example-systemd-unit-file) systemd unit file to start and stop conduwuit. The binary is installed at `/usr/sbin/conduwuit`.
At a later stage this packaging may support also setting up TLS and running
stand-alone. In this case, however, you need to set up some certificates and
renewal, for it to work properly.
This package assumes by default that conduwuit will be placed behind a reverse proxy. The default config options apply (listening on `localhost` and TCP port `6167`). Matrix federation requires a valid domain name and TLS, so you will need to set up TLS certificates and renewal for it to work properly if you intend to federate.
Consult various online documentation and guides on setting up a reverse proxy and TLS. Caddy is documented at the [generic deployment guide](../deploying/generic.md#setting-up-the-reverse-proxy) as it's the easiest and most user friendly.
If you run into any problems while setting up an Appservice, write an email to `timo@koesters.xyz`, ask us in [#conduit:fachschaften.org](https://matrix.to/#/#conduit:fachschaften.org) or [open an issue on GitLab](https://gitlab.com/famedly/conduit/-/issues/new).
If you run into any problems while setting up an Appservice: ask us in
[#conduwuit:puppygock.gay](https://matrix.to/#/#conduwuit:puppygock.gay) or
[open an issue on GitHub](https://github.com/girlbossceo/conduwuit/issues/new).
## Set up the appservice - general instructions
Follow whatever instructions are given by the appservice. This usually includes
downloading, changing its config (setting domain, homeserver url, port etc.)
and later starting it.
downloading, changing its config (setting domain, homeserver url, port etc.) and
later starting it.
At some point the appservice guide should ask you to add a registration yaml
file to the homeserver. In Synapse you would do this by adding the path to the
homeserver.yaml, but in Conduit you can do this from within Matrix:
homeserver.yaml, but in conduwuit you can do this from within Matrix:
First, go into the #admins room of your homeserver. The first person that
First, go into the `#admins` room of your homeserver. The first person that
registered on the homeserver automatically joins it. Then send a message into
the room like this:
@conduit:your.server.name: register-appservice
!admin appservices register
```
paste
the
@ -31,13 +33,13 @@ the room like this:
```
You can confirm it worked by sending a message like this:
`@conduit:your.server.name: list-appservices`
`!admin appservices list`
The @conduit bot should answer with `Appservices (1): your-bridge`
The server bot should answer with `Appservices (1): your-bridge`
Then you are done. Conduit will send messages to the appservices and the
Then you are done. conduwuit will send messages to the appservices and the
appservice can send requests to the homeserver. You don't need to restart
Conduit, but if it doesn't work, restarting while the appservice is running
conduwuit, but if it doesn't work, restarting while the appservice is running
could help.
## Appservice-specific instructions
@ -46,16 +48,6 @@ could help.
To remove an appservice go to your admin room and execute
**Conduit** is configured using a TOML file. The configuration file is loaded from the path specified by the `CONDUIT_CONFIG` environment variable.
This chapter describes various ways to configure conduwuit.
> **Note:** The configuration file is required to run Conduit. If the `CONDUIT_CONFIG` environment variable is not set, Conduit will exit with an error.
## Basics
> **Note:** If you update the configuration file, you must restart Conduit for the changes to take effect
conduwuit uses a config file for the majority of the settings, but also supports
setting individual config options via commandline.
> **Note:** You can also configure Conduit by using `CONDUIT_{field_name}` environment variables. To set values inside a table, use `CONDUIT_{table_name}__{field_name}`. Example: `CONDUIT_SERVER_NAME="example.org"`
Please refer to the [example config
file](./configuration/examples.md#example-configuration) for all of those
settings.
Conduit's configuration file is divided into the following sections:
The config file to use can be specified on the commandline when running
conduwuit by specifying the `-c`, `--config` flag. Alternatively, you can use
the environment variable `CONDUWUIT_CONFIG` to specify the config file to used.
Conduit's environment variables are supported for backwards compatibility.
- [Global](#global)
- [TLS](#tls)
- [Proxy](#proxy)
## Option commandline flag
conduwuit supports setting individual config options in TOML format from the
`-O` / `--option` flag. For example, you can set your server name via `-O
server_name=\"example.com\"`.
## Global
Note that the config is parsed as TOML, and shells like bash will remove quotes.
So unfortunately it is required to escape quotes if the config option takes a
string. This does not apply to options that take booleans or numbers:
- `--option allow_registration=true` works ✅
- `-O max_request_size=99999999` works ✅
- `-O server_name=example.com` does not work ❌
- `--option log=\"debug\"` works ✅
- `--option server_name='"example.com'"` works ✅
The `global` section contains the following fields:
## Execute commandline flag
> **Note:** The `*` symbol indicates that the field is required, and the values in **parentheses** are the possible values
conduwuit supports running admin commands on startup using the commandline
argument `--execute`. The most notable use for this is to create an admin user
on first startup.
| Field | Type | Description | Default |
| --- | --- | --- | --- |
| `address` | `string` | The address to bind to | `"127.0.0.1"` |
| `port` | `integer` | The port to bind to | `8000` |
| `tls` | `table` | See the [TLS configuration](#tls) | N/A |
| `server_name`_*_ | `string` | The server name | N/A |
| `database_backend`_*_ | `string` | The database backend to use (`"rocksdb"` *recommended*, `"sqlite"`) | N/A |
| `database_path`_*_ | `string` | The path to the database file/dir | N/A |
| `db_cache_capacity_mb` | `float` | The cache capacity, in MB | `300.0` |
| `enable_lightning_bolt` | `boolean` | Add `⚡️` emoji to end of user's display name | `true` |
| `allow_check_for_updates` | `boolean` | Allow Conduit to check for updates | `true` |
| `conduit_cache_capacity_modifier` | `float` | The value to multiply the default cache capacity by | `1.0` |
| `rocksdb_max_open_files` | `integer` | The maximum number of open files | `1000` |
| `pdu_cache_capacity` | `integer` | The maximum number of Persisted Data Units (PDUs) to cache | `150000` |
| `cleanup_second_interval` | `integer` | How often conduit should clean up the database, in seconds | `60` |
| `max_request_size` | `integer` | The maximum request size, in bytes | `20971520` (20 MiB) |
| `max_concurrent_requests` | `integer` | The maximum number of concurrent requests | `100` |
| `max_fetch_prev_events` | `integer` | The maximum number of previous events to fetch per request if conduit notices events are missing | `100` |
| `allow_registration` | `boolean` | Opens your homeserver to public registration | `false` |
| `registration_token` | `string` | The token users need to have when registering to your homeserver | N/A |
| `allow_encryption` | `boolean` | Allow users to enable encryption in their rooms | `true` |
| `allow_federation` | `boolean` | Allow federation with other servers | `true` |
> **Note:** To run and use Conduit you should probably use it with a Domain or Subdomain behind a reverse proxy (like Nginx, Traefik, Apache, ...) with a Lets Encrypt certificate.
# conduwuit for Docker
## Docker
To run Conduit with Docker you can either build the image yourself or pull it from a registry.
To run conduwuit with Docker you can either build the image yourself or pull it
from a registry.
### Use a registry
OCI images for Conduit are available in the registries listed below. We recommend using the image tagged as `latest` from GitLab's own registry.
OCI images for conduwuit are available in the registries listed below.
The Dockerfile provided by Conduit has two stages, each of which creates an image.
1. **Builder:** Builds the binary from local context or by cloning a git revision from the official repository.
2. **Runner:** Copies the built binary from **Builder** and sets up the runtime environment, like creating a volume to persist the database and applying the correct permissions.
To build the image you can use the following command
which also will tag the resulting image as `matrixconduit/matrix-conduit:latest`.
### Run
When you have the image you can simply run it with
```bash
docker run -d -p 8448:6167 \
-v db:/var/lib/matrix-conduit/ \
-e CONDUIT_SERVER_NAME="your.server.name" \
-e CONDUIT_DATABASE_BACKEND="rocksdb" \
-e CONDUIT_ALLOW_REGISTRATION=true \
-e CONDUIT_ALLOW_FEDERATION=true \
-e CONDUIT_MAX_REQUEST_SIZE="20000000" \
-e CONDUIT_TRUSTED_SERVERS="[\"matrix.org\"]" \
-e CONDUIT_MAX_CONCURRENT_REQUESTS="100" \
-e CONDUIT_PORT="6167" \
--name conduit <link>
-v db:/var/lib/conduwuit/ \
-e CONDUWUIT_SERVER_NAME="your.server.name" \
-e CONDUWUIT_DATABASE_BACKEND="rocksdb" \
-e CONDUWUIT_ALLOW_REGISTRATION=false \
--name conduit $LINK
```
or you can use [docker compose](#docker-compose).
The `-d` flag lets the container run in detached mode. You now need to supply a `conduit.toml` config file, an example can be found [here](../configuration.md).
You can pass in different env vars to change config values on the fly. You can even configure Conduit completely by using env vars, but for that you need
to pass `-e CONDUIT_CONFIG=""` into your container. For an overview of possible values, please take a look at the `docker-compose.yml` file.
The `-d` flag lets the container run in detached mode. You may supply an
optional `conduwuit.toml` config file, the example config can be found
[here](../configuration/examples.md). You can pass in different env vars to
change config values on the fly. You can even configure conduwuit completely by
using env vars. For an overview of possible values, please take a look at the
[`docker-compose.yml`](docker-compose.yml) file.
If you just want to test Conduit for a short time, you can use the `--rm` flag, which will clean up everything related to your container after you stop it.
If you just want to test conduwuit for a short time, you can use the `--rm`
flag, which will clean up everything related to your container after you stop
it.
### Docker compose
### Docker-compose
If the `docker run` command is not for you or your setup, you can also use one of the provided `docker compose` files.
If the `docker run` command is not for you or your setup, you can also use one
of the provided `docker-compose` files.
Depending on your proxy setup, you can use one of the following files;
- If you already have a `traefik` instance set up, use [`docker-compose.for-traefik.yml`](docker-compose.for-traefik.yml)
- If you don't have a `traefik` instance set up (or any other reverse proxy), use [`docker-compose.with-traefik.yml`](docker-compose.with-traefik.yml)
- If you already have a `traefik` instance set up, use
- If you want a setup that works out of the box with `caddy-docker-proxy`, use
[`docker-compose.with-caddy.yml`](docker-compose.with-caddy.yml) and replace all
`example.com` placeholders with your own domain
- For any other reverse proxy, use [`docker-compose.yml`](docker-compose.yml)
When picking the traefik-related compose file, rename it so it matches `docker-compose.yml`, and
rename the override file to `docker-compose.override.yml`. Edit the latter with the values you want
for your server.
Additional info about deploying Conduit can be found [here](generic.md).
When picking the traefik-related compose file, rename it so it matches
`docker-compose.yml`, and rename the override file to
`docker-compose.override.yml`. Edit the latter with the values you want for your
server.
When picking the `caddy-docker-proxy` compose file, it's important to first
create the `caddy` network before spinning up the containers:
```bash
docker network create caddy
```
After that, you can rename it so it matches `docker-compose.yml` and spin up the
containers!
Additional info about deploying conduwuit can be found [here](generic.md).
### Build
To build the Conduit image with docker compose, you first need to open and modify the `docker-compose.yml` file. There you need to comment the `image:` option and uncomment the `build:` option. Then call docker compose with:
To build the conduwuit image with docker-compose, you first need to open and
modify the `docker-compose.yml` file. There you need to comment the `image:`
option and uncomment the `build:` option. Then call docker compose with:
```bash
docker compose up
```
This will also start the container right afterwards, so if want it to run in detached mode, you also should use the `-d` flag.
This will also start the container right afterwards, so if want it to run in
detached mode, you also should use the `-d` flag.
### Run
If you already have built the image or want to use one from the registries, you can just start the container and everything else in the compose file in detached mode with:
If you already have built the image or want to use one from the registries, you
can just start the container and everything else in the compose file in detached
mode with:
```bash
docker compose up -d
@ -112,106 +118,22 @@ docker compose up -d
### Use Traefik as Proxy
As a container user, you probably know about Traefik. It is a easy to use reverse proxy for making
containerized app and services available through the web. With the two provided files,
As a container user, you probably know about Traefik. It is a easy to use
reverse proxy for making containerized app and services available through the
[`docker-compose.with-traefik.yml`](docker-compose.with-traefik.yml)) and [`docker-compose.override.yml`](docker-compose.override.yml) from the repository and remove `.for-traefik` (or `.with-traefik`) from the filename.
2. Open both files and modify/adjust them to your needs. Meaning, change the `CONDUIT_SERVER_NAME` and the volume host mappings according to your needs.
3. Create the `conduit.toml` config file, an example can be found [here](../configuration.md), or set `CONDUIT_CONFIG=""` and configure Conduit per env vars.
4. Uncomment the `element-web` service if you want to host your own Element Web Client and create a `element_config.json`.
5. Create the files needed by the `well-known` service.
- `./nginx/matrix.conf` (relative to the compose file, you can change this, but then also need to change the volume mapping)
7. Connect to your homeserver with your preferred client and create a user. You should do this immediately after starting Conduit, because the first created user is the admin.
[`docker-compose.override.yml`](docker-compose.override.yml), it is equally easy
to deploy and use conduwuit, with a little caveat. If you already took a look at
the files, then you should have seen the `well-known` service, and that is the
little caveat. Traefik is simply a proxy and loadbalancer and is not able to
serve any kind of content, but for conduwuit to federate, we need to either
expose ports `443` and `8448` or serve two endpoints `.well-known/matrix/client`
and `.well-known/matrix/server`.
With the service `well-known` we use a single `nginx` container that will serve
those two files.
## Voice communication
In order to make or receive calls, a TURN server is required. Conduit suggests using [Coturn](https://github.com/coturn/coturn) for this purpose, which is also available as a Docker image. Before proceeding with the software installation, it is essential to have the necessary configurations in place.
### Configuration
Create a configuration file called `coturn.conf` containing:
```conf
use-auth-secret
static-auth-secret=<asecretkey>
realm=<yourserverdomain>
```
A common way to generate a suitable alphanumeric secret key is by using `pwgen -s 64 1`.
These same values need to be set in conduit. You can either modify conduit.toml to include these lines:
Run the [Coturn](https://hub.docker.com/r/coturn/coturn) image using
```bash
docker run -d --network=host -v $(pwd)/coturn.conf:/etc/coturn/turnserver.conf coturn/coturn
```
or docker compose. For the latter, paste the following section into a file called `docker-compose.yml`
and run `docker compose up -d` in the same directory.
```yml
version: 3
services:
turn:
container_name: coturn-server
image: docker.io/coturn/coturn
restart: unless-stopped
network_mode: "host"
volumes:
- ./coturn.conf:/etc/coturn/turnserver.conf
```
To understand why the host networking mode is used and explore alternative configuration options, please visit the following link: https://github.com/coturn/coturn/blob/master/docker/coturn/README.md.
For security recommendations see Synapse's [Coturn documentation](https://github.com/matrix-org/synapse/blob/develop/docs/setup/turn/coturn.md#configuration).
Although you might be able to compile Conduit for Windows, we do recommend running it on a Linux server. We therefore
only offer Linux binaries.
You may simply download the binary that fits your machine. Run `uname -m` to see
what you need.
You may simply download the binary that fits your machine. Run `uname -m` to see what you need. For `arm`, you should use `aarch`. Now copy the appropriate url:
Prebuilt fully static musl binaries can be downloaded from the latest tagged
release [here](https://github.com/girlbossceo/conduwuit/releases/latest) or
`main` CI branch workflow artifact output. These also include Debian packages.
These binaries have jemalloc and io_uring statically linked and included with
them.
**Stable/Main versions:**
Alternatively, you may compile the binary yourself. We recommend using
Nix (or [Lix](https://lix.systems) to build conduwuit as this has the most guaranteed
reproducibiltiy and easiest to get a build environment and output going.
Conduit uses the ports 443 and 8448 both of which need to be open in the firewall.
conduwuit uses the ports 443 and 8448 both of which need to be open in the
firewall.
If Conduit runs behind a router or in a container and has a different public IP address than the host system these public ports need to be forwarded directly or indirectly to the port mentioned in the config.
## Optional: Avoid port 8448
If Conduit runs behind Cloudflare reverse proxy, which doesn't support port 8448 on free plans, [delegation](https://matrix-org.github.io/synapse/latest/delegate.html) can be set up to have federation traffic routed to port 443:
[SRV DNS record](https://spec.matrix.org/latest/server-server-api/#resolving-server-names) delegation is also [possible](https://www.cloudflare.com/en-gb/learning/dns/dns-records/dns-srv-record/).
If conduwuit runs behind a router or in a container and has a different public
IP address than the host system these public ports need to be forwarded directly
or indirectly to the port mentioned in the config.
## Setting up a systemd service
Now we'll set up a systemd service for Conduit, so it's easy to start/stop Conduit and set it to autostart when your
server reboots. Simply paste the default systemd service you can find below into
`/etc/systemd/system/conduit.service`.
The systemd unit for conduwuit can be found
[here](../configuration/examples.md#example-systemd-unit-file). You may need to
change the `ExecStart=` path to where you placed the conduwuit binary.
**You need to make some edits again.** When you are done, run
```bash
# Debian
$ sudo systemctl reload apache2
# Installed from source
$ sudo apachectl -k graceful
```
Lighttpd is not supported as it seems to mess with the `X-Matrix` Authorization
header, making federation non-functional. If using Apache, you need to use
`nocanon` to prevent this.
### Caddy
Create `/etc/caddy/conf.d/conduit_caddyfile` and enter this (substitute for your server name).
Create `/etc/caddy/conf.d/conduwuit_caddyfile` and enter this (substitute for
your server name).
```caddy
```caddyfile
your.server.name, your.server.name:8448 {
reverse_proxy /_matrix/* 127.0.0.1:6167
# TCP reverse_proxy
127.0.0.1:6167
# UNIX socket
#reverse_proxy unix//run/conduwuit/conduwuit.sock
}
```
That's it! Just start or enable the service and you're set.
That's it! Just start and enable the service and you're set.
```bash
$ sudo systemctl enable caddy
sudo systemctl enable --now caddy
```
### Nginx
## You're done
If you use Nginx and not Apache, add the following server section inside the http section of `/etc/nginx/nginx.conf`
```nginx
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
listen 8448 ssl http2;
listen [::]:8448 ssl http2;
server_name your.server.name; # EDIT THIS
merge_slashes off;
# Nginx defaults to only allow 1MB uploads
# Increase this to allow posting large files such as videos
client_max_body_size 20M;
location /_matrix/ {
proxy_pass http://127.0.0.1:6167;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_read_timeout 5m;
}
ssl_certificate /etc/letsencrypt/live/your.server.name/fullchain.pem; # EDIT THIS
ssl_certificate_key /etc/letsencrypt/live/your.server.name/privkey.pem; # EDIT THIS
ssl_trusted_certificate /etc/letsencrypt/live/your.server.name/chain.pem; # EDIT THIS
include /etc/letsencrypt/options-ssl-nginx.conf;
}
```
**You need to make some edits again.** When you are done, run
Now you can start conduwuit with:
```bash
$ sudo systemctl reload nginx
```
## SSL Certificate
If you chose Caddy as your web proxy SSL certificates are handled automatically and you can skip this step.
The easiest way to get an SSL certificate, if you don't have one already, is to [install](https://certbot.eff.org/instructions) `certbot` and run this:
```bash
# To use ECC for the private key,
# paste into /etc/letsencrypt/cli.ini:
# key-type = ecdsa
# elliptic-curve = secp384r1
$ sudo certbot -d your.server.name
```
[Automated renewal](https://eff-certbot.readthedocs.io/en/stable/using.html#automated-renewals) is usually preconfigured.
If using Cloudflare, configure instead the edge and origin certificates in dashboard. In case you’re already running a website on the same Apache server, you can just copy-and-paste the SSL configuration from your main virtual host on port 443 into the above-mentioned vhost.
## You're done!
Now you can start Conduit with:
```bash
$ sudo systemctl start conduit
sudo systemctl start conduwuit
```
Set it to start automatically when your system boots with:
```bash
$ sudo systemctl enable conduit
sudo systemctl enable conduwuit
```
## How do I know it works?
You can open [a Matrix client](https://matrix.org/ecosystem/clients), enter your homeserver and try to register. If you are using a registration token, use [Element web](https://app.element.io/), [Nheko](https://matrix.org/ecosystem/clients/nheko/) or [SchildiChat web](https://app.schildi.chat/), as they support this feature.
You can open [a Matrix client](https://matrix.org/ecosystem/clients), enter your
homeserver and try to register.
You can also use these commands as a quick health check.
You can also use these commands as a quick health check (replace
Conduit can be acquired by Nix from various places:
conduwuit can be acquired by Nix (or [Lix][lix]) from various places:
* The `flake.nix` at the root of the repo
* The `default.nix` at the root of the repo
* From Nixpkgs
* From conduwuit's binary cache
The `flake.nix` and `default.nix` do not (currently) provide a NixOS module, so
(for now) [`services.matrix-conduit`][module] from Nixpkgs should be used to
configure Conduit.
A community maintained NixOS package is available at [`conduwuit`](https://search.nixos.org/packages?channel=unstable&show=conduwuit&from=0&size=50&sort=relevance&type=packages&query=conduwuit)
If you want to run the latest code, you should get Conduit from the `flake.nix`
A binary cache for conduwuit that the CI/CD publishes to is available at the
following places (both are the same just different names):
Here are some of the most frequently asked questions about Conduit, and their answers.
## Why do I get a `M_INCOMPATIBLE_ROOM_VERSION` error when trying to join some rooms?
Conduit doesn't support room versions 1 and 2 at all, and doesn't properly support versions 3-5 currently. You can track the progress of adding support [here](https://gitlab.com/famedly/conduit/-/issues/433).
## How do I backup my server?
To backup your Conduit server, it's very easy.
You can simply stop Conduit, make a copy or file system snapshot of the database directory, then start Conduit again.
> **Note**: When using a file system snapshot, it is not required that you stop the server, but it is still recommended as it is the safest option and should ensure your database is not left in an inconsistent state.
## How do I setup sliding sync?
If you use the [automatic method for delegation](delegation.md#automatic-recommended) or just proxy `.well-known/matrix/client` to Conduit, sliding sync should work with no extra configuration.
If you don't, continue below.
You need to add a `org.matrix.msc3575.proxy` field to your `.well-known/matrix/client` response which contains a url which Conduit is accessible behind.
Here is an example:
```json
{
~ "m.homeserver": {
~ "base_url": "https://matrix.example.org"
~ },
"org.matrix.msc3575.proxy": {
"url": "https://matrix.example.org"
}
}
```
## Can I migrate from Synapse to Conduit?
Not really. You can reuse the domain of your current server with Conduit, but you will not be able to migrate accounts automatically.
Rooms that were federated can be re-joined via the other participating servers, however media and the like may be deleted from remote servers after some time, and hence might not be recoverable.
## How do I make someone an admin?
Simply invite them to the admin room. Once joined, they can administer the server by interacting with the `@conduit:<server_name>` user.
In order to make or receive calls, a TURN server is required. conduwuit suggests
using [Coturn](https://github.com/coturn/coturn) for this purpose, which is also
available as a Docker image.
* It is assumed you have a [Coturn server](https://github.com/coturn/coturn) up and running. See [Synapse reference implementation](https://github.com/element-hq/synapse/blob/develop/docs/turn-howto.md).
### Configuration
## Edit/Add a few settings to your existing conduit.toml
Create a configuration file called `coturn.conf` containing:
```
# Refer to your Coturn settings.
# `your.turn.url` has to match the REALM setting of your Coturn as well as `transport`.
