TudbuT/conduit
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3967 commits 50 branches 7 tags 14 MiB
Commit graph

116 commits

Author SHA1 Message Date
strawberry
60927c1c72 ci: try using sccache 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-25 22:50:10 -04:00
strawberry
934ab6a4fe nix: add dpkg to devshell packages for dpkg-deb 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-25 22:50:10 -04:00
strawberry
09680f653f nix: add outputs for debug builds 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-24 19:55:49 -04:00
strawberry
1797fec3c9 nix: disable USE_RTTI for rocksdb 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-15 22:13:05 -04:00
strawberry
f0c63c539b nix: dont build more unnecessary tests/tools for rocksdb 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-14 12:08:18 -04:00
strawberry
df28359a19 nix: dont build rocksdb core tools (ldb / sst_dump) (-DWITH_CORE_TOOLS=1) 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-14 12:08:18 -04:00
strawberry
a1bfd7a018 nix: dont include experimental feature on all-features builds 
this is just future infra for it

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-14 12:06:52 -04:00
strawberry
2c0bfac43e nix: bump flake.lock and rocksdb to v9.4.0 
• Updated input 'attic':
    'github:zhaofengli/attic/717cc95983cdc357bc347d70be20ced21f935843' (2024-06-01)
  → 'github:zhaofengli/attic/6139576a3ce6bb992e0f6c3022528ec233e45f00' (2024-07-09)
• Updated input 'complement':
    'github:matrix-org/complement/bc97f1ddc1cd7485faf80c8935ee2641f3e1b57c' (2024-07-02)
  → 'github:matrix-org/complement/0d14432e010482ea9e13a6f7c47c1533c0c9d62f' (2024-07-10)
• Updated input 'crane':
    'github:ipetkov/crane/0aed560c5c0a61c9385bddff471a13036203e11c' (2024-07-06)
  → 'github:ipetkov/crane/2d83156f23c43598cf44e152c33a59d3892f8b29' (2024-07-09)
• Updated input 'fenix':
    'github:nix-community/fenix/abc0549e3560189462a7d394cc9d50af4608d103' (2024-07-08)
  → 'github:nix-community/fenix/5087b12a595ee73131a944d922f24d81dae05725' (2024-07-13)
• Updated input 'fenix/rust-analyzer-src':
    'github:rust-lang/rust-analyzer/a5b21ea0aa644dffd7cf958b43f11f221d53404e' (2024-07-07)
  → 'github:rust-lang/rust-analyzer/ffbc5ad993d5cd2f3b8bcf9a511165470944ab91' (2024-07-11)
• Updated input 'liburing':
    'github:axboe/liburing/7b3245583069bd481190c9da18f22e9fc8c3a805' (2024-06-22)
  → 'github:axboe/liburing/1d674f83b7d0f07553ac44d99a401b05853d9dbe' (2024-07-12)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/655a58a72a6601292512670343087c2d75d859c1' (2024-07-08)
  → 'github:NixOS/nixpkgs/7e7c39ea35c5cdd002cd4588b03a3fb9ece6fad9' (2024-07-12)
• Updated input 'rocksdb':
    'github:girlbossceo/rocksdb/a935c0273e1ba44eacf88ce3685a9b9831486155' (2024-07-02)
  → 'github:girlbossceo/rocksdb/911f4243e69c2e320a7a209bf1f5f3ff5f825495' (2024-07-13)

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-14 12:06:52 -04:00
strawberry
f129d90900 nix: delete -DWITH_SNAPPY=1 from rocksdb cmakeFlags 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-10 22:00:46 -04:00
strawberry
a8e690f22b nix: unset patches in rocksdb 
we have the patch already and i dont want to make it nix
exclusive by reverting it in my rocksdb fork

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-10 22:00:46 -04:00
strawberry
7a4bbe2ff6 nix: drop hardened_malloc feature from all-features builds 
this is non-functional and i've spent far too long trying to figure
out what the issue is

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-10 22:00:46 -04:00
strawberry
53fa7c3729 nix: add all-features outputs/packages 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-10 22:00:46 -04:00
strawberry
68ad351f84 nix: partially remove some liburing overrides 
the argument one is weird and causing build script invalid arguments

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-10 22:00:46 -04:00
strawberry
74791642c1 add markdownlint and run lychee in documentation CI workflow 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-03 11:30:16 -04:00
strawberry
8deba94a12 update various deps, update rocksdb to v9.3.1 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-07-02 16:46:13 -04:00
strawberry
cd3e7394bf nix: support pushing to conduwuit.cachix.org 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-06-16 13:16:51 -04:00
Benjamin Lee
8a1848a814 Revert "nix: default output to scopeHostStatic instead of scopeHost" 
This reverts commit a37b2b9e64.

Dynamic builds are working again, so we'd prefer having that be the
default output for consistency with nixpkgs.
 2024-05-27 04:54:57 -04:00
Benjamin Lee
b4cd8e9140 fix dynamic builds with liburing 
The original implementation of this was really weird, so I restructed it
a lot while debugging, and am just gonna leave the restructured version.

Root cause of the segfault seems to be that upstream nixpkgs liburing
derivation is generating both static and dynamic libraries, causing
rocksdb to statically link liburing in a dynamic build, pulling in some
allocator stuff at the same time. I created a PR[1] to fix this upstream,
but it probably won't be available on nixos-unstable for quite a while,
so we can also patch it locally.

[1]: https://github.com/NixOS/nixpkgs/pull/314945
 2024-05-27 04:54:57 -04:00
Benjamin Lee
207979579c fix dynamically linked devshell 
This failed to inherit the fix from bec507d7390dda9ca2e624ef846521a20e2252c7
because the crane package's buildInputs become propagatedBuildInputs in
a static stdenv, but become normal buildInputs in a dynamic stdenv. Since
we were only pulling propagatedBuildInputs into the devshell, dynamically
linked devshells did not include the rust-jemalloc-sys package. This
causes tikv-jemalloc-sys to build it's own static jemalloc package, and
we end up loading libc before jemalloc at runtime.
 2024-05-27 04:54:57 -04:00
strawberry
a37b2b9e64 nix: default output to scopeHostStatic instead of scopeHost 
defaults to static builds instead of dynamically linked builds

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
strawberry
630760b5da bump rocksdb to v9.2.1 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
strawberry
bef7dbd1cb finally error on complement diff mismatch, remove jemalloc builds from 
CI

jemalloc is now a default feature

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
strawberry
ba2f22b5d3 nix: remove jemalloc (now default) targets, add jq input for default 
jq input change was from 17eb354590
to prevent unnecessary bindgen rebuilds

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
morguldir
8ba9b33a95 Make sure we use the liburing of the platform we're building for 
Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-05-24 12:46:15 -04:00
morguldir
70047ff26d Make rocksdb include liburing, and tell gcc the path during the build 
With: strings /nix/store/9skicdac6xs4yww1nd3h7m6xydv4hxlj-rocksdb-9.1.1/lib/librocksdb.so.9|rg io_uring|wc -l
112
With: strings result/bin/conduit |rg io_uring|wc -l
5

Without: strings static-x86_64-unknown-linux-musl-jemalloc |rg io_uring | wc -l
0

Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-05-24 12:46:15 -04:00
Benjamin Lee
1d57e14dc0 set C/LDFLAGS for complement dependencies directly 
Previously we were relying on NIX_CFLAGS_COMPILE, but this is not being
set in static devshells. A cleaner solution for complement would likely
be to build the tests in their own nix derivation instead of building
them in the devshell, but this change unblocks CI for now.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
5d81203277 use a statically-linked binary for complement 
Dynamically-linked jemalloc is broken.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
ad39a34c16 add a dynamically-linked devshell 
This is broken on linux, but can be used by darwin users for development,
since static/jemalloc/darwin is broken.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
a007338b34 mark dynamic jemalloc builds as broken on linux 2024-05-24 10:53:47 -04:00
Benjamin Lee
17cc02ff99 add a 'no-features' devshell for local testing 2024-05-24 10:53:47 -04:00
Benjamin Lee
c0f8253fc5 enable all-features in nix for CI builds 
CI is running `cargo build --all-features`, so we should be passing all
the features to nix as well.

The only thing this currently affects is the jemalloc_prof feature, but if
we add any non-default features that affect nix in the future they should
also be handled correctly now.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
0fd0a5d73c switch default devshell to static linking 
Dynamically-linked jemalloc doesn't work due to link-order issues, and we
want CI to be testing a static binary anyway since that's what we're
publishing in releases.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
4e6fc2f2df factor devshell out into a helper function 
We're planning to add a second devshell with `all-features` for CI.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
a6742ce8a7 remove liburing from devshell 
This doesn't seem to be necessary to build, and the derivation is broken
in pkgsStatic.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
a7fe434086 only link to one jemalloc build 
Without setting JEMALLOC_OVERRIDE, we end up linking to two different
jemalloc builds. Once dynamically, as a transitive dependency through
rocksdb, and a second time to the static jemalloc that tikv-jemalloc-sys
builds.
 2024-05-24 10:53:47 -04:00
strawberry
67569cb9c8 nix: switch to fork of rocksdb input 
db6df0b185
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-06 03:45:10 -04:00
strawberry
a198f0481a nix: add liburing to devshell 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-06 03:45:10 -04:00
strawberry
b66d2d44d0 chore: bump MSRV to 1.77.0 as 1.78.0 came out 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-06 03:45:10 -04:00
strawberry
1181a7a7a9 nix: specify explicit branches/refs for flake inputs 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-03 01:52:29 -04:00
strawberry
a97520b0e9 bump MSRV to 1.76.0 
there's really no point in trying to stay as low as possible for us,
and this makes development easier. Debian users should just use rustup,
Nix users already get the proper toolchains.

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-03 01:52:29 -04:00
strawberry
3c45a468f1 bump rocksdb to 9.1.1 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
Charles Hall
06e8b63a3c add cargo-audit to the devshell 
Apparently github actions VMs ship with it and that's how it was working
before? Cursed. We should control our own supply chain and also ensure
that local development uses the same version as CI.
 2024-04-26 02:03:40 -04:00
Charles Hall
63fe828120 use lib.makeScope and files to organize packages 
Some of the improvements here include:

* rocksdb can actually use jemalloc now instead of just pulling in a
  second rocksdb for no reason
* "complement-runtime" factored back out into shell file
* complement image no longer uses `mkDerivation` for `copyToRoot`
  because that's what `buildEnv` is for
* complement image no longer sets `SERVER_NAME`, complement already does
  that
* all packages were factored out into `callPackage`-able files for use
  with a custom `lib.makeScope pkgs.newScope`
* new version of `mkPackage` has options that are easier to use and
  override such as `features`
 2024-04-26 02:03:40 -04:00
Charles Hall
36774322e1 always go through inputs 2024-04-26 02:03:40 -04:00
Charles Hall
5476a36a0b remove dead code 2024-04-26 02:03:40 -04:00
Charles Hall
d2c3275323 get complement via flake inputs 
Flake lock file updates:

• Added input 'complement':
    'github:matrix-org/complement/d73c81a091604b0fc5b6b0617dcac58c25763f57?narHash=sha256-hom/Lt0gZzLWqFhUJG0X2i88CAMIILInO5w0tPj6G3s%3D' (2024-04-18)
 2024-04-26 02:03:40 -04:00
Charles Hall
b635e807ef get rocksdb via flake inputs 
Flake lock file updates:

• Added input 'rocksdb':
    'github:facebook/rocksdb/bcf88d48ce8aa8b536aee4dd305533b3b83cf435?narHash=sha256-vRPyrXkXVVhP56n5FVYef8zbIsnnanQSpElmQLZ7mh8%3D' (2024-04-16)
 2024-04-26 02:03:40 -04:00
Charles Hall
503c0f1076 flatten and sort all flake inputs 2024-04-26 02:03:40 -04:00
Charles Hall
a260308bc9 unpin crane because the bug was fixed 
Flake lock file updates:

• Updated input 'crane':
    'github:ipetkov/crane/2c653e4478476a52c6aa3ac0495e4dea7449ea0e?narHash=sha256-XoXRS%2B5whotelr1rHiZle5t5hDg9kpguS5yk8c8qzOc%3D' (2024-02-11)
  → 'github:ipetkov/crane/55f4939ac59ff8f89c6a4029730a2d49ea09105f?narHash=sha256-Vz1KRVTzU3ClBfyhOj8gOehZk21q58T1YsXC30V23PU%3D' (2024-04-21)
 2024-04-26 02:03:40 -04:00
AwesomeQubic
999cc7ccf5 possibly fix macOS builds for nix 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00