TudbuT/conduit
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3380 commits 50 branches 7 tags 14 MiB
Commit graph

98 commits

Author SHA1 Message Date
Benjamin Lee
207979579c fix dynamically linked devshell 
This failed to inherit the fix from bec507d7390dda9ca2e624ef846521a20e2252c7
because the crane package's buildInputs become propagatedBuildInputs in
a static stdenv, but become normal buildInputs in a dynamic stdenv. Since
we were only pulling propagatedBuildInputs into the devshell, dynamically
linked devshells did not include the rust-jemalloc-sys package. This
causes tikv-jemalloc-sys to build it's own static jemalloc package, and
we end up loading libc before jemalloc at runtime.
 2024-05-27 04:54:57 -04:00
strawberry
a37b2b9e64 nix: default output to scopeHostStatic instead of scopeHost 
defaults to static builds instead of dynamically linked builds

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
strawberry
630760b5da bump rocksdb to v9.2.1 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
strawberry
bef7dbd1cb finally error on complement diff mismatch, remove jemalloc builds from 
CI

jemalloc is now a default feature

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
strawberry
ba2f22b5d3 nix: remove jemalloc (now default) targets, add jq input for default 
jq input change was from 17eb354590
to prevent unnecessary bindgen rebuilds

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-25 22:28:43 -04:00
morguldir
8ba9b33a95 Make sure we use the liburing of the platform we're building for 
Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-05-24 12:46:15 -04:00
morguldir
70047ff26d Make rocksdb include liburing, and tell gcc the path during the build 
With: strings /nix/store/9skicdac6xs4yww1nd3h7m6xydv4hxlj-rocksdb-9.1.1/lib/librocksdb.so.9|rg io_uring|wc -l
112
With: strings result/bin/conduit |rg io_uring|wc -l
5

Without: strings static-x86_64-unknown-linux-musl-jemalloc |rg io_uring | wc -l
0

Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-05-24 12:46:15 -04:00
Benjamin Lee
1d57e14dc0 set C/LDFLAGS for complement dependencies directly 
Previously we were relying on NIX_CFLAGS_COMPILE, but this is not being
set in static devshells. A cleaner solution for complement would likely
be to build the tests in their own nix derivation instead of building
them in the devshell, but this change unblocks CI for now.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
5d81203277 use a statically-linked binary for complement 
Dynamically-linked jemalloc is broken.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
ad39a34c16 add a dynamically-linked devshell 
This is broken on linux, but can be used by darwin users for development,
since static/jemalloc/darwin is broken.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
a007338b34 mark dynamic jemalloc builds as broken on linux 2024-05-24 10:53:47 -04:00
Benjamin Lee
17cc02ff99 add a 'no-features' devshell for local testing 2024-05-24 10:53:47 -04:00
Benjamin Lee
c0f8253fc5 enable all-features in nix for CI builds 
CI is running `cargo build --all-features`, so we should be passing all
the features to nix as well.

The only thing this currently affects is the jemalloc_prof feature, but if
we add any non-default features that affect nix in the future they should
also be handled correctly now.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
0fd0a5d73c switch default devshell to static linking 
Dynamically-linked jemalloc doesn't work due to link-order issues, and we
want CI to be testing a static binary anyway since that's what we're
publishing in releases.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
4e6fc2f2df factor devshell out into a helper function 
We're planning to add a second devshell with `all-features` for CI.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
a6742ce8a7 remove liburing from devshell 
This doesn't seem to be necessary to build, and the derivation is broken
in pkgsStatic.
 2024-05-24 10:53:47 -04:00
Benjamin Lee
a7fe434086 only link to one jemalloc build 
Without setting JEMALLOC_OVERRIDE, we end up linking to two different
jemalloc builds. Once dynamically, as a transitive dependency through
rocksdb, and a second time to the static jemalloc that tikv-jemalloc-sys
builds.
 2024-05-24 10:53:47 -04:00
strawberry
67569cb9c8 nix: switch to fork of rocksdb input 
db6df0b185
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-06 03:45:10 -04:00
strawberry
a198f0481a nix: add liburing to devshell 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-06 03:45:10 -04:00
strawberry
b66d2d44d0 chore: bump MSRV to 1.77.0 as 1.78.0 came out 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-06 03:45:10 -04:00
strawberry
1181a7a7a9 nix: specify explicit branches/refs for flake inputs 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-03 01:52:29 -04:00
strawberry
a97520b0e9 bump MSRV to 1.76.0 
there's really no point in trying to stay as low as possible for us,
and this makes development easier. Debian users should just use rustup,
Nix users already get the proper toolchains.

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-05-03 01:52:29 -04:00
strawberry
3c45a468f1 bump rocksdb to 9.1.1 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
Charles Hall
06e8b63a3c add cargo-audit to the devshell 
Apparently github actions VMs ship with it and that's how it was working
before? Cursed. We should control our own supply chain and also ensure
that local development uses the same version as CI.
 2024-04-26 02:03:40 -04:00
Charles Hall
63fe828120 use lib.makeScope and files to organize packages 
Some of the improvements here include:

* rocksdb can actually use jemalloc now instead of just pulling in a
  second rocksdb for no reason
* "complement-runtime" factored back out into shell file
* complement image no longer uses `mkDerivation` for `copyToRoot`
  because that's what `buildEnv` is for
* complement image no longer sets `SERVER_NAME`, complement already does
  that
* all packages were factored out into `callPackage`-able files for use
  with a custom `lib.makeScope pkgs.newScope`
* new version of `mkPackage` has options that are easier to use and
  override such as `features`
 2024-04-26 02:03:40 -04:00
Charles Hall
36774322e1 always go through inputs 2024-04-26 02:03:40 -04:00
Charles Hall
5476a36a0b remove dead code 2024-04-26 02:03:40 -04:00
Charles Hall
d2c3275323 get complement via flake inputs 
Flake lock file updates:

• Added input 'complement':
    'github:matrix-org/complement/d73c81a091604b0fc5b6b0617dcac58c25763f57?narHash=sha256-hom/Lt0gZzLWqFhUJG0X2i88CAMIILInO5w0tPj6G3s%3D' (2024-04-18)
 2024-04-26 02:03:40 -04:00
Charles Hall
b635e807ef get rocksdb via flake inputs 
Flake lock file updates:

• Added input 'rocksdb':
    'github:facebook/rocksdb/bcf88d48ce8aa8b536aee4dd305533b3b83cf435?narHash=sha256-vRPyrXkXVVhP56n5FVYef8zbIsnnanQSpElmQLZ7mh8%3D' (2024-04-16)
 2024-04-26 02:03:40 -04:00
Charles Hall
503c0f1076 flatten and sort all flake inputs 2024-04-26 02:03:40 -04:00
Charles Hall
a260308bc9 unpin crane because the bug was fixed 
Flake lock file updates:

• Updated input 'crane':
    'github:ipetkov/crane/2c653e4478476a52c6aa3ac0495e4dea7449ea0e?narHash=sha256-XoXRS%2B5whotelr1rHiZle5t5hDg9kpguS5yk8c8qzOc%3D' (2024-02-11)
  → 'github:ipetkov/crane/55f4939ac59ff8f89c6a4029730a2d49ea09105f?narHash=sha256-Vz1KRVTzU3ClBfyhOj8gOehZk21q58T1YsXC30V23PU%3D' (2024-04-21)
 2024-04-26 02:03:40 -04:00
AwesomeQubic
999cc7ccf5 possibly fix macOS builds for nix 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
morguldir
6c0d527b90 Use jemalloc/hmalloc for cross builds 
Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-04-26 02:03:40 -04:00
morguldir
056c9d6920 Since we use crane.buildPackage we need to use cargoExtraArgs 
Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-04-26 02:03:40 -04:00
morguldir
3ebf1082d6 Base oci-images on their matching alloc variant 
Co-authored-by: AwesomeQubic <ThatQubicWah@protonmail.com>
Signed-off-by: morguldir <morguldir@protonmail.com>
 2024-04-26 02:03:40 -04:00
AwesomeQubic
30b5142ecc fix flake for macos, fix jemalloc/hmalloc builds 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
strawberry
2a987ca67a try using upstream rocksdb again 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
AwesomeQubic
90fc2bf53e add Complement support to the nix flake 
Co-authored-by: strawberry <strawberry@puppygock.gay>
Signed-off-by: AwesomeQubic <ThatQubicWah@protonmail.com>
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
strawberry
3d445dd984 bump rocksdb to 9.1.0 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-26 02:03:40 -04:00
strawberry
11a2da3819 fix flake for other oci images too 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-17 14:51:08 -04:00
strawberry
fcda7252c3 fix flake 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-17 14:51:08 -04:00
strawberry
e95e4b9200 revert tag name in nix flake for OCI images 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-17 14:51:08 -04:00
morguldir
b303a774d8 Set the time of the HEAD commit as the OCI created field 
Apparently it uses `date -Iseconds` to parse, so we can use @ with a timestamp

Also it doesn't parse `created` in buildImage, only buildLayeredImage

Signed-off-by: morguldir <morguldir@protonmail.com>
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-17 14:51:08 -04:00
strawberry
5015fc7a2c add ci and flake support for using ref name to docker image tag 
also runs ci on dev branch

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-17 14:51:08 -04:00
strawberry
38b15418ca remove created date from OCI image generation 
dockerhub and github container registry don't like this,
and i have no idea what to do.

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-15 07:01:00 -04:00
strawberry
775191d5c1 dont include the timestamp in the docker image created date 
i hate this

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-15 00:55:07 -04:00
strawberry
71611f0ae5 nix: try lastModifiedDate for oci image created date 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-14 22:35:23 -04:00
strawberry
89c22435d5 nix: simplify malloc outputs 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-14 22:35:23 -04:00
strawberry
354487fa10 nix: use HEAD commit's date for docker image creation and reproducible images 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-14 22:35:23 -04:00
strawberry
a94387dcdc maybe add multiple outputs support to nix flake for jemalloc and hardened_malloc 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-14 22:35:23 -04:00