aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2023-12-04misc: don't allow udisks to automount bcachefs filesystems with no promptHEADorigin/master_2023-12-04masterDarrick J. Wong2-1/+19
The unending stream of syzbot bug reports and overwrought filing of CVEs for corner case handling (i.e. things that distract from actual user complaints) in XFS has generated all sorts of of overheated rhetoric about how every bug is a Serious Security Issue(tm) because anyone can craft a malicious filesystem on a USB stick, insert the stick into a victim machine, and mount will trigger a bug in the kernel driver that leads to some compromise or DoS or something. I thought that nobody would be foolish enough to automount an XFS filesystem. What a fool I was! It turns out that udisks can be told that it's okay to automount things, and then GNOME will do exactly that. Including mounting mangled XFS filesystems! Same with bcachefs! <delete angry rant about poor decisionmaking and armchair fs developers blasting us on X while not actually doing any of the work> Turn off /this/ idiocy by adding a udev rule to tell udisks not to automount bcachefs filesystems. This will not stop a logged in user from unwittingly inserting a malicious storage device and pressing [mount] and getting breached. This is not a substitute for a thorough audit. This is not a substitute for lklfuse. This does not solve the general problem of in-kernel fs drivers being a huge attack surface. I just want to give Kent a break from some of the oceans of bu******. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-30Update bcachefs sources to 71a5b27e017d bcachefs: Make backpointer fsck wb ↵Kent Overstreet12-38/+34
flush check more rigorous Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-29Update bcachefs sources to 676dd269f0f8 mean and variance: Promote to lib/mathKent Overstreet3-5/+15
2023-11-29Update bcachefs sources to eb83f1f842bb mean and variance: Promote to lib/mathKent Overstreet9-107/+100
2023-11-29add a fallback __DECLARE_FLEX_ARRAYKent Overstreet1-2/+5
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-29Update bcachefs sources to c3e4d892b77b mean and variance: Promote to lib/mathKent Overstreet29-176/+224
2023-11-27man: Fix cmd-option formatBrett Holman1-2/+2
Options prgquote and degraded only had a single leading hyphen. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27bcachefs-tools: don't skip stale superblock wipe in force modeBrian Foster1-6/+5
Even though the blkid wipe has been fixed to detect and wipe old superblocks, we still incorrectly skip the wipe when in force mode. Update the force logic in open_for_format() to bypass the user request and otherwise proceed with the wipe. Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27cmd_list_journal: Apply star_start_of_lines() correctlyKent Overstreet1-2/+2
In list_journal, we note which journal entries are being ignored - this was forgetting to mark entries newer than the newest flush. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27Update bcachefs sources to feaca6edbd24 mean and variance: Promote to lib/mathKent Overstreet45-989/+515
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-27Convert to BCH_IOCTL_DEV_USAGE_V2Kent Overstreet3-19/+34
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-25Update bcachefs sources to 8c94740b1bf8 bcachefs: Add missing vaidation for ↵Kent Overstreet36-426/+610
jset_entry_data_usage
2023-11-25cmd_fs_usage: Print out replica set durabilityKent Overstreet3-7/+23
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-24docs: mention that libclang is requiredAlexander Batischev1-1/+2
This fixes the following build failure on Debian bookworm: error: failed to run custom build command for `clang-sys v1.6.1` Caused by: process didn't exit successfully: `/home/minoru/src/bcachefs-tools/rust-src/target/release/build/clang-sys-df95f6d1266be773/build-script-build` (exit status: 101) --- stdout cargo:warning=could not execute `llvm-config` one or more times, if the LLVM_CONFIG_PATH environment variable is set to a full path to valid `llvm-config` executable it will be used to try to find an instance of `libclang` on your system: "couldn't execute `llvm-config --prefix` (path=llvm-config) (error: No such file or directory (os error 2))" --- stderr thread 'main' panicked at /home/minoru/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clang-sys-1.6.1/build/dynamic.rs:206:45: called `Result::unwrap()` on an `Err` value: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])" note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-24improve kmalloc performanceDaniel Hill2-34/+12
Reading from /proc/meminfo is really slow We don't want to start swapping to disk. Deceptively, memory available goes up when we start to swap to disk making performance even worse. To mitigate this: 1. replace reading from meminfo with proper system calls. 2. attempt to lock allocations in physical memory space. 3. check our own allocated memory instead of available memory. 4. still check available memory in the off chance we're trying to play nice with other apps. Signed-off-by: Daniel Hill <daniel@gluo.nz> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-23rust: fix MSRV buildAlexander Fougner5-38/+38
- downgrade clap_completion to 4.3.x (1.65 compatible) - dependency updates - remove unused imports Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-11-22Update bcachefs sources to 783085c3cc44 kbuild: Allow gcov to be enabled on ↵Kent Overstreet71-627/+1304
the command line Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-20nix: add bcachefs-fuse packageFinn Behrens2-2/+6
Add a fuseSupport argument and bcachefs-fuse as flake output with fuse enabled. Signed-off-by: Finn Behrens <me@kloenk.de> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-18fix manpage dateZhai Can1-1/+1
Signed-off-by: Zhai Can <bczhc0@126.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-18manpage: add options for `set-option`Zhai Can1-0/+120
Signed-off-by: Zhai Can <bczhc0@126.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-18update manpageZhai Can1-31/+242
Signed-off-by: Zhai Can <bczhc0@126.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-17Fix build for cmd_fusemount.cKent Overstreet1-4/+4
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-17bcachefs-tools: fix broken libblkid superblock wipeBrian Foster1-3/+8
When playing around with comparing some fstests results on different filesystems, I noticed that a 'bcachefs format' of a previously btrfs-formatted device still continued to mount as btrfs. The reason for this is that the blkid wipe invoked via open_for_format() is not working correctly. blkid_do_wipe() depends on the "SBMAGIC[_OFFSET]" values to do any work, and the associated superblock magic flag is not enabled on the probe. Set the probe flags to explicitly enable the values the bcachefs code depends on in the probe. This includes the type, label and superblock magic information. There are also a couple quirks in the libblkid code that might be worth noting. One is that the superblock enablement and flag setting functions appear hardcoded to return zero, so we just combine the error checks. Second, while blkid_do_wipe() can return an error, it actually doesn't in the scenario being addressed here because it doesn't seem to distinguish between the values being absent because nothing was found by the probe or because the values weren't enabled in the first place. Regardless, add an error check here in the event the wipe does explicitly fail for some unexpected reason. Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16Bump rust edition to 2021Kent Overstreet2-4/+4
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16delete dead codeKent Overstreet1-7/+0
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16Update bcachefs sources to 938f680845d1 fixup! rename and export ↵Kent Overstreet17-267/+162
__kern_path_locked()
2023-11-16Create COPYING.rust-dependencies in release tarballsKent Overstreet1-1/+7
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-16Version is now specified in the makefileKent Overstreet2-2/+3
git describe doesn't work for tarball releases Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-12Update bcachefs sources to 3ca08ab51ec9 bcachefs: six locks: Simplify ↵Kent Overstreet65-973/+1034
optimistic spinning Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-12rust keylocation add none variant and implement ValueEnumFinn Behrens2-23/+32
This enables a possible values help in the clap help text. Signed-Off-By: Finn Behrens <me@kloenk.de> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-10rust-cli: add help on `list` commandZhai Can1-0/+1
2023-11-10add command to generate Rust-part CLI completionsZhai Can8-21/+85
2023-11-09nix: add devShellFinn Behrens1-0/+8
Add a nix devShell output to provide a development environment for nix users. Signed-off-by: Finn Behrens <me@kloenk.de>
2023-11-09nix: fix build add overlay and formatter to flakeFinn Behrens4-48/+37
Adds overlay flake output to expose the bcachefs package and add and run a nix formatter. Co-authored-by: Leona Maroni <dev@leona.is> Signed-off-by: Leona Maroni <dev@leona.is> Signed-off-by: Finn Behrens <me@kloenk.de>
2023-11-08btree_write_buffer: ensure atomic64_sub_return_release availabilityNicholas Sielicki2-0/+7
prior to this patch, on certain platforms (ie: armv7l), compilation fails due to atomic64_sub_return_release not being defined here. Ensure that the atomics header is pulled in, and ensure that it is available in all cases, regardless of whether ATOMIC64_SPINLOCK is defined. Signed-off-by: Nicholas Sielicki <linux@opensource.nslick.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-08rust: mount: use libc::c_ulong for flagsNicholas Sielicki1-2/+2
libc proper treats mount flags as an unsigned long, which is usually u64, except when it isn't. When preparing mount flags, use the libc::c_ulong type instead of u64 to allow for this. This fixes compiling this file under armv7l. Signed-off-by: Nicholas Sielicki <linux@opensource.nslick.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-07Fix build on 32 bitKent Overstreet1-1/+2
size_t is apparently not an unsigned long on 32 bit, which is what rounddown_pow_of_two() returns. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-07make-release-tarball.shKent Overstreet1-0/+54
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-07ci: skip install of recommended packagesAlexander Fougner1-2/+2
Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-11-07ci: bump github actionsAlexander Fougner2-5/+5
- cachix/install-nix-action v22 -> v23 - actions/checkout v3 -> v4 Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-11-06Avoiding picking different bucket sizes for different devicesKent Overstreet3-30/+52
Erasure coding can't create stripes across devices with different bucket sizes - therefore, format shouldn't pick different bucket sizes for different devices. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-06Fix build for cmd_fusermount.cKent Overstreet1-21/+22
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-06cmd_format: Check for device options after device argumentsKent Overstreet1-0/+11
It's a common user error to specify device specific options at the end of a format command, and then not have them apply to any devices - add a check for this. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-04downgrade rust deps to compile for MSRVAlexander Fougner2-22/+66
- add helper for cargo update, whilst respecting MSRV Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-11-03Update bcachefs sources to d464ec667b2b bcachefs: Add missing printk newlinesKent Overstreet16-59/+118
2023-11-02Update, slim down rust dependenciesKent Overstreet5-772/+341
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-11-02Update bcachefs sources to b9bd69421f73 bcachefs: x-macro-ify inode flags enumKent Overstreet38-309/+348
2023-10-27Update bcachefs sources to 6628827a8707 bcachefs: Skip deleted members in ↵Kent Overstreet14-51/+109
member_to_text()
2023-10-27Update bcachefs sources to 7250b2ee5574 bcachefs: Fix deleted inodes btree ↵Kent Overstreet56-882/+1697
in snapshot deletion
2023-10-25Update bcachefs sources to 0d63ed13ea3d closures: Fix race in closure_sync()Kent Overstreet52-798/+1419
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-20Update bcachefs sources to f70a3402188e bcachefs: Fix ca->oldest_gen allocationKent Overstreet8-36/+46
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-19Update bcachefs sources to 24bdb6fed91c bcachefs: bch2_btree_id_str()Kent Overstreet34-130/+138
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-18Update bcachefs sources to a180af9dd349 bcachefs: Refactor memcpy into ↵Kent Overstreet30-126/+156
direct assignment Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-08Change open_for_format to the block io apiHunter Shaffer7-26/+34
Upcoming patch will add device benchmarking at format time, which needs the bio API. Signed-off-by: Hunter Shaffer <huntershaffer182456@gmail.com>
2023-10-08deleted unused path to device in get_size, get_blocksizeHunter Shaffer6-10/+10
Signed-off-by: Hunter Shaffer <huntershaffer182456@gmail.com>
2023-10-08Update bcachefs sources to 7bf1ac0d46 bcachefs: Correctly initialize new ↵Kent Overstreet26-292/+530
buckets on device resize Signed-off-by: Hunter Shaffer <huntershaffer182456@gmail.com>
2023-09-29Handle -EAGAIN from io_submit()Kent Overstreet1-1/+8
If io_submit() returns -EAGAIN, that just means the io context is full and we need to wait for completions - no need to die. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-09-26Update bcachefs sources to a1b6677dca57 bcachefs: Fix looping around ↵Kent Overstreet6-140/+89
bch2_propagate_key_to_snapshot_leaves()
2023-09-26Update bcachefs sources to 1336a995cbc3 bcachefs: Silence transaction ↵Kent Overstreet8-14/+41
restart error message
2023-09-26Create symlinks on install with ln -sfKent Overstreet1-6/+6
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-09-23Update bcachefs sources to f9c612bbf82d bcachefs: Fixes for building in ↵Kent Overstreet114-3673/+4091
userspace Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-09-22Kill shell script wrappersKent Overstreet6-23/+26
nixos has trouble with the shell script wrappers - readlink isn't available by default! But, there's a better approach: just get rid of them and use symlinks instead, and have main() check what we're supposed to be. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-09-19bcachefs-tools: fix endian problems between bit spinlocks and futexesBrian Foster1-11/+51
bcachefs format on a big-endian (s390x) machine crashes down in the rhashtable code imported from the kernel. The reason this occurs lies within the rht_lock() -> bit_spin_lock() code, the latter of which casts bitmaps down to 32-bits to satisfy the requirements of the futex interface. The specific problem here is that a 64 -> 32 bit cast doesn't refer to the lower 8 bytes on a big endian machine, which means setting bit number 0 in the 32-bit map actually corresponds to bit 32 in the 64-bit map. The rhashtable code specifically uses bit zero of the bucket pointer for exclusion and uses native bitops elsewhere (i.e. __rht_ptr()) to identify NULL pointers. If bit 32 of the pointer is set by the locking code instead of bit 0, an otherwise NULL pointer looks like a non-NULL value and results in a segfault. The bit spinlock code imported by the kernel is originally intended to work with unsigned long. The kernel code is able to throttle the cpu directly when under lock contention, while the userspace implementation relies on the futex primitives to emulate reliable blocking. Since the futex interface introduces the 32-bit requirement, isolate the associated userspace hack to that particular code. Restore the native bitmap behavior of the bit spinlock code to address the rhashtable problem described above. Since this is not compatible with the futex interface, create a futex wrapper specifically to convert the native bitmap type to a 32-bit virtual address and mask value for the purposes of waiting/waking when under lock contention. Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-09-19libbcachefs: convert sb features mask to little endianBrian Foster1-1/+1
bch_sb.features is an array of __le64. Convert the native endian format of the features mask appropriately. This causes a bcachefs format to produce an unmountable fs when run from a big endian system. Signed-off-by: Brian Foster <bfoster@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-09-08cmd_mount: request passphrase if the existing key is revokedColin Gillespie1-3/+4
Signed-off-by: Colin Gillespie <colin@cgillespie.xyz>
2023-09-08cmd_set_passphrase: revoke the invalidated keyColin Gillespie3-0/+20
After setting a new passphrase, the previous key is left untouched. This revokes the old key, preventing future actions from using it in error. Signed-off-by: Colin Gillespie <colin@cgillespie.xyz>
2023-09-03cmd_mount: Fix test-only mount pathChristopher James Halse Rogers1-8/+17
The comman line help claims that `bcachefs mount <DEV>` without a mount point will do a dry-run mount - all the steps required to mount the fs, but without actually doing the final real mount. Make the code actually do this, rather than complain that you haven't supplied a mountpoint if you don't provide a mountpoint Signed-off-by: Christopher James Halse Rogers <raof@ubuntu.com>
2023-09-01Fix one second delay when exitingTorge Matthies1-1/+15
Signed-off-by: Torge Matthies <openglfreak@googlemail.com>
2023-08-28Update bcachefs sources to e7f6215768 bcachefs: Fix snapshot_skiplist_good()Kent Overstreet15-672/+971
2023-08-22Fix null ptr deref in bbpos_parse()Kent Overstreet1-0/+4
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-18Update bcachefs sources to bed61fae3b bcachefs: Delete a faulty assertionKent Overstreet3-6/+23
2023-08-17Update bcachefs sources to a8115093df bcachefs: Fix divide by zero in ↵Kent Overstreet43-1251/+1484
rebalance_work()
2023-08-12cmd_mount: use the correct keyring key typeColin Gillespie1-2/+2
Mount is checking for and adding encryption keys using the logon key type instead of the user key type. This was causing it to not be able to unlock volumes on its own, and ask for a passphrase on already unlocked volumes. Signed-off-by: Colin Gillespie <colin@cgillespie.xyz>
2023-08-11Update bcachefs sources to b0788c47d9 bcachefs: Fix check_version_upgrade()Kent Overstreet32-74/+98
2023-08-11cmd_dump: Also set read_onlyKent Overstreet1-0/+1
In nochanges mode, without read_only, we can go into a fake rw mode where we allow writes but hold them in memory. That's not what we want for the dump tool - this fixes a bug where btree nodes don't always get dumped correctly. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-11fuse: Set fsname correctlyKent Overstreet4-1/+21
Also, add helpers for the fuse.bcachefs filesystem type; this means we can now test the fuse version with fstests. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-10cmd_mount: don't return 0 on mount failureLinus Heckemann3-5/+6
Signed-off-by: Linus Heckemann <git@sphalerite.org>
2023-08-08fuse: Update for snapshotsKent Overstreet2-76/+75
cmd_fusermount.c builds again Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-06Update bcachefs sources to 717b356d1d bcachefs: Convert journal validation ↵Kent Overstreet52-5467/+5775
to bkey_invalid_flags Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-04ci: set and verify MSRVAlexander Fougner4-2/+33
To prevent uncontrolled usage of new rust features, directly or from dependencies, set a minimum supported Rust version and verify it builds. Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-08-04Remove sparse protocol envvarAlexander Fougner1-3/+0
Default as of rust 1.70 Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-08-03cmd_dump: Making dumping the entire journal the defaultKent Overstreet1-6/+15
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-03Update bcachefs sources to 5b8c4a1366 bcachefs: ↵Kent Overstreet11-107/+349
bcachefs_metadata_version_deleted_inodes
2023-08-01Update bcachefs sources to 33a60d9b05 bcachefs: Assorted fixes for clangKent Overstreet32-477/+460
2023-08-01cmd_list_journal: Tweak extent matchingKent Overstreet1-8/+7
Extents can now additionally be matched by the end position, the same as normal keys. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-08-01bpos_parse() now handles symbol constantsKent Overstreet1-3/+32
We can now parse U32_MAX/U64_MAX, as printed by bch2_bpos_to_text(). Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-20Run shrinkers when pthread_create failsKent Overstreet1-1/+7
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-20Update bcachefs sources to 4b5917839c bcachefs: Fix a null ptr deref in ↵Kent Overstreet4-8/+11
check_xattr()
2023-07-20Fix 'bcachefs fsck -y'Kent Overstreet1-3/+1
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-20Update bcachefs sources to ee560a3929 bcachefs: Print version, options ↵Kent Overstreet32-501/+611
earlier in startup path
2023-07-20ci: upgrade to checkout action v3Alexander Fougner1-1/+1
v2 is deprecated Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-07-20.gitignore: replace travis with github-ci configJörg Thalheim1-1/+1
2023-07-20add dependabot config to keep github ci actions up-to-dateJörg Thalheim1-0/+6
2023-07-20ci: unpin nix install urlJörg Thalheim1-2/+0
I don't think we no longer need this and the install-nix-action is not compatible with it.
2023-07-20ci: bump install-nix-action/cachix to fix nixos buildJörg Thalheim1-3/+3
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2023-07-15Update bcachefs sources to e14d7c7195 bcachefs: Compression levelsKent Overstreet39-237/+645
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-10Add additional information about encryptionDemi Marie Obenour1-6/+60
This adds additional details about how encryption works in bcachefs, along with a warning regarding snapshots. Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
2023-07-10Update bcachefs sources to 04f2d2ae5b bcachefs: Fix build error on weird gccKent Overstreet2-4/+2
2023-07-10Update bcachefs sources to 070ec8d07b bcachefs: Snapshot depth, skiplist fieldsKent Overstreet12-102/+354
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-10Update bcachefs sources to fb39031ade bcachefs: bch2_sb_maybe_downgrade(), ↵Kent Overstreet6-15/+49
bch2_sb_upgrade() Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-10Update bcachefs sources to 1e3ca87f7b bcachefs: ↵Kent Overstreet8-62/+133
bcachefs_metadata_version_major_minor
2023-07-09Update bcachefs sources to da7d42a9a2 bcachefs: Add new assertions for ↵Kent Overstreet45-583/+514
shutdown path Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-07-07Update bcachefs sources to 25de2b00dc bcachefs: Change check for invalid key ↵Kent Overstreet78-479/+674
types Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-06-27get_random_u32_below()Kent Overstreet1-0/+24
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-06-27Update bcachefs sources to 84f132d569 bcachefs: fsck: Break walk_inode() up ↵Kent Overstreet61-717/+851
into multiple functions Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-06-18Update bcachefs sources to bca25b802d fixup! bcachefs: Fix ↵Kent Overstreet2-8/+10
bch2_check_discard_freespace_key()
2023-06-18Update bcachefs sources to 99175e5712 bcachefs: Fix ↵Kent Overstreet11-47/+92
bch2_check_discard_freespace_key()
2023-06-18Switch to CLOCK_MONOTONIC_COARSEKent Overstreet1-1/+1
CLOCK_MONOTONIC requires a syscall, and is much more expensive. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-06-16build.nix: add instruction to keep cargoDeps hashes up-to-dateJörg Thalheim1-0/+3
2023-06-16nix: replace deprecated cargo/rustc aliases with new versionJörg Thalheim1-2/+4
2023-06-16flake.lock: UpdateJörg Thalheim1-6/+24
This is needed so that nix-update can update cargo dependencies. Flake lock file updates: • Updated input 'nixpkgs': 'github:nixos/nixpkgs/b1f87ca164a9684404c8829b851c3586c4d9f089' (2023-02-25) → 'github:nixos/nixpkgs/0eeebd64de89e4163f4d3cf34ffe925a5cf67a05' (2023-06-12) • Updated input 'utils': 'github:numtide/flake-utils/3db36a8b464d0c4532ba1c7dda728f4576d6d073' (2023-02-13) → 'github:numtide/flake-utils/a1720a10a6cfe8234c0e93907ffe81be440f4cef' (2023-05-31) • Added input 'utils/systems': 'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e' (2023-04-09)
2023-06-15fix reading keys from non-tty inputsJörg Thalheim1-1/+7
read_password_from_tty doesn't have a fallback if input is a pipe. This makes scripting and integration testing harder. Noticed while updating our nixos integration test. Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2023-06-15flake.nix: add missing bindgen output hashJörg Thalheim1-0/+3
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2023-06-13Update bcachefs sources to 01d7ad6d95 bcachefs: snapshot_to_text() includes ↵Kent Overstreet8-128/+70
snapshot tree
2023-06-10Update bcachefs sources to ed6b7f81a7 six locks: Disable percpu read lock ↵Kent Overstreet18-144/+226
mode in userspace Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-06-10cmd_fs_usage: Implement --helpKent Overstreet3-14/+25
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-06-08Sort input file listBernhard M. Wiedemann1-1/+1
so that libbcachefs.so builds in a reproducible way in spite of non-deterministic filesystem readdir order. See https://reproducible-builds.org/ for why this is good.
2023-06-04Update bcachefs sources to 7c0fe6f104 bcachefs: Fix bch2_fsck_ask_yn()Kent Overstreet47-435/+531
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-05-25Update bcachefs sources to 31c09369cd six locks: Fix an unitialized varKent Overstreet21-692/+865
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-05-19Update bcachefs sources to 799716df00 bcachefs: Delete an incorrect ↵Kent Overstreet6-6/+5
bch2_trans_unlock() Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-05-13Update bcachefs sources to 400f275d46 bcachefs: Fix check_overlapping_extents()Kent Overstreet7-43/+27
2023-05-08Update bcachefs sources to 5074caad6a fixup! bcachefs: BTREE_ID_snapshot_treeKent Overstreet2-36/+41
2023-05-08Update bcachefs sources to e990c131de fixup! bcachefs: BTREE_ID_snapshot_treeKent Overstreet4-18/+33
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-05-02Update bcachefs sources to 6a20aede29 bcachefs: Fix quotas + snapshotsKent Overstreet77-712/+1244
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-04-27chore(rust): add opt_get! and expose some FMODE_* as Rust constTruongSinh Tran-Nguyen3-2/+46
In an effort to rewrite `bch2_read_super` from C to Rust, it is neccessary to have `opt_get!` macro defined, and some FMODE_* consts (defined as macro in `include/linux/blkdev.h`) defined as Rust const. Bindgen is currently unable to exapnd C functional macro [1], this this commit use the workaround as introduced in [2]. [1] https://github.com/rust-lang/rust-bindgen/issues/753 [2] https://github.com/rust-lang/rust-bindgen/issues/753#issuecomment-608546390 Signed-off-by: TruongSinh Tran-Nguyen <i@truongsinh.pro>
2023-04-26feat(cmd_mount): Add support for magic keyword `OLD_BLKID_UUID`TruongSinh Tran-Nguyen1-11/+33
This commit enhances mount sub command by allowing it to accept `OLD_BLKID_UUID=xxx` as the device, enabling whole fs mounting without relying on colon-separated devices. Using colon-separated devices in fstab is discouraged due to potential device reference issues after reboot. Additionally, using `UUID=xxx` in fstab or with `mount -t bcachefs` is currently infeasible as it depends on blkid, which prior to v2.39, does not recognize bcachefs. This update enables mounting via fstab using `OLD_BLKID_UUID=xxx` as obtained from `bcachefs show-super` "External UUID". Signed-off-by: TruongSinh Tran-Nguyen <i@truongsinh.pro>
2023-04-26chore: logger for idiomatic style and expanded logging levelsTruongSinh Tran-Nguyen8-66/+48
Improve the Rust logger by adhering to idiomatic Rust conventions and incorporating additional logging levels: warn, debug, and trace. Signed-off-by: TruongSinh Tran-Nguyen <i@truongsinh.pro>
2023-04-26A small compile fixMikulas Patocka1-0/+1
Hi Here I'm sending a small compile fix for bcachefs-tools. Without this patch, I get this error: cargo build --release --manifest-path rust-src/Cargo.toml Compiling bch_bindgen v0.1.0 (/usr/src/git/bcachefs-tools/rust-src/bch_bindgen) error: failed to run custom build command for `bch_bindgen v0.1.0 (/usr/src/git/bcachefs-tools/rust-src/bch_bindgen)` Caused by: process didn't exit successfully: `/usr/src/git/bcachefs-tools/rust-src/target/release/build/bch_bindgen-733e88995ce9eab7/build-script-build` (exit status: 101) --- stderr warning: optimization flag '-fkeep-inline-functions' is not supported [-Wignored-optimization-argument] ../../include/linux/bit_spinlock.h:20:3: error: call to undeclared function 'futex'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] ../../include/linux/bit_spinlock.h:28:2: error: call to undeclared function 'futex'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] ../../include/linux/bit_spinlock.h:39:2: error: call to undeclared function 'futex'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] The futex() function is declared in /usr/include/x86_64-linux-gnu/urcu/futex.h It is not declared in linux/futex.h, so we need to include urcu/futex.h Mikulas Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-04-25Update bcachefs sources to fd6fb298aa bcachefs: Make sure hash info gets ↵Kent Overstreet4-15/+19
initialized in fsck Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-04-21Update bcachefs sources to 2115a2ffde bcachefs: Kill ↵Kent Overstreet10-128/+122
bch2_verify_bucket_evacuated()
2023-04-20Add a NO_RUST optionKent Overstreet2-2/+17
For systems without working rust/llvm/bindgen, add an option for building without Rust. This will be less of an option in the future, as more code gets rewritten in Rust. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-04-20Delete pytestKent Overstreet4-157/+0
These tests have never been useful; drop them. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
2023-04-16Update bcachefs sources to fd381c355c bcachefs: Fix a null ptr deref in fsck ↵Kent Overstreet3-4/+13
check_extents()
2023-04-16Update bcachefs sources to 22ccceee15 bcachefs: Fix a slab-out-of-boundsKent Overstreet2-2/+2
2023-04-15Update bcachefs sources to 504729f99c bcachefs: Allow answering y or n to ↵Kent Overstreet9-27/+170
all fsck errors of given type Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-31Update bcachefs sources to 8fd009dd76 bcachefs: Rip out code for storing ↵Kent Overstreet39-1233/+1546
backpointers in alloc keys Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-19Fix build dependenciesKent Overstreet1-2/+2
Incremental builds are fast again Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-19Add packaging workflowAlexander Fougner1-0/+66
Package and publish deb+rpm for easy testing Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-03-15Update bcachefs sources to 0342eebf85 bcachefs: Improve the backpointer to ↵Kent Overstreet34-178/+306
missing extent message
2023-03-14Update bcachefs sources to 72405e7ff8 bcachefs: Fix ↵Kent Overstreet40-520/+822
bch2_check_extents_to_backpointers()
2023-03-08Add manpage for mount commandAlexander Fougner2-0/+49
Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-03-08Remove cmd_assemble docsAlexander Fougner1-23/+1
Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-03-08linux shim: Fix dropped O_DIRECT flagKent Overstreet1-2/+2
A recent libbcachefs update accidentally committed a change that dropped the O_DIRECT flag - we definitely didn't want to do that. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-08add rpm make target and fix up spec fileEric Sandeen2-34/+33
Add a new make target to build an rpm. This uses rpmbuild --build-in-place, so no SRPM is created. Also clean up the specfile to remove manually-added Requires: that will be auto-generated, add a couple new BuildRequires: and use macros for file locations. Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
2023-03-07Update docsAlexander Fougner4-87/+128
- add some project info for the main readme - migrate readmes to markdown for improved formatting - The main binary now requires the Rust toolchain, so make note of it in the build instructions. Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-03-06cmd_list.rs: Finish list modes, delete cmd_list.cKent Overstreet8-441/+71
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-06Update bcachefs sources to 3856459b1b bcachefs: ↵Kent Overstreet31-195/+404
bch2_btree_iter_peek_node_and_restart() Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-05Add rust toolchain to debian build-depsAlexander Fougner1-2/+3
Fixes building the .deb - also add missing libudev to debian build deps Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-03-04rust: Implement BtreeNodeIterKent Overstreet2-2/+101
This implements BtreeNodeIter, and adds it to cmd_list.rs - the next step in having a full replacement for cmd_list.c Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-04Update bcachefs sources to 171da96d76 bcachefs: Drop some anonymous structs, ↵Kent Overstreet30-143/+297
unions
2023-03-04rust: Fix ptr casting in Fs::open()Kent Overstreet10-83/+136
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-03Update bcachefs sources to 2272c5f5b7 bcachefs: Mark stripe buckets with ↵Kent Overstreet29-157/+328
correct data type
2023-03-02linux shim: hlist_unhashed()Kent Overstreet1-0/+13
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-03-02rust: BkeySC now has correct lifetime on BtreeIterKent Overstreet2-4/+7
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-28rust: Filesystem options now supportedKent Overstreet9-6/+47
This implements opt_set!(), which works exactly the same as the C version and allows filesystem options to be specified in Rust code. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-28rust: BtreeTrans lifetime now marked w.r.t. FsKent Overstreet1-11/+12
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-28rust: BkeyValCKent Overstreet4-42/+121
Now we have a rust-style enum for key types Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-28rust: BkeySCKent Overstreet3-34/+44
Implement a rust equivalent to bkey_s_c, which uses references with the correct lifetimes: now cmd_list.rs doesn't need unsafe. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-28bcachefs: Annotate BtreeIter with BtreeTrans lifetimeKent Overstreet1-7/+10
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-28rust: Simplify bpos FromStrKent Overstreet1-3/+3
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-27rust: BtreeIterFlagsKent Overstreet6-7/+34
This adds a new wrapper type for btree iterator flags. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-27Update bcachefs sources to a5da815430 bcachefs: Convert constants to constsKent Overstreet2-17/+17
2023-02-27Rust: Start of cmd_list rewriteKent Overstreet17-90/+635
This is a _very_ preliminary rewrite of the cmd_list tool in rust, which is intended to be a testing ground for a safe interface in Rust to the core btree interface. This adds rust wrappers for: bch_fs: provides bch2_fs_open(), bch2_fs_stop btree_trans: provides bch2_trans_init(), bch2_trans_exit() btree_iter: provides peek, peek_and_restart, advance bch_errcode: implements Display (wraps bch2_err_str()) bpos: implements Ord (wraps bpos_cmp()) bkey_s_c: implements Display (wraps bch2_bkey_val_to_text()) and other assorted types. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-27Update bcachefs sources to ca97ee3577 bcachefs: ↵Kent Overstreet25-305/+524
bch2_btree_iter_peek_and_restart_outlined() Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-27nix fix up! Rust now integrated into bcachefs binaryDaniel Hill9-356/+74
Heavily simplified to just call make with the required rust dependencies. Signed-off-by: Daniel Hill <daniel@gluo.nz>
2023-02-22Made the -k option for mount.bcachefs default to ask, without messing up ↵Westly Ward1-1/+1
formatting Signed-off-by: Westly Ward <sonicrules1234@gmail.com>
2023-02-21cmd_list_journal: Highlight entries matching transaction filterKent Overstreet1-18/+38
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-21rust-src: Clean up read_super bindingsKent Overstreet4-43/+26
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-21More rust improvementsKent Overstreet6-278/+179
- passing of arguments from c -> rust code now works correctly - 'bcachefs mount' now handles being passed a device or devices Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-21fixup! Rust now integrated into bcachefs binaryKent Overstreet1-1/+1
2023-02-21Rust now integrated into bcachefs binaryKent Overstreet18-173/+151
Rust is now required for building the bcachefs tool, and rust code is now fully integrated with the C codebase - meaning it is possible to call back and forth. The mount helper is now a subcommand, 'mount.bcachefs' is now a small shell wrapper that invokes 'bcachefs mount'. This will make it easier to start rewriting other subcommands in rust, and eventually the whole command line interface. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-20Update bcachefs sources to 8e1519ccb6 bcachefs: Add tracepoint & counter for ↵Kent Overstreet24-278/+480
btree split race Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-17Update bcachefs sources to 1b14994029 bcachefs: Fragmentation LRUKent Overstreet29-344/+474
2023-02-13Fix cpu_relax()Kent Overstreet1-2/+2
cpu_relax() is supposed to be a compiler barrier - this fixes a bug with btree_write_buffer_flush() getting stuck. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-13Update bcachefs sources to 8dbfede1d9 fixup! bcachefs: More info on ↵Kent Overstreet11-61/+106
check_bucket_ref() error
2023-02-12cmd_list_journal: Add filter optionsKent Overstreet3-71/+213
Instead of having to use grep, this adds the ability to print out only transactions that update a particular key, or to filter out entirely keys except those updating certain btrees. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-12Update bcachefs sources to 3e0c5b0722 fixup! bcachefs: Btree write bufferKent Overstreet32-298/+1051
2023-02-11mount: support remountDaniel Hill1-1/+2
remount is handled by libc, not the kernel mount procedure, this fixes remounts. Also sorted the list of flags. Signed-off-by: Daniel Hill <daniel@gluo.nz>
2023-02-10Update bcachefs sources to dab31ca168 bcachefs: Add some logging for btree ↵Kent Overstreet4-10/+43
node rewrites due to errors
2023-02-09Update bcachefs sources to ea93c26e98 fixup! bcachefs: We can handle missing ↵Kent Overstreet105-1442/+2683
btree roots for all alloc btrees Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-02nix: fix compilation issuesDaniel Hill3-158/+545
Update Cargo.lock files to match new dependancies Bump nixpkgs pin
2023-02-01Makefile: build with -Wno-deprecated-declarationsKent Overstreet1-0/+1
This fixes a ZSTD warning. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-02-01cmd_fsck: Add --ratelimit_errors optionKent Overstreet1-8/+14
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-01-31fixup! nix: overhaul build system.Daniel Hill2-9/+4
Signed-off-by: Daniel Hill <daniel@gluo.nz>
2023-01-18rust: replace tracing with loggerAlexander Fougner11-344/+154
tracing framework is a overengineered for simple mount helper. Add a few very barebone logging macros to allow configurable verbosity and colorized output with a small footprint. Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-01-16rust: use rustfmt defaultsAlexander Fougner10-530/+560
follow the kernel style guide, i.e idiomatic rust style. Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-01-15rust: suppress errors on fs probingAlexander Fougner1-0/+5
suppress errors for any non-bchfs blockdevs found during probing Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-01-15rust: support fstab style mountAlexander Fougner7-192/+278
- add support for fstab format, UUID=<uuid> - structopt is no longer actively maintained, replace with clap v4 which support everything structopt can and more. - update dependencies Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2023-01-03make: remove libbcachefs.so when cleaning.Daniel Hill1-1/+1
Signed-off-by: Daniel Hill <daniel@gluo.nz>
2023-01-03nix: overhaul build system.Daniel Hill17-480/+305
Removed outdated overlay. Simply build tooling using bingenHook and propagated*Inputs Signed-off-by: Daniel Hill <daniel@gluo.nz>
2023-01-03rust: update bindgen to 0.63Daniel Hill6-36/+16
Signed-off-by: Daniel Hill <daniel@gluo.nz>
2022-12-29Merge https://github.com/fougner/bcachefs-toolsKent Overstreet1-8/+8
2022-12-29doc: fix formattingAlexander Fougner1-8/+8
fix misnomer and some formatting Signed-off-by: Alexander Fougner <fougner89@gmail.com>
2022-12-21Get current metadata version from sysfsKent Overstreet2-1/+8
This changes the default version for the format command to be the currently supported version, via /sys/modules/bcachefs/parameters/version. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2022-12-19Update bcachefs sources to 0939e1c732 fixup! bcachefs: Rework lru btreeKent Overstreet3-18/+12
2022-12-19Update bcachefs sources to 2e8463c770 bcachefs: Rework lru btreeKent Overstreet7-124/+85
2022-12-19Update bcachefs sources to c801fa69f0 bcachefs: Fix bch_alloc_to_text()Kent Overstreet2-2/+5
2022-12-19Use bch2_err_str() instead of strerror()Kent Overstreet11-34/+41
This correctly prints out our private error codes. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2022-12-19Change memory reclaimKent Overstreet2-22/+63
- Spin up a background thread to call the shrinkers every 1 second - Memory allocations will only call reclaim after a failed allocation, not every single time This will be a major performance boost on allocation intensive workloads. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2022-12-19Update bcachefs sources to 7958ebe324 bcachefs: Fix alloc_v4_backpointers()Kent Overstreet23-145/+328