| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The test case relies on `--delete-qgroup` option which has been removed.
The feature needs to be implemented in kernel as it's more complex than
just calling an ioctl. The test case does not take the complexity of
subvolume dropping into consideration and only tested the simplest
cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
This reverts commit 9da773aa46ba33a9c3bdd83b31e15b031b3bfe4d.
There are several problems related to the --delete-qgroup option:
- Currently kernel doesn't allow to delete non-empty qgroups
- A qgroup can only be empty after fully dropped and a transaction is
committed
The tool doesn't take either factor into consideration
- Things like drop_subtree_threshold or other operations can mark qgroup
inconsistent and skip accounting
This can mean the target qgroup will never be empty until next rescan
On the other hand, even we do it the proper way, it would hugely delay
the command (wait until the subvolume to be cleaned).
Furthermore, even if the waiting is handled properly,
drop_subtree_threshold can still prevent us deleting the qgroup (qgroup
numbers are inconsistent, and accounting is skipped completely).
So the qgroup cleanup needs kernel to make it work properly.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
Since commit 8049446bb0ba ("btrfs-progs: docs: placeholder for
contents.rst file on older sphinx version"), on systems with much newer
sphinx-build, "make" would not work for Documentation directory:
$ make clean-all && ./autogen.sh && ./configure --prefix=/usr/ && make -j12
$ ls -alh Documentation/_build
ls: cannot access 'Documentation/_build': No such file or directory
The sphinx-build has a much newer version:
$ sphinx-build --version
sphinx-build 7.2.6
[CAUSE]
On systems which don't need the workaround, the phony target of
contents.rst seems to cause a dependency loop:
GNU Make 4.4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'Makefile'...
Updating makefiles....
Considering target file 'Makefile'.
Looking for an implicit rule for 'Makefile'.
Trying pattern rule '%:' with stem 'Makefile'.
Found implicit rule '%:' for 'Makefile'.
Finished prerequisites of target file 'Makefile'.
No need to remake target 'Makefile'.
Updating goal targets....
Considering target file 'contents.rst'.
File 'contents.rst' does not exist.
Finished prerequisites of target file 'contents.rst'.
Must remake target 'contents.rst'.
Makefile:35: update target 'contents.rst' due to: target is .PHONY
if [ "$(sphinx-build --version | cut -d' ' -f2)" \< "1.7.7" ]; then \
touch contents.rst; \
fi
Putting child 0x64ee81960130 (contents.rst) PID 66833 on the chain.
Live child 0x64ee81960130 (contents.rst) PID 66833
Reaping winning child 0x64ee81960130 PID 66833
Removing child 0x64ee81960130 PID 66833 from chain.
Successfully remade target file 'contents.rst'.
All the default make doing is just try to generate contents.rst, but
since we have much newer version, we won't generate the file at all.
[FIX]
Instead of a phony target, just move the contents.rst generation into
man page target so that we won't cause loop target on contents.rst.
Fixes: 8049446bb0ba ("btrfs-progs: docs: placeholder for contents.rst file on older sphinx version")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add some missing entries. Changes to supported levels:
- increase to 6.8 from 6.7 where applicable, there were fixes to squota
and temp-fsid
- raid-stripe-tree declares support from 6.7, however this is still
behind CONFIG_BTRFS_DEBUG option in kernel, there are some bugs
and the known lack of RAID56 support
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use the objectid, type, offset natural order as it's more readable and
we're used to read keys like that.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
device
A user who wants to shrink a btrfs filesystem within some other logical
device (like a partition) will likely want to adapt the size of the
underlying device, too.
This commit adds documentation that describes how the length of the
portion that btrfs uses of some device can be found out.
Thanks go out to Roman Mamedov <rm@romanrm.net> for hinting `btrfs
filesystem show` as alternative command.
Note: the granularity is one sectorsize and the input values are
silently rounded down to avoid bugs from converted filesystems that
would not adhere to the native btrfs constraints.
[ci skip]
Pull-request: #775
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Being able to expand (“can”) the partition beforehand is not enough – it must
actually be done.
Pull-request: #775
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Pull-request: #773
Author: Chris Mayo <aklhfex@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Copy the images from wiki so that we don't need to jump around the web
search results.
[ci skip]
Pull-request: #771
Signed-off-by: Austin Chang <austin880625@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
This is a followup to 884a609a77a6dd ("btrfs-progs: add basename
wrappers for unified semantics"). Test cli/019-subvolume-create-parents
fails as there are paths with trailing slashes.
The GNU semantics does not change the argument of basename(3) but this
is problematic with trailing slashes. This is not uncommon and could
potentially break things.
To minimize impact of the basename behaviour depending on the include of
libgen.h use the single wrapper in path utils that has to include libgen
anyway for dirname. Our code passes writable buffers to basename.
Issue: #778
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
What basename(3) does with the argument depends on _GNU_SOURCE and
inclusion of libgen.h. This is problematic on Musl (1.2.5) as reported.
We want the GNU semantics that does not modify the argument. Common way
to make it portable is to add own helper. This is now implemented in
path_basename() that does not use the libc provided basename but preserves
the semantics. The path_dirname() is just for parity, otherwise same as
dirname().
Sources:
- https://bugs.gentoo.org/926288
- https://git.musl-libc.org/cgit/musl/commit/?id=725e17ed6dff4d0cd22487bb64470881e86a92e7
Issue: #778
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
GCC 14 is available, add it as default compiler to tumbleweed image to
catch new warnings.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Correct a typo by replacing "then" with "than".
Signed-off-by: oluceps <linux@oluceps.uk>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Some steps don't seem to have a message printed when they start, like
the tree-log clearing or checksum fill phase.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There's an early check of some critical roots right after opening the
filesystem but there's only one message. Check the same roots but print
message for each so it's more specific.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use the right helper for unrecognized options so only the unknown one is
printed instead of the whole help text. Also move the case to the end as
is common elsewhere.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
cmds/inspect.c:1193:1: warning: leak of ‘ctx.stats’ [CWE-401] [-Wanalyzer-malloc-leak]
There are mixed returns and gotos for error handling and the returns
miss freeing of the ctx.stats. Unify all paths to the single label that
frees the buffers and rename it.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
cmds/scrub.c:1150:25: warning: use of possibly-NULL ‘path’ where non-null expected [CWE-690] [-Wanalyzer-possible-null-argument]
Initialization of the datafile path is done from a static string but the
strdup() call is not handled. Store the path directly to the buffer,
it's later modified by mkdir_p().
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
cmds/subvolume.c:1078:39: warning: use of possibly-NULL ‘name’ where non-null expected [CWE-690] [-Wanalyzer-possible-null-argument]
The failure name duplication is not handled and can potentially lead to
a NULL dereference later. Handle the error properly and return template
error message.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_util_deleted_subvolumes_fd()
Reported by 'gcc -fanalyzer':
libbtrfsutil/subvolume.c:415:20: warning: dereference of NULL ‘subvol’ [CWE-476] [-Wanalyzer-null-dereference]
The analyzer found a path where the NULL pointer passed as argument to
btrfs_util_subvolume_info_fd() could be dereferenced. This is unlikely
unless there's a corruption on the disk as the header->offset would have
to be 0. Pass a valid temporary buffer so this does not happen but
there's no use of it otherwise.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_test_for_multiple_profiles()
Reported by 'gcc -fanalyzer':
common/utils.c:1203:9: warning: use of uninitialized value ‘data’ [CWE-457] [-Wanalyzer-use-of-uninitialized-value]
There are several return parameters passed to
btrfs_get_string_for_multiple_profiles(), in case it fails early no
values are assigned so the free() would be called on some stack
initialization value. Initialize all the pointers.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
kernel-shared/print-tree.c:1745:12: warning: check of ‘eb’ for NULL after already dereferencing it [-Wanalyzer-deref-before-check]
The fs_info is initialized before we check 'eb' but we always get a
valid one so no need to validate it.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
kernel-shared/extent_io.c: In function ‘read_raid56’:
./include/kerncompat.h:393:18: warning: dereference of NULL ‘pointers’ [CWE-476] [-Wanalyzer-null-dereference]
After allocation of the pointers array fails it's dereferenced in the
exit block. We can return immediately instead.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
cmds/replace.c:357:17: warning: double ‘close’ of file descriptor ‘fdmnt’ [CWE-1341] [-Wanalyzer-fd-double-close]
The first close is done right before going to the label
'leave_with_error' but the variable is not reset to -1 so in the exit
block close() is called again.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
common/format-output.c:168:1: warning: missing call to ‘va_end’ [-Wanalyzer-va-list-leak]
There's a temporary va_list used infmt_set_unquoted() but va_copy() must
be paired with va_end(), which is missing.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
common/path-utils.c:401:16: warning: use of possibly-NULL ‘curr_dir’ where non-null expected [CWE-690] [-Wanalyzer-possible-null-argument]
There's an unhandled strdup() call in path_is_in_dir() so tmp could be
potentially NULL and passed down in the function. This is in the path
utilities so we assume the buffer is a path and can use the safe copy.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer:
common/string-table.c:62:17: warning: leak of ‘msg’ [CWE-401] [-Wanalyzer-malloc-leak]
The 'msg' still allocated when returning from the function due to error,
free it.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported by 'gcc -fanalyzer':
common/device-scan.c:222:20: warning: dereference of NULL ‘device’ [CWE-476] [-Wanalyzer-null-dereference]
If the allocation of device fails then we can't free device->zone_info
at the out label. To fix that return immediately as it's at the
beginning of the function.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Pick a few platforms to verify build of configure --experimental
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The 'box' target is not that different from the default one regarding
dependencies so build it as well by default for better coverage.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add a new test case to make sure:
- btrfstune can convert a zoned btrfs with extent tree to bgt
- btrfstune can convert a zoned btrfs with bgt back to extent tree
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
There is a report that, for zoned devices btrfstune is unable to convert
it to block group tree.
# btrfstune /dev/nullb0 --convert-to-block-group-tree
Error reading 1342193664, -1
Error reading 1342193664, -1
ERROR: cannot read chunk root
ERROR: open ctree failed
[CAUSE]
For read-write opened zoned devices, all the read/write has to be
aligned to its sector size.
However btrfs stores its metadata by extent_buffer::data[], which has
all the structures before it, thus never aligned to zoned device sector
size.
Normally we would require btrfs_pread() and btrfs_pwrite() to do the
extra alignment, but during open_ctree(), we are not aware if a device
is zoned or not.
Thus we rely on if the fd is opened with O_DIRECT flag, if the fd has
O_DIRECT, then we would temporarily set fs_info->zoned for chunk tree
read.
Unforunately not all open_ctree_fd() callers have the flags set
properly, and btrfstune is one of the missing call site.
This makes all the read not properly aligned and cause read failure.
[FIX]
Just manually check if the target device is a zoned one, and set
O_DIRECT accordingly.
Issue: #765
Pull-request: #767
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
There is a bug report that mkfs.btrfs can not specify block-group-tree
feature along with zoned devices:
# mkfs.btrfs /dev/nullb0 -O block-group-tree,zoned
btrfs-progs v6.7.1
See https://btrfs.readthedocs.io for more information.
Resetting device zones /dev/nullb0 (40 zones) ...
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)
ERROR: error during mkfs: Invalid argument
[CAUSE]
During mkfs, we need to write all the 7 or 8 tree blocks into the
metadata zone, and since it's zoned device, we need to fulfill all the
requirement for zoned writes, including:
- All writes must be in sequential bytenr
- Buffer must be aligned to sector size
The sequential bytenr requirement is already met by the mkfs design, but
the second requirement on memory alignment is never met for metadata, as
we put the contents of a leaf in extent_buffer::data[], which is after a
lot of small members.
Thus metadata IO buffer would never be aligned to sector size (normally
4K).
And we require btrfs_pwrite() and btrfs_pread() to handle the memory
alignment for us.
However in create_block_group_tree() we didn't use btrfs_pwrite(), but
plain pwrite() call directly, which would lead to -EINVAL error due to
memory alignment problem.
[FIX]
Just call btrfs_pwrite() instead of the plain pwrite() in
create_block_group_tree().
Issue: #765
Pull-request: #767
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There is a missing newline for a successful
--convert-from-block-group-tree run, meanwhile
--convert-to-block-group-tree has the correct newline.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Inspired by https://www.reddit.com/r/btrfs/comments/1bkdewb/btrfs_errors_in_dmesg/ .
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The error message for ro->rw is not clear enough, add more context for
the received_uuid and send and point to the manual page that explains
more.
Asked at:
https://www.reddit.com/r/btrfs/comments/1bkqor2/received_uuid_error_after_btrfs_send/
[ci skip]
Issue: #763
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
It was reported in https://lore.kernel.org/linux-btrfs/e7ce9995-93cb-4904-875c-684d4494765f@web.de/
that compression does not happen on direct io files. This is incorrectly
documented that it works but this is not true. Compression works on
buffered writes and relies on page cache, while direct io avoids that
and takes a different path in code.
[ci skip]
Pull-request: #764
Author: HAN Yuwei <hrx@bupt.moe>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
verity -> verify
[ci skip]
Pull-request: #759
Author: Thomas Bertels <3265870+tbertels@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use the templated error message for transaction failures, use the same
pattern assigning the ret and errno.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
CONFIG_BTRFS_DEBUG)
The send v3 protocol is enabled in kernel by a different config option
than in btrfs-progs to actually work. Now v3 can be tested when
configured and built with --enable-experimental.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use the templated error message for transaction start failures, use the
same pattern assigning the ret and errno.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Turn all BUG_ONs to error handling and push it to the caller. The error
conditions are almost certainly corruptions so we can't do anything
about that.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The error values of enter_shared_node() are mixing int and bool, unify
that to be 1 == true, 0 == false, <0 errors. Update callers to handle
errors.
Inline the add_shared_node() helper as it's trivial and makes handling
errors easier. As all errors can be now returned, do proper error
handling instead of all remaining BUG_ONs.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
to the callers
Handle the BUG_ONs inside splice_shared_node() and move them to the
callers. As there's a big loop and external tree cache updated there's
not error cleanup done.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
to callers
Handle the two BUG_ONs inside add_inode_backref() and move them to the
callers.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Free the newly allocated structures when 'mod' is requests and insertion
fails. All exit paths from the function now don't leave anything to
clean up.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use the template to be verbose about device zeroing failure, this can be
called repeatedly.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add template for read/write error messages and use it for write of
superblock when adding a device. sbwrite() is wrapper around write that
makes sure the zoned devices are accessed correctly.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use a local copy of the search header for proper aligned access instead
of the unaligned helpers, move the definitions to the closest scope.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use tree search ioctl wrappers for code that is considered internal, ie.
leaving out libbtrfs (legacy), libbtrfsutil (needs own API for that).
Conversion is mostly direct of what the API provides.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For unclear reasons using the v2 ioctl leads to an infinite loop in
'btrfs fi usage' in load_chunk_info() when there's only one valid item
returned and then it keeps looping. Can be reproduced by mkfs-tests/001.
After debugging, from second item in the buffer there's all zeros, while
it's returned nr_items=4. Switching the same code to use v1 makes it
work again. It's puzzling as it's the same code in kernel.
We want to make the switch eventually so only disable the detection so
other code can use the new API.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add wrappers around v1 and v2 of TREE_SEARCH ioctl so it can be
transparently used by code. The structures partially overlap but due to
the buffer size the v2 is offset and also needs a filler to expand the
flexible buffer.
Usage:
- define struct btrfs_tree_search_args, all zeros
- btrfs_tree_search_sk() reads offset of the search key within the
structures
- btrfs_tree_search_ioctl() detect support and call the highest
supported ioctl version, v2 has been supported since 3.14 but we want
to keep backward compatibility
- btrfs_tree_search_data() read data from the buffer previously filled
by ioctl, a sequence of (search header, data)
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Replace BUG_ONs with proper error handling.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The buffer size check is needed and has already caught problems when
adding the raid-stripe-tree, do a better error reporting.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Do proper error handling and use the template error message when setting
the label fails.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add new error message template and use it to report invalid range
overlaps and do proper error handling.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Do proper error handling like in the rest of the function.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Tree comparators never change parameters, make them all const and also
change the rb-tree prototypes.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Sync a few more file on the source level with kernel 6.8.
- type cleanups
- defines and enums
- comments
- parameter updates
- error handling
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There's an error in resource control command name, the argument is
IOReadBandwidthMax instead of IOBandwidthReadMax. See
https://www.freedesktop.org/software/systemd/man/latest/systemd.resource-control.html
[ci skip]
Pull-request: #744
Author: Termuellinator <saldorin@web.de>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Sync structures definitions and constants from kernel source.
- update btrfs_ioctl_fs_info_args
- update btrfs_ioctl_send_args
- verity, block-group-tree, raid-stripe-tree, simple quota
- ioctl definition updates
- copy comments
- diff minimization (whitespace changes, moved definitions)
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Copy what's in kernel header, new structures and constants.
- raid-stripe-tree
- verity
- simple quota
Deleted:
- btrfs_extent_ref_v0 (obsolete, kernel support removed)
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The initial version of libbtrfsutil did not follow a unified naming
scheme that's usually used for libraries like those provide by
util-linux. Add aliases that are "btrfs_util_" + object + action +
suffix.
The library version changes to 1.3 but there's no new functionality,
only the aliases added. New functions can be added in the future without
possible confusion when the same action could apply to different
objects.
Issue: #574
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
With the latest kernel patch to reject invalid qgroupids in
btrfs_qgroup_inherit structure, "btrfs subvolume create" or "btrfs
subvolume snapshot" can lead to the following output:
# mkfs.btrfs -O quota -f $dev
# mount $dev $mnt
# btrfs subvolume create -i 2/0 $mnt/subv1
Create subvolume '/mnt/btrfs/subv1'
ERROR: cannot create subvolume: No such file or directory
The "btrfs subvolume" command output the first line, seemingly to
indicate a successful subvolume creation, then followed by an error
message.
This can be a little confusing on whether if the subvolume is created or
not.
[FIX]
Fix the output by only outputting the regular line if the ioctl
succeeded.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Remove btrfs_qgroup_inherit_add_copy() and the command line interface.
This was designed to add a pair of source/destination qgroups into
btrfs_qgroup_inherit structure, so that rfer/excl numbers would be
copied from the source qgroup into the destination one.
This behavior has been intentionally hidden since 2016, as such copy will
cause qgroup inconsistent immediately and a rescan would reset whatever
numbers copied anyway.
Now we're going to reject the copy behavior from kernel, there is no
need to keep those hidden (and already disabled for "subvolume create")
case.
Remove btrfs_qgroup_inherit_add_copy() call, and cleanup the
undocumented options.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Feature from https://github.com/sphinx-doc/sphinx/pull/2064 enable
navigation to be able to navigate documentation using the arrow keys.
Pull-request: #754
Author: Martin <spleefer90@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use a more descriptive name, the interface is generic so it should use
the generic term for file/directory.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The arguments now match the wrapper, use it.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
All calls now pass true, drop the parameter.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There are some cases that disable verbosity (of errors) and then print
own message. Enable the verbose error messages printed by
btrfs_open_fd2() as they are specific.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
All calls now pass true, drop the parameter.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The code in scrub predates the global verbosity options and sets its own
quiet status. This is still used only for error messages that should be
printed even with -q. Drop that or replace with bconf.verbose status
check.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There are many places that pass false as verbosity argument and then
print an error message, or don't print any message in error cases.
Use btrfs_open_file_or_dir_fd() that will be verbose in case of an error
with the same semantics.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
It's commonly used elsewhere in the code to return the -errno values if
possible, do that for the open helpers too.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Remove the following unused functions:
- btrfs_open_dir()
- open_file_or_dir()
- btrfs_open_file_or_dir()
- btrfs_open()
- open_path_or_dev_mnt()
- open_file_or_dir3()
- close_file_or_dir()
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return
only the fd.
Replace the last reference to btrfs_open_file_or_dir3() with
btrfs_open_fd2().
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return
only the fd.
Replace open_file_or_dir() with btrfs_open_fd2() removing any reference
to the unused/useless dirstream variables. btrfs_open_fd2() is required
to avoid spurious error messages.
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return
only the fd.
Replace btrfs_open_file_or_dir() with btrfs_open_file_or_dir_fd()
removing any references to the unused/useless dirstream variables.
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return
only the fd.
Replace open_file_or_dir3() with btrfs_open_fd2() removing any reference
to the unused/useless dirstream variables. btrfs_open_fd2() is needed
because sometime the callers need to set the RDONLY/RDWRITE mode, and to
avoid spurious messages.
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return
only the fd.
Replace open_path_or_dev_mnt() with btrfs_open_mnt_fd() removing
any reference to the unused/useless dirstream variables.
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return
only the fd.
Replace the last btrfs_open_dir() call with btrfs_open_dir_fd()
removing any reference to the unused/useless dirstream variables.
Also update the add_seen_fsid() function removing any reference to dir
stream (again this is never used).
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
Replace btrfs_open_dir() with btrfs_open_dir_fd() removing
any reference to the unused/useless dirstream variables.
Calling btrfs_open_dir_fd() with only the path is equivalent to
btrfs_open_dir(_, _, 1).
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.
However this is never used. So avoid calling diropen() and return only
the fd. This is a preparatory patch.
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add a note summarizing the problems.
[ci skip]
Issue: #257
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Update wording and add an example.
[ci skip]
Issue: #729
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There's a report that 'btrfs balance start --enqueue' does not properly
wait when there are multiple instances started. The command does a busy
wait instead of timeouts.
Strace output:
0.000006 pselect6(5, NULL, NULL, [4], {tv_sec=60, tv_nsec=0}, NULL) = 1 (except [4], left {tv_sec=59, tv_nsec=999999716})
0.000008 pselect6(5, NULL, NULL, [4], {tv_sec=29, tv_nsec=999999000}, NULL) = 1 (except [4], left {tv_sec=29, tv_nsec=999998786})
After the first select there's almost the entire time left, the second
one starts right after it.
Polling/selecting sysfs files is possible under some conditions:
- the file descriptor must be reopened before each poll/select
- the whole buffer must be read too
With that in place it now works as expected. The remaining timeout logic
is slightly adjusted to wait at most 10 seconds so the pending jobs do
not wait too long if there's still a lot of time left from the first
select.
Issue: #746
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add more packages providing some utilities and add new DM targests.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add visuals of the tabulated stats.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add a new role to render manual page references and also link to the web
pages on man7.org.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Use the new docref and duplabel syntax to fix build warnings and fix the
link targets in the pages.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Sphinx/RST does not(?) have native support for cross references to
labels in specific documents (doc, ref but not both at the same time),
also allowing duplicate labels. We have web and manual pages and to
share the same text there are common files included from each, defining
labels. This leads to warnings and clicking the links ends up in the
included document (like ch-volume-management-intro.rst) instead of the
parent.
As a last resort, implement own role that does the doc and ref in one
go. A special directive is used to define a label that is expected
to be duplicate (but otherwise it's an ordinary label) and this gets rid
of the warning too.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Subpage promoted to 'OK', no serious problems since 6.0. Otherwise
references, adjustments, wording updates.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Minor wording and clarifications for 'mkfs.btrfs -O list-all'.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
We've deprecated the -R option and runtime feature distinction, there's
only one option -O recommended so let it also print on the same line.
Incompat/compat/runtime status of a feature shall be consulted with the
documentation.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Be verbose about the potential compatibility problems with the
sectorsize and page size. Also print the page size on the overview.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There are new backends and builtin accelerated implementations.
Recalculate the table results, a different host than before, with SHANI
extension.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add 6.6 and 6.7 changes, mention the subpage vs zoned limitation found
recently.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
"desinations" => "destinations"
[ci skip]
Pull-request: #743
Signed-off-by: Max-Julian Pogner <max-julian@pogner.at>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Sync a few more file on the source level with kernel 6.8-rc3, no
functional changes.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Pull-request: #740
Author: LiuYan <lovetide@qq.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
Although commit b2a1be83b85f ("btrfs-progs: mkfs: keep file descriptors
open during whole time") is making sure we're only closing the writeable
fds after the fs is properly created, there is still a missing fd not
following the requirement.
And this explains the issue why sometimes after mkfs.btrfs, lsblk still
doesn't give a valid uuid.
Shown by the strace output (the command is "mkfs.btrfs -f
/dev/test/scratch1"):
openat(AT_FDCWD, "/dev/test/scratch1", O_RDWR) = 5 <<< Writeable open
fadvise64(5, 0, 0, POSIX_FADV_DONTNEED) = 0
sysinfo({uptime=2529, loads=[8704, 6272, 2496], totalram=4104548352, freeram=3376611328, sharedram=9211904, bufferram=43016192, totalswap=3221221376, freeswap=3221221376, procs=190, totalhigh=0, freehigh=0, mem_unit=1}) = 0
lseek(5, 0, SEEK_END) = 10737418240
lseek(5, 0, SEEK_SET) = 0
......
close(5) = 0 <<< Closed now
pwrite64(6, "O\250\22\261\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1163264) = 16384
pwrite64(6, "\201\316\272\342\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1179648) = 16384
pwrite64(6, "K}S\t\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1196032) = 16384
pwrite64(6, "\207j$\265\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1212416) = 16384
pwrite64(6, "q\267;\336\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 5242880) = 16384
fsync(6) <<< But we're still writing into the disk.
[CAUSE]
After more digging, it turns out we have a very obvious escape in
open_ctree_fs_info():
open_ctree_fs_info()
|- fp = open(oca->filename, flags);
|- info = __open_ctree_fd();
|- close(fp);
As later we only do IO using the device fd, this close() seems fine.
But the truth is, for mkfs usage, this fs_info is a temporary one, with
a special magic number for the disk. And since mkfs is doing writeable
operations, this close() would immediately trigger udev scan.
And since at this stage, the fs is not yet fully created, udev can race
with mkfs, and may get the invalid temporary superblock.
[FIX]
Introduce a new btrfs_fs_info member, initial_fd, for
open_ctree_fs_info() to record the fd.
And on close_ctree(), if we find fs_info::initial_fd is a valid fd, then
close it.
By this, we make sure all writeable fds are only closed after we have
written valid super blocks into the disk.
Issue: #734
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
In "btrfs tune" subcommand, we're using open_ctree_fd(), which
requires passing a valid fd, and the caller is responsible to properly
handling the lifespan of the fd.
But we just call close_ctree() and btrfs_close_all_devices(), without
closing the fd.
[FIX]
Just manually close the fd.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
In cmd_rescue_clear_ino_cache(), we opened the fs, but without
closing it using close_ctree().
[CAUSE]
This was introduced in 42404a4e448c2e ("btrfs-progs: move inode cache
removal to rescue group"), the original code inside btrfs check
had a "goto out_close;" to properly close the fs.
[FIX]
Manually call close_ctree() on the fs_info->tree_root.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The symbol BTRFS_UPDATE_KERNEL seems to be unused since 2f55fd70191ed3
("btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls"), remove
it.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Wherever it makes sense, save the logs as artifacts when something
fails (most likely the main step with real tests).
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There's a report that passing raw device mapper path and -d don't work
together:
yyy@xxx ~ $ sudo btrfs filesystem show /dev/dm-0
Label: none uuid: a7fbb8d6-ec5d-4e88-bd8b-c686553e0dc7
Total devices 1 FS bytes used 144.00KiB
devid 1 size 256.00MiB used 88.00MiB path /dev/mapper/da0972636816-LogVol00
With --all-devices
yyy@xxx ~ $ sudo btrfs filesystem show --all-devices /dev/dm-0
ERROR: not a valid btrfs filesystem: /dev/dm-0
Where dm-0 corresponds to the LogVol00 device from above.
Passing the option -d skips some steps but still uses the real path of
the device that is required for scanning and identification, while
blkid uses the canonicalized path.
The combination of raw device name and -d was not handled as the raw
path is not in cache and thus not recognized. Canonicalization fixes
that although this changes the device name in the output.
Issue: #732
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There are now two tests with 060, we'd like to be that a unique number,
rename the one that was added more recently.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The raid-stripe-tree can be enabled for convert, though it's still
considered incomplete and slightly experimental. Due to that the tests
need to be adjusted to check for support and skip mount eventually.
Possible remaining options to add: quota, squota
Issue: #694
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Now docker hub images can be pulled for build tests (sources are
downloaded) and this is faster than rebuilding them each time so this
can be enabled for all ci/* and devel branches.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The runner scripts ci/ci-build... pass build options that work for each
distro, however this was not matching some Dockerfiles. Pulling such
image would then fail due to missing dependencies (namely libudev).
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There was a bug when a branch contained a slash then the file with
downloaded sources was not found. Update all, all images have to be
rebuilt and pushed to docker hub so the changes are applied inside
github actions.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add new script to upload updated images to my docker hub, add comments
in which order.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add a possibility to test branches independent of devel, the pattern is
prefix "ci/" or "CI/".
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
We have had working subpage support in Btrfs for many cycles now.
Generally, we do not want people creating filesystems by default
with non-4k sectorsizes since it creates portability problems.
As the subpage has stabilized it seems to be safe to do the switch.
This may still affect users that relying on the previous behaviour.
Issue: #604
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eric Curtin <ecurtin@redhat.com>
Signed-off-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
In case of a raid5/6 filesystem 'btrfs fi us' returns wrong values
without the root capabilities:
$ sudo btrfs fi us /tmp/raid5fs # as root
Overall:
Device size: 3.00GiB
Device allocated: 1.51GiB <--- OK
Device unallocated: 1.49GiB <--- OK
Device missing: 0.00B
Device slack: 0.00B
Used: 769.03MiB <--- OK
Free (estimated): 1.32GiB (min: 1.32GiB) <-OK
Free (statfs, df): 1.32GiB
Data ratio: 1.50 <--- OK
Metadata ratio: 1.50 <--- OK
Global reserve: 5.50MiB (used: 0.00B)
Multiple profiles: no
[...]
$ btrfs fi us /tmp/raid5fs # as user
WARNING: cannot read detailed chunk info, per-device usage will not be shown, run as root
Overall:
Device size: 3.00GiB
Device allocated: 0.00B <--- WRONG
Device unallocated: 3.00GiB <--- WRONG
Device missing: 0.00B
Device slack: 0.00B
Used: 0.00B <--- WRONG
Free (estimated): 0.00B (min: 8.00EiB) <- WRONG
Free (statfs, df): 1.32GiB
Data ratio: 0.00 <--- WRONG
Metadata ratio: 0.00 <--- WRONG
Global reserve: 5.50MiB (used: 0.00B)
Multiple profiles: no
[...]
The reason is that the BTRFS_IOC_SPACE_INFO ioctl doesn't return enough
information. To bypass it a scan of the chunks is required when a
raid5/6 profile is present.
To avoid providing wrong information, in case of a raid5/6 filesystem
without the root capabilities the "btrfs fi us" is not executed at all
and a warning with a suggestion to run it as root is printed.
$ ./btrfs fi us /tmp/t/
WARNING: cannot read detailed chunk info, per-device usage will not be shown, run as root
WARNING: due to the presence of a raid5/raid6 profile, we cannots compute some values;
WARNING: run as root instead.
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If "btrfs dev us" is invoked by a not root user, it is impossible to
collect the chunk info data (not enough privileges). This causes
"btrfs dev us" to print as "Unallocated" value the size of the disk.
This patch handles the case where print_device_chunks() is invoked
without the chunk info data, printing "Unallocated N/A":
Before the patch:
$ btrfs dev us t/
WARNING: cannot read detailed chunk info, per-device usage will not be shown, run as root
/dev/loop0, ID: 1
Device size: 5.00GiB
Device slack: 0.00B
Unallocated: 5.00GiB <-- Wrong
$ sudo btrfs dev us t/
[sudo] password for ghigo:
/dev/loop0, ID: 1
Device size: 5.00GiB
Device slack: 0.00B
Data,single: 8.00MiB
Metadata,DUP: 512.00MiB
System,DUP: 16.00MiB
Unallocated: 4.48GiB <-- Correct
After the patch:
$ ./btrfs dev us /tmp/t/
WARNING: cannot read detailed chunk info, per-device usage will not be shown, run as root
/dev/loop0, ID: 1
Device size: 5.00GiB
Device slack: 0.00B
Unallocated: N/A
$ sudo ./btrfs dev us /tmp/t/
[sudo] password for ghigo:
/dev/loop0, ID: 1
Device size: 5.00GiB
Device slack: 0.00B
Data,single: 8.00MiB
Metadata,DUP: 512.00MiB
System,DUP: 16.00MiB
Unallocated: 4.48GiB
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
This patch introduces a new parser helper, parse_u64_with_suffix(),
which has a better error handling, following all the parse_*()
helpers to return non-zero value for errors.
This new helper is going to replace parse_size_from_string(), which
would directly call exit(1) to stop the whole program.
Furthermore most callers of parse_size_from_string() are expecting
exit(1) for error, so that they can skip the error handling.
For those call sites, introduce a wrapper, arg_strtou64_with_suffix(),
to do that. The only disadvantage is a little less detailed error
report for why the parse failed, but for most cases the generic error
string should be enough.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Both functions are just doing the same thing, the only difference is
only in error handling, as parse_u64() requires callers to handle it,
meanwhile arg_strtou64() would call exit(1).
This patch would convert arg_strtou64() to utilize parse_u64(), and use
the return value to output different error messages.
This also means the return value of parse_u64() would be more than just
0 or 1, but -EINVAL for invalid string (including no numeric string at
all, has any tailing characters, or minus value), and -ERANGE for
overflow.
The existing callers are only checking if the return value is 0, thus
not really affected.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The CI test uses openssl 3.2 but it still hasn't made it to Tumbleweed,
it's still waiting in the queue. It'll be enabled again.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Enhance the conversion macro to recognize the filesystem type in case
the ntfs-specific converter utility has to be used. Its current feature
set does not match btrfs-convert.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The convert tests weren't enabled in the CI due to some problems that
seem to be fixed now. Add it to the default and coverage workflows, the
run time is about 2 minutes which is acceptable for coverage and for
devel it's running in parallel.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The test does not pass in environments where /bin/sh is not bash (like
github CI). Unify the initial setup like the other tests do: source,
compatibility, prerequisities, the rest.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
There is a report about failed btrfs-convert, which shows the following
error:
Create btrfs metadata
corrupt leaf: root=5 block=5001931145216 slot=1 ino=89911763, invalid previous key objectid, have 89911762 expect 89911763
leaf 5001931145216 items 336 free space 7 generation 90 owner FS_TREE
leaf 5001931145216 flags 0x1(WRITTEN) backref revision 1
fs uuid 8b69f018-37c3-4b30-b859-42ccfcbe2449
chunk uuid 448ce78c-ea41-49f6-99dc-46ad80b93da9
item 0 key (89911762 INODE_REF 3858733) itemoff 16222 itemsize 61
index 171 namelen 51 name: [FILENAME1]
item 1 key (89911763 INODE_REF 3858733) itemoff 16161 itemsize 61
index 103 namelen 51 name: [FILENAME2]
[CAUSE]
When iterating a directory, btrfs-convert would insert the DIR_ITEMs,
along with the INODE_REF of that inode.
This leads to above stray INODE_REFs, and trigger the tree-checker.
This can only happen for large fs, as for most cases we have all these
modified tree blocks cached, thus tree-checker won't be triggered.
But when the tree block cache is not hit, and we have to read from disk,
then such behavior can lead to above tree-checker error.
[FIX]
Insert a dummy INODE_ITEM for the INODE_REF first, the inode items would
be updated when iterating the child inode of the directory.
Issue: #731
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The strict check is enabled for both check_image() and
convert_test_do_convert() to detect chunk alignment related problems.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Recently we had a scrub use-after-free caused by unaligned chunk
length, although the fix was submitted, we may want to do extra checks
for a chunk's alignment.
This patch adds such check for the starting bytenr and length of a
chunk, to make sure they are properly aligned to 64K stripe boundary.
By default, the check only leads to a warning but is not treated as an
error, as we expect kernel to handle such unalignment without any
problem.
But if the new debug environmental variable,
BTRFS_PROGS_DEBUG_STRICT_CHUNK_ALIGNMENT, is specified, then we will
treat it as an error. So that we can detect unexpected chunks from
btrfs-progs, and fix them before reaching the end users.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
aligned
Although kernel scrub code has been updated to handle the unaligned
chunk length, there is also no harm if we can allocate data chunk with
both start and length aligned.
This patch handles this by rounding up the end bytenr when allocating
data chunks for the conversion.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
On multi-device filesystems, a scrub limit may be applied to any of the
devices. Ensure that any limit found is not disregarded.
Since it's more intuitive, keep the lowest non-zero limit found, even
though at the present we don't actually use the exact value.
Pull-request: #733
Issue: #727
Fixes: 7e4a235df1ac ("btrfs-progs: scrub status: print device speed limit in status if set")
Signed-off-by: Jonas Malaco <jonas@protocubo.io>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
On multi-device filesystems, a scrub limit may be applied to any of the
devices. Ensure that any limit found is not disregarded.
Since it's more intuitive, keep the lowest non-zero limit found, even
though at the present we don't actually use the exact value.
Pull-request: #733
Issue: #727
Fixes: 7e4a235df1ac ("btrfs-progs: scrub status: print device speed limit in status if set")
Signed-off-by: Jonas Malaco <jonas@protocubo.io>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
On multi-device filesystems, scrub status should report "some limits
set" if at least one device has a scrub limit set.
However, with btrfs-progs 6.6.3, this was being reported regardless of
whether any limit actually being set:
# sudo btrfs scrub limit /more/butter
UUID: 989129d9-c96f-4d52-9d68-cbb6d9b2c499
Id Limit Path
-- ----- ---------
1 - /dev/sdc1
2 - /dev/sdd1
# sudo btrfs scrub status /more/butter/
UUID: 989129d9-c96f-4d52-9d68-cbb6d9b2c499
Scrub started: Mon Jan 15 02:00:30 2024
Status: running
Duration: 6:23:19
Time left: 0:49:08
ETA: Mon Jan 15 09:12:57 2024
Total to scrub: 9.83TiB
Bytes scrubbed: 8.72TiB (88.64%)
Rate: 397.47MiB/s (some device limits set)
Error summary: no errors found
Fix it by only setting `limit` to the special marker value 1 if at least
one actual limit is found.
Pull-request: #733
Issue: #727
Fixes: 7e4a235df1ac ("btrfs-progs: scrub status: print device speed limit in status if set")
Signed-off-by: Jonas Malaco <jonas@protocubo.io>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
converting a large fs
[BUG]
There is a report about failed btrfs-convert, which shows the following
error:
corrupt leaf: root=5 block=5001928998912 slot=1 ino=89911763, invalid previous key objectid, have 89911762 expect 89911763
ERROR: failed to copy ext2 inode 89911320: -5
ERROR: error during copy_inodes -5
WARNING: error during conversion, the original filesystem is not modified
[CAUSE]
Above error is triggered when checking the following items inside a
subvolume:
- inode ref
- dir item/index
- file extent
- xattr
This is to make sure these items have correct previous key.
However btrfs-convert is not following this requirement, it always
inserts those items first, then creates a btrfs_inode for it.
Thus it can lead to the error.
This can only happen for large fs, as for most cases we have all these
modified tree blocks cached, thus tree-checker won't be triggered.
But when the tree block cache is not hit, and we have to read from disk,
then such behavior can lead to above tree-checker error.
[FIX]
Make sure we insert the inode item first, then the file extents/dir
items/xattrs. And after the file extents/dir items/xattrs inserted, we
update the existing inode (to update its size and bytes).
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Unlike kernel where tree-checker would provide enough info so later we
can use "btrfs inspect dump-tree" to catch the offending tree block, in
progs we may not even have a btrfs to start "btrfs inspect dump-tree".
E.g during btrfs-convert.
To make later debuging easier, let's call btrfs_print_tree() for every
error we hit inside tree-checker.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Older versions of sphinx require the contents.rst file otherwise the
build fails, while new versions don't need it and use index.rst.
Sphinx error:
master file btrfs-progs/Documentation/contents.rst not found
make[1]: *** [Makefile:37: man] Error 2
make: *** [Makefile:502: build-Documentation] Error 2
This build error is seen on version 1.7.6-3.
To make it work on old and new versions create a placeholder empty file
but make it a phony build target so new sphinx does not see it and
report as not in any TOC.
[ci skip]
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There are several warnings regarding the absence of an argument for the
code-block directive on some build servers using python3-sphinx 0.2.2-17.
For example:
Making all in Documentation
[SPHINX] man
ch-subvolume-intro.rst:141: WARNING: Error in "code-block" directive:
1 argument(s) required, 0 supplied.
.. code-block::
27 21 0:19 /subv1 /mnt rw,relatime - btrfs /dev/sda rw,space_cache
Etc...
Add the none argument.
[ci skip]
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
create"
The test case would check if "btrfs subvolume create":
- Report error on an existing path
- Still report error if mulitple paths are given and one of them already
exists
- For above case, still created a subvolume for the good parameter
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
When try to create a subvolume where the target path already exists, the
"btrfs" command doesn't return error code correctly.
# mkfs.btrfs -f $dev
# mount $dev $mnt
# touch $mnt/subv1
# btrfs subvolume create $mnt/subv1
ERROR: target path already exists: $mnt/subv1
# echo $?
0
[CAUSE]
The check on whether target exists is done by path_is_dir(), if it
returns 0 or 1, it means there is something in that path already.
But unfortunately commit 5aa959fb3440 ("btrfs-progs: subvolume create:
accept multiple arguments") only changed the out label, which would
directly return @ret, not updating the return value correctly.
[FIX]
Make sure all error out branch has their @ret manually updated.
Fixes: 5aa959fb3440 ("btrfs-progs: subvolume create: accept multiple arguments")
Issue: #730
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Bump version, all the new features in 6.7 have been already added.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The autoconf manual recommends AS_IF [1] nowadays rather than bare shell
if tests as they can interfere with quoting and macro expansion.
[1] https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.72/autoconf.html#index-AS_005fIF-1
Pull-request: #721
Signed-off-by: Sam James <sam@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The "=" operator should be used as configure may be run by a POSIX shell
at /bin/sh (like dash). Bash recognises "=" too so this retains
compatibility with it.
Pull-request: #721
Signed-off-by: Sam James <sam@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add new options to set the per-device limit (requires root privileges as
it writes to the sysfs files).
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
To be consistent with the rest of the code the sysfs helper should
return the -errno instead of passing -1 from various syscalls. Update
callers that relied on -1 as the invalid file descriptor.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add convenience wrappers for writing a buffer or u64 to toplevel or FSID
file in sysfs.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Scrubs which complete in under one second may carry a duration rounded
down to zero. This subsequently results in a bytes_per_sec value of
zero, which corresponds to the Rate metric output, causing intermittent
tests/btrfs/282 failures.
This change ensures that Rate reflects any sub-second bytes processed.
Time left and ETA metrics are also affected by this change, in that they
increase to account for (sub-second) bytes_per_sec.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Issue: #402
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Print one message per scrubbed device and also print the limit if set:
$ btrfs scrub start /mnt
scrub started on /mnt, fsid 9ee93131-f680-4d6c-8ca4-a194506e3081 (pid=27257)
Starting scrub on devid 1 (limit 100.00MiB/s)
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add new command to read the scrub limits set via the sysfs file (no root
access needed).
Example output:
$ btrfs scrub limit /mnt
UUID: 57a05502-9e81-4b21-ad9d-0fc31863ed11
Id Limit Path
-- ----- --------------
1 - /dev/nvme0n1p1
2 - /dev/nvme0n1p2
3 - /dev/nvme0n1p3
4 - /dev/nvme2n1p4
5 - /dev/nvme0n1p5
6 - /dev/nvme0n1p6
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The statfs(2) syscall is deprecated by LSB in favor of statvfs(2),
however we can't replace all uses because we still need the
statfs::f_type to determine the filesystem by magic numer.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The subvolume cleaning is done by polling but it's possible that the
filesystem turns to read-only (as reported), either due to an error
intentionally. In that case the waiting would be indefinite without an
obvious reason.
To fix that check if the filesystem is still writable in each iteration.
Issue: #535
Link: https://github.com/btrfs/fstests/issues/40
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
When there's a speed limit set for a device via
/sysfs/fs/btrfs/FSID/devinfo/scrub_speed_max, show it in the scrub status
output like below:
$ btrfs scrub status -d /mnt
...
Rate: 47.98MiB/s (limit 60MiB/s)
...
If the limit is 0 this means unlimited and is not printed.
For a single device filesystem the limit is printed even without '-d' as
it's clear which device limit applies. For multi-device filesysetms,
without any limits nothing is printed, if there at least one device
limit set then the following is printed:
Rate: 36.37MiB/s (some device limits set)
More details with the -d option.
Issue: #531
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If zstd is not compiled in then a stream fails with a generic error
message:
ERROR: unknown compression: 2
Where BTRFS_ENCODED_IO_COMPRESSION_ZSTD is 2 and there's a case for that
but behind the '#if COMPRESSION_ZSTD'.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
With the recent updates to documentation build the theme must be now
installed as a package. Disable building documentation in all workflows
that do functional tests.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The DocConventions are now fairly complete and can be moved to the
proper section.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Lowmem mode has improved quite a lot since its introduction, for
read-only check it's definitely fine.
For repair mode, both lowmem and original mode are considered dangerous
especially for complex corruptions with unknown cause.
For now lowmem mode is only bad at fixing fundamentally corrupted cases,
like bad shift offsets or transid, which in real world it's not an easy
repair for the original mode either.
This patch would move the --mode option out of the dangerous section and
update the notes for the lowmem mode on its limitation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Currently the path of deleted subvolume is printed, we should also print
the numeric id as it's another identifier commonly found and can be used
for a cross reference. In connection with the qgroup deletion it's
making the output clear:
...
Delete subvolume 258 (no-commit): '/mnt/subv1'
Delete qgroup 0/258
...
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
There's another config required for building the RTD documentation,
https://docs.readthedocs.io/en/stable/config-file/ .
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The 0/subvolid qgroups are not automatically deleted when the subvolume
is deleted, for historical reasons. There's a command to clean up all
such stale qgroups (btrfs qgroup clean-stale) but this should be also
possible with the subvolume deletion.
With the options we can switch the default to delete the qgroup by
default eventually, if somebody depends on the not deleting behaviour
the negation option can be used.
Issue: #366
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Compiling with clang on aarch64 leads to an error when detecting the
SIMD instruction support. Gcc ignores the arch/feature mismatch.
Conditionally detect the -m flags only on x86_64.
Issue: #712
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Add clang as compiler for the basic build checks in the CI.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Centos 8 does not provide the package at all, on musl it's in the
package 'pahole'.
Issue: #710
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The enqueue option should let the user know that the expected operation
hasn't started yet and that it's waiting for another one. Although the
exclusive operations can take long, the two reason should be
distinguished.
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Reported on IRC, that it's unexpected that passing several devices on
command line for 'btrfs device delete' still uses some of the devices
during deletion. The expectation was that they'd be removed at once (and
thus not used for the intermediate chunk relocation).
As it works now, the ioctl removes only one device. As a workaround, add
a timeout (like we have for the full balance and others) when there are
more devices passed on the command line. This can be skipped by the
--force parameter.
Issue: #708
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
strtoull may return the boundary values, if the callers could expect
that and verify it then the errno must be reset before the call.
Signed-off-by: David Sterba <dsterba@suse.com>
|
