Xsphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget-/translations/zh_CN/filesystems/iomap/portingmodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget-/translations/zh_TW/filesystems/iomap/portingmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget-/translations/it_IT/filesystems/iomap/portingmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget-/translations/ja_JP/filesystems/iomap/portingmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget-/translations/ko_KR/filesystems/iomap/portingmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget-/translations/sp_SP/filesystems/iomap/portingmodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhhG/var/lib/git/docbuild/linux/Documentation/filesystems/iomap/porting.rsthKubhtarget)}(h.. _iomap_porting:h]h}(h] iomap-portingah ]h"] iomap_portingah$]h&]uh1hhKhhhhhhubh)}(hDumb style notes to maintain the author's sanity: Please try to start sentences on separate lines so that sentence changes don't bleed colors in diff. Heading decorations are documented in sphinx.rst.h]hDumb style notes to maintain the author's sanity: Please try to start sentences on separate lines so that sentence changes don't bleed colors in diff. Heading decorations are documented in sphinx.rst.}hhsbah}(h]h ]h"]h$]h&]hhuh1hhhhhhhhK ubhsection)}(hhh](htitle)}(hPorting Your Filesystemh]hPorting Your Filesystem}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhK ubhtopic)}(hTable of Contents h](h)}(hTable of Contentsh]hTable of Contents}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhKubh bullet_list)}(hhh](h list_item)}(hhh]h paragraph)}(hhh]h reference)}(hhh]h Why Convert?}(hj hhhNhNubah}(h]id1ah ]h"]h$]h&]refid why-convertuh1jhjubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1hhhubh)}(hhh]j)}(hhh]j )}(hhh]hHow Do I Convert a Filesystem?}(hj,hhhNhNubah}(h]id2ah ]h"]h$]h&]refidhow-do-i-convert-a-filesystemuh1jhj)ubah}(h]h ]h"]h$]h&]uh1jhj&ubah}(h]h ]h"]h$]h&]uh1hhhubeh}(h]h ]h"]h$]h&]uh1hhhhhhNhNubeh}(h]table-of-contentsah ](contentslocaleh"]table of contentsah$]h&]uh1hhhhKhhhhubh)}(hhh](h)}(h Why Convert?h]h Why Convert?}(hj[hhhNhNubah}(h]h ]h"]h$]h&]refidjuh1hhjXhhhhhKubj)}(h;There are several reasons to convert a filesystem to iomap:h]h;There are several reasons to convert a filesystem to iomap:}(hjjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjXhhubh block_quote)}(hX1. The classic Linux I/O path is not terribly efficient. Pagecache operations lock a single base page at a time and then call into the filesystem to return a mapping for only that page. Direct I/O operations build I/O requests a single file block at a time. This worked well enough for direct/indirect-mapped filesystems such as ext2, but is very inefficient for extent-based filesystems such as XFS. 2. Large folios are only supported via iomap; there are no plans to convert the old buffer_head path to use them. 3. Direct access to storage on memory-like devices (fsdax) is only supported via iomap. 4. Lower maintenance overhead for individual filesystem maintainers. iomap handles common pagecache related operations itself, such as allocating, instantiating, locking, and unlocking of folios. No ->write_begin(), ->write_end() or direct_IO address_space_operations are required to be implemented by filesystem using iomap. h]henumerated_list)}(hhh](h)}(hXThe classic Linux I/O path is not terribly efficient. Pagecache operations lock a single base page at a time and then call into the filesystem to return a mapping for only that page. Direct I/O operations build I/O requests a single file block at a time. This worked well enough for direct/indirect-mapped filesystems such as ext2, but is very inefficient for extent-based filesystems such as XFS. h]j)}(hXThe classic Linux I/O path is not terribly efficient. Pagecache operations lock a single base page at a time and then call into the filesystem to return a mapping for only that page. Direct I/O operations build I/O requests a single file block at a time. This worked well enough for direct/indirect-mapped filesystems such as ext2, but is very inefficient for extent-based filesystems such as XFS.h]hXThe classic Linux I/O path is not terribly efficient. Pagecache operations lock a single base page at a time and then call into the filesystem to return a mapping for only that page. Direct I/O operations build I/O requests a single file block at a time. This worked well enough for direct/indirect-mapped filesystems such as ext2, but is very inefficient for extent-based filesystems such as XFS.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubah}(h]h ]h"]h$]h&]uh1hhjubh)}(hoLarge folios are only supported via iomap; there are no plans to convert the old buffer_head path to use them. h]j)}(hnLarge folios are only supported via iomap; there are no plans to convert the old buffer_head path to use them.h]hnLarge folios are only supported via iomap; there are no plans to convert the old buffer_head path to use them.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubah}(h]h ]h"]h$]h&]uh1hhjubh)}(hUDirect access to storage on memory-like devices (fsdax) is only supported via iomap. h]j)}(hTDirect access to storage on memory-like devices (fsdax) is only supported via iomap.h]hTDirect access to storage on memory-like devices (fsdax) is only supported via iomap.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhK"hjubah}(h]h ]h"]h$]h&]uh1hhjubh)}(hXCLower maintenance overhead for individual filesystem maintainers. iomap handles common pagecache related operations itself, such as allocating, instantiating, locking, and unlocking of folios. No ->write_begin(), ->write_end() or direct_IO address_space_operations are required to be implemented by filesystem using iomap. h]j)}(hXBLower maintenance overhead for individual filesystem maintainers. iomap handles common pagecache related operations itself, such as allocating, instantiating, locking, and unlocking of folios. No ->write_begin(), ->write_end() or direct_IO address_space_operations are required to be implemented by filesystem using iomap.h]hXBLower maintenance overhead for individual filesystem maintainers. iomap handles common pagecache related operations itself, such as allocating, instantiating, locking, and unlocking of folios. No ->write_begin(), ->write_end() or direct_IO address_space_operations are required to be implemented by filesystem using iomap.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhK%hjubah}(h]h ]h"]h$]h&]uh1hhjubeh}(h]h ]h"]h$]h&]enumtypearabicprefixhsuffix.uh1j~hjzubah}(h]h ]h"]h$]h&]uh1jxhhhKhjXhhubeh}(h]jah ]h"] why convert?ah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hHow Do I Convert a Filesystem?h]hHow Do I Convert a Filesystem?}(hjhhhNhNubah}(h]h ]h"]h$]h&]jij5uh1hhjhhhhhK-ubj)}(hX8First, add ``#include `` from your source code and add ``select FS_IOMAP`` to your filesystem's Kconfig option. Build the kernel, run fstests with the ``-g all`` option across a wide variety of your filesystem's supported configurations to build a baseline of which tests pass and which ones fail.h](h First, add }(hj hhhNhNubhliteral)}(h``#include ``h]h#include }(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubh from your source code and add }(hj hhhNhNubj)}(h``select FS_IOMAP``h]hselect FS_IOMAP}(hj(hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubhO to your filesystem’s Kconfig option. Build the kernel, run fstests with the }(hj hhhNhNubj)}(h ``-g all``h]h-g all}(hj:hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubh option across a wide variety of your filesystem’s supported configurations to build a baseline of which tests pass and which ones fail.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhK/hjhhubj)}(hXEThe recommended approach is first to implement ``->iomap_begin`` (and ``->iomap_end`` if necessary) to allow iomap to obtain a read-only mapping of a file range. In most cases, this is a relatively trivial conversion of the existing ``get_block()`` function for read-only mappings. ``FS_IOC_FIEMAP`` is a good first target because it is trivial to implement support for it and then to determine that the extent map iteration is correct from userspace. If FIEMAP is returning the correct information, it's a good sign that other read-only mapping operations will do the right thing.h](h/The recommended approach is first to implement }(hjRhhhNhNubj)}(h``->iomap_begin``h]h ->iomap_begin}(hjZhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjRubh (and }(hjRhhhNhNubj)}(h``->iomap_end``h]h ->iomap_end}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjRubh if necessary) to allow iomap to obtain a read-only mapping of a file range. In most cases, this is a relatively trivial conversion of the existing }(hjRhhhNhNubj)}(h``get_block()``h]h get_block()}(hj~hhhNhNubah}(h]h ]h"]h$]h&]uh1jhjRubh" function for read-only mappings. }(hjRhhhNhNubj)}(h``FS_IOC_FIEMAP``h]h FS_IOC_FIEMAP}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjRubhX is a good first target because it is trivial to implement support for it and then to determine that the extent map iteration is correct from userspace. If FIEMAP is returning the correct information, it’s a good sign that other read-only mapping operations will do the right thing.}(hjRhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhK5hjhhubj)}(hXNext, modify the filesystem's ``get_block(create = false)`` implementation to use the new ``->iomap_begin`` implementation to map file space for selected read operations. Hide behind a debugging knob the ability to switch on the iomap mapping functions for selected call paths. It is necessary to write some code to fill out the bufferhead-based mapping information from the ``iomap`` structure, but the new functions can be tested without needing to implement any iomap APIs.h](h Next, modify the filesystem’s }(hjhhhNhNubj)}(h``get_block(create = false)``h]hget_block(create = false)}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh implementation to use the new }(hjhhhNhNubj)}(h``->iomap_begin``h]h ->iomap_begin}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubhX  implementation to map file space for selected read operations. Hide behind a debugging knob the ability to switch on the iomap mapping functions for selected call paths. It is necessary to write some code to fill out the bufferhead-based mapping information from the }(hjhhhNhNubj)}(h ``iomap``h]hiomap}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh\ structure, but the new functions can be tested without needing to implement any iomap APIs.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhK@hjhhubj)}(hXwOnce the read-only functions are working like this, convert each high level file operation one by one to use iomap native APIs instead of going through ``get_block()``. Done one at a time, regressions should be self evident. You *do* have a regression test baseline for fstests, right? It is suggested to convert swap file activation, ``SEEK_DATA``, and ``SEEK_HOLE`` before tackling the I/O paths. A likely complexity at this point will be converting the buffered read I/O path because of bufferheads. The buffered read I/O paths doesn't need to be converted yet, though the direct I/O read path should be converted in this phase.h](hOnce the read-only functions are working like this, convert each high level file operation one by one to use iomap native APIs instead of going through }(hjhhhNhNubj)}(h``get_block()``h]h get_block()}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh>. Done one at a time, regressions should be self evident. You }(hjhhhNhNubhemphasis)}(h*do*h]hdo}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubhf have a regression test baseline for fstests, right? It is suggested to convert swap file activation, }(hjhhhNhNubj)}(h ``SEEK_DATA``h]h SEEK_DATA}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh, and }(hjhhhNhNubj)}(h ``SEEK_HOLE``h]h SEEK_HOLE}(hj,hhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubhX  before tackling the I/O paths. A likely complexity at this point will be converting the buffered read I/O path because of bufferheads. The buffered read I/O paths doesn’t need to be converted yet, though the direct I/O read path should be converted in this phase.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhKIhjhhubj)}(hX&At this point, you should look over your ``->iomap_begin`` function. If it switches between large blocks of code based on dispatching of the ``flags`` argument, you should consider breaking it up into per-operation iomap ops with smaller, more cohesive functions. XFS is a good example of this.h](h)At this point, you should look over your }(hjDhhhNhNubj)}(h``->iomap_begin``h]h ->iomap_begin}(hjLhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjDubhS function. If it switches between large blocks of code based on dispatching of the }(hjDhhhNhNubj)}(h ``flags``h]hflags}(hj^hhhNhNubah}(h]h ]h"]h$]h&]uh1jhjDubh argument, you should consider breaking it up into per-operation iomap ops with smaller, more cohesive functions. XFS is a good example of this.}(hjDhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhKUhjhhubj)}(hXThe next thing to do is implement ``get_blocks(create == true)`` functionality in the ``->iomap_begin``/``->iomap_end`` methods. It is strongly recommended to create separate mapping functions and iomap ops for write operations. Then convert the direct I/O write path to iomap, and start running fsx w/ DIO enabled in earnest on filesystem. This will flush out lots of data integrity corner case bugs that the new write mapping implementation introduces.h](h"The next thing to do is implement }(hjvhhhNhNubj)}(h``get_blocks(create == true)``h]hget_blocks(create == true)}(hj~hhhNhNubah}(h]h ]h"]h$]h&]uh1jhjvubh functionality in the }(hjvhhhNhNubj)}(h``->iomap_begin``h]h ->iomap_begin}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjvubh/}(hjvhhhNhNubj)}(h``->iomap_end``h]h ->iomap_end}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjvubhXO methods. It is strongly recommended to create separate mapping functions and iomap ops for write operations. Then convert the direct I/O write path to iomap, and start running fsx w/ DIO enabled in earnest on filesystem. This will flush out lots of data integrity corner case bugs that the new write mapping implementation introduces.}(hjvhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhK[hjhhubj)}(hNow, convert any remaining file operations to call the iomap functions. This will get the entire filesystem using the new mapping functions, and they should largely be debugged and working correctly after this step.h]hNow, convert any remaining file operations to call the iomap functions. This will get the entire filesystem using the new mapping functions, and they should largely be debugged and working correctly after this step.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKdhjhhubj)}(hXMost likely at this point, the buffered read and write paths will still need to be converted. The mapping functions should all work correctly, so all that needs to be done is rewriting all the code that interfaces with bufferheads to interface with iomap and folios. It is much easier first to get regular file I/O (without any fancy features like fscrypt, fsverity, compression, or data=journaling) converted to use iomap. Some of those fancy features (fscrypt and compression) aren't implemented yet in iomap. For unjournalled filesystems that use the pagecache for symbolic links and directories, you might also try converting their handling to iomap.h]hXMost likely at this point, the buffered read and write paths will still need to be converted. The mapping functions should all work correctly, so all that needs to be done is rewriting all the code that interfaces with bufferheads to interface with iomap and folios. It is much easier first to get regular file I/O (without any fancy features like fscrypt, fsverity, compression, or data=journaling) converted to use iomap. Some of those fancy features (fscrypt and compression) aren’t implemented yet in iomap. For unjournalled filesystems that use the pagecache for symbolic links and directories, you might also try converting their handling to iomap.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhhjhhubj)}(hThe rest is left as an exercise for the reader, as it will be different for every filesystem. If you encounter problems, email the people and lists in ``get_maintainers.pl`` for help.h](hThe rest is left as an exercise for the reader, as it will be different for every filesystem. If you encounter problems, email the people and lists in }(hjhhhNhNubj)}(h``get_maintainers.pl``h]hget_maintainers.pl}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh for help.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhKuhjhhubeh}(h]j;ah ]h"]how do i convert a filesystem?ah$]h&]uh1hhhhhhhhK-ubeh}(h]porting-your-filesystemah ]h"]porting your filesystemah$]h&]uh1hhhhhhhhK ubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksentryfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerj(error_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourceh _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}refids}nameids}(hhjjjUjPjjjj;u nametypes}(hjjUjjuh}(hhjhjPhjjXj;jjj j5j,u footnote_refs} citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes] citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}j6KsRparse_messages]transform_messages]hsystem_message)}(hhh]j)}(hhh]h3Hyperlink target "iomap_porting" is not referenced.}hjsbah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]levelKtypeINFOsourcehlineKuh1juba transformerN include_log] decorationNhhub.