sphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget./translations/zh_CN/admin-guide/media/mali-c55modnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget./translations/zh_TW/admin-guide/media/mali-c55modnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget./translations/it_IT/admin-guide/media/mali-c55modnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget./translations/ja_JP/admin-guide/media/mali-c55modnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget./translations/ko_KR/admin-guide/media/mali-c55modnameN classnameN refexplicituh1hhh ubh)}(hhh]hPortuguese (Brazilian)}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget./translations/pt_BR/admin-guide/media/mali-c55modnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget./translations/sp_SP/admin-guide/media/mali-c55modnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhhH/var/lib/git/docbuild/linux/Documentation/admin-guide/media/mali-c55.rsthKubhsection)}(hhh](htitle)}(h*ARM Mali-C55 Image Signal Processor driverh]h*ARM Mali-C55 Image Signal Processor driver}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(h Introductionh]h Introduction}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh paragraph)}(hThis file documents the driver for ARM's Mali-C55 Image Signal Processor. The driver is located under drivers/media/platform/arm/mali-c55.h]hThis file documents the driver for ARM’s Mali-C55 Image Signal Processor. The driver is located under drivers/media/platform/arm/mali-c55.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hXGThe Mali-C55 ISP receives data in either raw Bayer format or RGB/YUV format from sensors through either a parallel interface or a memory bus before processing it and outputting it through an internal DMA engine. Two output pipelines are possible (though one may not be fitted, depending on the implementation). These are referred to as "Full resolution" and "Downscale", but the naming is historic and both pipes are capable of cropping/scaling operations. The full resolution pipe is also capable of outputting RAW data, bypassing much of the ISP's processing. The downscale pipe cannot output RAW data. An integrated test pattern generator can be used to drive the ISP and produce image data in the absence of a connected camera sensor. The driver module is named mali_c55, and is enabled through the CONFIG_VIDEO_MALI_C55 config option.h]hXQThe Mali-C55 ISP receives data in either raw Bayer format or RGB/YUV format from sensors through either a parallel interface or a memory bus before processing it and outputting it through an internal DMA engine. Two output pipelines are possible (though one may not be fitted, depending on the implementation). These are referred to as “Full resolution” and “Downscale”, but the naming is historic and both pipes are capable of cropping/scaling operations. The full resolution pipe is also capable of outputting RAW data, bypassing much of the ISP’s processing. The downscale pipe cannot output RAW data. An integrated test pattern generator can be used to drive the ISP and produce image data in the absence of a connected camera sensor. The driver module is named mali_c55, and is enabled through the CONFIG_VIDEO_MALI_C55 config option.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hThe driver implements V4L2, Media Controller and V4L2 Subdevice interfaces and expects camera sensors connected to the ISP to have V4L2 subdevice interfaces.h]hThe driver implements V4L2, Media Controller and V4L2 Subdevice interfaces and expects camera sensors connected to the ISP to have V4L2 subdevice interfaces.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubeh}(h] introductionah ]h"] introductionah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hMali-C55 ISP hardwareh]hMali-C55 ISP hardware}(hj%hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj"hhhhhKubh)}(hA high level functional view of the Mali-C55 ISP is presented below. The ISP takes input from either a live source or through a DMA engine for memory input, depending on the SoC integration.::h]hA high level functional view of the Mali-C55 ISP is presented below. The ISP takes input from either a live source or through a DMA engine for memory input, depending on the SoC integration.:}(hj3hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj"hhubh literal_block)}(hX+---------+ +----------+ +--------+ | Sensor |--->| CSI-2 Rx | "Full Resolution" | DMA | +---------+ +----------+ |\ Output +--->| Writer | | | \ | +--------+ | | \ +----------+ +------+---> Streaming I/O +------------+ +------->| | | | | | | | |-->| Mali-C55 |--+ | DMA Reader |--------------->| | | ISP | | | | | / | | | +---> Streaming I/O +------------+ | / +----------+ | | |/ +------+ | +--------+ +--->| DMA | "Downscaled" | Writer | Output +--------+h]hX+---------+ +----------+ +--------+ | Sensor |--->| CSI-2 Rx | "Full Resolution" | DMA | +---------+ +----------+ |\ Output +--->| Writer | | | \ | +--------+ | | \ +----------+ +------+---> Streaming I/O +------------+ +------->| | | | | | | | |-->| Mali-C55 |--+ | DMA Reader |--------------->| | | ISP | | | | | / | | | +---> Streaming I/O +------------+ | / +----------+ | | |/ +------+ | +--------+ +--->| DMA | "Downscaled" | Writer | Output +--------+}hjCsbah}(h]h ]h"]h$]h&]hhuh1jAhhhK#hj"hhubeh}(h]mali-c55-isp-hardwareah ]h"]mali-c55 isp hardwareah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hMedia Controller Topologyh]hMedia Controller Topology}(hj\hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjYhhhhhK4ubh)}(hAn example of the ISP's topology (as implemented in a system with an IMX415 camera sensor and generic CSI-2 receiver) is below:h]hAn example of the ISP’s topology (as implemented in a system with an IMX415 camera sensor and generic CSI-2 receiver) is below:}(hjjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK6hjYhhubkfigure kernel_figure)}(hhh]hfigure)}(hhh]himage)}(hX.. kernel-figure:: mali-c55-graph.dot :alt: mali-c55-graph.dot :align: center h]h}(h]h ]h"]h$]h&]altmali-c55-graph.doturi$admin-guide/media/mali-c55-graph.dot candidates}*jsuh1jhjhhhKubah}(h]h ]h"]h$]h&]aligncenteruh1j~hj{ubah}(h]h ]h"]h$]h&]uh1jyhjYhhhhhNubh)}(h!The driver has 4 V4L2 subdevices:h]h!The driver has 4 V4L2 subdevices:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK>hjYhhubh bullet_list)}(hhh](h list_item)}(ha`mali_c55 isp`: Responsible for configuring input crop and color space conversionh]hdefinition_list)}(hhh]hdefinition_list_item)}(hQ`mali_c55 isp`: Responsible for configuring input crop and color space conversionh](hterm)}(hF`mali_c55 isp`: Responsible for configuring input crop and color spaceh](htitle_reference)}(h`mali_c55 isp`h]h mali_c55 isp}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh8: Responsible for configuring input crop and color space}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1jhhhK@hjubh definition)}(hhh]h)}(h conversionh]h conversion}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKAhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhK@hjubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1jhjhhhNhNubj)}(hF`mali_c55 tpg`: The test pattern generator, emulating a camera sensor.h]h)}(hjh](j)}(h`mali_c55 tpg`h]h mali_c55 tpg}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh8: The test pattern generator, emulating a camera sensor.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKBhjubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubj)}(h7`mali_c55 resizer fr`: The Full-Resolution pipe resizerh]h)}(hj:h](j)}(h`mali_c55 resizer fr`h]hmali_c55 resizer fr}(hj?hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj<ubh": The Full-Resolution pipe resizer}(hj<hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKChj8ubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubj)}(h2`mali_c55 resizer ds`: The Downscale pipe resizer h]h)}(h1`mali_c55 resizer ds`: The Downscale pipe resizerh](j)}(h`mali_c55 resizer ds`h]hmali_c55 resizer ds}(hjehhhNhNubah}(h]h ]h"]h$]h&]uh1jhjaubh: The Downscale pipe resizer}(hjahhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKDhj]ubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubeh}(h]h ]h"]h$]h&]bullet-uh1jhhhK@hjYhhubh)}(h$The driver has 3 V4L2 video devices:h]h$The driver has 3 V4L2 video devices:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKFhjYhhubj)}(hhh](j)}(h8`mali-c55 fr`: The full-resolution pipe's capture deviceh]h)}(hjh](j)}(h `mali-c55 fr`h]h mali-c55 fr}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh-: The full-resolution pipe’s capture device}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKHhjubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubj)}(h2`mali-c55 ds`: The downscale pipe's capture deviceh]h)}(hjh](j)}(h `mali-c55 ds`h]h mali-c55 ds}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh': The downscale pipe’s capture device}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKIhjubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubj)}(h6`mali-c55 3a stats`: The 3A statistics capture device h]h)}(h5`mali-c55 3a stats`: The 3A statistics capture deviceh](j)}(h`mali-c55 3a stats`h]hmali-c55 3a stats}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh": The 3A statistics capture device}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKJhjubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubeh}(h]h ]h"]h$]h&]jjuh1jhhhKHhjYhhubh)}(hFrame sequences are synchronised across to two capture devices, meaning if one pipe is started later than the other the sequence numbers returned in its buffers will match those of the other pipe rather than starting from zero.h]hFrame sequences are synchronised across to two capture devices, meaning if one pipe is started later than the other the sequence numbers returned in its buffers will match those of the other pipe rather than starting from zero.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKLhjYhhubh)}(hhh](h)}(hIdiosyncrasiesh]hIdiosyncrasies}(hj#hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhKQubh)}(hX**mali-c55 isp** The `mali-c55 isp` subdevice has a single sink pad to which all sources of data should be connected. The active source is selected by enabling the appropriate media link and disabling all others. The ISP has two source pads, reflecting the different paths through which it can internally route data. Tap points within the ISP allow users to divert data to avoid processing by some or all of the hardware's processing steps. The diagram below is intended only to highlight how the bypassing works and is not a true reflection of those processing steps; for a high-level functional block diagram see ARM's developer page for the ISP [3]_::h](hstrong)}(h**mali-c55 isp**h]h mali-c55 isp}(hj7hhhNhNubah}(h]h ]h"]h$]h&]uh1j5hj1ubh The }(hj1hhhNhNubj)}(h`mali-c55 isp`h]h mali-c55 isp}(hjIhhhNhNubah}(h]h ]h"]h$]h&]uh1jhj1ubhXi subdevice has a single sink pad to which all sources of data should be connected. The active source is selected by enabling the appropriate media link and disabling all others. The ISP has two source pads, reflecting the different paths through which it can internally route data. Tap points within the ISP allow users to divert data to avoid processing by some or all of the hardware’s processing steps. The diagram below is intended only to highlight how the bypassing works and is not a true reflection of those processing steps; for a high-level functional block diagram see ARM’s developer page for the ISP }(hj1hhhNhNubhfootnote_reference)}(h[3]_h]h3}(hj]hhhNhNubah}(h]id1ah ]h"]h$]h&]refidid6docnameadmin-guide/media/mali-c55uh1j[hj1resolvedKubh:}(hj1hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKShj hhubjB)}(hX +--------------------------------------------------------------+ | Possible Internal ISP Data Routes | | +------------+ +----------+ +------------+ | +---+ | | | | | Colour | +---+ | 0 |--+-->| Processing |->| Demosaic |->| Space |--->| 1 | +---+ | | | | | | Conversion | +---+ | | +------------+ +----------+ +------------+ | | | +---+ | +---------------------------------------------------| 2 | | +---+ | | +--------------------------------------------------------------+h]hX +--------------------------------------------------------------+ | Possible Internal ISP Data Routes | | +------------+ +----------+ +------------+ | +---+ | | | | | Colour | +---+ | 0 |--+-->| Processing |->| Demosaic |->| Space |--->| 1 | +---+ | | | | | | Conversion | +---+ | | +------------+ +----------+ +------------+ | | | +---+ | +---------------------------------------------------| 2 | | +---+ | | +--------------------------------------------------------------+}hj{sbah}(h]h ]h"]h$]h&]hhuh1jAhhhK^hj hhubhtable)}(hhh]htgroup)}(hhh](hcolspec)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjubhthead)}(hhh]hrow)}(hhh](hentry)}(hhh]h)}(hPadh]hPad}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKohjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h Directionh]h Direction}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKphjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hPurposeh]hPurpose}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKqhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1jhjubhtbody)}(hhh](j)}(hhh](j)}(hhh]h)}(h0h]h0}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKshjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hsinkh]hsink}(hj2hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKthj/ubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h3Data input, connected to the TPG and camera sensorsh]h3Data input, connected to the TPG and camera sensors}(hjIhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKuhjFubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(h1h]h1}(hjihhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKwhjfubah}(h]h ]h"]h$]h&]uh1jhjcubj)}(hhh]h)}(hsourceh]hsource}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKxhj}ubah}(h]h ]h"]h$]h&]uh1jhjcubj)}(hhh]h)}(h8RGB/YUV data, connected to the FR and DS V4L2 subdevicesh]h8RGB/YUV data, connected to the FR and DS V4L2 subdevices}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKyhjubah}(h]h ]h"]h$]h&]uh1jhjcubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(h2h]h2}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK{hjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hsourceh]hsource}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK|hjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h3RAW bayer data, connected to the FR V4L2 subdevicesh]h3RAW bayer data, connected to the FR V4L2 subdevices}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK}hjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]colsKuh1jhjubah}(h]h ]h"]h$]h&]uh1jhj hhhNhNubh)}(hThe ISP is limited to both input and output resolutions between 640x480 and 8192x8192, and this is reflected in the ISP and resizer subdevice's .set_fmt() operations.h]hThe ISP is limited to both input and output resolutions between 640x480 and 8192x8192, and this is reflected in the ISP and resizer subdevice’s .set_fmt() operations.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj hhubh)}(h**mali-c55 resizer fr** The `mali-c55 resizer fr` subdevice has two _sink_ pads to reflect the different insertion points in the hardware (either RAW or demosaiced data):h](j6)}(h**mali-c55 resizer fr**h]hmali-c55 resizer fr}(hj$hhhNhNubah}(h]h ]h"]h$]h&]uh1j5hj ubh The }(hj hhhNhNubj)}(h`mali-c55 resizer fr`h]hmali-c55 resizer fr}(hj6hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubhy subdevice has two _sink_ pads to reflect the different insertion points in the hardware (either RAW or demosaiced data):}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj hhubj)}(hhh]j)}(hhh](j)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjQubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjQubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjQubj)}(hhh]j)}(hhh](j)}(hhh]h)}(hPadh]hPad}(hj{hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjxubah}(h]h ]h"]h$]h&]uh1jhjuubj)}(hhh]h)}(h Directionh]h Direction}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjuubj)}(hhh]h)}(hPurposeh]hPurpose}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjuubeh}(h]h ]h"]h$]h&]uh1jhjrubah}(h]h ]h"]h$]h&]uh1jhjQubj)}(hhh](j)}(hhh](j)}(hhh]h)}(hjh]h0}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hsinkh]hsink}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h4Data input connected to the ISP's demosaiced stream.h]h6Data input connected to the ISP’s demosaiced stream.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hjkh]h1}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hsourceh]hsource}(hj5hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj2ubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h1Data output connected to the capture video deviceh]h1Data output connected to the capture video device}(hjLhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjIubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hjh]h2}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjiubah}(h]h ]h"]h$]h&]uh1jhjfubj)}(hhh]h)}(hsinkh]hsink}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjfubj)}(hhh]h)}(h1Data input connected to the ISP's raw data streamh]h3Data input connected to the ISP’s raw data stream}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjfubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjQubeh}(h]h ]h"]h$]h&]colsKuh1jhjNubah}(h]h ]h"]h$]h&]uh1jhj hhhNhNubh)}(hmThe data source in use is selected through the routing API; two routes each of a single stream are available:h]hmThe data source in use is selected through the routing API; two routes each of a single stream are available:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj hhubj)}(hhh]j)}(hhh](j)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthK!uh1jhjubj)}(hhh]j)}(hhh](j)}(hhh]h)}(hSink Padh]hSink Pad}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h Source Padh]h Source Pad}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hPurposeh]hPurpose}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj,ubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh](j)}(hhh]h)}(hjh]h0}(hjXhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjUubah}(h]h ]h"]h$]h&]uh1jhjRubj)}(hhh]h)}(hjkh]h1}(hjnhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjkubah}(h]h ]h"]h$]h&]uh1jhjRubj)}(hhh]h)}(hDemosaiced data routeh]hDemosaiced data route}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjRubeh}(h]h ]h"]h$]h&]uh1jhjOubj)}(hhh](j)}(hhh]h)}(hjh]h2}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hjkh]h1}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hRaw data routeh]hRaw data route}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjOubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]colsKuh1jhjubah}(h]h ]h"]h$]h&]uh1jhj hhhNhNubh)}(hIf the demosaiced route is active then the FR pipe is only capable of output in RGB/YUV formats. If the raw route is active then the output reflects the input (which may be either Bayer or RGB/YUV data).h]hIf the demosaiced route is active then the FR pipe is only capable of output in RGB/YUV formats. If the raw route is active then the output reflects the input (which may be either Bayer or RGB/YUV data).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj hhubeh}(h]idiosyncrasiesah ]h"]idiosyncrasiesah$]h&]uh1hhjYhhhhhKQubeh}(h]media-controller-topologyah ]h"]media controller topologyah$]h&]uh1hhhhhhhhK4ubh)}(hhh](h)}(h!Using the driver to capture videoh]h!Using the driver to capture video}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hXUsing the media controller APIs we can configure the input source and ISP to capture images in a variety of formats. In the examples below, configuring the media graph is done with the v4l-utils [1]_ package's media-ctl utility. Capturing the images is done with yavta [2]_.h](hUsing the media controller APIs we can configure the input source and ISP to capture images in a variety of formats. In the examples below, configuring the media graph is done with the v4l-utils }(hj,hhhNhNubj\)}(h[1]_h]h1}(hj4hhhNhNubah}(h]id2ah ]h"]h$]h&]jlid4jnjouh1j[hj,jpKubhH package’s media-ctl utility. Capturing the images is done with yavta }(hj,hhhNhNubj\)}(h[2]_h]h2}(hjHhhhNhNubah}(h]id3ah ]h"]h$]h&]jlid5jnjouh1j[hj,jpKubh.}(hj,hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hhh](h)}(hConfiguring the input sourceh]hConfiguring the input source}(hjehhhNhNubah}(h]h ]h"]h$]h&]uh1hhjbhhhhhKubh)}(hThe first step is to set the input source that we wish by enabling the correct media link. Using the example topology above, we can select the TPG as follows:h]hThe first step is to set the input source that we wish by enabling the correct media link. Using the example topology above, we can select the TPG as follows:}(hjshhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjbhhubjB)}(hhmedia-ctl -l "'lte-csi2-rx':1->'mali-c55 isp':0[0]" media-ctl -l "'mali-c55 tpg':0->'mali-c55 isp':0[1]"h]hhmedia-ctl -l "'lte-csi2-rx':1->'mali-c55 isp':0[0]" media-ctl -l "'mali-c55 tpg':0->'mali-c55 isp':0[1]"}hjsbah}(h]h ]h"]h$]h&]hhƌforcelanguagenonehighlight_args}uh1jAhhhKhjbhhubeh}(h]configuring-the-input-sourceah ]h"]configuring the input sourceah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(h0Configuring which video devices will stream datah]h0Configuring which video devices will stream data}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hX-The driver will wait for all video devices to have their VIDIOC_STREAMON ioctl called before it tells the sensor to start streaming. To facilitate this we need to enable links to the video devices that we want to use. In the example below we enable the links to both of the image capture video devicesh]hX-The driver will wait for all video devices to have their VIDIOC_STREAMON ioctl called before it tells the sensor to start streaming. To facilitate this we need to enable links to the video devices that we want to use. In the example below we enable the links to both of the image capture video devices}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjB)}(humedia-ctl -l "'mali-c55 resizer fr':1->'mali-c55 fr':0[1]" media-ctl -l "'mali-c55 resizer ds':1->'mali-c55 ds':0[1]"h]humedia-ctl -l "'mali-c55 resizer fr':1->'mali-c55 fr':0[1]" media-ctl -l "'mali-c55 resizer ds':1->'mali-c55 ds':0[1]"}hjsbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhKhjhhubeh}(h]0configuring-which-video-devices-will-stream-dataah ]h"]0configuring which video devices will stream dataah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(h>Capturing bayer data from the source and processing to RGB/YUVh]h>Capturing bayer data from the source and processing to RGB/YUV}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hXTo capture 1920x1080 bayer data from the source and push it through the ISP's full processing pipeline, we configure the data formats appropriately on the source, ISP and resizer subdevices and set the FR resizer's routing to select processed data. The media bus format on the resizer's source pad will be either RGB121212_1X36 or YUV10_1X30, depending on whether you want to capture RGB or YUV. The ISP's debayering block outputs RGB data natively, setting the source pad format to YUV10_1X30 enables the colour space conversion block.h]hX To capture 1920x1080 bayer data from the source and push it through the ISP’s full processing pipeline, we configure the data formats appropriately on the source, ISP and resizer subdevices and set the FR resizer’s routing to select processed data. The media bus format on the resizer’s source pad will be either RGB121212_1X36 or YUV10_1X30, depending on whether you want to capture RGB or YUV. The ISP’s debayering block outputs RGB data natively, setting the source pad format to YUV10_1X30 enables the colour space conversion block.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(heIn this example we target RGB565 output, so select RGB121212_1X36 as the resizer source pad's format:h]hgIn this example we target RGB565 output, so select RGB121212_1X36 as the resizer source pad’s format:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjB)}(hX# Set formats on the TPG and ISP media-ctl -V "'mali-c55 tpg':0[fmt:SRGGB20_1X20/1920x1080]" media-ctl -V "'mali-c55 isp':0[fmt:SRGGB20_1X20/1920x1080]" media-ctl -V "'mali-c55 isp':1[fmt:SRGGB20_1X20/1920x1080]" # Set routing on the FR resizer media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[1],2/0->1/0[0]]" # Set format on the resizer, must be done AFTER the routing. media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/1920x1080]"h]hX# Set formats on the TPG and ISP media-ctl -V "'mali-c55 tpg':0[fmt:SRGGB20_1X20/1920x1080]" media-ctl -V "'mali-c55 isp':0[fmt:SRGGB20_1X20/1920x1080]" media-ctl -V "'mali-c55 isp':1[fmt:SRGGB20_1X20/1920x1080]" # Set routing on the FR resizer media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[1],2/0->1/0[0]]" # Set format on the resizer, must be done AFTER the routing. media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/1920x1080]"}hj sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhKhjhhubh)}(hThe downscale output can also be used to stream data at the same time. In this case since only processed data can be captured through the downscale output no routing need be set:h]hThe downscale output can also be used to stream data at the same time. In this case since only processed data can be captured through the downscale output no routing need be set:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjB)}(h`# Set format on the resizer media-ctl -V "'mali-c55 resizer ds':1[fmt:RGB121212_1X36/1920x1080]"h]h`# Set format on the resizer media-ctl -V "'mali-c55 resizer ds':1[fmt:RGB121212_1X36/1920x1080]"}hj sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhKhjhhubh)}(hsFollowing which images can be captured from both the FR and DS output's video devices (simultaneously, if desired):h]huFollowing which images can be captured from both the FR and DS output’s video devices (simultaneously, if desired):}(hj. hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjB)}(h[yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 yavta -f RGB565 -s 1920x1080 -c10 /dev/video1h]h[yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 yavta -f RGB565 -s 1920x1080 -c10 /dev/video1}hj< sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhKhjhhubh)}(hhh](h)}(hCropping the imageh]hCropping the image}(hjO hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjL hhhhhKubh)}(hBoth the full resolution and downscale pipes can crop to a minimum resolution of 640x480. To crop the image simply configure the resizer's sink pad's crop and compose rectangles and set the format on the video device:h]hBoth the full resolution and downscale pipes can crop to a minimum resolution of 640x480. To crop the image simply configure the resizer’s sink pad’s crop and compose rectangles and set the format on the video device:}(hj] hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjL hhubjB)}(hmedia-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(480,270)/640x480 compose:(0,0)/640x480]" media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" yavta -f RGB565 -s 640x480 -c10 /dev/video0h]hmedia-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(480,270)/640x480 compose:(0,0)/640x480]" media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" yavta -f RGB565 -s 640x480 -c10 /dev/video0}hjk sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhMhjL hhubeh}(h]cropping-the-imageah ]h"]cropping the imageah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(hDownscaling the imageh]hDownscaling the image}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhM ubh)}(hX.Both the full resolution and downscale pipes can downscale the image by up to 8x provided the minimum 640x480 output resolution is adhered to. For the best image result the scaling ratio for each direction should be the same. To configure scaling we use the compose rectangle on the resizer's sink pad:h]hX0Both the full resolution and downscale pipes can downscale the image by up to 8x provided the minimum 640x480 output resolution is adhered to. For the best image result the scaling ratio for each direction should be the same. To configure scaling we use the compose rectangle on the resizer’s sink pad:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hj hhubjB)}(hmedia-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(0,0)/1920x1080 compose:(0,0)/640x480]" media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" yavta -f RGB565 -s 640x480 -c10 /dev/video0h]hmedia-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(0,0)/1920x1080 compose:(0,0)/640x480]" media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" yavta -f RGB565 -s 640x480 -c10 /dev/video0}hj sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhMhj hhubeh}(h]downscaling-the-imageah ]h"]downscaling the imageah$]h&]uh1hhjhhhhhM ubh)}(hhh](h)}(hCapturing images in YUV formatsh]hCapturing images in YUV formats}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMubh)}(hX If we need to output YUV data rather than RGB the color space conversion block needs to be active, which is achieved by setting MEDIA_BUS_FMT_YUV10_1X30 on the resizer's source pad. We can then configure a capture format like NV12 (here in its multi-planar variant)h]hX If we need to output YUV data rather than RGB the color space conversion block needs to be active, which is achieved by setting MEDIA_BUS_FMT_YUV10_1X30 on the resizer’s source pad. We can then configure a capture format like NV12 (here in its multi-planar variant)}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubjB)}(hmmedia-ctl -V "'mali-c55 resizer fr':1[fmt:YUV10_1X30/1920x1080]" yavta -f NV12M -s 1920x1080 -c10 /dev/video0h]hmmedia-ctl -V "'mali-c55 resizer fr':1[fmt:YUV10_1X30/1920x1080]" yavta -f NV12M -s 1920x1080 -c10 /dev/video0}hj sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhM hj hhubeh}(h]capturing-images-in-yuv-formatsah ]h"]capturing images in yuv formatsah$]h&]uh1hhjhhhhhMubeh}(h]>capturing-bayer-data-from-the-source-and-processing-to-rgb-yuvah ]h"]>capturing bayer data from the source and processing to rgb/yuvah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(hFCapturing RGB data from the source and processing it with the resizersh]hFCapturing RGB data from the source and processing it with the resizers}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhM&ubh)}(hX%The Mali-C55 ISP can work with sensors capable of outputting RGB data. In this case although none of the image quality blocks would be used it can still crop/scale the data in the usual way. For this reason RGB data input to the ISP still goes through the ISP subdevice's pad 1 to the resizer.h]hX'The Mali-C55 ISP can work with sensors capable of outputting RGB data. In this case although none of the image quality blocks would be used it can still crop/scale the data in the usual way. For this reason RGB data input to the ISP still goes through the ISP subdevice’s pad 1 to the resizer.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM(hj hhubh)}(hTo achieve this, the ISP's sink pad's format is set to MEDIA_BUS_FMT_RGB202020_1X60 - this reflects the format that data must be in to work with the ISP. Converting the camera sensor's output to that format is the responsibility of external hardware.h]hXTo achieve this, the ISP’s sink pad’s format is set to MEDIA_BUS_FMT_RGB202020_1X60 - this reflects the format that data must be in to work with the ISP. Converting the camera sensor’s output to that format is the responsibility of external hardware.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM-hj hhubh)}(hWIn this example we ask the test pattern generator to give us RGB data instead of bayer.h]hWIn this example we ask the test pattern generator to give us RGB data instead of bayer.}(hj& hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM2hj hhubjB)}(h{media-ctl -V "'mali-c55 tpg':0[fmt:RGB202020_1X60/1920x1080]" media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]"h]h{media-ctl -V "'mali-c55 tpg':0[fmt:RGB202020_1X60/1920x1080]" media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]"}hj4 sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhM5hj hhubh)}(hUCropping or scaling the data can be done in exactly the same way as outlined earlier.h]hUCropping or scaling the data can be done in exactly the same way as outlined earlier.}(hjD hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM:hj hhubeh}(h]Fcapturing-rgb-data-from-the-source-and-processing-it-with-the-resizersah ]h"]Fcapturing rgb data from the source and processing it with the resizersah$]h&]uh1hhjhhhhhM&ubh)}(hhh](h)}(h?Capturing raw data from the source and outputting it unmodifiedh]h?Capturing raw data from the source and outputting it unmodified}(hj] hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjZ hhhhhM>ubh)}(hThe ISP can additionally capture raw data from the source and output it on the full resolution pipe only, completely unmodified. In this case the downscale pipe can still process the data normally and be used at the same time.h]hThe ISP can additionally capture raw data from the source and output it on the full resolution pipe only, completely unmodified. In this case the downscale pipe can still process the data normally and be used at the same time.}(hjk hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM@hjZ hhubh)}(hTo configure raw bypass the FR resizer's subdevice's routing table needs to be configured, followed by formats in the appropriate places:h]hTo configure raw bypass the FR resizer’s subdevice’s routing table needs to be configured, followed by formats in the appropriate places:}(hjy hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMDhjZ hhubjB)}(hX`media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[0],2/0->1/0[1]]" media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]" media-ctl -V "'mali-c55 resizer fr':2[fmt:RGB202020_1X60/1920x1080]" media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB202020_1X60/1920x1080]" # Set format on the video device and stream yavta -f RGB565 -s 1920x1080 -c10 /dev/video0h]hX`media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[0],2/0->1/0[1]]" media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]" media-ctl -V "'mali-c55 resizer fr':2[fmt:RGB202020_1X60/1920x1080]" media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB202020_1X60/1920x1080]" # Set format on the video device and stream yavta -f RGB565 -s 1920x1080 -c10 /dev/video0}hj sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhMGhjZ hhubhtarget)}(h.. _mali-c55-3a-stats:h]h}(h]h ]h"]h$]h&]jlmali-c55-3a-statsuh1j hMQhjZ hhhhubeh}(h]?capturing-raw-data-from-the-source-and-outputting-it-unmodifiedah ]h"]?capturing raw data from the source and outputting it unmodifiedah$]h&]uh1hhjhhhhhM>ubeh}(h]!using-the-driver-to-capture-videoah ]h"]!using the driver to capture videoah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hCapturing ISP Statisticsh]hCapturing ISP Statistics}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMTubh)}(hXlThe ISP is capable of producing statistics for consumption by image processing algorithms running in userspace. These statistics can be captured by queueing buffers to the `mali-c55 3a stats` V4L2 Device whilst the ISP is streaming. Only the :ref:`V4L2_META_FMT_MALI_C55_STATS ` format is supported, so no format-setting need be done:h](hThe ISP is capable of producing statistics for consumption by image processing algorithms running in userspace. These statistics can be captured by queueing buffers to the }(hj hhhNhNubj)}(h`mali-c55 3a stats`h]hmali-c55 3a stats}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubh3 V4L2 Device whilst the ISP is streaming. Only the }(hj hhhNhNubh)}(hB:ref:`V4L2_META_FMT_MALI_C55_STATS `h]hinline)}(hj h]hV4L2_META_FMT_MALI_C55_STATS}(hj hhhNhNubah}(h]h ](xrefstdstd-refeh"]h$]h&]uh1j hj ubah}(h]h ]h"]h$]h&]refdocjo refdomainj reftyperef refexplicitrefwarn reftargetv4l2-meta-fmt-mali-c55-statsuh1hhhhMVhj ubh8 format is supported, so no format-setting need be done:}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMVhj hhubjB)}(hX# We assume the media graph has been configured to support RGB565 capture # from the mali-c55 fr V4L2 Device, which is at /dev/video0. The statistics # V4L2 device is at /dev/video3 yavta -f RGB565 -s 1920x1080 -c32 /dev/video0 && \ yavta -c10 -F /dev/video3h]hX# We assume the media graph has been configured to support RGB565 capture # from the mali-c55 fr V4L2 Device, which is at /dev/video0. The statistics # V4L2 device is at /dev/video3 yavta -f RGB565 -s 1920x1080 -c32 /dev/video0 && \ yavta -c10 -F /dev/video3}hj sbah}(h]h ]h"]h$]h&]hhjjnonej}uh1jAhhhM\hj hhubh)}(hXThe layout of the buffer is described by :c:type:`mali_c55_stats_buffer`, but broadly statistics are generated to support three image processing algorithms; AEXP (Auto-Exposure), AWB (Auto-White Balance) and AF (Auto-Focus). These stats can be drawn from various places in the Mali C55 ISP pipeline, known as "tap points". This high-level block diagram is intended to explain where in the processing flow the statistics can be drawn from::h](h)The layout of the buffer is described by }(hj hhhNhNubh)}(h:c:type:`mali_c55_stats_buffer`h]hliteral)}(hj' h]hmali_c55_stats_buffer}(hj+ hhhNhNubah}(h]h ](j cc-typeeh"]h$]h&]uh1j) hj% ubah}(h]h ]h"]h$]h&]refdocjo refdomainj5 reftypetype refexplicitrefwarnj mali_c55_stats_bufferuh1hhhhMehj ubhXr, but broadly statistics are generated to support three image processing algorithms; AEXP (Auto-Exposure), AWB (Auto-White Balance) and AF (Auto-Focus). These stats can be drawn from various places in the Mali C55 ISP pipeline, known as “tap points”. This high-level block diagram is intended to explain where in the processing flow the statistics can be drawn from:}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMehj hhubjB)}(hX2 +--> AEXP-2 +----> AEXP-1 +--> AF-0 | +----> AF-1 | | | | +---------+ | +--------------+ | +--------------+ | | Input +-+-->+ Digital Gain +---+-->+ Black Level +---+---+ +---------+ +--------------+ +--------------+ | +-----------------------------------------------------------------+ | | +--------------+ +---------+ +----------------+ +-->| Sinter Noise +-+ White +--+--->| Lens Shading +--+---------------+ | Reduction | | Balance | | | | | | +--------------+ +---------+ | +----------------+ | | +---> AEXP-0 (A) +--> AEXP-0 (B) | +--------------------------------------------------------------------------+ | | +----------------+ +--------------+ +----------------+ +-->| Tone mapping +-+--->| Demosaicing +->+ Purple Fringe +-+-----------+ | | | +--------------+ | Correction | | | +----------------+ +-> AEXP-IRIDIX +----------------+ +---> AWB-0 | +----------------------------------------------------------------------------+ | +-------------+ +-------------+ +------------------->| Colour +---+--->| Output | | Correction | | | Pipelines | +-------------+ | +-------------+ +--> AWB-1h]hX2 +--> AEXP-2 +----> AEXP-1 +--> AF-0 | +----> AF-1 | | | | +---------+ | +--------------+ | +--------------+ | | Input +-+-->+ Digital Gain +---+-->+ Black Level +---+---+ +---------+ +--------------+ +--------------+ | +-----------------------------------------------------------------+ | | +--------------+ +---------+ +----------------+ +-->| Sinter Noise +-+ White +--+--->| Lens Shading +--+---------------+ | Reduction | | Balance | | | | | | +--------------+ +---------+ | +----------------+ | | +---> AEXP-0 (A) +--> AEXP-0 (B) | +--------------------------------------------------------------------------+ | | +----------------+ +--------------+ +----------------+ +-->| Tone mapping +-+--->| Demosaicing +->+ Purple Fringe +-+-----------+ | | | +--------------+ | Correction | | | +----------------+ +-> AEXP-IRIDIX +----------------+ +---> AWB-0 | +----------------------------------------------------------------------------+ | +-------------+ +-------------+ +------------------->| Colour +---+--->| Output | | Correction | | | Pipelines | +-------------+ | +-------------+ +--> AWB-1}hjQ sbah}(h]h ]h"]h$]h&]hhuh1jAhhhMlhj hhubh)}(hXBy default all statistics are drawn from the 0th tap point for each algorithm; I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF statistics from AF-0. This is configurable for AEXP and AWB statsistics through programming the ISP's parameters.h]hX By default all statistics are drawn from the 0th tap point for each algorithm; I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF statistics from AF-0. This is configurable for AEXP and AWB statsistics through programming the ISP’s parameters.}(hj_ hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubj )}(h.. _mali-c55-3a-params:h]h}(h]h ]h"]h$]h&]jlmali-c55-3a-paramsuh1j hMhj hhhhubeh}(h](capturing-isp-statisticsj eh ]h"](capturing isp statisticsmali-c55-3a-statseh$]h&]uh1hhhhhhhhMTexpect_referenced_by_name}j~ j sexpect_referenced_by_id}j j subh)}(hhh](h)}(hProgramming ISP Parametersh]hProgramming ISP Parameters}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMubh)}(hThe ISP can be programmed with various parameters from userspace to apply to the hardware before and during video stream. This allows userspace to dynamically change values such as black level, white balance and lens shading gains and so on.h]hThe ISP can be programmed with various parameters from userspace to apply to the hardware before and during video stream. This allows userspace to dynamically change values such as black level, white balance and lens shading gains and so on.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hThe buffer format and how to populate it are described by the :ref:`V4L2_META_FMT_MALI_C55_PARAMS ` format, which should be set as the data format for the `mali-c55 3a params` video node.h](h>The buffer format and how to populate it are described by the }(hj hhhNhNubh)}(hD:ref:`V4L2_META_FMT_MALI_C55_PARAMS `h]j )}(hj h]hV4L2_META_FMT_MALI_C55_PARAMS}(hj hhhNhNubah}(h]h ](j stdstd-refeh"]h$]h&]uh1j hj ubah}(h]h ]h"]h$]h&]refdocjo refdomainj reftyperef refexplicitrefwarnj v4l2-meta-fmt-mali-c55-paramsuh1hhhhMhj ubh8 format, which should be set as the data format for the }(hj hhhNhNubj)}(h`mali-c55 3a params`h]hmali-c55 3a params}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubh video node.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj hhubeh}(h](programming-isp-parametersjw eh ]h"](programming isp parametersmali-c55-3a-paramseh$]h&]uh1hhhhhhhhMj }j jm sj }jw jm subh)}(hhh](h)}(h Referencesh]h References}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMubhfootnote)}(h&https://git.linuxtv.org/v4l-utils.git/h](hlabel)}(h1h]h1}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh)}(hj h]h reference)}(hj h]h&https://git.linuxtv.org/v4l-utils.git/}(hj hhhNhNubah}(h]h ]h"]h$]h&]refurij uh1j hj ubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubeh}(h]jCah ]h"]1ah$]h&]j>ajnjouh1j hhhMhj hhjpKubj )}(h&https://git.ideasonboard.org/yavta.gith](j )}(h2h]h2}(hj> hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj: ubh)}(hj< h]j )}(hj< h]h&https://git.ideasonboard.org/yavta.git}(hjO hhhNhNubah}(h]h ]h"]h$]h&]refurij< uh1j hjL ubah}(h]h ]h"]h$]h&]uh1hhhhMhj: ubeh}(h]jWah ]h"]2ah$]h&]jRajnjouh1j hhhMhj hhjpKubj )}(h-https://developer.arm.com/Processors/Mali-C55h](j )}(h3h]h3}(hjn hhhNhNubah}(h]h ]h"]h$]h&]uh1j hjj ubh)}(hjl h]j )}(hjl h]h-https://developer.arm.com/Processors/Mali-C55}(hj hhhNhNubah}(h]h ]h"]h$]h&]refurijl uh1j hj| ubah}(h]h ]h"]h$]h&]uh1hhhhMhjj ubeh}(h]jmah ]h"]3ah$]h&]jgajnjouh1j hhhMhj hhjpKubeh}(h] referencesah ]h"] referencesah$]h&]uh1hhhhhhhhMubeh}(h]*arm-mali-c55-image-signal-processor-driverah ]h"]*arm mali-c55 image signal processor driverah$]h&]uh1hhhhhhhhKubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksjfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerj error_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourcehnj _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}(3]j]a1]j4a2]jHaurefids}(j ]j ajw ]jm aunameids}(j j jjjVjSjjjj j j jjjjj j j j} j j j j jW jT j j j~ j j} jz j jw j j j j j7 jCjg jWj jmu nametypes}(j jjVjjj jjj j j j jW j j~ j} j j j j7 jg j uh}(j hjhjSj"jjYj j jgj]j jj>j4jRjHjjbjjj jj} jL j j j j jT j j jZ j j jz j jw j j j j j jCj jWj: jmjj u footnote_refs}(j ]j]aj ]j4aj ]jHau citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes](j j: jj e citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}j KsRparse_messages]transform_messages](hsystem_message)}(hhh]h)}(hhh]h7Hyperlink target "mali-c55-3a-stats" is not referenced.}hj@ sbah}(h]h ]h"]h$]h&]uh1hhj= ubah}(h]h ]h"]h$]h&]levelKtypeINFOsourcehnjlineMQuh1j; ubj< )}(hhh]h)}(hhh]h8Hyperlink target "mali-c55-3a-params" is not referenced.}hj[ sbah}(h]h ]h"]h$]h&]uh1hhjX ubah}(h]h ]h"]h$]h&]levelKtypejU sourcehnjlineMuh1j; ube transformerN include_log] decorationNhhub.