V4L2 drivers are implemented as kernel modules, loaded manually by the system administrator or automatically when a device is first opened. The driver modules plug into the "videodev" kernel module. It provides helper functions and a common application interface specified in this document.

Each driver thus loaded registers one or more device nodes with major number 81 and a minor number between 0 and 255. Assigning minor numbers to V4L2 devices is entirely up to the system administrator, this is primarily intended to solve conflicts between devices.^[1] The module options to select minor numbers are named after the device special file with a "_nr" suffix. For example "video_nr" for /dev/video video capture devices. The number is an offset to the base minor number associated with the device type. ^[2] When the driver supports multiple devices of the same type more than one minor number can be assigned, separated by commas:

> insmod mydriver.o video_nr=0,1 radio_nr=0,1

In /etc/modules.conf this may be written as:

alias char-major-81-0 mydriver
alias char-major-81-1 mydriver
alias char-major-81-64 mydriver              
options mydriver video_nr=0,1 radio_nr=0,1   
	  

When no minor number is given as module option the driver supplies a default. Chapter 4, Interfaces recommends the base minor numbers to be used for the various device types. Obviously minor numbers must be unique. When the number is already in use the offending device will not be registered.

By convention system administrators create various character device special files with these major and minor numbers in the /dev directory. The names recommended for the different V4L2 device types are listed in Chapter 4, Interfaces.

The creation of character special files (with mknod) is a privileged operation and devices cannot be opened by major and minor number. That means applications cannot reliable scan for loaded or installed drivers. The user must enter a device name, or the application can try the conventional device names.

Under the device filesystem (devfs) the minor number options are ignored. V4L2 drivers (or by proxy the "videodev" module) automatically create the required device files in the /dev/v4l directory using the conventional device names above.

Devices can support several related functions. For example video capturing, video overlay and VBI capturing are related because these functions share, amongst other, the same video input and tuner frequency. V4L and earlier versions of V4L2 used the same device name and minor number for video capturing and overlay, but different ones for VBI. Experience showed this approach has several problems^[3], and to make things worse the V4L videodev module used to prohibit multiple opens of a device.

As a remedy the present version of the V4L2 API relaxed the concept of device types with specific names and minor numbers. For compatibility with old applications drivers must still register different minor numbers to assign a default function to the device. But if related functions are supported by the driver they must be available under all registered minor numbers. The desired function can be selected after opening the device as described in Chapter 4, Interfaces.

Imagine a driver supporting video capturing, video overlay, raw VBI capturing, and FM radio reception. It registers three devices with minor number 0, 64 and 224 (this numbering scheme is inherited from the V4L API). Regardless if /dev/video (81, 0) or /dev/vbi (81, 224) is opened the application can select any one of the video capturing, overlay or VBI capturing functions. Without programming (e. g. reading from the device with dd or cat) /dev/video captures video images, while /dev/vbi captures raw VBI data. /dev/radio (81, 64) is invariable a radio device, unrelated to the video functions. Being unrelated does not imply the devices can be used at the same time, however. The open() function may very well return an EBUSY error code.

Besides video input or output the hardware may also support audio sampling or playback. If so, these functions are implemented as OSS or ALSA PCM devices and eventually OSS or ALSA audio mixer. The V4L2 API makes no provisions yet to find these related devices. If you have an idea please write to the linux-media mailing list: http://www.linuxtv.org/lists.php.

In general, V4L2 devices can be opened more than once. When this is supported by the driver, users can for example start a "panel" application to change controls like brightness or audio volume, while another application captures video and audio. In other words, panel applications are comparable to an OSS or ALSA audio mixer application. When a device supports multiple functions like capturing and overlay simultaneously, multiple opens allow concurrent use of the device by forked processes or specialized applications.

Multiple opens are optional, although drivers should permit at least concurrent accesses without data exchange, i. e. panel applications. This implies open() can return an EBUSY error code when the device is already in use, as well as ioctl() functions initiating data exchange (namely the VIDIOC_S_FMT ioctl), and the read() and write() functions.

Mere opening a V4L2 device does not grant exclusive access.^[4] Initiating data exchange however assigns the right to read or write the requested type of data, and to change related properties, to this file descriptor. Applications can request additional access privileges using the priority mechanism described in the section called “Application Priority”.

V4L2 drivers should not support multiple applications reading or writing the same data stream on a device by copying buffers, time multiplexing or similar means. This is better handled by a proxy application in user space. When the driver supports stream sharing anyway it must be implemented transparently. The V4L2 API does not specify how conflicts are solved.

To open and close V4L2 devices applications use the open() and close() function, respectively. Devices are programmed using the ioctl() function as explained in the following sections.

Video input devices can have one or more tuners demodulating a RF signal. Each tuner is associated with one or more video inputs, depending on the number of RF connectors on the tuner. The type field of the respective struct v4l2_input returned by the VIDIOC_ENUMINPUT ioctl is set to V4L2_INPUT_TYPE_TUNER and its tuner field contains the index number of the tuner.

Radio devices have exactly one tuner with index zero, no video inputs.

To query and change tuner properties applications use the VIDIOC_G_TUNER and VIDIOC_S_TUNER ioctl, respectively. The struct v4l2_tuner returned by VIDIOC_G_TUNER also contains signal status information applicable when the tuner of the current video input, or a radio tuner is queried. Note that VIDIOC_S_TUNER does not switch the current tuner, when there is more than one at all. The tuner is solely determined by the current video input. Drivers must support both ioctls and set the V4L2_CAP_TUNER flag in the struct v4l2_capability returned by the VIDIOC_QUERYCAP ioctl when the device has one or more tuners.

Video output devices can have one or more modulators, uh, modulating a video signal for radiation or connection to the antenna input of a TV set or video recorder. Each modulator is associated with one or more video outputs, depending on the number of RF connectors on the modulator. The type field of the respective struct v4l2_output returned by the VIDIOC_ENUMOUTPUT ioctl is set to V4L2_OUTPUT_TYPE_MODULATOR and its modulator field contains the index number of the modulator. This specification does not define radio output devices.

To query and change modulator properties applications use the VIDIOC_G_MODULATOR and VIDIOC_S_MODULATOR ioctl. Note that VIDIOC_S_MODULATOR does not switch the current modulator, when there is more than one at all. The modulator is solely determined by the current video output. Drivers must support both ioctls and set the V4L2_CAP_MODULATOR flag in the struct v4l2_capability returned by the VIDIOC_QUERYCAP ioctl when the device has one or more modulators.

To get and set the tuner or modulator radio frequency applications use the VIDIOC_G_FREQUENCY and VIDIOC_S_FREQUENCY ioctl which both take a pointer to a struct v4l2_frequency. These ioctls are used for TV and radio devices alike. Drivers must support both ioctls when the tuner or modulator ioctls are supported, or when the device is a radio device.

The video standards discussed so far has been dealing with Analog TV and the corresponding video timings. Today there are many more different hardware interfaces such as High Definition TV interfaces (HDMI), VGA, DVI connectors etc., that carry video signals and there is a need to extend the API to select the video timings for these interfaces. Since it is not possible to extend the v4l2_std_id due to the limited bits available, a new set of IOCTLs is added to set/get video timings at the input and output:

To enumerate and query the attributes of DV presets supported by a device, applications use the VIDIOC_ENUM_DV_PRESETS ioctl. To get the current DV preset, applications use the VIDIOC_G_DV_PRESET ioctl and to set a preset they use the VIDIOC_S_DV_PRESET ioctl.

To set custom DV timings for the device, applications use the VIDIOC_S_DV_TIMINGS ioctl and to get current custom DV timings they use the VIDIOC_G_DV_TIMINGS ioctl.

Applications can make use of the Table A.28, “Input capabilities” and Table A.31, “Output capabilities” flags to decide what ioctls are available to set the video timings for the device.

The control mechanism as originally designed was meant to be used for user settings (brightness, saturation, etc). However, it turned out to be a very useful model for implementing more complicated driver APIs where each driver implements only a subset of a larger API.

The MPEG encoding API was the driving force behind designing and implementing this extended control mechanism: the MPEG standard is quite large and the currently supported hardware MPEG encoders each only implement a subset of this standard. Further more, many parameters relating to how the video is encoded into an MPEG stream are specific to the MPEG encoding chip since the MPEG standard only defines the format of the resulting MPEG stream, not how the video is actually encoded into that format.

Unfortunately, the original control API lacked some features needed for these new uses and so it was extended into the (not terribly originally named) extended control API.

Even though the MPEG encoding API was the first effort to use the Extended Control API, nowadays there are also other classes of Extended Controls, such as Camera Controls and FM Transmitter Controls. The Extended Controls API as well as all Extended Controls classes are described in the following text.

Three new ioctls are available: VIDIOC_G_EXT_CTRLS, VIDIOC_S_EXT_CTRLS and VIDIOC_TRY_EXT_CTRLS. These ioctls act on arrays of controls (as opposed to the VIDIOC_G_CTRL and VIDIOC_S_CTRL ioctls that act on a single control). This is needed since it is often required to atomically change several controls at once.

Each of the new ioctls expects a pointer to a struct v4l2_ext_controls. This structure contains a pointer to the control array, a count of the number of controls in that array and a control class. Control classes are used to group similar controls into a single class. For example, control class V4L2_CTRL_CLASS_USER contains all user controls (i. e. all controls that can also be set using the old VIDIOC_S_CTRL ioctl). Control class V4L2_CTRL_CLASS_MPEG contains all controls relating to MPEG encoding, etc.

All controls in the control array must belong to the specified control class. An error is returned if this is not the case.

It is also possible to use an empty control array (count == 0) to check whether the specified control class is supported.

The control array is a struct v4l2_ext_control array. The v4l2_ext_control structure is very similar to struct v4l2_control, except for the fact that it also allows for 64-bit values and pointers to be passed.

It is important to realize that due to the flexibility of controls it is necessary to check whether the control you want to set actually is supported in the driver and what the valid range of values is. So use the VIDIOC_QUERYCTRL and VIDIOC_QUERYMENU ioctls to check this. Also note that it is possible that some of the menu indices in a control of type V4L2_CTRL_TYPE_MENU may not be supported (VIDIOC_QUERYMENU will return an error). A good example is the list of supported MPEG audio bitrates. Some drivers only support one or two bitrates, others support a wider range.

The recommended way to enumerate over the extended controls is by using VIDIOC_QUERYCTRL in combination with the V4L2_CTRL_FLAG_NEXT_CTRL flag:

struct v4l2_queryctrl qctrl;

qctrl.id = V4L2_CTRL_FLAG_NEXT_CTRL;
while (0 == ioctl (fd, VIDIOC_QUERYCTRL, &qctrl)) {
	/* ... */
	qctrl.id |= V4L2_CTRL_FLAG_NEXT_CTRL;
}

The initial control ID is set to 0 ORed with the V4L2_CTRL_FLAG_NEXT_CTRL flag. The VIDIOC_QUERYCTRL ioctl will return the first control with a higher ID than the specified one. When no such controls are found an error is returned.

If you want to get all controls within a specific control class, then you can set the initial qctrl.id value to the control class and add an extra check to break out of the loop when a control of another control class is found:

qctrl.id = V4L2_CTRL_CLASS_MPEG | V4L2_CTRL_FLAG_NEXT_CTRL;
while (0 == ioctl (fd, VIDIOC_QUERYCTRL, &qctrl)) {
	if (V4L2_CTRL_ID2CLASS (qctrl.id) != V4L2_CTRL_CLASS_MPEG)
		break;
		/* ... */
		qctrl.id |= V4L2_CTRL_FLAG_NEXT_CTRL;
	}

The 32-bit qctrl.id value is subdivided into three bit ranges: the top 4 bits are reserved for flags (e. g. V4L2_CTRL_FLAG_NEXT_CTRL) and are not actually part of the ID. The remaining 28 bits form the control ID, of which the most significant 12 bits define the control class and the least significant 16 bits identify the control within the control class. It is guaranteed that these last 16 bits are always non-zero for controls. The range of 0x1000 and up are reserved for driver-specific controls. The macro V4L2_CTRL_ID2CLASS(id) returns the control class ID based on a control ID.

If the driver does not support extended controls, then VIDIOC_QUERYCTRL will fail when used in combination with V4L2_CTRL_FLAG_NEXT_CTRL. In that case the old method of enumerating control should be used (see 1.8). But if it is supported, then it is guaranteed to enumerate over all controls, including driver-private controls.

It is possible to create control panels for a graphical user interface where the user can select the various controls. Basically you will have to iterate over all controls using the method described above. Each control class starts with a control of type V4L2_CTRL_TYPE_CTRL_CLASS. VIDIOC_QUERYCTRL will return the name of this control class which can be used as the title of a tab page within a control panel.

The flags field of struct v4l2_queryctrl also contains hints on the behavior of the control. See the VIDIOC_QUERYCTRL documentation for more details.

Below all controls within the MPEG control class are described. First the generic controls, then controls specific for certain hardware.

The Camera class includes controls for mechanical (or equivalent digital) features of a device such as controllable lenses or sensors.

The FM Transmitter (FM_TX) class includes controls for common features of FM transmissions capable devices. Currently this class includes parameters for audio compression, pilot tone generation, audio deviation limiter, RDS transmission and tuning power features.

For more details about RDS specification, refer to [EN 50067] document, from CENELEC.

Different devices exchange different kinds of data with applications, for example video images, raw or sliced VBI data, RDS datagrams. Even within one kind many different formats are possible, in particular an abundance of image formats. Although drivers must provide a default and the selection persists across closing and reopening a device, applications should always negotiate a data format before engaging in data exchange. Negotiation means the application asks for a particular format and the driver selects and reports the best the hardware can do to satisfy the request. Of course applications can also just query the current selection.

A single mechanism exists to negotiate all data formats using the aggregate struct v4l2_format and the VIDIOC_G_FMT and VIDIOC_S_FMT ioctls. Additionally the VIDIOC_TRY_FMT ioctl can be used to examine what the hardware could do, without actually selecting a new data format. The data formats supported by the V4L2 API are covered in the respective device section in Chapter 4, Interfaces. For a closer look at image formats see Chapter 2, Image Formats.

The VIDIOC_S_FMT ioctl is a major turning-point in the initialization sequence. Prior to this point multiple panel applications can access the same device concurrently to select the current input, change controls or modify other properties. The first VIDIOC_S_FMT assigns a logical stream (video data, VBI data etc.) exclusively to one file descriptor.

Exclusive means no other application, more precisely no other file descriptor, can grab this stream or change device properties inconsistent with the negotiated parameters. A video standard change for example, when the new standard uses a different number of scan lines, can invalidate the selected image format. Therefore only the file descriptor owning the stream can make invalidating changes. Accordingly multiple file descriptors which grabbed different logical streams prevent each other from interfering with their settings. When for example video overlay is about to start or already in progress, simultaneous video capturing may be restricted to the same cropping and image size.

When applications omit the VIDIOC_S_FMT ioctl its locking side effects are implied by the next step, the selection of an I/O method with the VIDIOC_REQBUFS ioctl or implicit with the first read() or write() call.

Generally only one logical stream can be assigned to a file descriptor, the exception being drivers permitting simultaneous video capturing and overlay using the same file descriptor for compatibility with V4L and earlier versions of V4L2. Switching the logical stream or returning into "panel mode" is possible by closing and reopening the device. Drivers may support a switch using VIDIOC_S_FMT.

All drivers exchanging data with applications must support the VIDIOC_G_FMT and VIDIOC_S_FMT ioctl. Implementation of the VIDIOC_TRY_FMT is highly recommended but optional.

Apart of the generic format negotiation functions a special ioctl to enumerate all image formats supported by video capture, overlay or output devices is available.^[11]

The VIDIOC_ENUM_FMT ioctl must be supported by all drivers exchanging image data with applications.

Multi-planar API introduces new multi-planar formats. Those formats use a separate set of FourCC codes. It is important to distinguish between the multi-planar API and a multi-planar format. Multi-planar API calls can handle all single-planar formats as well (as long as they are passed in multi-planar API structures), while the single-planar API cannot handle multi-planar formats.

Figure 1.1. Image Cropping, Insertion and Scaling

For capture devices the coordinates of the top left corner, width and height of the area which can be sampled is given by the bounds substructure of the struct v4l2_cropcap returned by the VIDIOC_CROPCAP ioctl. To support a wide range of hardware this specification does not define an origin or units. However by convention drivers should horizontally count unscaled samples relative to 0H (the leading edge of the horizontal sync pulse, see Figure 4.1, “Line synchronization”). Vertically ITU-R line numbers of the first field (Figure 4.2, “ITU-R 525 line numbering (M/NTSC and M/PAL)”, Figure 4.3, “ITU-R 625 line numbering”), multiplied by two if the driver can capture both fields.

The top left corner, width and height of the source rectangle, that is the area actually sampled, is given by struct v4l2_crop using the same coordinate system as struct v4l2_cropcap. Applications can use the VIDIOC_G_CROP and VIDIOC_S_CROP ioctls to get and set this rectangle. It must lie completely within the capture boundaries and the driver may further adjust the requested size and/or position according to hardware limitations.

Each capture device has a default source rectangle, given by the defrect substructure of struct v4l2_cropcap. The center of this rectangle shall align with the center of the active picture area of the video signal, and cover what the driver writer considers the complete picture. Drivers shall reset the source rectangle to the default when the driver is first loaded, but not later.

For output devices these structures and ioctls are used accordingly, defining the target rectangle where the images will be inserted into the video signal.

Video hardware can have various cropping, insertion and scaling limitations. It may only scale up or down, support only discrete scaling factors, or have different scaling abilities in horizontal and vertical direction. Also it may not support scaling at all. At the same time the struct v4l2_crop rectangle may have to be aligned, and both the source and target rectangles may have arbitrary upper and lower size limits. In particular the maximum width and height in struct v4l2_crop may be smaller than the struct v4l2_cropcap.bounds area. Therefore, as usual, drivers are expected to adjust the requested parameters and return the actual values selected.

Applications can change the source or the target rectangle first, as they may prefer a particular image size or a certain area in the video signal. If the driver has to adjust both to satisfy hardware limitations, the last requested rectangle shall take priority, and the driver should preferably adjust the opposite one. The VIDIOC_TRY_FMT ioctl however shall not change the driver state and therefore only adjust the requested rectangle.

Suppose scaling on a video capture device is restricted to a factor 1:1 or 2:1 in either direction and the target image size must be a multiple of 16 × 16 pixels. The source cropping rectangle is set to defaults, which are also the upper limit in this example, of 640 × 400 pixels at offset 0, 0. An application requests an image size of 300 × 225 pixels, assuming video will be scaled down from the "full picture" accordingly. The driver sets the image size to the closest possible values 304 × 224, then chooses the cropping rectangle closest to the requested size, that is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is still valid, thus unmodified. Given the default cropping rectangle reported by VIDIOC_CROPCAP the application can easily propose another offset to center the cropping rectangle.

Now the application may insist on covering an area using a picture aspect ratio closer to the original request, so it asks for a cropping rectangle of 608 × 456 pixels. The present scaling factors limit cropping to 640 × 384, so the driver returns the cropping size 608 × 384 and adjusts the image size to closest possible 304 × 192.

Source and target rectangles shall remain unchanged across closing and reopening a device, such that piping data into or out of a device will work without special preparations. More advanced applications should ensure the parameters are suitable before starting I/O.

struct v4l2_cropcap cropcap;
struct v4l2_crop crop;

memset (&cropcap, 0, sizeof (cropcap));
cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
	perror ("VIDIOC_CROPCAP");
	exit (EXIT_FAILURE);
}

memset (&crop, 0, sizeof (crop));
crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
crop.c = cropcap.defrect;

/* Ignore if cropping is not supported (EINVAL). */

if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
    && errno != EINVAL) {
	perror ("VIDIOC_S_CROP");
	exit (EXIT_FAILURE);
}
      

struct v4l2_cropcap cropcap;
struct v4l2_format format;

reset_cropping_parameters ();

/* Scale down to 1/4 size of full picture. */

memset (&format, 0, sizeof (format)); /* defaults */

format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

format.fmt.pix.width = cropcap.defrect.width >> 1;
format.fmt.pix.height = cropcap.defrect.height >> 1;
format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;

if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) {
	perror ("VIDIOC_S_FORMAT");
	exit (EXIT_FAILURE);
}

/* We could check the actual image size now, the actual scaling factor
   or if the driver can scale at all. */
	

struct v4l2_cropcap cropcap;
struct v4l2_crop crop;

memset (&cropcap, 0, sizeof (cropcap));
cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;

if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) {
	perror ("VIDIOC_CROPCAP");
	exit (EXIT_FAILURE);
}

memset (&crop, 0, sizeof (crop));

crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
crop.c = cropcap.defrect;

/* Scale the width and height to 50 % of their original size
   and center the output. */

crop.c.width /= 2;
crop.c.height /= 2;
crop.c.left += crop.c.width / 2;
crop.c.top += crop.c.height / 2;

/* Ignore if cropping is not supported (EINVAL). */

if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
    && errno != EINVAL) {
	perror ("VIDIOC_S_CROP");
	exit (EXIT_FAILURE);
}

struct v4l2_cropcap cropcap;
struct v4l2_crop crop;
struct v4l2_format format;
double hscale, vscale;
double aspect;
int dwidth, dheight;

memset (&cropcap, 0, sizeof (cropcap));
cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
	perror ("VIDIOC_CROPCAP");
	exit (EXIT_FAILURE);
}

memset (&crop, 0, sizeof (crop));
crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) {
	if (errno != EINVAL) {
		perror ("VIDIOC_G_CROP");
		exit (EXIT_FAILURE);
	}

	/* Cropping not supported. */
	crop.c = cropcap.defrect;
}

memset (&format, 0, sizeof (format));
format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) {
	perror ("VIDIOC_G_FMT");
	exit (EXIT_FAILURE);
}

/* The scaling applied by the driver. */

hscale = format.fmt.pix.width / (double) crop.c.width;
vscale = format.fmt.pix.height / (double) crop.c.height;

aspect = cropcap.pixelaspect.numerator /
	 (double) cropcap.pixelaspect.denominator;
aspect = aspect * hscale / vscale;

/* Devices following ITU-R BT.601 do not capture
   square pixels. For playback on a computer monitor
   we should scale the images to this size. */

dwidth = format.fmt.pix.width / aspect;
dheight = format.fmt.pix.height;