2.4. Video device’ s internal representation¶
The actual device nodes in the /dev
directory are created using the
video_device
struct (v4l2-dev.h
). This struct can either be
allocated dynamically or embedded in a larger struct.
To allocate it dynamically use video_device_alloc()
:
struct video_device *vdev = video_device_alloc();
if (vdev == NULL)
return -ENOMEM;
vdev->release = video_device_release;
If you embed it in a larger struct, then you must set the release()
callback to your own function:
struct video_device *vdev = &my_vdev->vdev;
vdev->release = my_vdev_release;
The release()
callback must be set and it is called when the last user
of the video device exits.
The default video_device_release()
callback currently
just calls kfree
to free the allocated memory.
There is also a video_device_release_empty()
function that does
nothing (is empty) and should be used if the struct is embedded and there
is nothing to do when it is released.
You should also set these fields of video_device
:
video_device
->v4l2_dev: must be set to thev4l2_device
parent device.video_device
->name: set to something descriptive and unique.video_device
->vfl_dir: set this toVFL_DIR_RX
for capture devices (VFL_DIR_RX
has value 0, so this is normally already the default), set toVFL_DIR_TX
for output devices andVFL_DIR_M2M
for mem2mem (codec) devices.video_device
->fops: set to thev4l2_file_operations
struct.video_device
->ioctl_ops: if you use thev4l2_ioctl_ops
to simplify ioctl maintenance (highly recommended to use this and it might become compulsory in the future!), then set this to yourv4l2_ioctl_ops
struct. Thevideo_device
->vfl_type andvideo_device
->vfl_dir fields are used to disable ops that do not match the type/dir combination. E.g. VBI ops are disabled for non-VBI nodes, and output ops are disabled for a capture device. This makes it possible to provide just onev4l2_ioctl_ops
struct for both vbi and video nodes.video_device
->lock: leave toNULL
if you want to do all the locking in the driver. Otherwise you give it a pointer to a structmutex_lock
and before thevideo_device
->unlocked_ioctl file operation is called this lock will be taken by the core and released afterwards. See the next section for more details.video_device
->queue: a pointer to thestruct vb2_queue
associated with this device node. If queue is notNULL
, and queue->lock is notNULL
, then queue->lock is used for the queuing ioctls (VIDIOC_REQBUFS
,CREATE_BUFS
,QBUF
,DQBUF
,QUERYBUF
,PREPARE_BUF
,STREAMON
andSTREAMOFF
) instead of the lock above. That way the vb2 queuing framework does not have to wait for other ioctls. This queue pointer is also used by the vb2 helper functions to check for queuing ownership (i.e. is the filehandle calling it allowed to do the operation).video_device
->prio: keeps track of the priorities. Used to implementVIDIOC_G_PRIORITY
andVIDIOC_S_PRIORITY
. If left toNULL
, then it will use thestruct v4l2_prio_state
inv4l2_device
. If you want to have a separate priority state per (group of) device node(s), then you can point it to your own structv4l2_prio_state
.video_device
->dev_parent: you only set this if v4l2_device was registered withNULL
as the parentdevice
struct. This only happens in cases where one hardware device has multiple PCI devices that all share the samev4l2_device
core.The cx88 driver is an example of this: one core
v4l2_device
struct, but it is used by both a raw video PCI device (cx8800) and a MPEG PCI device (cx8802). Since thev4l2_device
cannot be associated with two PCI devices at the same time it is setup without a parent device. But when thestruct video_device
is initialized you do know which parent PCI device to use and so you setdev_device
to the correct PCI device.
If you use v4l2_ioctl_ops
, then you should set
video_device
->unlocked_ioctl to video_ioctl2()
in your
v4l2_file_operations
struct.
In some cases you want to tell the core that a function you had specified in
your v4l2_ioctl_ops
should be ignored. You can mark such ioctls by
calling this function before video_register_device()
is called:
v4l2_disable_ioctl
(vdev
, cmd).
This tends to be needed if based on external factors (e.g. which card is
being used) you want to turns off certain features in v4l2_ioctl_ops
without having to make a new struct.
The v4l2_file_operations
struct is a subset of file_operations.
The main difference is that the inode argument is omitted since it is never
used.
If integration with the media framework is needed, you must initialize the
media_entity
struct embedded in the video_device
struct
(entity field) by calling media_entity_pads_init()
:
struct media_pad *pad = &my_vdev->pad;
int err;
err = media_entity_pads_init(&vdev->entity, 1, pad);
The pads array must have been previously initialized. There is no need to
manually set the struct media_entity
type and name fields.
A reference to the entity will be automatically acquired/released when the video device is opened/closed.
2.4.1. ioctls and locking¶
The V4L core provides optional locking services. The main service is the
lock field in struct video_device
, which is a pointer to a mutex.
If you set this pointer, then that will be used by unlocked_ioctl to
serialize all ioctls.
If you are using the videobuf2 framework, then there
is a second lock that you can set: video_device
->queue->lock. If
set, then this lock will be used instead of video_device
->lock
to serialize all queuing ioctls (see the previous section
for the full list of those ioctls).
The advantage of using a different lock for the queuing ioctls is that for some
drivers (particularly USB drivers) certain commands such as setting controls
can take a long time, so you want to use a separate lock for the buffer queuing
ioctls. That way your VIDIOC_DQBUF
doesn’t stall because the driver is busy
changing the e.g. exposure of the webcam.
Of course, you can always do all the locking yourself by leaving both lock
pointers at NULL
.
If you use the old videobuf framework then you must
pass the video_device
->lock to the videobuf queue initialize
function: if videobuf has to wait for a frame to arrive, then it will
temporarily unlock the lock and relock it afterwards. If your driver also
waits in the code, then you should do the same to allow other
processes to access the device node while the first process is waiting for
something.
In the case of videobuf2 you will need to implement the
wait_prepare()
and wait_finish()
callbacks to unlock/lock if applicable.
If you use the queue->lock
pointer, then you can use the helper functions
vb2_ops_wait_prepare()
and vb2_ops_wait_finish()
.
The implementation of a hotplug disconnect should also take the lock from
video_device
before calling v4l2_device_disconnect. If you are also
using video_device
->queue->lock, then you have to first lock
video_device
->queue->lock followed by video_device
->lock.
That way you can be sure no ioctl is running when you call
v4l2_device_disconnect()
.
2.4.2. Video device registration¶
Next you register the video device with video_register_device()
.
This will create the character device for you.
err = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (err) {
video_device_release(vdev); /* or kfree(my_vdev); */
return err;
}
If the v4l2_device
parent device has a not NULL
mdev field,
the video device entity will be automatically registered with the media
device.
Which device is registered depends on the type argument. The following types exist:
Device name |
Usage |
|
---|---|---|
|
|
for video input/output devices |
|
|
for vertical blank data (i.e. closed captions, teletext) |
|
|
for radio tuners |
|
|
for V4L2 subdevices |
|
|
for Software Defined Radio (SDR) tuners |
|
|
for touch sensors |
The last argument gives you a certain amount of control over the device
node number used (i.e. the X in videoX
). Normally you will pass -1
to let the v4l2 framework pick the first free number. But sometimes users
want to select a specific node number. It is common that drivers allow
the user to select a specific device node number through a driver module
option. That number is then passed to this function and video_register_device
will attempt to select that device node number. If that number was already
in use, then the next free device node number will be selected and it
will send a warning to the kernel log.
Another use-case is if a driver creates many devices. In that case it can be useful to place different video devices in separate ranges. For example, video capture devices start at 0, video output devices start at 16. So you can use the last argument to specify a minimum device node number and the v4l2 framework will try to pick the first free number that is equal or higher to what you passed. If that fails, then it will just pick the first free number.
Since in this case you do not care about a warning about not being able
to select the specified device node number, you can call the function
video_register_device_no_warn()
instead.
Whenever a device node is created some attributes are also created for you.
If you look in /sys/class/video4linux
you see the devices. Go into e.g.
video0
and you will see ‘name’, ‘dev_debug’ and ‘index’ attributes. The
‘name’ attribute is the ‘name’ field of the video_device struct. The
‘dev_debug’ attribute can be used to enable core debugging. See the next
section for more detailed information on this.
The ‘index’ attribute is the index of the device node: for each call to
video_register_device()
the index is just increased by 1. The
first video device node you register always starts with index 0.
Users can setup udev rules that utilize the index attribute to make fancy
device names (e.g. ‘mpegX
’ for MPEG video capture device nodes).
After the device was successfully registered, then you can use these fields:
video_device
->vfl_type: the device type passed tovideo_register_device()
.video_device
->minor: the assigned device minor number.video_device
->num: the device node number (i.e. the X invideoX
).video_device
->index: the device index number.
If the registration failed, then you need to call
video_device_release()
to free the allocated video_device
struct, or free your own struct if the video_device
was embedded in
it. The vdev->release()
callback will never be called if the registration
failed, nor should you ever attempt to unregister the device if the
registration failed.
2.4.3. video device debugging¶
The ‘dev_debug’ attribute that is created for each video, vbi, radio or swradio
device in /sys/class/video4linux/<devX>/
allows you to enable logging of
file operations.
It is a bitmask and the following bits can be set:
Mask |
Description |
---|---|
0x01 |
Log the ioctl name and error code. VIDIOC_(D)QBUF ioctls are only logged if bit 0x08 is also set. |
0x02 |
Log the ioctl name arguments and error code. VIDIOC_(D)QBUF ioctls are only logged if bit 0x08 is also set. |
0x04 |
Log the file operations open, release, read, write, mmap and get_unmapped_area. The read and write operations are only logged if bit 0x08 is also set. |
0x08 |
Log the read and write file operations and the VIDIOC_QBUF and VIDIOC_DQBUF ioctls. |
0x10 |
Log the poll file operation. |
0x20 |
Log error and messages in the control operations. |
2.4.4. Video device cleanup¶
When the video device nodes have to be removed, either during the unload of the driver or because the USB device was disconnected, then you should unregister them with:
This will remove the device nodes from sysfs (causing udev to remove them
from /dev
).
After video_unregister_device()
returns no new opens can be done.
However, in the case of USB devices some application might still have one of
these device nodes open. So after the unregister all file operations (except
release, of course) will return an error as well.
When the last user of the video device node exits, then the vdev->release()
callback is called and you can do the final cleanup there.
Don’t forget to cleanup the media entity associated with the video device if it has been initialized:
media_entity_cleanup
(&vdev->entity);
This can be done from the release callback.
2.4.5. helper functions¶
There are a few useful helper functions:
file and
video_device
private data
You can set/get driver private data in the video_device struct using:
Note that you can safely call video_set_drvdata()
before calling
video_register_device()
.
And this function:
video_devdata
(struct file *file);
returns the video_device belonging to the file struct.
The video_devdata()
function combines video_get_drvdata()
with video_devdata()
:
video_drvdata
(struct file *file);
You can go from a video_device
struct to the v4l2_device struct using:
struct v4l2_device *v4l2_dev = vdev->v4l2_dev;
Device node name
The video_device
node kernel name can be retrieved using:
The name is used as a hint by userspace tools such as udev. The function should be used where possible instead of accessing the video_device::num and video_device::minor fields.
2.4.6. video_device functions and data structures¶
-
enum
vfl_devnode_type
¶ type of V4L2 device node
Constants
VFL_TYPE_VIDEO
for video input/output devices
VFL_TYPE_VBI
for vertical blank data (i.e. closed captions, teletext)
VFL_TYPE_RADIO
for radio tuners
VFL_TYPE_SUBDEV
for V4L2 subdevices
VFL_TYPE_SDR
for Software Defined Radio tuners
VFL_TYPE_TOUCH
for touch sensors
VFL_TYPE_MAX
number of VFL types, must always be last in the enum
-
enum
vfl_devnode_direction
¶ Identifies if a
struct video_device
corresponds to a receiver, a transmitter or a mem-to-mem device.
Constants
VFL_DIR_RX
device is a receiver.
VFL_DIR_TX
device is a transmitter.
VFL_DIR_M2M
device is a memory to memory device.
Note
Ignored if enum vfl_devnode_type
is VFL_TYPE_SUBDEV
.
-
enum
v4l2_video_device_flags
¶ Flags used by
struct video_device
Constants
V4L2_FL_REGISTERED
indicates that a
struct video_device
is registered. Drivers can clear this flag if they want to block all future device access. It is cleared by video_unregister_device.
V4L2_FL_USES_V4L2_FH
indicates that file->private_data points to
struct v4l2_fh
. This flag is set by the core whenv4l2_fh_init()
is called. All new drivers should use it.
V4L2_FL_QUIRK_INVERTED_CROP
some old M2M drivers use g/s_crop/cropcap incorrectly: crop and compose are swapped. If this flag is set, then the selection targets are swapped in the g/s_crop/cropcap functions in v4l2-ioctl.c. This allows those drivers to correctly implement the selection API, but the old crop API will still work as expected in order to preserve backwards compatibility. Never set this flag for new drivers.
V4L2_FL_SUBDEV_RO_DEVNODE
indicates that the video device node is registered in read-only mode. The flag only applies to device nodes registered for sub-devices, it is set by the core when the sub-devices device nodes are registered with
v4l2_device_register_ro_subdev_nodes()
and used by the sub-device ioctl handler to restrict access to some ioctl calls.
-
struct
v4l2_prio_state
¶ stores the priority states
Definition
struct v4l2_prio_state {
atomic_t prios[4];
};
Members
prios
array with elements to store the array priorities
Description
Note
The size of prios array matches the number of priority types defined
by enum v4l2_priority
.
-
void
v4l2_prio_init
(struct v4l2_prio_state *global)¶ initializes a
struct v4l2_prio_state
Parameters
struct v4l2_prio_state *global
pointer to
struct v4l2_prio_state
-
int
v4l2_prio_change
(struct v4l2_prio_state *global, enum v4l2_priority *local, enum v4l2_priority new)¶ changes the v4l2 file handler priority
Parameters
struct v4l2_prio_state *global
pointer to the
struct v4l2_prio_state
of the device node.enum v4l2_priority *local
pointer to the desired priority, as defined by enum
v4l2_priority
enum v4l2_priority new
Priority type requested, as defined by enum
v4l2_priority
.
Description
Note
This function should be used only by the V4L2 core.
-
void
v4l2_prio_open
(struct v4l2_prio_state *global, enum v4l2_priority *local)¶ Implements the priority logic for a file handler open
Parameters
struct v4l2_prio_state *global
pointer to the
struct v4l2_prio_state
of the device node.enum v4l2_priority *local
pointer to the desired priority, as defined by enum
v4l2_priority
Description
Note
This function should be used only by the V4L2 core.
-
void
v4l2_prio_close
(struct v4l2_prio_state *global, enum v4l2_priority local)¶ Implements the priority logic for a file handler close
Parameters
struct v4l2_prio_state *global
pointer to the
struct v4l2_prio_state
of the device node.enum v4l2_priority local
priority to be released, as defined by enum
v4l2_priority
Description
Note
This function should be used only by the V4L2 core.
-
enum v4l2_priority
v4l2_prio_max
(struct v4l2_prio_state *global)¶ Return the maximum priority, as stored at the global array.
Parameters
struct v4l2_prio_state *global
pointer to the
struct v4l2_prio_state
of the device node.
Description
Note
This function should be used only by the V4L2 core.
-
int
v4l2_prio_check
(struct v4l2_prio_state *global, enum v4l2_priority local)¶ Implements the priority logic for a file handler close
Parameters
struct v4l2_prio_state *global
pointer to the
struct v4l2_prio_state
of the device node.enum v4l2_priority local
desired priority, as defined by enum
v4l2_priority
local
Description
Note
This function should be used only by the V4L2 core.
-
struct
v4l2_file_operations
¶ fs operations used by a V4L2 device
Definition
struct v4l2_file_operations {
struct module *owner;
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
__poll_t (*poll) (struct file *, struct poll_table_struct *);
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
#ifdef CONFIG_COMPAT;
long (*compat_ioctl32) (struct file *, unsigned int, unsigned long);
#endif;
unsigned long (*get_unmapped_area) (struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*open) (struct file *);
int (*release) (struct file *);
};
Members
owner
pointer to struct module
read
operations needed to implement the read() syscall
write
operations needed to implement the write() syscall
poll
operations needed to implement the poll() syscall
unlocked_ioctl
operations needed to implement the ioctl() syscall
compat_ioctl32
operations needed to implement the ioctl() syscall for the special case where the Kernel uses 64 bits instructions, but the userspace uses 32 bits.
get_unmapped_area
called by the mmap() syscall, used when %!CONFIG_MMU
mmap
operations needed to implement the mmap() syscall
open
operations needed to implement the open() syscall
release
operations needed to implement the release() syscall
Description
Note
Those operations are used to implemente the fs struct file_operations at the V4L2 drivers. The V4L2 core overrides the fs ops with some extra logic needed by the subsystem.
-
struct
video_device
¶ Structure used to create and manage the V4L2 device nodes.
Definition
struct video_device {
#if defined(CONFIG_MEDIA_CONTROLLER);
struct media_entity entity;
struct media_intf_devnode *intf_devnode;
struct media_pipeline pipe;
#endif;
const struct v4l2_file_operations *fops;
u32 device_caps;
struct device dev;
struct cdev *cdev;
struct v4l2_device *v4l2_dev;
struct device *dev_parent;
struct v4l2_ctrl_handler *ctrl_handler;
struct vb2_queue *queue;
struct v4l2_prio_state *prio;
char name[32];
enum vfl_devnode_type vfl_type;
enum vfl_devnode_direction vfl_dir;
int minor;
u16 num;
unsigned long flags;
int index;
spinlock_t fh_lock;
struct list_head fh_list;
int dev_debug;
v4l2_std_id tvnorms;
void (*release)(struct video_device *vdev);
const struct v4l2_ioctl_ops *ioctl_ops;
unsigned long valid_ioctls[BITS_TO_LONGS(BASE_VIDIOC_PRIVATE)];
struct mutex *lock;
};
Members
entity
intf_devnode
pointer to
struct media_intf_devnode
pipe
fops
pointer to
struct v4l2_file_operations
for the video devicedevice_caps
device capabilities as used in v4l2_capabilities
dev
struct device
for the video devicecdev
character device
v4l2_dev
pointer to
struct v4l2_device
parentdev_parent
pointer to
struct device
parentctrl_handler
Control handler associated with this device node. May be NULL.
queue
struct vb2_queue
associated with this device node. May be NULL.prio
pointer to
struct v4l2_prio_state
with device’s Priority state. If NULL, then v4l2_dev->prio will be used.name
video device name
vfl_type
V4L device type, as defined by
enum vfl_devnode_type
vfl_dir
V4L receiver, transmitter or m2m
minor
device node ‘minor’. It is set to -1 if the registration failed
num
number of the video device node
flags
video device flags. Use bitops to set/clear/test flags. Contains a set of
enum v4l2_video_device_flags
.index
attribute to differentiate multiple indices on one physical device
fh_lock
Lock for all v4l2_fhs
fh_list
List of
struct v4l2_fh
dev_debug
Internal device debug flags, not for use by drivers
tvnorms
Supported tv norms
release
video device release() callback
ioctl_ops
pointer to
struct v4l2_ioctl_ops
with ioctl callbacksvalid_ioctls
bitmap with the valid ioctls for this device
lock
pointer to
struct mutex
serialization lock
Description
Note
Only set dev_parent if that can’t be deduced from v4l2_dev.
-
media_entity_to_video_device
(__entity)¶ Returns a
struct video_device
from thestruct media_entity
embedded on it.
Parameters
__entity
pointer to
struct media_entity
-
to_video_device
(cd)¶ Returns a
struct video_device
from thestruct device
embedded on it.
Parameters
cd
pointer to
struct device
-
int
__video_register_device
(struct video_device *vdev, enum vfl_devnode_type type, int nr, int warn_if_nr_in_use, struct module *owner)¶ register video4linux devices
Parameters
struct video_device *vdev
struct video_device
to registerenum vfl_devnode_type type
type of device to register, as defined by
enum vfl_devnode_type
int nr
which device node number is desired: (0 == /dev/video0, 1 == /dev/video1, …, -1 == first free)
int warn_if_nr_in_use
warn if the desired device node number was already in use and another number was chosen instead.
struct module *owner
module that owns the video device node
Description
The registration code assigns minor numbers and device node numbers based on the requested type and registers the new device node with the kernel.
This function assumes that struct video_device
was zeroed when it
was allocated and does not contain any stale date.
An error is returned if no free minor or device node number could be found, or if the registration of the device node failed.
Returns 0 on success.
Note
This function is meant to be used only inside the V4L2 core.
Drivers should use video_register_device()
or
video_register_device_no_warn()
.
-
int
video_register_device
(struct video_device *vdev, enum vfl_devnode_type type, int nr)¶ register video4linux devices
Parameters
struct video_device *vdev
struct video_device
to registerenum vfl_devnode_type type
type of device to register, as defined by
enum vfl_devnode_type
int nr
which device node number is desired: (0 == /dev/video0, 1 == /dev/video1, …, -1 == first free)
Description
Internally, it calls __video_register_device()
. Please see its
documentation for more details.
Note
if video_register_device fails, the release() callback of
struct video_device
structure is not called, so the caller
is responsible for freeing any data. Usually that means that
you video_device_release()
should be called on failure.
-
int
video_register_device_no_warn
(struct video_device *vdev, enum vfl_devnode_type type, int nr)¶ register video4linux devices
Parameters
struct video_device *vdev
struct video_device
to registerenum vfl_devnode_type type
type of device to register, as defined by
enum vfl_devnode_type
int nr
which device node number is desired: (0 == /dev/video0, 1 == /dev/video1, …, -1 == first free)
Description
This function is identical to video_register_device()
except that no
warning is issued if the desired device node number was already in use.
Internally, it calls __video_register_device()
. Please see its
documentation for more details.
Note
if video_register_device fails, the release() callback of
struct video_device
structure is not called, so the caller
is responsible for freeing any data. Usually that means that
you video_device_release()
should be called on failure.
-
void
video_unregister_device
(struct video_device *vdev)¶ Unregister video devices.
Parameters
struct video_device *vdev
struct video_device
to register
Description
Does nothing if vdev == NULL or if video_is_registered()
returns false.
-
struct video_device *
video_device_alloc
(void)¶ helper function to alloc
struct video_device
Parameters
void
no arguments
Description
Returns NULL if -ENOMEM
or a struct video_device
on success.
-
void
video_device_release
(struct video_device *vdev)¶ helper function to release
struct video_device
Parameters
struct video_device *vdev
pointer to
struct video_device
Description
Can also be used for video_device->release().
-
void
video_device_release_empty
(struct video_device *vdev)¶ helper function to implement the video_device->release() callback.
Parameters
struct video_device *vdev
pointer to
struct video_device
Description
This release function does nothing.
It should be used when the video_device is a static global struct.
Note
Having a static video_device is a dubious construction at best.
-
void
v4l2_disable_ioctl
(struct video_device *vdev, unsigned int cmd)¶ mark that a given command isn’t implemented. shouldn’t use core locking
Parameters
struct video_device *vdev
pointer to
struct video_device
unsigned int cmd
ioctl command
Description
This function allows drivers to provide just one v4l2_ioctl_ops struct, but disable ioctls based on the specific card that is actually found.
Note
This must be called before video_register_device. See also the comments for determine_valid_ioctls().
-
void *
video_get_drvdata
(struct video_device *vdev)¶ gets private data from
struct video_device
.
Parameters
struct video_device *vdev
pointer to
struct video_device
Description
returns a pointer to the private data
-
void
video_set_drvdata
(struct video_device *vdev, void *data)¶ sets private data from
struct video_device
.
Parameters
struct video_device *vdev
pointer to
struct video_device
void *data
private data pointer
-
struct video_device *
video_devdata
(struct file *file)¶ gets
struct video_device
from struct file.
Parameters
struct file *file
pointer to struct file
-
void *
video_drvdata
(struct file *file)¶ gets private data from
struct video_device
using the struct file.
Parameters
struct file *file
pointer to struct file
Description
This is function combines both video_get_drvdata()
and video_devdata()
as this is used very often.
-
const char *
video_device_node_name
(struct video_device *vdev)¶ returns the video device name
Parameters
struct video_device *vdev
pointer to
struct video_device
Description
Returns the device name string
-
int
video_is_registered
(struct video_device *vdev)¶ returns true if the
struct video_device
is registered.
Parameters
struct video_device *vdev
pointer to
struct video_device
Description