4. Media Controller devices¶
4.1. Media Controller¶
The media controller userspace API is documented in the Media Controller uAPI book. This document focus on the kernel-side implementation of the media framework.
4.1.1. Abstract media device model¶
Discovering a device internal topology, and configuring it at runtime, is one of the goals of the media framework. To achieve this, hardware devices are modelled as an oriented graph of building blocks called entities connected through pads.
An entity is a basic media hardware building block. It can correspond to a large variety of logical blocks such as physical hardware devices (CMOS sensor for instance), logical hardware devices (a building block in a System-on-Chip image processing pipeline), DMA channels or physical connectors.
A pad is a connection endpoint through which an entity can interact with other entities. Data (not restricted to video) produced by an entity flows from the entity’s output to one or more entity inputs. Pads should not be confused with physical pins at chip boundaries.
A link is a point-to-point oriented connection between two pads, either on the same entity or on different entities. Data flows from a source pad to a sink pad.
4.1.2. Media device¶
A media device is represented by a struct media_device
instance, defined in include/media/media-device.h
.
Allocation of the structure is handled by the media device driver, usually by
embedding the media_device
instance in a larger driver-specific
structure.
Drivers register media device instances by calling
__media_device_register()
via the macro media_device_register()
and unregistered by calling media_device_unregister()
.
4.1.3. Entities¶
Entities are represented by a struct media_entity
instance, defined in include/media/media-entity.h
. The structure is usually
embedded into a higher-level structure, such as
v4l2_subdev
or video_device
instances, although drivers can allocate entities directly.
Drivers initialize entity pads by calling
media_entity_pads_init()
.
Drivers register entities with a media device by calling
media_device_register_entity()
and unregistred by calling
media_device_unregister_entity()
.
4.1.4. Interfaces¶
Interfaces are represented by a
struct media_interface
instance, defined in
include/media/media-entity.h
. Currently, only one type of interface is
defined: a device node. Such interfaces are represented by a
struct media_intf_devnode
.
Drivers initialize and create device node interfaces by calling
media_devnode_create()
and remove them by calling:
media_devnode_remove()
.
4.1.5. Pads¶
Pads are represented by a struct media_pad
instance,
defined in include/media/media-entity.h
. Each entity stores its pads in
a pads array managed by the entity driver. Drivers usually embed the array in
a driver-specific structure.
Pads are identified by their entity and their 0-based index in the pads array.
Both information are stored in the struct media_pad
,
making the struct media_pad
pointer the canonical way
to store and pass link references.
Pads have flags that describe the pad capabilities and state.
MEDIA_PAD_FL_SINK
indicates that the pad supports sinking data.
MEDIA_PAD_FL_SOURCE
indicates that the pad supports sourcing data.
Note
One and only one of MEDIA_PAD_FL_SINK
or MEDIA_PAD_FL_SOURCE
must
be set for each pad.
4.1.6. Links¶
Links are represented by a struct media_link
instance,
defined in include/media/media-entity.h
. There are two types of links:
1. pad to pad links:
Associate two entities via their PADs. Each entity has a list that points to all links originating at or targeting any of its pads. A given link is thus stored twice, once in the source entity and once in the target entity.
Drivers create pad to pad links by calling:
media_create_pad_link()
and remove with
media_entity_remove_links()
.
2. interface to entity links:
Associate one interface to a Link.
Drivers create interface to entity links by calling:
media_create_intf_link()
and remove with
media_remove_intf_links()
.
Note
Links can only be created after having both ends already created.
Links have flags that describe the link capabilities and state. The
valid values are described at media_create_pad_link()
and
media_create_intf_link()
.
4.1.7. Graph traversal¶
The media framework provides APIs to iterate over entities in a graph.
To iterate over all entities belonging to a media device, drivers can use
the media_device_for_each_entity macro, defined in
include/media/media-device.h
.
struct media_entity *entity;
media_device_for_each_entity(entity, mdev) {
// entity will point to each entity in turn
...
}
Drivers might also need to iterate over all entities in a graph that can be reached only through enabled links starting at a given entity. The media framework provides a depth-first graph traversal API for that purpose.
Note
Graphs with cycles (whether directed or undirected) are NOT
supported by the graph traversal API. To prevent infinite loops, the graph
traversal code limits the maximum depth to MEDIA_ENTITY_ENUM_MAX_DEPTH
,
currently defined as 16.
Drivers initiate a graph traversal by calling
media_graph_walk_start()
The graph structure, provided by the caller, is initialized to start graph traversal at the given entity.
Drivers can then retrieve the next entity by calling
media_graph_walk_next()
When the graph traversal is complete the function will return NULL
.
Graph traversal can be interrupted at any moment. No cleanup function call is required and the graph structure can be freed normally.
Helper functions can be used to find a link between two given pads, or a pad
connected to another pad through an enabled link
media_entity_find_link()
and
media_entity_remote_pad()
.
4.1.8. Use count and power handling¶
Due to the wide differences between drivers regarding power management
needs, the media controller does not implement power management. However,
the struct media_entity
includes a use_count
field that media drivers
can use to track the number of users of every entity for power management
needs.
The media_entity
.use_count
field is owned by
media drivers and must not be
touched by entity drivers. Access to the field must be protected by the
media_device
.graph_mutex
lock.
4.1.9. Links setup¶
Link properties can be modified at runtime by calling
media_entity_setup_link()
.
4.1.10. Pipelines and media streams¶
When starting streaming, drivers must notify all entities in the pipeline to
prevent link states from being modified during streaming by calling
media_pipeline_start()
.
The function will mark all entities connected to the given entity through enabled links, either directly or indirectly, as streaming.
The struct media_pipeline
instance pointed to by
the pipe argument will be stored in every entity in the pipeline.
Drivers should embed the struct media_pipeline
in higher-level pipeline structures and can then access the
pipeline through the struct media_entity
pipe field.
Calls to media_pipeline_start()
can be nested.
The pipeline pointer must be identical for all nested calls to the function.
media_pipeline_start()
may return an error. In that case,
it will clean up any of the changes it did by itself.
When stopping the stream, drivers must notify the entities with
media_pipeline_stop()
.
If multiple calls to media_pipeline_start()
have been
made the same number of media_pipeline_stop()
calls
are required to stop streaming.
The media_entity
.pipe
field is reset to NULL
on the last
nested stop call.
Link configuration will fail with -EBUSY
by default if either end of the
link is a streaming entity. Links that can be modified while streaming must
be marked with the MEDIA_LNK_FL_DYNAMIC
flag.
If other operations need to be disallowed on streaming entities (such as changing entities configuration parameters) drivers can explicitly check the media_entity stream_count field to find out if an entity is streaming. This operation must be done with the media_device graph_mutex held.
4.1.11. Link validation¶
Link validation is performed by media_pipeline_start()
for any entity which has sink pads in the pipeline. The
media_entity
.link_validate()
callback is used for that
purpose. In link_validate()
callback, entity driver should check
that the properties of the source pad of the connected entity and its own
sink pad match. It is up to the type of the entity (and in the end, the
properties of the hardware) what matching actually means.
Subsystems should facilitate link validation by providing subsystem specific helper functions to provide easy access for commonly needed information, and in the end provide a way to use driver-specific callbacks.
-
struct
media_entity_notify
¶ Media Entity Notify
Definition
struct media_entity_notify {
struct list_head list;
void *notify_data;
void (*notify)(struct media_entity *entity, void *notify_data);
};
Members
list
- List head
notify_data
- Input data to invoke the callback
notify
- Callback function pointer
Description
Drivers may register a callback to take action when new entities get registered with the media device. This handler is intended for creating links between existing entities and should not create entities and register them.
-
struct
media_device_ops
¶ Media device operations
Definition
struct media_device_ops {
int (*link_notify)(struct media_link *link, u32 flags, unsigned int notification);
};
Members
link_notify
- Link state change notification callback. This callback is called with the graph_mutex held.
-
struct
media_device
¶ Media device
Definition
struct media_device {
struct device *dev;
struct media_devnode *devnode;
char model[32];
char driver_name[32];
char serial[40];
char bus_info[32];
u32 hw_revision;
u64 topology_version;
u32 id;
struct ida entity_internal_idx;
int entity_internal_idx_max;
struct list_head entities;
struct list_head interfaces;
struct list_head pads;
struct list_head links;
struct list_head entity_notify;
struct mutex graph_mutex;
struct media_graph pm_count_walk;
void *source_priv;
int (*enable_source)(struct media_entity *entity, struct media_pipeline *pipe);
void (*disable_source)(struct media_entity *entity);
const struct media_device_ops *ops;
};
Members
dev
- Parent device
devnode
- Media device node
model
- Device model name
driver_name
- Optional device driver name. If not set, calls to
MEDIA_IOC_DEVICE_INFO
will returndev->driver->name
. This is needed for USB drivers for example, as otherwise they’ll all appear as if the driver name was “usb”. serial
- Device serial number (optional)
bus_info
- Unique and stable device location identifier
hw_revision
- Hardware device revision
topology_version
- Monotonic counter for storing the version of the graph topology. Should be incremented each time the topology changes.
id
- Unique ID used on the last registered graph object
entity_internal_idx
- Unique internal entity ID used by the graph traversal algorithms
entity_internal_idx_max
- Allocated internal entity indices
entities
- List of registered entities
interfaces
- List of registered interfaces
pads
- List of registered pads
links
- List of registered links
entity_notify
- List of registered entity_notify callbacks
graph_mutex
- Protects access to struct media_device data
pm_count_walk
- Graph walk for power state walk. Access serialised using graph_mutex.
source_priv
- Driver Private data for enable/disable source handlers
enable_source
- Enable Source Handler function pointer
disable_source
- Disable Source Handler function pointer
ops
- Operation handler callbacks
Description
This structure represents an abstract high-level media device. It allows easy access to entities and provides basic media device-level support. The structure can be allocated directly or embedded in a larger structure.
The parent dev is a physical device. It must be set before registering the media device.
model is a descriptive model name exported through sysfs. It doesn’t have to be unique.
enable_source is a handler to find source entity for the sink entity and activate the link between them if source entity is free. Drivers should call this handler before accessing the source.
disable_source is a handler to find source entity for the sink entity and deactivate the link between them. Drivers should call this handler to release the source.
Use-case: find tuner entity connected to the decoder entity and check if it is available, and activate the the link between them from enable_source and deactivate from disable_source.
Note
Bridge driver is expected to implement and set the
handler when media_device
is registered or when
bridge driver finds the media_device during probe.
Bridge driver sets source_priv with information
necessary to run enable_source and disable_source handlers.
Callers should hold graph_mutex to access and call enable_source
and disable_source handlers.
-
int
media_entity_enum_init
(struct media_entity_enum * ent_enum, struct media_device * mdev)¶ Initialise an entity enumeration
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration to be initialised
struct media_device * mdev
- The related media device
Return
zero on success or a negative error code.
-
void
media_device_init
(struct media_device * mdev)¶ Initializes a media device element
Parameters
struct media_device * mdev
- pointer to struct
media_device
Description
This function initializes the media device prior to its registration. The media device initialization and registration is split in two functions to avoid race conditions and make the media device available to user-space before the media graph has been completed.
So drivers need to first initialize the media device, register any entity
within the media device, create pad to pad links and then finally register
the media device by calling media_device_register()
as a final step.
-
void
media_device_cleanup
(struct media_device * mdev)¶ Cleanups a media device element
Parameters
struct media_device * mdev
- pointer to struct
media_device
Description
This function that will destroy the graph_mutex that is
initialized in media_device_init()
.
-
int
__media_device_register
(struct media_device * mdev, struct module * owner)¶ Registers a media device element
Parameters
struct media_device * mdev
- pointer to struct
media_device
struct module * owner
- should be filled with
THIS_MODULE
Description
Users, should, instead, call the media_device_register()
macro.
The caller is responsible for initializing the media_device
structure
before registration. The following fields of media_device
must be set:
media_entity.dev
must point to the parent device (usually apci_dev
,usb_interface
orplatform_device
instance).media_entity.model
must be filled with the device model name as a NUL-terminated UTF-8 string. The device/model revision must not be stored in this field.
The following fields are optional:
media_entity.serial
is a unique serial number stored as a NUL-terminated ASCII string. The field is big enough to store a GUID in text form. If the hardware doesn’t provide a unique serial number this field must be left empty.media_entity.bus_info
represents the location of the device in the system as a NUL-terminated ASCII string. For PCI/PCIe devicesmedia_entity.bus_info
must be set to “PCI:” (or “PCIe:”) followed by the value ofpci_name()
. For USB devices,theusb_make_path()
function must be used. This field is used by applications to distinguish between otherwise identical devices that don’t provide a serial number.media_entity.hw_revision
is the hardware device revision in a driver-specific format. When possible the revision should be formatted with theKERNEL_VERSION()
macro.
Note
- Upon successful registration a character device named media[0-9]+ is created. The device major and minor numbers are dynamic. The model name is exported as a sysfs attribute.
- Unregistering a media device that hasn’t been registered is NOT safe.
Return
returns zero on success or a negative error code.
-
media_device_register
(mdev)¶ Registers a media device element
Parameters
mdev
- pointer to struct
media_device
Description
This macro calls __media_device_register()
passing THIS_MODULE
as
the __media_device_register()
second argument (owner).
-
void
media_device_unregister
(struct media_device * mdev)¶ Unregisters a media device element
Parameters
struct media_device * mdev
- pointer to struct
media_device
Description
It is safe to call this function on an unregistered (but initialised) media device.
-
int
media_device_register_entity
(struct media_device * mdev, struct media_entity * entity)¶ registers a media entity inside a previously registered media device.
Parameters
struct media_device * mdev
- pointer to struct
media_device
struct media_entity * entity
- pointer to struct
media_entity
to be registered
Description
Entities are identified by a unique positive integer ID. The media controller framework will such ID automatically. IDs are not guaranteed to be contiguous, and the ID number can change on newer Kernel versions. So, neither the driver nor userspace should hardcode ID numbers to refer to the entities, but, instead, use the framework to find the ID, when needed.
The media_entity name, type and flags fields should be initialized before
calling media_device_register_entity()
. Entities embedded in higher-level
standard structures can have some of those fields set by the higher-level
framework.
If the device has pads, media_entity_pads_init()
should be called before
this function. Otherwise, the media_entity.pad
and media_entity.num_pads
should be zeroed before calling this function.
Entities have flags that describe the entity capabilities and state:
MEDIA_ENT_FL_DEFAULT
- indicates the default entity for a given type. This can be used to report the default audio and video devices or the default camera sensor.
Note
Drivers should set the entity function before calling this function.
Please notice that the values MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN
and
MEDIA_ENT_F_UNKNOWN
should not be used by the drivers.
-
void
media_device_unregister_entity
(struct media_entity * entity)¶ unregisters a media entity.
Parameters
struct media_entity * entity
- pointer to struct
media_entity
to be unregistered
Description
All links associated with the entity and all PADs are automatically unregistered from the media_device when this function is called.
Unregistering an entity will not change the IDs of the other entities and the previoully used ID will never be reused for a newly registered entities.
When a media device is unregistered, all its entities are unregistered automatically. No manual entities unregistration is then required.
Note
The media_entity instance itself must be freed explicitly by the driver if required.
-
int
media_device_register_entity_notify
(struct media_device * mdev, struct media_entity_notify * nptr)¶ Registers a media entity_notify callback
Parameters
struct media_device * mdev
- The media device
struct media_entity_notify * nptr
- The media_entity_notify
Description
Note
When a new entity is registered, all the registered media_entity_notify callbacks are invoked.
-
void
media_device_unregister_entity_notify
(struct media_device * mdev, struct media_entity_notify * nptr)¶ Unregister a media entity notify callback
Parameters
struct media_device * mdev
- The media device
struct media_entity_notify * nptr
- The media_entity_notify
-
void
media_device_pci_init
(struct media_device * mdev, struct pci_dev * pci_dev, const char * name)¶ create and initialize a struct
media_device
from a PCI device.
Parameters
struct media_device * mdev
- pointer to struct
media_device
struct pci_dev * pci_dev
- pointer to struct pci_dev
const char * name
- media device name. If
NULL
, the routine will use the default name for the pci device, given bypci_name()
macro.
-
void
__media_device_usb_init
(struct media_device * mdev, struct usb_device * udev, const char * board_name, const char * driver_name)¶ create and initialize a struct
media_device
from a PCI device.
Parameters
struct media_device * mdev
- pointer to struct
media_device
struct usb_device * udev
- pointer to struct usb_device
const char * board_name
- media device name. If
NULL
, the routine will use the usb product name, if available. const char * driver_name
- name of the driver. if
NULL
, the routine will use the name given byudev->dev->driver->name
, with is usually the wrong thing to do.
Description
Note
It is better to call media_device_usb_init()
instead, as
such macro fills driver_name with KBUILD_MODNAME
.
-
media_device_usb_init
(mdev, udev, name)¶ create and initialize a struct
media_device
from a PCI device.
Parameters
mdev
- pointer to struct
media_device
udev
- pointer to struct usb_device
name
- media device name. If
NULL
, the routine will use the usb product name, if available.
Description
This macro calls media_device_usb_init()
passing the
media_device_usb_init()
driver_name parameter filled with
KBUILD_MODNAME
.
-
struct
media_file_operations
¶ Media device file operations
Definition
struct media_file_operations {
struct module *owner;
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
__poll_t (*poll) (struct file *, struct poll_table_struct *);
long (*ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*open) (struct file *);
int (*release) (struct file *);
};
Members
owner
- should be filled with
THIS_MODULE
read
- pointer to the function that implements
read()
syscall write
- pointer to the function that implements
write()
syscall poll
- pointer to the function that implements
poll()
syscall ioctl
- pointer to the function that implements
ioctl()
syscall compat_ioctl
- pointer to the function that will handle 32 bits userspace
calls to the the
ioctl()
syscall on a Kernel compiled with 64 bits. open
- pointer to the function that implements
open()
syscall release
- pointer to the function that will release the resources allocated by the open function.
-
struct
media_devnode
¶ Media device node
Definition
struct media_devnode {
struct media_device *media_dev;
const struct media_file_operations *fops;
struct device dev;
struct cdev cdev;
struct device *parent;
int minor;
unsigned long flags;
void (*release)(struct media_devnode *devnode);
};
Members
media_dev
- pointer to struct
media_device
fops
- pointer to struct
media_file_operations
with media device ops dev
- pointer to struct
device
containing the media controller device cdev
- struct cdev pointer character device
parent
- parent device
minor
- device node minor number
flags
- flags, combination of the
MEDIA_FLAG_*
constants release
- release callback called at the end of
:c:func:`media_devnode_release()`
routine at media-device.c.
Description
This structure represents a media-related device node.
The parent is a physical device. It must be set by core or device drivers before registering the node.
-
int
media_devnode_register
(struct media_device * mdev, struct media_devnode * devnode, struct module * owner)¶ register a media device node
Parameters
struct media_device * mdev
- struct media_device we want to register a device node
struct media_devnode * devnode
- media device node structure we want to register
struct module * owner
- should be filled with
THIS_MODULE
Description
The registration code assigns minor numbers and registers the new device node with the kernel. An error is returned if no free minor number can be found, or if the registration of the device node fails.
Zero is returned on success.
Note that if the media_devnode_register call fails, the release()
callback of
the media_devnode structure is not called, so the caller is responsible for
freeing any data.
-
void
media_devnode_unregister_prepare
(struct media_devnode * devnode)¶ clear the media device node register bit
Parameters
struct media_devnode * devnode
- the device node to prepare for unregister
Description
This clears the passed device register bit. Future open calls will be met
with errors. Should be called before media_devnode_unregister()
to avoid
races with unregister and device file open calls.
This function can safely be called if the device node has never been registered or has already been unregistered.
-
void
media_devnode_unregister
(struct media_devnode * devnode)¶ unregister a media device node
Parameters
struct media_devnode * devnode
- the device node to unregister
Description
This unregisters the passed device. Future open calls will be met with errors.
Should be called after media_devnode_unregister_prepare()
-
struct media_devnode *
media_devnode_data
(struct file * filp)¶ returns a pointer to the
media_devnode
Parameters
struct file * filp
- pointer to struct
file
-
int
media_devnode_is_registered
(struct media_devnode * devnode)¶ returns true if
media_devnode
is registered; false otherwise.
Parameters
struct media_devnode * devnode
- pointer to struct
media_devnode
.
Note
If mdev is NULL, it also returns false.
-
enum
media_gobj_type
¶ type of a graph object
Constants
MEDIA_GRAPH_ENTITY
- Identify a media entity
MEDIA_GRAPH_PAD
- Identify a media pad
MEDIA_GRAPH_LINK
- Identify a media link
MEDIA_GRAPH_INTF_DEVNODE
- Identify a media Kernel API interface via a device node
-
struct
media_gobj
¶ Define a graph object.
Definition
struct media_gobj {
struct media_device *mdev;
u32 id;
struct list_head list;
};
Members
mdev
- Pointer to the struct
media_device
that owns the object id
- Non-zero object ID identifier. The ID should be unique
inside a media_device, as it is composed by
MEDIA_BITS_PER_TYPE
to store the type plusMEDIA_BITS_PER_ID
to store the ID list
- List entry stored in one of the per-type mdev object lists
Description
All objects on the media graph should have this struct embedded
-
struct
media_entity_enum
¶ An enumeration of media entities.
Definition
struct media_entity_enum {
unsigned long *bmap;
int idx_max;
};
Members
bmap
- Bit map in which each bit represents one entity at struct media_entity->internal_idx.
idx_max
- Number of bits in bmap
-
struct
media_graph
¶ Media graph traversal state
Definition
struct media_graph {
struct {
struct media_entity *entity;
struct list_head *link;
} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
struct media_entity_enum ent_enum;
int top;
};
Members
stack
- Graph traversal stack; the stack contains information on the path the media entities to be walked and the links through which they were reached.
stack.entity
- pointer to
struct media_entity
at the graph. stack.link
- pointer to
struct list_head
. ent_enum
- Visited entities
top
- The top of the stack
-
struct
media_pipeline
¶ Media pipeline related information
Definition
struct media_pipeline {
int streaming_count;
struct media_graph graph;
};
Members
streaming_count
- Streaming start count - streaming stop count
graph
- Media graph walk during pipeline start / stop
-
struct
media_link
¶ A link object part of a media graph.
Definition
struct media_link {
struct media_gobj graph_obj;
struct list_head list;
union {
struct media_gobj *gobj0;
struct media_pad *source;
struct media_interface *intf;
};
union {
struct media_gobj *gobj1;
struct media_pad *sink;
struct media_entity *entity;
};
struct media_link *reverse;
unsigned long flags;
bool is_backlink;
};
Members
graph_obj
- Embedded structure containing the media object common data
list
- Linked list associated with an entity or an interface that owns the link.
{unnamed_union}
- anonymous
gobj0
- Part of a union. Used to get the pointer for the first graph_object of the link.
source
- Part of a union. Used only if the first object (gobj0) is a pad. In that case, it represents the source pad.
intf
- Part of a union. Used only if the first object (gobj0) is an interface.
{unnamed_union}
- anonymous
gobj1
- Part of a union. Used to get the pointer for the second graph_object of the link.
sink
- Part of a union. Used only if the second object (gobj1) is a pad. In that case, it represents the sink pad.
entity
- Part of a union. Used only if the second object (gobj1) is an entity.
reverse
- Pointer to the link for the reverse direction of a pad to pad link.
flags
- Link flags, as defined in uapi/media.h (MEDIA_LNK_FL_*)
is_backlink
- Indicate if the link is a backlink.
-
struct
media_pad
¶ A media pad graph object.
Definition
struct media_pad {
struct media_gobj graph_obj;
struct media_entity *entity;
u16 index;
unsigned long flags;
};
Members
graph_obj
- Embedded structure containing the media object common data
entity
- Entity this pad belongs to
index
- Pad index in the entity pads array, numbered from 0 to n
flags
- Pad flags, as defined in
include/uapi/linux/media.h
(seek for
MEDIA_PAD_FL_*
)
-
struct
media_entity_operations
¶ Media entity operations
Definition
struct media_entity_operations {
int (*get_fwnode_pad)(struct fwnode_endpoint *endpoint);
int (*link_setup)(struct media_entity *entity,const struct media_pad *local, const struct media_pad *remote, u32 flags);
int (*link_validate)(struct media_link *link);
};
Members
get_fwnode_pad
- Return the pad number based on a fwnode endpoint or a negative value on error. This operation can be used to map a fwnode to a media pad number. Optional.
link_setup
- Notify the entity of link changes. The operation can return an error, in which case link setup will be cancelled. Optional.
link_validate
- Return whether a link is valid from the entity point of
view. The
media_pipeline_start()
function validates all links by calling this operation. Optional.
Description
Note
Those these callbacks are called with struct media_device.graph_mutex
mutex held.
-
enum
media_entity_type
¶ Media entity type
Constants
MEDIA_ENTITY_TYPE_BASE
- The entity isn’t embedded in another subsystem structure.
MEDIA_ENTITY_TYPE_VIDEO_DEVICE
- The entity is embedded in a struct video_device instance.
MEDIA_ENTITY_TYPE_V4L2_SUBDEV
- The entity is embedded in a struct v4l2_subdev instance.
Description
Media entity objects are often not instantiated directly, but the media entity structure is inherited by (through embedding) other subsystem-specific structures. The media entity type identifies the type of the subclass structure that implements a media entity instance.
This allows runtime type identification of media entities and safe casting to
the correct object type. For instance, a media entity structure instance
embedded in a v4l2_subdev structure instance will have the type
MEDIA_ENTITY_TYPE_V4L2_SUBDEV
and can safely be cast to a v4l2_subdev
structure using the container_of()
macro.
-
struct
media_entity
¶ A media entity graph object.
Definition
struct media_entity {
struct media_gobj graph_obj;
const char *name;
enum media_entity_type obj_type;
u32 function;
unsigned long flags;
u16 num_pads;
u16 num_links;
u16 num_backlinks;
int internal_idx;
struct media_pad *pads;
struct list_head links;
const struct media_entity_operations *ops;
int stream_count;
int use_count;
struct media_pipeline *pipe;
union {
struct {
u32 major;
u32 minor;
} dev;
} info;
};
Members
graph_obj
- Embedded structure containing the media object common data.
name
- Entity name.
obj_type
- Type of the object that implements the media_entity.
function
- Entity main function, as defined in
include/uapi/linux/media.h
(seek for
MEDIA_ENT_F_*
) flags
- Entity flags, as defined in
include/uapi/linux/media.h
(seek for
MEDIA_ENT_FL_*
) num_pads
- Number of sink and source pads.
num_links
- Total number of links, forward and back, enabled and disabled.
num_backlinks
- Number of backlinks
internal_idx
- An unique internal entity specific number. The numbers are re-used if entities are unregistered or registered again.
pads
- Pads array with the size defined by num_pads.
links
- List of data links.
ops
- Entity operations.
stream_count
- Stream count for the entity.
use_count
- Use count for the entity.
pipe
- Pipeline this entity belongs to.
info
- Union with devnode information. Kept just for backward compatibility.
info.dev
- Contains device major and minor info.
info.dev.major
- device node major, if the device is a devnode.
info.dev.minor
- device node minor, if the device is a devnode.
Description
Note
stream_count and use_count reference counts must never be
negative, but are signed integers on purpose: a simple WARN_ON(<0)
check can be used to detect reference count bugs that would make them
negative.
-
struct
media_interface
¶ A media interface graph object.
Definition
struct media_interface {
struct media_gobj graph_obj;
struct list_head links;
u32 type;
u32 flags;
};
Members
graph_obj
- embedded graph object
links
- List of links pointing to graph entities
type
- Type of the interface as defined in
include/uapi/linux/media.h
(seek for
MEDIA_INTF_T_*
) flags
- Interface flags as defined in
include/uapi/linux/media.h
(seek for
MEDIA_INTF_FL_*
)
Description
Note
Currently, no flags for media_interface
is defined.
-
struct
media_intf_devnode
¶ A media interface via a device node.
Definition
struct media_intf_devnode {
struct media_interface intf;
u32 major;
u32 minor;
};
Members
intf
- embedded interface object
major
- Major number of a device node
minor
- Minor number of a device node
-
u32
media_entity_id
(struct media_entity * entity)¶ return the media entity graph object id
Parameters
struct media_entity * entity
- pointer to
media_entity
-
enum media_gobj_type
media_type
(struct media_gobj * gobj)¶ return the media object type
Parameters
struct media_gobj * gobj
- Pointer to the struct
media_gobj
graph object
-
u32
media_id
(struct media_gobj * gobj)¶ return the media object ID
Parameters
struct media_gobj * gobj
- Pointer to the struct
media_gobj
graph object
-
u32
media_gobj_gen_id
(enum media_gobj_type type, u64 local_id)¶ encapsulates type and ID on at the object ID
Parameters
enum media_gobj_type type
- object type as define at enum
media_gobj_type
. u64 local_id
- next ID, from struct
media_device.id
.
-
bool
is_media_entity_v4l2_video_device
(struct media_entity * entity)¶ Check if the entity is a video_device
Parameters
struct media_entity * entity
- pointer to entity
Return
true
if the entity is an instance of a video_device object and can
safely be cast to a struct video_device using the container_of()
macro, or
false
otherwise.
-
bool
is_media_entity_v4l2_subdev
(struct media_entity * entity)¶ Check if the entity is a v4l2_subdev
Parameters
struct media_entity * entity
- pointer to entity
Return
true
if the entity is an instance of a v4l2_subdev
object and can
safely be cast to a struct v4l2_subdev
using the container_of()
macro, or
false
otherwise.
-
int
__media_entity_enum_init
(struct media_entity_enum * ent_enum, int idx_max)¶ Initialise an entity enumeration
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration to be initialised
int idx_max
- Maximum number of entities in the enumeration
Return
Returns zero on success or a negative error code.
-
void
media_entity_enum_cleanup
(struct media_entity_enum * ent_enum)¶ Release resources of an entity enumeration
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration to be released
-
void
media_entity_enum_zero
(struct media_entity_enum * ent_enum)¶ Clear the entire enum
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration to be cleared
-
void
media_entity_enum_set
(struct media_entity_enum * ent_enum, struct media_entity * entity)¶ Mark a single entity in the enum
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration
struct media_entity * entity
- Entity to be marked
-
void
media_entity_enum_clear
(struct media_entity_enum * ent_enum, struct media_entity * entity)¶ Unmark a single entity in the enum
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration
struct media_entity * entity
- Entity to be unmarked
-
bool
media_entity_enum_test
(struct media_entity_enum * ent_enum, struct media_entity * entity)¶ Test whether the entity is marked
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration
struct media_entity * entity
- Entity to be tested
Description
Returns true
if the entity was marked.
-
bool
media_entity_enum_test_and_set
(struct media_entity_enum * ent_enum, struct media_entity * entity)¶ Test whether the entity is marked, and mark it
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration
struct media_entity * entity
- Entity to be tested
Description
Returns true
if the entity was marked, and mark it before doing so.
-
bool
media_entity_enum_empty
(struct media_entity_enum * ent_enum)¶ Test whether the entire enum is empty
Parameters
struct media_entity_enum * ent_enum
- Entity enumeration
Return
true
if the entity was empty.
-
bool
media_entity_enum_intersects
(struct media_entity_enum * ent_enum1, struct media_entity_enum * ent_enum2)¶ Test whether two enums intersect
Parameters
struct media_entity_enum * ent_enum1
- First entity enumeration
struct media_entity_enum * ent_enum2
- Second entity enumeration
Return
true
if entity enumerations ent_enum1 and ent_enum2 intersect,
otherwise false
.
-
gobj_to_entity
(gobj)¶ returns the struct
media_entity
pointer from the gobj contained on it.
Parameters
gobj
- Pointer to the struct
media_gobj
graph object
Parameters
gobj
- Pointer to the struct
media_gobj
graph object
-
gobj_to_link
(gobj)¶ returns the struct
media_link
pointer from the gobj contained on it.
Parameters
gobj
- Pointer to the struct
media_gobj
graph object
-
gobj_to_intf
(gobj)¶ returns the struct
media_interface
pointer from the gobj contained on it.
Parameters
gobj
- Pointer to the struct
media_gobj
graph object
-
intf_to_devnode
(intf)¶ returns the struct media_intf_devnode pointer from the intf contained on it.
Parameters
intf
- Pointer to struct
media_intf_devnode
-
void
media_gobj_create
(struct media_device * mdev, enum media_gobj_type type, struct media_gobj * gobj)¶ Initialize a graph object
Parameters
struct media_device * mdev
- Pointer to the
media_device
that contains the object enum media_gobj_type type
- Type of the object
struct media_gobj * gobj
- Pointer to the struct
media_gobj
graph object
Description
This routine initializes the embedded struct media_gobj
inside a
media graph object. It is called automatically if media_*_create
function calls are used. However, if the object (entity, link, pad,
interface) is embedded on some other object, this function should be
called before registering the object at the media controller.
-
void
media_gobj_destroy
(struct media_gobj * gobj)¶ Stop using a graph object on a media device
Parameters
struct media_gobj * gobj
- Pointer to the struct
media_gobj
graph object
Description
This should be called by all routines like media_device_unregister()
that remove/destroy media graph objects.
-
int
media_entity_pads_init
(struct media_entity * entity, u16 num_pads, struct media_pad * pads)¶ Initialize the entity pads
Parameters
struct media_entity * entity
- entity where the pads belong
u16 num_pads
- total number of sink and source pads
struct media_pad * pads
- Array of num_pads pads.
Description
The pads array is managed by the entity driver and passed to
media_entity_pads_init()
where its pointer will be stored in the
media_entity
structure.
If no pads are needed, drivers could either directly fill
media_entity->num_pads
with 0 and media_entity->pads
with NULL
or call
this function that will do the same.
As the number of pads is known in advance, the pads array is not allocated dynamically but is managed by the entity driver. Most drivers will embed the pads array in a driver-specific structure, avoiding dynamic allocation.
Drivers must set the direction of every pad in the pads array before calling
media_entity_pads_init()
. The function will initialize the other pads fields.
-
void
media_entity_cleanup
(struct media_entity * entity)¶ free resources associated with an entity
Parameters
struct media_entity * entity
- entity where the pads belong
Description
This function must be called during the cleanup phase after unregistering the entity (currently, it does nothing).
-
int
media_create_pad_link
(struct media_entity * source, u16 source_pad, struct media_entity * sink, u16 sink_pad, u32 flags)¶ creates a link between two entities.
Parameters
struct media_entity * source
- pointer to
media_entity
of the source pad. u16 source_pad
- number of the source pad in the pads array
struct media_entity * sink
- pointer to
media_entity
of the sink pad. u16 sink_pad
- number of the sink pad in the pads array.
u32 flags
- Link flags, as defined in
include/uapi/linux/media.h
( seek for
MEDIA_LNK_FL_*
)
Description
Valid values for flags:
MEDIA_LNK_FL_ENABLED
- Indicates that the link is enabled and can be used to transfer media data. When two or more links target a sink pad, only one of them can be enabled at a time.
MEDIA_LNK_FL_IMMUTABLE
- Indicates that the link enabled state can’t be modified at runtime. If
MEDIA_LNK_FL_IMMUTABLE
is set, thenMEDIA_LNK_FL_ENABLED
must also be set, since an immutable link is always enabled.
Note
Before calling this function, media_entity_pads_init()
and
media_device_register_entity()
should be called previously for both ends.
-
int
media_create_pad_links
(const struct media_device * mdev, const u32 source_function, struct media_entity * source, const u16 source_pad, const u32 sink_function, struct media_entity * sink, const u16 sink_pad, u32 flags, const bool allow_both_undefined)¶ creates a link between two entities.
Parameters
const struct media_device * mdev
- Pointer to the media_device that contains the object
const u32 source_function
- Function of the source entities. Used only if source is NULL.
struct media_entity * source
- pointer to
media_entity
of the source pad. If NULL, it will use all entities that matches the sink_function. const u16 source_pad
- number of the source pad in the pads array
const u32 sink_function
- Function of the sink entities. Used only if sink is NULL.
struct media_entity * sink
- pointer to
media_entity
of the sink pad. If NULL, it will use all entities that matches the sink_function. const u16 sink_pad
- number of the sink pad in the pads array.
u32 flags
- Link flags, as defined in include/uapi/linux/media.h.
const bool allow_both_undefined
- if
true
, then both source and sink can be NULL. In such case, it will create a crossbar between all entities that matches source_function to all entities that matches sink_function. Iffalse
, it will return 0 and won’t create any link if both source and sink are NULL.
Description
Valid values for flags:
- A
MEDIA_LNK_FL_ENABLED
flag indicates that the link is enabled and can be - used to transfer media data. If multiple links are created and this flag is passed as an argument, only the first created link will have this flag.
- A
MEDIA_LNK_FL_IMMUTABLE
flag indicates that the link enabled state can’t - be modified at runtime. If
MEDIA_LNK_FL_IMMUTABLE
is set, thenMEDIA_LNK_FL_ENABLED
must also be set since an immutable link is always enabled.
It is common for some devices to have multiple source and/or sink entities
of the same type that should be linked. While media_create_pad_link()
creates link by link, this function is meant to allow 1:n, n:1 and even
cross-bar (n:n) links.
Note
Before calling this function, media_entity_pads_init()
and
media_device_register_entity()
should be called previously for the
entities to be linked.
-
void
media_entity_remove_links
(struct media_entity * entity)¶ remove all links associated with an entity
Parameters
struct media_entity * entity
- pointer to
media_entity
Description
Note
This is called automatically when an entity is unregistered via
media_device_register_entity()
.
-
int
__media_entity_setup_link
(struct media_link * link, u32 flags)¶ Configure a media link without locking
Parameters
struct media_link * link
- The link being configured
u32 flags
- Link configuration flags
Description
The bulk of link setup is handled by the two entities connected through the link. This function notifies both entities of the link configuration change.
If the link is immutable or if the current and new configuration are identical, return immediately.
The user is expected to hold link->source->parent->mutex. If not,
media_entity_setup_link()
should be used instead.
-
int
media_entity_setup_link
(struct media_link * link, u32 flags)¶ changes the link flags properties in runtime
Parameters
struct media_link * link
- pointer to
media_link
u32 flags
- the requested new link flags
Description
The only configurable property is the MEDIA_LNK_FL_ENABLED
link flag
flag to enable/disable a link. Links marked with the
MEDIA_LNK_FL_IMMUTABLE
link flag can not be enabled or disabled.
When a link is enabled or disabled, the media framework calls the link_setup operation for the two entities at the source and sink of the link, in that order. If the second link_setup call fails, another link_setup call is made on the first entity to restore the original link flags.
Media device drivers can be notified of link setup operations by setting the
media_device.link_notify
pointer to a callback function. If provided, the
notification callback will be called before enabling and after disabling
links.
Entity drivers must implement the link_setup operation if any of their links is non-immutable. The operation must either configure the hardware or store the configuration information to be applied later.
Link configuration must not have any side effect on other links. If an
enabled link at a sink pad prevents another link at the same pad from
being enabled, the link_setup operation must return -EBUSY
and can’t
implicitly disable the first enabled link.
Note
The valid values of the flags for the link is the same as described
on media_create_pad_link()
, for pad to pad links or the same as described
on media_create_intf_link()
, for interface to entity links.
-
struct media_link *
media_entity_find_link
(struct media_pad * source, struct media_pad * sink)¶ Find a link between two pads
Parameters
struct media_pad * source
- Source pad
struct media_pad * sink
- Sink pad
Return
returns a pointer to the link between the two entities. If no
such link exists, return NULL
.
-
struct media_pad *
media_entity_remote_pad
(const struct media_pad * pad)¶ Find the pad at the remote end of a link
Parameters
const struct media_pad * pad
- Pad at the local end of the link
Description
Search for a remote pad connected to the given pad by iterating over all links originating or terminating at that pad until an enabled link is found.
Return
returns a pointer to the pad at the remote end of the first found
enabled link, or NULL
if no enabled link has been found.
-
struct media_entity *
media_entity_get
(struct media_entity * entity)¶ Get a reference to the parent module
Parameters
struct media_entity * entity
- The entity
Description
Get a reference to the parent media device module.
The function will return immediately if entity is NULL
.
Return
returns a pointer to the entity on success or NULL
on failure.
-
int
media_entity_get_fwnode_pad
(struct media_entity * entity, struct fwnode_handle * fwnode, unsigned long direction_flags)¶ Get pad number from fwnode
Parameters
struct media_entity * entity
- The entity
struct fwnode_handle * fwnode
- Pointer to the fwnode_handle which should be used to find the pad
unsigned long direction_flags
- Expected direction of the pad, as defined in
include/uapi/linux/media.h
(seek for
MEDIA_PAD_FL_*
)
Description
This function can be used to resolve the media pad number from a fwnode. This is useful for devices which use more complex mappings of media pads.
If the entity does not implement the get_fwnode_pad()
operation
then this function searches the entity for the first pad that
matches the direction_flags.
Return
returns the pad number on success or a negative error code.
-
int
media_graph_walk_init
(struct media_graph * graph, struct media_device * mdev)¶ Allocate resources used by graph walk.
Parameters
struct media_graph * graph
- Media graph structure that will be used to walk the graph
struct media_device * mdev
- Pointer to the
media_device
that contains the object
-
void
media_graph_walk_cleanup
(struct media_graph * graph)¶ Release resources used by graph walk.
Parameters
struct media_graph * graph
- Media graph structure that will be used to walk the graph
-
void
media_entity_put
(struct media_entity * entity)¶ Release the reference to the parent module
Parameters
struct media_entity * entity
- The entity
Description
Release the reference count acquired by media_entity_get()
.
The function will return immediately if entity is NULL
.
-
void
media_graph_walk_start
(struct media_graph * graph, struct media_entity * entity)¶ Start walking the media graph at a given entity
Parameters
struct media_graph * graph
- Media graph structure that will be used to walk the graph
struct media_entity * entity
- Starting entity
Description
Before using this function, media_graph_walk_init()
must be
used to allocate resources used for walking the graph. This
function initializes the graph traversal structure to walk the
entities graph starting at the given entity. The traversal
structure must not be modified by the caller during graph
traversal. After the graph walk, the resources must be released
using media_graph_walk_cleanup()
.
-
struct media_entity *
media_graph_walk_next
(struct media_graph * graph)¶ Get the next entity in the graph
Parameters
struct media_graph * graph
- Media graph structure
Description
Perform a depth-first traversal of the given media entities graph.
The graph structure must have been previously initialized with a call to
media_graph_walk_start()
.
Return
returns the next entity in the graph or NULL
if the whole graph
have been traversed.
-
int
media_pipeline_start
(struct media_entity * entity, struct media_pipeline * pipe)¶ Mark a pipeline as streaming
Parameters
struct media_entity * entity
- Starting entity
struct media_pipeline * pipe
- Media pipeline to be assigned to all entities in the pipeline.
Description
Mark all entities connected to a given entity through enabled links, either directly or indirectly, as streaming. The given pipeline object is assigned to every entity in the pipeline and stored in the media_entity pipe field.
Calls to this function can be nested, in which case the same number of
media_pipeline_stop()
calls will be required to stop streaming. The
pipeline pointer must be identical for all nested calls to
media_pipeline_start()
.
-
int
__media_pipeline_start
(struct media_entity * entity, struct media_pipeline * pipe)¶ Mark a pipeline as streaming
Parameters
struct media_entity * entity
- Starting entity
struct media_pipeline * pipe
- Media pipeline to be assigned to all entities in the pipeline.
Description
..note:: This is the non-locking version of media_pipeline_start()
-
void
media_pipeline_stop
(struct media_entity * entity)¶ Mark a pipeline as not streaming
Parameters
struct media_entity * entity
- Starting entity
Description
Mark all entities connected to a given entity through enabled links, either
directly or indirectly, as not streaming. The media_entity pipe field is
reset to NULL
.
If multiple calls to media_pipeline_start()
have been made, the same
number of calls to this function are required to mark the pipeline as not
streaming.
-
void
__media_pipeline_stop
(struct media_entity * entity)¶ Mark a pipeline as not streaming
Parameters
struct media_entity * entity
- Starting entity
Description
Note
This is the non-locking version of media_pipeline_stop()
-
struct media_intf_devnode *
media_devnode_create
(struct media_device * mdev, u32 type, u32 flags, u32 major, u32 minor)¶ creates and initializes a device node interface
Parameters
struct media_device * mdev
- pointer to struct
media_device
u32 type
- type of the interface, as given by
include/uapi/linux/media.h
( seek for
MEDIA_INTF_T_*
) macros. u32 flags
- Interface flags, as defined in
include/uapi/linux/media.h
( seek for
MEDIA_INTF_FL_*
) u32 major
- Device node major number.
u32 minor
- Device node minor number.
Return
- if succeeded, returns a pointer to the newly allocated
media_intf_devnode
pointer.
Note
Currently, no flags for media_interface
is defined.
-
void
media_devnode_remove
(struct media_intf_devnode * devnode)¶ removes a device node interface
Parameters
struct media_intf_devnode * devnode
- pointer to
media_intf_devnode
to be freed.
Description
When a device node interface is removed, all links to it are automatically removed.
-
media_create_intf_link
(struct media_entity * entity, struct media_interface * intf, u32 flags)¶ creates a link between an entity and an interface
Parameters
struct media_entity * entity
- pointer to
media_entity
struct media_interface * intf
- pointer to
media_interface
u32 flags
- Link flags, as defined in
include/uapi/linux/media.h
( seek for
MEDIA_LNK_FL_*
)
Description
Valid values for flags:
MEDIA_LNK_FL_ENABLED
Indicates that the interface is connected to the entity hardware. That’s the default value for interfaces. An interface may be disabled if the hardware is busy due to the usage of some other interface that it is currently controlling the hardware.
A typical example is an hybrid TV device that handle only one type of stream on a given time. So, when the digital TV is streaming, the V4L2 interfaces won’t be enabled, as such device is not able to also stream analog TV or radio.
Note
Before calling this function, media_devnode_create()
should be called for
the interface and media_device_register_entity()
should be called for the
interface that will be part of the link.
-
void
__media_remove_intf_link
(struct media_link * link)¶ remove a single interface link
Parameters
struct media_link * link
- pointer to
media_link
.
Description
Note
This is an unlocked version of media_remove_intf_link()
-
void
media_remove_intf_link
(struct media_link * link)¶ remove a single interface link
Parameters
struct media_link * link
- pointer to
media_link
.
Description
Note
Prefer to use this one, instead of __media_remove_intf_link()
-
void
__media_remove_intf_links
(struct media_interface * intf)¶ remove all links associated with an interface
Parameters
struct media_interface * intf
- pointer to
media_interface
Description
Note
This is an unlocked version of media_remove_intf_links()
.
-
void
media_remove_intf_links
(struct media_interface * intf)¶ remove all links associated with an interface
Parameters
struct media_interface * intf
- pointer to
media_interface
Description
Note
- This is called automatically when an entity is unregistered via
media_device_register_entity()
and bymedia_devnode_remove()
. - Prefer to use this one, instead of
__media_remove_intf_links()
.
-
media_entity_call
(entity, operation, args...)¶ Calls a struct media_entity_operations operation on an entity
Parameters
entity
- entity where the operation will be called
operation
- type of the operation. Should be the name of a member of
struct
media_entity_operations
. args...
- variable arguments
Description
This helper function will check if operation is not NULL
. On such case,
it will issue a call to operation(entity, args).