Rpi/v6.6/streams #200
Open
Rpi/v6.6/streams #200
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update the kernel headers with the definition of two device-specific line-based metadata formats, and with the definition of the MEDIA_PAD_FL_INTERNAL and V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flags. The new definitions will allow to support handling line-based metadata streams exposed by the sensor driver through an internal sink pad. While the changes have not yet been collected in the official linux-media tree, they're available in the 'metadata' branch of https://git.linuxtv.org/sailus/media_tree.git, at revision: 8d8fdc8a8809 ("media: Documentation: Add subdev configuration models, raw sensor model"). The patches are expected to land in Linux v6.13. Signed-off-by: Jacopo Mondi <[email protected]> Acked-by: Kieran Bingham <[email protected]>
Add the definition for the generic line based pixelformats.
The formats has been added in upstream Linux by commit
1d9215233958 ("media: uapi: v4l: Add generic 8-bit metadata format
definitions") which got merged in Linux v6.10.
The formats however are not yet available to userspace, as they
have been made only available to the kernel by commit
d69c8429ea80 ("media: uapi: v4l: Don't expose generic metadata formats
to userspace") to let the line-based metadata support stabilize before
allowing applications to use it.
With the forthcoming completion of the line-based metadata upstreaming
manually add the generic line based pixel format to prepare libcamera
to support them.
Signed-off-by: Jacopo Mondi <[email protected]>
V4L2_CID_USER_BCM2835_ISP_BASE is defined in RPi BSP as
(V4L2_CID_USER_BASE + 0x10e0)
However in mainline this address has been occupied by other control
bases. Align V4L2_CID_USER_BCM2835_ISP_BASE to the last available
identifier available in mainline.
This commit requires kernel commit d1b7fb3b95c6 ("uapi: bcm2835-isp:
Re-assign V4L2_CID_USER_BCM2835_ISP_BASE")
Signed-off-by: Jacopo Mondi <[email protected]>
Support the newly introduced V4L2 media bus formats for metadata. This includes generic metadata formats, and two sensor-specific embedded data formats. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>
With support for metadata in the streams API, the v4l2_meta_format structure has been extended with width, height and bytesperline fields. Support them in the V4L2VideoDevice getFormat() and setFormat() functions is the video device is meta capture device and if the pixel format is one of the generic line-based metadata formats. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]> --- Compared to Larent's first version, this one sets width, height and bytesperline only if the device isMetaCap() and the V4L2 pixel format is one of the generic line based ones. Signed-off-by: Jacopo Mondi <[email protected]>
Introduce a factory to create CameraSensor derived classes instances by inspecting the sensor media entity name and provide a convenience macro to register specialized sensor handlers. Signed-off-by: Jacopo Mondi <[email protected]> Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Stefan Klug <[email protected]>
With a camera sensor factory in place, the next step is to create an abstract base class that all camera sensors implement, providing a uniform API to pipeline handler. Turn all public functions of the CameraSensor class into pure virtual functions, and move the implementation to the CameraSensorLegacy class. Part of the code is likely worth keeping as common helpers in a base class. However, to follow the principle of not designing helpers with a single user, this commit moves the whole implementation. Common helpers will be introduced later, along with other CameraSensor subclasses. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Stefan Klug <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>
In order to support a default implementation for camera sensors when no better implementation matches, libcamera needs to try "specialized" implementations first and pick the default last. Make this possible by adding a priority value for factories. Newly registered factories are inserted in the factories list sorted by descending priority, and the default factory uses a negative priority to be inserted as the last element. This mechanism may be a bit overkill in the sense that there is no expected use cases for priorities other than trying the default last, but the implementation is simple and easy to understand. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Stefan Klug <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>
Add a new CameraSensorRaw implementation of the CameraSensor interface tailored to devices that implement the new V4L2 raw camera sensors API. This new class duplicates code from the CameraSensorLegacy class. The two classes will be refactored to share code. Signed-off-by: Laurent Pinchart <[email protected]> --- Changes since combined RFC: - Set factory priority explicitly - Set flipsAlterBayerOrder_ - Fix build warning due to missing default case in cfa switch - Check for read-only hblank using V4L2_CTRL_FLAG_READ_ONLY
Some sensors support producing and transmitting embedded data over a stream separate from the image stream. Add support for this feature in the CameraSensor interface, and implement it for the CameraSensorRaw class. The CameraSensorLegacy uses the default stub implementation, as the corresponding kernel drivers don't support embedded data. Signed-off-by: Laurent Pinchart <[email protected]>
Now that the CameraSensor class implements support for embedded data, use the corresponding API to enable or disable the embedded data stream. This changes requires sensor kernel drivers that implement the upstream embedded data API based on V4L2 streams. As the API is under development and not merged in the upstream kernel yet, this breaks compatibility with the downstream Raspberry Pi kernel. Signed-off-by: Laurent Pinchart <[email protected]>
Use the stream formatting operator for V4L2VideoFormat instead of calling .toString() manually. As the .toString() member function is a wrapper around the stream formatting operator, this makes the code slightly more efficient. Signed-off-by: Laurent Pinchart <[email protected]>
The Vc4CameraData::platformConfigure() function configures inicam and the ISP. The corresponding configuration steps are interleaved, making it more difficult to read the code and follow the flow. Reorganize the function to improve readability. Signed-off-by: Laurent Pinchart <[email protected]>
Now that the CameraSensor class implements support for embedded data, use the corresponding API instead of retrieving the embedded data format from the subdev manually. This changes requires sensor kernel drivers that implement the upstream embedded data API based on V4L2 streams. As the API is under development and not merged in the upstream kernel yet, this breaks compatibility with the downstream Raspberry Pi kernel. Signed-off-by: Laurent Pinchart <[email protected]>
The mainline Unicam driver unconditionally creates the embedded data video device. Create the corresponding stream unconditionally. Drop the warning in case of mismatch between Unicam and the CamHelper, as this becomes a normal situation when the sensor doesn't support emebedded data. Signed-off-by: Laurent Pinchart <[email protected]>
The mainline Unicam driver creates a V4L2 subdevice, which needs to be configured. Create and open a corresponding V4L2Subdevice instance and configure the format on its sink pad in the platformConfigure() function. Presence of the Unicam subdev is required, to avoid extra complexity. This drops support for the driver from the Raspberry Pi downstream kernel. Users are expected to update their kernel to use the mainline Unicam driver. Signed-off-by: Laurent Pinchart <[email protected]>
The Unicam embedded data video device suffers from two issues with the upstream driver: - The driver uses the generic metadata V4L2_META_FMT_GENERIC_* formats, instead of the downstream V4L2_META_FMT_SENSOR_DATA. - The driver requires the width and height of the embedded data stream to be configured. Fix them both. Signed-off-by: Laurent Pinchart <[email protected]>
The V4L2_META_FMT_SENSOR_DATA format was used by the Raspberry Pi VC4 pipeline handler and the downstream Unicam kernel driver. Now that they have both moved to an upstream API, drop the format definition from the kernel header. This brings videodev2.h closer to the mainline kernel. Signed-off-by: Laurent Pinchart <[email protected]>
Enable embedded data usage with the IMX219 for testing purpose. Signed-off-by: Laurent Pinchart <[email protected]>
Also update the entity names to match the upstream driver. Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Companion PR of raspberrypi/linux#6437
Tested with imx219 on pi5 and pi4. On pi4 @jailuthra reported instabilities in the embedded data reading. Maybe that's the reason why embedded data were originally disabled on that sensor.