Picamera2 with custom compiled ffmpeg on raspberry pi 5

[matthiasharrer]

Hey all,

I am facing some issues with making the picamera work on raspberry pi 5 (with the libav encoders) with a custom version of ffmpeg. Maybe someone could guide me to the specific features that are needed in ffmpeg? Or what else might be the issue.

I was able to run the same script on a fresh raspberry pi os install with picamera2 installed from apt.

I am trying to record an mjpeg encoded video with something like the following code:

from picamera2 import Picamera2, encoders, outputs
from time import sleep

camera = Picamera2()

config = camera.create_video_configuration(
    main={"size" : (1640, 1232), "format": "BGR888"},
    lores={"size": (640, 480)},
    encode="lores",
    controls={"FrameRate": 20}
)
camera.configure(config)

camera.start_recording(
    encoder=encoders.MJPEGEncoder(framerate=20),
    output=outputs.FileOutput("video.mjpeg")
)

sleep(10)

camera.stop_recording()
camera.close()

I installed libcamera from apt, but picamera2 + av with pip inside a venv (including system side packages for libcamera).
pyav (10.0.0) was compiled from sources.

I am getting the following error:

Traceback (most recent call last):
  File "[...]/test_camera.py", line 17, in <module>
    camera.start_recording(
  File "[...]/.venv/lib/python3.11/site-packages/picamera2/picamera2.py", line 1743, in start_recording
    self.start_encoder(encoder, output, pts=pts, quality=quality, name=name)
  File "[...]/.venv/lib/python3.11/site-packages/picamera2/picamera2.py", line 1683, in start_encoder
    _encoder.start(quality=quality)
  File "[...]/.venv/lib/python3.11/site-packages/picamera2/encoders/encoder.py", line 227, in start
    self._start()
  File "[...]/.venv/lib/python3.11/site-packages/picamera2/encoders/libav_mjpeg_encoder.py", line 59, in _start
    self._stream.codec_context.qmin = self.qp
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'av.video.codeccontext.VideoCodecContext' object has no attribute 'qmin'

[simalessandri]

I'm encountering the same issue. My code runs perfectly on Raspberry Pi 4, but fails on Raspberry Pi 5. Have you discovered any workarounds?

[matthiasharrer]

I have not so far .. are you also using an ffmpeg version that you compiled yourself?
The major difference between Pi 4 and Pi 5 is, that I think on the Pi 4 it uses a hardware accelerated encoder, which is not available on Pi 5. So only there it will try to use the ffmpeg based encoder.

[matthiasharrer]

I did a bit more research and got it to run and record the video, however I don't really understand why the changes would be necessary and what the potential problems could be..

Just modifying the lines in libav_mjpeg_encoder.py that are raising the errors in the following way makes it work:

"""This is a base class for a multi-threaded software encoder."""

from fractions import Fraction

import av

from picamera2.encoders.encoder import Encoder, Quality

from ..request import MappedArray


class LibavMjpegEncoder(Encoder):
    """Encoder class that uses libx264 for h.264 encoding."""

    def __init__(self, bitrate=None, repeat=True, iperiod=30, framerate=30, qp=None):
        """Initialise"""
        super().__init__()
        self._codec = "mjpeg"
        self.repeat = repeat
        self.bitrate = bitrate
        self.iperiod = iperiod
        self.framerate = framerate
        self.qp = qp

    def _setup(self, quality):
        # If an explicit quality was specified, use it, otherwise try to preserve any bitrate/qp
        # the user may have set for themselves.
        if quality is not None or \
           (getattr(self, "bitrate", None) is None and getattr(self, "qp", None) is None):
            quality = Quality.MEDIUM if quality is None else quality
            QP_TABLE = {Quality.VERY_LOW: 31,
                        Quality.LOW: 15,
                        Quality.MEDIUM: 10,
                        Quality.HIGH: 5,
                        Quality.VERY_HIGH: 3}
            self.qp = QP_TABLE[quality]

    def _start(self):
        self._container = av.open("/dev/null", "w", format="null")
        self._stream = self._container.add_stream(self._codec, rate=self.framerate)

        self._stream.codec_context.thread_count = 8
        self._stream.codec_context.thread_type = av.codec.context.ThreadType.FRAME

        self._stream.width = self.width
        self._stream.height = self.height
        self._stream.pix_fmt = "yuv420p"

        # This is all rather arbitrary but comes out with a vaguely plausible a quantiser. I
        # found that the sqrt of the quantiser times the bitrate was approximately constant with
        # the value 64000000 for a 1080p30 stream, though obviously it will depend on content,
        # and probably the phase of the moon.
        if self.qp is None:
            reference_complexity = 1920 * 1080 * 30
            actual_complexity = self.width * self.height * getattr(self, "framerate", 30)
            reference_bitrate = self.bitrate * reference_complexity / actual_complexity
            self.qp = max(min(round(64000000 / reference_bitrate) ** 2, 127), 1)
        # Comment out qmin / qmax non-existing property access 
        # self._stream.codec_context.qmin = self.qp
        # self._stream.codec_context.qmax = self.qp
        self._stream.codec_context.color_range = 2  # JPEG (full range)
        self._stream.codec_context.flags |= av.codec.context.Flags.QSCALE

        self._stream.codec_context.time_base = Fraction(1, 1000000)

        FORMAT_TABLE = {"YUV420": "yuv420p",
                        "BGR888": "rgb24",
                        "RGB888": "bgr24",
                        "XBGR8888": "rgba",
                        "XRGB8888": "bgra"}
        self._av_input_format = FORMAT_TABLE[self._format]

    def _stop(self):
        for packet in self._stream.encode():
            self.outputframe(bytes(packet), packet.is_keyframe, timestamp=packet.pts)
        self._container.close()

    def _encode(self, stream, request):
        timestamp_us = self._timestamp(request)
        with MappedArray(request, stream) as m:
            # Removed non-existing width paramter
            frame = av.VideoFrame.from_ndarray(m.array, format=self._av_input_format)
            frame.pts = timestamp_us
            for packet in self._stream.encode(frame):
                self.outputframe(bytes(packet), packet.is_keyframe, timestamp=packet.pts)

I do still wonder where those properties / kwargs are coming from.. Looking at the PyAV documentation/code I cannot find them

[matthiasharrer]

A as I was running into this again myself .. don't forget to compile pyav from source poetry run pip install av --no-binary av

[hugokernel]

Same error here but without custom compiled ffmpeg (I just use picamera2.encoders.MJPEGEncoder)

I removed the av package (pip uninstall av) and error disappears.

[agrear]

Checkout the official Picamera2 documentation.

In section 7.1 Encoders the following is stated:

On Pi 4 and earlier devices there is dedicated hardware for H264 and MJPEG encoding. On a Pi 5, however, these codecs are implemented in software using FFmpeg libraries. The performance on a Pi 5 is similar or better, and the images are encoded with better quality. The JPEG encoder runs in software on all platforms.

Further in section 7.1.2. JpegEncoder:

The JpegEncoder class implements a multi-threaded software JPEG encoder, which can also be used as a motion JPEG ("MJPEG") encoder.

And finally in section 7.1.3. MJPEGEncoder:

The MJPEGEncoder class implements an MJPEG encoder using the Raspberry Pi’s in-built hardware, accessed through the V4L2 kernel drivers.

AFAIK you are not supposed to use the MJPEGEncoder on a Raspberry Pi 5 but instead use the JpegEncoder for MJPEG encoding.