[Support]: ffmpeg still decoding stream with detection disabled

[track0x1]

Describe the problem you are having

I have two same model cameras (Wyze Cam v3, being broadcast through docker-wyze-bridge)- one was setup with detection and a low-res substream (for detection purposes), while the other is setup without detection (just recording). Both cameras are outputting the same resolution. I noticed the one without detection had significantly higher average CPU usage 50%+ while the other was consistently below 10%. This feels weird to be expected behavior because if I enable detection and setup the low-res substream on the other camera, its CPU usage drops to that of the other camera.

Version

0.12.1-367D724

Frigate config file

mqtt:
  enabled: true
  host: ***
  user: ***
  password: ***

detectors:
  coral:
    type: edgetpu
    device: usb

birdseye:
  enabled: false

ffmpeg:
  hwaccel_args: preset-rpi-64-h264

go2rtc:
  streams:
    side_yard:
      - rtsp://192.168.0.10:8559/side-yard
    den:
      - rtsp://192.168.0.10:8559/camv3

cameras:
  den:
    ffmpeg:
      input_args: -correct_ts_overflow 0
      inputs:
        - path: rtsp://127.0.0.1:8554/den
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      enabled: false
  side_yard:
    ffmpeg:
      input_args: -correct_ts_overflow 0
      inputs:
        - path: rtsp://192.168.0.10:8559/side-yard-sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/side_yard
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      enabled: true
      width: 640
      height: 360

model:
  labelmap:
    15: animal
    16: animal
    17: animal
    18: animal
    19: animal
    20: animal
    21: animal
    22: animal
    23: animal
    24: animal

record:
  enabled: true
  retain:
    days: 2
snapshots:
  enabled: true
objects:
  track:
    - person
    - animal
  filters:
    person:
      threshold: 0.72

Relevant log output

n/a

FFprobe output from your camera

CAMERA 1:
[rtsp @ 0x55b8d21840] method SETUP failed: 461 Unsupported Transport
Input #0, rtsp, from 'rtsp://192.168.0.10:8559/side-yard':
  Metadata:
    title           : No Name
  Duration: N/A, start: 2.250956, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 20 fps, 20 tbr, 90k tbn, 40 tbc

CAMERA 2:
[rtsp @ 0x559a00b840] method SETUP failed: 461 Unsupported Transport
Input #0, rtsp, from 'rtsp://192.168.0.10:8559/camv3':
  Metadata:
    title           : No Name
  Duration: N/A, start: 1.992944, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 20 fps, 20 tbr, 90k tbn, 40 tbc

Frigate stats

{"cpu_usages":{"%Cpu(s):":{"cpu":"id,","mem":"0.1"},"1":{"cpu":"0.0","mem":"0.0"},"107":{"cpu":"0.0","mem":"0.0"},"110":{"cpu":"11.6","mem":"1.7"},"117":{"cpu":"0.0","mem":"0.2"},"134":{"cpu":"0.0","mem":"0.1"},"135":{"cpu":"0.0","mem":"0.1"},"136":{"cpu":"0.0","mem":"0.1"},"137":{"cpu":"0.0","mem":"0.1"},"15":{"cpu":"0.0","mem":"0.0"},"17":{"cpu":"0.0","mem":"0.0"},"24":{"cpu":"0.0","mem":"0.0"},"25":{"cpu":"0.0","mem":"0.0"},"26":{"cpu":"0.0","mem":"0.0"},"27":{"cpu":"0.0","mem":"0.0"},"28":{"cpu":"0.0","mem":"0.0"},"282":{"cpu":"0.3","mem":"1.0"},"288":{"cpu":"0.3","mem":"0.1"},"289":{"cpu":"0.0","mem":"1.3"},"29":{"cpu":"0.0","mem":"0.0"},"291":{"cpu":"2.0","mem":"1.1"},"295":{"cpu":"1.3","mem":"1.1"},"297":{"cpu":"0.0","mem":"0.5"},"299":{"cpu":"2.0","mem":"1.2"},"30":{"cpu":"0.0","mem":"0.0"},"300":{"cpu":"0.0","mem":"0.5"},"301":{"cpu":"4.3","mem":"1.1"},"303":{"cpu":"2.7","mem":"1.1"},"31":{"cpu":"0.0","mem":"0.0"},"311":{"cpu":"52.8","mem":"0.8"},"312":{"cpu":"6.6","mem":"0.6"},"316":{"cpu":"1.7","mem":"0.6"},"32":{"cpu":"0.0","mem":"0.0"},"41":{"cpu":"0.0","mem":"0.0"},"42":{"cpu":"0.0","mem":"0.0"},"568":{"cpu":"0.0","mem":"0.0"},"606":{"cpu":"0.0","mem":"0.0"},"611":{"cpu":"0.3","mem":"0.0"},"616":{"cpu":"0.0","mem":"0.0"},"78":{"cpu":"0.0","mem":"0.0"},"81":{"cpu":"0.0","mem":"0.0"},"82":{"cpu":"0.0","mem":"0.0"},"89":{"cpu":"3.3","mem":"0.2"},"MiB":{"cpu":"6534.0","mem":"avail"},"PID":{"cpu":"%CPU","mem":"%MEM"},"Tasks:":{"cpu":"stopped,","mem":"0"},"top":{"cpu":"users,","mem":"load"}},"den":{"camera_fps":5.1,"capture_pid":301,"detection_enabled":0,"detection_fps":0.0,"ffmpeg_pid":311,"pid":295,"process_fps":5.1,"skipped_fps":0.0},"detection_fps":0.0,"detectors":{"coral":{"detection_start":0.0,"inference_speed":20.4,"pid":289}},"gpu_usages":{"rpi-v4l2m2m":{"gpu":-1,"mem":-1}},"service":{"last_updated":1699299703,"latest_version":"0.12.1","storage":{"/dev/shm":{"free":131.7,"mount_type":"tmpfs","total":134.2,"used":2.5},"/media/frigate/clips":{"free":337300.0,"mount_type":"ext4","total":472246.5,"used":115696.5},"/media/frigate/recordings":{"free":337300.0,"mount_type":"ext4","total":472246.5,"used":115696.5},"/tmp/cache":{"free":998.9,"mount_type":"tmpfs","total":1000.0,"used":1.1}},"temperatures":{},"uptime":247,"version":"0.12.1-367d724"},"side_yard":{"camera_fps":6.0,"capture_pid":303,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":312,"pid":299,"process_fps":6.0,"skipped_fps":0.0}}

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

Wyze Cam V3 via docker wyze bridge

Any other information that may be helpful

No response

[NickM-27]

Even when detect is disabled the stream is still decoded, if you don't provide a sub stream then the main stream will be decoded which will increase CPU usage.

[track0x1]

@NickM-27 Is this expected behavior or a known issue? Why does Frigate need to decode the stream if I’m not using it for detection purposes? If it’s expected then perhaps we add a note to the docs that states the stream will be decoded regardless and this can significantly impact CPU usage?

[NickM-27]

The decoded stream is used for other things like:

1. motion detection
   a. for recordings
   b. for the motion detector in HA integration
   c. for birdseye in motion mode
2. the latest.jpg and other various APIs / live view
3. for birdseye in general

There is a feature request to make this not always be the case but as outlined above many things depend on this decoded stream so it will be complicated to implement.

The docs can be updated