Curtis,
Have you heard of an open source project called go2rtc? It is used to process video streams via ffmpeg and convert to webrtc compatible format, as far as I know. Might be useful to help preprocess your video input. Supports not only Intel VAAPI, but also Nvidia and AMD. I realize you guys have put considerable effort into your existing solution. Still wanted to point this one out. It is used in the FrigateNVR project, and works quite well there.
Thanks for the information!