qsee h.265
  • I have 24 cameras running in SS and several different models of qsee cameras, most of them supporting h.265. Any time I try to enable h.265 on the stream I'll get a nice clear picture in SS for a moment, then bad data, timeout and the feed will start over. I've tried playing with the FPS, bitrate, etc on the cameras but the results are the same. I see a couple other threads of people getting grey or green screens but not quite this issue. Any ideas?
  • Could you please check the log (File menu in SecuritySpy -> Open log) and let me know the exact error messages/codes you are getting?
  • Sure thing. Here is what I get in the log when I switch a camera to h.265.
    I change to h.265 on the camera and save that setting, and I get a successful stream in SS for a moment. Then it goes dark, and I get these errors. It will come back after a moment, but then the cycle repeats.

    11/21/2019 09:30:20: Error from camera "Blvd Ext Driveway North", it will be closed. 5.1.0,22185,71042 Failed to decompress incoming video frame. Decompression failed.

    11/21/2019 09:35:52: Error from camera "Blvd Ext Driveway ", it will be closed. 5.1.0,22185,-12909 Failed to decompress incoming video frame. Image data decompression failed - bad data.
  • For what it's worth, I played around with the image settings on the cameras some more and I can get the stream to stay up in SS if I set the camera to h.265 with a value of 1 FPS. Any other FPS value will cause the issue described above.
  • These errors are being returned by Apple’s VideoToolbox framework, which SecuritySpy uses for decompression of the incoming video feeds, and they indicate that the data that the camera is sending cannot be decoded. I’m not sure if there will be anything that can be done about this from our end unfortunately. One thing to check is if there is a firmware update for the cameras from the manufacturer?

    Do the cameras work OK in H.264 mode? If so, then this may be the only option for these cameras, at least for the time being.
  • Interesting. On at least one of the models I know I have the latest firmware. I will check the other model that supports h.265. They do all work fine in h.264
  • It appears everything is on the latest firmware (there is one model - I only have two of this particular camera - that I haven't been able to find the latest firmware for yet. Qsee doesn't make that easy). Interesting enough, I updated my SS server to Catalina this morning to see if it helped with the other issue I was seeing where SS would crash due to that known issue in MacOS. I bumped the FPS value up on the 6 cameras set to h.265 and after the Catalina upgrade they are all stable so far. So the issue appears to be tied to VideoToolbox pre-Catalina for me at least
  • That's great to hear! It's likely that there is a new/better H.265 decoder in Catalina. This is very useful information, we'll now know what to advise other users in the future in the same situation.
  • Makes sense! I was hesitant to install Catalina as I had a couple issues on my iMac with it (user login/switching was terribly slow, and overall performance took a hit). I ended up installing the 10.15.2 beta a couple days ago on the iMac and that cleared up those issues. I installed the general release on my Mac mini for SS and didn't run into any of those issues though, and it fixed two problems for me so I'll take it!
  • Now I wonder if I spoke too soon. Things were nice and stable for some hours after that upgrade, then some of the streams started that drop and reconnect cycle again. It’s strange though, it isn’t quite like it was before. Before the Catalina upgrade it would happen quickly- maybe 5 seconds, disconnect and reconnect, another 5 seconds and repeat. Now it goes several minutes. Again it is only h265, if I switch to h264 things are stable. But, performance with h265 is soooo much better. CPU usage is down, and the gpu isn’t constantly spiked as with h264, so my remote viewing is much more smooth.
  • This continues to get more bizarre. The streams all stayed pretty stable during the day yesterday. Last night there were a couple of bounces, then this morning two of the streams had disconnected but had not come back. I was able to login to the cameras directly. I rebooted one and the stream resumed in SS. Before I rebooted the other I thought I would try connecting to the camera directly via rtsp in VLC. That came up fine. I disabled and re-enabled the camera in SS and the stream resumed (no reboot).
    I haven't been able to find an action that definitively causes a stream to freeze up, but I have been able to get it to happen a couple times since. I've tried switching a camera that is doing it back to h.264, and it would still happen (where previously h.264 was quite stable). It almost seems like if I have any h.265 streams coming in to SS it impacts stability, but not necessarily on that camera.
    I've had several camera streams open in VLC directly (against the cameras' IPs) and those have been fine. I've considered picking up a camera from a different vendor where someone has had good luck with h.265, setting all mine back to h.264 and seeing if the new camera remains stable - if there is something with the qsee cameras factoring in here hopefully that would expose it. The strange thing is the qsee cameras are actually manufactured buy a couple different OEMs - the older ones I have are made by Dahua I believe, and the newer models are Uniview. Everything is the latest firmware based on what I am able to find.
    Other than that I am sort of running out of ideas here. I did introduce the Catalina upgrade as well, which initially seemed to really help (not just on the h.265 issue, but with that startup error that caused the crash loop as well - that remains stable at this time). I hate to go backward from here and recreate that crash on startup, but the instability of the cameras isn't exactly ideal either.. but, I don't know that Catalina is to blame on that.
  • One possible factor here could be whether the video is being decoded in hardware or in software. One method could be more reliable than the other. To check this, go to the Debug menu (via the SecuritySpy menu) and check the "Disable Hardware Accelerated Video Decoding" option. Then quit and reopen SecuritySpy. Run it like this for a while to see if this is more reliable, it will be an interesting test.
  • Sounds good, I switched that and will let it run for a bit. As expected that bumped my cpu usage from about 35% to around 85%.
    I setup another test for this last night that yielded some interesting results. I installed windows 10 via boot camp on my MacBook so I could keep the mini running as-is for the moment. I installed I spy connect on windows 10 and added all the same cameras to it.
    First thing I noticed was that the two cameras I have that display a 2/3 grey screen in SS (I just have 2 of that particular model and they both do this) when set to h.265 display fine in ispyconnect.
    It isn’t a perfect comparison as the MacBook has a Radeon 560 in it. However, once I had the cameras all added I played around with config a little bit both in ispyconnect and on the camera feeds. There is an option on the RTSP config per camera in ispyconnect to use gpu. If I don’t enable that you can see the impact of adding each camera on cpu usage, and once they are all added I hit about 70% overall. If I do check that the cpu is at maybe 5% with all the cameras running, but the gpu stays at 100%. I have this running side by side currently and none of the feeds- h 264 or 265- have dropped l. They have several times on the mini in SS, though they have reconnected themselves. Also interesting is the MacBook is connected to WiFi and is still stable (obviously not what you’d want for a permanent install, but interesting) and the mini is connected to gigabit Ethernet
  • I've had SS running with the "disable hardware accelerated video decoding" option checked for a little over 24 hours now. Fairly interesting results.
    The issue I’ve seen with the two cameras of the specific model where the stream will come through but be 2/3 grey stayed the same (though the same cameras work fine on the test install I did on my MacBook).
    The other streams seem to have stayed fairly stable, but a couple of the cameras would still freeze, drop and reconnect. In the iOS app this would show as grey and blocky in the preview for a moment then the normal image returns. I was able to take a few screenshots of that.
    With the hardware acceleration option disabled I did see the cpu usage run very high as expected. That seemed to lead to some other lags and stability issues on the system.
    I just tried enabling hardware acceleration again and the first two times restarting SS after that change I received the libdispatch crash. The third time it started back up. I am not sure if this is related to the hardware acceleration or is coincidental.
    I’ve let the test install of ispyconnect continue to run on my MacBook with the same cameras and it has remained stable.
  • @Ben Disable Hardware Accelerated Video Decoding in the Debug and things seem to be working just fine - will let it run and see how it goes.
  • @Ben in my pursuit of a solution to the stability/performance of my camera streams I did some additional shuffling of installations and addition of some hardware.

    First, I moved my SS install over to my 2017 27" iMac, which has a built-in Radeon Pro 850. This has been flawless for 5 days now, and the gpu utilization is very comfortable with all the cameras configured normally using h.264. I use the iMac for work extensively, so I would prefer to have SS and other home server/automation functionality on the mini, if I can get performance where it needs to be.
    To that end, I came across a deal on an egu setup and picked it up to see if I could get anywhere with it. As expected and noted in other threads checking the "prefer egpu" option on the SS application doesn't appear to impact where stream decoding takes place - still maxes out the intel graphics on my mini. However, functionality that uses metal does engage the egpu - when I open the all cameras view I see some usage show up for SS on the AMD.

    I certainly don't know the ins and outs of Videotoolbox, but from what I've been reading the latest version of VLC does use it on MacOS. As an experiment I opened the stream from one of my cameras (h.264) with VLC, making sure hardware acceleration is on, and I see VLC hit the AMD eGPU. That seems to be the case whether or not I have the "prefer eGPU" option checked on the VLC application, so it seems to be making the decision to use the higher powered gpu on its own (either within VLC, or once it has engaged Videotoolbox).

    So, I've reached a couple tentative conclusions based on what I've done so far. The bottleneck on the mini is the gpu, and all else being equal (camera settings, SS version, Catalina, etc) everything works fine on the iMac as it has the Radeon built-in to use. If I could get the eGPU working on the mini I would be fantastic shape and that box would run more cameras than I will likely ever install.. but I'm not sure where else to go on that at the moment if Videotoolbox, SS or a combo thereof are only sticking to the intel graphics.
  • Thanks for the detailed testing report, this is all very interesting.

    I'm not sure why the work doesn't get shifted to the eGPU when you have set the option to prefer it. Unfortunately there is no way to tell VideoToolbox where to do the decoding: SecuritySpy can only ask for software or hardware. If it asks for hardware, it's then up to the system where to schedule the task.

    VLC uses Libav for its video decoding, and it seems that with Libav you can choose which hardware to prefer. But unfortunately we can't use Libav in SecuritySpy, mainly due to licensing restrictions, but also because it would be a lot of work to integrate, with uncertain results and with the possibility of breaking other things.

    So I think the best thing to suggest at this point would be to reduce the frame rates of your cameras to keep the CPU and GPU usage moderate on your Mac mini, so that it can handle the load.
  • Thanks Ben. It does seem strange and counterintuitive that MacOS would not use the available horsepower when hardware has been selected. But if there isn't a way to tell it which GPU to use then there isn't much left to try!
    I have the frame rates dialed back pretty far and it is hanging in there right now, but I have several more cameras to install and I just dont think the mini will get me there, at least not with tolerable quality on the recordings. I think I'm going to do a little bit of a computer shuffle - put the MacBook back to the way it was originally, move SS off the mini and onto the iMac (since I know that runs well with the Radeon 580), then get rid of the mini and replace the iMac with a new work system.
  • It does sound like your Mac mini may not be up to the job, but before you buy new hardware we can try one more thing: if we were to implement a more flexible control in SecuritySpy (e.g. "decode half the cameras in hardware"), this may spread the load better between software vs. hardware and improve the overall performance. What do you think?

    You also mentioned above a "libdispatch crash" - having not seen the crash log, I'm not sure if this is to do with decoding or not. If you can email me the crash log I can take a look. While you're at it, please email me the general log file too (File menu in SecuritySpy -> Open Log).

    Other questions:

    What is the exact model of Mac mini you are using?
    Are you using SecuritySpy's AI features at all? These can use significant CPU and GPU.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!