Unstable Event Stream after 5.5 upgrade

TwitchCaptain
TwitchCaptain
in SecuritySpy

Hello!

My dad hit me up recently about flaky behavior from his cameras, so I began investigating. What I found is that almost every time a request to a video ++stream is made the web server in SecuritySpy crashes and restarts. There is nothing in either log file about this crash/restart of the web server. If often returns a 404 for a valid resource when this happens, but sometimes the request are quick enough to return a well known 'connection refused' message.

Example 1 (logs from my custom app):

[MSGS] 2022/08/13 13:51:39 [NZYB] Telegram Received from 1581483199 (admin:true, ignored:false), size: 9, cmd: vid
[ERROR] 2022/08/13 13:51:39 SecuritySpy Event Stream Disconnected
[MSGS] 2022/08/13 13:51:39 [NZYB] Telegram: Sending Video (/private/tmp/chat_command_NZYB_South.mp4, 0.13Mb) to 1581483199
[MSGS] 2022/08/13 13:51:42 [raAk] Telegram Received from 1581483199 (admin:true, ignored:false), size: 4, cmd: pics
2022/08/13 13:51:42 [ERROR] [raAk] cam.SaveJPEG: getting jpeg: getting image: http request: Get "http://127.0.0.1:8000/++image?auth=<snip>&cameraNum=2": dial tcp 127.0.0.1:8000: connect: connection refused

Example 2:

[MSGS] 2022/08/13 13:51:23 [NV4B] Telegram Received from 1581483199 (admin:true, ignored:false), size: 3, cmd: vid
[ERROR] 2022/08/13 13:51:23 SecuritySpy Event Stream Disconnected
2022/08/13 13:51:23 [ERROR] [NV4B] cam.SaveVideo: subcommand failed: exit status 1: rtsp://127.0.0.1:8000/++stream?auth=<snip>%3D%3D&cameraNum=6&codec=h264&height=600&quality=20&req_fps=5: Invalid data found when processing input
2022/08/13 13:51:23 [ERROR] [NV4B] cam.SaveVideo: subcommand failed: exit status 1: rtsp://127.0.0.1:8000/++stream?auth=<snip>%3D%3D&cameraNum=0&codec=h264&height=600&quality=20&req_fps=5: Invalid data found when processing input
2022/08/13 13:51:23 [ERROR] [NV4B] cam.SaveVideo: subcommand failed: exit status 1: rtsp://127.0.0.1:8000/++stream?auth=<snip>%3D%3D&cameraNum=1&codec=h264&height=600&quality=20&req_fps=5: Invalid data found when processing input
2022/08/13 13:51:23 [ERROR] [NV4B] cam.SaveVideo: subcommand failed: exit status 1: rtsp://127.0.0.1:8000/++stream?auth=<snip>%3D%3D&cameraNum=2&codec=h264&height=600&quality=20&req_fps=5: Invalid data found when processing input
[MSGS] 2022/08/13 13:51:23 [NV4B] Telegram: Sending Video (/private/tmp/chat_command_NV4B_South.mp4, 0.07Mb) to 1581483199
[MSGS] 2022/08/13 13:51:24 [NV4B] Telegram: Sending Video (/private/tmp/chat_command_NV4B_Front.mp4, 0.07Mb) to 1581483199
[INFO] 2022/08/13 13:51:28 SecuritySpy Event Stream Connected!

Please let me know if I can provide any more information about this. It happens so regularly I assume it can be easily reproduced with a simple bash script. If not, let me know and I'll cook something up for you.

In the most basic usage: connect to the event stream with 1 thread, and make video stream requests from another (using ffmpeg and rtsp/rtsps). One of my systems has a valid SSL cert; this problems happens with rtsp and rtsps.

Thanks,

-captain

Comments

  • TwitchCaptain
    TwitchCaptain
    edited August 2022

    I just downgraded to Security Spy 5.3.4 and the problem is gone. Luckily I still had a copy because you don't seem to make that version available for download on your website. Hoping for a quick fix!

    This is what the logs are supposed to look like (no errors):

    [INFO] 2022/08/13 14:22:33 SecuritySpy Event Stream Connected!
[MSGS] 2022/08/13 14:22:59 [NjGM] Telegram Received from 1581483199 (admin:true, ignored:false), size: 3, cmd: vid
[MSGS] 2022/08/13 14:23:06 [NjGM] Telegram: Sending Video (/private/tmp/chat_command_NjGM_South.mp4, 0.40Mb) to 1581483199
[MSGS] 2022/08/13 14:23:09 [NjGM] Telegram: Sending Video (/private/tmp/chat_command_NjGM_Front.mp4, 0.45Mb) to 1581483199
[MSGS] 2022/08/13 14:23:11 [NjGM] Telegram: Sending Video (/private/tmp/chat_command_NjGM_Mailbox.mp4, 0.60Mb) to 1581483199
[MSGS] 2022/08/13 14:23:14 [NjGM] Telegram: Sending Video (/private/tmp/chat_command_NjGM_Back.mp4, 0.51Mb) to 1581483199
[MSGS] 2022/08/13 14:23:16 [NjGM] Telegram: Sending Video (/private/tmp/chat_command_NjGM_Shop.mp4, 0.55Mb) to 1581483199
[MSGS] 2022/08/13 14:23:18 [NjGM] Telegram: Sending Video (/private/tmp/chat_command_NjGM_Corner.mp4, 0.55Mb) to 1581483199
  • Ben
    Ben

    Hi Captain, I think this one is best dealt with directly - I've sent you an email asking for a few things. I'm sure we'll be able to locate and fix the problem.

Categories