Vidio feed failures

[Secret spuk]

I have a question I'd like to ask of the techie, and survailance types.

When a camera, or video feed go's down is it investigated imediately as possible sabotage, or taken care of the next day/week?

Scenario; In a large metropolitan hospital located in a middle class area of the city. The hospital has some physical security, and a vidio survailance system. The person in the video room notice that a camera in a remote parking lot has gone off line. There is no overlaping camera. This hospital has a member of the IT team dedicated to maintaining the survailance system. But she only works 9 to 5 monday through friday.

What would be normal procedure?



Spuk

[HotelSecurity]

A slight correction - it's video.

[Secret spuk]

oops

Spuk

[Silva Consultants]

I think that most organizations would wait until the next business day to correct a camera failure of this type. They may send someone to the camera location when the failure occurs to see what's happening, but it is unlikely that they would call out a service technician at night or on a weekend to fix it.

Probably not the best practice from a security standpoint, but reality nonetheless.

[SecTrainer]

This is a very important camera, given its location and the well-known risks related to remote hospital parking lots. Someone should be dispatched to determine whether the camera has been deliberately disabled, perhaps in preparation for some other action, or whether the problem is something simple that can be handled immediately (e.g., simple vandalism like a cut cable or power failure), perhaps even by on-duty maintenance staff who in a hospital will typically have such basic skills. (Unfortunately, sometimes warranty provisions are voided if systems are repaired by "unauthorized" persons.) A power problem usually doesn't invoke the warranty provisions, though, since many contracts make the system owner responsible for maintaining power to the system.

If it's an IP camera, there could also be a very simple network problem that on-duty IT people could address. Maybe someone accidentally disconnected the camera from the hub or a connection panel, or mistakenly assigned its IP address to another node.

While I agree with Michael that organizations are reluctant to call contract service people out at their outrageous overtime rates, it's a whole lot cheaper than a lawsuit for "negligent security". In any event, I would not hesitate to call in the employee who is responsible for these systems to ascertain what can be done. (Indeed, being on-call would be one of the provisions in this person's job description. Limited M-F 9-5 coverage doesn't feed this bulldog.)

One other option would be to focus more security attention to this area temporarily until the camera can be replaced or repaired at the very first possible opportunity.

One thing is for sure: Once your organization knows that a security system has been significantly compromised (deliberately or not), doing nothing to compensate or to correct the problem is never an option unless you really like losing lawsuits. As I've already said, the risks to hospital employees and visitors in remote parking lots are very well known by everyone in the industry, including the tort lawyers.

[integrator97]

The reality is that in most situations a technician is not immediately notified. In 20 years in the industry, the only customer I remember ever calling after hours for camera problems was the county jail, though it may have happened with others and I've forgotten. And even then it was only if it was something extraordinary; mostly they would call the next morning or leave a message. And I've got banks, fortune 100 customers, all kinds.

[SecTrainer]

The reality is that in most situations a technician is not immediately notified. In 20 years in the industry, the only customer I remember ever calling after hours for camera problems was the county jail, though it may have happened with others and I've forgotten. And even then it was only if it was something extraordinary; mostly they would call the next morning or leave a message. And I've got banks, fortune 100 customers, all kinds.

That's right, but the steps I suggested can and should be taken in the meantime - check to see if it's a simple problem that the facility staff on duty can remediate immediately, or take steps to compensate for the fact that this important resource has been compromised until it can be fixed.

The one thing you don't want is finding yourself on a witness stand with this:

"So, then, when you realized that you no longer could observe your remote parking lot, where many employees and visitors must park, and knowing that hospital parking lots are attractive to violent criminals, what did you do?"

"Ummm....nothing."

"No one looked at the camera to see what was wrong with it?"

"No."

"No one checked the camera's network configuration to see if it had been changed, or perhaps a simple power supply or network connectivity problem?"

"No."

"You assigned no extra patrols to that area?"

"No."

"Any extra escorts?"

"No."

"You didn't close off that area of the parking lot or try to limit its use?"

"No."

"Then, did you at least warn employees and visitors parking there of the diminished security situation?"

"No."

KA-CHING!

Okay, so you don't want to eat the extra cost of calling a CCTV tech in on high-dollar hours, thereby risking pounds in order to save ha'-pennies. But man oh man, you don't just do NOTHING in a situation of this nature!

[Secret spuk]

Bravo Sec trainer. My personal thanks.

Spuk.

[Curtis Baillie]

That's right, but the steps I suggested can and should be taken in the meantime - check to see if it's a simple problem that the facility staff on duty can remediate immediately, or take steps to compensate for the fact that this important resource has been compromised until it can be fixed.

The one thing you don't want is finding yourself on a witness stand with this:

"So, then, when you realized that you no longer could observe your remote parking lot, where many employees and visitors must park, and knowing that hospital parking lots are attractive to violent criminals, what did you do?"

"Ummm....nothing."

"No one looked at the camera to see what was wrong with it?"

"No."

"No one checked the camera's network configuration to see if it had been changed, or perhaps a simple power supply or network connectivity problem?"

"No."

"You assigned no extra patrols to that area?"

"No."

"Any extra escorts?"

"No."

"You didn't close off that area of the parking lot or try to limit its use?"

"No."

"Then, did you at least warn employees and visitors parking there of the diminished security situation?"

"No."

KA-CHING!

Okay, so you don't want to eat the extra cost of calling a CCTV tech in on high-dollar hours, thereby risking pounds in order to save ha'-pennies. But man oh man, you don't just do NOTHING in a situation of this nature!Excellent - and I've seen that play out many times.

[CameraMan]

I've gotten three AM calls for burglar alarms and fire alarms- anything that makes a noise, basically- but never for a non-functional camera or DVR.

Plenty of "yeah, the cops are here and we can't figure out how to download the video onto a DVD", but never for just a non-functional camera.

[SIW Editor]

Great discussion.

Now consider this: How many users actually know when their cameras/DVRs/NVRs have gone down? I'd venture to guess that maybe 1% of end users have some sort of automatic notification process that a camera has gone down. But with the number of these units (even the DVRs) hooked to Internet, why are email alerts not standard?!? Cameraman, in your experience, is this a standard feature on DVRs and NVRs today?

[CameraMan]

There are DVRs with email alerts for failure, but often the failure it's warning you about prevents it from sending you the alert (most common example: power reset, power comes back on, the DVR simply isn't recording).

My favorite service phone call: customer calls my cell, tells me he had a break-in, but pressing "play" on his DVR is giving him six month old video (because his DVR power reset six months ago, and most of the cheap ones we were using back then did not automatically start recording again unless you actually pressed 'record').

Me: I'd love to help you, but I'm actually out of town.
Him: I'll pay you an extra hundred bucks.
Me: Umm... I'm out of town becuase I'm getting married.
Him: When?
Me: An hour.
Him: Oh. Congatulations, call me when you get back.

Me, two weeks later: So, this is a UPS, or uninteruptible power supply, you decided not to get one when we did the installation to save $80?
Him: Thanks. And here's a little something extra (slips me another $100) congratulations again!

[integrator97]

Most of the DVR's I use will email on video loss eg: camera loss.

And I will add in reference to my earlier post that most of my systems are not monitored by an officer or guard, but are passive recording.

I concur with CM that many don't look at video until they need it. I couldn't begin to count the number of times I made service calls on VCR's in ATM's where they needed the video, and I'd go back 2 or 3 months or more to find a tape with good video. The good thing is DVR's are much more reliable. But it was easier to train people to rewind for 10 seconds and push play to check video when changing tapes.

[SecTrainer]

There is far too much "magical thinking" on the part of the public when it comes to CCTV, access control, intrusion and fire systems. Some of this is no doubt due to the modern culture of techno-religion. We believe systems can "walk on water". Some is sheer ignorance. But I also think we bear some responsibility for the problem:

1. Some make exaggerated claims for their systems. This can happen inadvertently (by implication) when the sales pitch makes little reference to system limitations, user responsibilities, maintenance requirements/costs, etc. - in other words, the "downside" that every system has. There have been court cases in which this failure has led to the assessment of liability, incidentally! The seller is responsible for affirmatively informing customers about system limitations or anything else that is "reasonably material" to the customer's purchasing decision and his understanding of what he is buying (and what he is NOT buying). This is based on a presumption of "superior knowledge" on the part of the seller, so "He never asked me about that" is NOT a defense. Even if it were, it's not good business, either.

2. User training isn't done properly. Hand off the system documents, wiring schematic and the user manual and the customer should be good to go. Even those who have a decent initial training program often don't have provisions for ongoing training to allow for things like employee turnover. Another training failure involves the failure to involve all of the employees who interface with the system (e.g., IT people).

3. Followup isn't conducted regularly and on-site. Maybe the customer gets a phone call or visit after 30 days or so, and that's it unless the customer calls with a problem. (This is unfortunate because followup provides an opportunity not only to head off problems before they reach critical mass, but can also present opportunities for up-selling.) Followup is also the only way to learn that the customer has done something (perhaps not even related to the system itself) that compromises the system. There's always equipment being installed or replaced (surges, EMI, ground loops, etc.), and so a run of unshielded wire is no longer appropriate. Or, there are changes in IP network configuration that impact traffic loads, etc. Perhaps an item of gear we sold was faulty and starts to fail intermittently shortly after installation and long before expected. In multiple-occupancy venues, maybe it's something that another tenant or the landlord does that our customer doesn't even know about. Maybe an adjoining facility installs new lights in their parking lot which "pollutes" our client's space and requires adjustment to his cameras. Maybe the power grid itself is experiencing changes.

How do we know any of these things without dropping by periodically, just to make sure our systems are still functioning according to the original specs? We can't rely on the customer to know there's a problem until it's too late, as mentioned by others above. That's OUR obligation to the customer! (And customers do not believe us when we tell them that merely turning a camera a matter of an inch or two can completely screw up the image, or that a little dirt in a card swipe track can degrade access throughput.)

4. System integration isn't tested thoroughly enough to account for all reasonably foreseeable contingencies, or without a sufficient understanding of the other system(s) involved.

5. We overlooked some "little" thing in our design that later turns out to be not so little.

The list goes on...

[integrator97]

1. Some make exaggerated claims for their systems.
My favorite is "I want to keep 60 days of video on my 12 camera system. The other company said I only need an 80 gig DVR." Yea, it'll do that. Of course you only get one picture per camera every 2 seconds, and it's only CIF, 320x240, so if you blow it up it's worthless. But yea, they didn't lie to you.


4. System integration isn't tested thoroughly enough ... I can't tell you how many techs I've had to retrain to test every point on a system, not just every zone. Just because the center window in that bank of 3 works, doesn't mean they all do.

[SecTrainer]

My favorite is "I want to keep 60 days of video on my 12 camera system. The other company said I only need an 80 gig DVR." Yea, it'll do that. Of course you only get one picture per camera every 2 seconds, and it's only CIF, 320x240, so if you blow it up it's worthless. But yea, they didn't lie to you.

...

...or at one picture per day, Mr. Scrooge, you could get by with a floppy diskette and save yourself a bundle! :D

Actually, I think the other company did lie - by omission.

[CameraMan]

Actually, I think the other company did lie - by omission.

Sorry, but this had to be repeated.