Whatever happened to redunancy?
Vandals managed to cut off a large portion of phone service in the southern Santa Clara County, parts of San Benito County, and much of Santa Cruz County, for most of the day. That included 911, cell coverage, and most local coverage; I could call Scotts Valley but not Aptos, for instance. Most numbers I tried were just busy.
One of the things that worries me about this is that the first question one asks when running infrastructure is, "Is there a single place I could cut that would bring down a big area?" As far as I can tell, this outage was caused by a cut in a single location. While it did likely require inside knowledge (a special tool to open the manhole cover, and knowledge of where it would do the most harm), it shouldn't actually be that easy to cause a major disruption. I hope people are asking Verizon some very hard questions right now. We're lucky that no one died or was injured due to being cut off from 911 today.*
It may not be possible to avoid an SPOF, but if there are SPOFs... then they need better security than "heavy manholes requiring special tools."
(Poking around, there were two cable cuts, but the second, in San Carlos County, didn't cause an outage and wasn't part of the South Bay outage.)
*Given how hard I used to work to avoid SPOFs when the consequences of failure were... people not seeing their pictures up on the Interwebs, oh noes! I am a little alarmed that all of the physical assets are in one basket. And yes, I actually spent time wondering, in a serious fashion, how hard it would be for a guy with a pair of cable snips, or perhaps even a bomb, to take out my site. Yeah, Fiber Is Hard To Pull, but then maybe 911, at least, should have local redundancy. Why is Santa Cruz's 911 service dependant on a fiber bundle running through San Jose?
One of the things that worries me about this is that the first question one asks when running infrastructure is, "Is there a single place I could cut that would bring down a big area?" As far as I can tell, this outage was caused by a cut in a single location. While it did likely require inside knowledge (a special tool to open the manhole cover, and knowledge of where it would do the most harm), it shouldn't actually be that easy to cause a major disruption. I hope people are asking Verizon some very hard questions right now. We're lucky that no one died or was injured due to being cut off from 911 today.*
It may not be possible to avoid an SPOF, but if there are SPOFs... then they need better security than "heavy manholes requiring special tools."
(Poking around, there were two cable cuts, but the second, in San Carlos County, didn't cause an outage and wasn't part of the South Bay outage.)
*Given how hard I used to work to avoid SPOFs when the consequences of failure were... people not seeing their pictures up on the Interwebs, oh noes! I am a little alarmed that all of the physical assets are in one basket. And yes, I actually spent time wondering, in a serious fashion, how hard it would be for a guy with a pair of cable snips, or perhaps even a bomb, to take out my site. Yeah, Fiber Is Hard To Pull, but then maybe 911, at least, should have local redundancy. Why is Santa Cruz's 911 service dependant on a fiber bundle running through San Jose?