It Can’t Happen To Me!

I can imagine the surprise when those who trust the Internet and view it as a 99.999% (“5-9s”) reliable network suddenly found themselves without it. In reality, the on and off ramps may not always be able to provide access.

On February 25, 2015, Northern Arizona went quiet. Cell phones and landlines did not work, cable TV did not work, and there were no connections to the Internet. This quiet period meant no 9-1-1 emergency calls, no credit card transactions for merchants, and no access to cloud-based applications and data. Service went down at noon and was partially restored to some in the area by 6pm but most had to wait until 3am for their service to be restored.

CenturyLink, provider of all these services, lost connectivity in one of its fiber routes to Northern Arizona from Southern Arizona and guess what? While CenturyLink is “planning” a second, redundant route, it is not yet in place so there is only one set of fiber cables running from south to north. To make matters worse, according to the police who are investigating, the cable might have been cut. We later learned the cable was in fact cut near a riverbed in an area that is not easily accessible by vehicle. Beyond that, the police won’t share any additional information.

According to CenturyLink, the cable had to be inspected mile by mile until the cut could be found and repaired. Meanwhile, 69,000 people in Flagstaff and hundreds of others along the cable route had no services. Reports are many about people trying to use ATMs without success, students trying to access their cloud-based papers so they could finish them and turn them in on time, merchants only being able to take cash and, of course, companies unable to function with all of their normal channels of communications cut off.

I can imagine the surprise when those who trust the Internet and view it as a 99.999% (“5-9s”) reliable network suddenly found themselves without it. In reality, the on and off ramps may not always be able to provide access. Is Northern Arizona a lone example of poor planning by a service provider? Hardly, most cable companies do not have multiple routes to their facilities. They might have redundant or what they think is redundant access routes to the Internet, but unless and until proven by real-world test and trials, no one knows for sure.

A case in point was a large 9-1-1 center that paid a telephone company extra money to have redundant trunks run into the center. Basically, one came in from the east side and one from the west, yet the entire system failed. Why? Because a few miles down the road these “redundant” trunks all came together in a single underground pipe system to carry them the rest of the way. A backhoe tore up the pipe and wires so all 9-1-1 calls ceased being delivered!

Where I live the fiber that feeds my Internet and cable TV is from a different company than the fiber that feeds the cell sites near me, so we probably would not lose all communications. However, while there are two different companies, their fiber runs lie in side-by-side conduit under the road so a backhoe could take it all out—plus the phone lines since they share a fiber route with one of the companies.

As a result of my extensive experience in Public Safety communications I have become very untrusting of promised “redundant” systems. Often they are not truly redundant as in the 9-1-1 center above or as with the fiber that is all in the same trench under the street where I live. When working with the wireless network operators on behalf of the Public Safety community, I quickly learned a term they all use to describe their wireless networks. That term is “best effort.” They all provide service that normally stays up, but at the end of the day they will all tell you they are geared up to provide best-effort service. As far as I am concerned, DSL, cable, and fiber to the home, office, and you should also be classified as best-effort connections. They will usually work, but those of you who are familiar with the Law named after Mr. Murphy, a guy we have come to hate in wireless, know he will appear out of nowhere when we really, really need to access our work, data, or now even applications, or when we simply must send a file to a client or risk losing a contract.

I guess I have become jaded living with and being involved in communications and computing for as long as I have and I question much about today’s trend to keep everything in “the cloud.” The only way for mere mortals to reach the cloud is via the Internet and access to the Internet starts at our home, office, the nearest Starbucks, or an airport. We all know or should know that public Wi-Fi is not secure. Yet I receive company confidential documents from people working for one of my clients while they are sitting at an airport using the free Wi-Fi and without the benefit of a Virtual Private Network (VPN) or even data encryption.

Next up is our connection to the Internet itself. Be it cable, DSL, or even fiber, once our data leaves our premises it is either carried overhead on a pole or buried underground and the data travels to a point where it is then handed off to the Internet. This link is a single pipe between us and the Internet. A car can hit a pole taking the connection down, a backhoe can make a mistake and rip out the conduit holding the wire, coax, or fiber, a flood can ruin the fiber connections underground at the junction points as it did in Hurricane Sandy, and a major storm or earthquake can disrupt our access.

There is little we can do about such outages, even if we subscribe to both DSL and cable, or like some, cable or fiber with an automatic failover to 4G wireless. Once the wires and/or fiber are in the ground or on a pole, the chances of a disruption to one or more of your connections is always a possibility.

That is why I do not use cloud-based applications, and it is why even though I do use cloud-based back-up I also have a set of hard drives on my network to back up my data. I am paranoid about failures in commercial systems, because I am accustomed to working with Public Safety-grade networks and infrastructure, as well as off-Internet pipes that carry data privately to and from their destinations. Even then I will tell you that none of these systems is truly 5-9’s reliable. No matter how many ways there are to back up—wires, fiber, cell sites, and all of the other things between us and the Internet or remote storage—we are still running a risk, slight though it may be, of having some down time that will come at the worst possible time. Just ask those in Flagstaff how productive they were in the 15 hours they had no cell phones, no Internet, no wired phones, and no cable TV.

The bottom line for me is that trusting our most important information and applications to a cloud only, and believing our connection to the Internet and, in fact, the Internet itself will always be there and accessible is much like not backing up a PC in the early days and then having your hard drive freeze up only to lose all of your valuable data and personal information. If you google Internet outages for 2014 you will find a list of outages that is, I am sure, not complete since the FCC does not yet require the same level of reporting on Internet outages as it does on wireless networks. Still, the list includes outages on Comcast, Charter, Verizon, Frontier, CenturyLink, and Level 3. If you go to outageanalyzer.com you will find some stats on outages and a list of current outages. When I went to the site as I was writing this article I found the following list of outages:

It is difficult to tell how many outages are within the United States but there were at least 18 in the Central Mountain area according to the site. Failures do happen.

Are you ready for a failure of your vital communications systems? Your Internet, your cell phone, and/or your landline phone? Like everything else, most of us are not prepared and simply assume that everything will hum along perfectly all of the time.

There are some ways to prepare for family communications during a failure. You can use Family Radio Walkie-Talkies with a one or two-mile range, or become a licensed Amateur Radio Operator. However, there is no way I know of to augment access to the Internet to retrieve your information if there is a failure. Ask the folks in Flagstaff and other parts of Arizona what it was like when they lost not only the Internet but ALL of their communications for up to 15 hours!

Andrew M. Seybold