Recently we wrote about complexity in IT and cyber security. Because of the myriad of systems that must work in tandem, troubleshooting issues sometimes gets very complicated. And sometime the simple answer is not the answer at all. We see complicated issues all the time.
We recently received report from a client late Friday afternoon that some of their clients have reported they cannot access our client’s website. Of course, when our support team goes to the URL it loads just fine. Right off the bat issues like this are tricky to troubleshoot we cannot replicate the issue it’s not even from our client who has the issue but a subset of their customers. After a few minutes of investigating our team is fairly certain the issue is related to the end users Internet provider. While we like to think the Internet is perfect and works at all the time. The truth is it’s not at all perfect and routing issue happen all the time. Usually, these routing issues resolve themselves in a few hours and sometimes it takes a few days. But there is not much you can do about it. For good measure we reached out to the hosting provider of our client’s website; to make sure they don’t see any issues on their end. They also confirmed they see no issues and agreed with our team that the issue is related to a routing issue with the users Internet provider and should clear itself up soon. While our client wasn’t very happy with this answer, we decided it’s best to wait until after the weekend and see how it goes. On Monday our client is still reporting that the issue is not resolved, and they want us to keep working on it. They’re internal marketing team is asking us to check the web hosting and DNS provider. Of course, we check on all of this but again all the vendors confirm do not see any issues. At this point our team was even more confident the issue was with this one national Internet service provider. One of the benefits of having 40+ plus people on our team is we all have may contacts at large companies that are not easy to contact such as Microsoft, Google, Facebook and in this case a very large nationwide ISP. Our team started to reach out to some folks and sure enough they did confirm it’s an issue on their end and they are working to fix it. It took a few days finally we did get confirmation that the routing issue was resolved.
We see many examples of complicated issues that don’t have a simple answer. We hope to share more with you soon!