What happens when 10s of 1000s of people have to work from home and access the same platforms?

IT departments are scrambling really fast, trying to now rethink a lot of application connectivity and delivery. Max Clark talks with William Wohnoutka, CenturyLink’s Vice President of Product Management for Global Internet and Content Delivery Services, about connectivity: Customers aren’t going to have to deal with the physical much longer, physical still matters, and where physical can be used to provide a differentiated service or a service that stands out becomes it meets a specific set of requirements… That has to be software activated now.

Episode Transcript:

INTRO: [00:00] Welcome to the Tech Deep Dive podcast, where we let our inner nerd come out and have fun getting into the weeds on all things tech. At Clarksys, we believe tech should make your life better, searching Google is a waste of time, and the right vendor is often one you haven’t heard of before. 

Max: [00.18] Hi I’m Max Clark and I’m talking with Bill Wohnoutka who is the VP of Product Management, Global Internet and Delivery Services for CenturyLink. Bill, thank you for joining.

Bill: [00.27] Thanks Max.

Max: [00.28] So Bill, you have a fascinating job to me, and a fascinating history that goes all the way back to the start of the internet with Global Center – I’m going to call you out there with how long you’ve been doing this. You came into CenturyLink by way of Level 3’s acquisition, and have really been at the core of the internet for a long time now. I mean just to start here, when you look back and we talk about a twenty-plus year horizon of time, what’s that been like and how much has surprised you in that span and how much of it was expected as we save the growth of the internet?

Bill: [01.03] You know, I hate to say that I invented the internet or something silly like that, but back in the Global Center days we were working under a leadership team that had run Maze to Mae West so I got exposed to internet connection challenges pretty early on and at that time, you know, marketing internet services was really about overcoming the challenges of congestion on shared routers in Maze to Mae West and certainly other important interexchange locations came forward… I think if anything surprised me honestly, it’s actually how long it took for the internet to develop a deep interconnection topology, and the move to do that took as many decades as it did — today, we’re incredibly, deeply interconnected and incredibly resilient in North America, and of course, I think the rest of the regions of the world don’t have the same benefit of homogeneity and culture and content, and so interconnection becomes increasingly more difficult as you leave the US. Certainly it works and it has a way of working, and there are certainly nuances by each region, depending on – again, those cultures, and then also the separation of countries from each other, right? In the US it’s very easy to build deep interconnections if you have the right amount of traffic and the right resources, but as you get into the rest of the world: EMEA, APAC and LATAM, it’s really a different story. So, we’re really happy about the way the US has evolved. There was pain along the way, certainly plenty of things in the public record to suggest not everyone was getting along, and every decade — but now, we are really – have a resilient, robust internet infrastructure, and it’s showing its ability to withstand the increased traffic from the COVID stay at home workers.

Max: [02.54] So the general population or public is not really aware of interconnection until you have a public dispute, and that’s kind of the equivalent of DirecTV cancelling, you know, somebody’s streaming package, right? But at the same time, I mean it feels like capacity is always limited. I mean, there’s this growth and this hunger related to interconnection and capacity, just doesn’t seem to really stop. I mean, is that really coming up in the US still, or is that really a — I mean, we see it with the APAC region network, Europe seems to be a little bit easier in most cases, LATAM is difficult as well. How do you make plans around this and deal with this capacity requirement as it grows?

Bill: [03.37] You know, back to what’s special about the US is that there’s so much geography and the density of that geography is certainly concentrated in a couple of hours but as you then start to fan out, there are a lot of dense populations through the middle of the country and on the coasts that are not the original two cities, which were more or less San Jose and Virginia, which is where the two interconnection points were. And so, because the US has such a distributed population and because the peers of ours here, the trading partners we work with here are the largest mobile, cable and wireline broadband suppliers, have a footprint that is broadly geographic, interconnection happens within as few as nine locations and, for some of our larger trading partners, it happens in as many as twenty five locations. The fibre richness, we certainly have enjoyed with the number of companies that have become a part of CenturyLink over the years, I think it has become an asset rich territory where interconnection is now considered a priority for user experience, and we see some of the most impressive trading partners here really leaning in in the face of COVID to proactively upgrade — in the case of one large trading partner, more than doubling our capacity between us in this period of time that we’ve been in lockdown, and so the situation is very different in EMEA, in particular. There has been a lot of concern about congestion being caused by various events that are happening: certainly, game popularity is increasing; I think all of our kids are playing Fortnite or Call of Duty at this point, and that has created a lot of strain in Europe in particular, where we’ve seen inquiries both formal and informal from regulatory groups there, and we’ve had to take steps and measure not just to accelerate the capacity and deployments in this specific companies, but we’ve been very busy writing software to control peak traffic and to help control peak traffic during business hours, which obviously, congestion would have an impact on commerce and the economy there, as well as peak hours and peak viewing  hours on the internet which are 5pm to 11pm.

Max: [05.54] I mean, do you think that the issues there are related to the interconnection capacity being strained, or the regional access networks having – you know, the equivalent of like a head unit, where capacity and strain are the last mile into the subscriber location. I mean, what is actually driving that concern, and that capacity crunch? 

Bill: [06.13] You made a really thoughtful statement earlier which is, there’s a finite amount of capacity – I’ll paraphrase it a bit – but there’s a finite amount of capacity in any country in any location. Interconnections in Europe tend to be concentrated within specific countries, and so — as you start to look at what a big internet event… You know, the sort of impact that a big internet event can have in a single country. I mean, take Germany, France, Spain or Great Britain; you could probably pretty quickly tally up just exactly how much capacity exists in the interconnection and down to the last mile, and over the last couple of years, we’ve had to really think hard about that, because for some of the large events that we’re now starting to see in the internet, whether they’re game downloads, or whether it’s streaming video, the infrastructure has reached those points where you actually start to see it congest. It’s a little hard to pinpoint, it’s a little bit more about your weakest link is somewhere in that mix on any given day, but you know, on the whole, there’s really a finite amount of headroom that is there to be absorbed. There’re lots of good ways to continue to grow and expand your scale in Europe, what you don’t have the benefit of is the same type of availability for Great Britain to handle a Germany in the event of hitting the large constraints in those countries. While the carriers there do interconnect outside of their home countries, the capacity that they carry is very much oriented around actually us getting content into their edge and localizing it into their networks and therefore the backload has become thinner than the edge. At this point our edge growth is far exceeding our backbone capacity, so a catastrophic failure on any other pinch point that could occur could make for a bad day in one of these countries, and certainly it doesn’t even take it to failure at this point, the popularity of games and streaming video are I think pressing up against what’s possible in those countries. That’s why software has had to be introduced into the solution and into the mix to control session rate limits, so if I have a great broadband connection, you have a great broadband connection, we live in the same country, the same city, we’re not going to get the same service, obviously. So, finding ways to democratize and flatten that, not only across users on the endpoint who are demanding the content but also across the suppliers of the content, has become really important in this event.

Max: [08.47] So I mean, I hate to use the term because it’s been in the huge marketing hype cycle for a while, but isn’t this really the play for edge and why edge is becoming such a big thing in conversation about getting content and services closer to the subscriber and beyond the interconnection locations? How does this affect edge strategy for CenturyLink and when you’re looking at product planning and service delivery?

Bill: [09.11] That’s a great question. In the content delivery world, we’re primarily using HTTP protocol to deliver content to consumers on a mobile device, on a connected television set, on a laptop… I think increasingly though, we’re seeing really strong competition from the over the top video folks, the esports folks, and the gaming companies, and so what they’ve needed from us and I think what they need from us going forward and the big trend that we see is that they will demand more flexibility in that edge. I can tell you right now, we are already working on releasing a product in the fourth quarter that will give a compliment to our content delivery network in the form of bare metal as a service, beyond that we’ll be launching various layers of virtualized services on top of it. One of our very, very aggressive customers has acquired a proprietary streaming protocol platform that enhances user experience, and also enhances the ability to monetize the content and that is now something that can run on a convention CDN, content delivery network edge, it’s complimentary to those services that still need to serve all the devices we normally serve, but they are absolutely striving to deliver a differentiated user experience and to do that, you know, they’re using proprietary software. So we’re looking at – not just enterprise applications there, because there’s a number of enterprise applications where the edge can be used to reduce the congestion that’s sent upstream, as you say, collecting video, security or surveillance footage from a location, edge compute can be used to reduce that data set before it has to go back to a core cloud service… But in the case of the entertainment industry they are absolutely focused on finding ways to compete and win user attention by delivering better experiences. So, we’re seeing a lot of competition there from our largest gaming customers and our largest OTT customers.

Max: [11.16] So I mean a criticism of CenturyLink would be that you guys are slow moving because you’re so large, and a lot of other CDNs have come up and become very specialized, and what you’re talking about really is… Customers that need a very specialized and unique experience or service tailored to their exact business model or service delivery model, can’t be accomplished by a generic CDN that can service everyone, and how do you actually partition out your platform? I mean, there’s other global players but you seem like you have a lot of unique assets and abilities to officially take your scale and leverage and deliver a unique product and experience for people looking for these things. 

Bill: [11.57] Thanks for that, I think that’s a backhanded compliment and I appreciate it! I think it’s an opportunity for me to declare that we have looked at our network assets and we’ve looked at what’s happening in more traditional enterprise space with web applications and industrial grade computing needs, meaning… I need to have a location where I can run workloads that is, from a latency standpoint, within five milliseconds of a user location, whether that’s a manufacturing location or it’s a retail location, or any type of enterprise location where latency actually dictates the ability to run the workload. The cost of running a workload in an enterprise location is high relative to break fix, relative to the ability to service and manage the equipment on a day to day basis, and so as we looked at their network and we looked at our on net locations in the US, we were in a position to provide edge computing services that really no one else was, within you know, five milliseconds of user locations on our network and covering most of the population of the US, most of the business centers… We’ve pivoted, I think, pretty quickly toward enabling industrial use applications across our private networks, our IP VPN, our SD-WAN products and services, and on the internet-facing side, we also enjoy this very similar, low-latency and you know, high degree of ability to put content very close to there the users are. We have over a hundred and twenty POPs in our CDN today, and that’s actually growing, not declining; we’re really trying to ensure that localization of those services can enable these types of new applications and I’ll give you some examples. You know tactical applications like augmented reality will absolutely require sub-ten millisecond latency. Any kind of industrial or manufacturing controls are sub five millisecond latency types of applications. So, the pivot here has to do with the fact that, you know, we looked very closely at our assets and our products and decided that we were going to need to become a technology innovator if we were going to take advantage of the physical networks, so things in the physical world move, somewhat slow relative to the software world. So you’ve seen us do an acquisition quite recently, in September, we acquired a company out of Paris called StreamRoot, and they make a technology that allows for the actual sharing of popular content… So, if you and I were watching the same show at the same time and we’re sitting in the same market on our broadband connections, the software is clever enough to determine whether you and I are watching the same content, and if we are, can the network be more efficient by downloading chunks of that content to each of us, and reducing the amount of downstream bandwidth that’s required. This kind of technology works incredibly well behind a firewall in enterprise locations, and in fact we have several customers in the UCC space that are deploying this technology inside of their applications to improve high quality bandwidth in locations that are hard to reach, is the term I’ve been using. It sounds like a place in your back you can’t quite scratch, but it is the case, you know? Downstream constraints in enterprise, when everyone wants to watch a high quality video of the CEO talking about the state of the company, those things have traditionally delivered very poor video experiences, so… I think again, we’re really looking for the assets to give us an advantage but I don’t think — we have an amazingly unique set of fibre assets, we also have several layers of network services above that, and now we’re really looking for software that can help us accelerate the ability for customers to consume that asset.

Max: [15.53] And I think you defined edge here very interestingly, which is, within five milliseconds. A lot of the original edge hype related to installing equipment at the base of a cell phone tower and building out all these little datacenters in shipping containers or something, and that has not proven to be an effective deployment of edge, but when you look at five milliseconds from a network asset, that’s a pretty big distance. You know, in terms of mileage, that’s a relatively large metro area you can cover in five milliseconds.

Bill: [16.23] That’s right, yeah, it’s a large metro area. There’s a number of players in the space that are focusing on the radio access networks and supplying bare metal as a service and other virtualized services to enable, you know, virtualized network functions, and we think that that’s a really good trend. We do do fibre in the metro from mobile traffic backhaul, but we haven’t jumped out and entered the space for mobile, you know, carrier-grade edge compute services. Our footprint is, I think, well-suited to enterprise use cases and applications. We’re certainly talking to all of the organizations that are trying to do the mobile cell tower enablement, and I think it’s going to be a very complementary relationship where we’ll be the next hop back from those providers, you know, you can think of the popular providers in that space, and assume that they’re going to be using our fibre and our assets and they’re probably going to want interconnections and more importantly, breakout of layer 3 IP traffic. It’s sort of silly to think that we’re going to localize some of these applications possibly as close as within the metro, and then fallback to even nine locations in the US to actually exchange the layer 3 internet traffic. As much as those interconnections in the US today are somewhere between nine and twenty five markets, I do see a world where we’re connecting in all metros and we’ve connected a much more scalable internet. So edge for us at that point becomes… We are sitting as the next hop back from the aggregator who is delivering the mobile tower edge compute. They do not expect to have enterprise or consumer applications running there, the space and the power is far too limited. Max, that’s the other thing that I think is going on – any time you have a new space like this, there’s a lot of hype. What was important for us was to take the time and really look at the assets and determine if we had a sustainable competitive advantage here, and I think that the other way that we’re capitalized and our profitability and our ability to invest in the future, those are the things that are going to differentiate us from the startups that have tried to enter the space, or those companies whose physical footprint and assets don’t match the type of deep localization we’re talking about.

Max: [18.36] I mean, you’ve got decades of a headstart and billions invested in this, and you are the largest network on the planet by every survey that comes out. I mean, you should be able to leverage that weight and get something out of it. So, COVID has driven a massive shift in workforce. We’ve seen — what was a slow uptake in remote or distributed environments go all remote overnight, and also at the same time a fantastic increase in residential demand in, you know, other services. You know, telemedicine, schooling, video gaming, et cetera. So, I understand if you can’t answer this specifically for CenturyLink, but for a you know, industry, I mean what kind of numbers are we actually talking here in terms of growth and capacity demand in the last month as it relates to what has become this event?

Bill: [19.24.] We’ve seen a fifty percent increase in UNINTELLIGIBLE 19.26 traffic daily, since around the time of stay at home orders. It was kind of a perfect storm as well; you have several new products that were being launched, certainly new innovations in the online gaming space… I think the demand for UCC and video conferencing is obvious to all of us, and that has hit all verticals. We’ve also seen a number of new OTT products hit the market in this period of time. We happen to be in a pretty good position ourselves, because we were anticipating the launch of the new OTT products and we were anticipating the launch of the gaming products, so we had been in a pretty heavy investment cycle leading up to this, and I think that again, the cooperation collaboration with the carriers and our trading partners globally has just been outstanding. We’ve really seen a lot of willingness for everyone to come together and work to ensure that we’re reducing the impact of the different types of traffic that are potentially impacting – and I think at the start did impact, in many cases – internet performance. Every area of demand that you could think of relative to internet access and content delivery has seen this type of really radical overnight growth, but again where there have been hot spots, there’s really been a really strong willingness of the ISPs, you know in the case of game downloads, game publishers, game distributors and CDNs to work together… More transparency than we would probably have if we didn’t have a global pandemic occurring, and certainly there’s been encouragement to lean in on aligning practices so that we don’t have one or two bad actors that are out there spoiling it for everyone else – and I’m talking daily phone calls with you know, multiple customers and multiple ISPs that could go on for a week or two at the start but in general, when we were figuring out where and how to deploy capacity, improve the situation, how to shift traffic around to improve the situation, or in many cases for us, writing software to help some of these customers that didn’t have controls on their platforms to control that peak, that actual spike when everybody wants to get the same video or everybody wants to get the same game… You know, it blows up all the other traffic — that has been a phenomenal story, and you’ve seen them in the press; certainly some of the major OTT providers came out and mentioned that they were taking the steps of — obviously we’re the company behind a lot of those customers and then you’ve seen it in the gaming space as well.

Max: [21.58] So it seems like… I think it was somewhere toward the mid-to-end 2018, early 2019 that the cost of running hundred gig interconnects… The economics equalized, or it got improved over ten gig legs. And I mean, this is a factor; how much does it cost for the router, and the blade and the optic, plus how much power does the optic suck in order to deliver these circuits. And since then, there’s been this explosion of hundred gig interconnection, and it seems to be accelerating rapidly. I mean, this goes back to an earlier comment you said about you know, capacity and constraints, and as we look forward now, there’s always, you know, some manufacturer, you know, political battles related to what the next spec will be: is it two hundred gig ethernet, is it four hundred gig ethernet, is it faster than four hundred gig ethernet? How do you plan for that, and where are you in the planning cycle of, “Okay, now we’ve got x by one hundred gig legs employed as our interconnection but we already need more,” and how long does it take to get ahead of that curve and where are you guys in that cycle?

Bill: [23.00] I mean, I think hundred gig ports are in their sweet spot at this point in terms of economics, and we will alway swinsist on hundred gig ports unless there just happens to be no other option for interconnection options. From a customer standpoint, I think ten gig is going to have a long shelf life, and certainly, you know, our backbone is made up of a lot of ten gig customer ports, and now actually thousands of hundred gig ports from the customer side, but from the interconnection side we really drive towards all hundred gig interconnection. There’s a new project we’re working in internally to bring the largest carriers and cloud providers together in the US to interconnect at terabit speeds, and some of the thinking there is that by building a switching fabric that everybody can access in these metros, that we can actually accelerate the rate of turning up, you know, the interconnection services, so the largest cloud providers and the largest interconnection partners and trading partners have come together… We’ve kind of led that charge, and we’re building out optical platforms now that will be standing in place with, you know, minimum upgrade capacities of terabit at a time. So, we are planning for an ever great acceleration of traffic exchange between content providers, cloud providers and eyeball networks playing a pretty central role in getting that project off the ground. It hasn’t been announced yet, but I think heading into the future — but to answer your question directly, I mean… Hundred gig line speeds are what make up that terabit connection, so the focus is still on where the economics are best, and then certainly — we tend to believe in the multiples of ten, I don’t know if that’s been circumstance or not, but it feels like the interim steps that happen, the forty gigs, the four hundred gig steps, those haven’t really ever been something that we could justify economically in making the change, so we tend of go in the cycles of ten-x multiples, here.

Max: [25.01] Well, I mean you said a hundred and twenty POPs for just a CDN, when you look at that times the number of routers and devices you have across those POPs, all of a sudden you’ve got quite a large fleet of equipment that you have to maintain and upgrade in those cycles.

Bill: [25.14] Yeah, that’s absolutely true. The goal I think for us, is to continue to expand the network into the places where customers want to go. So, I mentioned before that we had done a lot of pre-building before we hit COVID and it has to do not with the fact that we anticipated COVID, nobody can anticipate a black spawn event, otherwise it’s not a black spawn event, but the fact of the matter was is that because of our investment position and our ability to spend capital in advance of customer demand, we went to the places that customers said they needed us to go, and I think that’s something you can always come back to central and say, if I’ve got a critical initiative to digital transform my company, who are the players in the market that I can go to, show them what I’m going to do, and convince them that it’s in our mutual interest to build – those are the projects that I love and I think that’s why we get the crack at some of the biggest projects in the world, it’s because we have that appetite to build where our customers need us, and whether that’s a single enterprise location, and that’s bringing fibre into that location to ensure it’s continuously upgradable from ten gig to a hundred gig to a terabit, those are bets that we take with you all day long, and those are the bets that our company’s success is predicated on.

MID-ROLL: [26.32] Hi I’m Max Clark and you’re listening to the Tech Deep Dive podcast. At Clarksys, we believe tech should make your life better, searching Google is a waste of time, and the right vendor is often one you haven’t heard of before. With thousands of negotiated contracts, Clarksys has helped hundreds of businesses source and implement the right tech at the right price. If you’re looking for a new vendor and want to have peace of mind knowing you’ve made the right decision, visit us at Clarksys.com to schedule an intro call.

Max: [27.00] CenturyLink has been building like crazy in APAC, and there’s a couple of interesting things going on in the APAC reason, and one of them is it just doesn’t’ seem like there can ever be enough capacity. You’ve got almost unlimited demand and at the same time, as soon as you upgrade capacity it’s almost entirely saturated… How much of that is just a frustration cycle, and how do you make plans for that and keep up with it and balance the financial cost of those investments and upgrades, as well as with your trading partners? I mean, you’ve got two people here that have to make decisions and come to an agreement to collectively invest a lot of money to increase capacity. 

Bill: [27.38] Supply chains certainly become an issue… I don’t know that any of us were prepared in the sense that supply chain intervals have gone in some cases for some types of equipment for some of the competitors, and I think for all of us, have extended to more than 2x what they were before. Dabbling in Asia PAC is particularly hard because of the lack of cultural homogeneity and because of the water that separates some of the most important countries over there for US customers. The other thing about building – this is funny because it’s such a chicken and an egg – it’s a question of, if you have the traffic that needs tobe delivered, there will be trading partner who is there to receive that traffic, and those negotiations tend to go more quickly and are usually financially neutral for most parties. When you don’t have the traffic, everything is starting from scratch and trying to build from the ground up, and so the economics of the business really demands that you have to have the traffic in order to actually build the scale that’s required to handle industrial grade or consumer applications. And so, we did do a big lift in Asia PAC, we announced that at the end of last year, and it was driven on the back of our conviction that the customers we were working with were going to have a product that — have products that were extremely popular in those countries… So ,we looked at their roadmap and their build loans and – actually – frankly started building prikor to some of those even emerging but, again , looking at the way that these US customers wanted to enter that region, we were able to determine which countries would most likely be prioritized for them, start that building process, and as… Just as you said, pretty much as soon as we opened up the ports and started opening up the traffic to our internet backbone and our content delivery network, teh traffic just flowed. And again,. Once the traffic is flowing it’s a whole different game. You can go much, much faster if you have the balance of traffic, and people have to take you much, much more seriously. It’s been a really rewarding experience, but the supply chain fears are real. We’re not particularly dependent on any one country or any one vendor, we’re extremely diverse in our use of multiple vendors for many different reasons, so I think that’s allowed us to be pretty nimble in the face of what, for others, might be supply chain issues. That also having the deep bench strength and the ability to invest capital. And what we’ve seen is as traffic has started to go into countries that werne;t in our original build plan, we’ve had a lot of our carrier partners in these countries  come forward and ask us to expand more quickly into their countries to reduce the burden of subsea backhaul and capacity challenges that they face, and so we’re taking those investment opportunities incredibly seriously and putting compatible in flight to improve user experience in those countries, and ensure that those networks are operating more healthily as well. I think it’s in our interest to help any other carrier who’s come to us and said, “I’m seeing so much traffic from you in this country, we’re interconnected there but I’ve got to backhaul it across the ocean and deliver it to my eyeballs over here,” so that’s created a lot of really interesting and compelling opportunities for us to accelerate that growth now that the traffic’s there. There’s no doubt that there’s daily consumption lift from COVID traffic as well, we’ve been very fortunate to have the right customers and to have picked the bet on the right customers and grown with the right customers… We are also seeing a general lfit in traffic that is absolutely based on the increase in consumption, and that’s created new opportunities for us to lean in even heavier in Asia PAC.

Max: [31.19] How difficult — I mean, so the Chinese market is a massive market for companies, and in a couple of different ways you know? Companies that need to get access to the Chinese market or get access to Chinese eyeballs and vice versa, you know? But at the same time, you’re dealing with a — a regulatory, it’s more like the geopolitical becomes a driving factor in these things. And based on rhetoric and the press or on social media now, it has a pretty wide ranging impact on your business and your planning… How do you, I mean is it just keep your head down and kind of ignore it and just keep moving along and know that these disruptions are going to happen and you might get derailed, how do you maintain and manage that when there’s you know, this additional layer of complexity that can come out of nowhere all of a sudden?

Bill: [32.06] China is a very unique place in the world, relative to interconnection. I’ll start there and maybe we’ll go to the supply chain after that. It’s a very unique location in the world due to the fact that the government is using essential a large firewall that all of the firewalls implement to enter and exit the country, and then operating there is incredibly company, when I say that I mean it’s very different in operating in the rest of the countries in Asia PAC or everywhere else in the world. So, we tended to use interconnection outside of the country to ensure that we’re delivering sufficient access from our network into China, and then from China back to our network, and then… We use partners in the country to do last mile internet connectivity and broadband, and that has worked extremely well for us. I think the complexity of operating there, given all the other complexities and challenges that we can go tackle, outweighs the benefit at this time for us to go in and try to establish a business entity locally. It’s definitely on the list of places we’re going, but it’s not the priority. I can certainly point to other markets in Asia PAC that we want to continue to grow scale in more quickly. I think being tied to China from a supply chain standpoint has some significant drawbacks at the moment, and has for a long time. That’s been well covered in the press and I think everyone knows what we’d be talking about there, but it’s the last place I want to be dependent on China for my supply chain, and again, that’;s why the diversity of vendors that we’ve deployed in our networks allows us to have pretty good resiliency in an event like this, where I think it’s pretty clear that if you were dependent on tech from China at his point, you would have seen some slowdowns and extension of intervals in supply chain. Yeah, it’s an incredibly important place to be — we work with partners there, and we’ll continue to work with partners there, we’re about to bring a CDN partner on in China; most of our competitors in the CDN space are generally reselling services that can be acquired at some level of effort, and we want to ease that for our customer base. So, China is on our roadmap from a CDN perspective, as using a partner in the country to deliver. THat’s also to sy, the streaming technology that we’ve acquired has incredible applications in a location like China where our ability to use that software technology, it’s going to be a real advantage for anybody who decides to take advantage of that. The technology itself and this time of peer assisted delivery has caught a lot of attention and actually gained quite a bit of traction inside China as well, so it’s generally regarded as a good network traffic control tactic to deploy. We’re excited about the opportunity for our technology there.

Max: [34.50] So, I mean, CenturyLink, you go all the way to the bottom, right? You have fibre in the ground, or in poles or in the water, and a network is built on top of that and then services are built on top of that network, and you know, we’ve talked briefly on internet or MPLS or SD-WAN and content delivery and these sorts of things, but there is a lot of additional stuff that you guys do that people don’t really talk about or know about. I mean, you see a massive swath of internet traffic on a daily basis, you have very popular recursive name server, I don’t even know the scale of how many queries you’re doing per day, and on top of that you layer a threat intelligence product, and denial of service mitigation products, and you’ve got Ansular security services… These all kind of work in harmony but have competing interests, you know? The need and interests of content delivery and rapid interconnection and capacity is different from, “Okay, we’ve just now enabled additional ports and put more people on that can DDoS somebody,” I mean, how do you balance that? What does that look like and how competing are these objectives for the business with each other, as you scale and continue to grow this?

Bill: [35.57] One of the things I’ve always liked about working at CenturyLink, and I’ve got a twenty-two year run from the acquired company at Level 3, is that our CEO is very focused on making sure that decisions can be pushed to the right level of employee – and I’ll just say this: all those products that you just covered, my peers and I all sit underneath one executive product leader, and we meet weekly, we meet daily, our projects intersect and intertwine. For example, I’ve just launched a product called CDN Edge Compute, which allows us to deliver web application firewalls and bot management, bot detection; that’s actually stuff that sits in my peer on the security side, because you know, that’s in his realm as well, so we launched this product together with the DDoS, it’s actually a bundle of services that allows us to provide application layer protection as well as volumetric DDoS mitigation. I think one of the things that helps that happen is we have an incredibly tight management team that’s worked together for twenty-plus years, and has in many cases, all of us have worked on every product at some point in time, so I think there’s this strong awareness and frankly the desire to help each other to be successful, and I don’t know that you see that in every large company. I feel sometimes like… Jeff Storey – our CEO – his desire to push that decision making dowen the levels where real change to be made is what’s enabling us to go faster now as the new CenturyLink, and you’re allowed to say, you know, with all the products we can talk about, there’s probably hundreds of products in that portfolio, that is run by a team of about eight product managers that report to one person. So, our interests are incredibly aligned, and as these products start to come together – you think about SD-WAN and its reliance on the security side of the business, and you think about CDN and its reliance on the security side of the business, you think about our voice services and their suitability to our SD-WAN products; these products are coming closer together. I think our goal is really to drive towards a single platform for consumption, especially as we’ve looked at virtualization as being absolutely critical to getting new products and services out within a network, the whole edge compute story takes on such an important, central function for all the product managers to rally around. So, you know, as we’re looking to transform customer experience, I think our goal is to make the products and services we sell easier to consume for customers, and remove the friction that sometimes exists when you have multiple products that were not contemplated in the same organization. So, our go forward is very much about using a common core platform for CPE, a common core platform for edge compute, and then landing virtualized services on top of that that can be connected with software, and that’s where we’re spending our time.  

Max: [38.48] So as you look at common edge platforms for edge compute and the ability to actually launch containers and services for customers on the edge, that puts you into the cloud space. Public clouds are very large customers for you. You know, does that create a conflict for you at some point, you know, is that even perceived as competitive, given their size and your size in the market as it relates to compute and capacity? 

Bill: [39.13] We’re obviously a critical supplier to all of the major cloud providers, and we also play a role in managed services, where we’ll support cloud migrations. So, we’ve also actually become a very critical partner to those major cloud suppliers, we’re certified from multiple platforms to handle migration, as well as application development… You know, the relationship we have with them is so multifaceted, you take any of the large hyperscalers and you’ll look at see that we’re involved in the delivery of their consumer services, we’re involved in the core infrastructure for their computing as a service platforms, we’re involved in helping them deploy their edge capabilities into our footprint, so we’ve talked about our edge compute services coming from bare metal as a service up to multiple levels of virtualized services, but we’re also pursuing the ability to allow customers to take and put those core cloud companies’ edge offerings into our datacenters. A big part of our edge strategy is a lift on colo, that would enable us to support the proprietary edge cloud services of the largest hyperscalers, and then having the ceritifcaitons to do that transformation, I think makes us an even more valuable partner to them. Our fortunes are very much tied to the success of cloud, as a business, and I think it’s become imperative that we get off the sideline. It’s one of the really nice things having come from Level 3 into CenturyLink is that I’m part of the cloud team, and the cloud product manager is sitting right next to me in the same meetings. We all look to him — it’s kind of funny, you’ve got the fibre and the colocation guy, and he’s at the first layer of our stack here, and then you know, you’ve got the next product management function which is coming from our edge cloud product team and core cloud product team, and the rest of us at the application layer are all dependent on that guy and on myself, on the internet guy. So, yeah… I think we’ve really reorientated the company to become not just a cloud networking company, but a company that can use its assets to enable its customers in any layer that they really need to be. 

Max: [41.26] About ten years ago, there were some financial personalities on television related to the stock market predicting that datacenters were dead, and the cloud was the way and get rid of the datacenters, and it was funny to me at the time, because this misconception that clouds were in datacenters and you needed the datacenters for the clouds to exist, but just to relate the perception… And it feels like at the same time, fibre and the network, like… Nobody has cared for a long time and now it feels like all of a sudden people appreciate and care about the network experience and are seeing the impact of having strong networks and strong interconnection and capacity, and it’s gone from this invisible thing to a very visible forefront conversation of, “Okay great, we have our application that’s sitting somewhere, how do we connect to it?” What happens when tens of thousands of people have to work from home and access whatever platform it is, and what does that look like and how do we manage those things? I think IT departments are scrambling really fast, trying to now rethink a lot of application connectivity and delivery. So forget just public internet, but also internal internet, and I feel like, you know, you guys have some interesting stories, and if you guys went out and shouted from the rooftops a little bit more, you could really take advantage and benefit a lot from this. 

Bill: [42.41] Yeah, you know, we might be one of the best kept secrets, certainly there’s been lots of focus and attention on 5G as being where edge and edge cloud is happening, but 5G is incredibly dependent on power and space and colo, totally dependant on fibre and internet connection. You know, you get off the radio network pretty quick, and certainly with this work at home situation, it feels like broadband has taken on a whole new, important — kind of what you said about datacenter versus cloud, home broadband is where I imagine much of this traffic is concentrated right now. Yeah, putting connectivity services — it’s funny, we launched a service – and I’m sure you’re familiar with our Cloud Connect – several years ago, and it was generally a datacenter to cloud operator product, where customers were using that to migrate applications to the cloud, or they were using that for load balancing of their applications between the cloud and physical datacenter, and these connections were really very physical in the past but we’ve evolved that stack to become completely virtualized. That wasn’t the most interesting use case, well it’s still datacenter and a datacenter is that service now enables us to turn up dedicated connectivity between two instances of AWS cloud services, and so rather than using the routing that AWS provides by default, which is called the default routing, we’ve put premium services on the market that allows a customer to virtually instantiate a connection across our network, between two AWS availability zones. We’re going to be extending that capability to support multi cloud, and I think what it comes down to is… You’re right in the sense that customers aren’t going to have to deal with the physical much longer, physical still matters, and where physical can be used to provide a differentiated service or a service that stands out becomes it meets a specific set of requirements… That has to be software activated now. I do think they’re moving away from the days when enterprises wanted to employ a datacenter manager and there was a buyer of bandwidth who dealt with the long lead times that were required to get physical service in place. The country is really well fibred now and the game is moving to software defined capabilities, and that is absolutely where the focus is in terms of our core networking capabilities – SDN, software defined networking – is really where the investments are going, and the goal is to enable enterprises to be much more nimble in their ability to acquire a quality of service that’s appropriate for the task and acquire that instantaneously. That’s the challenge that we’re rising to.

Max: [45.28] So when we look at global internet, interconnection relationships, management, et cetera, how much of this is science and how much of this is soft skills and relationships, and when you look at the soft skills and the relationships, at some time in the next thirty years you’re probably going to want to stop doing this and go out and retire on a boat somewhere, right? But how do you develop your staff working for you, and bring up this next generation of leaders that are going to be driving the charge and managing you know, these networks and planning going forward?

Bill: [46.00] Your question makes me feel a bit like a dinosaur, but I will say this: I think what’s incredibly important here, first of all, is that we make the transformation — acquisitions often are about the product, but in many cases your acquisitions are also about the acquihire, and being able to bring, you know, innovative and creative minds into the company and people that are going to fulfil really strong roles. You know, the organization that I’m in today is really thinking hard about how to increase diversity within the workforce, and to me that includes gender, race and age – so, absolutely getting a diversity of backgrounds and skill sets to the table is critical. You know, from a customer standpoint, I think it’s as difficult as ever to navigate the number of choices out there. I really believe that you for example play an absolutely critical role in helping customers navigate what is otherwise a very confusing part of the market, and it is hard to tell where cloud blurs with network at this point… Take it as why we’ve tried to make this transaction that I described earlier. But, I think it is very challenging now for an enterprise to go to the next step unless they have somebody that is on their side that is very familiar with us and has the relationships with us, and has the ability to reach out and you know, get a conversation with our executive team. At this point, for most enterprises, they don’t have dedicated resources that are doing that and want to be expected to do it all virtually but at the end of the day, it’s a big world, with a lot of vendors and a very confusing marketplace.

Max: [47.39] So we’ve been through periods of consolidation, expansion, startups and consolidation and bankruptcy… You know, you look at the internet and the providers in general, I mean… What’s your forecast for the next five years here? I mean, we’ve got a massive amount of capacity being required with COVID, likely a pretty significant change in how people work as a result of the COVID. I mean, what are you forecasting and what are you guys getting ready for in your next cycle?

Bill: [48.02] We’re getting ready for our customers to start to prefer their digital distribution channels over their physical, that’s a pretty clear line of sight to that. We’re certainly cognisant of the fact that over the top content will continue to grow at a more rapid pace. The online gaming space is absolutely heating up, as we’ve said before, and I think that will continue to morph into more new experiences that are delivered to consumers that will require more flexibility on the edge, security for work at home employees of enterprises is absolutely going to be critical… We do expect to see a major shift in the way that companies view their real estate investments, and this will have a sustained transformation in the way that we work, so that depends on tools to enable those sort of trends, which were there always, but accelerating at a rate that was slower than I think, you know, what we’re experiencing now. This is going to be a very quick acceleration in the cycle of those trends that were already underway.

Max: [49.13] You say dinosaur, I mean I got on the internet with a 1200 baud modem… Now, I’ll pretend like I was three when that happened, but it’s amazing to think back at you know, where this started and where we are now, and just the scale of it and how quickly it keeps growing and how that continues to increase. Bill, I’ll give you the closing thought here – anything we haven’t touched on that you want to talk about?

Bill: [49.38] I think the human impact of this global pandemic is just incredible. I know I’m stating the obvious there… We’re very fortunate at CenturyLink to be able to play a role in maintaining high performance in the internet throughout this event. You know, our services are sitting behind the many, many, many of the UCC tools that have become critical, our internet backbone, our content delivery networks have become integral in helping people stay sane during this process and I just think we’re very fortunate to be able to have the opportunity to play a role in helping commerce go forward through this event, and helping families to stay entertained and not go crazy at each other at home with nothing to do… I’ve just been very fortunate that I was here to be a part of this and my company was here to be a part of it.

Max: [50.28] Awesome. Bill, it’s always a pleasure speaking with you.

Bill: [50.31] Thank you, Max.

OUTRO: [50.32] Thanks for joining the Tech Deep Dive podcast. At Clarksys we believe tech should make your life better, searching Google is a waste of time, and the right vendor is often one you haven’t heard of before. We can help you buy the right tech for your business, visit us at Clarksys.com to schedule an intro call.