In the last few years the demands from webscale network operators (WNOs, Figure 1) on transmission network architectures have changed considerably. From pure raw capacity requirements and lower costs, webscale players now prioritize highly scalable and advanced point to point bandwidth bundling interface technologies.
Webscale operators’ field of expertise is the data center, and most planned at least initially to rent capacity as leased lines from telcos (or telecommunications network operators, aka TNOs). However, many TNOs did not have the end to end transmission networks able to support webscale needs in terms of capacity, latency, cost objectives. Further, telco networks were not flexible enough to follow rapidly WNOs’ needs for modifications, additions, and changes of the services they needed.
Hence several years ago, WNOs themselves decided to build their own backbone and regional transmission networks, sometimes linking continents. Undersea, the WNOs either leased capacity from existing submarine consortia systems or started to build submarine cables for their own dedicated use. The largest WNOs, such as Microsoft, Facebook and Alphabet, have increasingly favored the latter (self-build) approach. With these initiatives, WNOs seek to have full control of the transmission network, and adequate time to market for their needs.
As webscale players have built out their networks, they have become more influential across the industry. Their buying power alone is a major reason; Figure 2 shows how webscale operator capex has grown dramatically since 2012, while telco capex has stagnated.
At the same time, the telco market is far larger, and the largest integrated telcos spend well over 10% of capex on their transport networks. These telcos are heavily investing in the transformation of their transport networks, and supporting 5G is a central goal.
In the past, mobile services have been sold on the back of convenience of use. With very little considerations beyond coverage and without a capacity objective, there was never a firm commitment to service quality. 5G is probably the first access network with services subject to a wide variety of SLAs, from best effort to non-congested, and very low latency services with limits as low as 1ms. 5G will help operators to move from best effort services for all, to a tiered service level agreement (SLA)-based portfolio. Telcos hope this will help them to be more profitable, at least for the more sophisticated services.
Network slicing poised to play important role
The big change in direction in the strategy of transmission networks is that the planning, design, engineering and operations of a telecom operator will soon be subject to much tighter contracts and commitments.
For years, wild overbooking levels have been the norm, especially for mobile services, and networks were in most cases engineered for coverage alone. This won’t be possible for the next generation of services, which will require more than a 10x increase in bandwidth, and 10x less latency than the current generation.
TNOs and WNOs both accept that the demanding requirements on bandwidth, latency and operational scalability to ensure short time to market for 5G services cannot be supported with existing network architectures.
A potential solution is “network slicing”, which starts with adding more TDM capabilities in the data plane to be able to provide a hard separation in the way services with different KPIs use network resources. This separation is orchestrated by an SDN centralized control and management plane.
Network slicing brings improvements to traffic engineering, with clear KPIs for bandwidth, latency and packet congestion. That helps to support all types of services over the same network infrastructure. Low priority services such as web browsing are effectively separated from network resources dedicated to services with demanding SLAs such as low latency leased lines or 5G inter-vehicle communications.
Operators pursuing FlexO technology to help cope with looming Shannon limit
Historically the main requirements telcos have standardized for transmission network architectures and platforms are high resiliency, powerful operations administration maintenance features, multiservice support, and backwards compatibility with legacy platforms.
In recent years, technological developments have pushed transmission networks towards the limit in the bandwidth per distance product, or the “Shannon limit”. The transmission technology is starting to hit the limits of the fiber medium.
One way to cope with this comes from the ITU, with its Flexible OTN, or FlexO, standard (G.709.1/Y.1331.1). FlexO allows client OTN handoffs above 100Gbps by defining an “OTUCn” modular structure: “an aggregate OTUCn (n ≥ 1) can be transferred using bonded FlexO short-reach interfaces as lower bandwidth elements.” FlexO also supports standard 100GbE optical modules.
True to their backwards compatible requirements, telcos have made sure that FlexO supports 100G transmission requirements, and is an extension of existing OTN standards. This should simplify the roll out of FlexO on existing platforms.
FlexE to improve utilization, end-end manageability and router-transport connectivity
Operators – both telco & webscale – have also been exploring breakthroughs in the interfaces between transmission systems and servers and routers.
Aligned with FlexO, the Optical Internetworking Forum’s Flexible Ethernet (FlexE) supports similar schemes of bundling and multiplexing of interfaces between routers and transmission systems. FlexE offers a way to transport a range of Ethernet MAC rates whether or not they correspond to existing physical (Ethernet PHY) rates. Network utilization should improve, as should end-end manageability. One key element of FlexE was that Ethernet would grow within a TDM frame. This may pave the way to network slicing through the use of hard boundaries between tranches of services with different SLAs.
Conclusions
There is a growing alignment in the requirements for transmission network architectures across telecommunications and webscale network operators. They both need more flexible ways to grow their networks and manage them on an end to end basis. They want to benefit from low cost, open source components and procurement, but adapt technology to suit their customer base. They need to be able to support different classes of service and traffic. Even when providing free services, operators need to deliver a high quality of experience in order to monetize.
5G transport and data center interconnectivity services pose such a challenge to both TNOs and WNOs that work-arounds will not make up for limitations in either the data centers nor the network. For many operators, building a transmission network that supports network slicing principles will require a fresh start and new investments.
Matt Walker is chief analyst at MTN Consulting, LLC, an independent market research and advisory firm founded in 2017