The transition to widely available broadband internet has dramatically changed the behavior of some end-users who consume inordinate amounts of bandwidth. The unexpected rise in demand from these users invalidated residential internet business models and forced service providers to introduce network congestion and traffic management practices, the most controversial of which is deep packet inspection (DPI), and sometimes go as far as introducing traffic caps on users.
The tug-of-war between a small percentage of bandwidth-abusing users and service providers has escalated into a full-scale conflict in terms of how much bandwidth users are paying for and what constitutes abusing the network. Bandwidth management issues that have come to the surface include cable operators trying to stop third parties from delivering video over their infrastructure, regulatory actions to oversee traffic management policies, and congressional hearings on broadband services. Add users who hog bandwidth and carriers that perform covert traffic management actions to that explosive mix -- the most infamous being the Comcast peer-to-peer (P2P) packet monitoring incident -- and you have all the necessary ingredients for a good technology soap opera.
The whole traffic management issue would be a gripping tragicomedy if it didn't affect the performance of the Internet, a vital piece of mission-critical infrastructure.
How we got here from there
Before broadband Internet technologies (cable, DSL and fiber) were introduced, almost all residential Internet services were based on dial-up access, which contains an implicit usage model: the longer you are connected, the larger your phone charges. Even when ISPs offered flat-rate services and there were no hidden charges, access speeds were so low that average line utilization closely matched maximum line speed.
The dial-up access model also provided clear separation between individual users, so their traffic mixed only on backhaul and transit links. The limited number of dial-in ports at each point of presence (POP) represented another bottleneck that limited core network congestion.
Fast-forward a few years to the broadband explosion, where the landscape has changed dramatically.
Numerous service providers offer broadband access that uses shared media (cable). The bandwidth of the access link is shared by many users, and a single misbehaving user can affect the whole segment.
The average bandwidth consumption has not grown in proportion to increased access-line speeds. It's hard to stress a 10 Mbit connection when you're reading email, chatting on IM, commenting on friends' Facebook photos or even watching YouTube videos.
Users are online all the time, and the dial-up port barrier has been busted.
Furthermore, ruthless competition among providers has eroded most of the profit margin, and they are forced to utilize large access-to-backhaul and even larger access-to-transit oversubscription ratios.
Realistic service definitions
All this wouldn't be so bad if service providers were transparent and truthful in their service offerings. In most cases, their marketing departments committed the original sin by launching fuzzy "service definitions" along the lines of "20 Mbps Amazingly Fast Internet Connections." The actual speed (beyond the local office) or user behavior expectations were never specified, apart from vague acceptable-use policies.
No wonder some users took the offer at face value, tried to use as much bandwidth as possible, and cried foul when service providers tried to renege on the deal with traffic management or transfer caps.
As a result of all of this, a number of technical issues plague the residential ISP market. For example:
- Users who don't conform to "expectations." The average usage of a typical residential user is a small percentage of the access speed. Service providers rely on this behavior and use high oversubscription ratios in their calculations, otherwise they could never achieve the financial break-even point with current low prices. A small percentage of "abnormal" users that saturate access links for prolonged periods, sometimes days or weeks, can adversely affect a large user population (more than on shared media systems like cable). The undesired behavior of "abnormal" users is often due to the usage of peer-to-peer applications but could also be a result of a popular Web server or file sharing server run by a residential user.
- Peer-to-peer applications. One of the fundamental design assumptions of the Internet is that each application session gets its fair share of bandwidth. The TCP protocol and existing queuing mechanisms in routers and switches are fine-tuned to achieving this goal, resulting in fair service delivery across a large user base, as long as each customer uses approximately the same number of TCP sessions. Peer-to-peer applications typically open a large number of constantly active TCP sessions, seizing a disproportionate amount of bandwidth for a long period of time.
- Video streaming affects other applications. Some video applications use TCP to download the content (YouTube or Netflix, for example). Others, particularly the streaming applications, use the User Datagram Protocol (UDP). Since UDP doesn't respond to congestion and packet loss as well as TCP, users receiving UDP video streams could squeeze out TCP users (Web browsing, email, etc.) in addition to using large amounts of bandwidth for a long time.
- Undesired applications tend to be obfuscated and encrypted. The moment a noticeable percentage of users started to use bandwidth-hogging applications, service providers engaged in a lose-lose game of trying to identify the offending applications and slow or block them. The developers of these applications responded by obfuscating them (using random TCP or UDP port numbers or pretending to be Web requests) and encrypting the content to prevent packet inspection looking for application-specific signatures
Introducing above-board bandwidth management
Some service providers have tried to use concealed mechanisms to minimize the impact of undesired applications, going as far as interfering with user sessions. To their dismay, they quickly discovered that the most affected users were also the most technically savvy and the most vocal, resulting in a huge public relations mess, which prompted the FCC to step in and take corrective actions.
Other service providers tried to introduce transfer caps -- metered and/or tiered service plans that bill customers according to the amount of data consumed -- that started with ridiculously low numbers clearly aimed at business problems other than solving traffic management issues. No wonder bloggers and industry press had a field day.
To avoid repeating these mistakes, other approaches to rational bandwidth use and traffic management are in order.
- Be honest with your users. Educate them about the issues, the limitations, and what you're doing to manage traffic and ensure that everyone has a good Internet experience. Some service providers are very specific in their definitions, including a real-time traffic graph of bandwidth use for users.
- Give customers realistic choices and usage plans. Some customers only want to surf the Web and read email. Others want to run P2P applications or watch streaming video. The choices aren't good or bad; they're just different. Give users a choice of usage plans and prioritize traffic based on that choice.
- Don't penalize your customers. It's usually better to downgrade the service of customers going over the monthly (or weekly) quota than to charge them automatically. Obviously, you should immediately inform them that they've exceeded the limits and give them an option to buy an additional transfer quota.
- Be as fair as possible. Networks aren't usually busy in the middle of the night. Service providers can give customers a free ride during that time frame. The knowledgeable users of peer-to-peer applications will quickly reconfigure their clients and stay off your network during peak hours.
Numerous mechanisms and technical solutions can be used to achieve these goals. Sometimes you might need dedicated boxes that can perform deep DPI, or it might be enough to monitor per-port utilization and interface counters, then react accordingly.
Don't forget: Whatever you do, keep your customers informed and happy.
Ivan Pepelnjak is a 25-year veteran of the networking industry
This article originally appeared on SearchTelecom.com