Realistic service definitions
All this wouldn't be so bad if service providers were transparent and truthful in their service offerings. In most cases, their marketing departments committed the original sin by launching fuzzy "service definitions" along the lines of "20 Mbps Amazingly Fast Internet Connections." The actual speed (beyond the local office) or user behavior expectations were never specified, apart from vague acceptable-use policies.
No wonder some users took the offer at face value, tried to use as much bandwidth as possible, and cried foul when service providers tried to renege on the deal with traffic management or transfer caps.
As a result of all of this, a number of technical issues plague the residential ISP market. For example:
- Users who don't conform to "expectations." The average usage of a typical residential user is a small percentage of the access speed. Service providers rely on this behavior and use high oversubscription ratios in their calculations, otherwise they could never achieve the financial break-even point with current low prices. A small percentage of "abnormal" users that saturate access links for prolonged periods, sometimes days or weeks, can adversely affect a large user population (more than on shared media systems like cable). The undesired behavior of "abnormal" users is often due to the usage of peer-to-peer applications but could also be a result of a popular Web server or file sharing server run by a residential user.
- Peer-to-peer applications. One of the fundamental design assumptions of the Internet is that each application session gets its fair share of bandwidth. The TCP protocol and existing queuing mechanisms in routers and switches are fine-tuned to achieving this goal, resulting in fair service delivery across a large user base, as long as each customer uses approximately the same number of TCP sessions. Peer-to-peer applications typically open a large number of constantly active TCP sessions, seizing a disproportionate amount of bandwidth for a long period of time.
- Video streaming affects other applications. Some video applications use TCP to download the content (YouTube or Netflix, for example). Others, particularly the streaming applications, use the User Datagram Protocol (UDP). Since UDP doesn't respond to congestion and packet loss as well as TCP, users receiving UDP video streams could squeeze out TCP users (Web browsing, email, etc.) in addition to using large amounts of bandwidth for a long time.
- Undesired applications tend to be obfuscated and encrypted. The moment a noticeable percentage of users started to use bandwidth-hogging applications, service providers engaged in a lose-lose game of trying to identify the offending applications and slow or block them. The developers of these applications responded by obfuscating them (using random TCP or UDP port numbers or pretending to be Web requests) and encrypting the content to prevent packet inspection looking for application-specific signatures