The mobile data apocalypse, and what it means to you

The mobile industry is now completing a huge shift in its attitude toward mobile data. Until pretty recently, the prevailing attitude among mobile operators was that data was a disappointment. It had been hyped for a decade, and although there were some successes, it had never lived up to the huge growth expectations that were set at the start of the decade. Most operators viewed it as a nice incremental add-on rather than the driver of their businesses.

But in the last year or so, the attitude has shifted dramatically from "no one is using mobile data" to "oh my God, there's so much demand for mobile data that it'll destroy the network." A lot of this attitude shift was caused by the iPhone, which has indeed overloaded some mobile networks. But there's also a general uptick in data usage from various sources, and the rate of growth seems to be accelerating.

Extrapolating the trend, most telecom analyst firms are now producing mobile data traffic forecasts that look something like this:




The forecasts are driven by a couple of simple observations:

--Smartphones produce much more data traffic than traditional mobile phones. Cisco estimates that a single smartphone produces as much data traffic as 40 traditional feature phones. So converting 10 million people from feature phones to smartphones is like adding 390 million new feature phone users, in terms of impact on the data network. The more popular smartphones get, the busier the network becomes.

--A notebook PC generates far more traffic than a smartphone. According to Cicso, a single notebook computer generates the same data traffic as 450 feature phones. As notebook users convert to 3G-enabled netbooks and add 3G dongles to their computers, they dramatically increase the data traffic load on the network.

You can read Cisco's analysis here.

This becomes especially interesting when you look at the forecasts for growth of 3G-equipped netbooks and notebooks. Mobile operators in many countries have started subsidizing sales of those devices if you pay for a data service plan. It's an attractive deal for many people. Say your son or daughter is going off to college. Do you buy them a regular notebook computer and also pay for the DSL service to their apartment, or do you buy them a 3G data plan for about the same price as DSL and get the netbook for free?

The forecasting firm In-Stat recently predicted that by 2013, 30% of all notebook computers will be sold through mobile operators and bundled with 3G data plans (link). Notebook computer sales worldwide are about 150 million units a year, so that's 45 million new 3G notebooks a year -- or the data equivalent of adding 20 billion more feature phones to the network every year.

Jeepers.

These forecasts are producing a behind-the-scenes panic among mobile network operators. The consensus is that there's no way their networks can grow quickly enough to support all that data traffic. There are several reasons:

--They can't afford to build that much infrastructure.

--Even if they could afford the buildout, they won't have enough bandwidth available to carry all that data, even with 4G.

--Traffic-shaping techniques like tiered pricing and usage caps can't restrain usage growth enough to save them, because

--Fear of losing customers to a competitor will force them to continue to subsidize sales of 3G dongles and offer relatively generous caps in their data plans.

There are a number of projections that show the operators losing money on wireless data a few years from now, as costs continue to increase faster than revenue. The danger isn't so much that they will all go broke, but they're very afraid that they'll turn into zero-profit utilities.

Many operators now seem to be counting on WiFi as their ultimate savior. The theory is that if they can offload enough of the data traffic from their networks to WiFi base stations connected to wired networks, then maybe other measures like 4G, usage caps, and aggressive improvements to the network will let them squeak through.

It's an ironic situation. For a long time the mobile operators thought of themselves as the future lords of data communication. All devices would have 3G connections, the thinking went, and the fixed-line data carriers such as Comcast and BT would fade away just like the fixed-line voice companies are doing.

Instead, the new consensus is that we're moving to a world where the fixed-line vendors will be expected to carry most consumer data traffic for the foreseeable future. They'll provide your wireless connectivity at home and work, while the mobile network will fill in the gaps when you're on the move. The area of disagreement, of course, is who will get the majority of the access revenue. We'll let the fixed-line and mobile operators argue over that one; I want to talk about some of the other impacts of this weird new hybrid wireless world that we're heading into.

(I touched on some of this in my post on net neutrality a couple of weeks ago (link), but I want to go into more detail here.)


The brave new world of scarce mobile bandwidth

Built-in WiFi is now good. For a long time many mobile operators resisted selling smartphones with WiFi built in. They viewed WiFi networks as competitors for customer control, and wanted to prevent usage of them. Now that they see WiFi as their savior, the operators are suddenly encouraging its inclusion in phones. Don't be surprised if in the near future it becomes impossible to get a subsidized price for any smartphone that doesn't have WiFi built in.


Traffic shaping is a fact of life, and a likely source of irritation. Many mobile operators are starting to limit the performance of applications that consume the most data bandwidth (today that's mostly video and file sharing). It's already being done today, and in most cases the operators won't even tell you they're doing it, unless the government requires them to. Certain apps will just communicate more slowly, or fail altogether, when the network gets busy.

There are a couple of exceptions where operators have been more public about their traffic shaping activity. The 3 network in the UK recently announced restrictions (link). And O2 in the UK has given details on exactly which applications it restricts in its home wireless data service (link).

Current traffic shaping hasn't generated a firestorm of complaints from the average customer (as distinct from net neutrality advocates), in part because it is very hard for users to tell why a website runs slowly on a particular day. But as mobile traffic continues to increase, operators are going to find that it's cheaper to ratchet up the restrictions bit by bit rather than pay for more capacity. Eventually people will notice, and I worry that we'll end up in a situation in which the operators carefully balance out how much they can piss off their customers without creating an outright revolt. It's a lot like the way the US airline industry operates today, and it's a miserable experience for everyone involved.

What to do. There are better ways to shape traffic. I think operators should give customers more information on how much data they're using at any given time, so they can manage it themselves. Then let them make an informed decision about which apps they'll use their bandwidth on. It would be relatively simple to create an on-screen widget showing how much data is being transferred at any time, just like the signal strength and battery life indicators on today's phones.

It's also possible to create some APIs that would tell a website how much bandwidth is available to it, so the developer could adjust its features accordingly. This idea is being tossed around between web companies and operators, but I don't know how much is actually being done about it.

Combine those changes with usage-based pricing (my next point) and customers will shape their own traffic. Then there won't be any need for covert manipulation of the network.


Say hello to capped data plans. Completely unlimited wireless data plans are not sustainable long term; the economics of them just don't work. And in fact, virtually no data plans today are completely uncapped; there is almost always some fine print about the maximum amount of traffic allowed before surcharges kick in or the user is tossed off the network.

Some people are saying that the operators should go back to charging by the byte, and in some parts of the world (particularly Asia), there is a long history of per-byte pricing. But the experience in most of the world has been that per-byte pricing makes users so nervous about their expenses that they won't use data services at all.

(DoCoMo in Japan has an interesting hybrid approach (link) in which it charges per-packet until the user hits a maximum charge of about $70 per month. Additional usage beyond that cap is free. So that's capped pricing rather than capped usage. This reduces customer fear of accidentally running up a gigantic bill, but I wonder how DoCoMo prevents power users from flooding the network with traffic. Maybe there's a second, hidden cap on total usage.)

What to do. I think the right answer in most of the world is going to be flat-rate data plans in which there's a clearly-communicated cap, with tiered charges beyond that. The cap will need to be set at a level that moderate users won't ever reach, so they don't become gun-shy about data. To alleviate the fear of accidentally running up a huge bill, there will also need to be an on-device meter showing how much of the user's monthly data allocation has been used (just telling them to go look at a website is not enough; it should be on-screen). I'm told that on-screen meters like this are already being offered on netbooks by some European operators.

Today most operators are pretty up-front about communicating the data limits when a computer is connected to a mobile network. But many of them are still deceptive toward smartphone customers. AT&T's Smartphone Personal service, for example, promises the following for $35 a month:

Included Data: Unlimited; Additional data: $0 per MB

Sounds pretty straightforward. No asterisks, no fine print. But if you click on the terms of service (link), you'll find a long list of banned application types, followed by this general provision:

"AT&T reserves the right to (i) deny, disconnect, modify and/or terminate Service, without notice, to anyone...whose usage adversely impacts its wireless network or service levels or hinders access to its wireless network... and (ii) otherwise protect its wireless network from harm, compromised capacity or degradation in performance."

In other words, if the network is getting slow, they can do anything to your service, at any time, without notice.

There is also a hidden 5G per month maximum:

"If you are on a data plan that does not include a monthly MB/GB allowance and additional data usage rates, you agree that AT&T has the right to impose additional charges if you use more than 5 GB in a month."

This is not just an American problem. Orange in the UK calls its iPhone data service "unlimited," but there's a footnote saying that "unlimited" actually means 750 megabytes a month, a surprisingly low cap compared to AT&T's.

If we're ever going to collectively manage mobile network overload, we'll all need to be much more up-front about the way it operates and what a particular service plan will and won't do.


Is residential 3G really a good idea? Especially in Europe, it's common for operators to tell people that they should ditch their DSL or cable modem at home and replace it with a 3G modem. That works out well only when the network has excess capacity. As soon as the networks start to get congested, the operators will need to offload traffic to residential WiFi routers connected to DSL or cable. If those residential fixed lines have been removed, the operators can't offload.

What to do. I think this one is going to be self-limiting. Once 3G bandwidth gets scarce, the operators will realize that they can get a lot more revenue feeding data to smartphones than to PCs. The math works like this: With a given amount of bandwidth, you could support a single notebook computer and charge about $50 a month, or support 11 smartphones at $30 a month each. Hmm, $330 a month versus $50, seems like a pretty easy decision.

But there are two circumstances in which it would make sense for the operators to keep subsidizing PC sales:

1. If smartphone sales plateau. If this happens, eventually the network will catch up with demand and then there will be excess capacity for PCs; or

2. If operators can route most of the actual data traffic from PCs through WiFi connected to landlines. In this case they could sell you data plans knowing that you won't affect their networks much. That brings us to the next point...


Operators have a huge vested interest in unlocking WiFi access points. Most WiFi access points today are encrypted and inaccessible to other devices in the area. I think there's a strong financial incentive for mobile operators to work with fixed-line access companies to get those access points unlocked. The benefit for the wireless companies is clear -- the more WiFi points they can talk to, the fewer cell towers they need to build. But the benefits for the fixed-line operators are much less clear. Why should they help the mobile operators with their bandwidth crunch?

What to do. The ideal situation would be a revenue-sharing deal in which the operators share some money with the fixed-line companies to encourage them to open up access to their networks. In this scenario, your DSL or cable provider would give you a WiFi router that has been pre-configured to automatically and securely share excess bandwidth with mobile devices in the area. Your own traffic would get priority, but any extra capacity could be shared automatically. The benefit for you as a consumer would be a free router, and/or a lower DSL bill as the cable company passes along some of the revenue it gets from the mobile operators.

The effectiveness of this sort of approach is going to depend on the relative cost for an operator of subsidizing a set of WiFi base stations in an area, versus the cost of installing more wireless capacity. I wonder about weird scenarios like a DSL provider auctioning off excess WiFi capacity to wireless operators in a particularly congested area.


Femtocells for the rest of us. Another very logical step for the operators is to start pushing femtocells aggressively. (Femtocells are radios that work like a short-range cell tower, but are the size of a WiFi router. You connect one to your DSL or cable line, and it offloads traffic from the wireless network. Link)

What to do. Today femtocells are generally sold as signal boosters in areas with marginal wireless coverage. But in the future I think it may make sense for operators to give away femtocells, or at least subsidize them, for customers who live in areas where the data network is congested.


What it all means: Fixed-mobile convergence with a twist

If you step back from the details, the big picture is that we really need a single integrated data network that encompasses mobile and fixed connections, and switches between them seamlessly. People have been talking about this sort of thing for years (check out the Wikipedia article on fixed-mobile convergence here), but the focus has generally been on handing voice calls between WiFi and cellular. That's hard to do technologically (because you can't interrupt a voice conversation during the handover for more than a fraction of a second). Besides, it doesn't solve a significant customer problem -- the voice network isn't the thing that's overloaded.

The place where we could really, really use fixed-mobile convergence is in data. I'm worried, though, that the intense competition between the wireless and wired worlds will make it difficult and slow to achieve the coordination needed. This might be a useful place for government to put its attention. Not in terms of regulating the integrated network into existence (that would be the kiss of death), but to grease the skids for cooperation between the mobile and fixed-line worlds.


Just one more thing...

Everything above is based on the assumption that those Cisco and analyst forecasts are correct. But Cisco has a vested interest in hyping fear of the data apocalypse (Emergency! Buy more routers now!!), and my general rule about tech analysts is that every time they all agree on something you should bet against them.

There is a genuine crunch in mobile data capacity going on at the moment; you can read about network outages caused by the iPhone even today. And I can assure you that for every network failure you read about, there are dozens of other failures and near-failures that don't get reported. Many wireless data networks are very stressed.

And the situation will get worse.

But there's no such thing as infinite demand. At some point the growth of mobile data will slow down, and it's very important to try to estimate how and when that'll happen, so we as an industry do not overshoot too badly. The question isn't whether the growth forecasts are wrong, it's when they will be wrong.

I'll write about that next week...