Telecosm - Ike Elliott

Common VoIP Security Problem - Spoof Attacks

Today, let's talk about spoofing in VoIP.  It's a real security threat in a lot of VoIP implementations, unlike the hysteria surrounding the SIPtap tool that supposedly allows easy eavesdropping on VoIP calls. 

One of the more common spoof attacks in VoIP results from unauthenticated messages.   What can happen in a lot of VoIP implementations is that any computer on the Internet can send a message to your VoIP client and your VoIP client doesn't know that it is a message from a scammer or spammer. 

Here's an example:  Your Vonage VT 2142-VD phone adapter (from Motorola) receives a SIP INVITE message (trying to start a phone call with you) from a spam source.  This particular phone adapter doesn't authenticate incoming messages, so it responds to the SIP INVITE and sets up a call with the spam source.  Your Vonage phone rings and in a few seconds you are listening to a recording of someone trying to sell you Viagra.  Kind of circumvents the "do not call list", doesn't it?

How do you fix this kind of problem?  We have a three-layer toolbox for fighting spoof attacks:

1. Application layer: message authentication using SIP Digest.
2. Transport layer: authenticating a session so you only accept messages from know senders, using Transport Layer Security (TLS).
3. Network layer: encrypting messages using an IPSec mechanism.

Let's start with application layer message authentication.  SIP provides an HTTP digest authentication mechanism that allows any message to be challenged.  This means that the client could ask the sender to authenticate the message it just received, and the sender is required to re-send the message with credentials that the receiver can check to see if the message is legit.  In our example, the Vonage client could challenge every SIP INVITE and could avoid spam calls, assuming the credential mechanism used is hard to hack. 

Next: transport layer mechanisms.  Transport Layer Security allows the Vonage client to authenticate the session it sets up with its SIP proxy server at the time the session is set up.  This should be a bi-directional authentication, so that both the proxy server and the client are authenticated.  In our example, the Vonage client could then trust any message it receives on the authenticated TCP/IP session with the Proxy Server. (Note, SIP can use either UDP/IP or TCP/IP).

Network Layer Security using IPSec extends the IP packet header to allow encryption of any higher layer protocol.  You can secure all communication with a VoIP client by requiring that all traffic with that VoIP client be encrypted using an IPSec mechanism.  In our example, if the Vonage client received an unencrypted SIP INVITE, it would discard it. 

While you can use any of these three mechanisms in combination, that is kind of like having a belt and two sets of suspenders.  Of the three, the TLS-based mechanism is my preference.  It is more efficient than SIP Digest (not requiring the message overhead of challenges and responses on every message), and simpler than IPSec (not so many crypto keys and security associations to manage), and it gets the job done.

The problem is that so many VoIP services and VoIP clients use NONE of the above mechanisms, and that is a real vulnerability.

Verizon Move is a Small Step in the Right Direction

Verizon announced this week that they would open their wireless network to any cellphone that passes a quality test, becoming the first of the major US carriers to make such an announcement.  Blogs and news articles around the net are hailing the move as the dawn of a new era for the US wireless market.  I do think it is a step in the right direction, but I don't think that we are going to see an overnight transition to the Chinese model, where nobody buys their cell phone from the carrier any more. 

First, the good stuff.  Verizon's move is a real boon for the fixed-mobile convergence world and for VoIP, because now multi-mode 3G/WiFi phones can presumably be connected to Verizon's network, and other networks, and Verizon can't object.  A likely result is that we in the US will see phones and services like the Skype's 3 Skypephone, which is available in the UK, but not available here.  In effect, Verizon has opened the door to wireless minutes being siphoned off of their network and onto other wireless networks that use VoIP, like WiFi and WiMax networks.

Now, the bad stuff.  In the US, wireless carriers will continue to subsidize cellphones in exchange for a two-year service contract commitment, and as a result most phones will continue to be distributed through the carriers.  And the carriers can still insist that the phones that they subsidize are phones that don't let you use VoIP on services that they don't control.  So, in order to get an unlocked phone that can use multiple networks, you are probably going to have to pay a bit more for the phone. 

That's the model that predominates in Europe, where you can get an unlocked phone on any network but you are going to have to pay more for an un-subsidized phone.  According to the article in the Wall Street Journal this morning, only 5% of phones sold in Europe are unsubsidized. 

Also, the wireless carrier's own 2.5G and 3.5G data networks will have greater coverage areas and be more reliable than WiFi hotspots, so VoIP will probably use the wireless carrier's own data networks pretty often, meaning the wireless carriers will win anyway.

Om Malik is also skeptical that this announcement is as big as some think, citing the fact that Verizon's network is CDMA and not GSM so you can't just pop a SIM card from your Verizon phone and put it in a GSM phone and use that phone on another network, like you can in much of Asia.  Read his post here.

So, Verizon deserves credit for opening the door a crack.  However, we are not about to walk through the door into a new and completely open world, at least not yet.

FCC Backs Down on Cable Regulation

Last night, FCC Chairman Kevin Martin couldn't get a majority of commissioners to vote with him on his plan to extend FCC regulation over the cable industry.  It was a serious embarrassment for Martin, but it was the right call for the country.  The story broke in early November, when Martin started saying that he would use the 70/70 rule in the 1984 Cable Act to extend the FCC's power over the cable industry.  It turns out that most of his fellow commissoners didn't believe Martin's statistics.  Good thing.

FCC May Have to Back Down on Cable Regulation

There's an article on page 3 of the Wall Street Journal this morning, saying that the FCC vote that had been scheduled for today to initiate a new era of cable regulation may be canceled.  I reported on this blog a few weeks ago about FCC Chairman Kevin Martin's plan to use faulty statistics to justify regulation.  The FCC is getting a great deal of pressure from both parties in congress to back down, and there is internal disagreement among FCC comissioners, notably from commissioner Robert McDowell.   Even Jesse Jackson thinks Martin is too regulatory.

I'm hoping the meeting is not only postponed but canceled completely.  The FCC under Martin's leadership has been overactive on this issue.

VoIP Security Scare Tactics

There has been another VoIP Security scare in the past few days, as Peter Cox demonstrated his SIPtap tool that enables unauthorized wiretapping of voice over IP calls.  The problem with the kind of reporting in the PC World article on the topic is that it exaggerates the the threat of this kind of security breach.  Mr. Blog has a good post describing why this kind of eavesdropping is a lot harder to do than Peter Cox would have you believe. 

It's a shame, too, because there are very real VoIP security threats that people should be worried about, but people are often distracted by trying to deal with this kind of lower-probability threat.   

I'll be posting this week on some of the real VoIP security threats that service providers and users really do need to be worried about:

• Identity spoofing (user authentication issues)
• Network spoofing (network authentication issues)
• Denial of service attacks
• Undermining firewalls
• Privacy issues
• Topology hiding

A lot of these issues are easier or harder to deal with depending on the network architecture you or your service provider have chosen. 

Stay tuned for more, and if you want to add to the list of VoIP security issues you would like to see a post about, let me know.

This Google Rumor Might Actually Be True

Rich Tehrani posted a rumor a couple of days ago that Google might be interested in buying Skype from eBay.  Unlike Rich's post from last week about Google possibly buying Sprint, I think Rich might be on to something with the Skype rumor.  eBay certainly hasn't done enough with Skype since they bought it, and took a big write-down in the value of Skype recently, so they could be interested in unloading it.  Google has the right platform to generate a lot more revenue from those Skype subscribers. 

Jemima Kiss first reported the rumor on her blog.  A commenter there thinks that Google is too tied up with GrandCentral's involvement with the Gizmo project, and that Google would favor Gizmo's SIP-based approach rather than Skype's proprietary peer-to-peer technology.  To me, though, Google could come up with a solid business case for an acquisition of Skype just by figuring out how much ad revenue they could make on Skype users, and they could remain indifferent as to choice of underlying protocol. 

What do you think?  Is Google ready to gun for Skype?

An Attitude of Gratitude

Wow, do I have a lot to be thankful for!  It is Thanksgiving day here in the US, and I have a roof over my head in a warm home, family by my side, plenty of food in the house, great friends, 145 great subscribers to this young blog...the list goes on and on. 

A friend of mine who is building a very successful VoIP service provider tells me the most important characteristic he looks for when hiring people is "Do they have an attitude of gratitude, or a sense of entitlement?"  I think most folks would agree that they like to be around grateful people.

That's part of why Thanksgiving is my favorite holiday of the year, a full day set aside to be thankful, to cook and eat great food, to be with family and friends.  And, as a reminder to myself that I have been blessed, I try to remember that there are many people around the world and in our communities who are far less fortunate, and I try to share my good fortune with them. 

Most countries don't have a Thanksgiving holiday, so I often ask my American friends who live in foreign countries what they do on Thanksgiving.  They generally celebrate as best they can (they can't always get the same kinds of food), and try to share the celebration with the people around them.  That feels right to me...we can do a lot to spread the attitude of gratitude to everyone we come into contact with by keeping perspective and making sure we remain grateful ourselves, and by sharing.

Happy Thanksgiving,

Ike Elliott

Wall Street Journal's Holman Jenkins vs. Kevin Martin in Cable War

Holman Jenkins wrote a rambling editorial in this morning's Wall Street Journal, titled "Web War III" , in which he opines about a number of loosely connected convergence topics, including:

• The big telcos, Verizon and AT&T, have the pole position in the convergence wars
• Google is no match for the big telcos, not owning a network of either the wireline or wireless variety
• Content providers don't need regulatory protection against network owners
• FCC Chairman Kevin Martin's crusade against the "cable monopoly" is off base and unnecessary.

This last point echoes a point made on this and other blogs in recent posts.  The FCC doesn't have a clear legislative mandate to extend its cable regulation powers, and if it does continue hand-cuffing the cable industry then it is only beating down an industry that is already hurting, as Comcast and the other cable companies are reporting subscriber losses. 

Only one other comment on the Jenkins article... I think Google IS a match for the telcos in some important ways.  Sure, the software/search/advertising giant doesn't own a network, but they do own a ton of know-how and intellectual property (57 US patents and counting).  Having worked in software development departments for a couple of telcos, I am certain that Google does software development better than any telco.  And being unencumbered with operating a network, Google is free to distribute its software-based services on any public Internet, with the notable exception of the walled-garden wireless phone networks.  I wouldn't count Google out yet.

Comcast's Acceptable Use Policy Permits Delaying of BitTorrent Traffic

There is a good post over on the Technology Liberation Front, in which Cord Blomquist has done the research into Comcast's Acceptable Use Policy to find that Comcast does have the right to block BitTorrent traffic if it wants to.  Whenever a Comcast subscriber signs up for service they have to acknowledge that they have read this policy and that they accept its terms, so it looks like all of the screaming about Comcast violating net neutrality morals is just an argument that Comcast should change its Acceptable Use Policy, and not a justification for regulation. 

I posted on this topic last month, shortly after the AP broke the story that Comcast was sometimes delaying BitTorrent traffic, and I thought then and I still think now that Comcast is doing nothing wrong. 

That doesn't mean that I am a softie on net neutrality issues...I'm just saving what powder I have for bigger battles. 

Why Doesn't Verizon Pick On Somebody Their Own Size?

I had a comment on yesterday's post about the validity of Verizon's VoIP patents, asking why Verizon sued Vonage, rather than the gatway-manufacturer Cisco, or softswitch manufacturer Broadsoft?  This is one of the easier questions on the "Big List of VoIP Patent Questions" post from a few days ago, so we'll tackle it today.

How does a patent holder decide who to target?  They usually consider the following:
1. Are we pretty sure the target is infringing our patent?
2. Are we infringing any of the target's patents?
3. Can we win a lawsuit if it goes to court?
4. Is it important to maintain positive relations with the target?
5. Is the target a competitor that we would like to impair?

In the case of Verizon and Vonage, they came up with the right answer to all of the above questions.  Verizon was pretty sure Vonage was infringing.  My searches on the patent search engines didn't turn up any patents that have been granted to Vonage (at least, not yet), so Verizon was pretty confident they wouldn't run into a counter-suit.  Verizon knew they had a good chance of winning in court because they could afford really good lawyers, and maybe Vonage wouldn't hire really good lawyers.  Most importantly (to Verizon), Vonage was a sworn enemy and Verizon liked the idea of impairing Vonage's business.  Check marks on all the criteria.

Compare this checklist to Cisco, now.  Cisco may be infringing Verizon's patent, check.  However, Cisco has one of the largest VoIP patent portfolios in the industry, and Verizon is probably infringing a bunch of those patents, so a counter-suit would be probable.  Cisco can afford great lawyers, so Verizon might not win in court.  Cisco is a big supplier to Verizon, so Verizon would want to maintain good relations there.  And Verizon has no reason to want to hurt Cisco's business.  Only one check out of five, so it's not likely that Verizon would sue Cisco.

If you do the same analysis with softswitch-maker Broadsoft, you'd find some similar results.  Broadsoft has a couple of patents (6,625,274 and 7,213,073), and Broadsoft is a supplier to Verizon (see Broadsoft's home page).  Broadsoft might not be able to afford as much legal counsel as Verizon, but with only two checks out of five, Verizon has no reason to pick a fight with Broadsoft.

Another competitor in the consumer VoIP market, Packet 8, has a dozen US patents.  They wouldn't be an easy target, either. 

The painful truth for Vonage is that they were a big, easy target, and didn't pay enough attention to protecting their business with key patents.  I'm sure they've learned their lesson now, but is it too late?

Are Verizon's VoIP Patents Valid?

One of the readers of this blog asked if Level 3 Communications VoIP Patents would be considered prior art when analyzing the Verizon VoIP Patents.  I think the broader question is also interesting:  was there enough prior art laying around at the time that Verizon filed for their patent that Verizon's invention should have been "obvious to one of ordinary skill in the art"?  The court obviously didn't think so, or they wouldn't have found Vonage to be infringing.  Not being a patent lawyer, I couldn't tell you with certainty if the court was right or not.  I can provide some background on what was known at the time the patents were filed, though.

These are the Verizon patents that Vonage was initially found to infringe:

6,104,711, filed March 6, 1997
6,359,880, filed July 30, 1999
6,282,574, filed February 24, 2000

Now, the filing date is not necessarily the "date of invention", so you can't really go on the filing date by itself.  A company could have compelling evidence (in engineering notebooks and email records, for example) that the technology was invented somewhat earlier than the filing date, and that they only got around to filing the patents later on.  However, there are limits on how big this lag time between invention and filing can be, so filing date is a decent proxy for invention date for our little analysis here.

Also, patents can be filed in such a way that they are associated with patents that were filed earlier, and therefore get the benefit of the earlier filing date.  In Verizon's case, for example, the 574 patent was related to the earlier 711 patent, so both patents could claim the 1997 filing date. 

So, what kind of prior art was laying around in early 1997?  I already mentioned in yesterday's post that Jeff Pulver had already published a book about his Free World Dialup project and how it worked, even before Verizon's patents were filed.   Also, companies like VocalTec and NetSpeak already had VoIP gateway products on the market and in use at that time.   The ITU had already developed the H.323 VoIP standards by then, publishing their first specs in November, 1996.  These specs included a mechanism for translating names or phone numbers into IP addresses, of the sort described in the Verizon patent.  The first drafts of the Session Initiation Protocol (SIP) had already been submitted to the Internet Engineering Task Force in February, 1996. 

My own first VoIP patents were filed in November of 1996, while I was at MCI, so these patents could come into consideration as prior art when analyzing the Verizon patents.  My Level 3 VoIP Patents were filed in November, 1998, so their filing dates are later than the earliest Verizon patents. 

Were the Verizon patents obvious?  Timothy Lee, publishing on the Ars Technica blog, seemed to think so, as did Jeff Pulver.  I also was aware of the kinds of signaling protocols being developed for VoIP at the time, and would have found the sorts of things claimed in the Verizon patents to be obvious.  The question is whether folks like Jeff Pulver or myself would be considered by a court to of "ordinary skill in the art", or having somewhat greater expertise. 

So, this leaves an impression that Verizon saw a threat in the development of VoIP technology, and tried to mitigate that threat by patenting as much as it could, as early as it could, around the use of that technology in real networks.  They weren't alone in that effort as demonstrated by the long list of other carriers with lots of VoIP patents.  I think what strikes most people as wrong is the perception that these carriers patented the technology defensively, with no intention to aggressively develop and use the technology, instead allowing others, like Vonage, to try to build a customer base.   

This is the same thing that we find offensive about patent trolls.  Shouldn't an inventor of a technology (or their company) be required to provide reasonable and non-discriminatory licensing terms for their technology, if they aren't going to use and develop the technology themselves?  Shouldn't the patent office establish a higher standard for granting a patent, so that companies can't go out and patent what is already in the public domain in an effort to corner a piece of the market? 

One consolation in all of this is that utility patents such as these have a life of 20 years from the filing date, after which anyone can use the technology.  So, the Verizon patents dating back to 1997 have less than 10 years of life left.   

If patents are going be used primarily to defend the market position of incumbents, maybe it is time to reduce patent duration from 20 years to something lower, say 12 years? 

The VoIP Patent Landscape

Yesterday, I posted a big list of VoIP Patent questions that friends have been asking me ever since I started blogging.  Since I've got clients who retain me as an expert witness in patent lawsuits, and since I'm an inventor on 18 patents, they figure I might know the answers to questions like "Don't other companies have VoIP patents, too, not just Verizon, AT&T, and Sprint?"   

The truth is, anybody can find out just by doing some targeted searches on either the patent office web site, or on Google Patents.  I'll save you a little searching though...there are on the order of about 1,000 VoIP patents out there, and many, many companies own little corners of the VoIP intellectual property universe. 

Is there really that much to patent about VoIP that we need 1,000 patents?  No, of course not.  The US Patent Office reported a 3.5% patent allowance error rate last year, meaning that of the over 164,000 patents the office allowed last year, close to  6,000 of them shouldn't have been granted.  That's just the ones the patent office knows about. 

The truth is that the patent office does grant patents that it shouldn't, because there is no way for the USPTO to employ true experts in every field as patent examiners.  So, patents are sometimes granted when there is already another patent on the same technology, or when there is a lot of "prior art" (evidence that the invention is already widely known before it was "invented").  Many times, this kind of error doesn't get sorted out unless and until there is a patent lawsuit to draw attention to the issue. 

I think that's the process that has started with the Vonage suits.  Vonage might have really been infringing some of these patents, but there is a question as to whether some of the patent claims should have been granted in the first place, since there was prior art out there, such as Jeff Pulver's book, "The Internet Telephone Toolkit."  In Vonage's case, either they did not present a clear enough case to invalidate the patent claims, or they didn't have very good legal representation, or they couldn't really find good prior art for the claims in question, and the were found to be infringing. 

And so, the dance of the elephants begins.  There are many large companies with large VoIP patent portfolios.  Having a large portfolio of good VoIP claims can really help you preserve freedom to operate a VoIP service, since you can cross-license your patents with other patent holders instead of paying royalties to each other.   Here is an informal table of some of the large VoIP patent holders:

	Voice over Internet Protocol	Internet Telephony	Total
Cisco	125	36	161
Nortel	79	77	156
AT&T	52	93	145
Lucent	26	59	85
Verizon	30	42	72
Ericsson	23	39	62
Siemens	14	19	33
Sprint	17	6	23
Level 3	15	6	21

*NOTE: above table contains results of a search on Google Patents for patents assigned to each company with either the phrase "voice over internet protocol" or "internet telephony" somewhere in the patent. The same patent could result from either search, meaning there could be some double-counting in the above table.  Also note that AT&T's total includes SBC's patents, Verizon's total includes MCI's patents, and Level 3's total includes Genuity's patents.

Without actually diving into the detailed specifications and claims of each of these patents, it is hard to say who is really well-protected against VoIP lawsuits.  I'd guess that most of the companies on the list above are pretty free to do what they want in the VoIP space by virtue of their patent portfolios, but it could be that some of the players have weaker claims than others. 

But when the elephants start dancing, it is usually the smaller animals that get hurt.  That's what happened to Vonage.

VoIP Patent Questions

A friend of mine saw yesterday's post about the bleak prospects for Vonage, and how one factor contributing to the uncertainty surrounding the company's future is potential for patent infringement.  Vonage has already paid over $240M to settle patent infringement claims from AT&T, Verizon, and Sprint Nextel. 

My friend proceeded to ask me about a jillion questions about VoIP patents, more than I can possibly answer in one post, and some of which I probably can't answer at all.  See, I am an inventor named on 18 patents, several of them in the VoIP space, but I'm not a patent lawyer, so I'm not really qualified to answer some of the questions.  I'll do my best, though.

I've adapted my friend's questions and added a few of my own here, and will commit to starting to answer some of these questions as best I can in a series of posts over time.  Drumroll, please....the questions:

1. Why is Vonage the target of all of these patent lawsuits?  Aren't there other VoIP infringers?
2. Don't other companies have VoIP patents, too, not just Verizon, AT&T, and Sprint?
3. Was Vonage really infringing, or did they just have inferior legal representation?
4. How broad are the patents that Vonage supposedly infringed?  Would any VoIP application also infringe?
5. Why haven't Verizon, AT&T, and Sprint gone after other infringers?  Are they allowed to be selective about who they target?
6. Why aren't VoIP software and equipment manufacturers also being targeted with VoIP patent lawsuits?
7. Is it now impossible for an independent VoIP service provider to make a go of it?  Can any VoIP service provider survive?
8. Some specific questions about Level 3 Communications:
   a) Would LVLT VoIP patents qualify as prior art and thus negate the Verizon, ATT, and Sprint suits?
   b) What keeps LVLT from coming to Vonage's defense... are LVLT's patents not strong enough?
   c) Is the LVLT Voice Termination business in jeopardy because of these patent wars?

I'm sure there are other questions that can be asked that are just as interesting as these, so bring 'em on! 

Meanwhile, here is just a little bit of background, just to get started, which might shed a little light on why some of the above questions may be hard to answer. 

I did some searching of online patent databases on a few key VoIP phrases, just to see how many patents mention these phrases, and here is what I came up with:

Phrase	Patent Count
Voice Over Internet Protocol	525
Internet Telephony	548
Packet Voice	476

Of course, many of the patents that mention "voice over internet protocol" might also mention "internet telephony", so I am sure theere is quite a bit of double-counting in the table above.  But I think it is safe to say that there are at least 600, and probably a lot more, patents that explicitly say something about VoIP or packet voice applications. 

The earliest patent in the above table is a Stanford University patent that was filed in 1981, patent number 4,503,533, and the patent specification explicity discussed the performance of packet voice applications on a certain kind of network design.   So, the fundamental idea of using packet networks for interactive voice conversations has been around for a long time.  Internet founder Vint Cerf once told me that they were thinking about using voice applications on packet networks way back in his DARPA days, which I think was as early as 1969.  Every little packet voice improvement, though, is patentable, and that's how we ended up where we are today.  More to come.

Google's Next Mobile Move

This morning, the Wall Street Journal summarized a lot of the speculation that is going around about Google's potential next steps in their drive to expand into the mobile world.  Options include:

• Buying a mobile carrier
• Licensing spectrum and becoming a mobile carrier, with or without a partner
• Building and marketing a mobile handset
• Continuing driving the open-source Android operating system as a lever to get mobile carriers to open up

Om Malik posted today on the topic, asking what people think Google will do.  I am already on record as saying that I don't think Google will buy a carrier like Sprint, and I don't think Google will build a handset.  Too much money and too far outside of Google's sweet spot. 

The spectrum licensing idea, though, may have merit, especially if Google bids on and wins the restricted "C" block that the FCC carved out for open handsets.  (I reported on this in the context of the "unlocked" iPhone, back in October).   The FCC placed the restrictions on the C block in response, in part, to Google lobbying for an open wireless network, and while Google didn't get all of its wishes, the FCC did give Google some of what it wanted: licensees of the "C" block are required to allow any device or application to have access to this spectrum. 

That is no hardship for Google...it is exactly what Google wants.  However, the large and established wireless carriers like AT&T and Verizon Wireless are already on record as opposing open networks, claiming it is too much of a security risk.  The truth is they also see a business risk - the potential lost revenue as dual-mode handsets siphon off service revenue onto other networks.  So, to an AT&T or a Verizon Wireless, the "C" block is less valuable and therefore the licenses may go for less money.

Or will they?  Will the mega-carriers bid heavily on the "C" block just to keep Google out of the wireless game?  I think this is why Google has hired game-theory experts to help them in the bidding process, as reported by this morning's Journal article.

If Google does end up winning spectrum licenses in the auction this winter, I'd bet that Google will quickly turn around and partner with an established carrier to operate the network.   That way, Google has the best of both worlds: an open network to demonstrate a new wireless business model, and no network operations headaches to deal with.

Vonage's Goose is Cooked

More bad news for a consumer VoIP player this morning: Vonage lost their appeals request in the Verizon Communications intellectual property lawsuit.  They now have to pay $120M, including $2.5 million to certain charities and the remainder to Verizon.  This is on top of $80M to Sprint to settle another patent lawsuit, and an estimated $39M to AT&T, for a total of $239M to settle patent lawsuits.  Vonage raised about $531M in last year's IPO and now they have handed nearly half of the cash over to their bitter enemies, the telcos.

And that's not even the worst of Vonage's problems.  Their cost per gross subscriber line addition of $206 is still too high as compared to revenue, and when you combine that with their churn rates of 3% per month (reported in their 3rd quarter earnings), Vonage is only barely hanging on.   My quick calculations show that if Vonage does not drastically cut their SG&A costs, they will run out of cash next summer.

This comes after SunRocket closed its doors this summer, and Skype disappointed their acquirer, eBay, with slower growth in subscribers and ARPU than they hoped.  What's going on?

Independent consumer VoIP players are facing some mighty tough headwinds:

1. It takes Vonage 18 months to make back its cost of customer acquisition on operating cash flow (not including SG&A)...its a bad business that isn't likely to attract much additional investment now.  Vonage is stuck, because adding more subscribers at $206 per sub is only going to make them run out of cash sooner. 
2. The big carriers have much deeper patent portfolios than the consumer VoIP players and are using them as a competitive weapon.
3. Beyond patents, consumer VoIP is a market with no barriers to entry, meaning constant competition will cause constant downward pressure on prices.  The only way to improve margins is through cost-cutting, which is hard to do when you don't own your own network.
4. In addition to the telcos, the cable companies are winning market share in consumer VoIP, armed with a deep customer database and a low-cost network that is already paid-for by other services.

Vonage's only real asset now is their 2.5 million subscribers.  I expect Vonage will sell that asset to the highest bidder within the next 9 months, and call it quits.

Skype stands to benefit from the rapidly dwindling supply of consumer VoIP competitors, and will survive because of their lower cost position and lower prices.  eBay paid too much for Skype, but at least the one they picked will survive the shakeout.

Dennis Kyle's Words of Wisdom to Telecom Sales Teams

I think the managing telecom salespeople is one of the trickier roles in telecom.  It's not that telecom salespeople are particularly "high needs" or anything... it's just that telecom salespeople need to be good at so many different things.  Great people skills, a thorough understanding of their products, a detail-orientation so that a customer's needs can be accurately communicated to the operations team, and a thick skin...these traits can go a long way toward success in a telecom sales role. (Maybe this is true of any kind of sales?).  Managing these talented people is part cheerleader and part guidance counselor, part benefactor and part hammer.  So I'm always looking for words of wisdom on how to do the job well.

There is an excellent post on Dan Caruso's Bear on Business Blog this morning, telling the story of a technique that my good friend Dennis Kyle uses to set the rules for success with his sales team.   Dennis is a real pro, so his words are truly words of wisdom.  "We can work on one problem, but not two."  Check it out over on Bear on Business!

Level 3's Crowe a True Leader

Last month, when Level 3 Communications announced their 3rd quarter earnings and disappointing new guidance, I was pretty steamed, like a lot of shareholders.   The stock had enjoyed a premium valuation for the first half of the year due to management's optimism about the improving financial performance of the company.  Then, in the 3rd quarter report, we shareholders had the rug pulled out from under us, and it was not the first time.

The investor bulletin boards were riddled with calls for heads to roll, especially CEO Jim Crowe and President/COO Kevin O'Hara.  Having had three weeks to stew on it, I went back and listened to the earnings conference call again yesterday and listened carefully to the Q&A part where an analyst asked management who was to blame for the company's provisioning problems and asked if heads shouldn't roll.  Crowe's answer reminded me why I definitely would not favor his removal:  he is a true leader and would be very hard to replace.

Here is what he said: "I'll say again, the responsibility for the provisioning issues, which I've tried to be clear about, and the consequences, which I've tried to be clear about, flows up through the operating organization to Kevin and I.  If I thought there was a lack of recognition of the problem, if I thought there was a lack of appreciation of the causes or the seriousness of the problem, I think it would be completely appropriate to have answers to your questions.  I see none of those, and I can assure you, by the way, if I didn't feel that way, our board would. I'm not of the school that says a mistake, in and of itself, is cause to start terminating people.  We're moving quickly, we're an organization that has covered a lot of ground in a short period of time.  We've made mistakes in the past.  We'll make mistakes in the future.  Analyzing those mistakes, understanding them, and correcting them quickly is what I believe is important, and if I see any evidence that that is not occurring, then I do think it is appropriate to look for other individuals that will fix those problems.  But I don't see any evidence of that.  Next question."

As the leader of the organization, Crowe took the responsibility for the revenue miss, and put the focus on fixing the problem rather than fixing the blame.  He left the door open to making changes if the problems weren't fixed, but he backed up his existing team, saying, in effect, that we all make mistakes and we'll continue to do that but what should set us apart is how we work to fix the mistakes and how we work to avoid making the same mistakes again.

Having worked as an executive at Level 3, I know I had a great deal of respect for Crowe and his leadership qualities.  Taking the arrows, as he did on October 23rd, inspires a lot of loyalty in his team.  Crowe understands he can't do that every quarter, and that the company does need to turn it around or his own job may be in jeopardy.  However, it is refreshing to see a leader who puts himself at the head of the line when it is time for heads to roll.  It would be a shame to see him go.

Another Blog Joins the Fight Against FCC Bullying

It was good to see this afternoon's post on the Technology Liberation Front (one of my favorite tech blogs), echoing my post several days ago denouncing FCC Chairman Kevin Martin's plan to regulate Cable based on a flimsy interpretation of "statistics" and the 1984 Cable Act.  James Gattuso wrote an excellent summary of the issue, it is worth a read. 

I know it is easy to get caught up in the populist thinking, that cable rates are too high, and that channel bundling limits consumer choice, and yes, these are real problems.  However, in the absence of evidence of monopoly behavior on the part of the cable companies, I'd rather see the market do the dirty work of busting open the video services market, instead of seeing the heavy hand of the FCC intervene with regulations that may haunt us for years after the "problem" has morphed into something entirely different.  Keep in mind, congress and the FCC are the ones responsible for the strange "access charge regime" and "universal service fund" that have corrupted carrier economics for a few decades now, and it is long overdue for an overhaul.  Once you let the government in, it is really hard to get them out again.

Google Buying a Network?

Rich Tehrani posted his blog a few days ago on the rumor that Google was going to buy Sprint Nextel, and suggested that it could really happen, since Google tried to do video on its own and ended up acquiring YouTube when it realized YouTube had a lot more traction.  Rich's logic is that Google is in the same position in wireless, on the outside looking in, and will end up buying a wireless network to gain entry into the wireless club.

Andy Abramson responded with doubts on his VoIPWatch blog, and my thoughts are largely in line with Andy's.  I am very doubtful that Google would buy Sprint.  Google has their eyes on a bigger prize: dominating the mobile search and advertising space for all the carriers, not just one of the smaller US carriers.  In this case, I think that Rich is also a bit too America-centric.  The Google Open Handset Alliance already has more carrier traction outside fo the US, where mobile networks are already more open (see my post on VoIP Devices and the 3 Skypephone in the UK).  If Google's OHA gets traction in Europe and Asia, Google wins even if the two biggest US carriers don't open up.  And Sprint is already in the OHA, so why would Google need to buy them? 

Andy goes on to speculate that T-Mobile, Qwest, Verizon, or Level 3 might buy Sprint.  I'm open to the idea that any of the first three on this list might buy Sprint, but Level 3 wouldn't be a likely buyer.  Buying a retail consumer mobile service provider is nowhere near the center of the bullseye for Level 3, who took years to get comfortable with serving enterprises, let alone consumers.  I really doubt Level 3 will go after Sprint.

There has also been persistent speculation on some of the investor bulletin boards, for eons, that Google might buy Level 3.  Again, I hate to rain on the parade, but I think Google would be foolish to do that.  Yes, Google might realize some economies by owning a network, but not enough to justify buying the network.  Google has it good right now, getting great deals on bandwidth services from Level 3 and other carriers, and not having to worry about actually operating the network.  Why buy all those headaches?  I don't see a strategic driver for Google buying Level 3.

What Kinds of VoIP Clients Work Best, and Why?

Determinism, determinism, determinism.  Real-time network applications like VoIP work best in deterministic environments, where the network and the device on which the application is running provide predictable performance.

In VoIP clients, there is a pretty simple rank-ordering of the most deterministic, and therefore most reliable, kinds of devices, from "VoIP-friendliest" to "VoIP-unfriendliest":

1. Dedicated, wireline VoIP Phone
2. Dedicated, wireline Analog Telephone Adapter
3. Dedicated, cordless DECT VoIP Phone Adapter connected to a wireline IP network.
4. Wireless private-spectrum VoIP Phone
5. WiFi VoIP Phone
6. Wireline-connected PC
7. WiFi-connected PC

Let's look at this list from the bottom up.  PC's are pretty unpredictable environments for VoIP.  They usually run a multi-tasking operating system that was not designed for real-time applications and as a result, your VoIP application is often competing with other applications on your PC for a limited set of resources, and your application performance can suffer. 

WiFi networks are also highly unpredictable, because

• they are subject to wide variations in signal strength with a small shift in location,
• they are subject to interference from other nearby radio frequency signals because the spectrum is unlicensed, and
• they are usually a shared resource and you can't control the other users of the network

If you combine a PC with a WiFi network, you have the worst of both worlds: an unpredictable processor for running your application, and an unpredictable network for sending and receiving packets.

Yesterday, I blogged about Skype's recent troubles with service reliability on their WiFi phones, such as the Skype phones from Belkin and Netgear, and I commented that I expected Skype would have a hard time reaching service levels expected by business customers, because they don't control the quality of service (QoS) on their customer's Internet connection. 

I'm actually a big fan of Skype's strategy to escape the PC and offer alternative VoIP client devices.  Why? Because the PC is a highly unpredictable environment, very inhospitable to real-time applications like VoIP.

WiFi phones, however, won't offer the best user experience.  Yes, you get the joy of puddle-jumping mobility whenever you are in a hotspot.  However, the phone service will only work well if your WiFi signal is strong, and the WiFi network and its connection to the Internet are uncongested.  In many cases, neither of these factors will be under your control, and Skype has no control over these factors, either. Not very predictable quality = not very reliable service.

I think that over time, once the bugs are worked out, wireless VoIP phones that use licensed spectrum networks, like the 3 Skypephone that is currently offered in the UK, will work  better than WiFi phones and PC VoIP clients, because the network these phones use will presumably be more predictable than WiFi networks.  Also, a wireless handset may work better than a PC for VoIP, because the computer (the handset) that the VoIP application is running on can be designed for real time applications and is less likely to be distracted by running a bunch of other apps at the same time.  Right now, though, this new 3 Skypephone is experiencing some growing pains.

So far. the best experience I have had with VoIP service reliability has been with a dedicated desktop Soundpoint 501 VoIP phone from Polycom, connected to a wireline network with a quality Internet connection.  This is a quality phone for the price, and provides a dedicated environment for your VoIP application.  I have very rarely had any hiccups with this setup.   Even better, if you are willing to shell out some extra dough, is the Polycom Soundpoint 550 HD VoIP phone, which provides a High-Def codec for awesome high quality audio when you are communicating with another SoundPoint 550.  The audio quality is like listening to a CD.

I think Skype is on the right track with their strategy to diversify their VoIP clients beyond the PC.  If you are after just plain good phone service, though, you would do better to get a Philips VoIP841 Cordless Skype Phone than a WiFi phone.  It's a DECT cordless phone rather than a WiFi phone, so it works like an analog telephone adapter that is integrated with a cordless phone.  You'll get much more "deterministic" behavior on the Philips VoIP841 DECT phone, and that will result in better phone call quality!

Why Skype Fails as a Business Service Provider

Om Malik posted yesterday that Skype, in response to service outages, should pony up and pay for its own infrastructure instead of completely relying on its customers' computers.  Skype has had a tough few months, with outages the past few days and in August calling into question the reliability of the service, at a time when Skype is trying to cross the chasm and transform itself from a casual-use web telephony tool into a primary phone line replacement service.  Om suggests that Skype should deploy some of its own peer-to-peer supernodes to improve the reliability of the service.  That may help Skype improve its service for residential customers, but I don't think it will be enough for Skype to approach PSTN levels of reliability, which would be required if Skype ever hopes to be a serious enterprise-grade service provider.

Regardless of what is causing the most recent Skype outage, Skype has a key architectural weakness that will make it very difficult for them to serve businesses, and it has nothing to do with their acclaimed peer-to-peer architecture.   Skype's problem: they don't control the network their customers use to connect to the Internet, and therefore cannot guarantee a high-quality phone call.  Anybody that has used Skype knows that sometimes a Skype call goes bad, and they just chalk it up to Skype being a free VoIP service and re-try the call.  That works for a lot of consumers, but most businesses don't want to deal with that kind of uncertainty with their phones.

We end users do all kinds of wild things with our home or office networks, and Skype has no magic wand to prevent it.  This very freedom enjoyed by Skype's customers, the freedom to mess around on their own networks, is the seed of destruction for service quality on a VoIP service.

Other business VoIP service providers pay a lot of attention to quality of service on the customer's Internet connection, and some, such as Unity Business Networks, go so far as to provision their own T1 to the customer site and provision their own router on the customer site in order to guarantee a quality experience.   Unity's customers have great things to say about Unity's service.  To provide this level of service, Unity's team had to develop a set of operational skills (and associated costs) that Skype hasn't even begun to consider, and maybe never will.

I believe that if Skype can solve its reliability problems then consumers will stick with Skype and that Skype will make progress in its quest to become "the primary phone" in some households.  However, Skype will never cross the chasm and become a reliable business service provider until they can truly control the quality of service on the customer's connection to the Internet.  That costs a lot of money, and I don't think Skype's business model will support that architecture.

FCC Regulatory Land Grab in Cable Land?

The Wall Steet Journal reported this morning that the FCC might be preparing to "impose new programming and access requirements on carriers." (See FCC, Cable Firms Gird For Possible Access Row).  Apparently, there was a clause in the 1984 Cable Act that allowed the FCC to regulate the market by reducing the rates that programmers pay for access to spare channels on the cable networks, if certain market conditions are met.  The 1984 Act stated that when cable can reach 70% of US households, and when 70% of those househods subscribe to cable, then the FCC can intervene.  Apparently, the FCC now thinks these conditions are met.

The National Cable & Telecommunications Association (NCTA), disputes this claim, of course, with their own statistics, showing that cable can reach 97.5% of US TV Households, but that cable serves only about 59.8% of these households as of June, 2007. 

Who serves the remaning TV consumers?  The biggest competitor right now is Satellite TV, with about 25% of the market, and TV over Fiber Optic cable has about 4% market share.  Paul Carton provides lots of detail of a ChangeWave consumer market survey here.  The ChangeWave survey gives cable a higher percentage of the market than the NCTA, at 64%, but it is still well under the 70% threshold in the 1984 act.  What's more, the trend seems to be against cable, with Satellite and Fiber Optic providers gaining market share and enjoying higher levels of customer satisfaction.

This is a continuation of the FCC's recent ramp-up of pressure on the cable industry.  This morning's Wall Street Journal article on the same topic provides some good background of the FCC's actions against the cable industry, in hopes of giving consumers more choice of channels, and lower rates.  These are worthy goals, to be sure, but relying on the 1984 Telecom Act's 70/70 provision seems like a stretch to me.  Personally, I'd like to see any FCC action in this area supported by additional legislation before proceeding with an agency power grab.

So, why is the FCC considering regulation?  This smells like a regulatory body trying to expand its power base.  There's something rotten on 12th Street.

Thank You, Telecosm Readers

I want to thank all of the folks who are reading this blog on a regular basis.  The Telecosm blog is less than three full weeks old, and we already hit a peak of 100 subscribers this week.  Thanks for visiting, reading, and commenting at the Telecosm!
Ike

Free, Ad-Supported Cell Service? Don't Count On It

Ever since the Google Android announcement, the press has been littered with speculation that either your mobile phone or your mobile phone service could eventually be free, paid for by advertising.  I'm here to tell you, though, that it ain't gonna happen anytime soon, and I'm not the only one saying it.  Andy Rubin, Google's Director of Mobile Platforms, says "You won’t see a completely ad-driven cellphone based on this platform for some time.”

Why?  Because the math doesn't add up. 

I'm not sure how much Google really makes per page view in advertising, on average, because I can't decipher it from their quarterly reports.  Somebody out there does have the data, no doubt, and they can comment here to set me straight if that is the case.

What I do know, though, is that AOL makes about $0.003 per page view on advertising, based on Time Warner's 3rd quarter earnings report, and that AOL has about 140 page views per unique visitor to their web sites, per month.  That means that AOL makes about $0.42 per unique visitor to their web sites per month. 

Okay, you're thinking "how do I translate that to Google?"  I'm not really sure, so let's just make a bunch of optimistic assumptions and see where it leads us.   If we just assume that Google does a lot better than AOL, and instead of 140 page views they get 1,000 page views per month on that sweet new OHA handset.  Let's go on to assume that they make double what AOL makes per page view: instead of $0.003 per view, Google gets $0.006.  That still only gets you to $6.00 per month per subscriber in ad revenue. 

Of course, Google is already talking about sharing that ad revenue with handset manufacturers and/or service providers.  Plus, Google will want to keep some of that ad revenue for themselves, won't they?  How much will be left over to reduce the consumer's monthly service charges, or to pay for the handset? Not much. 

But let's play along and assume half of the ad revenue can be used to help pay for the handset.  That's $3 per month, and let's assume the handset lasts 2 years, so that's $60 for subsidizing the cost of the handset.  And that's with a lot of optimistic assumptions.

Yes, you can find cellphones now that cost less than $60, but they aren't cell phones that would run the Android code and provide a big screen for displaying those ads.  And, if you are getting your cell phone for free, who is going to force you to keep it for two years and view all of those ads with it, to make sure the ad revenue is really collected and shared with the handset manufacturer?

We've been looking at the case where we are just trying to subsidize the handset itself.  The math gets even worse when you are trying to use ad revenue to subsidize the monthly cellphone service charges.

No wonder Mr. Rubin is downplaying the possibility of completely ad-driven cellphones.  I'm guessing it isn't because of some big technical barrier.  No, I think it is just because the math doesn't work yet.

Someday, when electronics get cheap enough and inflation drives the price of advertising high enough, the lines will cross and the math of a completely ad-driven cell phone will work.  I'm guessing it is going to take a while.

Dropping Out Of the Rat Race Has Benefits

There is a good article in this morning's Wall Street Journal by Sue Shellenbarger titled "Men on the Daddy Track Find A Place of Their Own at Home."  I was interested in the article because four years ago I made a decision to drop out of full-time corporate work in order to place higher priority on other parts of my life, and I can say without a doubt it is one of the best decisions I have ever made.

At the time, my two sons were 8 and 11 years old.  Like a lot of families, my wife had been the primary care-giver for the boys, but now she was itching to get back into full-time work.  I was seeing my boys grow up and recognized that I had spent very little time with them, and wanted to try to fix that before it was too late and they were fully grown.  Also, I was looking for more meaning in my work, and was feeling the tug to do non-profit work.  However, the status quo would never satisfy these needs, as I was often working overtime as an exec at Level 3 Communications. 

Over a period of several months, my wife and I made the decision to send her back to work as a full-time teacher while I dropped out and became a part-time consultant, part-time non-profit executive director, and full-time Dad.  It wasn't an easy choice: intentionally cutt