How Linux Makes Companies Smarter
Two perspectives exist regarding Linux's success in the enterprise: one from the inside and one from the outside. Insiders credit the organization's intelligence, resourcefulness and expertise. Outsiders credit vendors with the same qualities. Credit is due to both, of course, and to some degree each side gives some credit to the other. But the press generally takes the outside view, giving the lion's share of credit to vendors. As a result, little credit goes to the qualities that really account for successful use of Linux in the enterprise.
No two inside stories are the same. Every vendor-side story basically is about how products solve problems—that's why vendors love to call their products “solutions”. But Linux isn't about “solutions”. That's a vendor word. Linux is about resourcefulness and intelligence. Those are customer words. Linux succeeds in the enterprise by helping smart companies make themselves smarter. With Linux and open source, the primary supply and demand sides of the marketplace both reside inside the company.
This coexistence is a hard development for many of us to understand. It's hard for vendors because they're used to a marketplace where the supply side is in control and wide profit margins are the ideal, even if they are no longer the norm. It's hard for the press to understand, because so many of the news stories they write are vendor-vs.-vendor sportscasts, in which the marketplace is a playing field and customers are prizes. It's hard for corporate leadership to understand, because they're so used to playing the prize role in vendor playoffs.
The people who don't find it hard to understand are the people who use Linux to make their companies more efficient, more reliable and more effective. Take the case of LSI Logic, the semiconductor maker. Roland Smith is the director of Global Operations there, running one of four groups that report to the company's CIO. The company's big switch to Linux happened a while back when Smith's team recommended replacing HP-UX with Linux, increasing performance five- or sixfold and cutting costs by two-thirds. Now, Smith says, “Whenever somebody comes in and wants to talk to the CIO about a new application, or a project or anything new, the CIO's question is, 'Does it run on Linux?”'
This environment has some interesting effects on the flow of help. Smith provides an excellent case of help flowing from customer to vendor:
In our SAP environment, we recently decided we wanted to begin inserting Linux application servers. So we went out and talked to both Dell and HP. We told them what we wanted from each of them was an eval server—one they think is best suited to run Linux. We wanted four processors and a gig of memory.
A couple of interesting things happened. One was HP fumbled around and took two weeks to come back and say, “What do you really need?” The other was that Dell took three days and shipped us a server. They said, “Here. You've got it for three months. Go tell us what you can do with it.” That was kind of wonderful.
So, then we went to SAP and said, “What do you recommend?” And SAP said, “Well, we think SuSE 8 runs okay. And we think that Red Hat 7.1 runs okay. But we don't really know.” Then we went and talked to Red Hat and they said, “We think it'll run nicely on our Advanced Server 2.1, but we don't really know. So how about you go try it out and tell us?”
So, we built it. We bought the Red Hat application server, built it, loaded SAP on it and put it in our test environment. And it's running incredibly well. We're very happy with it. But it's interesting to me that, of all the vendors out there, only Dell said, “We'll happily give you hardware to try it out on, but we don't know anything more than that.”
Where LSI is concerned, most of the intelligence and leadership is located on the customer side.
Another smart customer is Orbitz, the on-line travel company owned by five major airlines. Leon Chism, chief internet architect for Orbitz, says this about how the company applies its own intelligence internally:
Our developers are very good at leveraging open-source tools to improve our design, build and software management process. On their own they have built a number of tools that parse logs, store them in a PostgreSQL database and do ex post facto analysis to find errors and opportunities for improvement. The first I heard of one tool was after it had been used to do analysis and improve some sections of the code successfully several times. We also have a number of developers and engineers using tools like Perl, Python and Jython (we employ an official Jython developer) for their own ad hoc tools to help them monitor site behavior. All of this is in addition to the official production management infrastructure. Generally, after a tool has been developed, tested and proven useful, we migrate it into the rest of the management infrastructure our Network Operations Center uses.
He also sees the use of Linux and open source as competitive advantages for Orbitz:
Orbitz is a company that leverages open-source projects in its production environment and isn't afraid to be public about it. We currently use over 750 Linux machines in our production environment. Our web servers are Apache running on Linux, our application servers are proprietary servlet engines running on Linux and the back-end “booking engine” is comprised entirely of Java services running on Linux. Lastly, the software that does the low fare searching that is one of the key differentiators between Orbitz and our rivals also is running on Linux.
Companies have found ways to make money by providing services to open-source-software users or by packaging this free software with products they sell. With the profit motive driving its promotion, free software is cropping up all over corporate and government computers.
Why is it most important that “companies have found ways to make money”? Is that more important than the actual work that gets done?
Why does he say “companies” when he means “vendors”? Aren't customers companies too?
And let's face it: Linux users are tough customers. Here's Leon Chism again:
On the relationship front, there is a vendor that we have and are still considering replacing with an open-source solution. For the most part, the application works, but when we find bugs in it the resolution process leaves a lot to be desired. Escalation procedures, arguments about root cause, long discussions about “supported configurations” all lead to muddling the process rather than focusing on the solution.
When we have issues or questions with Linux or Apache, these things aren't at issue. We simply use the customer service application to end all customer service applications—www.google.com—and start researching. Or we start perusing source code. Orbitz is a customer that typically pushes the limits of the products we use. Some vendors are up for the challenge and are willing to make efforts to support that need. Some aren't. And we don't know who is who until, as they say, push comes to shove. Using open-source products removes that issue. We have an engineering and software development staff that is fully capable of paddling its own canoe, and the support we get from authors, list servs and Google is more than enough in most cases.
Yet the tough vendor story gets played while the tough customer story does not.
Dell may be the top-selling PC brand, but the real best-sellers are no-name “white” boxes that are bought by the ton or built out of industry-standard parts on an as-needed basis. Customers love them, because they're cheap to buy and easy to replace. But we don't hear much about them. One reason we don't is their builders don't spend much money on advertising and PR. But a bigger reason is the computer industry has a long-standing prejudice against the word “commodity” and has a huge fear of “commodification”.
Commodification has long been a fact of life in the hardware business, but software still is widely considered a big margin category. Microsoft has a bad quarter when gross profit margins slip below 80%, and Oracle's worst quarter in the last five (at the time of this writing) was still over 75%.
But the software industry will have to face the fact that customers love commodities. One good example is the University Corporation for Atmospheric Research (UCAR, www.ucar.edu/ucar/index.html) in Boulder, Colorado. In spite of its name, UCAR is a business with substantial customers that include the national air traffic control system. Greg Thompson, a scientist with UCAR, explains the decision-making process there:
Everybody here has a shelf full of O'Reilly books. We like to figure things out for ourselves. For example, I'm a scientist, not a programmer. But a while back I saw a need to get past manipulating data with Perl and text files. So I started reading up on databases and learning MySQL. I wouldn't have bothered if MySQL hadn't been free. We don't have five-digit figures around here to run Oracle. We don't need hard-core database stuff. We don't need transaction support. But we do need results that are Web-accessible, so MySQL made sense. Building internal expertise around something like MySQL is straightforward and easy. I do a little research on my own, see if something I need is already installed—and if it's not, I send a help-desk e-mail to a sysadmin. Then, the next day I download a Debian package, run apt-get and I'm in business. Without spending a dime.
It's amazing how small a software approval bureaucracy can get. When friction is low and costs are zero, free enterprise lives up to its name. “Our IT budget is zero”, says Elliot Noss, president and CEO of Tucows (tucows.com) in Toronto. Tucows' business is hosting one of the largest software download sites in the world, plus running one of the three top-level domain name registries (OpenSRS, resellers.tucows.com/opensrs), which manages around 3.5 million domain names, all on Linux, Apache, MySQL and PostgreSQL. “I can't even imagine what it would cost to work with a big Sun system or to run a big Oracle database”, Noss says.
Although network and technology companies are more willing to talk about using Linux and open source, other types of companies are quietly changing the way they handle software, too. Take this e-mail, for example, whose author requested anonymity:
I work for a Fortune 50 company whose IT has a new open-source policy, as well as a fully approved open-source toolkit (with an internal “brand name”) available for employee download from an internal web site. Already up to version 3.0, it comes in both UNIX and Windows flavors. It consists of GNU tools and others on the UNIX side and the Cygwin toolkit on the Windows side. Linux (Red Hat for now) is one of the “tools”.
The guy who heads up the toolkit project is a take-no-prisoners, free-software advocate, too. The stated purpose of the policy is to permit the company to save money by eliminating license fees and by streamlining the software acquisition process, but the clear subtext is to make it possible for the company to protect its intellectual property from third-party (that is, Microsoft) manipulation.
Part of the new policy is something of “don't ask, don't tell”. IT agrees not to complain if anyone decides to install these tools on whatever computer they use. All any nonmanagement employee now needs to do is get approval from his or her local manager to download and install the tools. Managers do not need anyone's permission to install the tools for themselves. Per the policy, IT does not get involved in any way. So spread of the tools—of Linux and the rest of the free software suite—happens naturally and organically.
Before the policy was implemented, even GNU Emacs required an official IT review/approval process. Even if your boss wanted it, IT could fight it if they wanted to. But with this new policy, that's now history.
Policy inevitably adapts to new facts of development life inside an organization. Today the trends among those facts favor Linux, big time. In a recent survey of Linux developers (who also work with other platforms), Evans Data Corp. observed a rapid shift away from proprietary UNIXes and Windows, in the direction of Linux:
Linux is the primary choice of host platform at 40%. Windows 2000 makes a strong showing at 29%, with Windows XP right behind at 12%. The landscape is about to change, however....Next year respondents plan to increase their use of Linux as the primary development platform by 15%, from 40% to 55%.
The new imperative for vendors is to give customers what they want and not bother trying to lock them into no-choice relationships. They've had enough of that, thank you. Mårten Mickos, CEO of MySQL (mysql.com), believes vendors like his can help raise IT consciousness. He says, MySQL successfully sells free software (MySQL is GPLed code) because “there is more value to code you can see than to code you can't see”, adding, “We are part of a huge community of customers and other developers who are all passionate about improving the code base. Every day we are proving it is possible to have a commercial relationship that benefits free software.”
That commercial relationship still is not one that happens between the tops of the vendor and customer bureaucracies. Among big customers it happens down among the middle tiers on both sides. For MySQL, that list of customers is impressive, and it includes Nokia, Yahoo, NASA, Silicon Graphics and Cisco. Jeremy Zawody, a self-described “technical yahoo” with Yahoo Finance, says, “MySQL will penetrate the enterprise similarly to the way (Microsoft) SQL Server did, but with much greater speed. MySQL is to Oracle as Linux is to Windows. It will slowly but steadily creep up the food chain, just like Linux has.” But when I asked Mårten Mickos if MySQL is competing with Oracle yet, he said no. “We complement Oracle far more than we compete with it.”
Still, we're at a point in history where the action is clearly shifting up the stack, from operating systems and applications to data. “We think it's the information age, not the operating system age”, Larry Ellison says. “The OS manages hardware; we manage the software.”
Because more and more of that software runs on Linux, Oracle has wisely chucked its long-standing OS agnosticism and repositioned itself, alongside IBM, as one of the world's leading “Linux companies”. Wim Coekaerts (otn.oracle.com/oramag/Coekaerts.html), head of Oracle's Linux kernel team, says “Linux is really, really important to Oracle. We are very much a Linux company.” He proudly credits the work his kernel development team has contributed to helping to make Linux “enterprise class”. There is proof in the customer pudding, too. Roland Smith says:
I would say that Oracle is probably the best vendor in the marketplace, in respect to Linux. When we told our Oracle account team that we wanted to put up an Oracle database on Linux, they were all over it. They really knew what they were doing. Within a week they had us in touch with their development folks back in Redwood Shores. They had good documentation. It was easy to put up. They were patient. It was easy to run, easy to connect to. We're very happy about it.
Smart companies naturally want smart relationships. Oracle seems to be doing a good job of meeting that market demand. It should help them continue to adapt to the successes of MySQL and PostgreSQL.
Like Oracle, other vendors will need to adapt to a world where Linux and its open-source companions serve as fundamental infrastructure for IT. That infrastructure is quickly becoming as standard as two-by-fours, ten-penny nails and sheetrock screws. Vendors always will be welcome to take advantage of that infrastructure and to contribute to its improvement; but their frame of reference will shift from the abstract to the concrete—from abstract playing fields to concrete IT projects where they have something useful to contribute. When that happens, and the software industry finishes growing up, credit will finally go where it's long overdue: to the smart people who used Linux to make their companies smarter, no matter what those companies bought and sold.
