What do you get when you mix engineers from some of the largest cloud computing companies including Google, Yahoo, Facebook and Oracle with a commercial Hadoop distribution? You get a free commercial cloud computing operating system that anyone can download freely.

Cloudera CEO Michael Olson has already built and sold an open source company to Oracle that made Berkeley DB. Now Olson has built a team from some of the largest online companies to launch Cloudera.

Joining Olson are Dr. Amr Awadallah of Yahoo, Christophe Bisciglia of Google, Jeff Hammerbacher formerly of Facebook and others. The team has a background in large-scale data analysis and built Cloudera with this purpose in mind.

Hadoop was created by Doug Cutting, now of Yahoo, and based on Google’s papers on MapReduce and distributed file system. The base Hadoop distribution is available from the Apache Foundation and includes an implementation of MapReduce and the Hadoop Distributed File System.

Cloudera takes working with large distributed Web applications one step further by implementing the Hive client library that allows data to be accessed using HQL. This Hive Query Language allows developers to access large data sets using SQL-like syntax.

Cloudera aims to make installing and managing Hadoop easier for the average technology company. “Adoption of Hadoop has been slow in mainstream computing because it is still hard to install, build, and maintain a Hadoop cluster,” says Olson.

Entrepreneurs that want to bootstrap their applications can also use Amazon’s EC2 to scale their Hadoop operations. Cloudera has created an EC2 image in order to make the process easier.

Olson and his team have their work cut out for them in making an open source software model profitable. Cloudera is challenging Red Hat at the high end of the Linux server market which is no small feat.

However, by focusing on companies that need to manage large sets of distributed data, Cloudera could become the Linux distribution of choice. That could lead to a sizable professional services team at Cloudera.

Related:

Entry was posted on Monday, March 16th, 2009 at 10:13 am under Open source.

Read more entries by Dave Jeyes.
Stumble It!