Sign up to receive 5 free days of our daily summary for tech news.

Q&A
Venture Capital

Lighting a Spark Beneath Big Data

The so-called Big Data industry is a jumble of open-source software projects with such odd mascots that it resembles a box of animal crackers. There’s Hadoop (an elephant), Pig (see Porky, The) and Hive (an elephant’s head and a hornet’s body). The latest flavor is called Spark, whose logo is (mercifully) just a spark.

Spark has become wildly popular with developers in a short time, gathering more than 400 contributors last year, according to Open Hub, which tracks activity around open-source projects. It promises to compute data up to 100 times faster than the standard methods used for bits sitting inside Hadoop, the software suite sold by Cloudera and Hortonworks.

Adatao, a startup with $13 million in funding from Lightspeed Venture Partners and Andreessen Horowitz, is one of the early firms building an app that uses Spark to analyze data stored in Hadoop file systems.

Adatao CEO Christopher Nguyen argues that Hadoop forms a foundational “storage layer” for data that companies could one day unlock using Spark’s computing speed. Mr. Nguyen talked with The Information about Spark’s promise and the direction the industry is headed. Edited excerpts follow.

What Is The Information?

Notes bb3aa5069205d702dda37fc71dc6f1c59df2310c4379304e9f1199f052af4884

Exclusive Articles

We broke it first. Receive original reporting you won't read anywhere else from the largest newsroom in tech.

Notebook b5103017ee163370a1667c9ce59ab0ef023875a17888105d26c185cadf31bb69

Daily news analysis

Every weeknight, we'll send you our reporters’ views on the day’s top tech news—distilled into one email.

Conference calls c9e664e8b96ee347d3e92b8309938f268422b9db98cfabca5c826fbb75054b23

Conference calls

Get access to our reporters and other top executives with monthly deep-dive calls into topics like startups and autonomous vehicles.

Events 499acedd16cffc41445edd76bfd302b2836c2a27419890f17130a6b10e2aa3df

Special Events

For no extra fee, get access to more than a dozen events yearly, from intimate dinners to larger gatherings with marquee speakers.

What else is included in my subscription?

Become a contributor

Share your views and find other subscribers by completing your profile. You’ll be listed in our contributor directory.

Slack community

Discuss topics and current events with our subscriber-only Slack group and share news about your company with other subscribers.

Annual Subscriber only benefits:

Org Charts

Access the only collection of tech company org charts. Our expanding database includes companies like Amazon, Snap, and Uber.

Crypto Newsletter

Our experts investigate the latest in the crypto world to keep you in-the-know about the movements of digital currencies.

Stay up to date on Silicon Valley

Sign up for Jessica Lessin’s (The Information’s CEO & Founder) free Saturday newsletter and also receive a complimentary week of our daily afternoon tech commentary email.

Already a subscriber? Log in here

Recent Articles

Venture Capital Markets

Pinterest’s IPO Winners

By Zoë Bernard and Martin Peers

Venture Capital Startups

Robinhood Nears Funding at Valuation Over $7 Billion

By Zoë Bernard

Venture Capital Enterprise Markets

Question for Slack: When Will Investors Cash In on Gains?

By Alfred Lee