Sign up to receive 5 free days of our daily summary for tech news.

Announcing The Information Courses: Sign up for our Micromobility 101 course, a 15-day guided primer on dockless e-scooters, bikes, and more.

Read this article for free.

Already a subscriber? Log in here

Venture Capital

Lighting a Spark Beneath Big Data

The so-called Big Data industry is a jumble of open-source software projects with such odd mascots that it resembles a box of animal crackers. There’s Hadoop (an elephant), Pig (see Porky, The) and Hive (an elephant’s head and a hornet’s body). The latest flavor is called Spark, whose logo is (mercifully) just a spark.

Spark has become wildly popular with developers in a short time, gathering more than 400 contributors last year, according to Open Hub, which tracks activity around open-source projects. It promises to compute data up to 100 times faster than the standard methods used for bits sitting inside Hadoop, the software suite sold by Cloudera and Hortonworks.

Adatao, a startup with $13 million in funding from Lightspeed Venture Partners and Andreessen Horowitz, is one of the early firms building an app that uses Spark to analyze data stored in Hadoop file systems.

Adatao CEO Christopher Nguyen argues that Hadoop forms a foundational “storage layer” for data that companies could one day unlock using Spark’s computing speed. Mr. Nguyen talked with The Information about Spark’s promise and the direction the industry is headed. Edited excerpts follow.

What Is The Information?

Notes bb3aa5069205d702dda37fc71dc6f1c59df2310c4379304e9f1199f052af4884

Exclusive Articles

We broke it first. Receive original reporting you won't read anywhere else from the largest newsroom in tech.

Notebook b5103017ee163370a1667c9ce59ab0ef023875a17888105d26c185cadf31bb69

Daily news analysis

Every weeknight, we'll send you our reporters’ views on the day’s top tech news—distilled into one email.

Conference calls c9e664e8b96ee347d3e92b8309938f268422b9db98cfabca5c826fbb75054b23

Conference calls

Get access to our reporters and other top executives with monthly deep-dive calls into topics like startups and autonomous vehicles.

Events 499acedd16cffc41445edd76bfd302b2836c2a27419890f17130a6b10e2aa3df

Special Events

For no extra fee, get access to more than a dozen events yearly, from intimate dinners to larger gatherings with marquee speakers.

Access the best reporting on the tech industry read by tens of thousands of global executives.

Become a contributor

Share your views and find other subscribers by completing your profile. You’ll be listed in our contributor directory.

Org Charts

Access the only collection of tech company org charts. Our expanding database includes companies like Amazon, Snap, and Uber.

Slack community

Discuss topics and current events with our subscriber-only Slack group and share news about your company with other subscribers.

Share with your team

Subscribers can unlock any article and share it with friends and co-workers through a special share link.

Stay up to date on Silicon Valley

Sign up for Jessica Lessin’s (The Information’s CEO & Founder) free Saturday newsletter and also receive a complimentary week of our daily afternoon tech commentary email.

Already a subscriber? Log in here

Recent Articles

Crypto Venture Capital Startups

Andreessen Digs Into Crypto

By Sarah Kuranda

Crypto Venture Capital

The People With Power at Coinbase

By Jon Victor


EOS Block Producers Under Pressure

By Jon Victor

Crypto Venture Capital

After the Crypto Boom, Investors Proceed With Caution

By Jon Victor