Firing on All Cylinders: The 2017 Big Data Landscape


It feels good to be a data geek in 2017.

Last year, we asked “Is Big Data Still a Thing?”, observing that since Big Data is largely “plumbing”, it has been subject to enterprise adoption cycles that are much slower than the hype cycle.  As a result, it took several years for Big Data to evolve from cool new technologies to core enterprise systems actually deployed in production.

In 2017, we’re now well into this deployment phase.  The term “Big Data” continues to gradually fade away, but the Big Data space itself is booming.  We’re seeing everywhere anecdotal evidence pointing to more mature products, more substantial adoption in Fortune 1000 companies, and rapid revenue growth for many startups.

Meanwhile, the froth has indisputably moved to the machine learning and artificial intelligence side of the ecosystem. AI experienced in the last few months a “Big Bang” in collective consciousness not entirely dissimilar to the excitement around Big Data a few years ago, except with even more velocity.

2017 is also shaping up to be an exciting year from another perspective: long-awaited IPOs.  The first few months of this year have seen a burst of activity for Big Data startups on that front, with warm reception from the public markets.

All in all, in 2017 the data ecosystem is firing on all cylinders.  As every year, we’ll use the annual revision of our Big Data Landscape to do a long-form, “State of the Union” roundup of the key trends we’re seeing in the industry.

Let’s dig in.

Continue reading “Firing on All Cylinders: The 2017 Big Data Landscape”

The New Gold Rush? Wall Street Wants your Data



A few months ago, Foursquare achieved an impressive feat by predicting, ahead of official company results, that Chipotle’s Q1 2016 sales would be down nearly 30%. Because it captures geo-location data from both check-ins and visits through its apps, Foursquare was able to extrapolate foot-traffic stats that turned out to be very accurate predictors of financial performance.
That a social media company could be building a data asset of immense value to Wall Street is part of an accelerating trend known as “alternative data”. As just about everything in our lives is getting sensed and captured by technology, financial services firms have been turning their attention to startups, with the hope of mining their data to extract the type of gold nuggets that will enable them to beat the market.
Could working with Wall Street be a business model for you?
The opportunity is open to a wide range of startups.  Many tech companies these days generate an interesting “data exhaust” as a by-product of their core activity.  If your company offers a payment solution, you may have interesting data on what people buy. A mobile app may accumulate geo-location data on where people shop or how often they go to the movies.  A connected health device may know who gets sick when and where.  A commerce company may have data on trends and consumer preferences. A SaaS provider may know what corporations purchase, or how many employees they hire, in which region. And so on and so forth.
At the same time, this is a tricky topic, with a lot of misunderstandings. The hedge fund world is very different from the startup world, and a lot gets lost in translation.  Rumors about hedge funds paying “millions” for data sets abound, which has created a distorted perception of the size of the financial opportunity.  A fair number of startups I speak with do incorporate idea of selling data to Wall Street into their business plan and VC pitches, but how that would work exactly remains generally very fuzzy.
If you’re one of the many startups sitting on a growing data asset and trying to figure out whether you can make money selling it to Wall Street, this post is for you: a deep dive to provide context, clarify concepts and offer some practical tips.

Continue reading “The New Gold Rush? Wall Street Wants your Data”

HyperScience and the Enterprise AI Opportunity


Today our portfolio company HyperScience is coming out of stealth and talking a bit more about what they’ve been working on for the last couple of years. We have been involved for a little while already as lead Series A investors, and we are excited to now be joined today by our friends at Felicis, a great addition to a strong syndicate from both coasts that also includes Shana Fisher (Third Kind) who led the seed, AME Cloud Ventures, Slow Ventures, Acequia, Box Group and Scott Belsky.  The company is announcing today a total of $18M in Series A investment.

HyperScience offers AI solutions targeting Global 2000 corporations and government institutions. Their products enable those customers to automate or accelerate a lot of dusty back office processes, particularly those that involve the manipulation and triage of large amounts of documents and images.

Continue reading “HyperScience and the Enterprise AI Opportunity”

Dataiku or the Early Maturation of Big Data

In the early days of Big Data (call it 2009 to 2014), a lot had to do with experimentation and discovery.  Early enterprise adopters would play around with Hadoop, the then-new open source framework with a funny name, trying to figure out where the technology fit in the broader landscape of databases and data warehouses.  People would also try to figure out what a “data scientist” was – a statistician who can code? An engineer who knows some math?  It was a time of hype, immature products and trial and error.

Continue reading “Dataiku or the Early Maturation of Big Data”

Investing in Frontier Tech


Over the last few months, the usual debate around unicorns and bubbles seems to have been put on hold a bit, as fears of a major crash have thankfully not materialized, at least for now.

Instead another discussion has emerged, one that’s actually probably more fundamental. What’s next in tech? Which areas will produce the Googles and Facebooks of the next decade?

What’s prompting the discussion is a general feeling that we’re on the tail end of the most recent big wave of innovation, one that was propelled by social, mobile and cloud.  A lot of great companies emerged from that wave, and the concern is whether there’s room for a lot more “category-defining” startups to appear.  Does the world need another Snapchat? (see Josh Elman’s great thoughts here).  Or another marketplace, on-demand company, food startup, peer to peer lending platform? Isn’t there a SaaS company in just about every segment now? And so on and so forth.

One alternative seems to be “frontier tech”: a seemingly heterogeneous group that includes artificial intelligence, the Internet of Things, augmented reality, virtual reality, drones, robotics, autonomous vehicles, space, genomics, neuroscience, and perhaps the blockchain, depending on who you ask.

Continue reading “Investing in Frontier Tech”

Phosphorus and the Rise of the New Genomics Startup


As we are perhaps reaching the end of a cycle of innovation in tech – the one that resulted from the simultaneous emergence of social, mobile and cloud – and collectively pondering what’s next, one of the areas I’ve found particularly exciting recently is the intersection of Big Data and life sciences.

A little over two years ago, in connection with my investment in Recombine, a genomics startup, I wrote (here) about another powerful combination of trends: the sharp drop in the cost of sequencing the human genome, the maturation of Big Data technologies, and the increasing commoditization of wet lab work.

The fundamental premise was, and still very much is, as follows:

Continue reading “Phosphorus and the Rise of the New Genomics Startup”

Is Big Data Still a Thing? (The 2016 Big Data Landscape)


In a tech startup industry that loves its shiny new objects, the term “Big Data” is in the unenviable position of sounding increasingly “3 years ago”.   While Hadoop was created in 2006, interest in the concept of “Big Data” reached fever pitch sometime between 2011 and 2014.  This was the period when, at least in the press and on industry panels, Big Data was the new “black”, “gold” or “oil”.  However, at least in my conversations with people in the industry, there’s an increasing sense of having reached some kind of plateau.  2015 was probably the year when the cool kids in the data world (to the extent there is such a thing) moved on to obsessing over AI and its many related concepts and flavors: machine intelligence, deep learning, etc.

Beyond semantics and the inevitable hype cycle, our fourth annual “Big Data Landscape” (scroll down) is a great opportunity to take a step back, reflect on what’s happened over the last year or so and ponder the future of this industry.

In 2016, is Big Data still a “thing”? Let’s dig in.

Continue reading “Is Big Data Still a Thing? (The 2016 Big Data Landscape)”

The Power of Data Network Effects

In the furiously competitive world of tech startups, where good entrepreneurs tend to think of comparable ideas around the same time and “hot spaces” get crowded quickly with well-funded hopefuls, competitive moats matter more than ever.  Ideally, as your startup scales, you want to not only be able to defend yourself against competitors, but actually find it increasingly easier to break away from them, making your business more and more unassailable and leading to a “winner take all” dynamic.  This sounds simple enough, but in reality many growing startups, including some well-known ones, experience exactly the reverse (higher customer acquisition costs resulting from increased competition, core technology that gets replicated and improved upon by competitors that started later and learned from your early mistakes, etc.).

While there are various types of competitive moats, such as a powerful brand (Apple) or economies of scale (Oracle), network effects are particularly effective at creating this winner takes all dynamic, and have been associated with some of the biggest success stories in the history of the Internet industry.

Network effects come in different flavors, and today I want to talk about a specific type that has been very much at the core of my personal investment thesis as a VC, resulting from my profound interest in the world of data and machine learning: data network effects.

Continue reading “The Power of Data Network Effects”

The Astounding Resurrection of AI [Slides]

A few days ago, I was invited to speak at a Yale Entrepreneurship Breakfast about about one of my favorite areas of interest, Artificial Intelligence.  Here are the slides from the talk — a primer on how AI rose from of the ashes to become a fascinating category for startup founders and venture capitalists.  Very much a companion to my earliest post about our investment in   Many thanks to my colleague Jim Hao, who worked with me on this presentation. and the emergence of the AI-powered application

AI is experiencing an astounding resurrection.  After so many broken promises, the term “artificial intelligence” had become almost a dirty word in technology circles.  The field is now rising from the ashes.  Researchers who had been toiling away in semi-obscurity over the last few decades have suddenly become superstars and have been aggressively recruited by the largest Internet companies:  Yann LeCun (see his recent talk at our Data Driven NYC event here) by Facebook; Geoff Hinton by Google; Andrew Ng by Baidu.  Google spent over $400 million to acquire DeepMind, a 2 year old secretive UK AI startup. The press and social media are awash with thoughts on AI.  Elon Musk cautions us against its perils.
What’s different this time? As Irving Wladawsky-Berger pointed out in a Wall Street Journal article, “a different AI paradigm emerged. Instead of trying to program computers to act intelligently–an approach that hadn’t worked because we don’t really know what intelligence is– AI now embraced a statistical, brute force approach based on analyzing vast amounts of information with powerful computers and sophisticated algorithms.”  In other words, the resurgence of AI is partly a child of Big Data, as better algorithms (in particular, what’s known as “deep learning”, pioneered by LeCun and others) have been enabled by larger than ever datasets and the ability to process those datasets at scale at reasonable cost.

Continue reading “ and the emergence of the AI-powered application”

The State Of Big Data in 2014: a Chart

Note: This post appeared on VentureBeat, here.

It’s been almost two years since I took a first stab at charting the booming Big Data ecosystem, and it’s been a period of incredible activity in the space. An updated chart was long overdue, and here it is:

(click on the arrows at the bottom right of the screen to expand)

A few thoughts on this revised chart, and the Big Data market in general, largely from a VC perspective:

Continue reading “The State Of Big Data in 2014: a Chart”

Can the Bloomberg Terminal be “Toppled”?

In the eye of some entrepreneurs and venture capitalists, the Bloomberg terminal is a bit of an anomaly, perhaps even an anachronism.  In the era of free information on the Internet and open source Big Data tools, here’s a business that makes billions every year charging its users to access data that it generally obtains from third parties, as well as the tools to analyze it.  You’ll hear the occasional jab at its interface as reminiscent of the 1980s.  And at a time of accelerating “unbundling” across many industries, including financial services, the Bloomberg terminal is the ultimate “bundling” play: one product, one price, which means that that the average user uses only a small percentage of the terminal’s 30,000+ functions.  Yet, 320,000 people around the world pay about $20,000 a year to use it.

Continue reading “Can the Bloomberg Terminal be “Toppled”?”


The field of bioinformatics is having its “big bang” moment.   Of course, bioinformatics is not a new discipline and it has seen various waves of innovations since the 1970s and 1980s, with its fair share of both exciting moments and disappointments (particularly in terms of linking DNA analysis to clinical outcomes).  But there is something special happening to the industry right now, accelerated by several factors:

Continue reading “Recombine”

Thomson Reuters CTO Series (Podcast)

Thomson Reuters CTO James Powell runs a great series of podcasts where he interviews people in the technology world about topics of relevance to his organization.  I was fortunate to be invited to speak with James about the Internet of Things and Big Data, and it was a lot of fun.   Below is the podcast, uploaded on SoundCloud.  Thanks to James Powell and Dan Cost for the opportunity.

Launching New Sites for Data Driven NYC and Hardwired NYC

Some updates on the event/community front:

1) A little while ago, I changed the name of the data event I’ve been organizing from “NYC Data Business Meetup” to “Data Driven NYC”.   I originally started the event mostly as experiment, and didn’t give much thought to branding (so yeah, that was a terrible name).  The event has now grown quite a bit (over 3.700 members as I write this), so it was time for a better name; also at this stage, it feels more like a community than “just” a meetup, so I wanted a name that reflected this reality.

2) Back in June, I launched a new community called “Hardwired NYC”.  It covers startups, technologies and products at the intersection of the physical and digital worlds, including topics like 3D printing, Internet of Things, wearable computing, etc.  I developed a strong interest in those areas through my involvement in the Big Data world – the Internet of Things, in particular, is deeply intertwined with Big Data (the proliferation of sensors has been contributing to the Big Data “problem”; equally  the Internet of Things will be highly dependent on Big Data technologies if it is to deliver on its promise).

3) As Hardwired NYC is taking off fast (more than 700 members after just two events), I figured that both events/communities should have their own website with full video libraries, including for people who don’t live in New York and are interested in the content. So, with the great help of my FirstMark colleague Dan Kozikowski,  I’m launching this week and  Both sites have a “Watch” section where, from now on, I will post pictures and videos of events (as opposed to this blog).

Data Driven NYC

Hardwired screenshot