How many people really are watching 'The Mandalorian' or 'Money Heist'?

For those obsessed with who is winning the video streaming wars, one metric matters: subscriber growth.

But Netflix and now Walt Disney – with its November launch of Disney+ – typically release that figure quarterly, leaving outsiders to guess at subscriber growth in any way they can.

A cottage industry of companies has sprung up to fill that vacuum. Firms like Apptopia, Sensor Tower and App Annie, born years ago to track how many people download mobile apps, are now playing a bigger role in the streaming war that kicks into gear this year as AT&T Inc’s WarnerMedia and Comcast Corp-owned NBCUniversal launch new services.

These firms sell mobile download data they arrive at by applying algorithmic magic to publicly available data and data from other apps. The process is propriety, they say, and opaque to outsiders.

The resulting figures – which are approximations of mobile downloads, not the new subscribers the companies disclose – do not correlate exactly with subscriber growth, but are influential.

Third-party data is widely reported in the press, including in Reuters stories. Bloomberg offers Apptopia’s mobile data to its clients. The data is also cited in research from Wall Street firms including Credit Suisse, Bank of America and Wells Fargo – sometimes as a worthwhile indication of performance, and other times dismissively.

The data moves markets: On Nov. 26, shortly after Apptopia released data indicating that Disney+ was averaging nearly a million new subscribers a day – a report that was covered widely in the press – Disney shares rose 2.3% to $153.43, setting a new record high.

To survey how often these firms get it right, Reuters reviewed eight quarters of data from Netflix, and the same amount of data from two of the third-party app measurement firms.

It found that Sensor Tower’s past eight quarters’ of Netflix mobile download data has directionally if not precisely mirrored Netflix global paid membership growth. Apptopia download data mirrored it directionally in all but two quarters.

Even so, the data is controversial: critics say these firms do a poor job of tracking how many people drop a streaming service, and as such, should not be viewed as a proxy for growth.

“If we had based our conclusions on app download data, we’d be very incorrect about what Netflix is doing and everything in any given quarter,” said MoffettNathanson analyst Michael Nathanson, who said his firm had used Apptopia and Sensor Tower, but no longer does so.

Netflix did not respond to requests for comment. Disney and App Annie declined to comment.

Executives from Sensor Tower and Apptopia emphasize that the data reflects trends, not precise growth.

“The reason people like and trust the mobile data is that mobile gets the most screen time — it’s indicative of how people are living their lives,” says Adam Blacker, a vice president at Apptopia. “What we’re doing is nailing the trends and the percentage swings.”

Recent quarters of Netflix mobile download data from Apptopia and Sensor Tower, while directionally mostly correct, have been off in notable ways. Apptopia recorded negative download growth for Netflix in the second and fourth quarters of 2019 — compared to the 22% and 20% global paid membership growth the company reported, respectively. In the third quarter of 2019, Apptopia reported single-digit growth compared with an increase of 21% reported by Netflix.

“We’re not going to be right 100% of the time,” says Blacker about those quarters. “We’re not going to tell you to trade on download data.”

Sensor Tower reported single-digit global mobile app install growth for Netflix in the second and fourth quarters of 2019, compared with growth of 22% and 20%, respectively, reported by Netflix.

“We’re only looking at mobile,” said Randy Nelson, head of mobile insights at Sensor Tower. “We only capture that first time install – it could be someone downloading on their phone; could be someone who’s been a Netflix subscriber for a while but never put it on their phone. That and the fact our figures are estimates is it will never be 1 to 1.”

Despite that limitation, the data may become more ubiquitous as new streaming services launch.

“I think everyone’s looking for an edge on subscribers,” says Nathanson. “These stocks trade on subscribers.”