A Quantitative Look at the Movie Business

How budgets and reviews drive box office results

By Harrison Burke

With most of our readers spending more time at home and more time watching Netflix, and with our team occupied by the end of the quarter, we asked our intern, Harrison Burke, to work on a rigorous quantitative analysis of the movie business. We hope you enjoy this change of pace from our normal financial wonkery.
 
Grab some popcorn and get ready to binge out on a quantitative look at what drives box office results and return on investment in the film business. Spoiler alert: it doesn’t work out the way you think it will.
 
I built a custom database of 6,621 films from 2000 to 2020 that includes each film’s budget, the gross revenue generated, the average critic rating from Metacritic, the average audience rating from IMDb, and details about the film like genre and description. I measured success with two metrics: total revenue generated and ROI (revenue/budget).
 
Audience rating on IMDB was, unsurprisingly, a strong and statistically significant predictor of both revenue and ROI. Below we show visualizations of both of these relationships.
 
Figure 1: Audience Ratings vs. Revenue (LHS) and ROI (RHS)

Source: IMDb, Metacritic, Kaggle, Verdad Analysis

The highest grossing films score 6-8 on average out of 10. Films that score much lower struggle to generate big box office results. It’s very hard to get very high rankings from a very high number of people, so movies that score 9 or 10 tend to be smaller budget, niche films. ROI is more linearly related to audience score, as those niche films with very high audience ratings tend to be quite profitable to produce. The above results are unsurprising: films audiences like better should generate higher revenues and higher return on investment.

Harvard Business School Professor Anita Elberse argued in an influential book that big money in the entertainment industry is made by making a few very large bets on a few very big winners – and that the film industry should migrate to focusing on a few big blockbusters rather than a hodge podge of smaller bets.

To test Dr. Elberse’s claim that bigger is better in the movie business, we ran regression analyses testing the relationship between budget and revenue and budget and ROI, controlling for audience score, critic score, and genre. Figure 2 shows the relationship between budget and revenue. Elberse's theory seems to broadly hold true here: higher budget films tend to earn higher revenues at the box office.

Figure 2: Budget (Natural Log) vs. Revenue (Natural Log)

Source: IMDb, Metacritic, Kaggle, Verdad Analysis

But Elberse’s theory does not hold for ROI. Below we compare budget to ROI, which shows a strong negative relationship.

Figure 3: Budget (Natural Log) vs. ROI (Natural Log)

Source: IMDb, Metacritic, Kaggle, Verdad Analysis

The graph shows clearly that budget and ROI are negatively correlated: smaller movies, not bigger ones, are more profitable for investors. When it comes to movie budgets, you get more bang, but not for your buck.

In addition to understanding the drivers of revenue and ROI, we also wanted to understand the relationship between critics and audiences and where the two most differed. Critics and audience scores were highly correlated, but there were some big differences.The below chart shows the difference in ratings on a 10-point scale between the audiences and the critics. The critics, unsurprisingly, had lower scores across the board than the audiences, but they had a particularly dim view of action, horror, and mystery films, while critics tended to be less negative relative to audiences when it came to rating musicals, westerns, and animated films.

Figure 4: Differences in Ratings by Genre

Source: IMDb, Metacritic, Kaggle, Verdad Analysis

For fun, we built a list of the 10 films with revenue greater than $20M where the difference between the audience score and the critic score was the greatest. Below is a list of the top 10 movies audiences loved but critics thought were awful.

Figure 5: Audiences’ Favorite Films, Relative to Critical Opinion

Source: IMDb, Metacritic, Kaggle, Verdad Analysis

We also looked at the inverse: the films critics thought were great that audiences despised. Figure 6 shows the top 10 film titles that grossed greater than $20M where critics diverge favorably from audiences.

Figure 6: The Critics’ Favorite Films, Relative to Broader Audiences

Source: IMDb, Metacritic, Kaggle, Verdad Analysis

By and large, the films that were a hit with audiences but despised by critics had big budgets and produced strong ROIs, whereas the films that were a hit with critics but had lower audience ratings were smaller budget films that produced extraordinarily high ROIs – small budget critical successes have historically been highly profitable investments.

We hope you enjoyed this quantitative deep-dive on the movie business. We’ll be back to our regular programming next week.

Acknowledgment: This piece was authored by our fall intern, Harrison Burke. Harrison is a junior at Harvard, majoring in philosophy. He was a member of the USRowing U19 National Team and rowed on the Harvard 2nd varsity eight that won gold at Eastern Springs in 2019. He is interested in pursuing a career in finance or data analytics.