As of two weeks ago, a dozen brands were planning to skip Super Bowl LV. Since then Budweiser, Microsoft, and Walmart each announced they would also sit out this year’s Big Game.
Even so, a bunch of other brands filled the void and ViacomCBS sold out of its Super Bowl LV ad inventory.
(I guess that explains why the game didn’t end after three quarters even though the score would have been the same.)
So, who made the right decision?
Was it the roughly 20 former Super Bowl stalwarts that opted out of this year’s Big Game, the 20-some first-time advertisers who opted in, or the approximately 30 brands that never left?
It all depends on how they measure the impact of their Super Bowl ads.
In this column, we’ll check out the best and most effective Super Bowl ads and share marketing takeaways, as well as exploring how three different ad measurement methodologies stacked up.
What Does Super Bowl Ads Measurement Matter to Me?
Digital marketers are data-driven.
And being data-driven is an advantage – if you know which data should drive which decisions.
Or, as Avinash Kaushik advised in a tweet back on Sept. 28, 2017, “Never judge a fish by its ability to climb a tree.”
It’s worth noting that the primary business goal of many of the brands that spent an average of $5.5 million on a 30-second Super Bowl spot wasn’t driving traffic to their website.
Their goals were to increase brand awareness, consideration, favorability, and purchase intent of the 96.4 million viewers who watched this year’s not-so-big game.
(Interestingly, that’s the lowest number of people who watched a Super Bowl since 2007.)
This explains why it’s well worth analyzing the methodologies used to determine this year’s best Super Bowl ads even if you aren’t the CMO of a consumer brand company… yet.
Heck, it will provide you with short-term benefits, if you direct display advertising or head up content marketing.
And, down the road, it will help you to harness brand-building campaigns to improve the results of your performance marketing campaigns.
So, where to begin?
I reached out to Jeff Ferguson, a partner at Amplitude Digital and fellow SEJ columnist who leads university classes in Advanced Digital Marketing and Search Engine Optimization, to get his take on the ads that aired during Super Bowl LV.
Below each ad, you’ll find either my take (GJ) or his (JF).
Bronze Medal in Big Game Measurement Methodology Goes to USA Today’s Ad Meter
If there were medals for methodology, I’d award the bronze to USA Today’s Ad Meter.
Ad Meter’s approach to measuring public opinion surrounding Super Bowl ads has a flaw that dates back to its inception in 1989.
Panelists are still asked to rate more than 50 national commercials using a meter that registers their favorable or unfavorable impressions from 1 to 10.
However, a favorable impression of an ad may or may not impact the metrics that matter: lifts in brand awareness, brand consideration, brand favorability, or purchase intent.
In addition, Ad Meter’s methodology in 2021 has a new flaw.
Although USA Today expects to have thousands of panelists, these consumers may or may not be a representative sample of U.S. adults over 18.
This is because Ad Meter’s registration process no longer gathers data from users about their age, gender, location, or salary because of the California Consumer Privacy Act (CCPA).
Finally, Ad Meter only rates the national commercials aired between the coin toss and kickoff all the way to the two-minute warning in the fourth quarter.
They don’t rate the teasers, extended cuts, or ads like “Tom Brady & Rob Gronkowski | Big Game Ad | #TheGOATin5G | T-Mobile,” which T-Mobile claims was “banned” from airing during the Super Bowl because of a “protected telco rights deal.”
Nevertheless, the winners below – deemed the top 10 Super Bowl LV ads, according to Ad Meter – got a lot of publicity.
Top 10 Super Bowl LV Ads According to USA Today’s Ad Meter
So, you will want to know which commercials “won,” even if USA Today doesn’t use a best-in-class methodology to gather feedback directly from consumers.
1. Rocket Mortgage: “Certain Is Better – Tracy Morgan, Dave Bautista & Liza Koshy” (with an Ad Meter score of 7.38)
JF: I call this type of spot “a joke looking for a brand.”
While it does a great job of leading with the brand, the rest of the spot has nothing to do with the brand (or mortgages, for that matter).
It’s like someone wrote the spot and waited for a client to approve it someday. Funny, but forgettable.
2. Rocket Mortgage: “Certain Is Better – Tracy Morgan & Joey Bosa” (7.30)
JF: Same gag, still funny… at least now we know who it’s for.
3. Amazon Echo: “Alexa’s Body” (6.75)
GJ: Okay, I get the joke. “Who knew Alexa had abs?”
I support empowering women in product management and advocating for equal representation in advertising.
But I found myself relating to the poor husband in the ad, who kept saying stuff like, “Honey, I already ran the sprinklers. Things are getting way too wet around here.”
4. M&M’S: “Come Together” (6.73)
JF: M&M are masters of brand; I mean their brand just walks around and talks to people.
This year’s spot does a nice job of getting the message across and features a great cameo at the end.
Nice work.
5. Toyota: “Upstream” (6.71)
JF: First spot to make me cry so far.
Very well done, even if they waited until halfway through to subtly reveal the brand, people are going to talk about this one for a while.”
6. General Motors: “No Way Norway” (6.67)
JF: Funny and a nice early brand reveal made this ad much more effective than anything that came before it today.
Solid work.
7. Cheetos: “It Wasn’t Me” (6.52)
JF: Didn’t see that coming…
This was both a great use of brand and entertaining.
I mean, maybe it was a bit of a reach to use this song, but whatever. Solid work, Cheetos.
8. State Farm: “Drake from State Farm” (6.50)
GJ: I’m a big fan of Jake from State Farm. But, why add Drake?
In his classic book, Ogilvy on Advertising, David Ogilvy said that ads featuring celebrities tend to score below average in their ability to change brand preference.
He added, “Viewers guess that the celebrity has been bought, and they are right.”
9. Doritos: “Flat Matthew” (6.40)
JF: Matt McConaughey can usually do no wrong, but from luxury cars to this? Come on.
Plus, another wait until the end to reveal the brand. Lame… sub-par for Doritos.
10. Bud Light Seltzer Lemonade: Last Year’s Lemons (6.36)
JF: A weird ad for a product that shouldn’t exist.
How do you talk about a pandemic without actually mentioning the pandemic?
Make it rain lemons on everybody.
Great brand mention right out the gate from a brand that always gets that part right, but otherwise, meh.
Silver Medal in Big Game Measurement Methodology Goes to Ace Metrix
Ace Metrix gets my silver for a methodology that measures the strength of the emotional connection of video ads using a proprietary approach that quantifies 57 emotions and reactions including Funny, Heartfelt, and Empowering, as well as Dishonest, Preachy, and Sexist.
Their scores are passively derived from voluntary verbatim comments by a panel of 500 people.
Comments are analyzed using Natural Language Processing (NLP) and machine learning.
Each of the 57 emotions is then compared to all of the other ads in the Ace Metrix database, which totals over 10 million verbatims.
Now, digital marketers and SEO professionals are familiar with NLP and machine learning.
So, the data-driven among us may not be overly concerned that a computer is looking at the words used by people to describe an ad and then using machine learning to work out how happy, angry, or sad they were while watching it.
While this methodology is scalable, machines still have a long way to go before they can truly understand how we use language.
There are a lot of nuances to the way we speak.
For example, if your cousin from Boston says. “Sam Adams’ new Wicked Hazy is the juiciest IPA ever,” then he thinks that it was really good.
Top 5 Funniest Super Bowl LV Ads
According to the Ace Metrix methodology, here are the five funniest Super Bowl LV ads:
1. Cheetos: “It Wasn’t Me” (Funny Score 8.4).
See ad above.
2. Tide: “The Jason Alexander Hoodie” (Funny Score 8.3).
JF: ‘Hey, we’ve got access to Jason Alexander. Can we use him for anything?’
Yet another joke looking for a home, and it found one with Tide.
At least they tied it in with washing clothes at the end, but otherwise, this brand has done much better.
3. Bud Light Seltzer: “Last Year’s Lemons” (Funny Score 7.8).
We also looked at this ad above, so nothing to see here. Move along.
4. M&M’s: “Come Together” (Funny Score 7.2).
Yes, we look at this ad above, too. So, I guess people like funny ads.
5. Amazon Echo: “Alexa’s Body” (Funny Score 6)
Guess what… we looked at this ad above, as well.
It appears that a lot of decision-makers at brands read the story in Ad Age about the AdColony survey that found 81% of people wanted Super Bowl ads to be funny this year.
But, as I reported back in April, a majority of consumers were seeking uplifting YouTube content during the COVID-19 pandemic.
So, let’s look at the five most empowering Super Bowl LV ads, according to Ace Metrix.
Top 5 Most Empowering Super Bowl LV Ads
1. Indeed: “The Rising” (Empower Score 5.1)
JF: Unknown Jobs Website… oh wait, it was Indeed.
Very emotional, really touched a few nerves for me, but once again, they waited until the end for the reveal.
This could have been for any job website.
2. NFL Inspire Change: “As One” (Empower Score 4.9)
JF: Wow, Hologram Lombardi. Oddly effective. Second set of tears tonight.
3. Toyota: “Upstream” (Empower Score 4.4)
We covered this above, so need we say more?
4. Guaranteed Rate: “Believe You Will” (Empower Score 4.4)
GJ: Back in October 2018, I wrote How to Connect with Industry Influencers: Schmooze Optimization.
Well, that’s exactly what Guaranteed Rate did with this Super Bowl ad that features their brand ambassadors, who believe in their abilities to reach above and beyond.
If influencers represent your brand’s core values, you should also feature them in your digital marketing campaigns – even if you don’t air a Super Bowl ad.
5. Bass Pro Shops: “Get Back to Nature” (Empower Score 3.5).
GJ: Should your brand take a stand on controversial issues?
Encouraging Americans to participate in outdoor activities during the pandemic shouldn’t be controversial, but seems to be.
Kudos to Bass Pro Shops for doing the right thing by running this spot.
Gold Medal in Big Game Measurement Methodology Goes to Unruly EQ
Unruly EQ takes the gold.
What makes their approach to measuring Super Bowl ads better than the other two mentioned above?
First of all, their content measurement tool, Unruly EQ, uses a combination of audience panels and facial coding to measure viewers’ emotional responses to advertising.
The results are then compared to Unruly’s database of thousands of ads to see which ads generated the highest levels of a wide spectrum of psychological responses.
For example, here’s a typical EQ Lite report that Unruly generates for each ad it tests.
They also have reports that look deeper, such as second-by-second facial reactions and content recommendations.
Unruly also measures a number of different metrics, including:
Emotions – Includes overall emotional intensity and specific emotions. In addition to telling you which ads generated the strongest emotions, Unruly EQ can also tell you which ads made people the happiest, saddest, most inspired, etc.
Brand metrics – Also measures brand metrics such as purchase intent, brand favorability, brand recall, and interest in finding out more.
Brand values – Looks at other metrics such as authenticity, trustworthiness, and relatability. It also asks whether the content included various stereotypes (race, gender, etc.) and whether the brand came across as sustainable.
Most of these metrics then contribute to an overall effectiveness score – or EQ Score – which ranks all Super Bowl ads from 1 to 10.
Like Ad Meter, Unruly uses thousands of panelists for their Super Bowl rankings.
But, unlike Ad Meter, Unruly’s panels are nationally representative and collated using local third-party panel providers.
And unlike Ace EMO, Unruly actively asks people how they feel. Panelists are asked to rank their emotions for each ad, so there is no mistake.
Unruly also uses facial coding to supplement this for some ads, because sometimes what people say and how they react are different.
Pay close attention to Unruly’s ranking of the 10 Most Effective Ads of Super Bowl LV.
Top 10 Most Effective Super Bowl LV Ads
1. Toyota: “Upstream” (EQ Score of 8.1 out of 1))
This is the most effective ad of Super Bowl LV, according to new research from Unruly.
The study found the ad from the auto brand, which features 13-time Paralympic gold medalist Jessica Long, was the campaign most likely to have the biggest impact on brand and business metrics.
It appears that Toyota made the right decision to tell the heart-warming story of Team Toyota athlete Long’s incredible journey, which began when she was adopted by an American couple from a Russian orphanage.
In fact, “Upstream” was the most inspiring of all the Super Bowl ads tested, 163% more than the average US ad.
The ad also generated intense feelings of warmth, happiness, and amazement, but also had viewers reaching for the tissues, with the ad 164% sadder than the US norm.
Such an intense emotional response also had a significant impact on its brand metrics.
Along with Doritos’ “Flat Matthew”, which we’ll look at again below, Toyota managed to drive the highest level of desire to find out more among consumers (70%), while brand favorability (66%) and purchase intent (66%) were also well above the US norm.
Now, four ads finished with an EQ score of 8.1 out of 10.
But, Toyota’s minute-long commercial took the Super Bowl ad crown after generating intense emotional engagement from 63% of viewers, more than any other Super Bowl LV ad – and almost twice the average US ad.
2. Jeep: “The Middle” and Bass Pro Shop: “Get Back to Nature” (tied with an EQ Score 8.1)
Jeep’s call for national unity, “The Middle,” starring Bruce Springsteen, and Bass Pro Shops’ tribute to The Great Outdoors, “Get Back to Nature” were tied for second, with both attracting intense emotions from 60% of viewers.
And just as Bass Pro Shops deserves kudos for doing the right thing by running their spot, so does Jeep.
It takes courage to tackle a controversial issue.
4. M&Ms: “Come Together” (EQ Score 8.1)
Although we covered this ad above, Unruly reports that it attracted intense emotions from 55% of viewers.
5. NFL: “As One” (EQ Score 8.0).
We also covered this ad above, but notice how empowering ads are outperforming funny ones when you use a composite score based on a campaign’s ability to not only engage consumers but to drive brand metrics such as brand favorability and purchase intent.
6. Doritos: “Flat Matthew” (EQ Score 8.0)
We covered this ad above, too. So, funny ads work.
However, this demonstrates that there is more than one way to engage consumers and drive brand metrics at the same time.
7. DoorDash: “The Neighborhood” (EQ Score 7.9)
GJ: This ad didn’t appear above. Still, it made it into Unruly’s list of the 10 Most Effective Ads of Super Bowl LV.
So, is your marketing objective to get lots of people to like your ad or like your brand?
As I mentioned earlier, a favorable impression of an ad may or may not impact the metrics that matter: lifts in brand awareness, brand consideration, brand favorability, or purchase intent.
8. Anheuser-Busch: “Let’s grab a beer” and Remy Cointreau: “Love Letter” (EQ Score of 7.8)
JF: Now that was a hell of an ad. They should have lead with that one. Third cry of the night.
GJ: Like the DoorDash ad, this one didn’t appear above. But, Cointreau and The Independent Restaurant Coalition are asking viewers to send a love letter to their favorite local restaurants and bars.
Restaurants and bars need our support now more than ever before.
The COVID-19 pandemic impacted nearly 500,000 independent establishments and over 11 million people, leaving restaurants venerable to permanent closures and restaurant staff without work.
Making local restaurants and bars the heroes of your ad is something that digital marketers should also consider doing.
10. Fiverr: “Opportunity Knocks” and Cadillac: “ScissorHandsFree” (EQ Score of 7.8)
JF: That’s a hell of an ad.
I’m not sure why they went for a Super Bowl spot, but hey, at least the creative got the point across well.
Nicely done!
JF: I really wanted this one to work, but yeah, it just didn’t.
Little too far from this movie for me to really be that sentimental.
Super Bowl LV Hit Us in the Feels
Finally, Super Bowl 2021 was a particularly high-scoring year, according to Unruly, with ads attracting an average EQ score of 7.4 — almost three points higher than Super Bowl 2020’s average.
Ads also generated 53% more emotional engagement on average than the previous Super Bowl.
According to Terence Scroope, the VP of Insight & Solutions for Unruly and Tremor Video:
“Emotions were certainly a lot more heightened this year, with viewers’ emotional responses on average a lot more intense than any of the five previous Super Bowls we’ve tested. With performance and brand metrics also a lot higher, it meant this year’s crop of ads attracted the highest EQ Scores we have ever seen at the Super Bowl.”
He noted too that while many ads featuring celebrities tried to make us laugh, some of the top performers focused on emotion and creating a narrative.
Overall, Bud Light’s “Legends” and Cheetos’ “It Wasn’t Me” were most likely to make people laugh, with 17% of viewers finding the content funny (more than 4 times the U.S. average).
Even so, it’s worth noting that funny ads from Amazon Echo, Bud Light, and Tide did not make Unruly’s top 10.
So, when selecting a methodology to measure the impact of your ads—whether for the Super Bowl or a brand-building campaign, “You must choose, but choose wisely.”
It’s an important lesson to learn, even if you aren’t the CMO of a consumer brand company… yet.
Resources
Here are additional resources for you to check out.
- Super Bowl 2021 Insights, Unruly
- The 5 Funniest Super Bowl Commercials, AdAge
- The top 10 Super Bowl LV ads, USA Today
- Super Bowl ratings: Game fails to attract more than 100 million viewers, CNBC
More Resources:
