TRAILR in action
TRAILR is the next-gen approach to consumer research - it has the analytical power of robust traditional consumer research paired with the speed, scalability and agility of AI-powered tech solutions.
But how does it look when it comes down to a trailer? Scroll down for a peek into how TRAILR works and reach out to schedule a full demo for your trailer!
STEP 1: SELECT TRAILER FOR ANALYSIS
We focused on Resident Evil 4’s Story Trailer on IGN - within 7 days, the trailer scored over 1M views and well over a thousand comments.
STEP 2: PULL RELEVANT DATA & PROCESS IT
Fresh data | We pulled all comments posted within 7 days of trailer publication and ran them through the TRAILR data processing framework
1,365 comments pulled in for analysis | Spam was removed, sarcasm was recoded
ClearVoice model injects context & deciphers meaning | Clean comments were run through our AI-powered models to ensure all references are clarified, context was given to commentary and semantic meaning was digested
Here is a sample of a data transformation that happens behind the scenes:
Gamer on IGN:
Ah... I'll buy it at a high price. 😁
TRAILR processed comment:
I am very interested in the game. (Commenter is likely already familiar with the game, as this is a reference to a famous line from the merchant character in the original Resident Evil 4 game.)
STEP 3: SELECTING A BENCHMARK
DATA ON ITS OWN IS IRRELEVANT WITHOUT CONTEXT
To know how well a trailer is doing, we need to understand how it scores compared to other trailers for similar types of games. When the data tells us that we have 30% interest to play, we need to know if this is strong or weak.
To achieve this, we use predictive benchmarking, which is only possible thanks to two features of the TRAILR framework:
-
All data is collected and processed meticulously to ensure that we measure the exact same metrics with a methodology that’s consistent wave on wave - this allows us to do apples-to-apples comparisons across trailers and over time.
-
We store all data in a curated database that holds all trailers and their metadata and allows us to pull out benchmarks for specific genres, types of trailers, channels or other relevant characteristics.
In the case of RE4, we selected a database of 26 trailers for games with a horror theme posted on IGN.
We also looked at a database of survival games and a larger set of action games, but for this specific trailer key take-aways remained similar across these benchmarks. We chose to go with the horror-themed games database, as it contains trailers that most closely match the RE4 one and allows for an apples-to-apples comparison across metrics.
Here are a few of the games, whose trailers made it into the database: The Last of Us Part II, Dying Light 2 Stay Hуman, Doom Eternal, Back 4 Blood.
Screenshots below are taken from IGN's official YouTube channel (https://www.youtube.com/@IGN)
STEP 4: UNDERSTANDING PERFORMANCE
Now the fun part - let’s look at results and see how well the RE4 Story Trailer performed!
Тo decipher performance, we focus on three key types of metrics:
Performance KPIs
These are core performance metrics (such as interest, uniqueness, advocacy); they impact how successfully the trailer sets the game for success at launch
Diagnostics
These are gamers' perceptions of the trailer and the game showcased (such as gameplay, story, characters, audio); it explains performance and areas for improvement
Deep Dives
These are custom metrics built to dig further into trailer-specific feedback on topics of deeper interest (such as drill downs on each character presented in the trailer)
Performance KPIs for RE4's story trailer:
Net performance - an overall performance rank, which takes into account both how many viewers talk about a given KPI (share of mentions) and how well they think the trailer delivers on it (Agreement)
Share of mentions - this is the % of all comments that contain feedback that can be matched to each KPI
Agreement - this is the share of comments within each KPI that agree the trailer delivers on a KPI. For example, 7% agreement to Uniqueness means that 7% of responses on this KPI find the trailer content unique; and 93% do not find it unique. This is a measurement that is unique to TRAILR and goes beyond the standard measure of sentiment that reflects how people feel about something - here we’re measuring performance and not how viewers feel about a given KPI. Example: “I love the game! It’s totally like all the other horror games I’ve played, but I don’t care” - the comment has a positive sentiment, but the respondent does not find the game in the trailer unique.
Each metric is benchmarked to give insight into how strong performance is on each dimension compared to typical levels for this type of genre and channel.
So, what are results telling us?
The RE4 Story Trailer performed very well and generated strong interest to purchase the game ahead of launch day.
It delivered on gamer expectations and inspired very strong advocacy - both scoring at the top of our database. Games loved the trailer, they are happy to endorse it and are excited to purchase the game.
There are two areas where performance is slightly softer (still Average vs the database, though), mostly because fewer than usual comments map to these KPIs:
-
General Interest - this is due to interest shifting towards Committed Interest, which is actually a sign of strong performance
-
Uniqueness - which is expected for a remake, esp. as part of an established IP
One area we could explore further is Liking - while commentary on this KPI is generally more intensive than usual for this genre, within the KPI itself agreement levels are only Average. This signals that this might be one area where further efforts could help push attachment even further ahead of release date.
STEP 5: EXPLAINING PERFORMANCE
A diagnostic deep dive: what drives & limits trailer potential for success
We track a series of categories and benchmark performance vs. the same database we used previously. This allows us to see where the trailer over- and under-performs and lets us diagnose its perceived strengths and weaknesses. We focus on three aspects of the data
-
how many people talk about each dimension
-
how strong their positive or negative sentiment is behind each dimension
-
to what extent positive and negative sentiment is linked to trailer performance on KPIs like Interest, Liking, Advocacy, etc.
Results for the Resident Evil 4 Story Trailer are below.
The RE4 Story Trailer impressed gamers almost universally across dimensions - they love the IP and are excited about what the remake will bring. They like the visuals, the gameplay, the story, the dark tone and atmosphere of the trailer. Feedback on Capcom as a studio is also very favourable and gamers appreciate the great work done on this game.
There are two dimensions that stand out with softer performance and where the analysis can bring valuable insights into how to address them prior to launch and whether it’s worth doing so - Audio and Characters.
Let's see more about these dimension of interest and see how they compare to other dimensions:
* Note that this is not a complete list of diagnostics within the TRAILR framework.
Excitement for a high-quality remake, engagement with the RE4 characters and liking for the trailer's execution are the core drivers of positive sentiment in the RE4 story trailer.
Audio is the only weak link, but it's not limiting sentiment strongly.
Audio commentary break-down: Let's see what's behind the Audio feedback
Break-down of AUDIO feedback:
Strong reactions to Ada voice in the trailer limit Liking and Advocacy
Trailer audio is the one area that received decidedly negative feedback. Since a significant share of comments goes to this dimension, it’s important to pay attention to it.
When it comes to Audio, the negative sentiment is mostly driven by dislike for voice acting in the trailer (mostly due to Ada’s voice over). The reason for the negative sentiment is a perceived misfit between the voice and Ada's character, which leads to a disconnect and is likely going to be immersion-breaking. Efforts towards helping gamers get used to the new voice will help address this.
This impacts two of the performance KPIs significantly - Liking and Advocacy - but doesn’t meaningfully limit interest in playing or buying the game. So making a change to how characters are voiced over or putting out further content to build attachment with the new voices can be helpful for further driving endorsement for RE4 - it will not increase net game sales, but will make marketing efforts more efficient and impact the urgency with which gamers buy the game.
Break-down of CHARACTERS feedback:
Character commentary break-down: Which characters are top of mind?
The trailer inspires rich commentary on characters - Ashley, Ada and Leon are the three top of mind protagonists.
-
Ashley | gamers talk about her updated appearance, her character, and her new-found love for weapons (the latter sparks some confusion and gamers would appreciate more backstory to this new take on Ashley)
-
Ada | feedback focuses on her appearance and her voice (gamers like her design but dislike her voice in the trailer)
-
Leon | gamers are excited to see their favourite lead character return and are happy with his design
.png)
TRAILR insight | Impact on trailer performance:
-
Leon & Ashley - boost trailer potential
-
Ada - limits Liking & Advocacy
Characters is a dimension which appears somewhat polarizing - there is a high share of positive AND a high share of negative sentiment behind character-related comments.
Diving deeper into Characters, it appears that negative feedback is mostly around character appearance, incl. their outfits. Here is what gamer feedback looks like for each character:
-
Ashley is the most talked about character, with largely positive sentiment. Some gamers are surprised to see her armed with weapons, because they are used to her being a helpless character, but this doesn't negatively impact overall trailer performance.
-
Gamers are excited to see Leon - some comment on missing his usual humour and personality and wish to see more of it.
-
Feedback on Ada sees some more negative sentiment, which is mostly linked to the voice in the trailer. This limits liking and advocacy to a certain extent, but doesn’t limit the overall interest in the game itself.
-
Luis is another character that draws some negative attention with his updated appearance in RE4, but overall this is a minor issue and doesn’t impact trailer performance.
Leon is a fan-favourite - showcase the personality & humour fans have come to expect of him
Ashley's transformation draws attention, highlight why she evolved
This kind of deep-dive can be done for any dimension of interest:
TRAILR lets you freely explore individual gamer comments. Comments are neatly organised in clusters by topic and color-marked to show you how positive/negative and how intense sentiment behind each comment is.
These are some of the highlights of the TRAILR analysis for this trailer. The tool itself can dig much deeper into any topic of interest and not only identify what gamers are talking about, but also give valuable insights on the sentiment behind comments and whether it drives or limits trailer performance in general or specific aspects of it.
Interested to see the full power of TRAILR? Contact us and schedule a demo!