CXL Institute CRO Minidegree Review | Weeks 7-9 of 12
Ali Q Naqvi
I build AI Tools backed by Behavioral Science to solve Business Problems | Ex. Ogilvy UK & Dentsu Aegis
Weeks 7-9 Overview
This is a rather long update- I have had to take a couple weeks off due to personal reasons. The team at CXL have been very understanding, for which I am very grateful.
I have now completed the course material. However, I will definitely need a couple of weeks to revise certain courses such as Advanced Experimental Analysis and Statistics for A/B Testing before I feel ready for the final exam.
Here is a summary of what I have been able to cover over the last two and a half weeks:
Course: Advanced Experimentation Analysis
Instructor: Chad Sanderson (https://www.dhirubhai.net/in/chad-sanderson-experimentation/)
Total time: 4 hrs 30 mins
Difficulty/Skill Level: Advanced
Key Takeaways
· Even if you don’t plan on doing any advanced computer programming, the R language is worth learning for analytics and CROs. It allows you to be much more hands on with your data manipulation and visualisation compared to a standard tool such as Google Analytics.
· On average, an optimisation budget should be split as follows: 10% on tech and 90% on people.
· We should have a clear ‘metric hierarchy’:
o At the top should be our ‘North Star’ metrics- those that are business-critical. E.g. Revenue per Visitor.
o Tier 2 metrics should be leading indicators that are strongly correlated with the top-level metrics. E.g. Conversions and Average Order Value.
o Tier 3 metrics are those with weak correlation with North Star metrics. E.g. Pageviews, clicks, etc.
· We should also consider creating custom metrics which are relevant and useful to our organisation. These may be blended or weighted according to observed business impact.
· There are 3 key requirements to have ‘good’ Experimental Analysis:
1. Statistical Comprehension
2. Trustworthy Analysis?
3. Honest Analysis
· One of the things we need to look out for when designing experiments is Sample Ratio Mismatch- this occurs when you have an uneven distribution of visitors between your Control and Variation treatment groups.
· Randomisation is a key component of conducting statistically robust and significant A/B tests. Outside of the marketing world, such tests are known as Randomised Control Trials.
· Whenever we see a significant result in a test, it is for one of two reasons:
1. We observed ‘significance’ by pure chance and there is no difference
2. We observed significance because the distribution is different
· Although it may be tempting, we should avoid stopping a test just because it hits significance.
· We should set the sample size for the test in advance, rather than deciding it on the fly. There are a number of sample size calculators available to help with this process.
· The larger the number of visitors in your test, the more likely you will see a significant result, IF it exists
· Just because we don’t observe a significant result, doesn’t mean there is not a true difference
· Statistical power is how often we would achieve a significant result if it exists. A test that is 80% powered, means that we would see our chosen degree of difference, 80% of the time
Course: Optimising for B2B
Instructor: Bill Leake (https://www.dhirubhai.net/in/leake/)
Total time: 20 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· The lead cycle for B2B is usually much longer compared to B2C. This needs to be factored into everything from website design, information architecture, content marketing efforts as well as the sales process.
· It is important to come up with an attribution model that makes sense for your organisation- this is usually based on observations of the role of different channels in a sales cycle. Due to long lead times, the last-click channel may look like it should get all the credit for the sale- this usually isn’t the case.
· Make sure the metrics you are tracking have a real connection to business results- e.g. observe connections between how much time users spend viewing content on your site and how many of them convert into leads.
· When fuelling your sales engine with leads, there is a delicate balance between the elements of quality, quantity, and cost. You must choose two out of Fast, Cheap and Good.
· Optimise for personas from the bottom of the funnel up (base them on the types of customers that actually deliver the most value for your organisation).
· You need to have a deep understanding of your sales engine before deciding how to optimise your B2B marketing- e.g. if most of the sales are done over the phone by the sales team, your PPC ads need to be optimised for generating calls.
Course: Customer Value Optimisation
Instructor: Justin Rondeau (https://www.dhirubhai.net/in/jtrondeau/)
Total time: 18 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· The first step in optimising customer value is to hone in on the key value proposition of your product or service. This will help determine your target market and messaging hierarchy down the line.
· A ‘lead magnet’ is a great way to get a prospective customer into your database without much of an ask- you usually ask for their email address in exchange for a free piece of information that will help them achieve their goals. E.g. free video, one-page checklist/guide, etc.
· The next level up of customer ‘asks’ is the trip-wire offer. This is usually an entry-level package or offer that doesn’t cost much money, however, it programs the customer to feel comfortable in spending money with you and receiving value in return.
· Once a customer has already bought the trip-wire offer, you have an opportunity to upsell your more profitable offerings, i.e. your ‘profit maximisers’. You must ensure that there is a clear difference in value between your different offerings.
· Always have paths available to customers to come back into your funnel (e.g. retargeting them with a special offer through Facebook ads if they have dropped off at the last stage of the checkout process).
Course: Creating a Segmentation Strategy
Instructor: Candace Ohm (https://www.dhirubhai.net/in/candaceohm/)
Total time: 18 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· Segmentation is a powerful way to personalise your marketing to key groups/personas within your target audience. Segmentation can be based on a number of factors including (but not limited to) location, past behaviour, marketing channel, etc.
· There are a number of tools available to help with segmentation, ranging from free tools such as Google Analytics to more sophisticated tools such as MixPanel which allow you to customise user experiences based on segmentation rules.
Course: Digital Psychology and Behavioural Design Training
Instructor: Brian Cugelman (https://www.dhirubhai.net/in/cugelman/)
Total time: 3 hrs 45 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· Two elements are required for mediated influence- the influence technique (e.g. educating, coaching, selling, etc.) and the media (e.g. websites, mobile apps, online ads, etc.)
· Superficial indicator metrics such as pageviews, scrolls, tagging, etc. can be connected to real impact on human experiences such as comprehending, desiring, deciding and acting.
· Contrary to the neat linear or funnel theories of a customer journey, Dr. Cugelman says that the order in which a customer comprehends, desires, acts, etc. is not particularly important. Customers tend to flit between these different stages multiple times within a standard journey anyway.
· Motivation is an emotion that facilitates action.
· The 4 Emotional Quadrants are: Insecure, Secure, Optimistic and Pessimistic. We want our customers to be in the Optimistic quadrant as this is where they are not too secure in the status quo (leave things as they are) and are motivated to act for their own benefit.
· Confirmation prompts once a customer has signed up are a very useful tool in negating buyer’s remorse and boosting customer retention.
· Using spatial metaphors is a way to help users navigate software (e.g. treating webpages as ‘rooms’ to give users a clear indication of where they are within their website journey).
· Using breadcrumbs and directional cues is a great way to guide users towards your key landing pages / CTAs.
· Features tell, but benefits sell.
Course: Applied Neuromarketing
Instructor: André Morys (https://www.dhirubhai.net/in/andremorys/)
Total time: 4 hrs
Difficulty/Skill Level: Intermediate-Advanced
Key Takeaways
· People may forget what you said, but they will never forget how you made them feel.
· It is important to have an understanding of basic neurobiology:
o The Brain Stem, also known as the Old Brain, controls our ‘auto’ functions such as breathing, blood pressure, motor control, etc.
o The Cerebellum, also known as the ‘Little Brain’, controls most of our motor functions. This is part of what Daniel Kahneman calls ‘System 1’.
o The Mid Brain is in charge of our Limbic System. This is the area of the brain that controls our emotions, motivations, anxieties, etc. This is also part of System 1.
o The Cerebral Cortex, also known as the New Brain, is what we use for logical thinking, calculations, reasoning, etc.
· Because our optical system is connected to our old brain, we ‘instinctively’ create stories out of whatever visual impulses we are receiving in order to give them meaning.
· The Limbic Map is an excellent tool to help us map different implicit codes in our advertising against various human emotions.
· A good way to see what implicit codes your website is projecting is by conducting a ‘5 second test’- give a user 5 seconds to look at your website and then tell you what they think it’s about.
· You have 50 milliseconds to show the website visitor that this website is relevant to them. You must capture their attention before guiding them down the user journey you have crafted for them.
· On average, users only ‘get’ about 5% of the message that the website creator was trying to convey. Every assumption must be tested thoroughly with the target audience.
· There are certain principles you can use to capture and maintain a user’s attention, such as:
1. Using Contrast
2. Utilising Negative Space
3. Showing People/faces to guide attention
4. Showing Movement
5. Breaking rules (pattern interruption)
6. Directional cues (arrows, dots, etc.)
7. Person’s name/their own face (personalisation).
· Most personas are useless. Identify the key emotional motivator for a persona, map it against the limbic map and use that as a guide to craft messaging and UX for them.
Course: How to Design, Roll Out & Scale an Optimisation Program
Instructor: Merritt Aho (https://www.dhirubhai.net/in/merrittaho/)
Total time: 20 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· For technologically complex organisations, a distributed optimisation model works better than a centralised model.
· For a centralised model, it is important to have an CRO program engagement manager that works closely with the key stakeholders within the organisation.
· Most companies should aspire to have an in-house CRO solution. However, hiring for in-house roles can take a long time- in the interim it is often advisable to work with external agencies and eventually build up your in house capabilities.
· Speed to execution is critical in the early stages of a CRO program in order to show progress and build momentum within the organisation.
· CRO gains tend to be greater in the early parts of a program launch as there is a lot of low hanging fruit for improvement in results.
Course: Evangelising for Optimisation in Enterprise
Instructor: Merritt Aho (https://www.dhirubhai.net/in/merrittaho/)
Total time: 20 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· When you try to launch a CRO program, there will be many within your organisation who think they are already doing optimisation. It is important to not overly criticise their approach, rather offer ways in which their optimisation efforts can be further enhanced for their own benefit.
· Building a clear business case for CRO involves developing key metrics in collaboration with the main stakeholders of the organisation and then providing reasonable projections of ROI and improvement.
· It is important to spread as much information about your CRO program as possible, however, you must make an effort to make it digestible for non-technical people. Data visualisation and summaries within newsletters are a great way of doing this.
Course: Building Your Optimisation Tech Stack
Instructor: Merritt Aho (https://www.dhirubhai.net/in/merrittaho/)
Total time: 20 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· Ensure that clear goals are defined for your CRO program before investing in tools.
· Find opportunities to get other departments within your organisation to pool in their resources for tools that have common benefits (e.g. Analytics tools).
· There are different levels of tools available depending on what stage of your CRO program you are in- do not over-invest in high tech ‘Run’ tools when you are in the ‘Crawl’ stage.
Course: Optimise Your Optimisation Program
Instructor: Claire Vo (https://www.dhirubhai.net/in/clairevo/)
Total time: 20 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· Track inputs rather than just outputs for your CRO program- this means setting process goals alongside results goals.
· Measure test quantity via testing velocity, testing capacity, and testing coverage. These are indicators of how prolific your program is and how much more opportunity there is on the table with more resource investment.
· Measure test quality via win rate and expected value- don’t just base your assessment of your program on your latest test results(!).
Course: CRO Agency Masterclass
Instructor: Peep Laja (https://www.dhirubhai.net/in/peeplaja/)
Total time: 1 hr 50 mins
Difficulty/Skill Level: Intermediate
Key Takeaways
· Sell units of activity rather than hours (e.g. an analytics audit vs. 5 hrs of an analyst’s time).
· You may need to start small and build credibility, however, ultimately you want to go for premium clients. They are usually willing to spend more and give you less problems than small budget clients whose whole marketing strategy may revolve around your program.
· Focus on inbound lead generation through sharing your expertise rather than being a door to door salesman.
· Client relationships are just as important as delivering results. You need both in order to become indispensable.
· You may not need a contract for every engagement, however, ensure that you have written agreement over the specific details of every deliverable you are offering to your client. There should be no ambiguity here.
Wrap Up of Weeks 7-9
What I Loved
· I love Andre’ Mory’s Limbic Map. I think it’s a great tool to make persona’s actionable and match our page copy to actual human motivations and emotions.
· Even though I am intimidated by the maths, the Advanced Experimentation Analysis course delivers great value- you learn how to use a new programming language (R) and apply advanced statistical methods to manipulate and visualise data. This is very cool.
What Could Be Better
· The Digital Psychology and Behavioural Design Training course had some interesting information, however, I felt a lot of it has already been covered in previous courses within this minidegree.
Plan for Week 10
· Review and complete exams for Google Analytics Audit, Advanced Statistical Analysis, Statistics for A/B Testing and A/B Testing Mastery.
About this series
On 23rd March, 2020, in quarantine indefinitely due to the UK’s (perfectly reasonable) nationwide lockdown, I was lucky enough to gain a scholarship to the CXL Institute’s CRO Minidegree program for 12 weeks.
I am writing a weekly update of my progress through the program. You can find my previous updates below:
Week 1: https://www.dhirubhai.net/pulse/cxl-institute-cro-minidegree-review-week-1-12-ali-qasim-naqvi/
Week 2: https://www.dhirubhai.net/pulse/cxl-institute-cro-minidegree-review-week-2-12-ali-qasim-naqvi/
Week 5: https://www.dhirubhai.net/feed/update/urn:li:activity:6660291575731503104/
The CXL Institute CRO Minidegree Program
Conversion Rate Optimisation (CRO) refers to the art and science of converting website/app visitors into customers.
The CXL Institute Conversion Optimisation Minidegree is an online training program designed to be “the most thorough CRO training in the world”. It is taught by CXL Institute’s in-house staff (including its founder Peep Laja) as well as a collection of the leading marketing practitioners in the business.
In its own words, CXL Institute describes the key objectives for students of this course as being to “learn how to convert your traffic into customers, run more successful experiments, and get the skills to deliver consistent revenue gains on any website you work on.”
The approximate total time for this course is: 72 hrs 48 mins.
You can find more information here: https://cxl.com/institute/programs/conversion-optimization/
Thanks for taking an interest in my learning experience. I look forward to sharing more next week!
Ali