We tier up all the job changes we automatically detect into layers of cross validation. For the top tier, detected job changes that are validated by our system’s internal checks as 100% correct, we automatically pass through to our customer. For the bottom tier, ones our system isn’t confident about, we run a final pass through our QA team.
While we’re constantly building and improving our automated data pipelines, we’ve found that having a set of human eyes is invaluable for catching edge cases. Each edge case lends itself to an insight. And each insight can inform incremental system level improvements that start to matter at scale. Our QA team is the final guard rail to catch the car before it swerves off the cliff into a sea of bad data.
QA Challenges We’ve Faced
As our customer base grows, so does our quality assurance team.
Unlike scaling software infrastructure, scaling a QA team comes with its own set of unique challenges. How do we onboard new QA members quickly? How do we ensure consistent output and accuracy? What happens when suddenly, overnight, the number of contacts we need to monitor increases 5 fold, and half our QA team isn’t available or is on religious holiday?
The most important of these scaling questions, the one that far eclipses the rest, is how do we establish trust?
Trust that a person is honest, that they will be available, that they will do what they say, that they act with our intentions in mind.
In the beginning trust was much easier to build because we were few (the QA team were people I knew I could trust: myself and my cofounder Carina). Then our needs outpaced our ability so we added a few more. But we could still meet with and get to know each person in the team and listen to their concerns and challenges.
Scaling Trust
At some point though as Warmly grew, I realized I needed to start hiring QA managers, in addition to more QA folks. I went from knowing everyone, to knowing OF everyone, to not knowing what many of the members looked like. At a certain point it became impossible to keep track of everyone.
We started to see a slip in our data’s quality. New hires were making careless mistakes. They approved job changes that were clearly incorrect, even though everyone was trained on and used the same guidelines.
When we pointed out the mistakes to the managers to let their team know, it didn’t always stick. The members would correct their mistakes. Then a week would go by, and the same mistakes would crop up again.
For the earliest QA members (now managers), we interacted with them on a regular basis and got to know each other more personally. They understood Warmly’s mission and what we were trying to accomplish. In turn we got to know their own life goals. We built a relationship, and we established trust. We really enjoyed working with one another and they knew they were part of the family.
When we introduced layers of management, it allowed us to plan and coordinate in much larger groups. But it also prevented us from extending the human connection and empathy with the QA folks we were not interacting with regularly.
We realized we couldn’t replicate these interactions with everyone, but we could try something else; write them a letter. We could try to unite the QA team under our company’s shared vision, explaining to them why the work they’re doing is so important, and inspire them to take the shared ownership of quality.
And for those of you who are curious, below I’m sharing the exact letter we now send each new member of the QA team on their first day at Warmly. While it’s still early, we’ve already seen a lift in data quality, but more importantly, trust.
----------
A letter to Warmly’s finest
To the good people of Warmly QA,
All of you have been working hard, and we notice and appreciate the dedication. Thank you for your diligence and willingness to push through. Thank you for your positive attitude. And thank you for being a source of inspiration for the rest of the team.
I’d like to share with you all a bit about Warmly and how your work is contributing to changing the world.
Warmly’s Mission Statement
“Re-imagine the world of business through deeper connections”
What exactly does Warmly do?
In the simplest explanation, we help companies detect when their customers change jobs.
Why is this important for our customer?
Say, for example, one of our customers, Coca Cola, has a customer who moved from Google to Uber. Coca Cola wants to know about the job change ASAP so that they can find a new replacement advocate at Google or otherwise risk losing Google as a customer. At the same time, Coca Cola sales team wants to know about the job change so they can sell Coca Cola’s product to the new company, Uber, through the customer who just joined Uber.
Selling through a former customer has a 70% chance of conversion vs. cold outreach (throwing darts into the universe), which has only a 5% chance of conversion. It’s much easier to sell your product to someone who is already familiar and likes your product, just like it is much easier to work with someone who you are already familiar with and like working with!
How you are essential
The most crucial part of Warmly’s vision is realized through the work you all do. Every single contact you are able to find and every person you successfully detect as a job change is one more node that is added to the internal graph to build the customer network.
In the beginning the goal might look impossible. The QA team started with Carina, Carol, Emily and I searching google for people who might have changed jobs. It was slow but we never compromised on quality because we knew that high quality, accurate data was the only way we could win.
And we were right. Although we’re still a young budding startup, we’re growing fast. Because we are able to get the data back faster and more accurate than anyone else, we’ve received tons of positive emails from customers thanking Warmly for all the warm intros they were able to get.
Your efforts are being noticed from small startups like ours, to large, 500+ person companies. Customers across every industry are starting to look to Warmly, for warm intros to their future customers. And now that our QA team is growing, one job change detected will soon become 10,000, and one day, millions.
Our plan is to break into the world’s leading companies to revolutionize how sales is done across industries and geographies. People should not be sold products by salespeople, they should be referred products by their trusted friends, friends who want to advocate for a product they like and believe in. Warmly enables this to happen.
I’m not alone in being grateful for the amazing work you all are doing. It’s a hard road ahead, and at times stressful. But we’ll be with you each and every step of the way.
So on behalf of everyone here at Warmly,: Max, Val, Carina, Zack, Amanda, Emily, Brandon, Carol, Grant, Natalie, Sanil, Shahpar and myself, thank you again.
You are part of the family now and I hope you will see it as such. We rise and fall together.
Let’s go make a dent in the universe.
Warmly,
Sending you all our ❤️
What happened in Week 7 of Y Combinator?
Check out my cofounder Val Yermakova’s post Week 6 in Y Combinator.
. . .
Want to get in touch or send thoughts about the post? Would love to hear them ay [email protected]