The Algorithm

A finance bro and a math guy walk into a bar. Well, sorta. The two of us are an unlikely duo to run a marketing agency: Hamza was a math author and Andrew was deep in the weeds of private markets. So, naturally, the question comes up all the time - how and why are you guys doing what you’re doing? Off rip, math and finance are the furthest things away from marketing. But peel away at the layers, the Venn diagram has more surface-area than what meets the eye.

Why? Let us explain (in a very finance analogy). In the old days of Wall Street, relationships were key: alpha was earned through phone calls, negotiation prowess, and Le Bernardin dinners. This changed slowly - then all at once.

Starting in the early 2000s, financial networks stopped being run by people. Computers started replacing more and more work. By late 2007, Dan Spivey, backed by Barksdale (Netscape) and a $300mm war chest, started laying the groundwork of what would ultimately become Spread Networks. Reductively, the company was nothing more than a fiber optic cable from the Chicago Mercantile Exchange to the NASDAQ servers in Carteret, New Jersey. Companies had been laying subterranean cables since the late 19th century. But, this cable was far from ordinary. Aside from the cost, the 827-mile route was to be covered in a near dead straight line and full secrecy.

From the surface, this was a really dumb idea and a great way to light money on fire. But Spread Networks did one thing really well; they undercut the speed of existing networks that traders used to front-run pricing delta between CME futures and the underlying equites at the NASDAQ. Michael Lewis goes deep on Spread Networks and the murky world of electronic/quant/high frequency trading in Flash Boys. If you’re too lazy to read, you can watch Jesse Eisenberg explain all of the above in The Hummingbird Project. And, if you’re too lazy to watch, the major takeaway in Lewis’s exposé is the following:

Fast forward to today. As the era of active management sunsets and technology + liquidity prices in anomalies, your best bet is to go with the market. I’m sorry to say that 9 times of out of 10, you’re not going to beat the Carnegie Melon obsessive quant armed with Adderall, Asperger’s, and a much longer stick up their ass than yours. Just take Buffett’s advice: buy SPY and call it a day.

Ironically, Wall Street is a leading indicator to how shit gets built in Silicon Valley. Likely because the street is openly greedy, not the closeted greed of the peninsula as AGM points out:

Now back to 2007, the year Spread Networks started being built. At the same time, down to the month, another network was being laid, this time across the coast in Palo Alto. See, when Facebook first allowed ads on the platform in November ’07, it was adding to a seismic shift in how companies pay for a scintilla of attention in someone’s mind, which, after a few more clicks and electrons shuffling about, turns back into money. Just like The Street, the way ad-inventory was traded slowly transitioned to the whims of black boxes locked behind supercomputers.

These little boxes of computation, colloquially referred to as “The Algorithm,” remain a mystery (watch Amy Klobuchar try to figure it out). The reality is, it’s a black box and nobody has it figured out -- not even the engineers that build it. It’s a fool’s errand because it’s not real. The algorithm doesn’t exist.

No, really. We always get the question: “Have you cracked the algorithm?” What algorithm?

It’s basically all machine learning: matrices multiplied by other matrices to maximize some event. A math optimization problem at play. And each problem (i.e. campaign, day, cost for XYZ, etc.) is a different problem. Of course, this is very reductive, but what we’re getting at is there’s no formula or series of reproducible steps, so calling it an “algorithm” is misguided at best. If someone tells you they cracked the algorithm, run before you get scammed. As McConaughey put it, “It's all a fugazzi.”

For our friends at Facebook it wasn’t all sunshine and roses. Looking back at Facebook’s history, their Ad-algo had a rocky start: Facebook ads underperformed by as much as twenty times the worst targeted ads around the inception - another parallel to the early days of electronic trading on Wall Street (read Flash Crash).

The problem was handed off to Joaquin Quiñonero. Inspired by the Arab Spring, he joined Facebook and left his old role at Microsoft. Quiñonero began working on the ads and built up machine learning systems that improved their guesses over time. These algorithms would spot correlations between what users clicked on and their attributes (gender, age, location, content interests, etc), then use that information to guess which future ads they should be served. This algorithm began with near random guesses, but from its hits and misses, it kept refining.

This “algorithm” was, and still is, hardly great. Recommendations are still inexplicable at times. But given how cheap advertising was (and still relatively is), the hit rate doesn't need to be that high. A 2% click through rate is amazing for most advertisers. Quiñonero and his team would end up shipping updates at an unparalleled speed. Even a slight improvement would print money for Facebook. Facebook went from having a terrible algorithm to now having something that impresses even the most critical marketers. It can do a lot with very little and everyone wanted a piece of the action. So FB Learner was created. Basically a Wordpress for ML recommender algos. Instead of deliberately designing systems, engineers threw stuff at the wall and ML would figure it out. TL;DR, the “Algorithm(s)” we speak of.

So how do these algorithms work? There are some basic things that we can establish. They're at core—a market where ad inventory is the asset being traded. That’s why ad costs rise in certain times (spend is higher). You’re competing with other advertisers to reach the same eyeballs. Now the algo’s job is (very roughly speaking) to match eyeballs with ads to make the most money. If FB can make their advertisers money (or whatever their objective is), then they spend more and FB makes more. There is an optimization problem at play here. With limited eyeballs, how can you make the most money? That's what the algorithm solves for.

So, is there a way to “game” this algorithm? Hack it? The answer is as boring as Warren Buffet's portfolio strategy. You need to target broadly. 95% of the time, that's going to work better than trying to outsmart the algorithm. Just like Wall Street, there’s no longer alpha in complex targeting and strategies. You're competing for an ever-thinner edge. As Buffet tells you—simply buy the S&P and hold it. This is akin to letting the algorithm target broadly.

Why does this work? As mentioned, ad networks are like markets. That means everything is priced in given market efficiency. Your audience’s quality (let’s say their propensity to pay or convert in some capacity) is already priced in. So, trying to price it in yourself (targeting narrowly) won’t work and the algo will make sure to punish you. We oversee close to 100mm of spend annually. The bigger the client, the more they know simple=better.

Not only are you going to guess your target audience wrong by manually targeting (you're not capturing all the nuances), but you'll be punished for doing so by the algorithm through high CPMs. The algo has an opportunity cost function built in: if you make shitty content that gets users to scroll and exit the app, the paid algo will spike your CPM. It's an unforgiving machine aimed at maximizing every ad dollar.

When you target narrowly, you’re obviously not doing a better job than the algo. Just as the markets have an information advantage, so does the algo -- shuffling through millions of data points on each of their users. By targeting narrowly, you’re getting less than ideal users but the algorithm is forced to serve them. And the more aggressively narrow the targeting, the more CPMs go through the roof. The algorithm knows users will churn if shown the same ad with high frequency. Again, the algorithm is predictive and the opportunity cost is clear. Why show an ad to someone that'll cause them to leave the platform, meaning less overall ad dollars?

A lot of people ask us how these algorithms are so good. If you optimize for an install vs a purchase, you notice a 2x, maybe even 3x lift in purchases with 100 events in a day. How can it target so well with so little data? Algorithms don’t work on a user level anymore—they rely heavily on “collaborative filtering.” This is (partially) why your algorithm is so good. As unique as we think we all are—other users heavily inform your own algo/ad targeting. These inter-user relations are not captured in things like hashtag or topic interests because they are incredibly nuanced/unintelligible to a human. They are just matrices with numbers. This makes the algorithm even more inexplicable. No one can answer exactly why one ad appeared in your feed instead of another—it’s kind of the culmination of every other user and ad interaction. Obviously not, but re: butterfly effect for more.

You’re probably wondering, if what we’re saying is true (it is), how do you win? The key to winning is really simple:

● great data● great creative● great product(duh)

Simply focus on feeding the algorithm all the right signals at scale and none of the bad ones. No, like really avoid sending the bad ones back. For example, we had a fintech client who was feeding bank-add events back to TikTok. The only issue is—they were feeding back unsuccessful bank-add events as well. Cases where the login failed, cases where the user literally had a negative balance, etc. The algorithm (thinking it's maximizing the event) kept sending these low-quality users. Of course, they didn't get much out of their marketing.

The key is finding high signal, high frequency events so the algorithm can do what it does best: optimize. If you feed it shit, it gives you shit back. If you feed it gold... well, you get our point. Make sure your event infrastructure is built well.

The second focus is great creative. No algorithm can make your shitty creative work, sorry. There's no way around it. Your creative needs to be unforgivingly targeted to your main audience. Don't try trends. They don't work. Just make a good, engaging ad that your core audience can identify with. We do this at scale at NewForm. But you can try it too if you want. Just make good creative, pls.

PS: you gotta have a good product. Something people want. Duh. But a lot of people gloss over this step.

If you made it this far, we should chat. Email us at [email protected] / [email protected] or feel free to grab some time from our calendar here.