Design Is [Data] – A case study in designing Ads on Google Search

Design Is [Data] – A case study in designing Ads on Google Search


[MUSIC PLAYING] KAI HALEY: I’m Kai Haley. I’m one of the creators
of this program. And as many of you know,
we started this series to create a space for
the design community to come together to
explore the role of design in shaping the future. The processes and
tools and mindsets that we have in our
toolkit available to us are really essential for helping
us do a good job with that. And I am really excited to be
able to bring this topic to you all tonight, not
just because data is becoming the evermore
important component that we have to consider
in our design process, but also because
the work that Jason is going to present tonight has
been very influential for me personally in my
career as a designer. So I am very excited to
be able to have him share this work, which has been
invisible, largely, for a while now but is really
valuable to Google, so it’s really great that
we’re able to share it. Jason is a UX design
manager for Google Shopping. He is responsible for
all of the shopping experiences across Search,
Assistant, and Google Express. He also has a background
at Effective UI and agency, and prior to that, worked at
Kodak on consumer electronics. He has an MFA from RIT
in computer graphics and an associate’s
degree in illustration, minor in philosophy. Lots of great background there. And most importantly, Jason
has an incredible ability to keep white pants clean, even
with two small children, which is something I was impressed
with every day of the 3 and 1/2 years that I had the
privilege of working with him. So with that, I’m going
to hand it over to Jason. Thank you so much. [APPLAUSE] JASON BLYTHE: Hi, everyone. This is a big topic,
Design is Data, and so in 30 or so
minutes, it will almost be impossible to cover
every dimension of it. So I’m going to try to tell
a slice via a case study that focuses on a story about
designing ads and the ad badge or the ads label on Search. So you should learn a little
bit about Search, a little bit about designing ads on
Search, and how data plays a role in the design process. As Kai mentioned,
I’m the UX lead for Google Shopping,
responsible for the consumer experiences on
the shopping team. But I want to go further
back in my career at Google to talk about my time
leading the Search ads UX team, a group of designers that
designed the ads that show up on Search. And I think when I talk to lots
of designers outside Google, I usually get a quizzical
look, like, what do you mean there’s designers
on Search and there’s designers for Search ads? Like, Search doesn’t change. It’s been the same forever. And I think that’s
a neat trick, right? It’s literally
our sort of ethos. Search has become ubiquitous
for lots of users, and the idea here is to
design it in a seamless way. Jon Wiley is the former UX
lead for Search, currently a UX director for the Daydream team. He had a saying that
goes something like, Search is like trying
to overhaul a 727 airplane while it’s in flight
and have none of the passengers notice. So I’ll try to explain a
little bit of how that works and talk about ads in specific
here while we build our thesis. So what is data, anyway? Data at its heart is
just a bunch of facts. On their own, they
don’t do much, right? Whether they’re specific
facts, whether those facts are abstracted into numbers,
which I think people sort of associate with data, or they’re
statistical, probabilistic, an accumulation
of multiple facts. And data is everywhere. A random picture from
my photo library. We were shopping for a
few pots, ceramic pots, for some houseplants. And this picture has data. There’s a lot of ways to imagine
what data is in the picture. Like, I could tell you
it’s a grid of pixels. Each pixel has a hex code. Each pixel has a
location in the grid. That’s probably
unexciting data, right? But data has a
purpose in this story. Design is a
purpose-driven activity. So let’s imagine
another kind of data. Like, does anyone
sort of have a sense of what these numbers overlaid
on the image might be? AUDIENCE: Price. AUDIENCE: Size. JASON BLYTHE: Price, size. Now, see, this is good. You guys are starting
to realize that there’s lots of ways I
could layer on data or extract data from this image. In this case, it’s a
diameter, the opening diameter of each of the pots,
at least visible here. It’s just a number,
this uninteresting fact about this picture. So let’s isolate one. There is a thesis that I want
to start to weave in about data, about the purpose-driven
nature of data. It always starts with
a problem, like you’re trying to find something out. You’re trying to
derive something. So I could look at this
image and say, well, what are the chances it’s
going to topple over? Data can help you answer
that question, right? It might be unknowable, but
you could probably fake it. You could say, well, what’s
the overlap of the base versus the base that it’s on? What is the–
perhaps the weight? What’s the wind? What’s the surface
area of the pot? Is there something
in it that’s moving? Like, you could come up
with a lot of things. Each of these things
could be a data point. You could plug that data into
some kind of formula or rubric for understanding. You’re getting a
sense, like it’s data, data trying to solve a problem. There’s a purpose. Perhaps there’s a
formula, a relationship between the multiple
data points. And in the end, you’ll
end up with something that’s probabilistic
because you can’t know that. It hasn’t happened yet. And so that’s the core
part of the thesis here. Data is facts, but in order
to do something with data, you have to start orienting your
mind to getting the right data. You have to know
how to use data. You have to weave it
into a product story. And the example I gave,
that was a brief story of a probabilistic thing
that was very hard to know. And I think that enters into
a concept I want to enter, that data cannot
tell you everything. Data is as strong as the
data itself, in this case, trying to predict the future. It’s hard unless you
have a predictive model. There’s many talks
on machine learning. That’s one way of trying
to predict, right? And another concept that I
want to enter into our lexicon is causality. Data is always interested
in a causal relationship in many ways because
that’s how you derive power from storytelling. A happened, then B happened,
or if this, then that. That’s causality, not to be
confused with correlation and hopefully never to be
confused with coincidence. And if I can throw more
alliteration at you, there are moments where we
might even talk about confidence intervals and measuring the
hypothesis with counterfactuals and so forth. I’ll explain more later. With that said, deriving
knowledge is what data is for. And I’m not going to be
necessarily a student of data here. You could derive knowledge
in a multitude of ways. But that’s the point, right? The data needs to
add up to something that’s valuable to you to know. And it’s important
to understand that on your path of acquiring data
or getting the right data. So let’s take a
look at ads and what designing for ads looks
like, because if I dive right into a project example, it might
not be so clear what an ads designer does day to day. So let me kind of give
you a brief history and then lead into
the case study so that folks can kind of
get onboarded to what it looks like to design on Search. So this is what
Search looks like. You’re seeing a
comparison, 2011, that’s when I joined Google– versus today. The devices have
changed dramatically. The screen sizes are different. And this is a
proportional ratio here. But the Search experience
has changed a ton, right? Same query, flowers. A lot’s going on differently. In fact, I’m kind of proud
of the UI on the right. My shopping team
is there in force. But that’s a lot of
dramatic change in Search. So let me kind of walk you
through how Search and Search UX kind of thinks about Search. They live by principles. I’m just going to give
you a snapshot of some of the principles. A clear information design
with beautiful typography feels clean. It’s familiar across
queries, right? It should be obvious,
but these principles need to get articulated if
a design team is actually going to do things well. And what’s interesting
about Search is that it has to
be the same thing sort of over and over for
different queries to all people that use it. It has to be a really sort
of tightly integrated system of well-designed components. And that’s very
challenging, right? Because for a sports
query, we have to be the best
sports app we can be. For a shopping query, we
have to be the best shopping app we can be, right? And Search has to serve
a myriad of user needs and that makes it a very
challenging design problem. And Search at its heart is
a query and answer system, and so to do that
well, you kind of have to show that you
understand the user. If that doesn’t happen,
then we’re losing, right? And so this is just a snapshot
of some of the principles that Search lives by and that
govern the design decisions that they make. Here is a visual of what Google
Search looked like in 2000. This is when AdWords
was launched. This is when ads started
showing up on Google. And that’s kind of where
my background starts to become relevant here. I was not at Google
then, by the way. But some interesting
data for you here. Like Search, this was like
seven years before the iPhone, so it’s largely a desktop world. Search was a 20 million queries
a day product back then. That’s a long time ago. Now, that number
is– well, last time I checked, at
least, in like 2017, that number was
about 3.5 billion. And that’s just on desktop. We’ll not kind of give
you a scaling of mobile. And that’s old data. That’s not even recent
in the two years. And just to throw in a
gratuitous but simple data visualization because this is a
talk about designing with data. That just shows you the
scale of the product. And I think that’s
indicative of the metaphor that I gave in the beginning,
sort of overhauling a 727. It’s a really big 727. It may be a 747. I’m not even sure what the
biggest plane is nowadays. But at its inception, Search
was something quite different than its competitors, right? The UI you see on the left
is largely a directory. It has Search as a feature,
but it’s asking the user to click one of many things. On Google, I don’t want to be
so prideful as to say restraint shown in design. It’s just a belief that
if we get Search right, you don’t need to click
on anything else, right? And the design
community at Google sort of believed in white space
and the ability of white space to focus the user. So this is white space not for
visual design sake, simply. It’s white space for usability. It’s an information
design technique. These are the principles
that Search has to embody, and these are the principles
and answers– sorry, ads– have to live by. Here, you’re seeing a
snapshot of Search in 2010 when we introduced top ads. And so there’s ads not
just on the right-hand side but ads on the top. Notice that there’s many ads on
the right-hand side, just a few on the top. The idea here, if Search is
a query and answer system, ads can be answers. Perhaps not all,
just the top slice that are hyper-relevant
and actually useful to the user in the
moment for the query, right? That’s when we started to
create this dynamic of OK, there are some ads on the
page, but some of them are of a quality threshold
that are much higher. How do we design for
those versus the others? When I joined
Google, just a year after those top ads showed
up, Project Kennedy happened. Hands if anyone has heard
of Project Kennedy at all. Just a few people,
so I’ll explain. This was when Larry Page,
our CEO at the time, sort of said, hey, we need to
make Google more beautiful. Like, all of the
products work, but they look like they’re all crafted
by different companies in some ways, right? And so this was the
first cross-Google sort of like an organically
built sort of design system. It’s the precursor to
material design in many ways. And this was happening
when I joined Google. And Search had a project
aptly named Manhattan, or Project Manhattan– Googlers love their tie-ins from
Kennedy to Project Manhattan– to make Search better. That was just about Search
in this world of Kennedy. So it went from
this, pre-Kennedy, to that, which I think was
a powerful design change. They refreshed the
grid, modernized the page in multiple ways,
but the ads sort of still were perhaps even more
dominant as that main column got a little bit wider. So it’s at that point that I
had one of my first projects at Google, which is, OK,
take all the ads formats and design them for this
new world of Kennedy. Not the world before,
where ads were just on the right-hand
side, but we want to express the capability of ads
to be answers, serve unique use cases, reach for sort of
bespoke design and presentation, to be worthy of being
on the top of the page. So this was what designing
for ads looked like back then. And it’s interesting. Like, it’s hard to
tell on the projectors, but there’s a yellow
background behind some of them, the ones that sort of
lived on the top ads. So a little bit
about my team now that you kind of get a
sense of the kinds of things that we were designing
back in 2011. Simple rubric for my team’s
mission in Search ads– “To design for user
expectation, comprehension, and satisfaction.” A really simple rubric. Search, users give you a query. They have a need. So we need to anticipate. We need to serve their
expectation, right? With that need, the
users are reading. Search is a textual
reading interface. Like, what is reading if
users can’t comprehend? So if we’re meeting
users’ expectation by putting the right
thing on the page, we must ensure that
they’re comprehending what we’re putting there
and so that it’s worthwhile. And that’s where the
last step comes in. We have to sort of assess
the satisfaction of that. Like, if we meet
their expectation, if they can comprehend
what we’ve put on the page, then we should
sort of understand if we’re satisfying the
need and closing that loop. A really simple rubric. Like, the interaction
design on Search is actually quite
simple fundamentally. And as our team grew and as
ads became better answers, we had multiple expressions
of what ads could be. Here, you’re seeing just
the discrete formats that we had in ads in 2015, not
to mention the fact that all of these can be combined
in a myriad of ways with one another, right? So there’s like a huge
system underneath here, and ads are just only
a fraction of Search. That system allows us to build
and imagine new use cases. So there’s a structured system. A system of, quote unquote,
sort of a “single system of well-designed components
to use our Friends in Search,” Search UX’s vocabulary. But this is kind of
what my team was doing. So I’ll just get another
visual of the structure of ads and give you a hint to the
variations in the system and how we think about
that design structure. And I’ll touch briefly
on another angle of data that really
isn’t the thesis here, like the text that
you see in the ad is something we call data,
and even that was something that we actively design. Even though an advertiser
submits that data, we ask them to submit a
certain kind of data, right? The time that this was
done, we asked advertisers to submit one kind of data. And as we designed
for use cases, we realized, hey, that
didn’t make sense. Like if user intent is sharded
across multiple things, maybe we need to ask
for a plethora of data and design specifically
for that user intent with only a slice
of the information from the advertiser. What you’re seeing here
is an interesting example of experiments we ran in
Search ads back in 2015 about, where you’re seeing a smaller ad
on the left-hand side with less interactivity, the blue links,
less information density, but higher click-through rate. 30% higher click-through rate. That is abnormal to use
less space on Search and have higher engagement. And that’s all through
serving the right data, meeting the user’s expectation,
focusing on comprehension with a structured
presentation, and we’re able to sort of understand
satisfaction and see that in the click metrics. So just to give you
an example of what it’s like to
problem-solve on Search. You guys are all quiet, though. Feel free to ask
questions at any moment. So back to the ad badge. Here’s what Search looked like
sort of 2013 after Kennedy, and here is what the ad badge
does to the Search page. Probably easier to
tell where the ad is and where it isn’t
because that badge is quite distinct versus the yellow
background that lived before. But this was a
dramatic transformation for the Search page,
and I want to walk you through how we got there. This brings me
back to my thesis. The first thing that
matters, even before a design problem like this
comes your way, is to build intuition
in an ongoing way. This comes way before any
of our design process logic. You have to build intuition and
understand where it’s coming from, what it’s informed by. That’s a level of self-awareness
that I would encourage everyone to sort of aspire to. You have a notion that maybe
I would do this this way versus that way, and
variably at Google, someone will ask
you to prove it. What’s the data? Why do you believe the thing
that you believe, right? I’m a visual designer at heart. I studied graphic design. I don’t have data for
all the things I believe. Like, some of them
are best practices. My professor told me. I don’t know, right? That’s not good enough in
an organization that trades on data-based decision making. So even when you
don’t have data, you have to understand
the intuition and keep your intuition sort
of stream-of-consciousness running. That self-awareness will
matter as data starts to make decisions for you. What’s interesting is,
this was my first intuition on the onset of this project. Like, why are the ads on
that yellow background? Like, what artist would
pick a yellow canvas? I even said to some of the
leads in the Search Ads team, even if I was painting
a sunset, and I bought a ton of yellow
paint, I would still start with a white
canvas, right? And these ads are supposed
to be our hero answers. Why– why– why yellow? And so that leads us to
a mobile Search redesign, one that we did after
Kennedy to try to make sure our mobile experience was
not a translation of desktop but was truly mobile first. That was an opportunity
where the entire design system was trying to find its
way to a new level of beauty, right? Aptly, we kept the
Kennedy theme going and decided to call
it Project Monroe. To be fair, Bouvier
and Onassis were project names that were taken. Those were our first choices. But the theme of this project
was to find a beautiful expression of Search, one that
had the iconic representation that Search had– we did not
want to devolve away from that– and one that was timeless. We didn’t want to be sort
of casual or trendy here. We wanted to find robust
ways, principled ways of designing for Search. This was what we told
ourselves– beautiful typography, responsive to touch. We wanted our branding
to be clear and present. Information design at
the paramount, as usual with Search. Search is a textual interface,
a reading interface. The prose should be natural,
smooth, not robotic, right? And that set of
principles leads me to the next point in the thesis. After you’re building
intuition about your designs, when you’re solving
design problems, you also need to
live by principles. You need to understand
these principles, clearly articulate them, before
you start gathering data. Usually when I start gathering
data, one of two things happens. The data starts to
show me something I don’t want to believe, and
then I run away from the data, or I understand
the data clearly, and I do what any good
scientist would do– they go where the data takes them. Evidence based. And when that happens, the
way-finding is not up to you anymore. It’s up to the data. And so having principles
really matters if you’re going to use
data to actually problem solve as opposed to use
data to simply learn, right? So that’s something that I
want to sort of unpack for you. What are the
principles kind of that underlined that project more
than just the five bullets that I shared before? At Google at that time, we
were busy trying to figure out, what comes after Kennedy? Like what is Google made of? The words “material
design” didn’t exist, but we were asking the question,
what material was Google made of? These are some of the visuals
in the storytelling that was circulating across
the company then. And we started to imagine that
Google was made of this quantum paper, we called it. This paper with
magical properties. Like, this is the material. This material lives in a world. It has a physics to it. It’s in an environment. That environment has lighting. It has movement. It has rules. That’s what material
was coming to be in its sort of formative days. We even asked fun
questions, like, what are the physical
properties of material? What happens if I
zoomed in on that atom? What is it doing? Does it behave in
weird, unique ways? This is the storytelling that
principles are born from. And then we started
thinking about the ink, the way text and information
gets rendered on this paper. How does that ink behave? Is it different from the
properties of the paper? And how do I
construct interfaces with this new vocabulary? It was fun to be sort of
around while the formative days of material design were
kind of coming together. And what I’m sharing
with you guys are the kinds of questions
and theory building and world building that
existed at the time. So specific to this Search
project, we’re looking at, like, subtle things, things
like, OK, those underlines, the letting, the typography
going from sort of use of bold to sort of more controlled
use of bold, right? Just in the blue titles,
not in every line of text. Subtle typography changes
kind of can go a long way. But in spirit with
that material theme, this is what we
wanted Search to be. We wanted a cardified
interface, one that actually was built of
the same material substrate that we thought
Google was built from, and one that had a new
perspective on information, organization. Hat-tip to the
designers in Search UX. This is from them. But that ads yellow
background persisted. It was tough for a visual
designer to live with, right? It’s hard to see
on the projector, but that ad up there has a
yellow background for a card. I’m like, we’ve
talked about material. We’ve talked about its physics. I didn’t know it was yellow. I thought it was white. That became sort of
an existential problem from a designer’s perspective. So how do we get
something like this? This was our first
concept of doing better, with an ad badge
that was localized– one per ad– to try to improve
clarity, improve visual design, make a better sort of
information system of Search. So at that point, specifically
for the ad badge and ad labeling problem, these
were the key principles to try to unlock that problem. And let me break down
this framing, right? It’s about an
information design ethos. Search is a reading interface. Reading is something humans
have done for a long time. There are ergonomics here. Reading is natural
and intuitive. We need to make sure that we
understand, take advantage of those natural tendencies, to
make a reading interface that’s excellent, right? Search is a
typographic interface. Typography can be beautiful. Make it beautiful. Use typography. Celebrate typography so that we
can sort of elevate the reading experience. But thirdly, this is Search,
and Search lives by metrics. It works in a certain
way, and it works well. We can’t break Search in
trying to redesign it. So that becomes sort of that
third set of principles, that you need to make
sure you fit your solution to all of these, no matter
what the data is telling you. And that’s where the dynamic
and the tension starts to build. You have key principles. Now we’re going to
embark on a journey of trying to problem solve with
that and see how data plays in. Since you guys perhaps
don’t design on Search, let me explain a little bit
about the Search metrics. There are a lot of
metrics and data about Search, much of which
I can’t talk to you about. Like revenue, right? Google’s a
revenue-earning business. Some of those metrics
overlap with revenue, right? I’m not going to
talk about those. Sometimes we try to study
detailed problems on Search, but we have to do that in
an anonymized way, right? And so there are lots of
privacy rules, revenue rules, that control data sharing. So what you’re going to
see me walk through here is the basics that help you
understand the framework for decision making on Search. The basics, largely which
are all publicly known, but this was interesting. Here is three search experiences
that were largely all alike. And each one is different. Can anyone tell which
one is Google here? Anyone want to give a guess? Someone’s got to raise a hand. Say left, right, or center. AUDIENCE: Center. JASON BLYTHE: Center? That’s a good guess. Awesome. There are a few tells, but
it’s Yahoo, Google, and Bing, going from I guess, what
now is my left to right. And so there’s a reason
they’re all the same, right? We’re all looking
at the same metrics. There is a local maxima
here, no question. And so, looking at Google,
let me kind of unpack some of the rationality
behind that local maxima. Here’s a heat map
of Search results. Some of these heat maps I
can tell are click behavior. Some of the other heat maps
are based on eye tracking. You can do a search
and probably find any number of bloggers, industry
watchers, studying Search. So this is typical. But if you look closely,
they’re all one similar kind of pattern, right? Like folks read starting
from left going to right. They read top to bottom. There is a locus of
activity in the upper left. And if you look
even more closely, you’ll see that
there is a rhythm. This is what that
rhythm looks like. We kind of call it
the scan pattern. It’s sort of an induced
reading pattern because of the nature of Search. People want to go
one result at a time, and they move to
the second result if the one that they’re looking
at or the previous result didn’t meet their needs. There’s just staccato
rhythm down the Search page. So let’s move the Search
results away and just look at that rhythm. Here’s the ads problem. Users have a rhythm and
we’re fighting that rhythm with this logic of a background
in terms of a background color. In fact, much of that
background’s value is away from the scan pattern,
away from the reading pattern. It’s off to their right,
if you will, right? And what’s interesting
is in some cases, we have two ads,
not three, or one. And so if a user gets trained
that the ad pattern goes so deep, and we
change that on them, it makes it even harder to
learn conceptually, right? And so this was
the key proposal. This is like information
design basics, right? Like, change it. Change it to something that
lives with the scan pattern. That has the staccato
rhythm of the scan pattern. Hopefully this is intuitive. It kind of was surprising
to me at the time. Like in hindsight,
it seems obvious, but this was an
industry standard, these different background
colors, if folks remember a few years ago. So this was the key insight. And it allows for flexibility. It’s better information design. If we needed to move ads around
the page for whatever reason, we could. The labeling system held up. You couldn’t do this with a
yellow background, at least not without making
Search look ugly. So, at that time, this is me
stress testing the information design principles. So the old way of doing
things was a layout variable, a color variable, another layout
variable, two text variables. So multiple strategies
to try to do one thing. As a designer, I
believe in efficiency. This doesn’t look good to me. So how about a
precision labeling system that’s part, part
layout, but needs nothing else? This was the design insight
that led to the change. And so here you see a bunch of
information design principles being deployed before
we render anything. This is the kind of legwork that
has to go into design at Google and at Google Search
if you’re going to stand the test
of the data analysis that comes on the back end. So in going between
those two things, the next point of the
thesis is storytelling. Data will come into play. We must validate
our design decisions and our design thinking, so
we have to build a hypothesis, have to tell the
stories that connect your principles to the data. If those stories don’t
exist, then folks will be talking at
different wavelengths, and you’ll have this
sort of cross-functional, cross-discipline tension of
maybe the engineers think something different
from the design team. And that’s not healthy
when we’re all trying to solve the same problem. So let’s walk through
that story a little bit. From exposition
through to closing, we want users to trust that
they can find the ads easily, efficiently, right? But there’s many
ways that we ask users to find those ads– a
yellow background, or a text label, or on the
right-hand side. So, after examining
our principles, we think we have some techniques
that allow users one strategy. Maybe this will get us to
a better accuracy, right? So let’s go gather the
data that proves that, and if that’s the
case, maybe we might have a industry
standard on our hand, or at the very
minimum, a good launch, an improvement in Search. This story happens
right at the moment where we’re trying to figure
out where is this going. This story helps
inform how we’re going to gather data because
I know what the right data is, because I know what the
outcome I need to prove, right? So all of this is the way
the design process has to sort of flex in order
to anticipate and live well with the data-driven
process that comes after. So that key moment,
that’s the one that’s really the culmination of the
design work, quite frankly. But this, the next step, that’s
where the data process lives. So let’s take a look. Well, before we
take a look, this is what the designers
are doing at the time. This is what I’m
doing at the time. Like, I’m imagining,
OK, if we can get this thing to work, what are
all the things that we can do? We can throw more
structured data in. Like, we don’t have
a yellow background. So photography and map data
all can live in these card architectures, right? This is putting the cart
before the horse, by the way, but this is kind of
natural for a designer. Like you have a new
labeling system. It enables some new
things that you could do, things that were
hard to do before. I showed you ads with, like,
video thumbnails and imagery on a yellow background. It was horrible, in my opinion. And these are the hopes
and dreams of a designer. Like, if you get
trapped here, you will get frustrated because
there is no path here until the data stuff comes. So let’s look at that process
that’s connecting these two. Like, my thesis
kind of culminates with the interweaving
of the process here. So I’ll walk through
really quickly. Perhaps this is the key
slide, and then I’ll give you some highlights
of the process after, and then we can wrap
up and perhaps open up to more questions. It’s simple. Like, you could build
tighter rubrics here. And this is not the design
process as canonically told. This is nuanced to try
to make sure designers are robust when engaging with
the data-driven side here. Hopefully this makes
sense to folks, right? Build hypotheses, experiment,
AP test, analyze the data, validate, and find
your way to launch. But before you do that,
you kind of actually have to have a design candidate,
so your design process leads into that, right? So a few of the process
details, just to give you some highlights. So this is the fun stuff, right? This is saying, OK, how many
ways could we do this ad badge? We had dozens of examples
that we looked at. This isn’t the design
execution part. This is not where
the fun part is, even though this
is where the design team spent a lot of their time. The fun part starts after
that with the experimentation, of which we ran multiple. This is the first one. In fact, the media picked
up on this first one. So it’s a simple experiment. It should be obvious what we’re
trying to understand, right? What connotes it’s an ad? It’s that yellow background
and that text label that’s in the
upper-right that’s meant to live with the yellow
background, which ironically doesn’t live with the second ad,
which makes it a poor design, in my opinion. So we’re really
trying to figure out, can you just tell where
the ads are, right? That data is, like,
not the point. The point is, you have
to understand that. You have to understand
what you’re trying to ask from the experiment. Now you have a framework
to judge the data that’s coming back to you. Live traffic experiments
are data from real users– I’m not going to give you
the data points here– but what I want to give you
is a rubric for understanding. If you don’t bring this to
the table as a designer, the data scientists will. They’ll have their rubric. And they’re not understanding
the goals of the design, not necessarily,
so you have to make that connection as a designer. This is what the media
picked up, right? “Google Testing New Look
for Mobile AdWords Ads.” And one of the quotes here,
“The new label especially pops.” It goes on to say, because it’s
not on a yellow background. Because it’s on a white
background, so it pops. This is the kind of validation
that we’re looking for, right? Validation that it’s prominent. Users are seeing it. The media is seeing it, right? So this is a positive sign. Overwhelmingly
positive sign, right? So at that point,
I think what you need to understand is, like,
there’s first-order data, which is the question you’re asking. You can ask them directly. Like, can you tell
where the ads are? There’s second-order
data and sort of third-order data or context. That second-order
data is proxies. It’s things that are hard
to know that you maybe have to collect
multiple times of data, maybe think of either formulas
or think of them as proxies for other things which
you’re trying to tell. Like, one thing that we
do in Search is like, look at what people click on. It doesn’t mean that’s
a good click, right? Maybe you are happy with
the thing you clicked on. Maybe you’re unhappy with
the thing you clicked on. So it’s a weak signal
on its own, right? And then there’s tertiary bits,
like does the design learn? So maybe they’re behaving one
way now but another way a week later. How does it work on
mobile versus desktop? How does it work in
different languages and different locations
with different cultures? I’ll give you an example
of that cultural context to kind of show you how
the understanding of data sort of flows. All these graphs
are that experiment. It’s click behavior in
different countries. I’m not showing
you the countries. I’m not showing you what’s
on the x- and y-axis, just to kind of
protect the data. But one of these graphs
looks quite different. That bottom-right
graph– downward trend. What is going on? Because all of them
are relatively flat. The movement is
largely in the noise if you try to draw
a line of best fit. So that’s the trend. It was Japan. So that’s when the context and
the principles come into play. Like, why is this happening? What’s going on? How do we understand this? Like, we’re putting a
Japanese character ad label beside a Roman character URL. That’s something we
didn’t do before. These things didn’t juxtapose. We’re putting Japanese
characters in a bounding box. They’re much more
detailed characters than Roman characters. Is that affecting reading
and comprehension, right? How might we edit the design
based on that region behaving differently? So, like, that’s
the power of data. It can highlight
nuances of your design that weren’t clear,
like a few slides ago when I was so proud of the
information design logic, right? And so that’s what
data can allow for, like, detailed
problem solving to make sure that we land good
launches for Search at scale. And then that’s just
one of many experiments. We tried a ton of
colors as well. I can’t go through all
the experiments here, but this is the nature
of design on Search. It’s building through
design execution. It’s experimentation,
with storytelling that allows us to
gather the right data, explore the relevant context,
and improve our design making. So before and after. That’s what we were
able to accomplish with this particular story. And we had a fun moment
when we launched this. Like industry watchers,
like EyeQuant blog– EyeQuant, sorry. This is their blog page. They are researchers and
data scientists and so forth and they have worked
with Google in the past– not on this project– and so they know some of
the complexities here. It was fun to have
them say this, calling it a work of genius. I think it’s quite
too flattering. And Fast Company
even picked it up. I thought this was a
fun quote from them. “Even Google’s crown
jewel, its search results, accompanied by that
money-printing machine–” ads– “have developed taste
beyond the purview of the stereotypical
Google engineer.” And this is just
for the ad badge, which was awesome because
they’re talking about the ads here. So that was a moment that I
think our team was proud of. And we even started to see some
interesting before and afters. So this is an ad with a yellow
background– hard to tell. A Yelp ad in the right. Much better design,
in my opinion, that showed up afterwards
with an ad badge. A few more examples. I don’t want to make the
attribution here too direct, but when you make a launch on
Google, lots of folks see it. So in summary, build
intuition in an ongoing way, because those
intuitions are going to be the well that you
draw your principles from. You have to have
those principles and marry those principles
with storytelling before you invest in data. Those principles,
that storytelling, will help you get
the right data. That way, data isn’t
leading you astray, and if data goes where
you don’t expect it, you should probably
follow because you’re clear on what your problem is. And then once you get
the data, analyze it. Validate. That’s how you actually
end up making decisions. So that’s my thesis on
how to interweave data and a design-driven process. So I’ll close with a quote from
Charles Eames in an interview he gave in 1972. He was asked, does
design admit constraints? And his answer was,
“Design depends largely on constraints.” And my follow-through
on that is, what data is there for
is to help you understand those constraints,
understand those constraints with precision. So if I can leave you
with a closing thought, design isn’t data, to have a
play on words in the speaker series, but quite frankly,
data can make design better with precision. But it has to be guided with
design intuition and design principles. Thank you. [APPLAUSE] KAI HALEY: Thank
you, Jason, so much. It was really great to see
this incredible work shared. And I just want to say
what a privilege it was. I’ve worked with Jason for 3
and 1/2 years on this team, and thank you so much for
helping me build my intuition as a designer in that process. So thank you. JASON BLYTHE: Thanks, everyone. [APPLAUSE] [MUSIC PLAYING]


4 thoughts on “Design Is [Data] – A case study in designing Ads on Google Search

  1. These are great however is it possible to have 10minutes version as well? Sometimes watching an hour video is not ideal.

Leave a Reply

Your email address will not be published. Required fields are marked *