S2E16: Research, Questions, Bias, and Decisions with Carol Rossi

Carol Rossi has led research teams at NerdWallet, Kelley Blue Book, and Edmunds, where she started the research function from scratch. Carol has more than 25 years of experience in user experience, including a stint as the sole researcher at GeoCities in the late 90s. After 14 years leading teams in-house, Carol returned to consulting in 2022, this time focused on research leadership for orgs that have or want a research function but have no internal research leadership.

Carol holds Master's degrees in both Human Factors and Dance Ethnology. She has deep experience as an educator and loves to give back by regularly mentoring people interested in UX careers. She has taught both UX Research and yoga at community colleges and universities across Los Angeles.

www.carolrossi.com
https://www.linkedin.com/in/crossiux/
=====
Impactful research

• provides insights that change business outcomes
• serves the needs of a customer
• leads organizational culture change
• adds operational efficiency 

Value of research

• scrappy not crappy
  • rigorous but smart and situationally aware 
  • researchers need business acumen
    • help make the business successful by giving customers what they need
• in tech, researchers have faced a disproportionate number of layoffs
• researchers perhaps haven’t done a great job of highlighting its value
• research is meant to inform good decisions
• in academia we’re looking for ultimate truth and in industry we’re trying to get to good decisions

Decision making

• what’s usually missing is measuring the impact of decisions
• evaluating decisions informs future decisions
• what’s the thing we need to know?
  • what do we already know about that?
• identify the different sources of information you have
• bias
  • HiPPO (highest paid person’s opinion)
  • high value client’s need
  • ask question to mitigate 
  • it’s hard to see if insights are tainted and biased
  • people unintentionally introduce bias
    • asking leading questions, carrying assumptions, talking to wrong people
  • decisions are often based on biased information
    
    

Crafting good questions

• go in with learning mindset and sense of curiosity
• what are we trying to learn and what’s the best way to learn that?
• open-ended, non-leading
• we can introduce bias inadvertently by how we ask a question
• “tell me more” is a powerful question
• sometimes it may not be a questions, it m

Leadership Voyage
email: StartYourVoyage@gmail.com
youtube: https://www.youtube.com/@LeadershipVoyage
linkedin: https://www.linkedin.com/in/jasonallenwick/, https://www.linkedin.com/company/leadership-voyage-podcast/
music: by Napoleon (napbak)
https://www.fiverr.com/napbak
voice: by Ayanna Gallant
www.ayannagallantVO.com
==========

Instacart - Groceries delivered in as little as 1 hour.
Free delivery on your first order over $35.

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Transcript

1
00:00:01.740 --> 00:00:07.270
Jason Wick: Okay, welcome back everybody to another episode of leadership voyage. It's great to be with you all again.

2
00:00:07.290 --> 00:00:13.859
I am honored to be with the special guest today. Carol Rossi Carol. It's great to see you again today. How are you?

3
00:00:13.990 --> 00:00:40.899
Jason Wick: Hi? I'm well, thanks, Jason, for inviting me awesome. Well, it's good to see you. Carol you and I met. I might have been back in April or March, or something like that, and and had a nice conversation. And I think we're gonna have a really nice conversation with the microphones recording today, I think people are going to find it really interesting. We're going to hit on a lot of different topics. But to kind of let everybody get a glimpse into who you are, what you do.

4
00:00:40.960 --> 00:00:52.729
Jason Wick: When we spoke the first time. not only are you a user experience, research or consultant, but you also are trained in dance so professionally trained in tech and in dance.

5
00:00:53.350 --> 00:00:59.200
Jason Wick: Maybe you could give us a little glimpse how you ended up working in the field you're currently in today.

6
00:00:59.290 --> 00:01:21.759
Carol Rossi: Yeah. Cause those aren't necessarily related to each other. no. So yeah, I mean, I actually, my undergraduate degree was in sociology and psych. And then I uncovered this field called human factors when I was an undergraduate, and I decided to get a Master's degree in human factors. So I did that straight out of undergrad.

7
00:01:21.880 --> 00:01:34.169
Carol Rossi: And then I was. I started working in tech in my twenties, and I was studying dance as a hobby. And so, you know, ballet modern, the things that people study. And then, after a few years of working in these giant enterprise

8
00:01:34.190 --> 00:01:35.310
Carol Rossi: companies.

9
00:01:35.440 --> 00:01:56.959
Carol Rossi: I became disenchanted for actually some of the same reasons, people are disenchanted with a ux right now, and I decided to leave. And I thought, maybe what I want to do is teach, and I love this thing and dance. And so I decided to go back to school, get another master's degree in dance technology, which is an academic degree kind of like anthropology of movement.

10
00:01:57.020 --> 00:02:00.900
Carol Rossi: And then my plan was to get a Phd. In anthropology and become an academic.

11
00:02:00.910 --> 00:02:05.060
Carol Rossi: But while I was in grad school with the Ma, I fell in love with Yoga.

12
00:02:05.100 --> 00:02:10.429
Carol Rossi: and there were these opportunities. At that point Yoga was starting to become really popular. So

13
00:02:10.550 --> 00:02:22.440
Carol Rossi: there were all these opportunities to teach Yoga, and I was decided I wanted to teach at community colleges. So I taught at community colleges across La. I ended up teaching at a university for many years and starting a yoga program there

14
00:02:22.780 --> 00:02:34.220
Carol Rossi: and at the same time, I kind of kept the tech thing going by doing really like usability, consulting for a lot of clients throughout La, because I was living in Los Angeles at the time.

15
00:02:34.270 --> 00:02:42.580
Carol Rossi: So that's kind of how the 2 fields came together. And then at some point, I decided to go back into full time tech work.

16
00:02:42.660 --> 00:03:02.770
Carol Rossi: And at that point the technology degree was incredibly valuable, because that was the point in time in which everybody was working on personos and design thinking was popular. And so, because I had been in human factors before, most of my research was in more evaluative work. So the technology degree, I really learned how to do field work. I learned how to think through

17
00:03:02.830 --> 00:03:12.850
Carol Rossi: coming up with, you know, insights based on richer and more complex data. So then I I went to Kelly. Blue Book. Got a full time job there.

18
00:03:12.880 --> 00:03:25.550
Carol Rossi: worked there for a few years, went to Admins started the research team an admin stay there a long time went to nurture. So 14 years leading research teams in house. And then I started my consultancy last year.

19
00:03:26.040 --> 00:03:55.159
Jason Wick: Well, congratulations on starting the consultancy. And it's funny, isn't it? Because we all have our journeys, and you you laid yours out there and thank you for for providing all that context, I for everyone listening. And I I brought it up specifically here, just with Carol. Because, I have a music background, and it was just kind of fun at the end of our initial discussion how we started talking about dance and music, and even though you said things aren't related. We know they're kind of sort of related in some ways. They all, you know the ebbs and flows and the way that things work. But

20
00:03:55.160 --> 00:04:09.480
Jason Wick: thank you for describing all of that. We are you know, talking to a lot of different types of people, or have a lot of different listeners on the show. And we're gonna be talking a lot about your domain today. in the world of tech.

21
00:04:10.060 --> 00:04:24.750
Jason Wick: To make sure a few people who maybe aren't dialed into tech with our kind of level set. Would you mind just going through the mundane for us of defining a few things really, briefly, could you define ux

22
00:04:24.750 --> 00:04:42.179
Jason Wick: would be one. And then you just said one other term. Darn it, I was listening to you. And now I forgot ux would be good and maybe design design thinking something like that. If you could give us a couple of quick definitions of those. So that folks kinda know where we're at. As we proceed with some of the details on

23
00:04:42.460 --> 00:05:09.350
Carol Rossi: totally ux is user experience. And that's the field that's really responsible for all the things that you interact with as a user of anything. So everybody's got a smartphone. You know, most people have some kind of a computer that way that you interact with that device is the user experience, and then design thinking is a technique

24
00:05:09.350 --> 00:05:35.819
Carol Rossi: that actually without geeking out on it too much came out of systems, thinking from human factors and other fields. but it's really thinking about human-centered design. So thinking about starting with like understanding who the user is and what they need, what their unmet needs might be. And then what kinds of offerings do you give them to meet those needs. And how do you make those offerings easy for them to use? So there's a whole process that we go through

25
00:05:36.140 --> 00:05:38.370
Carol Rossi: to to do that.

26
00:05:38.580 --> 00:05:52.390
Jason Wick: Oh, that's great. Thank you. And and hopefully, everybody is is on track with us now, because we'll talk about some specific things here, Carol, about your domain of user experience research and consulting.

27
00:05:52.430 --> 00:06:02.280
Jason Wick: But I think what I'm really optimistic about in our discussion is we'll start to discuss some specifics of the process and and your experiences.

28
00:06:02.400 --> 00:06:19.629
Jason Wick: But a lot of these elements translate into other areas for people whether they can just take a slice out of this around questions or bias or decision making. I think there's a lot that can apply to a a bunch of different folks here. So thank you for helping set us up.

29
00:06:19.970 --> 00:06:29.349
Jason Wick: Okay? So your work these days, primarily, is to help organizations identify how research can have the maximal impact?

30
00:06:30.600 --> 00:06:38.000
Jason Wick: Can you start by giving us a little explanation of what does that mean? What does that process look like? And and then let's start to take it from there.

31
00:06:38.320 --> 00:06:46.870
Carol Rossi: Yeah. So maybe what I'll do is say, just a a bit about what impact means for research and then go into the process. So

32
00:06:46.890 --> 00:07:05.819
Carol Rossi: a lot of people have different ideas about what impact means. I think everybody agrees that one very important form of impact from research is that if you do a research study where you explore something like a product like if you're working on an iphone app, right? You put your app in front of people and you get feedback on it. Kind of thing

33
00:07:05.880 --> 00:07:11.330
Carol Rossi: that you want to see that any insights that you get from that research

34
00:07:11.700 --> 00:07:23.439
Carol Rossi: change the app in a way that gets the company more money, more engagement, whatever their goals are, and it serves the needs of the person using it. So that's the ideal.

35
00:07:23.470 --> 00:07:39.570
Carol Rossi: So you would see impact in terms of like the product being more popular or company making more money because people feel like they're getting what they need. So that's one type. Another type of impact is organizational, meaning within the company

36
00:07:39.600 --> 00:07:50.880
Carol Rossi: that the people that are working together are seeing more value from the research. And they're starting to like. Maybe the designers are starting to go to their research colleagues and ask

37
00:07:50.900 --> 00:08:03.829
Carol Rossi: for more information. Or what do you know about this topic or that topic. I need to know something right? That's one kind of organizational impact. Another kind of impact is operational. So

38
00:08:03.850 --> 00:08:17.009
Carol Rossi: more efficiency in the research process, because we don't want to do things in a way that's going to waste a lot of time of resources. We want to do things in a way that's going to be effective and get the answers we need without.

39
00:08:17.210 --> 00:08:43.009
Carol Rossi: You know, too much extra work. We don't need to overwork. So that's what we mean by impact in terms of the way that I work with companies. I start by investigating. I basically do like a little research project with the client and ask them a bunch of nosy questions like, what are their goals for research? So do they want research? Do they want to do research? So they get information that helps lead to business and product strategies. So it's really.

40
00:08:43.090 --> 00:08:51.170
Carol Rossi: you know, kind of leading the whole process. So they want to inform design decisions later in product development, like they already have something.

41
00:08:51.200 --> 00:09:05.609
Carol Rossi: And I just want to make sure they're on the right track. So they want to understand their core customers more deeply because they don't really know much about them. Right? So those kinds of things, what are their goals? And then do they already have researchers on staff.

42
00:09:05.610 --> 00:09:26.929
Carol Rossi: Do they want to hire researchers? If they don't, do? They have designers and product managers or engineers or other specialists doing research themselves? And if they do? Do they want that to continue? And how's that going? Because sometimes that can go really well, and other times not so do they need help making that process better. So I ask all these nosy questions.

43
00:09:27.050 --> 00:09:37.110
Carol Rossi: And then, if they don't actually know the answers to those questions already, because sometimes people don't know. because I start asking and they go. I never thought about that.

44
00:09:37.310 --> 00:09:48.880
Carol Rossi: Or if there isn't really agreement among the decision makers, the leaders that are responsible for making research happen. Then we do a half day or so workshop

45
00:09:48.920 --> 00:09:58.019
Carol Rossi: where we do that process together. You know, we outline their goals. We outline what they want to be doing, where they want to see it research in a year.

46
00:09:58.100 --> 00:09:59.200
Carol Rossi: And then we

47
00:09:59.480 --> 00:10:06.930
Carol Rossi: could talk about challenges that they see to that. And then we talk about what they can commit to doing in the next 3 months.

48
00:10:07.000 --> 00:10:21.500
Carol Rossi: and then it's it comes out in a nice little map so they can see on one page. You know all of those things, and then I print proposed to them ways that I can help them get to their first 3 month goal.

49
00:10:21.710 --> 00:10:32.520
Carol Rossi: So that might be like full partnership, like, I go in and work with them on a part time basis for some of that time, and actually do some of the work that needs to be done to set up.

50
00:10:32.550 --> 00:10:36.319
Carol Rossi: like the structure, the infrastructure, whatever needs to happen.

51
00:10:36.370 --> 00:10:54.439
Carol Rossi: Or it could be that I'm working one on one with somebody to help advise them on how they should do it. it could be that I'm doing workshops with some of the people on the team there. Like, I was saying, they might have designers doing their own research, and they need some help up leveling their research skill.

52
00:10:54.480 --> 00:11:16.450
Carol Rossi: or they might be wanting to hire a researcher, and I can help them put together the criteria they might think about to hire their first researcher and what success would look like for that person once they're on. But it just depends on what they need. So that's really the client based engagement. I also do public workshops. And so I have this workshop on prioritizing research, because

53
00:11:16.600 --> 00:11:23.369
Carol Rossi: there we usually get too many requests and and want to make sure people know how to think about

54
00:11:23.530 --> 00:11:26.769
Carol Rossi: what they really need to focus on

55
00:11:26.910 --> 00:11:38.299
Carol Rossi: and I also do some one on one coaching. Sometimes researchers will come to me, and they'll say, I really want to get a promotion. I want to work with somebody that's not my manager, or. you know, there's some specific thing that need to work on. So

56
00:11:38.670 --> 00:11:46.520
Carol Rossi: yeah, it's great to hear all the different angles and all the different roles that you can play based on what the needs are of your clients. And

57
00:11:46.620 --> 00:11:52.619
Jason Wick: you know it's like you're coming into this organization and you're trying to help them find, as we've said.

58
00:11:52.900 --> 00:12:05.040
Jason Wick: what do they want to accomplish with their research? How can they get the maximal impact and the variety of different ways that you outlined? And one thing, as we talked the first time, and that we continue to collaborate here over the last few days

59
00:12:05.170 --> 00:12:24.789
Jason Wick: we've been talking about like, you know, when I think about something that happens all the time, and businesses or management, or whatever it might be. People are making decisions all the time. I think a lot of times we're making decisions without even really being maybe conscious of it. And what I love about what you've talked through here, and what you bring to to an equation is rigor

60
00:12:25.310 --> 00:12:52.990
Jason Wick: and rigor around. What am I doing? Why am I doing it? What's the best way to to do anything in particular in this case? How you're going to you do your research. And so in this case, I would love to spin through a little bit around decision making and find the ways that we can apply the rigor that you bring to your consultancy to something that others can also apply to their lives. So it's a pretty broad thing to think through. But

61
00:12:53.320 --> 00:13:07.159
Jason Wick: we talked a little bit about the rational decision making model, and I'll quickly fly down it because I want you to talk more than me. But let's fly down it real quick. So the rational decision making model is to find the problem

62
00:13:07.360 --> 00:13:21.010
Jason Wick: identify the criteria you will use to judge possible solutions, decide how important each criterion is. Generate a list of possible alternatives, evaluate the alternatives and determine the best solution.

63
00:13:22.260 --> 00:13:23.110
Jason Wick: Now.

64
00:13:23.250 --> 00:13:31.530
Jason Wick: you already talked a little bit about like how you might work with someone in an organization if they were to consult you, and you know, kind of

65
00:13:31.810 --> 00:13:40.249
Jason Wick: overview and the big picture. But let's dive in a little a little more deeply here, I would say, into some of the specifics here of maybe decision making.

66
00:13:40.400 --> 00:13:45.950
Jason Wick: Now, one thing you pointed out, is this step 6 step model is nice. But

67
00:13:46.110 --> 00:13:53.300
Jason Wick: for anyone thinking about making decisions and taking the 6 step model, you suggested that there's something that's usually missing

68
00:13:53.360 --> 00:14:04.910
Jason Wick: in our decision making that follows all of this. And I love to hear you speak a little bit about what that that is, when people tend to neglect after they end up making decisions.

69
00:14:05.160 --> 00:14:16.629
Carol Rossi: Yeah? And it's it's something that's come up already. Right? It's I. I I really like this model. And we we don't have anything in here about measuring the impact of the solution that we came up with right? And

70
00:14:16.770 --> 00:14:37.120
Carol Rossi: and so I think that why are we making decisions? We're making decisions? Because we want to have some impact as a result of that decision. So if we don't ever go back and say, what was the impact of that decision? Then how do we know if it was a good decision, we might go through this process and there and check off the boxes. I've gone for these steps. But like.

71
00:14:37.650 --> 00:14:47.960
Carol Rossi: how do we know whether that worked? And then that can, of course, inform the next time we go through. Oh, this thing worked and that didn't. So maybe the next time we make a decision, we need to think about

72
00:14:48.130 --> 00:14:51.230
Carol Rossi: X, Y or Z. I think that

73
00:14:51.460 --> 00:15:01.680
Carol Rossi: one of the things that we seem to find ourselves one of the situation we seem to find ourselves in right now in user experience research is that. And this is debatable. It's a very hot debate right now about

74
00:15:01.790 --> 00:15:12.290
Carol Rossi: how we got to the place that we are in the industry in terms of like potentially seeing disproportionate number of lay offs over the last year impacting ux research

75
00:15:12.340 --> 00:15:17.049
Carol Rossi: relative to engineering product, even design.

76
00:15:17.270 --> 00:15:33.110
Carol Rossi: and there's one thought that perhaps we haven't articulated, measured, or articulated the impact as well as we could. And that's why I'm on this very passionate campaign to be talking about this and helping people think about

77
00:15:33.590 --> 00:15:43.930
Carol Rossi: making sure they're doing the research that's really going to have impact. And I think the other thing inherent in decision making that research. And it's so obvious. But I think we can eke out on the research itself

78
00:15:44.210 --> 00:15:50.240
Carol Rossi: and be really excited about talking about doing the research. But the reason we're doing it is so people can make good decisions.

79
00:15:50.550 --> 00:16:07.910
Carol Rossi: And so we don't want research to be an academic exercise where we're just trying to get to like. I was talking to someone the other day who outlined it. Really. Well, she said. you know in Academia we're looking for truth, and we're looking to further it. Understanding in the field and in industry, we're looking to make decisions. And I think it can be

80
00:16:08.250 --> 00:16:10.380
Carol Rossi: challenging sometimes to

81
00:16:10.960 --> 00:16:18.030
Carol Rossi: not remember that. So yeah, so I I definitely want to hit back. Go back to a few things you said. But

82
00:16:18.480 --> 00:16:35.479
Jason Wick: can you quickly give us a little more depth on that I find that a really insightful comment. And academia we're looking to discover truth, did you say? But in the application I I hate to say real life. That's so. That's a terrible way to say it. But we're looking to make a decision. Can you say that again?

83
00:16:35.570 --> 00:16:47.700
Carol Rossi: Yeah, I think that the way that I think about it, and with the way this person said it to me that makes a ton of sense to me in Academia, we're looking for ultimate truth. And we're looking to further understanding within a field.

84
00:16:47.770 --> 00:17:00.930
Carol Rossi: you know. And in industry, we're looking at decision making. We're doing research in order to make good decisions. And so we constantly have to be thinking, what is the decision that we're trying to make?

85
00:17:00.970 --> 00:17:09.810
Carol Rossi: What are we gonna do with this information. Once we have it rather than I'm going to publish this. So my peers can comment on it.

86
00:17:10.150 --> 00:17:17.260
Carol Rossi: And we can, you know debate, and they can add their research. And we can look at it.

87
00:17:17.319 --> 00:17:19.639
Carol Rossi: you know, as a exercise on its own.

88
00:17:19.750 --> 00:17:25.600
Jason Wick: Yeah, thanks for highlighting that distinction. last season. I I spoke with

89
00:17:25.710 --> 00:17:52.189
Jason Wick: Janelle Ward, who I believe that's how we met in the first place. But yeah, I mean, she specialized in in helping people make the transition between academia and industries as you're calling out. And I think it's really some light bulbs are going off for me. I mean. It's stuff I I knew, but I didn't really think about so clearly. Thank you for for calling out that kind of the distinction. Now, one thing I find interesting when it comes to

90
00:17:52.190 --> 00:17:57.080
Jason Wick: decision making kind of going through this process. We're talking through

91
00:17:57.200 --> 00:18:11.440
Jason Wick: the the sources of information from Academia. They they must be valuable right in in helping us kind of identify the validity of options or understanding a situation. Perhaps.

92
00:18:11.490 --> 00:18:24.739
Carol Rossi: Totally. I I think that you know  The ideal situation is that we look at what we're trying to uncover with research. What is the goal? What's the thing we need to know?

93
00:18:25.040 --> 00:18:31.849
Carol Rossi: And then what do we already know about that? And so we don't want to just jump in and say.

94
00:18:31.880 --> 00:18:49.189
Carol Rossi: Okay, you know, designer has a question about X, or we, we need to go. We need to uncover a new offering because we want to do something other than what we've been doing, and we don't just jump in and go. Okay, let's just start doing research before we go. What do we already know about that? And what do we already know? It could come from

95
00:18:49.410 --> 00:19:02.480
Carol Rossi: academic studies it could come from. If we are lucky enough to have a behavioral science or a data science team that's doing that kind of work. It could come from market research. It could come from analytics, it could come from customer support. It could come from anywhere.

96
00:19:02.560 --> 00:19:11.019
Carol Rossi: So I think we want to then parse out the validity and the the value that each of those sources frames.

97
00:19:11.030 --> 00:19:17.209
Carol Rossi: and we don't want to just go on corporate lore, you know or opinion.

98
00:19:17.270 --> 00:19:26.830
Carol Rossi: But we wanna if there is an opinion or there is this story that we've been telling ourselves, what's that about? Is that an assumption? Or is it based on some facts? Is it based on data that came from

99
00:19:27.130 --> 00:19:38.060
Carol Rossi: somewhere? So I think to me, academic research is another source, just like the sources that we might see in the company, or like white papers. And you know, public studies that we can call on.

100
00:19:38.600 --> 00:19:54.209
Jason Wick: No, I love that where you're identifying the different sources of information we have. we've got academic studies, our own data. you also, said Corporate Lower, which I find interesting, too. I want to ask you a little bit about

101
00:19:54.850 --> 00:20:01.820
Jason Wick:  bias. I think it's in in well, here what you know.

102
00:20:01.910 --> 00:20:11.680
Jason Wick: when people are presented with a a situation where they have a problem to solve or an opportunity to create. Or however, you want to think about that.

103
00:20:11.880 --> 00:20:19.329
Jason Wick: and we're identifying criteria and options and and assessing the validity of information and doing all of this stuff.

104
00:20:20.290 --> 00:20:23.080
Jason Wick: What? What part

105
00:20:23.170 --> 00:20:27.450
Jason Wick: can bias play in that scenario?

106
00:20:27.680 --> 00:20:28.510
Carol Rossi: Yeah.

107
00:20:28.630 --> 00:20:33.829
Carol Rossi: it's such an important topic. And it's something that we don't talk about. Enough.

108
00:20:34.000 --> 00:20:36.030
Carol Rossi: so I'll give you an example.

109
00:20:36.220 --> 00:20:43.080
Carol Rossi: so I think there are a couple of things. One, I think that there's the kind of organizational challenge

110
00:20:43.150 --> 00:20:46.889
Carol Rossi: that a lot of us have faced where

111
00:20:46.960 --> 00:20:56.949
Carol Rossi: and I think it's called hippo. I'm forgetting the acronym, but it's like the highest paid person. Yeah, so it's the like. Vp of such and such

112
00:20:57.030 --> 00:21:06.610
Carol Rossi: says, this is a truth, and people run with it. And maybe that came from some research that was done somewhere, you know.

113
00:21:06.810 --> 00:21:14.159
Carol Rossi: But I think we sometimes don't question those things. And so. or the sales team is telling us that

114
00:21:14.350 --> 00:21:22.320
Carol Rossi: the biggest clients need X. And so we should just do X for everybody, right? And so

115
00:21:22.620 --> 00:21:32.099
Carol Rossi: we're certainly not gonna put ourselves in the position to annoy the biggest client. And at the same time, does that mean we really need to do X, or does that mean there's a problem they're trying to solve.

116
00:21:32.370 --> 00:21:41.469
Carol Rossi: and there are many ways to solve it, and X is one of them, and so that and there could be a way that we can solve it for them that can also solve it for other people, you know. So I think there's like

117
00:21:41.520 --> 00:21:50.720
Carol Rossi: a level of clicking in and asking a few questions. to get at like, where is that really coming from? So that's one way to think about mitigate

118
00:21:50.880 --> 00:21:54.249
Carol Rossi: getting bias. Another thing that I think about a lot is that

119
00:21:54.970 --> 00:22:06.150
Carol Rossi: design is tangible, right? You can see whether something flows from one page to the next in terms of a flow. You can see whether the button

120
00:22:06.500 --> 00:22:29.989
Carol Rossi: is visible. You can see if the pixel line up like those are all tangible things. Code is tangible. It works, or it doesn't. You know. the final product that you're coming up with is tangible. Even if it's software, you can see it and interact with it. So our partners that work in design and engineering and product management. They're responsible for creating something that's tangible.

121
00:22:30.530 --> 00:22:39.030
Carol Rossi: The challenge with research is that it's the insights are intangible. You can't see them.

122
00:22:39.200 --> 00:22:50.090
Carol Rossi: So you can see you can interact with customers. And you can get input, and you can hear them tell stories. But the insights themselves are not as tangible. So

123
00:22:50.170 --> 00:23:07.700
Carol Rossi: it's not as easy to see if insights are tainted by bias, and it's really easy for people to inadvertently introduced bias into a study, and they may not even be thinking I'm doing a study. They might be thinking, I'm having a conversation with a customer.

124
00:23:08.110 --> 00:23:09.170
Carol Rossi: And

125
00:23:09.610 --> 00:23:22.820
Carol Rossi: then so if what but when you're having that conversation, if you're not really prepared, and you're not really engaging in a way that's going to get at the goal, you know, if you're asking leading questions

126
00:23:22.870 --> 00:23:28.120
Carol Rossi: for questions that don't address the objectives. If you let your assumptions based on

127
00:23:28.280 --> 00:23:35.559
Carol Rossi: you know what this Vp set, or whatever color your interpretation like there, all these things, if you're not talking to the right people.

128
00:23:35.960 --> 00:23:39.729
Carol Rossi: that are really the the target people for that.

129
00:23:39.760 --> 00:23:52.520
Carol Rossi: you know. Audience, the the audience for that product, you know, it's just like all these things can inadvertently introduce bias. And then you don't really know that you have a biased result. And you make decisions based on

130
00:23:53.090 --> 00:23:56.580
Carol Rossi: that biased result. And so

131
00:23:57.250 --> 00:24:01.359
Carol Rossi: it's challenging, you know. And I, this is a reason why

132
00:24:01.750 --> 00:24:20.920
Carol Rossi: researchers get very our feathers get kind of fluffed up when when we hear you know. Oh, people are doing their own research, and they don't really have guidance right so. And and I'm a big proponent of of designers and product managers getting in front of customers. And I'm also I've seen it work really. Well

133
00:24:21.250 --> 00:24:37.240
Carol Rossi: where I've done a lot of training and coaching, and people know the difference between when it's appropriate and and great and important for them to have those frontline conversations and went to partner with a researcher

134
00:24:37.400 --> 00:24:50.949
Carol Rossi: to do some of the heavier lifting. So I just think Bias is one of the reasons that it's really important for us to explore these things a little bit more deeply, and I also want to talk about rigor for 1 s. Since you mentioned that word.

135
00:24:51.610 --> 00:24:58.799
Carol Rossi: I think one of the things that I've been thinking a lot about, and and what I would say to my my nurball team a lot with scrappy, not crappy

136
00:24:59.000 --> 00:25:10.900
Carol Rossi: like. We want to do things in a way that's like as lean as possible, because we don't want to overdo it. But we can have this conversation and think through these things kind of quickly.

137
00:25:11.320 --> 00:25:22.439
Carol Rossi: and we may not need to do a big study. We might be able to do a really small study, or you know what we might be able to just go back and look at what we already know and not do fresh research at all.

138
00:25:22.470 --> 00:25:26.619
Carol Rossi: So I think that's when we start thinking about prioritization. What I think about is

139
00:25:26.880 --> 00:25:33.410
Carol Rossi:  When do we need to do the bigger thing? When do we need to the smaller thing. And when do we need to go back to the team and say.

140
00:25:33.460 --> 00:25:43.059
Carol Rossi: actually, we don't need to do any research on that. But here's what we already know. ABC used to be able to get started with that information and just save time.

141
00:25:43.890 --> 00:25:45.290
Jason Wick: So you're saying.

142
00:25:45.650 --> 00:26:00.950
Jason Wick: if I'm hearing you correctly. And then this last bit about yes, rigor. Rigor, of course, is good, but rigor in a vacuum. Perhaps we need to be situationally aware context driven with how we apply the right method to the right situation. I suppose? Right?

143
00:26:01.140 --> 00:26:16.119
Carol Rossi: Yeah, totally. And we need to have business acumen at a level that I think we necessary? Not necessarily. I don't know that we necessarily have had that level of business acumen for historical reasons. Like.

144
00:26:16.470 --> 00:26:38.500
Carol Rossi: you know, I think so. Fun. Fact, I was the sole researcher at Geo cities for a while. And we usually when I say that, you know, there are people go. Hi, I just say this website when I was 8 or whatever yeah, I was doing research there. So th, that was a very early days of the Internet, right? And so people hadn't really figured out how to monetize it yet. Advertising was starting to come up. But you know.

145
00:26:38.750 --> 00:26:40.150
Carol Rossi: And

146
00:26:40.530 --> 00:26:47.959
Carol Rossi: there was a lot of trial and error, and just sort of figuring things out in terms of how we were going to monetize. Then.

147
00:26:47.990 --> 00:27:00.650
Carol Rossi: you know, we went down this path of design, thinking like, we talked about earlier user center design that became sort of a main approach. And then we have these other things that are sort of tech, geeky, like agile and lean, and you know

148
00:27:00.680 --> 00:27:10.160
Carol Rossi: that we don't need to get into. But there are all these attempts to like. Figure out, how are we gonna make this a viable business. And so now we're at a point where

149
00:27:10.250 --> 00:27:37.339
Carol Rossi: guess what? It's like. 25 years in or more into the Internet as a business, and we know how to make it. Companies know how to make it a viable business. And so now we need to have a really high level of business acumen as researchers. So we're not coming in and saying, Let's do more studies. We're coming in and saying, Let's answer your questions, business people, so we can figure out how to make the business successful by giving the users what they need. It's not one or the other.

150
00:27:38.390 --> 00:27:53.550
Jason Wick: And, and, as you already said, I mean. you're using this research, whoever you is, you're using this research to make good decisions right for for the business that hopefully have positive impact on

151
00:27:53.580 --> 00:28:00.139
Jason Wick: organization users experience and and so on. Right?

152
00:28:01.060 --> 00:28:10.179
Jason Wick: gosh! You said so much good stuff in the bias section. I'm torn right now between going back and forging ahead. I think I do want to go back just for a second.

153
00:28:10.970 --> 00:28:22.869
Jason Wick: when we have, we're trying to make a decision. We're either strategically for a product or just as an individual. in a situation that we might be facing in our work.

154
00:28:23.330 --> 00:28:30.310
Jason Wick: you you outline some of the the impacts. or maybe caveats of of bias where.

155
00:28:30.910 --> 00:28:54.909
Jason Wick: like you said, I think it was hippo highest paid person opinion. I think it is. Yes, I'll have to look them up to make sure how we're swayed by that. You know the power dynamic and the organization already tells us that we don't want to disrupt the Vp. So we just kind of go that way. Path, least resistance, or a high value client, or or whatever that might be can over. We can over-optimize for them. So

156
00:28:55.540 --> 00:29:07.290
Jason Wick: part of the thing you said early on in in the process you follow. It's getting nosy with with nosy questions. And I'm kind of curious if there's a relationship here that we can help others with

157
00:29:08.130 --> 00:29:25.629
Jason Wick: reducing bias, getting the right information, whatever it means. How do we craft a good question? I know that can be answered in so many different ways. But I would love your take, Carol. How can someone listening out there. What can they go about? What steps to craft? Good questions?

158
00:29:26.180 --> 00:29:27.070
Carol Rossi: Yeah.

159
00:29:28.040 --> 00:29:29.890
yeah, it's

160
00:29:31.450 --> 00:29:38.440
Carol Rossi: I think the first thing is to go in with a sense of curiosity and a in a learning mindset.

161
00:29:38.550 --> 00:29:39.620
Carol Rossi: And

162
00:29:39.790 --> 00:29:57.629
Carol Rossi: because we can craft a perfect question. But if we're not open to what the answer really is. Then that's not gonna help. Right? So I think one, the first thing is go in with a sense of curiosity and a learning mindset. And then the next thing is, go back to the goal. What are we trying to learn?

163
00:29:57.970 --> 00:30:01.939
Carol Rossi: And what's the best way to learn that? And so

164
00:30:02.010 --> 00:30:13.320
Carol Rossi: if what we're trying to learn is, what are the unmet needs of our customer target customer? Right? What are the things that they can't necessarily tell you that could

165
00:30:13.420 --> 00:30:19.879
Carol Rossi: the like. There's a whole in in something that they're trying to do. Right? So like, for example,

166
00:30:20.680 --> 00:30:24.810
Carol Rossi: okay, this is where we're going to have to pause, because I want to come up with a good one.

167
00:30:25.100 --> 00:30:32.010
Carol Rossi:  So, for example, if

168
00:30:32.800 --> 00:30:35.140
Carol Rossi: I'm trying well, I'm going to use the.

169
00:30:35.300 --> 00:30:45.829
Carol Rossi: We may cut this Jason. Okay, I'm going to use the steep job saying, because I just read this yesterday somebody posted something on Linkedin that said, You know, we don't need to do a customer research, because people can't tell you what they want.

170
00:30:46.150 --> 00:30:51.560
Carol Rossi: And you know, all the researchers were like. And I thought.

171
00:30:51.670 --> 00:30:54.819
Carol Rossi: Yeah, they they can't tell you what they want.

172
00:30:55.850 --> 00:31:21.399
Carol Rossi: That's why we need to watch them. So sometimes when we go in and we look at our goal and we go. Our goal is to uncover unmet needs. Maybe we don't ask them questions. Maybe we observe what they're doing, or we ask them what they're doing, and we ask them how they're solving for it now, and we see where the holes are right. so I think one way to think about crafting questions is, maybe it's not even a question. Maybe it's an observation.

173
00:31:21.650 --> 00:31:28.340
Carol Rossi: And so if we go into the sense of curiosity and then we go in with the goal, what's the thing we're trying to cover. And then we figure out how to get

174
00:31:28.650 --> 00:31:34.660
Carol Rossi: to the right question. Often when we talk about good questions, we talk about good questions being open, ended.

175
00:31:34.800 --> 00:31:49.899
Carol Rossi: non-leading questions. So, for example, if I have a product that I'm trying to get feedback on, I might ask people like, how might you learn about retirement planning on this? You know financial app, right? As opposed to

176
00:31:50.320 --> 00:31:59.390
Carol Rossi: you know, something that's really specific and leading them down a path like Click on that button and tell me what you see. You know.

177
00:31:59.440 --> 00:32:04.940
Carol Rossi: or what's the purpose of this page might be another kind of open, ended question

178
00:32:05.020 --> 00:32:07.560
Carol Rossi: as opposed to

179
00:32:07.660 --> 00:32:10.979
Carol Rossi: Do you see that this page is trying to tell you

180
00:32:11.000 --> 00:32:18.329
Carol Rossi: about retirement planning? You know what I mean? It's like. And that's where we start introducing bias. By the way, we're asking the question.

181
00:32:18.630 --> 00:32:28.670
Carol Rossi: inadvertently, because we don't realize if I say, do you realize that this page is trying to tell you about retirement planning are not aware necessarily that that's a leading question.

182
00:32:28.970 --> 00:32:33.549
Carol Rossi: and that I'm like putting this thought into the person's head. I'm priming them, so to speak.

183
00:32:33.590 --> 00:32:50.739
Jason Wick: Well, and what I kind of love to is you. You kind of just gave me a dose of of medicine already, because my question, my question was, You know, how do you craft a good question? But what I actually really love about what you just said was.

184
00:32:50.790 --> 00:32:51.880
Jason Wick: Well.

185
00:32:52.260 --> 00:32:57.980
Jason Wick: yes, there are practices on how to craft good questions, but sometimes

186
00:32:58.870 --> 00:33:05.870
Jason Wick: it's not even the question. I think you were saying it's really, how do you procure the best information?

187
00:33:05.900 --> 00:33:08.949
Carol Rossi: And that doesn't always involve

188
00:33:09.090 --> 00:33:17.840
Jason Wick: asking the question like, like, you're just saying. And I actually just kinda love that that you're like, well, okay, the question answer to your question is this.

189
00:33:17.870 --> 00:33:33.259
Jason Wick: but sometimes the question isn't even necessarily the best way to get the information. You bring up observations and whatnot. But if you're going to write, going to ask the question open-ended, try not to be leading, etc., etc. All good practices.

190
00:33:33.420 --> 00:33:37.440
Jason Wick: anything else. You're thinking about that topic. I thought that was a nice spin on it.

191
00:33:37.610 --> 00:33:44.790
Carol Rossi: Yeah, I think the only other thing that comes up a lot when I'm What I'm helping

192
00:33:45.060 --> 00:34:05.159
Carol Rossi: people, you know, learn how to do. Research for themselves is like, what about follow up like they want to follow up, and they don't know what to do right? And there's this classic kind of tell me more about that, you know, which is like a therapist question. And it works really, really well in user experience research, too. Because

193
00:34:05.160 --> 00:34:21.010
Carol Rossi: if somebody is, you know, staring at the phone, and they say, Well, I think for retirement planning. I would do this with that. And then you are the person doing the research. And you're like, that's not really what I'm looking for. But you know. If you ask, tell me what more about that? Right? They'll go on

194
00:34:21.460 --> 00:34:37.540
Carol Rossi: And so I think that's the thing. The other thing that I was advised people on is that if you see someone squinting or like making a face, or they go you know, like there are things that you want to follow up on, and that's a Oh, I noticed your expression change. Tell me more about that, you know.

195
00:34:38.010 --> 00:34:52.579
Jason Wick: No, that's great thanks for adding that on, and and also just what I really loved you. You you opened up the answer to my initial question about asking good questions by saying, it's just also really important that you have the right mindset

196
00:34:52.659 --> 00:34:56.289
Carol Rossi: to hear the answer, or in this case

197
00:34:56.510 --> 00:34:58.460
Jason Wick: be open to

198
00:34:58.470 --> 00:35:12.150
Jason Wick: 3D. Listening, or 360 degree listening. Or, however, people want to characterize that global listening so that you're also watching, you're paying attention to the to the non verbal pieces as well. I love that element to add onto it.

199
00:35:12.890 --> 00:35:15.279
Jason Wick: Well, I I'm sorry. Go ahead. Yup.

200
00:35:15.900 --> 00:35:31.869
Carol Rossi: 

201
00:35:32.180 --> 00:35:36.760
Carol Rossi: 

202
00:35:36.830 --> 00:35:38.150
Jason Wick: 

203
00:35:38.410 --> 00:35:44.030
Carol Rossi: 

204
00:35:44.420 --> 00:35:49.649
Carol Rossi: the yeah, the the quick recap, I think is, you know. Just

205
00:35:49.730 --> 00:36:00.259
Carol Rossi: listen, observe, and if you don't know what to say, either, just maybe don't say anything and wait and see what happens, or you can always do the tell me more about that thing.

206
00:36:00.580 --> 00:36:04.869
Jason Wick: Thank you. Great tips. Carol really appreciate that. No others can apply that.

207
00:36:04.970 --> 00:36:18.500
Jason Wick: Probably right today a lot of situations for that for everybody. I can't believe we've already been talking for this long, but we we have to start wrapping things up. I want to ask you. I'm I'm really interested to hear the answer.

208
00:36:18.730 --> 00:36:25.139
Jason Wick: the question I ask everybody at the conclusion of every episode is, what is something that you've learned recently.

209
00:36:27.800 --> 00:36:41.789
Carol Rossi: 

210
00:36:42.020 --> 00:36:49.820
Carol Rossi: 

211
00:36:49.940 --> 00:36:54.260
Carol Rossi:  

212
00:36:54.320 --> 00:37:21.780
Carol Rossi: what I learned. You know, some people have a uniform because I don't want to have to think about what they wear every day. So there was the the Black Turtle. Next thing I don't do that, but I eat the same breakfast during the week, because I don't want to have to think about breakfast, and I need protein, and I need grain and blah blah blah. So I've been making this granola for ever, and I just finally got tired of the same Branola after I don't know how long. So I found a new granola recipe, and I'm very excited about it. So that's how I've learned how to make a different kind of granola.

213
00:37:21.880 --> 00:37:34.020
Jason Wick: That's very cool. I love it. I do a California answer to that. Does that sound like a California answer which I love? I I am wondering if you can try to get get close to the answer

214
00:37:34.080 --> 00:37:38.019
Jason Wick: for about how long do you think you were eating that same breakfast?

215
00:37:39.260 --> 00:37:53.270
Carol Rossi: Well, I've been in San Francisco 5 years, and I'm not sure I changed the recipe during the weekends. You mix it up right during the week. Yeah, there are like maybe one or 2 alternatives that come in. But it's probably been. That's great.

216
00:37:53.440 --> 00:37:58.870
Carol Rossi: A little bit of time. I love being funny.

217
00:37:59.070 --> 00:38:03.609
Carol Rossi: Get your same breakfast every week day. It frees up all this brain power right?

218
00:38:03.670 --> 00:38:14.980
Carol Rossi: No, no decision. Fatigue. In the morning you hit the ground running. Don't have to think about what to make that I love it.

219
00:38:15.370 --> 00:38:16.460
Carol Rossi: Well.

220
00:38:16.720 --> 00:38:30.920
Jason Wick: Carol, it's been my pleasure to talk to you today. for those who are interested in in the services you offer. Wanna learn more about some of the courses that you have Just learn more about you individually. where would we like to direct them.

221
00:38:31.370 --> 00:38:38.700
Carol Rossi: Yeah, they can look for me on Linkedin or at my website, which is just by name Carol rossi.com

222
00:38:38.770 --> 00:38:42.120
Jason Wick: perfect, and all that will be in the show notes for people to check out.

223
00:38:42.250 --> 00:38:51.769
Carol Rossi: Carol. Great to talk to you today. Thank you so much for being generous with your time, and have a good rest of your day. I appreciate it. You, too. Thanks so much. It's been really fun.

224
00:38:52.420 --> 00:38:53.309
Carol Rossi: Take care.