Experiencing Data w/ Brian T. O’Neill (UX for AI Data Products, SAAS Analytics, Data Product Management)

116 - 10 Reasons Your Customers Don’t Make Time for Your Data Product Initiatives + A Big Update on the Data Product Leadership Community (DPLC)

Do you ever find it hard to get the requirements, problems, or needs out of your customers, stakeholders, or users when creating a data product? This week I’m coming to you solo to share reasons your stakeholders, users, or customers may not be making time for your discovery efforts. I’ve outlined 10 reasons, and delve into those in the first part of this episode. 

 

In part two, I am going to share a big update about the Data Product Leadership Community (DPLC) I’m hoping to launch in June 2023. I have created a Google Doc outlining how v1 of the community will work as well as 6 specific benefits that I hope you’ll be able to achieve in the first year of participating. However, I need your feedback to know if this is shaping up into the community you want to join. As such, at the end of this episode, I’ll ask you to head over to the Google Doc and leave a comment. To get the document link, just add your email address to the DPLC announcement list at http://designingforanalytics.com/community and you’ll get a confirmation email back with the link. 

Links

• Join the Data Product Leadership Community at designingforanalytics.com/thecommunity
• My definition of “data product” is outlined on Experiencing Data Episode 105 
• Product vs. Feature Teams by Marty Cagan
• Email Brian at brian@designingforanalytics.com.

115 - Applying a Product and UX-Driven Approach to Building Stuart’s Data Platform with Osian Jones

Today I’m chatting with Osian Jones, Head of Product for the Data Platform at Stuart. Osian describes how impact and ROI can be difficult metrics to measure in a data platform, and how the team at Stuart has sought to answer this challenge. He also reveals how user experience is intrinsically linked to adoption and the technical problems that data platforms seek to solve. Throughout our conversation, Osian shares a holistic overview of what it was like to design a data platform from scratch, the lessons he’s learned along the way, and the advice he’d give to other data product managers taking on similar projects. 

Highlights/ Skip to:

• Osian describes his role at Stuart (01:36)
• Brian and Osian explore the importance of creating an intentional user experience strategy (04:29)
• Osian explains how having a clear mission enables him to create parameters to measure product success (11:44)
• How Stuart developed the KPIs for their data platform (17:09)
• Osian gives his take on the pros and cons of how data departments are handled in regards to company oversight (21:23)
• Brian and Osian discuss how vital it is to listen to your end users rather than relying on analytics alone to measure adoption (26:50)
• Osian reveals how he and his team went about designing their platform (31:33)
• What Osian learned from building out the platform and what he would change if he had to tackle a data product like this all over again (36:34)

Quotes from Today’s Episode

• “Analytics has been treated very much as a technical problem, and very much so on the data platform side, which is more on the infrastructure and the tooling to enable analytics to take place. And so, viewing that purely as a technical problem left us at odds in a way, compared to [teams that had] a product leader, where the user was the focus [and] the user experience was very much driving a lot of what was roadmap.” — Osian Jones (03:15)
  
  
• “Whenever we get this question of what’s the impact? What’s the value? How does it impact our company top line? How does it impact our company OKRs? This is when we start to panic sometimes, as data platform leaders because that’s an answer that’s really challenging for us, simply because we are mostly enablers for analytics teams who are themselves enablers. It’s almost like there’s two different degrees away from the direct impact that your team can have.” — Osian Jones (12:45)
  
  
• “We have to start with a very clear mission. And our mission is to empower everyone to make the best data-driven decisions as fast as possible. And so, hidden within there, that’s a function of reducing time to insight, it’s also about maximizing trust and obviously minimizing costs.” — Osian Jones (13:48)
  
  
• “We can track [metrics like reliability, incidents, time to resolution, etc.], but also there is a perception aspect to that as well. We can’t underestimate the importance of listening to our users and qualitative data.” — Osian Jones (30:16)
  
  
• “These were questions that I felt that I naturally had to ask myself as a product manager. … Understanding who our users are, what they are trying to do with data and what is the current state of our data platform—so those were the three main things that I really wanted to get to the heart of, and connecting those three things together.” – Osian Jones (35:29)
  
  
• “The advice that I would give to anyone who is taking on the role of a leader of a data platform or a similar role is, you can easily get overwhelmed by just so many different use cases. And so, I would really encourage [leaders] to avoid that.” – Osian Jones (37:57)
  
  
• “Really look at your data platform from an end-user perspective and almost think of it as if you were to put the data platform on a supermarket shelf, what would that look like? And so, for each of the different components, how would you market that in a single one-liner in terms of what can this do for me?” – Osian Jones (39:22)

Links

• Stuart: https://stuart.com/
• Article on IIA: https://iianalytics.com/community/blog/how-to-build-a-data-platform-as-a-product-a-retrospective
• Experiencing Data Episode 80 with Doug Hubbard: https://designingforanalytics.com/resources/episodes/080-how-to-measure-the-impact-of-data-productsand-anything-else-with-forecasting-and-measurement-expert-doug-hubbard/
• LinkedIn: https://www.linkedin.com/in/osianllwydjones/
• Medium: https://medium.com/@osianllwyd

114 - Designing Anti-Biasing and Explainability Tools for Data Scientists Creating ML Models with Josh Noble

Today I’m chatting with Josh Noble, Principal User Researcher at TruEra. TruEra is working to improve AI quality by developing products that help data scientists and machine learning engineers improve their AI/ML models by combatting things like bias and improving explainability. Throughout our conversation, Josh—who also used to work as a Design Lead at IDEO.org—explains the unique challenges and importance of doing design and user research, even for technical users such as data scientists. He also shares tangible insights on what informs his product design strategy, the importance of measuring product success accurately, and the importance of understanding the current state of a solution when trying to improve it.

Highlights/ Skip to:

• Josh introduces himself and explains why it’s important to do design and user research work for technical tools used by data scientists (00:43)
• The work that TruEra does to mitigate bias in AI as well as their broader focus on AI quality management (05:10)
• Josh describes how user roles informed TruEra’s design their upcoming monitoring product, and the emphasis he places on iterating with users (10:24) 
• How Josh approaches striking a balance between displaying extraneous information in the tools he designs vs. removing explainability (14:28)
• Josh explains how TruEra measures product success now and how they envision that changing in the future (17:59)
• The difference Josh sees between explainability and interpretability (26:56)
• How Josh decided to go from being a designer to getting a data science degree (31:08)
• Josh gives his take on what skills are most valuable as a designer and how to develop them (36:12)

Quotes from Today’s Episode

• “We want to make machine learning better by testing it, helping people analyze it, helping people monitor models. Bias and fairness is an important part of that, as is accuracy, as is explainability, and as is more broadly AI quality.” — Josh Noble (05:13)
  
  
• “These two groups, the data scientists and the machine-learning engineer, they think quite differently about the problems that they need to solve. And they have very different toolsets. … Looking at how we can think about making a product and building tools that make sense to both of those different groups is a really important part of user experience.” – Josh Noble (09:04)
  
  
• “I’m a big advocate for iterating with users. To the degree possible, get things in front of people so they can tell you whether it works for them or not, whether it fits their expectations or not.” – Josh Noble (12:15)
  
  
• “Our goal is to get people to think about AI quality differently, not to necessarily change. We don’t want to change their performance metrics. We don’t want to make them change how they calculate something or change a workflow that works for them. We just want to get them to a place where they can bring together our four pillars and build better models and build better AI.” – Josh Noble (17:38)
  
  
• “I’ve always wanted to know what was going on underneath the design. I think it’s an important part of designing anything to understand how the thing that you are making is actually built.” – Josh Noble (31:56)
  
  
• “There’s a empathy-building exercise that comes from using these tools and understanding where they come from. I do understand the argument that some designers make. If you want to find a better way to do something, spending a ton of time in the trenches of the current way that it’s done is not always the solution, right?” – Josh Noble (36:12)
  
  
• “There’s a real empathy that you build and understanding that you build from seeing how your designs are actually implemented that makes you a better teammate. It makes you a better collaborator and ultimately, I think, makes you a better designer because of that.” – Josh Noble (36:46)
  
  
• “I would say to the non-designers who work with designers, measuring designs is not invalidating the designer. It doesn’t invalidate the craft of design. It shouldn’t be something that designers are hesitant to do. I think it’s really important to understand in a qualitative way what your design is doing and understand in a quantitative way what your design is doing.” – Josh Noble (38:18)

Links

• Truera: https://truera.com/
• Medium: https://medium.com/@fctry2

113 - Turning the Weather into an Indispensable Data Product for Businesses with Cole Swain, VP Product at tomorrow.io

Today I’m chatting with Cole Swain, VP of Product at Tomorrow.io. Tomorrow.io is an untraditional weather company that creates data products to deliver relevant business insights to their customers. Together, Cole and I explore the challenges and opportunities that come with building an untraditional data product. Cole describes some of the practical strategies he’s developed for collecting and implementing qualitative data from customers, as well as why he feels rapport-building with users is a critical skill for product managers. Cole also reveals how scientists are part of the fold when developing products at Tomorrow.io, and the impact that their product has on decision-making across multiple industries. 

Highlights/ Skip to:

• Cole describes what Tomorrow.io does (00:56)
• The types of companies that purchase Tomorrow.io and how they’re using the products (03:45)
• Cole explains how Tomorrow.io developed practical strategies for helping customers get the insights they need from their products (06:10)
• The challenges Cole has encountered trying to design a good user experience for an untraditional data product (11:08)
• Cole describes a time when a Tomorrow.io product didn’t get adopted, and how he and the team pivoted successfully (13:01)
• The impacts and outcomes of decisions made by customers using products from Tomorrow.io (15:16)
• Cole describes the value of understanding your active users and what skills and attributes he feels make a great product manager (20:11)
• Cole explains the challenges of being horizontally positioned rather than operating within an [industry] vertical (23:53)
• The different functions that are involved in developing Tomorrow.io (28:08)
• What keeps Cole up at night as the VP of Product for Tomorrow.io (33:47)
• Cole explains what he would do differently if he could come into his role from the beginning all over again (36:14)

Quotes from Today’s Episode

• “[Customers aren't] just going to listen to that objective summary and go do the action. It really has to be supplied with a tremendous amount of information around it in a concise way. ... The assumption upfront was just, if we give you a recommendation, you’ll be able to go ahead and go do that. But it’s just not the case.” – Cole Swain (13:40)
  
  
• “The first challenge is designing this product in a way that you can communicate that value really fast. Because everybody who signs up for new product, they’re very lazy at the beginning. You have to motivate them to be able to realize that, hey, this is something that you can actually harness to change the way that you operate around the weather.” – Cole Swain (11:46)
  
• “People kind of overestimate at times the validity of even just real-time data. So, how do you create an experience that’s intuitive enough to be decision support and create confidence that this tool is different for them, while still having the empathy with the user, that this is still just a forecast in itself; you have to make your own decisions around it.” – Cole Swain (12:43)
  
  
• “What we often find in weather is that the bigger decisions aren’t made in silos. People don’t feel confident to make it on their own and they require a team to be able to come in because they know the unpredictability of the scenarios and they feel that they need to be able to have partners or comrades in the situation that are in it together with them.” – Cole Swain (17:24)
  
  
• “To me, there’s two super key capabilities or strengths in being a successful product manager. It’s pattern recognition and it’s the ability to create fast rapport with a customer: in your first conversation with a customer, within five minutes of talking with them, connect with them.” – Cole Swain (22:06)
  
  
• “[It’s] not about ‘how can we deliver the best value singularly to a particular client,’ but ‘how can we recognize the patterns that rise the tide for all of our customers?’ And it might sound obvious that that’s something that you need to do, but it’s so easy to teeter into the direction of building something unique for a particular vertical.” – Cole Swain (25:41)

• “Our sales team is just always finding new use cases. And we have to continue to say no and we have to continue to be disciplined in this arena. But I’d be lying to tell you if that didn’t keep me up at night when I hear about this opportunity of this solution we could build, and I know it can be done in a matter of X amount of time. But the risk of doing that is just too high, sometimes.” – Cole Swain (35:42)

Links

• Company website: https://Tomorrow.io
• Twitter: https://twitter.com/colemswain

112 - Solving for Common Pitfalls When Developing a Data Strategy featuring Samir Sharma, CEO of datazuum

Today I’m chatting with Samir Sharma, CEO of datazuum. Samir is passionate about developing data strategies that drive business outcomes, and shares valuable insights into how problem framing and research can be done effectively from both the data and business side. Samir also provides his definition of a data strategy, and why it can be complicated to uncover whose job it is to create one. Throughout the conversation, Samir and I uncover the value of including different perspectives when implementing a data strategy and discuss solutions to various communication barriers. Of course, dashboards and data products also popped up in this episode as well! 

 

Highlights/ Skip to:

• How Samir defines a data strategy and whose job it is to create one (01:39)
• The challenges Samir sees when trying to uncover and understand a company’s existing data strategy (03:39)
• The problem with the problem statements that Samir commonly encounters (08:37)
• Samir unpacks the communication challenges that lead to negative business outcomes when developing data products (14:05)
• An example of how improving research and problem framing solved a problem for Samir’s first big client (24:33)
• How speaking in a language your users understand can open the door to more exciting and valuable projects (31:08)

Quotes from Today’s Episode

• “I don’t think business teams really care how you do it. If you can get an outcome—even if it’s quick and dirty. We’re not supposed to be doing these things for months on end. We’re supposed to be iterating quickly to start to show that result and add value and then building on top of that to show more value, more results.” — Samir Sharma (07:29)
  
  
• “Language is so important for business teams and technical teams and data teams to actually be able to speak a common language which has common business constructs. Why are organizations trying to train 20,000 people on data literacy, when they’ve got a ten-person data team? Why not just teach the ten people in the data team business language?” — Samir Sharma (10:52)

 

• “I will continuously talk about processes because there’s not enough done actually understanding processes and how data is an event that occurs when a process is kicked off. … If you don’t understand the process and how data is enabling that process, or how data is being generated and the trigger points, then you’re just building something without really understanding where I need to fit that product in or where I need to fit that workflow in.” – Samir Sharma (11:46)

 

• “But I start with asking clear questions about if I built you this dashboard, what is the decision you’re going to make off the back of it? Nine times out of ten, that question isn’t asked, if I build you this widget on this dashboard, what decision or action are you going to make or take? And how is that going to be linked back to the map that strategic objective? And if you can ask that question, you can build with purpose.” – Samir Sharma (19:27)

 

• “You show [users] a bit of value, you show them what they’ve been dying to have, you give them a little bit extra in that so they can really optimize their decisions, and suddenly, you’ve got both sides now speaking a language that is really based on business outcomes and results.” – Samir Sharma (32:38)

 

•  “If the people in that conversation are the developers on one side, the business team, and they’re starting to see a new narrative, even the developers will start to say, “Oh! Now, I know exactly why I’m doing this. Now, I know why I’m building it.” So, they’re also starting to learn about the business, about what impacts sales, and maybe how marketing then intertwines into that. It’s important that that is done, but not enough time has been taken on that approach.” – Samir Sharma (24:05)
  
  
• The thing for me is, business teams don’t know what they don’t know, right? Most of the time, they’re asking a question. If I was on the data team and I’d already built a dashboard that would [answer that question], then I haven’t built it properly in the first instance. What I’ve done is I’ve built it for the beauty and the visualization instead of the what I would class is the ugliness and impact that I need.” – Samir Sharma (17:05)

Links

• datazuum: https://datazuum.com/
• LinkedIn: https://www.linkedin.com/in/samirsharma1/

111 - Designing and Monetizing Data Products Like a Startup with Yuval Gonczarowski

Today I’m chatting with Yuval Gonczarowski, Founder & CEO of the startup, Akooda. Yuval is a self-described “socially capable nerd” who has learned how to understand and meet the needs of his customers outside of a purely data-driven lens. Yuval describes how Akooda is able to solve a universal data challenge for leaders who don’t have complete visibility into how their teams are working, and also explains why it’s important that Akooda provide those data insights without bias. Yuval and I also explore why it’s so challenging to find great product leaders and his rule for getting useful feedback from customers and stakeholders. 

 

Highlights/ Skip to:

• Yuval describes what Akooda does (00:35)
• The types of technical skills Yuval had to move away from to adopt better leadership capabilities within a startup (02:15)
• Yuval explains how Akooda solves what he sees as a universal data problem for anyone in management positions (04:15)
• How Akooda goes about designing for multiple user types (personas) (06:29)
• Yuval describes how using Akooda internally (dogfooding!) helps inform their design strategy for various use cases (09:09)
• The different strategies Akooda employs to ensure they receive honest and valuable feedback from their customers (11:08)
• Yuval explains the three sales cycles that Akooda goes through to ensure their product is properly adapted to both their buyers and the end users of their tool (15:37)
• How Yuval learned the importance of providing data-driven insights without a bias of whether the results are good or bad (18:22)
• Yuval describes his core leadership values and why he feels a product can never be simple enough (24:22)
• The biggest learnings Yuval had when building Akooda and what he’d do different if he had to start from scratch (28:18)
• Why Yuval feels being the first Head of Product that reports to a CEO is both a very difficult position to be in and a very hard hire to get right (29:16)

Quotes from Today’s Episode

• “Re: moving from a technical to product role: My first inclination would be straight up talk about the how, but that’s not necessarily my job anymore. We want to talk about the why and how does the customer perceive things, how do they look at things, how would they experience this new feature? And in a sense, [that’s] my biggest change in the way I see the world.” — Yuval Gonczarowski (03:01)
  
  

• “We are a very data-driven organization. Part of it is our DNA, my own background. When you first start a company and you’re into your first handful of customers, a lot of decisions have to be made based on gut feelings, sort of hypotheses, scenarios… I’ve lived through this pain.” — Yuval Gonczarowski (09:43)

 

• “I don’t believe I will get honest feedback from a customer if I don’t hurt their pocket. If you want honest feedback [from customers], you got to charge.” — Yuval Gonczarowski (11:38)
  
  
• “Engineering is the most expensive resource we have. Whenever we allocate engineering resources, they have to be something the customer is going to use.” – Yuval Gonczarowski (13:04)

 

• When selling a data product: “If you don’t build the right collateral and the right approach and mindset to the fact that it’s not enough when the contract is signed, it’s actually these three sales cycles of making sure that customer adoption is done properly, then you haven’t finished selling. Contract is step one, installation is step two, usage is step three. Until step three is done, haven’t really sold the product.” — Yuval Gonczarowski (16:59)

 

• “By definition, all products are too complex. And it’s always tempting to add another button, another feature, another toggle. Let’s see what we can remove to make it easier.” – Yuval Gonczarowski (26:35)

Links

• Akooda: https://akooda.co/
• Yuval’s Email: y@akooda.co
• Yuval’s LinkedIn: https://www.linkedin.com/in/goncho/

110 - CDO Spotlight: The Value and Journey of Implementing a Data Product Mindset with Sebastian Klapdor of Vista

Today I’m chatting with Dr. Sebastian Klapdor, Chief Data Officer for Vista. Sebastian has developed and grown a successful Data Product Management team at Vista, and it all began with selling his vision to the rest of the executive leadership. In this episode, Sebastian explains what that process was like and what he learned. Sebastian shares valuable insights on how he implemented a data product orientation at Vista, what makes a good data product manager, and why technology usage isn’t the only metric that matters when measuring success. He also shares what he would do differently if he had to do it all over again.

 

Highlights/ Skip to:

• How Sebastian defines a data product (01:48)
• Brian asks Sebastian about the change management process in leadership when implementing a data product approach (07:40)
• The three dimensions that Sebastian and his team measure to determine adoption success (10:22)
• Sebastian shares the financial results of Vista adopting a data product approach (12:56)
• The size and scale of the data team at Vista, and how their different roles ensure success (14:30)
• Sebastian explains how Vista created and grew a team of 35 data product managers (16:47)
• The skills Sebastian feels data product managers need to be successful at Vista (22:02)
• Sebastian describes what he would do differently if he had to implement a data product approach at a company again (29:46)

Quotes from Today’s Episode

• “You need to establish a culture, and that’s often the hardest part that takes the longest -  to treat data as an asset, and not to treat it as a byproduct, but to treat it as a product and treat it as a valuable thing.” – Sebastian Klapdor (07:56)
  
  
• “One source of data product managers is taking data professionals. So, you take data engineers, data scientists, or former analysts, and develop them into the role by coaching them [through] the product management skills from the software industry.” – Sebastian Klapdor (17:39)

 

• “We went out there and we were hiring people in the market who were experienced [Product Managers]. But we also see internal people, actually grooming and growing into all of these roles, both from these 80 folks who have been around before, but also from other areas of Vista.” – Sebastian Klapdor (20:28)

 

• “[Being a good Product Manager] comes back to the good old classics of collaborating, of being empathetic to where other people are at, their priorities, and understanding where [our] priorities fit into their bigger piece, and jointly aligning on what is valuable for Vista.” – Sebastian Klapdor (22:27)

 

• “I think there’s nothing more detrimental than saying, ‘Yeah, sure, we can deliver things, and with data, it can do everything.’ And then you disappoint people and you don’t stick to your promises. … If you don’t stick to your promise, it will hurt you.” – Sebastian Klapdor (23:04)
  
  
• “You don’t do the typical waterfall approach of solving business problems with data. You don’t do the approach that a data scientist tries to get some data, builds a model, and hands it over to data engineer who should productionize that. And then the data engineer gets back and says certain features can’t be productionized because it’s very complex to get the data on a daily basis, or in real time. By doing [this work] in a data product team, you can work actually in Agile and you’re super fast building what we call a minimum lovable product.” – Sebastian Klapdor (26:15)

• “That was the biggest learning … whom do we staff as data product managers? And what do we expect of a good data product manager? How does a career path look like? That took us a really long time to figure out.” – Sebastian Klapdor (30:18)
  
  
• “We have a big, big, big commitment that we want to start stuffing UX designers onto our [data] product teams.” - Sebastian Klapdor (21:12)

Links

• Vista: https://vista.io
• LinkedIn: https://www.linkedin.com/in/sebastianklapdor/
• Vista Blog: https://vista.io/blog

109 - The Role of Product Management and Design in Turning ML/AI into a Valuable Business with Bob Mason from Argon Ventures

Today I’m chatting with Bob Mason, Managing Partner at Argon Ventures. Bob is a VC who seeks out early-stage founders in the ML/AI space and helps them inform their go-to-market, product, and design strategies. In this episode, Bob reveals what he looks for in early-stage data and intelligence startups who are trying to leverage ML/AI. He goes on to explain why it’s important to identify what your strengths are and what you enjoy doing so you can surround yourself with the right team. Bob also shares valuable insight into how to earn trust with potential customers as an early-stage startup, how design impacts a product’s success, and his strategy for differentiating yourself and creating a valuable product outside of the ubiquitous “platform play.” 

 

Highlights/ Skip to:

• Bob explains why and how Argon Ventures focuses their investments in intelligent industry companies (00:53)
• Brian and Bob discuss the importance of prioritizing go-to-market strategy over technology (03:42)
• How Bob views the career progression from data science to product management, and the ways in which his own career has paralleled that journey (07:21)
• The role customer adoption and user experience play for Bob and the companies he invests in, both pre-investment and post-investment (11:10)
• Brian and Bob discuss the design capabilities of different teams and why Bob feels it’s something leaders need to keep top of mind (15:25)
• Bob explains his recommendation to seek out quick wins for AI companies who can’t expect customers to wait for an ROI (19:09)
• The importance Bob sees in identifying early adopters during a sales cycle for early-stage startups (21:34)
• Bob describes how being customer-centric allows start-ups to build trust, garner quick wins, and inform their product strategy (23:42)
• Bob and Brian dive into Bob’s belief that solving intrinsic business problems by vertical increases a start-up’s chance of success substantially over “the platform play” (27:29)
• Bob gives insight into product trends he believes are going to be extremely impactful in the near future (29:05)

Quotes from Today’s Episode

• “In a former life, I was a software engineer, founder, and CTO myself, so I have to watch myself to not just geek out on the technology itself because the most important element when you’re determining if you want to move forward with investment or not, is this: is there a real problem here to be solved or is this technology in search of a problem?” — Bob Mason (01:51)
  
  
• “User-centric research is really valuable, particularly at the earliest stages. If you’re just off by a degree or two, several years down the road, that can be a really material roadblock that you hit. And so, starting off on the right foot, I think is super, super valuable.” – Bob Mason (06:12)

 

• “I don’t think the technical folks in an early-stage startup absolve themselves of not being really intimately involved with their go-to-market and who they’re ultimately creating value for.” – Bob Mason (07:07)

 

• “When we’re making an investment decision, startups don’t generally have any customers, and so we don’t necessarily use the signal of long-term customer adoption as a driver for our initial investment decision. But it’s very much top of mind after investment and as we’re trying to build and bring the first version of the product to market. Being very thoughtful and mindful of sort of customer experience and long-term adoption is absolutely critical.” – Bob Mason (11:23)

 

• “If you’re a scientist, the way you’re presenting both raw data and sort of summaries of data could be quite different than if you’re working with a business analyst that’s a few years out of college with a liberal arts degree. How you interpret results and then present those results, I think, is actually a very interesting design problem.” – Bob Mason (18:40)

 

• “I think initially, a lot of early AI startups just kind of assumed that customers would be patient and let the system run, [waiting] 3, 6, 9, 12 months [to get this] magical ROI, and that’s just not how people (buyers) operate.” – Bob Mason (21:00)

 

• “Re: platform plays: Obviously, you could still create a tremendous platform that’s very broad, but we think if you focus on the business problem of that particular vertical or domain, that actually creates a really powerful wedge so you can increase your value proposition. You could always increase the breadth of a platform over time. But if you’re not solving that intrinsic problem at the very beginning, you may never get the chance to survive.” – Bob Mason (28:24)

Links

• Argon Ventures: https://argon.vc/
• LinkedIn: https://www.linkedin.com/in/robertmason/details/experience/
• Email: bob@argon.vc

108 - Google Cloud’s Bruno Aziza on What Makes a Good Customer-Obsessed Data Product Manager

Today I’m chatting with Bruno Aziza, Head of Data & Analytics at Google Cloud. Bruno leads a team of outbound product managers in charge of BigQuery, Dataproc, Dataflow and Looker and we dive deep on what Bruno looks for in terms of skills for these leaders. Bruno describes the three patterns of operational alignment he’s observed in data product management, as well as why he feels ownership and customer obsession are two of the most important qualities a good product manager can have. Bruno and I also dive into how to effectively abstract the core problem you’re solving, as well as how to determine whether a problem might be solved in a better way. 

 

Highlights / Skip to:

• Bruno introduces himself and explains how he created his “CarCast” podcast (00:45)
• Bruno describes his role at Google, the product managers he leads, and the specific Google Cloud products in his portfolio (02:36)
• What Bruno feels are the most important attributes to look for in a good data product manager (03:59)
• Bruno details how a good product manager focuses on not only the core problem, but how the problem is currently solved and whether or not that’s acceptable (07:20)
• What effective abstracting the problem looks like in Bruno’s view and why he positions product management as a way to help users move forward in their career (12:38)
• Why Bruno sees extracting value from data as the number one pain point for data teams and their respective companies (17:55)
• Bruno gives his definition of a data product (21:42)
• The three patterns Bruno has observed of operational alignment when it comes to data product management (27:57)
• Bruno explains the best practices he’s seen for cross-team goal setting and problem-framing (35:30)

 

Quotes from Today’s Episode

 

• “What’s happening in the industry is really interesting. For people that are running data teams today and listening to us, the makeup of their teams is starting to look more like what we do [in] product management.” — Bruno Aziza (04:29)
  
  
• “The problem is the problem, so focus on the problem, decompose the problem, look at the frictions that are acceptable, look at the frictions that are not acceptable, and look at how by assembling a solution, you can make it most seamless for the individual to go out and get the job done.” – Bruno Aziza (11:28)

 

• “As a product manager, yes, we’re in the business of software, but in fact, I think you’re in the career management business. Your job is to make sure that whatever your customer’s job is that you’re making it so much easier that they, in fact, get so much more done, and by doing so they will get promoted, get the next job.” – Bruno Aziza (15:41)

 

• “I think that is the task of any technology company, of any product manager that’s helping these technology companies: don’t be building a product that’s looking for a problem. Just start with the problem back and solution from that. Just make sure you understand the problem very well.” (19:52)

 

• “If you’re a data product manager today, you look at your data estate and you ask yourself, ‘What am I building to save money? When am I building to make money?’ If you can do both, that’s absolutely awesome. And so, the data product is an asset that has been built repeatedly by a team and generates value out of data.” – Bruno Aziza (23:12)

 

• “[Machine learning is] hard because multiple teams have to work together, right? You got your business analyst over here, you’ve got your data scientists over there, they’re not even the same team. And so, sometimes you’re struggling with just the human aspect of it.” (30:30)

 

• “As a data leader, an IT leader, you got to think about those soft ways to accomplish the stuff that’s binary, that’s the hard [stuff], right? I always joke, the hard stuff is the soft stuff for people like us because we think about data, we think about logic, we think, ‘Okay if it makes sense, it will be implemented.’ For most of us, getting stuff done is through people. And people are emotional, how can you express the feeling of achieving that goal in emotional value?” – Bruno Aziza (37:36)

 

Links

• As referenced by Bruno, “Good Product Manager/Bad Product Manager”: https://a16z.com/2012/06/15/good-product-managerbad-product-manager/
• LinkedIn: https://www.linkedin.com/in/brunoaziza/
• Bruno’s Medium Article on Competing Against Luck by Clayton M. Christensen: https://brunoaziza.medium.com/competing-against-luck-3daeee1c45d4
• The Data CarCast on YouTube:  https://www.youtube.com/playlist?list=PLRXGFo1urN648lrm8NOKXfrCHzvIHeYyw

107 - Tom Davenport on Data Product Management and the Impact of a Product Orientation on Enterprise Data Science and ML Initiatives

Today I’m chatting with returning guest Tom Davenport, who is a Distinguished Professor at Babson College, a Visiting Professor at Oxford, a Research Fellow at MIT, and a Senior Advisor to Deloitte’s AI practice. He is also the author of three new books (!) on AI and in this episode, we’re discussing the role of product orientation in enterprise data science teams, the skills required, what he’s seeing in the wild in terms of teams adopting this approach, and the value it can create. Back in episode 26, Tom was a guest on my show and he gave the data science/analytics industry an approximate “2 out of 10” rating in terms of its ability to generate value with data. So, naturally, I asked him for an update on that rating, and he kindly obliged. How are you all doing? Listen in to find out!

Highlights / Skip to:

• Tom provides an updated rating (between 1-10) as to how well he thinks data science and analytics teams are doing these days at creating economic value (00:44)
• Why Tom believes that “motivation is not enough for data science work” (03:06)
• Tom provides his definition of what data products are and some opinions on other industry definitions (04:22)
• How Tom views the rise of taking a product approach to data roles and why data products must be tied to value (07:55)
• Tom explains why he feels top down executive support is needed to drive a product orientation (11:51)
• Brian and Tom discuss how they feel companies should prioritize true data products versus more informal AI efforts (16:26)
• The trends Tom sees in the companies and teams that are implementing a data product orientation (19:18)
• Brian and Tom discuss the models they typically see for data teams and their key components (23:18)
• Tom explains the value and necessity of data product management (34:49)
• Tom describes his three new books (39:00)

Quotes from Today’s Episode

• “Data science in general, I think has been focused heavily on motivation to fit lines and curves to data points, and that particular motivation certainly isn’t enough in that even if you create a good model that fits the data, it doesn’t mean at all that is going to produce any economic value.” – Tom Davenport  (03:05)
  
  
• “If data scientists don’t worry about deployment, then they’re not going to be in their jobs for terribly long because they’re not providing any value to their organizations.” – Tom Davenport (13:25)
  
  
• “Product also means you got to market this thing if it’s going to be successful. You just can’t assume because it’s a brilliant algorithm with capturing a lot of area under the curve that it’s somehow going to be great for your company.” – Tom Davenport (19:04)

 

• “[PM is] a hard thing, even for people in non-technical roles, because product management has always been a sort of ‘minister without portfolio’ sort of job, and you know, influence without formal authority, where you are responsible for a lot of things happening, but the people don’t report to you, generally.” – Tom Davenport (22:03)

 

• “This collaboration between a human being making a decision and an AI system that might in some cases come up with a different decision but can’t explain itself, that’s a really tough thing to do [well].” – Tom Davenport (28:04)
  
  
• “This idea that we’re going to use externally-sourced systems for ML is not likely to succeed in many cases because, you know, those vendors didn’t work closely with everybody in your organization” – Tom Davenport (30:21)

 

• “I think it’s unlikely that [organizational gaps] are going to be successfully addressed by merging everybody together in one organization. I think that’s what product managers do is they try to address those gaps in the organization and develop a process that makes coordination at least possible, if not true, all the time.” – Tom Davenport (36:49)

Links

• Tom’s LinkedIn: https://www.linkedin.com/in/davenporttom/
• Tom’s Twitter: https://twitter.com/tdav
• All-in On AI by Thomas Davenport & Nitin Mittal, 2023
• Working With AI by Thomas Davenport & Stephen Miller, 2022
• Advanced Introduction to AI in Healthcare by Thomas Davenport, John Glaser, & Elizabeth Gardner, 2022
• Competing On Analytics by Thomas Davenport & Jeanne G. Harris, 2007