AI and the Future of Work: Artificial Intelligence in the Workplace, Business, Ethics, HR, and IT for AI Enthusiasts, Leaders and Academics

Karl Mosgofian, CIO of unicorn Gainsight, shares his journey to $200M ARR and how AI is helping the organization grow

• Karl Mosgofian • Season 3 • Episode 27

Share episode

We’ve interviewed some legendary CIOs including Mark Settle from Okta (a repeat guest), Reza Nazeman from SAP Concur, and, more recently, Carter Busse from Workato.

We’re joined by another unicorn CIO today, Karl Mosgofian. Karl has helped grow Gainsight to more than $200M ARR and 1,200 employees. He has been leading the IT organization for nearly six years after having spent time at Harmonic, Apple, and Cadence Design.

Thanks to friend of the podcast Carter Busse for the intro to Karl.

Listen and learn...

  1. How Karl's role has changed since he joined Gainsight as a startup six years ago
  2. Why it's hard for CIOs to "just keep the lights on"
  3. How Karl navigates the duel role of enabling the business to innovate with technology while making sure teams stay focused on solving business problems
  4. How Karl formulated the Gainsight employee ChatGPT policy
  5. Why ChatGPT won't replace the help desk
  6. Karl's advice to vendors embedding AI in their products
  7. How Karl partners with his CISO and legal team to establish policies for LLM usage
  8. How Gainsight is using AI internally to improve productivity
  9. All about the quirky culture at Gainsight

References in this episode...

Speaker 1:

I want to make sure we have a sandbox internally that we control. I actually would like to get as much as possible off of the public chat GPT, And what I tell people is look, if you want to play, by all means play right. Have it write a song for you. Sure, Don't give it any information.

Speaker 2:

Good morning, good afternoon or good evening, depending on where you're listening. Welcome back to AI in the Future of Work. Thanks again for making this one of the most downloaded podcasts about the Future of Work. If you enjoy what we do, please like, comment and share in your favorite podcast app And we'll keep sharing amazing conversations like the one we have for today.

Speaker 2:

I'm your host, dan Turchin, ceo of PeopleRain, the AI platform for IT and HR employee service. I'm also an investor and an advisor to more than 30 AI-first companies And, as you know, a firm believer in the power of technology to make humans better. If you're passionate about changing the world with AI, or maybe just ready for your next adventure, let's talk, and each week we learn from thought leaders in the community. And the added bonus, of course, is yet one AI fun fact. Today's fun fact OpenAI recently published its GPT-4 technical report, which includes important ethical benchmarks and, more important, a small window into how it's evaluating itself. Here are a few self-reported metrics that are not surprising, given OpenAI is, let's say, grading its own homework. Openai says we've decreased the model's tendency to respond to requests for disallowed content by 82% compared to GPT-3.5. And GPT-4 responds to sensitive requests in accordance with our policy, the OpenAI's policies 29% more often versus GPT-3.5. On the real toxicity prompts data set, gpt-4 produces toxic generations only 0.73% of the time, while GPT-3.5 generates toxic content 6.48% of the time. As an industry, we should formalize reporting requirements and standardize the benchmarking process for all publicly available large language models. I applaud OpenAI for this starting point, but we can do better as an industry. As always, i'll link to the full article in today's show notes.

Speaker 2:

Now, shifting to this week's conversation, we've interviewed some legendary CIOs on this podcast, including personal friend Mark Settle, most recently from Okta, seven-time serial CIO. We interviewed Reza Nazamon from SAP Concur and, more recently, carter Bussey from Rokado. Today we're joined by another unicorn CIO, carl Moskofian. Carl has helped grow Gainsight for more than $200 million in ARR and more than 1200 employees. He has been leading the ITE organization there for nearly six years. After having spent time at Harmonic, apple and Cadence Design. You've heard me say it's harder than ever to be a CIO in a tech company.

Speaker 2:

Let's jump in and get a fresh perspective from one of the greats. Thanks to former guests and friend of the podcast, Carter Bussey, for the intro to Carl. Without further ado, Carl, my pleasure to welcome you to the podcast. Let's get started by having you share a bit more about your background.

Speaker 1:

Yeah, i'm super happy to be here and love the podcast, so it's kind of exciting to get to be on a podcast I actually listen to and enjoy. So my background I really started as a computer hobbyist as a kid back in the old days And so my path was maybe a little bit different in that I was largely self-taught and started a consulting company when I was quite young and went out and built a bunch of custom business applications for people, kind of learned a lot on the job or from mentors and also learned by having my own business, which is great, right, because I was everything. I was the programmer, the business analyst, the project manager, i was also sales and marketing. I was also doing my own books, right. So if you want to learn business process, it's a great way to do it is to actually start a business.

Speaker 1:

And then I came into the corporate IT world and sort of worked my way up the ladder. But I've always kind of gone back and forth between operational jobs and maybe more strategic ones. I was running a big enterprise applications group and then reinvented myself as an enterprise architect. Then I did some more operational stuff. Then I went to Apple and actually did some very strategic consulting with them. So I've kind of gone back and forth, which I enjoy, and it's kind of one of the best things about being a CIO is it does give you that opportunity to have a little bit of both worlds.

Speaker 2:

So six years at a unicorn like Gainsight. I'm curious to know what's a typical day in your life today, but then, equally important, what was it like when Gainsight was early and you weren't a rocket ride and it wasn't 1200 employees? what did you do then versus what do you do now?

Speaker 1:

Yeah, it's so different, right, And I reflect on having a handful of people you know, when I started to having a fairly large team now, and you know so it's massively different In some ways. And in some ways it's all the same right, because the challenges day to day are everything's always changing in the business, right? Certainly, especially in a business like Gainsight, where we're startup, we acquire companies, we build out new products, we go to market differently, and each of those things has all sorts of impacts to our technology. And then, oh, by the way, we had a pandemic that totally changed how we needed to actually work together right along the way. So there's always something changing that we've got to respond to and figure out how technology can help the company be successful with. But it's also, i would say, probably more operational than we would like.

Speaker 1:

So, i mean, i think CIOs all want to spend more time in that strategic realm, but the reality is, you know, sometimes, in a sort of denigrating way, i think, people talk about keeping the lights on, so they say, well, let's see, i just need to just keep the lights on. But that's like, who cares about that? We want, we want the strategic stuff, but, you know, keeping the lights on is actually pretty hard And I don't think it gets enough sort of appreciation or respect, and when the lights go off it's a big problem. So you know a lot of our time is spent on quote unquote keeping the lights on, but I think that's a bigger job than people realize or appreciate.

Speaker 2:

I've said before on the show that being a CIO in a technology company is like the hardest possible job, because everybody knows how to do your job better than you do. Is that the case at Gainstite, or how do you kind of navigate that challenge?

Speaker 1:

Yeah, i mean certainly when you're working with engineering teams, right, they're like software engineers and they are really smart and they probably do know a lot of things that I don't know. But I find that this is where I think the culture really helps. So Gainstite is such a values and culture driven company that I feel like the internal collaboration is really excellent And the ability of people to sort of leave their ego at the door and just work together to find the right answer is really high here. I've worked in much more dysfunctional organizations and this is a highly functional organization, you know, and so it's really.

Speaker 1:

I don't find that I have a ton of issues with that, but you know, there's always some of that, and of course, nowadays it doesn't have to even be engineers, right? Anybody can whip their credit card out and go, fire up some piece of technology and start using it, and they may or may not fully understand what they're getting themselves into. So, you know, part of just the natural cycle of things is that, especially at startups, people run around and they get a bunch of stuff And then eventually it breaks right, and it either breaks individually or it doesn't connect well with everything else, and you know that's. That's when they're like Well, maybe this IT thing's not such a bad idea after all. You know, i kind of thought I wanted to own this, but actually maybe you could open it now.

Speaker 2:

I've seen the evolution of some CEOs who are at companies that grow big fast, become kind of internal KTLO. keep the lights on kind of operators to at some point almost being product evangelists, where they're asked to oftentimes showcase in a technology company how they're using the tools in house. Give us a sense of how much your time are you spending on internal focused initiatives? and then how much is it okay for the business to rely on you to be an advocate for gain sites, products in sales situations?

Speaker 1:

Yeah, i love getting to get out of the you know, the virtual office and out and actually meeting directly with customers. So far it hasn't been a huge impact, like I haven't had a problem for balancing both of those roles. But I think part of that is that the actual amount of time that it takes for me to meet with customers and do those kind of things It's not that many hours in a week, and you know I mean, in other words, it's it may be very impactful, but it's not. It's not actually 20 hours a week or something. I actually would kind of love it if it was. In other words, i would love to be adding even more value externally, but you know it's for our product. We don't. We sell primarily to business owners And we're spending more and more time with CIOs because I think CIOs are starting to understand more and more that customer success is not really just a departmental solution but it's really an enterprise solution And more and more folks are having that sort of evolution where maybe the CCO brought in the insight But after a few years it's like actually this is really getting to be something that multiple departments are using.

Speaker 1:

It's very tied to our data. It would actually be better if IT was managing this. So we're seeing more of that, but it's still a little bit different than something like our friend Carter, where, you know, ricardo is selling directly to CIOs every day And so for him it's a bigger part of his life. But I love it and I'd love to do more.

Speaker 2:

So presumably GainSight. The customer success platform is like the biggest, most sophisticated user of GainSight. Talk about how you use it internally and then how that translates into what you're able to talk about when you evangelize GainSight to prospects.

Speaker 1:

Yeah, absolutely, it's huge. And the big thing everyone's talking about now is digital customer success. Right, and especially this year, as budgets have been cut and people are really taking even closer look at how they not just deliver great customer success but do it efficiently. You know, that's something that we've kind of pioneered internally for a long time, and so we've really tried to automate the heck out of just about everything we do. And you're right, our implementation of GainSight is really complicated, but we've seen some great results from it as well, and so you know, we feel like it's a great model for a lot of companies in how to really fully use these technologies.

Speaker 1:

You know, one of the things that happens a lot and I see it as a customer of lots of different SaaS software is that there's sort of a quick and dirty version right. There's a simplistic way you can use it, and then there's a more sophisticated, deeper way, and when you put that effort into the development and management of the platform right, you've actually got some good administrative people and some smart people who can help figure out how to really, in working with the vendor often, how do we get the most out of this? It's kind of like night and day. I mean the value you can get out of a platform that you're really getting the full use of is tremendous, and GainSight is definitely that kind of platform that you know. If you really figure out how to embed it in all of your processes, you'll get tremendous results.

Speaker 2:

So there are a lot of parallels between what your customers do with GainSight and how you deliver service to internal employees. Talk about how you measure the quality of employee experience and maybe how that does or doesn't relate to how your customers use GainSight.

Speaker 1:

Yeah, you know, in some ways I think it's almost easier from a product perspective to measure, measuring internal satisfaction or also effectiveness, right? productivity, i find, is a real challenge, right? So, for instance, gainsight has the ability to embed product analytics into your website, but I can't do that for the 100 vendors that I have. They have to do that right, and so I don't have as much information as I'd like about adoption, what people are really doing. I have some tools that help me a little bit with that.

Speaker 1:

I have surveys of internal users, but you know there's a lot of survey fatigue out there, so there's only so many times you can have a pop up that asks someone those questions before they just start ignoring all of them. So, honestly, to me it's a frustration that I don't feel like I can measure the quality of employee experience as well as I would like to, but a lot of it is anecdotal, right. A lot of it, honestly, is people coming to me and complaining And at the end of the day, you know, maybe that's, maybe that's just a significant part of keeping your finger on the pulse of what's actually happening within your company.

Speaker 2:

Carl, i'm going to limb and say the last couple months you've been inundated to request from the business to do various things with large language models or generative AI, chat, gpt, etc. As a CIO, you need to be a steward of doing these things responsibly, but also you know your customers are your employees and presumably you want to enable them with technology. Just talk us through that process and maybe, to the extent you have a policy around generative AI. what was the genesis of it?

Speaker 1:

Yeah, you know, when this thing first really hit, the biggest concern I had was IP leakage. Right, it was very easy to go to the public chat GPT and ask it to answer some questions, but in the process of that, give it all sorts of secret information that you really shouldn't give it. Happily, you know they now have the ability to say don't use my data to train, but that you're still counting on users to know that, right, so you can communicate. But you'd be amazed how many times IT can send a message out and still have people completely ignore it. So it's tough. So one of the things that I've really zeroed in on is that I wanna make sure we have a sandbox internally that we control. I actually would like to get as much as possible off of the public chat GPT And what I tell people is look, if you wanna play, by all means play right, have it, write a song for you, sure, but don't give it any information. So if you want to give it information, then get in the sandbox, right, and we're paying for. You know commercial environment. We manage the settings. You know we've told it not to train once and now we don't have to worry about each individual user remembering to do that. You know that's been the scariest thing, but the other thing is. So one of the things we spend a lot of time in IT doing is people come to us and say I want this thing And we say that's great, but can you back up and tell me what you're trying to accomplish and let us help you find the best solution Right? Sometimes the solution is technology Might or might not be the thing you were looking at. Sometimes the solution is process right. So let's step back and think about solving problems As opposed to just grabbing some cool looking technology. This is a space where I think that's especially true, because everybody's super excited and they should be. You know, i mean honestly. You know.

Speaker 1:

The writer, arthur C Clark, said a long time ago any sufficiently advanced technology is indistinguishable for magic, and chatGPT is one of the most magical things I've ever seen. And people see it and they're blown away. They get very excited. But I think there's a couple of concerns there. One is I think they can overestimate just what it's capable of and underestimate its weaknesses. Right, in terms of both the data that has available to it. On chatGPT, the hallucination issues things like that, right, but also it can become the answer to everything, and I think that's always a big danger, and so I'm trying to be kind of an evangelist within the company to say listen, ai is really interesting and important. Gpt is a very interesting breakthrough. However, let's understand what the right tool for the job is, in particular, in a B2B, in a business context, when you're not just fooling around.

Speaker 1:

Actually think that the applications for GPT are quite limited, because I think that any kind of real business problem you're trying to solve requires some domain expertise. So a general AI model that doesn't know anything about the particular subject that you're talking about is gonna have real limitations. So the obvious example of that is a lot of people are excited now and they see chatGPT and they say, wow, this is gonna transform the help desk right, because people are gonna interact with this thing. I'm saying, first of all, people are more organized on this stuff for a long time and there's some very good technology out there. Secondly, those technologies are good because they've been trained specifically in this domain.

Speaker 1:

They understand what certain words mean, they understand some context, and I think when you don't have that, if I just ask chatGPT how to fix my printer, it doesn't know what to do. It doesn't know where to go, it doesn't know anything, and I might get some interesting generic answer. Also, every once in a while, it might tell me that I should throw my printer out the window because it's hallucinating. So I don't know that. I really wanna turn my help desk over to chatGPT, and so I think that, as a CIO, i'm really looking to my vendors to bake in the appropriate technology to their products with context, with a semantic layer, right. That can really add business value as opposed to. Honestly, chatgpt is really interesting, but a lot of the stuff that it does is a bit of a party trick, right. I mean, it's really cool, but it's not necessarily accomplishing a business task very well.

Speaker 2:

Carl, when we were chatting before we started recording, i was saying I think one of the important roles that this podcast serves is helping technologists ask the question what could go wrong? In addition to the question we're really good at asking, which is what could go right? And two of the recurring themes that we've been talking about with respect to generative AI are one it's perfectly designed to replicate human bias, so it will often say things that are hurtful, inappropriate or just racist bias. And then two, all of the content that it's using to generate new content is owned by someone else, and so we often don't think about the ramifications of stealing someone else's content because it's so opaque where it came from. I'm wondering, within GainSight, do you have a CISO counterpart or legal, or is there like a? how are you managing that kind of check and balance when, let's say, marketing maybe only, or sales only, thinking about what could go right? But oftentimes there's a need to have a little bit more nuanced conversation. Do you have thought partners in the business that help you with that?

Speaker 1:

Absolutely, absolutely. It's a great partnership. Our general counsel is also our privacy officer And so he's very engaged in this, as is our CISO, and so the three of us really are communicating a lot right now to try and stand top of this over the last few months, to try and get smarter, share information with each other and really think through this, to publish some guidelines internally, to do things like set up the sandbox and things like that. So, yeah, i think it's really important to think about what your sort of governance approach is going to be for this And what is that virtual team that you're going to form, what is the structure the committee or the team or whatever you call it And who are the stakeholders, because I also think it's important to cast a bit of a wide net and make sure that all the folks who have a piece of this have a seat at the table and are communicating together and you don't have a group and engineering going off doing stuff but they're not really talking to the privacy team or vice versa.

Speaker 1:

The privacy team is off coming up with what they think are reasonable guidelines and then they roll them out and other people are like, hey, that doesn't work, that's not practical. So I think having everybody in the same Slack channel, so to speak, so that everybody is just very open and transparent and everybody knows what's going on, is super important right now, especially because things are moving so fast and we're all under so much pressure to do something. There's that sense that this is a new thing. We need to go do something, and that's great, it's good to have that energy, but you want to make sure that it doesn't lead you down a path that later you're kicking yourself. Now there's an article about some data leakage from you that you're really is creating reputational damage to you and that you're really wishing wasn't there.

Speaker 2:

Any examples of AI first applications that you've deployed so far and any indications of how that's working out?

Speaker 1:

Honestly, there's a few things. We're using some technology that we use to answer security questionnaires, and it matches people's questions with a set of standard answers and it's been incredibly successful and has saved us a ton of time. We're doing a few things around the help desk and things like that, but, honestly, probably the biggest use of AI in Gainsight is in Gainsight itself, because we've already so that's the thing. What is new? we started rolling out Horizon AI years ago as part of our product and we're using it all the time. What we haven't done in terms of internal applications, by the way, is we haven't gone out and built out our own thing.

Speaker 2:

As I said.

Speaker 1:

I'm not really crazy about that idea. I don't really want to go try and compete with Google to hire data scientists for my internal work, for product stuff. It's different product. We have dedicated people who are on that team For internal applications. I'm not big enough to go do that. That's another reason I'm really looking to my vendors. Where my vendors have offered me AI capabilities fantastic, bring it on. But what I haven't done is gone. Try to, from scratch, write my own thing to create some AI capability on top of some existing packaged SaaS application.

Speaker 2:

Wave your magic wand and tell me in, let's say, three or four years, when we're having another version of this conversation, what has changed about the workplace that maybe today would seem like science fiction.

Speaker 1:

Yeah, i was thinking about this recently that something that I think I probably always thought of as lazy writing in science fiction was this whole dialogue with the computer. Back in the 60s, star Trek was like, hey, computer, i'm going to give you a really vague, open-ended thing. The computer would just come back with some amazing answer. That was just ridiculous. It was like, okay, i get where you need that just for the writers to be able to move the plot along. But that's crazy talk. I really don't. Despite having thought about AI and stuff over the years, i'm just not sure I ever really believed we would get there in that way. But I think we're seeing the glimmers of it.

Speaker 1:

I think GPT and its current conversational capabilities is pointing the way to that science fiction experience of just saying computer, i'm going to have a dialogue with you. We're going to go back and forth. I'm going to ask you this is not Alexa, turn on the lights, this is. I'm going to ask you tough, weird, open-ended questions and you're actually going to come back with useful answers. We're going to talk back and forth and hone those and I'm going to get really useful information out of the end of that. I think that's a science fiction thing that's going to be happening quite soon. Actually, what I point out to people is look, this is GPT-4. What's GPT-12 going to look like, running maybe on quantum computing? This technology is moving quickly and I think some of those science fiction realities or fantasies are going to become reality sooner than we think.

Speaker 2:

I agree and I firmly believe that the gating factor it's not the technology. I mean, we see a, potentially we don't know for sure, but GPT-4 is like a trillion parameter model and the token count essentially the amount of memory that these models have is increasing. It's clear that what needs to catch up is the governance around LLMs and the cost models. They're phenomenally expensive. Right now it's costing about $700,000 a day of Microsoft's money to run chat GPT. That doesn't scale. You mentioned quantum computing. We just don't have the processing power. I get scared when I think about the environmental impact of these very power-hungry models. I feel like it's rare in that the rapid iteration or the improvement in the quality of the technology is really being constrained by these other factors.

Speaker 1:

Yeah, I think it is scary, although I also think we're at the peak of the hype cycle. if you're familiar with the famous hype cycle, We all know what's coming next. Things will calm down and we'll start to settle into where the real value ad workflows are. Then that cost will come down and the usage will come down. Right now, everybody in the world is playing around. Eventually, they're going to get past playing around and they're going to start building real stuff. I think then things are going to calm down.

Speaker 2:

We are bad at time. As hard as it seems to believe, this one flew by. I'm not letting you off the hot seat without answering one last question for me. You've been at Gainesight six years, which in Silicon Valley we live dog years, so call it 54 years.

Speaker 2:

You work for a notoriously quirky CEO in a notoriously awesome culture, i know a little bit about our karaoke and a little bit about some of the Taylor Swift songs and talk us through what it's like working for Nick Maitre, and just maybe a little bit about the culture that you not only participate in, but presumably you've been a part of creating it. What do we not know about Gainesight?

Speaker 1:

It is fantastic. Honestly, the number one thing anybody at Gainesight would tell you is that the culture, the values are real. They're not like words up on a poster, they're actually lived. I saw that pretty early on. I remember being in some early meetings with Nick and having these moments where he had to make a decision. It was a decision that was like we can either do something that's right for the customer but will cost us money, or we can do something that's best for us but it will hurt the customer. I saw him make those decisions in favor of the customer. I saw him make tough decisions that were hey, we have this thing, it's critical to our culture, but from a pure business standpoint maybe it's not the greatest thing, and he would choose the culture first.

Speaker 1:

What I figured out pretty quickly and the biggest thing I can tell you about Nick, is what you see is what you get. Some people maybe, when they're up on stage, they say all the right words and they do all these things. The Nick that you see on stage is exactly the Nick offstage. He's the real deal. He's created this culture and he's gathered people around him who believe in it, who carry the torch. We all drive that within our teams and it's just very focused on things that are not rocket science Again, they're not things that other people don't have up on their walls like success for all and the concept that we really think it's important that everybody wins. At a company level, that may mean investors, customers, employees, community Within a group, it may mean security and IT operations, but it's like, at whatever level you're at, we really believe that stuff and we really operate that way. It's a joy, which is part of the reason I'm still around, because CIOs do tend to bounce around a little.

Speaker 1:

I think sometimes that's just because the commitment to the operational side of the business tends to rise and fall. It's very tough When you're first hired. You're hired because the execs and the board are like yeah, we need a CIO to come in and fix all these problems with our processes and our systems. You come in, there's all this great energy. A couple years later, when you fixed a few things, they're like well, that's great, we don't need to give you any money now, right, suddenly that support starts to fade. Then it becomes a real slog. So the CIOs tend to get a little itchy, because the one thing about CIOs is we're pathologically like we want things to work. I think that's the most common trait you would find in a CIO. It just drives us crazy to have to be in a place where we're just hacking away every day but we can't actually solve the real problems. So when that support wanes, we're often starting to see the writing on the wall and starting to look out for that next place where they are ready for us, where they do want us to do the job.

Speaker 1:

But I've stayed because, first of all, we've had great support, a great appreciation for the importance and value of the operational side of the business. Also because it's been such a fascinating place. It's like I've worked at three different companies, because we keep changing and growing, but largely because of that culture, because of Nick, because of what GainSight means, not just as a company but as an experiment in human-first business. We're really trying to prove that this works, not just from an employee satisfaction standpoint or something like that, but that we'd actually be successful. You don't have to choose between being this hard-nosed I only care about money, business person and being human-first. You can actually do both. You can make money but also operate in a way that you feel good about. It's been great.

Speaker 2:

As prepped for this discussion, I watched a few of the very memorable videos on YouTube, some of the lip syncs and that sort of thing. I got to say, Carl, I didn't see you in any of them. When are you going to make your debut?

Speaker 1:

I've been in a couple but I'm not a big karaoke guy. But believe me, you can't be at GainSight and not participate in a parody. That's what I thought.

Speaker 2:

They're amazing.

Speaker 1:

They are truly legendary.

Speaker 2:

If you send me a link I'll send you. Okay, with the listening audience watching you, then I'll post the link in the show notes. Cool, carl, we're out of time, but where can the audience learn more about you and the great work GainSight is doing?

Speaker 1:

Yeah, I would say go to our website, find me on LinkedIn. I'm always happy to connect with folks. Yeah, Google me. Luckily, I have a weird name, so I'm easy to find.

Speaker 2:

Brilliant. It's been fun hanging out, carl. Yeah, this was awesome. Thanks so much. We'll come back and we'll do another version of this. That's all the time we have for this week on AI and the Future of Work. Thanks again to mutual friend Carter Busty for introducing Carl and I And of course, we're back next week with another fascinating guest.

People on this episode

Head Shot

Dan Turchin

Host