After the Blast: How One CEO Fought Back 💥 | A True Crime

Robert Cioffi, CEO of a small company discusses the highly emotional impact that a major ransomware attack had on his, his customers and the community.The Emotional Toll of Cyber Attack was horrific to witness and made national news. Learn how to get prepared in this True Cyber Crime Story.

Growth without Interruption. Get peace of mind. Stay Competitive-Get NetGain. Contact NetGain today at 844-777-6278 or reach out online at www.NETGAINIT.com  
 

Have a Guest idea or Story for us to Cover? You can now text our Podcast Studio direct. Text direct (904) 867-4466

🎧 Subscribe now http://www.youtube.com/@cybercrimejunkiespodcast and never miss a video episode!

Follow Us:
🔗 Website: https://cybercrimejunkies.com
📱 X/Twitter: https://x.com/CybercrimeJunky
📸 Instagram: https://www.instagram.com/cybercrimejunkies/

Want to help us out? Leave us a 5-Star review on Apple Podcast Reviews.
Listen to Our Podcast:
🎙️ Apple Podcasts: https://podcasts.apple.com/us/podcast/cyber-crime-junkies/id1633932941
🎙️ Spotify: https://open.spotify.com/show/5y4U2v51gztlenr8TJ2LJs?si=537680ec262545b3
🎙️ Youtube (FKA Google) Podcasts: http://www.youtube.com/@cybercrimejunkiespodcast

Join the Conversation: 💬 Leave your comments and questions. TEXT THE LINK ABOVE . We'd love to hear your thoughts and suggestions for future episodes!

Chapters

• 0:00 Ransomware Attack: A Business Catastrophe
• 3:09 Understanding the Human Impact of Cybercrime
• 6:25 Evolution of Progressive Computing
• 9:22 Unique Targeting of MSPs
• 13:06 Day of the Attack: A Personal Account
• 15:35 Mechanics of the Ransomware Attack
• 17:17 Emotional Toll of Being a Victim
• 22:55 The Aftermath: Recovery and Resilience
• 29:22 Navigating Crisis: Initial Responses to Cyber Attacks
• 32:45 Communication Strategies During a Crisis
• 37:09 Emotional Toll Of Cyber Attack
• 42:08 Actions To Take After A Cyber Attack
• 48:55 Empathy and Client Relationships in Crisis Management
• 54:13 Community Support and Collaboration in Cybersecurity
• 59:25 Legal and Ethical Considerations in Cyber Incidents
• 1:07:33 Lessons Learned and Future Preparedness

Transcript

Robert Cioffi, CEO of a small company discusses the highly emotional impact that a major ransomware attack had on his, his customers and the community.The Emotional Toll of Cyber Attack was horrific to witness and made national news. Learn how to get prepared in this True Cyber Crime Story.

Topics: Emotional Toll Of Cyber Attack,True Cyber Crime Story,Cyber Attack Emotional Affect,Actions To Take After A Cyber Attack,Life After A Data Breach,Rebounding From Boom,True Cyber Attack Story,True Cyber Crime Stories,Life After Data Breach,Data Breach Story,Cyber Attack Recovery,Ransomware Victim Story,What A Data Breach Feels Like,What Breaches Do To Small Business,Dean Mauro,What Happens After A Data Breach,Robert Cioffi Data Breach,Cyber Crime Junkies

Takeaways

•    Cybersecurity is fundamentally a people problem, not just an IT issue.

•    Ransomware attacks can devastate businesses in minutes.

•    The emotional impact of cybercrime is profound for victims.

•    Managed Service Providers (MSPs) are increasingly targeted by cybercriminals.

•    Understanding the mechanics of ransomware is crucial for prevention.

•    Victims of cybercrime often experience guilt and shame, despite being innocent.

•    The recovery process from a ransomware attack can be swift with the right strategies.

•    Human resilience plays a key role in overcoming cyber disasters.

•    The importance of empathy in discussing cybercrime is often overlooked.

•    Cybersecurity awareness is essential for all businesses, regardless of size. Engaging with cybersecurity experts early is crucial during a crisis.

•    Tabletop exercises help prepare teams for real-life incidents.

•    Core values can guide teams through challenging times.

•    Clear and empathetic communication is essential with clients during crises.

•    Legal advice should be balanced with the need for clear communication.

•    Community support can be invaluable in recovery efforts.

•    Understanding the emotional impact on clients is key to maintaining relationships.

•    Crisis situations can serve as catalysts for organizational improvement.

•    Don't hesitate to seek help from networks and professionals during crises.

•    Transparency with clients fosters trust and confidence.

Chapters

00:00

The Ransomware Attack: A Business Catastrophe

03:06

Understanding the Human Impact of Cybercrime

06:00

The Evolution of Progressive Computing

08:57

The Unique Targeting of MSPs

12:06

The Day of the Attack: A Personal Account

15:13

The Mechanics of the Ransomware Attack

18:09

The Emotional Toll of Being a Victim

21:08

The Aftermath: Recovery and Resilience

42:38

Navigating Crisis: Initial Responses to Cyber Attacks

45:00

Communication Strategies During a Crisis

51:08

Empathy and Client Relationships in Crisis Management

54:09

Community Support and Collaboration in Cybersecurity

57:54

Legal and Ethical Considerations in Cyber Incidents

01:02:59

Lessons Learned and Future Preparedness

Speaker 1 (00:00.654)
you

you

Speaker 2 (00:06.35)
Imagine this, it's a quiet Friday afternoon, your business is thriving, you have over 2,500 clients, depending on you and your team, great customers, things are rolling along. Your biggest worry that weekend is where to dock the boat. You come back from lunch and step back into the office, and then you see one of your top employees face pale like a ghost, and you know something, something is just not

And then you learn the hard real life version of boom. Your entire business has been wiped out, not in days, weeks or years, but in minutes. This isn't fiction. This isn't a drill. It happened to my friend and small business owner, Robert Siafi. And this is his story. In that instance, his entire world vanished, gone in minutes.

all thanks to one of the most infamous ransomware attacks in history. So if you think you'd stay calm when everything you've built starts crumbling in real time, think again. This story is an emotional roller coaster with human resilience, inspiration, and even a showdown with the most notorious ransomware gang in history. And if you ever thought that cybersecurity was just an IT problem, trust me.

By the end of this video, you'll realize it's a people problem. And as a leader, those are your people. It's your problem. Let's dive into one of the most draw dropping true survival stories in history. Small talk sucks. So let's dive in.

Speaker 2 (01:56.152)
Catch us on YouTube, follow us on LinkedIn, and dive deeper at cybercrimejunkies.com. Don't just watch, be the type of person that fights back. This is CYBER CRIME JUNKIES and now, the show.

Speaker 2 (02:15.598)
David Mauro in the studio today. We have a very special guest Robert Cioffi Robert, welcome.

Thank you, David. appreciate the invitation to come and share a bit about what happened to us with the hopes that the listeners will be able to better prepare for themselves for what may happen to them.

Robert, so welcome. So Robert, for those who don't know, please look him up, connect with him on LinkedIn, connect with him at his company, Progressive Computers. Let's start from the beginning. mean, you have a phenomenal story. It's really heartfelt and it gets past all of the kind of the standard media that we hear about ransomware, right? We always hear about, you know,

Oh, the code and things were exfiltrated and like, you know, I cybersecurity people and tech, IT people in general, we always tend to hide behind acronyms and, and hide behind the mirrored rays. Weren't doing them like that's fascinating for the minutiae, but the reason. Yeah.

when this happens. See it when we come in post-event sometimes, but you know there's a huge side that there's an emotional...

Speaker 1 (03:37.088)
came behind when events like this happened. And I think that's really Robert, what's your story speaks to? Yeah, I mean, it totally is. If I can just pick up on a few of the little cues there. was attending a conference over the summer where there was a speaker who, by the way, I admire greatly. I think he's brilliant. You know, somebody in this space that has is tremendously credentialed.

You don't have to embarrass me like that, Robert. mean, you don't. You just come out and say David. I'm in the room. You don't have to. It wasn't me, folks. Who was it? Do you mind?

So it wasn't, it wasn't, but I asked them this question because we were talking about this very topic of, you know, what makes the news in events like this and whether it's, you know, typically we hear about a large enterprise, you know, how could the CTO at Uber do this or how did the, you know, Target, CISO allow this to happen? And it, we, we,

Santa over sanitize in the media or when you hear stories going around and not understanding that there are human beings behind these stories And there are a lot more facts and evidence behind the scenes That we don't account for we we've lost our ability I think especially with social media and technology being the way it is and you know here This is coming from you know and uber geek techno file myself that

we've lost the ability to empathize or we forget to do so maybe we have lost the ability but we forget to empathize to understand there are victims here I was a victim my customers were victims we didn't do anything wrong a crime was committed can we start there please

Speaker 2 (05:20.406)
Yeah, exactly. Let's back up a little because understanding the context in your background really sets the tone because I'm first, second generation Italian American also. You born and raised in the New York area. You came out of school. You started this IT managed service provider at MSP, IT company. Actually, you guys were coders in the beginning, right?

Yeah, I mean it was 30 years ago. was this MSP back then. Right. I was building applications in something called Clipper, which is a V-based variant.

Thank you.

Speaker 2 (05:57.647)
I remember Clipper. That's how old I am. I, yeah.

Yeah, I mean, I tons of software from scratch. Some of it is still stored away in some file structure here. I got to go dig that out. yeah. You live my past and, remember that I was 21st. but yeah, that's how we built this business is basically building applications. And then it, you know, went into Novell and, know, windows NT and, know, for many listeners, you know, the rest of the story that I don't have to bore you with the,

metamorphosis that happened slowly over those 30 years in the way the world changed.

Yeah, absolutely. So then you guys developed this MSP. What's the name of it and how many?

Yeah, the company is Progressive Computing, Inc. We're an MSP based out of Yonkers, New York, which is just north of New York City. So, you know, our primary coverage is the county of Westchester, Westchester County, which is, you know, that sleepy bedroom community of a million people outside of New York City. And but, you know, our reach is far beyond that. We have customers coast to coast across all 48 states, which is part of the story.

Speaker 1 (07:07.95)
It's an important part of the story about where our customers were and our ability to be able to get to them. But yeah, mean, it's probably no different than many of the IT companies out there. Two guys, two college buddies got together. My business partner and I are still running the business today. We're about to hit our 30th anniversary in February.

That's fantastic. Congratulations. I know that's wonderful, man. I love that.

We accept gifts. I'm a big

All gifts are welcome. All gifts are welcome. so I mean, yeah, I mean, that's a very quick background about, know, what, what we are today and what our roots are. we, you we were traditional computer science students from, you know, college, you know, with, code lines of code running around in our head and, you know, aspirations to build a company.

How many employees did you guys have and what type of clientele were you guys?

Speaker 1 (08:03.438)
I mean, there's no specific vertical that we serve. mean, there are pockets, nonprofit, lot of professional services. Real estate has come up quite a bit for us as well. Construction, things like that.

And are you mostly in the S &P space or?

have mainly the space. mean, we generally try to start at a size of about 10 users and then go up from there into the 100, 150 space. mean, that's our sweet spot is in, you know, that 20 to 50, 60 range. And as far as employees were about, I kind of lost count because it's fluctuated a bit, especially post attack.

but we're about twenty five ish again i think were planning on a couple of hires for upcoming year was just just still finalizing some budgets

Great. Excellent. So you're a healthy MSP continuing on for, I mean, and for small businesses to survive for 10 years is defying all the odds. Right. And then in a very burgeoning growing, IT space in a competitive market, right. This isn't, this isn't Wichita, right. We're, we're in New York and, so let's start at the,

Speaker 2 (09:27.362)
Thank you.

Obviously your MSP was targeted through a ransomware attack, is, which was very unique. Is that a fair state? It was, it was unique in the sense that, I mean, ransomware is not new, but the, usually they hadn't gone after the support providers in the past historically.

Historically speaking, it's been theorized and the theory has been proven and it's and we're not the first, by the way, the incident that happened with us was not the first of its kind, but it certainly provided another major data point in the not only the suspect or the suspicion of a trend, but the actual trend now developing.

is that hackers know that MSPs have the keys to the castle for not just one company, but for all of the companies that they support. Right. So if I, if I am able to successfully attack an MSP, can potentially have hundreds of companies now at my fingertips. And obviously this is all through RLM tools. Right. So we were, you know, just for clarity purposes here.

We were one of the Kaseya VSA victims on July 2nd of 2021. There were about 60 customers and everything I'm about to tell you or will tell you just really kind of two points about it. Everything I'm going to speak about is mostly public information and I'll cite where it's not if I go there. And second, this is not a disparagement against Kaseya. I choose to come public with our story.

Speaker 1 (11:09.846)
not to beat anybody up because I felt like we were a victim. Kaseya was a victim too, right? And like I said before, a crime took place, a criminal, you know, did some really bad stuff to a lot of very good companies and a lot of very good people. So I want to just almost kind of start and end right there. You know, a zero day exploit was taken advantage of in the Kaseya VSA product.

And explain to the listeners what the VSA products and for those that might not even that might be listening to this as a podcast later, right? While they're driving, they're working out MSPs or managed service providers. They provide different models of variable capacity, unlimited support for businesses, organizations, government entities, et cetera. And MSP is the acronym for managed service provider. The

tools that they use to remote control to monitor manage to fix we call it remediate the the organizations the reason we're able to scale the reason it's affordable is we're able to Create incredible efficiencies by working remotely by remotely doing it across clients across And we use various different tool sets one of these at the time very popular. It still is the the

Kaseya portfolio of tools. the, so you guys were using these tools and the tool maker itself, Kaseya just to simplify it, right? Tool maker itself was compromised, meaning a criminal adversary got in and put some element that would allow them then when

those tools were launched legitimately through all of of Kaseya's clients, which were you, right? To your customers, to your customers then, then it reached the end point, which was the actual business, government entity, organization, small business, et cetera. And then they launched their ransom.

Speaker 1 (13:32.824)
Yep. These tools are just if I can add a little bit to that. These tools are live tools, right? There are there's a piece of software that we install on every one of the customers that we support.

Please.

Speaker 1 (13:45.102)
It's called an agent software that agent sits on that windows or Mac computer and allows us to do all of the things that you articulated already, right? The monitoring, the maintenance, and there are some automated processes that we can do that greatly make our lives and our customers lives so much better and allows us to deliver the service as you described at an affordable rate that makes sense, right? So we're the outsourced IT for our customers. So they trust us.

and there's the important word, they trust us that we are in care of their systems. And as a customer myself of a tool like Kaseya, we trust that their tools are sound, not only functionally and do everything that's advertised, but are secure. So the one term that hasn't been used yet, but a lot of people are familiar with is supply chain attack. And that's exactly what this was. So the threat actor, the adversary,

discovered a flaw in this software and then set out to find who is using this software. And we were one of those 60 some odd Kaseya customer victims that that attacker was able to use in a malicious way to deploy ransomware to all of the endpoints, all of the people that we manage, right? Over 2,500 endpoints for us. Over 2,000 users that we support were affected by this.

There's a supermarket chain in Sweden called Coop. They are in the news, like, so I'm not telling you anything proprietary and it's very public information. had 800 stores across Europe. About 500 of those stores were shut down because all their cash registers run Windows and all those cash registers were managed by a company like us who had Kaseya VSA installed on it. So they shut down, you know, about 500 stores for

about a week because they were unable to process transactions. So that's the magnitude of something like this.

Speaker 2 (15:52.608)
No, it's hugely impactful. mean, it's just it's really the it really is truly the supply chain, right? Because they're getting it at the core and it's just affecting thousands all over. So let's turn to the day you found out about this. There was no it was a zero day exploit. So that's a tech term for it. It wasn't known in the industry. When we think of

antivirus or firewalls, right? When you think of what ransomware is, it's a series of code, right? Right. Characters, letters, numbers, symbols, it's characters. And malware is bad code. It's code that harms or spies or does something that somebody doesn't authorize and doesn't know that it's going to do. And those things have lists of all of the known bad codes and they block from them. Right. And then here, this was a zero day.

which means it hadn't been known. Nobody knew that that code was bad and it got through to Kaseya. Is that a fair statement? Very simplistic.

I mean if you read online and again, I don't mean to disparage Kaseya by saying this, but the zero-day exploit was actually discovered a few months prior to the attack and the reason I'm bringing this up

is because I want the audience to understand that there are plenty of zero day exploits out there in the wild right now on a variety of things that you may be using. But what's meant by zero day is that only very few limited people know. A researcher, perhaps an engineer who discovered this fatal flaw. And you got to remember that software systems are very, very complex. You can't just write a new line of code, republish the software and push it out to everybody because

Speaker 1 (17:41.748)
No, that would be that's not how development works, right? It's almost like making a structural change to a skyscraper and not really going through all the right testing to ensure that when I go put this fix in that the whole building doesn't fall down. they knew about it and they were working feverishly to get a patch out for it. Interestingly enough, we were 100 percent patched on Kaseya up to the day of the attack.

and you know a patch was imminent it was coming out from them but this threat actor somehow you know sometimes when researchers put or publish information out there quietly to a vendor somehow that data gets leaked or or it's just discovered independently by you know by malicious people who are just looking for holes and again it's it's not to paint cassay in a bad

light because microsoft right now and apple right now who right now as well as everybody else on this planet knows about flaws in their software that the point is no about

That's why we get updates.

Right. It's why it's why updates happen. So this just happened to be one of those perfect storms where there's this major security hole and the bad guys found it before the software company could patch it.

Speaker 2 (19:05.984)
Yep. Yep. So turning to the day, what was the day that you first learned about?

my goodness. So if

know you know the day is like burned in energy.

Fitting into your memory. Yeah, that will live in an infamy here So if you live in the Northeast or the Midwest from what I understand I actually haven't Experienced winters in the Midwest or the like the North Miss Midwest. I have friends in Minnesota Iowa places like that and they tell me it can be brutal there even in this right so, you know that you know for half the year the weather is kind of cold and

either damp or cold or just not great. We relish the summer, right? The spring and the summer is really our time to kind of bloom and, you know, kind of shake off the winter blues. It was one of those, as the weatherman here in New York will say, and I'm sure in other parts of the country, one of the 10 best days of the year. The sky was cloudless. The temperature was like in the mid-80s. The air was not humid at all because it can get really hot and sticky here, right?

Speaker 1 (20:15.104)
uncomfortable. So, so much so the weather was so great. It was one of those rare days where we actually had the windows in the office open, right? It's either the heats on or the AC. Very few times do we actually allow the breeze to come into the office. And the forecast for the next few days, including the July 4th Independence Day weekend, July 4th being that Sunday, this was Friday, July 2nd, was the same exact weather.

Yeah.

Speaker 1 (20:44.126)
personally i had a lot of plans actually my wife had plans for us along with some friends we had three barbecues scheduled not at my house thank goodness there you go and a friend of ours has a boat and he was going to take us are two families out onto the hudson river and we're just gonna you know have a great time so you know we

I was, this is where my head was, right? It was on the weekend, on the three day weekend and just really enjoying the glorious weather. And it was about, I don't know, about noon. not, I usually, much to counter to my health, I should be eating lunch every day. A lot of times I end up skipping lunch, but I was determined that day, I'm gonna have lunch, I'm gonna go to the kitchen and sit down. You know, I was kind of in that.

more relaxed, jovial mood. And I was sitting in the kitchen, eating my lunch, window open behind me, sunshine coming in, joking with a colleague in the kitchen. And I see my director of ops ascend the staircase to the second floor where I'm sitting, turns the corner, starts walking down a long hall that I can see out the kitchen door. I've got a perfect line of sight down this long hallway. And I can just tell something was wrong.

I'm from New York, you know we throw around racial terms in a positive way all the time. He's Filipino, so he's dark-skinned, right? But he's like pale as a ghost.

There's something really wrong. And this was that moment. It was that proverbial moment in the movie. mean, hate to make it sound so cliche. We're sort of the sky darkens. And while it was still bright and sunny outside, there was a mood that started to set in. My reaction was somebody just died and Jay is coming upstairs to tell us, would be my partner, could be a client, maybe a family member just called the office and

Speaker 1 (22:54.978)
You know, he grabbed the phone because he heard, you know, who it was. I was a little scared. And I said to my colleague in the kitchen. I think somebody just died is literally the words that I said, and I was serious. I didn't mean that in a in a funny or in a.

No, the tone had turned somber. You could tell.

Yeah, so I sort of met Jay right at the threshold of the kitchen door and he is like slightly shaking like I said, pal. I mean, he lost his complexion and he wouldn't look me in the eye. Now, the thing you need to understand is Jay and I have such a great relationship. You know, we are very straightforward with each other. We're very friendly with each other. We know about each other's families like like.

something was definitely wrong and it started to confirm in my head that, you know, something really terrible did happen. And this is where my head went, right? Like in retrospect, what happened to us while being really awful. So I don't want to undermine that part of the story, but it certainly wasn't the death of somebody.

Right. was business, right? was a business catastrophe.

Speaker 1 (24:08.778)
Yeah, listen, we you know, we lost we lost money. We lost clients in the aftermath of this That's that's gonna be an obvious part of the story And it was terrible blah blah blah, but nobody died but so I grabbed Jay by the shoulders and I kind of like, you know I needed to get the truth out of him and I said Jay what's wrong and he kind of fixed his gaze on me and said Robert all of our customers are ransomware

And I couldn't process that, right? How do you even put that in context to process it? It was a little too surreal. what do you- and I even said to him, like, okay, Jay, like, what do mean all of them?

and

Speaker 2 (24:49.55)
Yeah, you've got to think he was exaggerating, right? Like, it's not that way. Yeah, like two or three at once and seems like a big deal, right?

or maybe my self preservation mode kicked in and said, I reject your reality.

me reconstruct this in my mind.

So it was in that moment that I think he realized he needed to hit me with the sledgehammer and he started to list our clients. And I think subconsciously, I've asked him about this and I can't get a good straight answer out of him because I don't think he really knows, but I think he thinks the way I was thinking was that he purposely started to list off the large customers, right? The ones that would really drive the point, not the...

Not that little ones aren't important. I don't mean it that way. Of revenue meant for the business, right? And he started to list them. And as he's listing them, now my senses change. Now it goes to moving my head around in the open office and I can see through different offices and glass walls and cubicles. I can hear and see people on the phone and they're having conversations that are going something like this.

Speaker 2 (25:42.83)
Of course, but in...

Speaker 2 (25:47.63)
Yeah.

Speaker 1 (26:09.708)
I don't know what's going on. Hold on. Let me find out. Yeah, I think something big is happening. Let me get a technician to call you back. I'm really sorry. And people were stuttering. People were very serious. You could tell that they were dealing, know, when you drew somebody off, having a bad conversation with somebody on the phone and you can only hear the other half of it.

The other half that's taking all the heat, that's the way it felt. And I definitely knew right then and there at that moment, our RMM tool, that Kaseya tool that we use to manage all of our customers must have been affected. And it was in that moment that I started to feel a lot of different emotions, guilt was also another one. I don't know if it's the Catholic upbringing,

guilt, came in because I immediately just assumed that we had done something wrong, that we had somehow allowed a threat actor to break into our systems. Right? Like in, my case, I knew nothing at that moment. is just my primal instincts. Right? We have a Facebook, group, like many people probably have in their communities and people will post all the time like, my car was broken into.

You didn't know the details of that.

Speaker 1 (27:35.628)
And people are always saying like, look, the local police department keeps telling you lock your car, lock your doors. There are people who come around and try to open doors cause they're stealing laptops. They're stealing electronics. They're stealing money. They're stealing whatever they can out of cars. And my head is like, Hey idiots, how many times do have to hear this lock your freaking door? Right. And you know, it's like, why don't they still get it? It's like, this has been going on. This conversation has been going on for years. And I felt.

like that idiot at that moment because it's where my head naturally went. I never imagined that it would be a crime of an international scope and affect so many businesses worldwide and that it would be something that I had done absolutely nothing wrong. I just felt as the steward, as the person or company entrusted to care for all of my customer systems.

And for me to somehow allow this to happen on my watch, that's why I felt so much guilt, right? Now that guilt went away with time and that time was only a matter of hours before the full story began to unfold. But those were my initial reactions. I was scared, you know, quite frankly, I was terrified. You know, I watched my business valuation kind of just evaporate on me, like within a split second. And I really didn't know what to do. I was frozen.

Now you're done.

This was you and your college buddy. You guys have been in business for.

Speaker 1 (29:05.24)
Well, my partner wasn't there. This was my director of ops that I was with at the moment. My partner was actually out of the office at that moment. I think he was actually on a sales call of all places to be at that moment. But, you know, I was really just feeling the sense that, you know, you won't find in any business book the term I'm about to use. I consider myself a fixer, right? If there's a problem, I can fix it.

It's a car something in your house. I just had a major leak in my bathroom I ripped up half the bathroom and put it all back together again It like you know people like oh, so you you know you know plumbing and I'm like no I figured it out, right, you know

engineering mind.

I mean, as a kid, my father used to make me, you know, come in the garage and help him out all the time. So I'm used to this. Right. And even in business, just being a computer scientist, like everything to me in life is a problem to work out. Even personal problems. just I always feel there's an answer here. Right. There's I don't have clarity, but there's an answer here. And I work through it's the way my mind works. But in that moment in time, I was so frozen.

I had no clear path in front of me. I had no idea what to do.

Speaker 2 (30:23.72)
man. man.

Well, the people understand, so the listeners understand. So the threat actor was able to exploit this zero day vulnerability. got into your system and then from there they were able to move to all of your ransomware them. Yeah. Because remember that that tool is something that we use to manage all of our customers. Right.

So if they're logged in now as an administrator, which is essentially what they were, right. They logged in and had full 100 % control over all systems, just like my own engineering for a T sorry, my own technicians and engineers, they have full access to that system or had, cause we don't use it any longer. but you know, that's what they use to do their jobs on a day to day basis. end user standpoint. So Mark.

and look at

Speaker 1 (31:21.016)
works at real estate office and he goes in to sit down in his cube and he's going to log on the network. What does he see? How did they know? would, what would trigger them to call you? So let me, let me, let me just maybe wrap up that part of the initial part of the story. So I grabbed Jay cause I didn't know what to do, right? I grabbed him and we went into my office where I'm sitting right now and literally right where I'm sitting at this moment, looking at this very screen and two screens to the side of me. I have three monitors.

I sat down and frantically thought, I need to try something. I need to figure out information. I started to log into these systems. And as within a few minutes, as I'm starting to my own crude investigation here and trying to just piece together what the heck is going on. It's almost like war is upon you and there's fog and smoke and explosions and bullets flying and sounds that you thought you'd never hear, smells that you'd never thought.

you'd smell and here I am trying to like bring some sanity and clarity into this mess. And as I was sitting at my desktop, now everybody on their desktop probably has icons like that trash bin and a word icon and an Excel icon and an icon for some software that you use, right? The little cute blue circle thing that you always click on and that starts your software or your CRM system. All those different icons started to turn into white boxes. Now,

I'm yours, I'm yours.

on my computer and what a technical person will tell will know immediately without me explaining is that's the telltale sign of the ransomware attack because what happens in a ransomware attack what that software does is it starts to encrypt every single file on right it kind of it scrambles all those files and now windows looks at that file and go i don't know what that is anymore

Speaker 2 (33:07.96)
It spreads like a weed.

Speaker 1 (33:20.014)
the

So you saw the ransomware as it was encrypting your own.

systems and then the second telltale sign is that now there is a Read me dot txt file a text file a plain text notepad file on my desktop that When I opened it, I mean, there's no I know I already know forensic we knew at that point in opening the text file But that's the instructions for paying the ransom

Now before your systems began to encrypt, were you guys able to remote in to your clients to see what they were seeing?

At that point, 100 %

Speaker 2 (34:14.478)
Or you were just fielding the phone calls because it had already happened.

So forensically, this is what happened at 1049 AM. And I have proof of this. I have actual forensic evidence of this. At 1049 AM, the threat actor started their attack on us. What they did is they broke into the system quietly. We didn't know that they were there. They uploaded this ransomware software to our system as an installation piece of software. It was just a file. That's all it was. They didn't encrypt anything. They didn't do any damage yet. We call this the payload, right? This is the bomb.

They uploaded this bomb into the system and then they told all 2,500 computers that we manage, download this file from the server that we have, our, you know, our Kaseya VSA server, and then run that program on your system. So that's exactly what it did. That file immediately got downloaded to 2,500 computers and those 2,500 computers were then sent an instruction to open and run that file. And that file ran.

And that's how the encryption process began on all those what we call endpoints, right? All those computers. So by 12 ish when I'm in the kitchen eating lunch, having a good old time thinking my weekend was going to be full of a lot of food and very much right.

The payload had been.

Speaker 1 (35:38.774)
Right. It had already been downloaded and running. And the way these things are designed is that they run in the background and it doesn't like sort of unleash itself till the very, very, very end of the process when it finalizes the encryption. And then that's it. By the time by the time that happens, it's way too late. Yeah.

So what was going through your mind as you began to see your own files begin to be encrypted? Because there you were trying to fix it, trying to understand it so that you can fix it. Same way my mind works, right? And while you're doing that, you start to see all of your own icons gray out, white out. So what was going through your mind? I would...

Yeah.

Speaker 2 (36:28.93)
personally, I would lose my mind. So what was happening?

So two things really, it's the emotional state of what I kind of had been describing, right? Is this, you know, I meet with my wife on a quarterly basis to talk about our life plan, right? It's about our goals, our dreams, our aspirations, what we want to do with life, the kids, you know, retirement, all these things, right? And the reason I bring this up as part of the story is because that life plan is inexorably tied to my business plan and my business goals and our valuation.

and my exit strategy from here, not that I'm going any way away anytime soon, but all of that stuff is tied together. So the business plan helps inform the life plan about what's possible for us. And I just watched like in my head, both of those plans disintegrate. Right, right before my eyes, because here I am thinking 100 % of my customers have ransomware installed on their machines that came through me. So there's the guilt.

right, even though I'm the victim. And so were they, we were both victims in this. But I still felt guilty. I'm sorry, it's just my wiring. It's just the way I felt. Sure. And I felt shame, right. And I've been on stage plenty of time saying there really is no shame in victimhood. But that's what I felt in that moment.

It's a natural feeling. It's a natural human.

Speaker 1 (37:53.006)
I wouldn't be a human if I didn't feel that way and anyone to say otherwise is really quite frankly full of you know what because that's a bunch of horse stuff. I felt guilt, I felt shame. It's who I am and it's who we are. It's our cultural values here.

So, you know, I kind of just like watched all of that just disappear. How am I going to recover from this? So all of that emotion was one parallel feeling that I was grappling with. And it was an incredibly heavy toll on me personally and professionally to feel this. But at the same time, another parallel thought was going through my mind. And that was math. I had a major math problem when we

in our 30 year history, every MSP has an IT services company like me has dealt with a ransomware attack. It's just it is what it is. It happens. Somebody brings a computer a road computer into the network.

drive something right click on something

And somewhere somebody clicked on something, it's something new, whatever it happens. When that happens, it's like five alarm fire here, right? It's all hands on deck. We're all focused. all, you know, we've got a plan. We're under control. We manage it. We know what to do. It's a problem.

Speaker 2 (39:20.578)
So you had dealt with ransomware in the past. I knew the answer to that question before I asked. But I mean, like in the MSP business, this is this is a common threat. It's not something that happens every day, but you know how to triage it. You know how to handle it.

or every one of us have.

Speaker 2 (39:53.582)
But this is the equivalent not to, this is the equivalent to a mass casualty. A crash, a mass shooting. is the ER is just overwhelming.

It's a mass

Speaker 1 (40:05.592)
We were overwhelmed. how do we, I mean, that was the question that popped into my head. The logical mind, the fixer said, how do I mathematically get to 80 customers that are sitting in 200 physical sites across four time zones? Like now. Right now. Like right now. Right. And there's no answer for that question. It's like, how do I carry a thousand pounds?

and a hundred pound sack, you don't. Right. So we have to get very creative with our solution. And there's a there's an entire thread of the story behind how we recovered about 95 percent of our endpoints in a matter of 17 calendar days, which I'm told is like even for a large banks to recover from a ransomware attack. takes them two and a half weeks. So we you know, we did what a large enterprise could do.

in the same amount of time with a

Let's not let's let's back up there for a second. So at the end of all this, you guys were able to recover 95 % of those endpoints within 17 business days.

calendar days not business calendar calendar day like Friday and yeah guess who didn't go to any barbecues or boat rides right right guess

Speaker 2 (41:23.042)
I was gonna say.

Speaker 2 (41:30.286)
I don't know why you didn't go. I'm sorry. I'm supposed to be in a boat Saturday guys. is yeah. Try and have this done by Monday when I get back.

The story is even crazier because, and there's parts of it sometimes I forget, but one of our chief engineers, we got two guys that have been working with us, working that we hired right out of college. They're both 24 year veterans with us. These guys are technical geniuses. One of them, his father dies that weekend.

And he's got to deal with a funeral that whole week. So I lost one of my top two guys. Oh, my God. Not only did I have this gigantic like mathematical. But now it's like someone, you know, it's almost like, you know, playing one of those card games and that card comes out that says your main, you know, resource to help on this is no longer in play, like just, like like, are you serious? Like, Oh, my gosh. Like, really? This is what you're going to do to me.

Thank

Speaker 1 (42:32.782)
So, you know, the story is

So at what point, yeah, mean, here's what, so from where you are, what you've described to where you're able to recover, at what point do you engage, like at what point do you just pick up the phone and call like the FBI? Or do you call your cybersecurity insurance provider? Like who did you call like first?

So, I mean.

Speaker 1 (43:01.336)
So I made a lot of phone calls. Some of them were to friends who I knew that were cybersecurity experts to try to get help. I didn't have a lot of clarity at that moment. And this is why tabletop exercises like role playing scenarios is really,

So critical, right? Operational resiliency. You've got to go through the motions so that you know in the heat of the moment when you have amygdala hijacking, cannot think process rationally. You at least have the game plan. You can go, here's the steps I have to take. Yep. Because we've

Every

Speaker 1 (43:36.184)
mean, we did a couple of things. One thing was I pulled all of our team in a conference room and got them on the phone if they were remote and basically just kind of gave them a pep talk to say, look, I don't have answers. I know you're getting pummeled to death. know every line of communication is exploding on you, but I just want you to take a breath. We don't have any good clarity right now. We don't have a clear path in front of us, but I'm telling you that we're going to survive this.

because of our core values. And I recited our our four core values back to them to make sure that they had a level set, a reset mentally to know that, you know, a team commitment, humble confidence and respect. Those are our four core values. We're going to drive us through this. And when I'm on stage talking about this to fellow MSPs, fellow IT people, I will often tell them our culture.

of all things was going to be a critical success factor in our ability to recover. And we just kept pounding that into our people's heads. Remember our core values, remember we're a team, we're gonna do this. Even though it's not clear what we're gonna do, we're gonna do this. So that was one thing I did. I started to make some calls to some security friends that specialize in security.

But then I remembered, my goodness, I have a cyber liability insurance policy and I probably should call them. They assigned what's called a breach counsel, which is a horrible name because you never want to use the word breach. has a legal connotation to it.

because it's right there's confidence. Yeah, there's

Speaker 1 (45:18.316)
there's there's yeah so if you ever if you're listening to me and you take away one little nugget like stay away from that word breach unless it truly is a breach but let a lawyer tell you that and let a qualified lawyer tell you that but they assigned counsel they assigned forensics and by that evening I had followed filed what's called an IC three report so if you go to FBI gov

you can report your crime there which i did and i was contacted by the fb i i sat on conference calls and spoken to them stack of letters from them in my draw about the the criminal which was apprehended by the way and there's a little bit that story i can share

What we want to get to next is exactly who is behind all this.

I can tell you names, so...

So, so you guys get to get to going there and we don't need to discuss the technical aspects of how you guys remediated it. But at some point, I guess my question is how did you handle the communication to the clients? Did you, did you get on the phone with the business owners? You have like 1500 business owners you have to get in touch with. Like how did you address that?

Speaker 1 (46:36.75)
So we set up different lines of communication. All of our tools were ransomware, by the way. Thankfully, our email is hosted at Microsoft 365, and so that wasn't affected. But we were able to do a little bit of self-recovery first. Physician heal thyself, right? We needed to kind of fix ourselves first before we can offer aid.

being on the plane and the airbag and the air things come down, you have to take it before you can give it to your child, right? Because otherwise, I'm going to pass out in the middle of helping

Exactly. quite frankly, we had a 48 hour waiting period because our attorneys had advised us do not take any action until we forensically know what happened. in retrospect, because I knew and I shared some of the details 1049 AM, right? I could tell you those things only because that information was provided after the fact. Right. But in that moment, we had no idea. Right. Were they there for months, a week, an hour?

We had no idea what our recovery point was, right? So it took us 48 hours to get that. So in that time, we were able to stand up some of our own systems internally so we can start communicating better. And we just set up a line of communication. My partner and our director of sales and marketing were kind of in charge of that. I took the legal parts of it and made sure that everything was kind of sanitized through them.

believe it or not, actually pushed back on my attorneys because they gave us a bunch of legal mumbo jumbo to send out to our customers. And I said, if I said, all my customers,

Speaker 2 (48:16.823)
set.

Yeah, that's exactly. I've seen that so many times. And this is where attorneys really need to understand the business.

Well, I told them, you know, I explained to them about, know, I don't sell IT services. I sell trust and confidence, right? Everything is built on relationships. I can't send this. So can we put this in plain English? And actually, I gave them my version of it and they blessed it more or less. And so we started to work that way. So even though was a little bit of legalese in some of those communications, really,

I need to do that. a link in the communication. Be like for terms and conditions or details, click here and then you can have exclusions and disclaimers, you know,

few parts over time where I have to send communications out that I said, look, I really hate that this is a legal thing, but I need you to kind of sign off on this so that you understand like what my limitations and obligations are. And most of our customers were most all of them were very fine with that. Right. They understood. They understood. Communication is a really important element. And I do have a presentation that I do on this topic.

Speaker 1 (49:32.846)
But I talk about the five C's of the communication plan Which is that it has to be clear consistent concise? Counseled right and it has to be it has to be caring right and you have to You have to put some emotion into it. You can't just hide behind an attorney Yeah, that's good

So and so the five C's clear consistent concise counseled and what was the last one caring absolutely the empathy actually is key here right yeah and

Ehring.

Speaker 1 (50:10.59)
I was so, we were all so numb from what was happening, but we have to remember every time I was speaking to somebody, okay, you're the 18th person in the last hour to get on the phone with me and scream in my ear with F bombs every third word. Like at some point, like you become so numb to that. You have to maintain a, my goodness, this is terrible. I'm really sorry. Like we're working really hard.

I and I had and I didn't do that just to do it for the sake of doing it. I had to put myself in that caring mode. So by the end of the day, we were just in I mean, not only were we working 16, 18 hour days to recover, but we were just emotional wrecks because of that. That was that was really the initial part. I think, you know, once we got past those first couple of days, people understood like me yelling at them and he is not going to make this thing go away. Right. And they started to really

understand the full magnitude of what the heck you know happened to them and

Yeah, paint a picture for people that may have to face this someday or that were curious. And that is when you get a call from a client and they're all all their systems are down. And for the first 48 hours, all of your systems are down. What was what happened in that phone call? Well, what did they say to you? What did you say to them in general?

You know, there were all sorts of statements about, it was a lot, was very emotional, right? It was, do you understand what this means for my business? Do you understand that I may be losing millions of dollars that I could be losing, you know, years of relationships. And I'm like, you know, in, you know, I didn't, I didn't combat that. But I also gently reminded them at the end, I'm in that same boat.

Speaker 2 (51:39.818)
Lots of emotion, I admit.

Speaker 2 (52:01.964)
Yeah.

Right. I'm the victim. And I really have to say that one of the things that I was very surprised about it, and those of us in the service industry, you know, I think we're just so bent on like, you know, I'm only like, I'm only as good as the last transaction, right? And, you know, we're so paranoid about like making sure that everybody is happy. But, you know, I had customers tell me later on, we thought about firing you, Robert, but

but we thought about it and said, you know, you've delivered such great service to us over the years. You've always been there when we've needed you. And now that you've gone through this, I can't think of anybody else. I'd rather face a problem with like this again than you. You've gone through battle. You have that experience.

Well, isn't that what this is all about? Right? Isn't that the key message here? Right? The moral to this story is it's not, unfortunately, it's not a one off. mean, hopefully it doesn't happen to all the clients all at once, whatever. But compromises are bad. Scenarios like this are going to happen. And what this showed was you, your organization, your team's approach to it with empathy.

with strategy, with counsel and being able to show clients, see when things, bad things happen, we've got your back. We're going to fight this through with you. That's really a testament to you personally, your character, but also a testament to, your organization, your organization's culture.

Speaker 1 (53:34.53)
And we did.

Speaker 1 (53:42.638)
the whole culture. appreciate that. We leaned heavily on a lot of community members that really stepped up to help us. So what I mean by community is our peer community from around the country. We had people walking in from almost any state that you can think of, Kansas, California, Vegas, Texas, Minneapolis, Florida, Massachusetts. I know I'm missing some. People drove in, people flew in because

They knew that we were in trouble and that we needed some help. And being connected in a community like that, these are essentially my competitors, but in non-competing geographies. But they knew that. And as I have said on stage, even internationally, I was in Italy a few months ago providing my presentation on this topic to my counterparts there.

You know, the one statement that I keep hearing people resonate back to me is that I always say when you attack one of us, you attack all of us. I can't combat the world's brightest hacker minds. I can't, I'm sorry. If you as a potential customer want me to guarantee that, I'm going to say I cannot do that. Impossible. And anybody telling you otherwise is full of themselves. And that's about it. I can tell you.

as a community, can stand and rise together. We can lift each other, we can help each other. We don't have to be adversarial with each other. We can selfishly say, David, you got hit. I don't know you or I only met you this one time, but dude, you ever call, if anything bad like this happens and you call me, I'm there for you.

Exactly. So what percentage after you guys get things resolved? And I guess my question is, of the first, let me ask this of the how many clients of yours were roughly 1500? 2500 PCs all at once.

Speaker 1 (55:41.038)
It was 2500 PCs and that represented about 80 customers.

Okay, so about 80 customers all at once. So how long until they were all up and running again?

Yes

Speaker 1 (56:01.774)
Yeah, so, you know, we calendar days, if you've got 95 % of them in that first 17 within 17, uh, calendar days, sorry. almost said business. Uh, within 17 calendar days, everybody was visited at least once. Right? So some of our initial passes were, uh, we need to get critical systems up. Please don't ask.

get the conference computer running because we're right right now it's not a priority list what's critical what's mission critical to your business what line of business applications do you have that you need there are customer facing or or revenue generating or whatever it let's work on those systems first what can you live without and

discussion is something that in a tabletop exercise can be you can set expectations, right? Right.

That's it, right? Yeah. Is that you? And I think, you know, while everybody tried to broker for getting 100 % of their so what do you mean? Everything is critical to be realistic here. Right. Right. me help the next guy who hasn't seen me yet. Right. That's that we haven't visited them yet. Right. It's been two days and I need to get to someplace else. Right. So, you know, lot of people

Did a decrypt key? Did like the FBI or somebody like actually find the key decrypt key? I'm asked that about ransom.

Speaker 1 (57:24.458)
So what happened here is that. So the FBI was actually inside their systems for about two weeks because what they were trying to do is build a case against the perpetrators of the crime. And if they had released that decryption key that would have told the hackers that, you know, law enforcement was was already in the systems and it would spook them and they would shut their systems down and, you know, fold back into the shadows.

Holy grail, like did we find it?

Speaker 1 (57:54.19)
So and people have asked me aren't you upset with the FBI? Well, you see that flag behind me. It really means something. This was an attack on you know, I think our Western values and who we are as as a society and I feel if it helped build a case against the perpetrators and the FBI was actually able to do something about it, which they were then I say keep the decryption key. Don't tell me about it now. Some of my customers were upset.

they knew it. They had it. They could have saved us all this heartache.

I remember reading all about that because there's a lot of order. There's a lot of discussions around people being upset. I had the decrypt key for a couple of weeks and that's not even why I asked. Actually, when I asked the question just a little while ago, I had forgotten that. And then as soon as I asked it and I was like, I remember this. I'm like, my gosh.

And because of their investigation, they were able to make an arrest. Yaroslav Vasinski is a Ukrainian national who on October 23rd of 2021, you can tell that I've studied this crossing the Ukraine Polish border was apprehended by Interpol because the FBI.

Poland, right? They caught him in Poland?

Speaker 1 (59:14.77)
They caught him in Poland as he was crossing the border from the Ukraine into Poland. We have extradition agreements with Poland. So he was extradited to the United States. I'm sorry. No, he's a Ukrainian national. And he is sitting in a jail cell in Dallas, Texas. I can think of no better state in this great nation of ours for him.

With Poland,

Speaker 2 (59:29.152)
Russian national.

Ukrainian nation.

Speaker 2 (59:43.608)
So

and I've often joked with the FBI. Well, they joked with them once I was like, Hey, if you guys ever want to like go out for lunch or donuts or whatever, you need somebody to like watch the prisoners.

little bit of that street kid from Yonkers sent me. And all my customers who are feeling that too, like I feel like on their behalf, I need to commit a And now he's facing what? hundred fourteen years. His sentencing is in, I think on March 20 something of 2023. So we'll see what happens. But

Here's a message that I want to make sure that everybody who's listening or still listening after I'm bloviating for about an hour here, that you understand that this is not the act of an individual, right? He had two other people that he was working with, right? I think are still kind of on that FBI list. And just like.

I don't manufacture Dell equipment or write Microsoft software, but in my business and wrapped within my services are a hundred other vendors like Dell and Microsoft and Apple and all this other stuff, right? Hackers too have their own supply chains, right? So the guy, this guy.

Speaker 1 (01:01:08.3)
ransomware game it was Reval Sodenokibi which is actually run by some Russian nationals

Yes, is our evil who we have several episodes on and have discussed in detail on.

more story

Speaker 1 (01:01:33.09)
Well, the leader of that group, which is I'm drawing a blank on his name is like something's like 30 something years old. His father-in-law is the head of the FSB. Now, the FSB is what those of us in our. That's the equivalent of what used to be the KGB. Right. And so he's like a senior director there. So please.

Pulled out of the FBI for what?

the KGB.

Speaker 1 (01:01:58.21)
you know slap me in the face and tell me that i'm being silly to think that there's no ties back to the russian government

always all independent and there's checks and balances. It's not a totality.

Right. It is widely known that this guy has five Lamborghinis are beautifully wrapped in all kinds of crazy Color schemes and one of his license plates in Russian I can't read Russian but I'm told that that particular license plate Translates into the word thief in English So that's like and he's known to be doing donuts and like Moscow Square and all around and nobody touches him like law enforcement like those Leave this kid alone

It's like the it's like the yeah, it's like the old days in the United States with the

So maybe I'm embellishing on that story a little bit. don't know. This is the information that's being told to me.

Speaker 2 (01:02:50.798)
I'm those those entities exist here as well, right? And they definitely existed 20 30 years ago

So the point about this is that Yaroslav was not an independent, he didn't write the software, he carried out the attack.

He might have even bought the access to cassia from an IAB from an from an from an initial access broker, right?

He subscribed to the access and then he gets a cut of whatever whatever payments are made, right? And that's how it works. And so the wheels of justice actually worked in our favor this time, at least on the front line of the attack.

Well, now the R Evil group was taken down shortly after this past what January, right? Or February. It was shortly after.

Speaker 1 (01:03:41.046)
Yeah, they went down nine days after the attack. They took their servers offline, yeah. Which was highly unusual. And this was right around the time that the White House made a call to the Kremlin. And apparently, I don't know, naked pictures of each other with farm animals were traded and said, you don't take this, I don't know what, some backroom deal was done.

It hits something because Wendy and and and they took down the heads of our evil in and we have a whole episode on on the our evil aspect but it was true Russian fashion right like busting down doors there's videos all over YouTube. yeah like

I don't know if that stuff is pomp and circumstance and video just to make the Americans feel good that you

Yeah, a lot of people, a lot of people feel it was a distraction because they were, you know, what they were about to do or in process of doing.

But they did go offline and so that was an interesting thing. they've been since back. I mean, they weren't down terribly long.

Speaker 2 (01:04:46.734)
No, no, they've they've they came back I think in April there was evidence of

shut down for a while, be quiet, we got you, we're gonna do anything to you, we're put this video together, you'll be back online in a couple months doing what you do. Take a little breather, you got a lot of money there. Lamborghini? Is that where the Lamborghini came from? Yeah.

Yeah, exactly. Right, exactly.

Yeah. Unbelievable. Yeah. So, so he's facing, so where we sit today about how many of the clients that you had before this attack, what percentage roughly are still with you today?

Speaker 1 (01:05:27.424)
It's got to be 85 % plus. It's I mean, you're going to some people left us for all sorts of reasons.

Wow.

Speaker 2 (01:05:37.322)
So there's natural attrition in MSBs. So every three to five years, there's usually a percentage that just go away for various...

It's completely obvious. We had some accounts that were sub 10 users that we were like, I, you know, I got healthcare facilities, a nursing home with 60 users and you've got five people running, you know, like whatever you're running. got it. I got to service them first. I'm sorry. We're talking about patient care here. Right. So I had to do also, or, and I say, I, anytime I say, I, by the way, it's really, we, we, the entire team. So,

course.

You know, we really had to prioritize some of the smaller ones like, look, I couldn't wait. I liked you guys, but I just couldn't wait. So I got somebody else and I'm just going to stick with them. Right. And so we lost those customers. had one customer, actually quite a large customer who their investors and their largest account that they had that represented over 10 % of their revenue walked. And their reasoning to us was our customer.

doesn't didn't like what happened and saying that we need to change our our service provider so I'm sorry guys we know how to go others you know I got that we just felt like it was time for a change I think trust or confidence was eroded especially from those decision makers that I want to say are a bit more disconnected from

Speaker 2 (01:06:46.797)
some

Speaker 2 (01:07:02.082)
They don't understand the tech, the technology. They really don't understand this wasn't you guys. This, yeah, this wasn't you. it.

We don't when we don't have that type relationship at the highest levels and I don't mean that like hey You might be a CEO listening to me right now of a 75 person company I'm not gonna sit there and chew your ear off quarterly for 90 minutes about you know bits and bytes and tech stuff I'm there to talk business, right? Right this too. I might be a geek on the inside, but I still business owner so you know

I mean, technology, I mean, we talk about this all the time and technology is a river. It flows through every aspect of an organization. So as you're going to build your business, we have to understand your business because as you have an initiative, trust me, technology is involved. And so we just need to know how can we configure things, support you, prepare for it, plan it out. There might be ways to save you money in your venture. there's it's not just about.

selling you more services. It's really understanding the business.

It's my job to understand what your goals and what your challenges are and then how do I align technology to help support those either support your goals or to help you overcome your challenges and that's what's really about when I don't have that report at that senior level and we get pushed down to the office manager we've got a great relationship there he or she loves us right but you know you've got someone who's a little bit

Speaker 2 (01:08:31.278)
not the same though. Right. think

I think you shoot yourself in the foot by not engaging. And again, like I said, it doesn't have to be, you know, a very boring technical conversation. It's not what it's about. It's about business. And how do we, how do we use technology to enhance your business? Right. So you're about to acquire somebody. How are we going to do that? Right. You have aspirations to double the size of your company in three years. How does, how is technology planned and aligned to help you achieve that goal?

Right? Systems that you have today may not support that growth. So shouldn't I know about it now?

Absolutely. So let me ask you this. How many like how is your house progressive computing? How has your company been since this? Have you guys solidified? Did you lose a lot of employees? You guys?

We had some, was funny because we lost two employees due to just some natural attrition right before the event.

Speaker 2 (01:09:32.118)
It was COVID too. had a pandemic that was affecting.

COVID Shmovid we went

I suffered far worse my I'm here to live and tell the tale. Yeah, so we had like two open positions. We were kind of casually looking. We were not in a rush. So we just decided not to replace those. We had one engineer that oddly enough just it was circumstantial had an opportunity to go back to a prior employer who threw some stupid offer at him. And we were like,

no.

Speaker 1 (01:10:11.712)
Okay. But we ended up with that happen within a couple of months.

That's pretty good. That's really good.

We ended up letting one person go, simply because we started to also go through a little bit of a restructuring at that point. took it as an opportunity to look ourselves in the mirror and go, well, how can we be a better organization? And when we did that, somebody unfortunately didn't survive that transition. He's off elsewhere.

And that and that happened. Of course. But overall, you guys are hiring now and that's.

We've replaced all that business with new and we're in a hiring mode at the moment. We, in fact, I changed, you know, I sat down with my partner one day and had one of those, I don't want to say come to Jesus moments, but like, you know, hey, look, can we, let's wipe the blood off our face here for a second and kind of figure out what do we need to do to make this a.

Speaker 1 (01:11:13.934)
So that's really helped us enhance our business. So I mean, in a way, like, I don't want to say, well, I'm really thankful that that happened to us. But, you know, make lemonade from lemons, you know, whatever that saying is. We just said, all right, look, I can't change the past. So I may as well use this as a catalyst to how do I get better?

Yep.

Sounds like he's really good. I'm glad to hear that

On a personal note, did you and your wife's quarterly plans and your life plans get back on track after a period of time?

It's a little up and down. I of got put on pause. We didn't actually talk or meet about it for a while, but towards the end of 21, we resumed those conversations. Sometimes we're a bit better at it than other times. I meet quarterly offsite at peer groups. actually do facilitation of other peers and helping them grow their businesses.

Speaker 2 (01:12:14.968)
That's great.

So I'm a paid facilitator for that, which is great. It's great opportunity for me also to grow as a leader. But I use that meeting cycle and cadence when I come back from those trips. That Saturday night that I'm back in New York, we use as what we call life plan date night, where we pull out the plan. It could be at the dining room table or it could be at a restaurant. We pull it out, we walk through it. How come I'm not meeting my health goals, my workout goals or my...

play guitar super badly and like that's on my life plan to not play as badly as I play it today.

Yep. I have the same life plan.

Hey, who's over in the other corner? Yeah, that's why I my hair out because I thought I was gonna be like an 80s rock star but think you play better if your hair was long and then I cut it does there is is an actual scientific connection. there is at least up in here there is Yeah, so there's all you know, like we're doing well with that Like I think my wife and I are pretty happy about like the progress there. mean listen, we can always do better, right?

Speaker 2 (01:13:05.15)
By the way, Mark, it does.

Speaker 2 (01:13:23.081)
yeah, I mean all of us can.

Yeah, there's there's there's

It was during this. If you don't mind me asking, I don't want to get too personal, but I mean, like, how are your kids during this? Like, were they aware their dad is involved in something like of international espionage and cyber crime? Like, holy cow, that's a lot for I try and explain to my own children sometimes what what we see and what we're dealing with. And they're like, if I get too technical, they get bored. But if I explain the emotional part, they're like, seriously, they're like,

Yes.

Speaker 1 (01:13:55.95)
One of the things that my wife and I have have always heard people comment about our kids is wow, they've got like such an amazing vocabulary and they speak like they're adults, right? Because we've always believed into just like treating like talk to them as I would. I'm not going to change my communication just because you're my child, right? I mean, obviously I'm going to not use gigantic words, but we we've always felt just be straight with them, right? And tell them what's going on.

My daughter my oldest daughter at the time was I'm trying to do the math here So she was a sophomore going into her junior year. She was home for that weekend from college and You know, so she was all ready to have you know to enjoy the weekend with us and at the time I guess my younger daughter was I'm thinking she was 12. Yeah, she was 12 at the time

And she was also pretty excited about the plans because all these people that we were gonna spend time with were, we were friends with them because our kids are friends, right? But yeah, I they just, I came home that Friday night and I hadn't seen my daughter from college in a few months and she was arriving that morning, right? And she's just sitting there waiting for me to come home and she's like, what happened, dad, right?

account.

the other one was asleep but you know we I spoke to them about it they they know they I just you know I was I was playing about it but like I wasn't coming home throwing things up against the wall or or you know custom like a drunken

Speaker 2 (01:15:42.602)
Yeah, they probably saw a very vulnerable side of their day.

I was I was emotionally drained. was physically drained. I just put it on the line. Look, something really bad happened. But, you know, that's going to you know, that's got a great group of people at the office, got a great partner, got a really solid company, got wonderful customers. It's going to really stink for July. You're not really going to see me too much. Just got to give me some space. I just got to work through this. Right. And they've now since seen me social media, hearing about trips I've been taking.

doing keynotes. got invited to go to Italy, all expenses paid to go deliver this story. So like they're excited for me, you know.

Well, it really is a truly inspiring story and to hear how you guys have recovered both personally, professionally, it's just absolutely wonderful. Robert Siafid, thank you so much, sir. We really, really appreciate it. you very much. I mean, this will not be the last time. Yeah, this will not be the last time we speak. We will definitely check you out. We'll have links to Robert's company and himself on LinkedIn in our show notes.

Thank you, sir, so much. Thanks for what you did. Thanks for the view that you had, the way that you handled this. is honestly, it's a great inspiration. It's a great, true cybercrime story that really knows that. it really is the human aspect of when a massive ransom.

Speaker 1 (01:17:12.822)
If I could just leave the listeners just with this final thought and it's been a comment. Everything that I've said here, but please, please, if something bad ever should happen to you and it doesn't even have to be a cyber crime, right? It could even just be something debilitating that's happened to your business. Don't hide it. Like you've got to get help. You've got to seek help. You've got to get the right legal help, the right law enforcement help. You've got to get the right technology help. You've got to.

Absolutely

Speaker 1 (01:17:37.676)
Reach out to your customers, you've got to reach out to your friends, you've got to reach out to your networks of people and just put it front and center. And I'm telling you, it's human nature for those of us that really care for you to want to be able to step up and help you, right? This is not something to hide. Hiding it does not solve any problems.

Yeah, absolutely. Well, thank you so much, sir. Thanks everybody for attending. Thanks for listening. And we will we'll catch everybody next time. Thank you so much. it. Robertson.