The ThinkND Podcast

The New AI, Part 15: The Computer Science Domain in the AI Revolution

• Think ND

Share episode

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 48:50

Episode Topic: The Computer Science Domain in the AI Revolution

What is the essential final step in a world where technology can generate endless data but still lacks wisdom? Listen in to a conversation about the fundamental evolution of computer science with Computer Science and Engineering assistant teaching professor William Theisen '18, '22 M.S., '24 Ph.D.

Featured Speakers:

  • William Theisen ’18, ’22 M.S., ’24 Ph.D., University of Notre Dame

Read this episode's recap over on the University of Notre Dame's open online learning community platform, ThinkND: https://go.nd.edu/7c5243.

This podcast is a part of the ThinkND Series titled The New AI.

Thanks for listening! The ThinkND Podcast is brought to you by ThinkND, the University of Notre Dame's online learning community. We connect you with videos, podcasts, articles, courses, and other resources to inspire minds and spark conversations on topics that matter to you — everything from faith and politics, to science, technology, and your career.

  • Learn more about ThinkND and register for upcoming live events at think.nd.edu.
  • Join our LinkedIn community for updates, episode clips, and more.

Welcome and Introductions

graham-wolfe_1_02-27-2026_140547

Welcome everybody to the New AI Project podcast. My name is Graham Wolf. I'm a senior at the University of Notre Dame and the program director of the New AI Projects. Today we're excited to welcome Dr. Bill Tison, professor in the University of Notre Dame's Computer Science and Engineering department, a triple Domer, PhD in computer science, and an expert on the role of misinformation and computer vision. He's also the current program director of the Bachelor of Arts and Computer Science, and we're thrilled to have him on today to apply some of our research into AI's sociotechnical impact to his, fascinating role. So it's a pleasure to have you here. anything else you'd like to add to that intro, Bill?

bill-theisen_1_02-27-2026_140547

Oh, no, that was-- thank, thanks for, thanks for having me.

graham-wolfe_1_02-27-2026_140547

On the New AI Project side, we're also, joined by Matthew Chow, who's done some really great work, writing our, what we call Why It Works column, which was formerly Research Revelations. But beyond that, we've, developed a team of those student experts, as we call them, each writing in their own domain of AI. So there's... It can be anything from Tech Titans, AI at Work, Taming AI, AI in Life, and then again, Research Revelations. So there's really something for everybody and, Matthew, I'll pass it over to you for your, introduction.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah, for sure. Hey, everyone. My name is Matthew Chow. I'm a current senior here at the University of Notre Dame, and I study computer science, and I have a specialized interest in AI, and specifically, I like building things that have AI in I'm currently the author of the Why It Works column, formerly known as Research Revelations of the New AI Project, and that's where we go over a lot of the basic technical details of various, you know, AI techniques and products that you might see out in the world today

graham-wolfe_1_02-27-2026_140547

Yeah, one of the ways we describe that is, really putting a lot of these cutting-edge products under really practical scrutiny and kind of unbuilding them and looking at them from the, you know, kind of from the ground up, the constituent parts and everything like that. I think that's really, a, a lot of the value that you bring to this column. And, in tandem with this conversation, we'll also be, releasing a piece about artificial intelligence and computer science curriculum that Matthew's, authoring as well. So I think just to kind of kick things off, we're gonna dig into your, perspective a little bit here, Bill, with, a question about sort of what your mindset looked like two to four years ago, as artificial intelligence reached everybody's, you know, the, the real public mainstream as opposed to just more of a niche or more research interest. did the world change for you at all as somebody who was, looking into becoming a professor of computer science and who was, you know, really living and breathing CS? you know, how, how did the world change before and after November twenty twenty-two?

bill-theisen_1_02-27-2026_140547

Which model was November 2022?

matthew-chou--he-him-_1_02-27-2026_140547

I

bill-theisen_1_02-27-2026_140547

What? That's a--

matthew-chou--he-him-_1_02-27-2026_140547

was GPT-3.

graham-wolfe_1_02-27-2026_140547

did

bill-theisen_1_02-27-2026_140547

Three or

graham-wolfe_1_02-27-2026_140547

it...

bill-theisen_1_02-27-2026_140547

and a

graham-wolfe_1_02-27-2026_140547

Yeah.

matthew-chou--he-him-_1_02-27-2026_140547

I think it was ChatGPT first release. I think it was three. Let me look that up really

bill-theisen_1_02-27-2026_140547

Okay. it doesn't-- I originally thought there was no way it was gonna work. I remember people telling me about it, and I-- without having tried it, I said, "No, that's dumb. It's not any good." and then I tried three and a half. It was three and a half, I remember, and I thought, "Oh my God, this is actually pretty good." And that was, you know, years ago actually compared to today. It was terrible. I wouldn't say my world changed, but it was certainly probably the most impressive application of AI I'd seen at that point, in my opinion. and even in the past four years, the models have gotten better and better and better and better. so it's been, it's been interesting to see how they've changed, especially I would say with the, advent or introduction of the coding models specifically, I think has made a larger difference for me than, say, three and a half or four made.

graham-wolfe_1_02-27-2026_140547

Gotcha. I mean, yeah, that makes sense. it's-- People talk about that, that era as if you could just kind of feel something in the air had changed, you know, the day before versus the day after. and of course, it's important to remember that there's much more tempered reactions you might have, especially as somebody who's, who's studying CS as an academic discipline. So yeah, I think that makes a lot of sense, and, and thanks for that perspective. And Matthew, I think I can kick it over to you if you wanna continue to kind of dig into this perspective, again, with an ear to, how these new, groundbreaking AI products have, impacted curriculum

matthew-chou--he-him-_1_02-27-2026_140547

for sure Yeah, I, I completely agree in the sense that, I remember, you know, fall of 2022, Graham and I, we were both freshmen first coming into college, and I remember, the general sentiment around it was like, "Oh, this is just a really, really cool, niche kind of tool." it's "Oh, write, write a poem about me and my roommate living together," or something like that. And it was something that seemed very, very silly and very, very kind of like, just kind of cool to use, but besides that, it didn't really seem to have a lot of use. And it was really cool to see, you know, as the years going on, you know, especially into our sophomore, junior, and especially even now into our senior year with things like agentic coding and stuff like that, that has completely just rocked not even industry, but it's rocked also the curriculum as well. You know, I remember multiple classes that I've been in where there's been, you know, assignments that I've had that you can just have ChatGPT or whatever model that you want at the time completely take over. You know, at a time I remember when I was in your class, Bill, there was... You were doing a live coding demonstration literally on the screen about how to make some sort of prediction model on Google Colab, and it was just filling in the blanks for you. All you had to do, just click tab, tab, tab, tab, and it just completed the entire assignment for you. So something I'm really curious about is, you know, I have never had basically an experience in computer science curriculum without AI. from the very beginning, AI has always been there, and while I haven't always used it, I can't really remember a time where it hasn't been an option.

CS Learning Before AI

matthew-chou--he-him-_1_02-27-2026_140547

So my question to you is what was it like going from a student, you know, in the mid-2010s all the way through to your PhD? what was it like before, and then afterwards, did-- could you see like a shift in how people were like approaching problems or completing their assignments and stuff like that?

bill-theisen_1_02-27-2026_140547

That is-- that's a good question. I think a year ago, I had a student who asked me how I could have possibly cheated on my assignments when I was an undergrad because chat didn't exist. And it's well, we had to do it the old-fashioned way. and, and the old-fashioned way, one, one big thing I've seen is collaboration, right? And so, you know, every office hours had twenty people in the class sitting outside the professor's office the entire time. And if you struggled-- Somehow, somehow computer science has gotten less social, which may be hard to believe, but it, it really does feel that way. it used to be if, if you were working on an assignment with someone, you were sat next to each other, you know, on the couch in the student lounge or wherever. And nowadays, I talk to students all the time, they're like, "Oh, my partner did this last night. I have no idea what they did." And there's actually quite, it seems somehow even a less social atmosphere now that, chat can handle all of your problems. We, we of course had Stack Overflow. but

graham-wolfe_1_02-27-2026_140547

Yeah,

matthew-chou--he-him-_1_02-27-2026_140547

and computer architecture and stuff like that. I also agree to now where I look at the curriculum that is being put out to

graham-wolfe_1_02-27-2026_140547

come

matthew-chou--he-him-_1_02-27-2026_140547

what they have to do and the assignments that they have been assigned. And in terms of difficulty, I can notice like a significant jump that has had to be made just by the nature of you can't have AI solve it completely for you. You can't just copy and paste the entire thing in. Given AI can give like a lot of help and stuff like that, like you said, but even as somebody who came in as AI was being, you know, created and it-- when it first hit the mainstream, I do notice even today the difference in the curriculum from when I was a freshman versus freshman today, or, you know, sophomore and vice versa and stuff like that. so, you know, I'm kind of curious as to, you know, I don't think I've ever explicitly asked a professor about this, but what has the like shift been like in terms of, you know, creating assignments or what do you value in terms of the academic curriculum here today in the age of AI, specifically to computer science? Are you trying to test mastery or are you trying to test more like, recognition, or is it the problem-solving aspect? What do you think is most valuable for you?

bill-theisen_1_02-27-2026_140547

That's interesting. I actually think it's way easier than when I was an undergrad. That was something that we used to complain about all the time, is how watered down it is compared to, compared to what it used to be. that's interesting that you think it's actually gotten harder. I, I would say actually that haven't professors designing assignments that differently. They've just decreased the points that all the assignments are worth and pumped those back into exams, like closed note exams or a big thing is, a shift towards oral exams. And so you just have to go in and stand up and know your stuff. part of the reason-- I mean, so you took AI with me, and that was pen and paper, closed note. You just gotta sit down and do it. The exams aren't that hard, right? I don't want them to be hard, but you have to be able to do it just using your brain. I think that's what's important to me.

matthew-chou--he-him-_1_02-27-2026_140547

Okay. Yeah, so the-- in terms of, the overall comprehension, it's comprehension from yourself and not with the aid of you know, internet and stuff like that. So I agree. I have seen also as well the evolution of... I remember going into my sophomore year, some of the classes I had heard from upperclassmen, they used to be put on things on like Google Forms or there was even like a live coding assignment where

bill-theisen_1_02-27-2026_140547

Yeah.

matthew-chou--he-him-_1_02-27-2026_140547

would code and then you'd run it on your local machine and then submit it through like a Dropbox or the student machines or stuff like that. But I also have noticed that, you know, almost every single exam or quiz now that I've had, whether it be for like the actual required classes that I take or even the extra, know, what's it called? Electives that I take, like almost all of the exams now are on paper because, you know, it's impossible to use AI you're writing with a pen and paper. So that's really interesting to know that... And also to your point of like it getting easier versus harder, think that is in terms of like and stuff like that, in the terms of while the assignments themselves might have gotten harder in difficulty in terms of what they are asking, it is not harder to complete it because you have the assistance of AI, right? So I think that that's a really interesting kind of, you know, nuance to kind of pick up on as is the computer science student, are we actually getting better or is it that with the new tools that we have, like we are able to accomplish more because we don't have to do as much?

bill-theisen_1_02-27-2026_140547

Is that, is that a question? Do you want me to make an assessment of you guys?

matthew-chou--he-him-_1_02-27-2026_140547

yeah. Is there, is there any Based off of what you've seen, you know, you were a student when there was no AI, do you think that today... This is kind of a loaded question, so feel free to you know, be a little vague about it, but you think that it's, you know, better for students today because they're able to do more with the AI, or do you think that there's still a level of foundation that is still needed that you think was better, you know, without AI?

Why Fundamentals Still Matter

bill-theisen_1_02-27-2026_140547

I think it's, I think it's both, I think it just changes what is useful to know. So it, it used to be that it was actually really valuable to know the syntax of a language or to know what, what libraries were available in a language and, and things like that. That, that was genuinely useful to have memorized because it would take you thirty minutes to go-- I mean, it would take you thirty minutes to find the right Stack Overflow link to figure out that you

graham-wolfe_1_02-27-2026_140547

Can

bill-theisen_1_02-27-2026_140547

and you probably produce things more slowly, but that was a direct speed up, right? Having all of that stuff memorized. I think now th-that is not actually quite so useful. What's useful though, that hasn't changed, and I think the undergrads don't have the best respect for this, is like it, it used to be back in my day, like Googling was a skill. Some people were better at Googling than others, right? And if you were really good at Googling, you were able to find your Stack Overflow answers faster and it was faster and faster, right? How, how are you good at Googling? You, you sort of knew what was out there and knew how to ask good questions How do you know how to ask good questions while you understand the, the fundamentals of the field in such a way that you can communicate about them clearly? I think the benefit of still teaching the fundamentals and learning the fundamentals at a level-- at an abstraction level above language, right? It's no longer useful to know that to define a function in Python you use def versus, I don't know, some other language. that's not useful, but you should know what a function is and how they operate and that they have arguments or return statements. So when you ask Chat, right, you can actually ask it a useful question and give it enough specifics for it to do that generation for you, And really at the core, all progra-- almost all programming languages that our average student will interact with, right, you're gonna have functions. You're gonna have some notion of loops. You'll have some notion of variables. Whether or not you need a semicolon here or, you know, a, a comma there is really not useful to know anymore. But you should still probably know the fundamental building blocks, right? You should know what an error message looks like, so when you get one, you're able to parse it, right? And then intelligently ask Chat about the error message.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah, for sure.

graham-wolfe_1_02-27-2026_140547

I jump in really quick, Matthew?

matthew-chou--he-him-_1_02-27-2026_140547

Oh, yeah, for sure

graham-wolfe_1_02-27-2026_140547

So I kinda wanna just, zoom in on here. You've started to do this evaluation of, you know, what's valuable and what's not valuable. and I think that's really interesting to sort of see a window into, you know, how you're making that assessment. But, I think a good case study on this would be your Intro to AI class. and that is the title of the class, right? Intro to AI?

bill-theisen_1_02-27-2026_140547

yeah, yeah.

graham-wolfe_1_02-27-2026_140547

Okay. Yeah. so I'm, I'm curious, what is-- how would you articulate the point of that class? you know, what are some of the big key takeaways that, that a student should have from it? And maybe Matthew, we can, compare and contrast with your experience as a student. but yeah, keeping in mind all these things that, that you're saying about, you know, what is and isn't valuable anymore, yeah, how have you designed that class?

bill-theisen_1_02-27-2026_140547

and I, sure, I'm sure Matt maybe would give you a better opinion of this than, than me. I think initially the class is extremely disappointing for many students, maybe in the first third to half of the class because AI is, is actually quite old. But every student only knows like anything that's happened in the last couple years, maybe five. so when they hear Intro to AI, that's sort of where, that's where their mind is, and that's not where the class starts. we start with some very historical stuff, stuff that most of the students won't think is AI. and I think that's very frustrating for the students, but I think My hope, my goal is that it's actually pretty difficult to jump straight into deep learning, I think, and, and really grasp it. And deep learning developed through machine learning, developed through this intro-- like this old AI stuff, it's actually a pretty natural progression of how we got to where we are, and I think it actually helps you learn if you almost follow along with that natural progression. And so I, I think most of the students come on board by the end of it. But I think the first bit is really frustrating. another thing that's a little different is they hear intro to AI and they're like, "Oh, nice, we're gonna do prompt engineering and drag and a bunch of stuff I can put on my resume." And then it's just-- it's not like that at all. It's like the actual matrix multiplications, and you're doing partial derivatives and, all kinds of squirrely stuff like by hand. And I think most of my students would tell me that that is not useful. but I think part of it is that it is setting you up to ask better questions later. if you have no idea what backprop is or what like a training loop is, then it's really hard if you end up in a data science job and you're asked to train a model and it's just not working. really hard to debug that if all you've sort of seen is like some web UI and you click play on train model and it just runs for you. so I would say it, it is like actually very foundational and annoying for the students, but the hope is that it-- you're never gonna do backprop by hand again, right? Much like you're, you're probably never gonna do some crazy calculus stuff you learned in calc one. But maybe the idea of a derivative and like understanding second order effects of mathematical functions is useful generally, and that is just the avenue through which you approach it. But I, I mean, Matt, you can really, you can lay it on me, right? I'm not grading you anymore, so

matthew-chou--he-him-_1_02-27-2026_140547

No, I definitely think that, you know, I also remember when I was taking Intro to AI with you, for your sifs, I was talking to a lot of people that are in the class, one of the main things that people were saying about the class was like, "Oh, I just wish that we had some more hands-on experience with you know, actually coding and, you know, making these models to see what these doing instead of you know, like you said, doing stuff by hand." for me and for a lot of, you know, my peers that go through the class as well, and as well as just students in general when they are studying things like introductory phases of AI, you know, the history of it and stuff like that, is exactly like you said, it's very intuitive. And I think that's something that I really enjoy about it because when you start to learn about how backpropagation or maybe like a neural net or even something like classification works, it makes a lot of sense intuitively because you realize, you know, these are things that we as humans, we just do naturally. We don't think about it, but it is something that is a building block that I'm sure when you were like five, when you were first gaining consciousness, when you were really, really little and you're learning things like math and how to talk, this is exactly what you did. You know, you were noticing patterns, and you're updating your own brain, your own knowledge based off of what you had seen. And so being able to that down into something that's logical, into something that's solid, into something that you can do manually is something that's really, really cool to see that you can learn, but you can artificially learn, quote-unquote, "through numbers." And based off of that, when we started to really get into the weeds of, you know- how an LLM works, you know, to the modern day of what AI looks like today, even though it's been super, super convoluted and complicated through all of these different techniques that have spawned in the past, you know, to 10 years, it does make a lot of intuitive sense. And by that making a lot of intuitive sense, to me, it lets you, like you said, ask a lot of the good questions that you need to know in order to do the things that you wanna do. I wouldn't have been able to build any of my projects without knowing at least a baseline of what an AI is capable of or how it works. Because based on that, I'm allowed to know, you know, okay, what can I actually accomplish here, and what can I not accomplish realistically? And I think that is the most valuable thing to me, and I think that is, you know, something that is valuable to a lot of students as well. So regardless of, you know, skills that you learn, the coding aspect of it, I do think that there is still value in it. I also think that it's like you said, where it's like the value that you're gonna get from the class is going to be the value that you seek from it as well. So if you're going into it looking for how to code a model and stuff like that, you might not learn that itself. But if you're going into it with more of, I don't wanna say open mind, but a more of a baseline type of perspective, you might get a little bit more from it than you would otherwise. I don't know, did I make sense, or was that just like

graham-wolfe_1_02-27-2026_140547

right.

bill-theisen_1_02-27-2026_140547

I, I would just say to add on that, like that's pretty in line with my view that like the code is cheap now. I think teaching code is borderline useless because I, I don't write code anymore. I just ask Claude Code to do it. But you have to be able to ask properly. I think that's currently the limiting factor. And so in my opinion, it's actually more useful to know the non-coding stuff because code is like trivial at this point.

graham-wolfe_1_02-27-2026_140547

I, I think that sounds right. And, I, I think a lot of people are, as you were indicating, both of you were, a lot of people in the CS world are very married to the hard skills, the value, like the kind of correlate hard-- learning hard skills and resume type skills with, you know, the value of an education. and people are just really married to the idea of job outcomes more so in CS than I think in a lot of other fields. and that fixation paired with the recent disruption in the job market and the big pivot that, CS education has had to take, I think that's led to a lot of cynicism, a lot of ideas about "Okay, I took all this time to learn this, and now," like you said, "it's super cheap. It's super, you know, it's not as employable anymore." and there's just a lot of this idea of you know, I have such a difficult major and I have such a difficult, you know, road ahead. and at the end of it, you know, the job market is not gonna, you know, reward me handsomely the way that it used to be set up to do. So, maybe I'm oversimplifying there, but how do you respond a little bit to that cynicism, if at all? and, you know, how do you recommend a student kind of navigate that from like a mindset perspective?

bill-theisen_1_02-27-2026_140547

Oh, that's a good question. let me... Let's see. I, I think-- I mean, this has happened a million times, right? Like compilers didn't used to exist, then compilers exist, and it's not it's not like computer science disappeared, right? co- as code becomes cheaper, like arguably there's more and more of it that needs to be maintained and, you know, the models actually do make mistakes, believe it or not. I, I could give you any number of examples of the most heinous code I've ever... I, I mean, every time I work with the models, I, if I do it for an hour, there's guaranteed to be at least one thing it does where it's just like no human, even like a sophomore who's taken a only a year of programming would ever come up with doing it that way. that's just bad. so what I've told the, what I've told the students is that like I think the future is just like maybe you're not writing as much code anymore, but someone still needs to architect the system and lay out the class structure and so okay, you don't have to write every line, but someone still needs to write the function headers. So I just-- I, I think it's different, and I actually think it starts bleeding more into a design type, Thing. I mean, that's what everyone in SF is saying right now, like taste matters. don't know, like that's, that's arguable. But I think, I think what's frustrating for kids is that for a long time it was a very rote path where grind LeetCode and you will get a job, right? That's basically the only thing that matters. But I'm of the opinion, and talked to students recently, like there basically aren't LeetCode interviews anymore. I don't, I don't know what you've seen, Matt, but like-- And so it's, it's, I think it's more a frustration with how easy it used to be. You knew exactly what you had to do, and it actually, grades almost didn't matter. It was just like, grind through these problems and then you will be given a good job. and it's no longer the case. Like you need to be a little more than that, and I think that's hard for the students that were only in it because they wanted to, you know, be told jump and then ask how high. so I, I, I think, I think the current-- I saw this tweet the other day and it was like, CS majors are down at Princeton by forty percent and there's been a huge increase in cybersecurity and data science majors. It's those are just branches of computer science. it's, it's, it's very silly. It's like saying, "Oh, nobody's majoring in business anymore. They're majoring in accounting." It's wait a second. So I, think maybe equate CS with software engineering. I think software engineering is undergoing a big change. I, I think, I think CS is several levels higher than that.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah. I-- To kinda add on to that, it's something like, you know, we hear the perspectives of San Fran and, the tech industry all the time on, things like social media and on LinkedIn, and we hear all these stories, you know, of people being like, "The coding of the future..." Jensen Huang said, "The coding language of the future is English." And then you hear these tech CEOs being like, "Software engineering's gonna be obsolete in about, two years." And they keep on saying these things. And on the other side of it, because we are in academia, you as a professor and us as students, we hear kind of like the opposite of that, but it's still the same type of cynicism where it's like, "Oh, AI is ruining the curriculum. Students aren't learning anymore. Like, all you all do is cheat." And you even see it sometimes, people are just like, they just cheat on their, what's it called? Assignments. All you have to do is chuck it into a, an AI model, and it'll just do most of it for you. I think part of the evolution here, the natural progression of things is like it's somewhere in the middle where it's not completely cynical to think of it in a specific way, but at the same time you have to recognize that the way that you have to learn and the road path that you're gonna take to maybe a successful career or even a successful academic career is going to be a lot different than it was, not even in like previous generations of college, but what we had experienced in something like high school or grade school And it's kind of hard to break out of that mindset, but at the same time, like I also have a difficult time kind of like differentiating this as well. But at the same time, the evolution of it, me, the idea of being able to stay not ahead of the curve, but on the curve, and being able to keep up of what is happening not only in industry, but also the opinions of academia as well. I feel like that is the most important skill set that I can have right now as, you know, somebody who's gonna be a recent graduate and somebody who is-- hasn't gotten their foot in the industry yet, or a solid viewpoint of what I want to do for the rest of my life professionally.

graham-wolfe_1_02-27-2026_140547

That sounds right, yeah.

Future of Software Engineering

graham-wolfe_1_02-27-2026_140547

I, I do kinda wanna zoom out a little bit and, and, and think about, like you were saying, Matthew, there's, there's a lot of these prescriptive ideas about, you know, what should somebody in our pr- shoes be doing? you know, what should a student be doing? What should a professor be doing? What should a young urban or a young professional be doing? I wanna talk a little bit about what should this CS world look like five, 10 years from now if it were to be, you know- Perfectly well-regulated and perfectly handled this shift, this adaptation to kind of accommodate, as you're saying, not just CS, but specifically software engineering. you know, what would curriculum, what would, the job market, what would the whole space look like in five to 10 years if we were to kind of nail this transition? again, s-specifically zooming in on software engineering because as you noted, it's, it's, sort of what people equate to.

bill-theisen_1_02-27-2026_140547

boy, that's a that's a tough question.

graham-wolfe_1_02-27-2026_140547

I mean, if you have any thoughts.

bill-theisen_1_02-27-2026_140547

Oh, yeah, yeah, yeah.

graham-wolfe_1_02-27-2026_140547

No pressure.

bill-theisen_1_02-27-2026_140547

there's a couple different, say, important parameters. One is just are we-- is it gonna be a sigmoid in terms of I mean, like the METR chart, like GP or, Opus 4.6 just obliterated it, right? So it's like super exponential, on-- in terms of like task length. if the models keep getting better and better and better, then like we're all dead, right? that's the, I don't know, slow takeoff or whatever. we all turn into paperclips. I'm not convinced that tax-- task length equates to usefulness. I, I don't know, I, I s- I sent Opus off the other day and it ran for forty-five minutes and came back and everything was done incorrectly. So there, there, there's a bit of a question there. another question is formal, if you have a task that you can prove formally, then that's really good, right? Because that's just like you just tell the model, "Iterate on this until you c- like you, you've formally proved your system." I think-- I don't think the models are going to us all. so I think then the job becomes, which is sort of what I was describing, the role of a software engineer becomes setting up the constraints in which these models will just churn through a billion tokens and, produce the answer. And so, essentially I think software engineering into writing tests, and that's all you ever do. And because once you have the tests, the AI will go generate like a trillion-line code base that will pass all of your tests. and like you probably have the AI to write the tests, right? But I've yet to see the models be particularly good about thinking about how to structure a code base. if you have five classes, like scaffolding then the, the verification, formal verification I think is where we go. And that, that's of course like I'm not convinced the models stays, like it's all VC-funded right now, right? if we have economic Armageddon or I-- like we're all-- everyone's in on data centers, right? But the grid literally can't support that. I think actually see a bifurcation where the top labs, if you have the money, you have access to the best models, but average people will have less and less access, or it'll become more and more cost-prohibitive. so I think we're actually in a bit of a golden era right now in terms of just like average Joe having access to close to state-of-the-art, and I think that changes. But right now, assuming we don't all get turned into paperclips, I think it becomes like a big formal verification, testing framework type of job.

graham-wolfe_1_02-27-2026_140547

So it's about, evaluation, verification, validation, that kind of thing. and I'd imagine that would trickle down into curriculum as well. That's the kind of thing that you could, I mean, just go all in on evaluation as opposed to, you know, writing and generation. I mean, there's design as well.

bill-theisen_1_02-27-2026_140547

Yeah.

graham-wolfe_1_02-27-2026_140547

Interesting.

bill-theisen_1_02-27-2026_140547

and, yeah, I mean, there's the there's like the old story of the guy who came in and smacked a computer with a hammer and charged two thousand dollars. And it was like hammer one dollar, knowing where to hit the hammer like one thousand nine hundred and ninety-nine dollars. Like I ver- I very much think that still holds true. the code, the hammer, super cheap, but there's a lot more that

graham-wolfe_1_02-27-2026_140547

Yeah.

bill-theisen_1_02-27-2026_140547

it,

graham-wolfe_1_02-27-2026_140547

Yeah.

bill-theisen_1_02-27-2026_140547

I think.

graham-wolfe_1_02-27-2026_140547

Matthew, does anything surprise you there or stand out to you?

matthew-chou--he-him-_1_02-27-2026_140547

No, I'm just like, the what, the what I'm thinking right now is kinda I feel like the way that, I guess like or like professors, what they want the students to learn kind of aligns with that kind of viewpoint of it seems like AI is leading to in the future. of a, not just an application of, you know, syntax and rules, but much more so understanding. You know, I've had not even just you, Bill, but so many professors and so many teachers and all of my education, like just drill into me like, "You can memorize all you want, but at the end of the day, it's all about understanding and applying." And I feel like that's tr- kinda like this is pointing to. S- But if that is where this is pointing to, I don't really know why there is such a huge divide within academia in terms of how to handle AI. Or maybe it's ty-type of thing where AI is just not understood enough right now to where there can be a consensus of how to handle that ty- sort of topic. But I don't know. Just kinda like my thoughts

Using AI to Learn Well

graham-wolfe_1_02-27-2026_140547

Okay, quick pause. We have about 10 minutes left. What do you think would be most productive or interesting to discuss? Matthew, any thoughts?

matthew-chou--he-him-_1_02-27-2026_140547

maybe we talked a little bit about the different opinions of the industry and academia. maybe sort of like in your-- 'cause academia is much more apprehensive about using AI from what I've seen. Feel free to butt in if I'm wrong. And then industry is much more like you should be using it for everything. but like kind of in the terms of today, like you said, Bill, it's kinda like the golden age where everybody has access, or everybody in academia right now kinda have-- has access to at least a free model. how should students and professors be using AI right now to learn as effectively as they can, and to what degree should they actually be using it?

bill-theisen_1_02-27-2026_140547

I start answering that?

graham-wolfe_1_02-27-2026_140547

Yeah. Wait, actually, let's kind of restart. I'll give a quick lead-in and,

matthew-chou--he-him-_1_02-27-2026_140547

Yeah,

graham-wolfe_1_02-27-2026_140547

stick to-- I like that. That's a good question. So there, here we go. So we started to kind of, hint at this a little bit, but there's seems to be a big divide between, the affinity for using AI tools and, and the apprehension or vice versa between academia and some would say the real world or the world outside of academia, the, the world of jobs and s- employers and stuff. So, Matthew, as, as somebody who's, just kind of navigated that process, talk to us a little bit about, your outlook. You know, why, why might that be and, and should it be that way? And, and, you know, just, just maybe give us your thoughts on that, on that space.

matthew-chou--he-him-_1_02-27-2026_140547

for sure. So like we know right now that like everybody, all the students here at Notre Dame, you know, everybody in the world that has access to the internet has like access to ChatGPT or a free version of it, and it's capable of a lot of things. right now it feels like from the people that I've talked to, that seem to be super pro-AI. "Oh, you should be using it for everything. It should be involved in everything that you do, so you can learn how to use it." the other half should be s- saying "AI is, at the end of the day, a b- a brain drain. Like you're not learning anything, actually. You're just offloading your thinking." So I'm just kinda curious, like in today's day and age, where AI is right now in its current state, how should students or professors too be using AI to kinda learn effectively to the most that they can, and to what degree should they be using it? Should they be using it, you know, to the degree that a lot of people think that- They should be using it as much as they can? Or is it a sign of something that you should kind of leave on the back burner? Or maybe is it somewhere in the middle?

bill-theisen_1_02-27-2026_140547

Yeah, I think people definitely use it a little too frequently. myself included. I tried to have it write one of my exams the other day, and it took me like eight hours, and then the exam was terrible, and I had to rewrite it myself anyway. And that took me about an hour. So I, I mean, I wasted like almost a full workday trying to get this silly little thing working. and it just didn't, right? It wasn't gonna do it up to snuff. but that being said, there's certain things that AI s-speeds up for me a lot or makes possible that I wouldn't, have been able to do otherwise. And specifically with learning, I mean, my favorite technique is just I'll ask it about a topic. You mentioned entropy earlier, so I'll ask it what entropy is and it'll It'll print out some long explanation, and I just read until I get to the first thing I don't understand, and then copy and paste that and say, "Explain this further." Read that explanation until I get to the first thing I don't understand, copy. and then I get down to something, right? Read back through what I've done until I get back to the original question. Read until I find the next thing I don't understand. and this isn't any different than it's been with like practice tests, right? You take a practice test, you did badly. You're like, "Oh, I'll just read over the solution," right? Or if, if you're gonna look at the solution manual for your homework, you're like, "Oh, I'll read over it." You read the solution, you're like, "Oh yeah, I get it." You just tell yourself that, but you don't actually, right? A good student's gonna sit down, try and work through the solution. ninety percent of students will read the solution, tell themselves they understand it and move on when they don't. And I, don't think that's changed at all, right? It's just a, a different form of it. so I, I mean, I use it, I use it every day, all, all day, basically. I've, I've got Claude Code windows open over here on my other monitor that I'm-- I chat with, right? I have the remote, the remote server set up that came out two days ago, and I talk to it on my phone, on my home PC or here when I'm out and about. So I'm very pro using it. think it's most useful when if, if I want to write a web page, I could sit down and write eighty lines of HTML, but I, I know how to do that, right? I'm not missing out on anything if I just have it generate those boilerplate eighty lines for me. I think the danger is when you need to understand it or someone else needs to. So like Claude to write an exam is really bad because the students have to interface with that, and so I need to have sort of my rubber stamp on it. But one thing I've been using Claude Code for is to write a lot of visualizations, interactive visualizations of AI techniques. And as long as the visualization works, I don't really care about the JavaScript that went into that, right? Because it's just some piddly little website for my students. It doesn't really matter. and so that's when it's useful, when you just kind of only care about the output, not necessarily the contents.

matthew-chou--he-him-_1_02-27-2026_140547

Okay, for sure. And a quick like Just follow up because I'm curious about that. what if the, goal of what you're trying to accomplish is a little bit of both? let's say that you're picking up a new project, and it required you to use a specific tech stack, and you needed to have a correct output is very easily achieved by asking AI. But at the same time, you also need to, you know, understand why it's doing that or how it's actually working. Do you think that that is, a good applical-- applicable use case for AI, or do you think at that point it's better for a student just to learn it all on their own and use... and not use AI at that point?

bill-theisen_1_02-27-2026_140547

I mean, I, I had Chegg when I took stats here and easy to just copy the solutions out of the manual, right? Oh, I didn't, I didn't learn stats at all, right? You know, I bombed all my exams. So yeah, I think you just have to be honest with yourself, but I don't think that's anything new, right? If you're actually-- If you're gonna use AI to write the code and then you're genuinely gonna sit down, read it, be able to understand how it works and take something from that, then by all means. But for me at least, the best way to learn is to actually do it and AI prevents you from that, but so did a solutions manual fifty years ago. So I don't, I don't think that's changed. it just seems like it has because you now have a solutions manual for almost everything, right? That's the only difference. But people who are gonna use the solutions manual like me will now just use it for literally everything because it works. But if you're honest with yourself, then I think it's completely fine.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah. And it seems like almost like the balance here is a little bit like, kind of like pair programming, right? Where you'll sit down with, you know, a senior dev or a, in this case, an AI model. You'll have it, watch what it does, and then when it gets, just like you said, with, you know, explaining entropy or a different school subject, when it gets to a point that you don't understand, you just ask it to elaborate further. And to that point, you can build the real understanding while still kind of seeing the actual workflow that it goes through and the actual steps that it takes to get there, and the explanation behind it as well. I feel like that is, to me, what I envision as a sweet spot. I don't know. Do you have any thoughts on that?

bill-theisen_1_02-27-2026_140547

I mean, yeah, I do that all the time. I mean, I, I've got a, like an LLM training right now that I wrote from scratch, but I-- there were loads of things that were broken when I wrote it, and so I'd paste it in the chat and be like, "Why? Why doesn't this train?" It'd be like, "Oh, well, you set up your, your loss function was expecting one hots and you don't have those or something." Right? So that, that was really useful because, okay, I, I understand what the difference is, just give me the code to fix that. But I had to get myself to that point first. Yeah, I think, I think that's probably a good analogy. I think that's a good solution, right? You have a tutor. but the danger is the tutor-- a student comes in and asks me for help on something, they have to answer to me. If you're working with chat, you can just close your laptop and walk away, right? There's not-- I think there's a bit of a social aspect that's missing there that makes it easier to lie to yourself about how much you're taking from it.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah, that definitely makes sense. And at the end of the day, it feels like a lot of the, you know, the most out of AI just comes down to the human, like self-discipline and being honest with yourself, which is, you know, skills that I still need to work on myself, but it is soft skills that are valuable, you know, just to have just a successful learning experience in general, general. Yeah. other than that, I don't really think... I think we ta- touched on almost everything that we have that I

graham-wolfe_1_02-27-2026_140547

No, I think we're in a good spot. Yeah, I think we're in a good spot. I think, kind of parting thoughts, Matthew, I'm thinking I'll, sort of tee you up with the opportunity to talk about your last article. just kind of go through some of the talking points there all related to what we've been talking about here. and then I think that could be, like, a good parting thought is just sort of the conclusion of that article, which is what I'm reminded of in a lot of ways here, which is that it's all about evaluation. and, and kind of just stick a landing on the conclusion you had for your article as well.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah, quick question. the article that we were planning on next, I thought that it was gonna be about OpenClaw and

graham-wolfe_1_02-27-2026_140547

Oh, no, no, no, the one from,

matthew-chou--he-him-_1_02-27-2026_140547

or...

graham-wolfe_1_02-27-2026_140547

ago.

matthew-chou--he-him-_1_02-27-2026_140547

Oh, okay. Okay. So specifically about Claw Code? I'm sorry.

graham-wolfe_1_02-27-2026_140547

Yeah.

matthew-chou--he-him-_1_02-27-2026_140547

misunderstanding

graham-wolfe_1_02-27-2026_140547

code, but, I think your takeaway there was, like, the value you added was, like, in the evaluation process.

matthew-chou--he-him-_1_02-27-2026_140547

Okay, sure.

graham-wolfe_1_02-27-2026_140547

Am I right? Did I remember that correctly?

matthew-chou--he-him-_1_02-27-2026_140547

think, think I did more of just what it's capable of, but I can, I mean, we- I can also just add onto that. That's perfectly fine. I think that's just a

graham-wolfe_1_02-27-2026_140547

Okay.

matthew-chou--he-him-_1_02-27-2026_140547

progression as well. Yeah

graham-wolfe_1_02-27-2026_140547

Yeah, let's do that. All

Evaluation as the New Skill

graham-wolfe_1_02-27-2026_140547

right. So, what I'm reminded of throughout this entire conversation, I think, Matthew, you touched on in your, in your most recent article, which I'll, I'll give you a chance to kind of walk us through here. but I, I, I think the, the really valuable takeaway for me was that the value that you added when you were going through the process of building from scratch a website was largely kind of evaluative. I'm searching for the right word. I mean, that's the word we've used in this, this conversation, but I feel like it even goes beyond that. It's almost, kind of, conversational the way that you guys w-were talking about where you're going back and forth with this, this agent that's, you know, highly competent, but, you know, you're giving it, it your creative direction, almost more like a design role that again plays out through this conversation with an AI. So maybe, maybe walk us through the-- your process there and, I'd love to hear, you know, just what it felt like to be kind of that evaluator and, you know, use that maybe as a signal for, for where this world is headed, and you know, where the field of software engineering might be going.

matthew-chou--he-him-_1_02-27-2026_140547

Yeah, for sure. I think that's something that is, really important to note about my previous article is when I-- the specific task I asked CloudCode to, to do on its own, basically, to plan out and everything on its own, at the end of the day, it was still very much largely guided by me. You know, the beginning prompt that I gave, it gave it the very basics of what I wanted to do. I wanted it to make a website about this specific thing. But in the planning process, in defining the actual scope of it, CloudCode did ask me a lot of questions, and at the end of the day, what it got was it just wanted something that was very, very basic, something that was very, very static, so it wouldn't have any sort of interactivity almost at all. And at the end of the day, that is exactly what it created. And if I had no technical knowledge about it and I just took the website itself just as face value, just looking at it, it would've looked perfectly fine. But as soon as you start digging just even a little bit into either the code or even the features that it did present out, you would know that, hey, a lot of these things that it did in the, you know, maybe like a button. I remember there was a button that you need to make to actually contact somebody or to sign up for the service. That needs to be connected to so many different things that is so much more complicated than just making a website that somebody can see and that can s- somebody can just scroll on. You know, there's like the database component of it. There's a lot of the technical aspects of communication and privacy about it, how are you gonna store things privately. And I think that that is something that is really, really valuable to kind of think about when you are using these tools like AI. You know, you have to understand the limitations of it, as well as you have to understand the limitations of yourself, as well as you have to understand the limitations of the project that you are working on or what you want to accomplish. And by kinda having that holistic viewpoint of it, not only can you get the most out of the AI tools that you're gonna use, but you can also get the most out of your own thinking process as well, because- You will know at the end of the day that you are in control as well as you know exactly what's going on and you're not just kinda handing off your thinking. And I think that that is the, like we said, the most valuable part of learning with AI, which is you yourself get a better idea. You can train yourself basically to get a better idea of what you can do and what you need help with, which is at the end of the day, I think is the coolest thing about working with something as cool as AI.

graham-wolfe_1_02-27-2026_140547

Yeah, that makes a lot of sense to me. And I think when we talk about this pivot from the hard skill, as some would call it, of coding to soft skill, as some would call it, of evaluation and, and validation. That sounds like it's a lot more accessible and in a lot of ways it probably is, and, you know, lower barrier to entry to, to being an evaluator of output than it is to being a coder. I think just hearing you kind of walk through that process that you went through in, in, in even just a very small project, Matthew, it's a reminder that it can be very sophisticated. This, this knowing what you want and, and, tailoring these tools to, to give it to you, and then, you know, validating and holding hands with it along the way can be a very sophisticated process. And when done right and when done valuably, it certainly is. yeah, I don't know. It's, it's reassuring in that way and it's, it's, it's fascinating and like you said, I think we're sort of on the forefront of it as, as these models are getting better and, and, perhaps, you know, they continue to, they will continue to get better for the next 10 years and then we'll turn into paperclips.

Closing Thanks and Wrap

graham-wolfe_1_02-27-2026_140547

But, yeah, I think that is an excellent place to kind of wrap up this conversation. but yeah, once again, thank you so much Bill for being on. we really appreciate your time and your, and your talents and your perspectives. Bill Tyson, thank you.

bill-theisen_1_02-27-2026_140547

Thank you.

matthew-chou--he-him-_1_02-27-2026_140547

you so

bill-theisen_1_02-27-2026_140547

Thank you.