Give your students a new exercise: have them write 30 essays on 30 unique topics related to the curriculum in one day. The reward is one bonus point on their final grade. Tell them to use LLMs for all the essays.
Once they have turned in their essays:
1. Ask them what they learned about the topics at hand
2. Have they received value for their tuition by depending on LLMs to possess all of the knowledge and they submit it as their own, while being unknown?
The lesson: have students see how the use of LLMs undermines the value of their education. In spite of making heavy use of LLMs they have learned nothing, yet generated large amounts of written content submitted under their names. Will this help them learn about the topic? If they show up to a research lab, or a workplace, with a lot of written works in their name but no real knowledge, will they look competent in front of their peers? Will they be able to reference their published essays when asked about them in the future?
The solution to LLM use is to make kids use too much of it, so they see how truly hollow and useless it is for doing their work for them.
But how do students pass their exams nowadays?
If they don't assimilate or learn anything. How do they pass exams, where there is no outside assistance?
Wait what?! I studied between 2018 and 2021 and we had to leave our smartphones (all electronics really) in our coat/jacket/backpack and those were left either along the back wall of the exam room or on cloath hangers along the back wall.
This system worked with up to 150 people (bigger rooms than that didn't exist) taking an exam at the same time. No problem.
Do colleges even have rooms that house 1K+ people taking an exam at the same time? Honestly it sounds absurd.
Test students inside a faraday cage.Now you would have to check for Meta-glasses (what do you mean, I can't wear my glasses?!? I can't see!), smart watches, and who knows what else..
You know, the more I think about this, the more reframing Idiocracy as the result of generations of LLM abuse (rather than eugenics) works conceptually.
"Water the fields with Brawndo so the plants get enough electrolytes" could EASILY come from an LLM, could it not?
Let me guess, you were told that schools are bad by your political party, and now you hate schools (and teachers). That makes you a perfect tool.
I’m seeing claims that Universities hand out degrees to everyone these days, which is BS in my experience. When I graduated, only 15% of the students ever enrolled in our engineering program had received a degree. If you’re below a certain GPA they will cut you, but if you’re above that you can spin your wheels wracking up debt for many years, but if you can’t complete the requirements, you get nothing. This is for an accredited University obviously, not some scam like Trump University.
My day job is coding (I ... work on AI infrastructure, I come in peaceDoes work require you to understand what you're doing? SOME work does, in theory, like coding, where you need to be able to fix it if there are issues, and if you know nothing it will be much harder.
As a writer, I find the fun of a writing project is selecting the words and tone. Do I understate a joke? Do I purposely make obscure references only a few people will understand? Do I make a joke and then over explain it for comedic effect?
In the same way, I've read many programmers on Ars write about how they relish crafting tight, light code and refining it until it sings.
However, I know from experience that many people regard writing as a chore. They simply want to put words on paper and be done. They derive no enjoyment from how to link ideas and flow paragraphs. They are not interested in explaining ideas or expressing viewpoints or presenting intriguing concepts. They want to be doing anything else.
There are always people who select professions based on projected salary. Doctors and programmers have been considered high-paying for ages and so you'll find loads of people more interested in their pay packets than the work they churn out.
So seems the case for your junior. He has no desire to be a craftsman.
The article mentions so many students seem interested solely in the end result. LLM make it trivial to receive an answer-shaped output which makes me wonder where their passions lie. If they're wholly disinterested in coursework, what is it that they want to spend time on?
The one student who claimed "This is my major, I actually need to learn stuff in this class. I use AI for my other classes." stumbles into the classic trap of believing they need focus solely on their discipline and everyone else is so much distraction.
I know there's a traditional fiery hatred of humanities amongst more technical-oriented people but the purpose is to understand how everything in the world fits together. If you've only experience with people similar to yourself then it's quite easy to overlook the fact that other people have different wants and needs.
For example, in the olden days, windows didn't support multiple languages. You needed to purchase different versions of windows in different languages. Eventually, they offered regional language packs you could install but then you still had to select a main region (and restart) and depending on which, your \ could display as ¥ which is a bit off-putting. Also, if you tried to install software using code pages in a different language, it might try to generate invalid characters for folders or filenames and the installation would fail. Clearly, whoever originally designed windows never considered people might need to use more than one language at a time.
I believe myself a rather curious person. I'm always interested in watching making-of clips of films. I like to see how food is prepared. However, I'm not very interested in sport so I wouldn't at all wish to watch someone prepare for a competition or event as I don't see that having any impact on my life at all.
It's the same as with that student who uses AI for the "other" classes. He doesn't see the benefit or utility in them.
You were arguing that University lecturers should not try to to prevent cheating. The result of that is that evaluation, even if performed, does not distinguish between students who succeeded through cheating and those who did classwork as intended.How did you turn "multiple choice tests are bad and university classes are often wildly hostile to learning" into "there should be no tests" ???
If you care about an indicator of whether someone has learned anything the current paradigm is objectively bad.
Remember those people in the movie Wall-E? They were happy consumers who could barely stand on their own two feet. That's the ultimate goal!We are sleepwalking into a world where people are so reliant on LLMs that society cannot function without them. The CEOs of these company know this...
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.My day job is coding...<big snip>...
So, would I use an LLM in a mandatory class for which I have no passion? Hell yeah. All day, every day. Would I be more inclined to either discard the LLM, or try and do things I enjoy by hand for a subject which I'm passionate about? Yes, at least ... yes if I had the actual opportunity in my life to devote the required time.
Yes and no. Time-inconsistent preferences are real. Part of the art of teaching is reeling in the rope students use to hang themselves.If more courses were based entirely on a single end-of-term proctored exam taken in the classroom there'd be a lot less chat gpt use in college. Busy work is treated as such.
It sounds like you should have gone to a trade school or technically certification program, not a liberal arts University. In general, there probably are too many students going to 4 year universities and getting well-rounded but expensive educations instead of going to more focused technical and professional certification programs instead.I just read several pages of these comments, and I’m amazed, stunned even, that no one has identified the actual root cause of this behavior.
The college education system is fundamentally broken - the quote of the student who only uses AI for the classes he cares about is exactly what should be focused on. We have to ask, why would he do that?
The cost of college tuition has risen dramatically and is effectively out of reach for many more people now than it ever has been. Why? What are we paying for?
We require students to take a minimum number of “units” across a variety of topics and subjects with the assumption that they will “learn something useful” by forcing them to take classes that they don’t care about, and from their perspective are in the way of doing the thing they actually do care about - e.g their major. Why?
This article is simply more evidence that our higher education system is fundamentally broken.
I, for one, support using AI for things that are burdensome and uninteresting. Taxes. Laundry. 18th century poetry. The correct formatting of citations. Downvote me if you must, but it’s clear that most college students also agree. We have better things to do.
The same thing happens in the US too.This is a really difficult issue to solve, because when policies are put in place to address the genuine needs of disadvantaged students, those policies are often actually exploited by students who are most advantaged.
Here in NSW, we have some robust and generous policies to assist disadvantaged and disabled students. Scribes, stress breaks, additional writing time, and so on.
However, this assistance is disproportionately used by students in wealthy areas and by private schools. One elite, exclusive private school had 26.3 per cent of students claiming assistance, while in Sydney’s disadvantaged south west, only 2.5 per cent claimed similar provisions.
Wealthy, advantaged students have the means to clear the hurdles to be granted assistance. Genuinely disadvantaged students simply can’t manage to get the paperwork in order or secure appointments with sympathetic specialists to gain recommendations for assistance. It’s become a tool that actually exacerbates the gap between advantaged and disadvantaged students.
Anal beads?Now you would have to check for Meta-glasses (what do you mean, I can't wear my glasses?!? I can't see!), smart watches, and who knows what else..
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.
The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!
This is not at all what young people are told. Acting like it is doesn't make it true, nor does it help young people who could never have the worldly experience to reach this conclusion on their own when making life-altering, potentially financially ruinous decisions about their future.Here’s the deal. You don’t need to go to college. Nobody does. Whilst it’s true that a lot of university and public policy types have forgotten that, I haven’t.
College is supposed to be hard. It’s supposed to separate the people who want to think from the people who don’t. That was the original social contract and I’m perfectly fine re-enforcing that. If it turns out that certain swaths of the employer class who long ago found out sorting these people to the top of the hiring pool was useful still find it useful, I’m also fine with that. If they don’t, welp, I can’t stop them. But I kinda suspect they will.
We have a chess fan in the audience, I see...Anal beads?
Apologies - I misread that trying to respond quickly between classes (the tuition costs at our institution are also a good bit lower than that even before you play the "net of waivers/scholarships" game, so it looked odd).
The other paragraphs remain true - especially the massive state disinvestments around the 2008 financial crisis.
Isn't this just two sets of inputs rather than one? Or maybe three, if you also upload what you turned in for assignment 3?I dont really recall getting that many assignments in university: the majority was learning principles and then applying them in an examination. If I had a time travelling AI helper back then I’d only be able to influence about 30% of my total score. Not insignificant but it would be clearly suspect if I aced assignments but failed exams consistently. And I seem to recall that you had to get a passing grade on exams as well as overall to get that module.
one possible defense against excessive AI dependency would be to require each assignment to have a reference back to a previous assignment, I.e. “consider how this differs from the area reviewed week 3”. That is much harder to feed into a machine to write as it doesn’t know what week 3 was, unless you pay for a lot of tokens.
either you have a job already lined up wherever your parents work, or you're going to be screwed by the system anyway and should maximize your GPA. I'm 10 years out of school and back then half my class either had jobs at their parents companies lined up, or stark unemployment and a career of under employment in their future.The problem of Fake It till you Make It, is that if you never have ever learned how to actually make it, your always just going to be a fake, and people will eventually figure that out.
I feel Hollywood Sci-Fi has set a, while story expedient, comically tropey bad example where the supposively very intelligent character simply asks the computer for the answer to a difficult question. Star Trek/Galaxy Quest
I get and sympathize with the sentiment, but you probably meant "make me want to launch myself into the Sun".
The problem with LLMs is that they take the old "they wouldn't print it if it weren't true" argument and amplify it by a million. Not only are they untrustworthy plagiarism machines, they also tend to lead people to abandon critical thinking and assume that the LLM output is correct. (It often isn't, as you well know.)
About the only way to kill this is to have the students ask the LLM an in-depth question about something that they know very well, just so they can see how wrong the LLM often is. And I fear that even that isn't enough.
ETA: In many subjects, there are misconceptions that can act as a block to learning. I am willing to bet that LLMs would get the answers to those misconceptions wrong in many cases, which suggests basing homework assignments and tests on misconceptions (which is where they really should be, honestly).
Yes and no. Time-inconsistent preferences are real. Part of the art of teaching is reeling in the rope students use to hang themselves.
This would work, but it would also result it a shit-ton of Fs. Then again, if the students are ready and willing to sign up for this, a lot of professors are more than ready to oblige. Say the word.
No, I argue the current university culture obsesses over cheating to the detriment of learning and that many aspects of modern class design incentivize gaming the system in various ways.You were arguing that University lecturers should not try to to prevent cheating. The result of that is that evaluation, even if performed, does not distinguish between students who succeeded through cheating and those who did classwork as intended.
I was only replying to that point, not anything else you are claiming.
I've mentioned it elsewhere: my daughter is studying CS at a junior college, transferring to a CSU school after 2 years. Right now, they're learning C++, and she seems to have a decent grasp of it. She said their prof had to make the assignments count for 0 because too many kids were using AI and leaving the AI's obvious comments / prompts IN the final submission. I.e. 0 effort. And these are a "CS cohort" - i.e. students specializing in CS. It's brutal.
In hindsight, the "2 years at junior college, transfer to a CSU" that both my daughters did (they tested out of high school after 2 years) has been somewhat of a disappointment - both the students at the junior college and now the 3rd/4th year students at CSU have been horribly unmotivated, unreliable, and basically just a shitshow - my autistic daughter has to constantly hunt down the other people in the group to get them to do the basics, and they can't even understand basic instructions or due dates. We expected 3rd/4th year students to at least be reasonably competent at a state school, but it feels like they wouldn't know how to walk to class without their phone.
AI is very, VERY bad for our intelligence levels. It's working amazingly well to create little robots that only know how to cut/paste what their magic hand-box tells them. Our daughters want more than the basics, and it's unbelievable how impossible writing 3 paragraphs seems for 4th year students.
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.
The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!
I would call that an "other effect."Except, you know, they're not.
People who would willingly rely on AI to get good grades and do their coursework are the types of people who will, by virtue of their academic record, subsequently occupy bureaucracies (e.g. all levels of government, insurance companies, etc.) in administrative positions (typically quite well compensated) and the rest of us -- the teachers, the technologists, the "doers" of the world -- will have to rely on them for anything to do with those organizations.
What a sad future awaits us.
Very thought-provoking article (and very well-written), @ScottJohnson. Thank you for putting in the real effort to produce this.
is your screen name indicative of where you grew up or just your ethnic background? My dad, funny enough, grew up in the Netherlands and I also have a lot of colleagues with ubdergrad degrees from European universities. They have no problem in most of those university systems handing out Fs and making people retake classes.Imho, there's too much effort and energy being expended by teachers and professors nowadays to drag unwilling students kicking and screaming to anything other than an F. IMHO providing the motivation for a student to put in the most basic of work is not a teachers job. It's the students (or maybe their parents if they're a minor). Don't do the work, get an F. I've had plenty of subjects in school and university that I had little interest in, but the motivation to still get a decent grade was entirely mine. Not that I always succeeded...
It has been devolving into that for decades. No LLMs were required.Maybe university is destined to devolve into remedial education and credentialism.
That's a rich man's perspective. A person making six figures and blowing it all on nonsense, then working overtime to try and earn even more? Sure, tell that to them, that's good advice for them.Life's too short to think of how you spend the time we have here that way.
YMMV, of course.
yes. And also where I currently live and work.is your screen name indicative of where you grew up or just your ethnic background?
I think about this a lot.1) Students are really bad at determining what is irrelevant for them in the future. Something that seems unrelated now becomes the edge that lands you the better job later
2) an education is different than training (though the distinction has been blurred by politics and school administrations). Training prepares you to do one particular kind of job; an education broadens your flexibility and adaptability because we all know the world and job markets are not nearly as certain as they were in the 50s and 60s where a high school degree and some job training set you up for life.
Have you ever worked inside a Faraday cage built to keep out cell phone and WiFi frequencies? I have, and it was very uncomfortable. I can't imagine building one big enough to accommodate more than a couple of students at a time, never mind 30 HS students or the 2k taking first year life science prerequisites.Test students inside a faraday cage.
The tendency of smart professionals whose field requires substantial investment of focus and learning to assume that expertise extends into areas they couldn't be bothered to care about in school is widespread, to the extent of infecting many of us in these forums, in fact. I don't consider myself immune to it, and do try to remind myself periodically the line between ignorance and knowing enough to be dangerous is super-duper thin and nigh invisible.I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.
The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!