To teach in the time of ChatGPT is to know pain
- Thread starter JournalBot
- Start date
Voix des Airs
Ars Praefectus
Let's think through the consequences of this. (And I am posting this from the perspective of someone with first-hand experience with what I'm about to talk about, so this isn't speculative.) Suppose Squiggit University stops enforcing rules involving cheating, and cheaters wind up with good grades and honors diplomas from Squiggit regardless of their actual talent and skills. How will this play out?
Employers notice Squiggit grads actual skills don't match their degrees and transcripts. And they will notice this. So they stop interviewing them because they don't trust Squiggit credentials.
[Background: Before I went back into academia I worked for a large, well known and well regarded technology company (not a "tech" company, a company that actually made physical stuff). After submitting a resume and transcript as a student I was selected for an on-campus interview. About a month after that I was contacted by HR from one of the company's divisions and offered a "plant trip" to interview for a position there. I accepted. The on-site interview was entirely with scientists/engineers and a couple of project managers. It lasted two full days, with one interview after another. It was hard and grueling but a week later one of the managers called me and offered a job, which I took. After a couple of years working there I became a member of the "interview committee" and started interviewing prospective hires on-site myself.
What I learned from all this is that the company had both company-wide and local knowledge regarding which school produced good candidates and which didn't and from that strongly favored universities that it had a good track record with. This was based on track record and not brand name, there were schools which were highly favored that you would likely not think of as being among those; and schools that you would think, based just on name alone, would be highly favored but in fact were disfavored. And this wasn't just a matter of technical skill, people, especially talented people, don't as a general rule want to work with cheaters (basically liars) - it creates a bad and hostile work environment. To be clear, none of this was a result of some kind of virtue or fairness on the employer's part, it was a result of the screening and interview process being very expensive and time consuming. The company didn't want to waste time and money on a candidate that was unlikely to make it through the process. And this was basically the rule through the entire industry - again, money, not virtue.
Then during the post-interview debrief process that the interviewers would go through the day after the interview, mention would be made of things like "you know, we always seem to get good results from this school" or "these guys always look good on paper but kinda tend to suck". And that information would work its way all the way up to corporate HR and hiring.
So suppose industry knows that University A has a strong honor code that it strictly enforces and Squiggit University doesn't and doesn't care about cheating. An employer gets equally strong resumes and transcripts from a student at University A and Squiggit. Which one do you think the employer is more likely to further pursue? I know first hand from my days involved in such things that the Squiggit student is going to get a polite "No thank you and we wish you the best" letter. Over time potential Squiggit students are going to start to realize that they don't have a very good opportunity to actually get a job as a Squigget grad... regardless of how good and honest they actually are. This situation is not good for anybody, not the least of which Squiggit.]
Last edited:
Upvote
11
(12
/
-1)
I think requiring people to take the courses is not necessarily a bad thing so long as they have a certain amount of freedom and choices to find something interesting (Don't like 18th century poetry? How about movie appreciation? Or classic sci-fi?, comic books?, all actual options when I went to school). The bigger issue is when those electives (and major courses for that matter) involve a ton of make work.
I have an engineering degree and minored in math and history just because I wanted to. The worst elective course I had (and in the bottom 3 for all my classes) was an anthropology course that I otherwise would have found fascinating, but I had to write a paper for every week and quickly discovered that if you didn't write to the professor's biases you'd get a shitty grade. On the other hand I had several history classes that I loved because the class was basically a discussion session and we only had a couple of papers and tests and they were written to encourage you to think about things.
Upvote
7
(7
/
0)
My problem when I was in college was that I could struggle to complete all the homework and get worse grades than many students, yet get the highest scores in the class on some tests. It was already the case that because of the popularity of plagiarism, collaboration, homework help services, etc, the average student completed homeworks which would take too much time to do properly on their own. Not because they were plagiarizing most of the assignment, but because every time they got stuck, these resources got them un-stuck in much less time than it actually takes to figure something out on your own.
Your intuition is right; with resources like sparknotes or chegg, as well as various organized "tutoring" groups that often taught using previous semesters' tests and assignments, it already got to the point where working honestly would lower your grades a noticeable amount on average - except when it came to test time, where it was harder to cheat. So even before AI it was hard; Now with LLM's I have to agree with your last couple paragraphs.
Upvote
3
(3
/
0)
But the point isn't to give up on stopping cheating, it's to create an environment that makes cheating harder organically and encourages students to learn, instead of building hostile environments that punish students for trying to learn and emphasizing assessment systems that are built to be gamed.
I can't even imagine how anyone who's taken a college class or helped build one in the last two or three years can honestly act like the status quo is good.
Upvote
3
(3
/
0)
How did you turn "multiple choice tests are bad and university classes are often wildly hostile to learning" into "there should be no tests" ???
If you care about an indicator of whether someone has learned anything the current paradigm is objectively bad.
Upvote
0
(1
/
-1)
Your course isn't standalone, it's taken for CREDITS as part of people's DEGREES. Your grades do matter for their future job prospects, and if you design things such that people with the ability (if it gets expensive) and the willingness to use LLMs can get full marks while people who actually do the work get worse marks? You're part of the problem, and your hands aren't clean. I paid my instructors not just to give me the chance to learn something, which I could have done elsewhere for far cheaper. I paid extra because I wanted the piece of paper at the end, and you have no problem with the idea of making my investment worthless.
Upvote
6
(6
/
0)
But you don't know the work until you do the work. And you can't do the work while talking. Or at least I can't. I guess you can, and it's normal in your world. Thanks for explaining.
To me it is like saying that a writer should be able to improvise a great story out loud, which no doubt some can, but it's just not their actual job.
I get that. Communication is key outside of academia as well. It's just very different from the work itself.
Also presentations are notoriously useless where I work, but maybe they mean more for you.
Last edited:
Upvote
1
(2
/
-1)
Even two cameras (and 'two reliable windows or apple devices with cameras' is already a tough ask for poor students) and the requirement to spin one around in a circle doesn't prevent you from having something out of sight. And to ensure one computer isn't a problem it requires you to install their draconian spyware onto your computer. which of course I would never agree to do to a computer I used for anything but school. And even some of the school's own software might set it off.
Upvote
4
(4
/
0)
Reading this, sadly, makes me feel some relief that I left college teaching in 2024.
Upvote
1
(1
/
0)
Let me guess, you were told that schools are bad by your political party, and now you hate schools (and teachers). That makes you a perfect tool.
I’m seeing claims that Universities hand out degrees to everyone these days, which is BS in my experience. When I graduated, only 15% of the students ever enrolled in our engineering program had received a degree. If you’re below a certain GPA they will cut you, but if you’re above that you can spin your wheels wracking up debt for many years, but if you can’t complete the requirements, you get nothing. This is for an accredited University obviously, not some scam like Trump University.
Upvote
16
(16
/
0)
To be fair, I've had professors like that inside the technical department. Shit professors are in every department sadly.
If anything, engineering students do not have nearly enough electives in the curriculum. It would need to be a five-year degree to do so, unfortunately. Engineers need to understand things beyond the equations and designs, because engineering has a significant effect on society.
University can't be the only place you get it from, but it can help to start to broaden people's horizons and make them better engineers and citizens.
Upvote
9
(9
/
0)
Even in controlled environments this sort of software tends to be fragile and present problems pretty regularly.
On top of that, the university couldn't afford the storage for this sort of operation.
Upvote
5
(5
/
0)
I dislike generative AI so much now. The South Park satire has become reality. I have made a promise to myself that I intend to keep forever: I will not use this to offload any of my own creativity.
As an adult in middle age, and as someone who didn't like school, I now understand the value of "education". It was a roadblock back when I was in my teens, something that got in the way of things I would have rather been doing, like looting corpses in Diablo 2, or playing Final Fantasy 7 for hours.
All of us now have anecdotes about younger people being stuck, any time a situation arises where some problem solving skills are needed. I see it too, I see it.
People that offload all their thinking, learning, and creative muscles to computers will be more and more obvious as the years go on.
As an adult in middle age, and as someone who didn't like school, I now understand the value of "education". It was a roadblock back when I was in my teens, something that got in the way of things I would have rather been doing, like looting corpses in Diablo 2, or playing Final Fantasy 7 for hours.
All of us now have anecdotes about younger people being stuck, any time a situation arises where some problem solving skills are needed. I see it too, I see it.
People that offload all their thinking, learning, and creative muscles to computers will be more and more obvious as the years go on.
Upvote
9
(9
/
0)
Edit, saw a followup where it may well be correct that the rate was 4.
Your examples of things they spend that money on is relevant, but you also leave out the fact that office work has much more productivity per dollar, at least operationally, than it used to. We can move electronic paperwork around far faster and cheaper than we ever moved physical paper - even if we now have much more of it, to compensate. And schools spend tons of money on things that only look good in tours and brochures, or only help athletics, or are just bloated expensive ways to do things because nobody has time to do them cheaper but they have enough money to buy whatever some company is selling. But that's another argument.
Upvote
3
(3
/
0)
Except, you know, they're not.
People who would willingly rely on AI to get good grades and do their coursework are the types of people who will, by virtue of their academic record, subsequently occupy bureaucracies (e.g. all levels of government, insurance companies, etc.) in administrative positions (typically quite well compensated) and the rest of us -- the teachers, the technologists, the "doers" of the world -- will have to rely on them for anything to do with those organizations.
What a sad future awaits us.
Very thought-provoking article (and very well-written), @ScottJohnson. Thank you for putting in the real effort to produce this.
Last edited:
Upvote
7
(7
/
0)
Make financial aid qualifications and revocations clearly contigent on course work authenticity. Whatever that means is up to the institution, as enforcement in the current climate would be top comedy. Certain places with reputations to keep would notionally promote their enforcement as your classic "our students excel through true rigor" pablum. Unfortunately, with all campuses having hollowed staff ranks due to extreme overindulgence on the gig worker adjunct rungs, it's far more likely that LLM'd undergrads will swim with LLM'd professors with the understanding that everyone has given up so judge not lest ye be judged.
Upvote
1
(1
/
0)
Apologies - I misread that trying to respond quickly between classes (the tuition costs at our institution are also a good bit lower than that even before you play the "net of waivers/scholarships" game, so it looked odd). The other paragraphs remain true - especially the massive state disinvestments around the 2008 financial crisis.
Upvote
3
(3
/
0)
I would have no problem with a student doing some work silently for a while, then pausing and explaining the thought process behind what they've done so far, and what they're going to do next. Some people can do the "chalk and talk" together; cognitively, it might be better for both the student and the examiner to alternate between modes, though.
The same could be said of class lectures and conference presentations: there's good, there's bad, and then there's downright ugly.
I don't know if you remember that fad for talking about "death by PowerPoint" back before Covid hit? There was a lot of pontification about both that and "the death of the lecture".
The bottom line is, we need to train people specifically on how to create and deliver effective presentations; if we don't, we can't really complain about the results.
I could rant for hours about this, but it would make for a really lousy presentation...
Upvote
5
(5
/
0)
What did the instructor and/or department chair have to say when you brought this to their attention?
Upvote
0
(0
/
0)
I’ve had luck building out Human + LLM learning infrastructure, but unless i have absolute control and good infrastructure, it’s all a non-starter. One of the biggest challenges in teaching, one that Id gotten pretty good at, was reeling in the rope students use to hang themselves. Reeling in that rope now requires extreme measures, often measures that administrators are just not willing to assist with and are often actively hostile to.
What’s bizarre is that nobody seems to get it or care. I can feel the earth shifting under my feet, but the administration and half the professor core refuse to rise to the challenge. The students are doomed. Which means in the long run, the people running the joint are doomed.
Upvote
4
(4
/
0)
I saw this got downvoted into oblivion, but FWIW I agree with you. The problem is that adapting is going to take some rather extreme and disruptive measures that people are not willing to accept. And by people I mean admin, professors, and students.
If any of us are going to make it out the other side, it’s going to require a social contract where everyone agrees to step up. I’m down for that, but it’s going to mean bringing effort and a lot of pain down on a lot of people.
Upvote
3
(3
/
0)
The issue I'm having with LLM's is, tools and better tools improved the production of something, be it a chair or software programme. LLM's claim to to "improve" the research and development of something. And while new tools might also have removed some skills and some work, LLM's will destroy skills and remove people from the craft they should really learn. So a lot of knowledge will be moved from people to LLM's. In this case an LLM will become a weird machine or a black box, something we might not be able to understand or repair in the future.
Upvote
5
(5
/
0)
Lewis Black segment Back In Black did a segment about AI and LLM. It seems for college students these days, it's "expected" that they're using these sorts of things (to the chagrin of many around them). We always knew the internet would get worse after it got better. I guess that day has arrived ![:\](https://cdn.arstechnica.net/civis/data/assets/smilies/slash-1x.png "Slash :\")
Upvote
1
(1
/
0)
The difference between earning a degree and being granted one is something to be pondered, particularly by those who see education as a means to an end instead of an end in itself.
Upvote
5
(5
/
0)
Employment is a means to an end (selling time for money), and getting an education from a modern expensive university is too - if you just wanted to spend a few years learning, you'd do it somewhere cheaper, and you wouldn't ask to be graded or given a piece of paper to prove you passed. Even if there was something uniquely valuable at the university you couldn't get anywhere else, you still only complete all the requirements for a multi-year degree because you want the piece of paper that says you did - otherwise you'd just take the classes you wanted and needed and leave the rest alone.
Upvote
5
(5
/
0)
Life's too short to think of how you spend the time we have here that way.
YMMV, of course.
Upvote
0
(0
/
0)
This makes sense.
Yep, I will never forget old Tufte. And I'd say it still applies to the business world-- everything is still a powerpoint, and powerpoints are still hot garbage, now with 70% more genai. Sometimes they are necessary. But usually, I get more from just reading the study, memo, code, model, etc. (That's not to say that talking is useless. Talking is nice. You get to smile and laugh.)
I think? academic lectures are a lot less annoying, and a lot more necessary. Though I couldn't tell you how much. Certainly we need teachers who can talk.
Upvote
2
(2
/
0)
This will be absolutely disastrous for elementary school students who will grow up not knowing how to think or do their own work.
Upvote
5
(5
/
0)
Funny enough, not too long ago I had to set up cameras like that for radiology fellows doing some certification exam online. We used what computers we could find, mostly just empty seats but actually borrowed an attending personal computer (and office) or two. I forget if we had to have two cameras or just the one to the side, but we had to buy extension cables and stands for the ones with the side view to get the distance they wanted. Then we had to call in within a day or two prior to the test to get their confirmations the setup was satisfactory.
Upvote
1
(1
/
0)
Proctored exams. Frequent, in class quizzes. In an ideal world, pre-imaged thin clients with special bespoke tooling and no access to the internet. (Honestly, you could manufacture such a device for $50 if you had the will to mass deploy it.)
In class: mandatory discussion and randomized cold calling of students in discussion sections. No exceptions. Flipped classrooms where students had to pre-read, pre-practice, pre-prepare in order to work problems in real time.
Outside of class: Rapid, timed responses of LLM generated questions about their essays, where the LLM had been given sufficient pre-compute time to generate simple, factual questions. Allowed challenges for evaluation by human graders that result in deductions the challenge fails. No exceptions. A massive increase in the buildout and adoption of remote, proctored testing. Ultra short testing windows. No exceptions.
Upvote
0
(0
/
0)
Here’s the deal. You don’t need to go to college. Nobody does. Whilst it’s true that a lot of university and public policy types have forgotten that, I haven’t.
College is supposed to be hard. It’s supposed to separate the people who want to think from the people who don’t. That was the original social contract and I’m perfectly fine re-enforcing that. If it turns out that certain swaths of the employer class who long ago found out sorting these people to the top of the hiring pool was useful still find it useful, I’m also fine with that. If they don’t, welp, I can’t stop them. But I kinda suspect they will.
Upvote
6
(7
/
-1)
i bet the real problem is that people don't want to be there. they are there because they need the document so they get a checkbox so they can get that high paying job that is really something boring and simple like poking excel all day despite requiring that long degree.
Upvote
3
(3
/
0)
Sure, but they weren't watching cartoons in the 1980s - I was. They were watching Jimmy Swaggart, believing every word, etc. Even now, "reformed" Jim Bakker is one of their biggest heroes. And of course, he says "Trump is a christian", etc.. Fox watching came much later, AFAIK (I moved across the continent ~20 years ago). They have never told me exactly which Fox show they watch - they just call it "their little show", so they do realize I would disapprove. From various hints, etc - it's Hannity. And my BIL who lives in the same town is a Tucker disciple.. FML.
Upvote
-1
(0
/
-1)
Well you are learning how to construct a good prompt, Trust me in directing the ai to change code that really matters.
As the last of the baby boomers we where not quite affected by calculators, cheap calculators with scientific functions arrived in 1977 and where not allowed in exam til perhaps mid 80's
http://www.arithmomuseum.com/album.php?cat=c&id=247&lang=en I had a fx81 purchased in 1980.
But we left school fully able to solve pretty much any practical problem from maths with just a peice of paper and working behind a busy store counter, I could do in my head some like 3 x 0.50+2x0.60+$2.00 as fast as they said I want 3 shellfish and two pieces of fish and $2 chips. I can still do say 1.99+2.xx+1.50 faster then a shop assistant laboriously typing it into a calculator and a few times over recent years I have said you got your maths wrong as they typed into a register or calculator and I did it in my head.
But after 40+ years of spreadsheets and calculators long division is gone, short division and multiplication above say 20x is pretty much gone and I am definitely slower to add 5-6 3 digit numbers.
Upvote
1
(1
/
0)
I've mentioned it elsewhere: my daughter is studying CS at a junior college, transferring to a CSU school after 2 years. Right now, they're learning C++, and she seems to have a decent grasp of it. She said their prof had to make the assignments count for 0 because too many kids were using AI and leaving the AI's obvious comments / prompts IN the final submission. I.e. 0 effort. And these are a "CS cohort" - i.e. students specializing in CS. It's brutal.
In hindsight, the "2 years at junior college, transfer to a CSU" that both my daughters did (they tested out of high school after 2 years) has been somewhat of a disappointment - both the students at the junior college and now the 3rd/4th year students at CSU have been horribly unmotivated, unreliable, and basically just a shitshow - my autistic daughter has to constantly hunt down the other people in the group to get them to do the basics, and they can't even understand basic instructions or due dates. We expected 3rd/4th year students to at least be reasonably competent at a state school, but it feels like they wouldn't know how to walk to class without their phone.
AI is very, VERY bad for our intelligence levels. It's working amazingly well to create little robots that only know how to cut/paste what their magic hand-box tells them. Our daughters want more than the basics, and it's unbelievable how impossible writing 3 paragraphs seems for 4th year students.
In hindsight, the "2 years at junior college, transfer to a CSU" that both my daughters did (they tested out of high school after 2 years) has been somewhat of a disappointment - both the students at the junior college and now the 3rd/4th year students at CSU have been horribly unmotivated, unreliable, and basically just a shitshow - my autistic daughter has to constantly hunt down the other people in the group to get them to do the basics, and they can't even understand basic instructions or due dates. We expected 3rd/4th year students to at least be reasonably competent at a state school, but it feels like they wouldn't know how to walk to class without their phone.
AI is very, VERY bad for our intelligence levels. It's working amazingly well to create little robots that only know how to cut/paste what their magic hand-box tells them. Our daughters want more than the basics, and it's unbelievable how impossible writing 3 paragraphs seems for 4th year students.
Upvote
10
(10
/
0)