To teach in the time of ChatGPT is to know pain

SojournerDeer

Smack-Fu Master, in training
53
Subscriptor
My general observation is that AI has been transformative in a positive way for those that have good attitude towards learning or have already developed the fundamental knowledge to leverage on it to go further. It's a drug that can be abused or do much good.

On the test better front, I think most educators are bringing a knife to a gun fight. Notwithstanding misgivings about AI companies, if AI is not used during evaluation, they are bound to lose. I don't mean using AI to detect AI (not mature enough) but using AI to enable resource intensive evaluation approaches that is unfeasible previously, e.g. individualised oral defense of submissions. Unfortunately, for specific scenarios such as the authors, it will be quite challenging.
 
Upvote
-6 (2 / -8)
Give your students a new exercise: have them write 30 essays on 30 unique topics related to the curriculum in one day. The reward is one bonus point on their final grade. Tell them to use LLMs for all the essays.

Once they have turned in their essays:

1. Ask them what they learned about the topics at hand
2. Have they received value for their tuition by depending on LLMs to possess all of the knowledge and they submit it as their own, while being unknown?


The lesson: have students see how the use of LLMs undermines the value of their education. In spite of making heavy use of LLMs they have learned nothing, yet generated large amounts of written content submitted under their names. Will this help them learn about the topic? If they show up to a research lab, or a workplace, with a lot of written works in their name but no real knowledge, will they look competent in front of their peers? Will they be able to reference their published essays when asked about them in the future?


The solution to LLM use is to make kids use too much of it, so they see how truly hollow and useless it is for doing their work for them.

Does work require you to understand what you're doing? SOME work does, in theory, like coding, where you need to be able to fix it if there are issues, and if you know nothing it will be much harder.

But what about an advertising campaign? If it works, who cares if you understand anything about it? Maybe the LLM found some weird correlation between people who buy lawnmowers and then need prosthetic feet 2 months later.. And that's the issue - there are jobs where just "finishing" the paperwork / following the script is all that's required. But those people ALSO want to live and breathe and eat. And they should be able to. We should NOT require a top-10%-IQ (based on today's results) just to be allowed to live a reasonable life.

I DO get what you're saying, and I agree in general - just that I don't think that the number of people who "understand" their job fully correlates to "having a job", and most people only care about "having a job", in the end. I do coding because it pays well, and I often enjoy it. But I would do other jobs that paid similarly, if it would keep a roof over my head. Heck, I'd happily report the outputs of a coin flip 100 times a day, or a magic 8 ball, if that would help me provide for my family.

Now, given a choice, I prefer the coding job (most days) and the chance to learn, and learn and learn, because I love reading about particle physics, etc.. But that same report would put my wife into a coma, along with many other people. They just don't care. eg: my dad would happily re-watch the same lion/wolf/bear documentary for the 97th time before watching even 5 minutes of Lord of the Rings.
 
Upvote
3 (3 / 0)
But how do students pass their exams nowadays?

If they don't assimilate or learn anything. How do they pass exams, where there is no outside assistance?

They'll find ways to sneak AI into their exams.

I remember one time, in a former life when I was a lecturer at a somewhat-known university. One of my students, a few minutes into the exam, took out their blackberry and read a message. I said "Ummm" and he broke down. He explained he was so used to always responding because he was an intern for blackberry at the time (this would have been ~2002) and he was just on auto-pilot. Given that he had it out for about 5 seconds, and it was really early in the exam, I just ignored it, because I couldn't imagine he had been able to read much, even if he had tried to cheat and I knew him as a hard worker / bright student. And he was sitting 1st or 2nd row - not exactly the place cheaters tend to sit.

Not sure if I could do the same if someone pulled out a phone during an exam today. I would imagine most exam policies are "phone = automatic fail" or "leave all phones in this box".

I had two brothers (twins, I believe) clearly passing notes / pointing to each other's papers during a midterm once that I was proctoring. They both failed pretty badly in spite of it (in the 25-35% range, IIRC), so I told the prof, but we didn't take it any further.
 
Upvote
1 (1 / 0)
Wait what?! I studied between 2018 and 2021 and we had to leave our smartphones (all electronics really) in our coat/jacket/backpack and those were left either along the back wall of the exam room or on cloath hangers along the back wall.

This system worked with up to 150 people (bigger rooms than that didn't exist) taking an exam at the same time. No problem.

Do colleges even have rooms that house 1K+ people taking an exam at the same time? Honestly it sounds absurd.

Now you would have to check for Meta-glasses (what do you mean, I can't wear my glasses?!? I can't see!), smart watches, and who knows what else..
 
Upvote
4 (4 / 0)
You know, the more I think about this, the more reframing Idiocracy as the result of generations of LLM abuse (rather than eugenics) works conceptually.

"Water the fields with Brawndo so the plants get enough electrolytes" could EASILY come from an LLM, could it not?

Pretty sure it's all over the training sets by now!
 
Upvote
1 (1 / 0)
Let me guess, you were told that schools are bad by your political party, and now you hate schools (and teachers). That makes you a perfect tool.

I’m seeing claims that Universities hand out degrees to everyone these days, which is BS in my experience. When I graduated, only 15% of the students ever enrolled in our engineering program had received a degree. If you’re below a certain GPA they will cut you, but if you’re above that you can spin your wheels wracking up debt for many years, but if you can’t complete the requirements, you get nothing. This is for an accredited University obviously, not some scam like Trump University.

The manosphere's alt-right bent will be the death of us all..
 
Upvote
3 (3 / 0)

doubleyewdee

Ars Scholae Palatinae
841
Subscriptor++
Does work require you to understand what you're doing? SOME work does, in theory, like coding, where you need to be able to fix it if there are issues, and if you know nothing it will be much harder.
My day job is coding (I ... work on AI infrastructure, I come in peace :))

Even in that vein the work itself varies wildly in terms of how much I need to concern myself with the quality, veracity, and suitability of the LLM output.

I rewrote our (sadly bespoke) infra-as-code template tooling/mini-DSL over the last ~month to fix several longstanding issues, improve performance, enhance the user experience during validation, etc. I did use Opus 4.6 a great deal in this effort, particularly at the beginning, but by the end I had touched ~95% of all net new code in some fashion. It's very awesome to watch an LLM just blit out an entirely correct AST parser and consumer while you go get a cup of coffee. It does feel transformational to the career. Of course, I then reviewed it, found a variety of testing gaps, found areas where my specification was either wrong or ambiguous, etc. Thorough reviews are the order of the day here.

On the other hand, I just slapped together a thing for my team to do some API calls to keep our CI path filters for build validations synched across N branches and M build tasks (embarassing that I find myself having to write such a banal, seemingly commonly desirable tool). That code needed considerably less hands-on, deep inspection. For one thing, it's a much simpler subject matter. That aforementioned template tooling runs on many thousands of CI runs every day, plus developer workstations, and of course we use it in many different ways. Getting another 5% performance gain from it matters. Keeping strict compatibility with the labyrinthine nigh-unreadable previous implementation is paramount. The tool that runs once every couple months on my MacBook to update our path filters? Completely different measures for quality.

I have a lot of empathy for the educators fighting the LLM insurgency because, genuinely, they're doing what they should be. However, I also have a lot of empathy for the people for whom education in a specific subject matter is perceived more as an obstacle they must surmount rather than an opportunity for growth. So much of my own schooling genuinely felt pointless at the time and, frankly, a lot of it was pointless in hindsight. Mandatory classes in things which, to this day, have never served me. Being dragooned into a typing class when I was already outpacing the instructor and every other student was absurd. Dragging myself through classes where the desired outcome was not improving my ability to apply gained knowledge but rather to recite memorized things or repeat application of the same formulae is an experience that many students have felt. The teachers are not to blame here, they're working inside a system with perverse objectives that work against the core concept of educating students. However, the students can't be blamed for looking around at this and thinking to themselves "why should I give a solitary shit about this if the AI gets me back to things I find meaning in faster"?

And yes, kids love absurd, seemingly imbecilic and infantile crap. I just don't think that's especially novel. I loved (and still do love) things which are almost certainly not what you'd classify a "good use of one's time." I think having the space to be inefficient and a bit feckless at times is an empowering and necessary part of growing up, and of being a human in general. Foisting a culture of "24x7 grindset" upon people leaves many people ... dissatisfied, to say the least.

Our society in places seems structured to pulverize people in the mill of grading, testing, ensuring an immaculate and entirely full dawn-to-dusk schedule of "things which will look good on your entrance exam," or "professional networking and socialization," or whatever mechanism to "prove" one is a "useful member of society," and to me it's no wonder those same people for whom these things are just not their passion are going to use any and every available tool to lose less of themselves to these sometimes soulless, unfulfilling drudgeries.

So, would I use an LLM in a mandatory class for which I have no passion? Hell yeah. All day, every day. Would I be more inclined to either discard the LLM, or try and do things I enjoy by hand for a subject which I'm passionate about? Yes, at least ... yes if I had the actual opportunity in my life to devote the required time.
 
Upvote
1 (3 / -2)
As a writer, I find the fun of a writing project is selecting the words and tone. Do I understate a joke? Do I purposely make obscure references only a few people will understand? Do I make a joke and then over explain it for comedic effect?

In the same way, I've read many programmers on Ars write about how they relish crafting tight, light code and refining it until it sings.

However, I know from experience that many people regard writing as a chore. They simply want to put words on paper and be done. They derive no enjoyment from how to link ideas and flow paragraphs. They are not interested in explaining ideas or expressing viewpoints or presenting intriguing concepts. They want to be doing anything else.

There are always people who select professions based on projected salary. Doctors and programmers have been considered high-paying for ages and so you'll find loads of people more interested in their pay packets than the work they churn out.

So seems the case for your junior. He has no desire to be a craftsman.

The article mentions so many students seem interested solely in the end result. LLM make it trivial to receive an answer-shaped output which makes me wonder where their passions lie. If they're wholly disinterested in coursework, what is it that they want to spend time on?

The one student who claimed "This is my major, I actually need to learn stuff in this class. I use AI for my other classes." stumbles into the classic trap of believing they need focus solely on their discipline and everyone else is so much distraction.

I know there's a traditional fiery hatred of humanities amongst more technical-oriented people but the purpose is to understand how everything in the world fits together. If you've only experience with people similar to yourself then it's quite easy to overlook the fact that other people have different wants and needs.

For example, in the olden days, windows didn't support multiple languages. You needed to purchase different versions of windows in different languages. Eventually, they offered regional language packs you could install but then you still had to select a main region (and restart) and depending on which, your \ could display as ¥ which is a bit off-putting. Also, if you tried to install software using code pages in a different language, it might try to generate invalid characters for folders or filenames and the installation would fail. Clearly, whoever originally designed windows never considered people might need to use more than one language at a time.

I believe myself a rather curious person. I'm always interested in watching making-of clips of films. I like to see how food is prepared. However, I'm not very interested in sport so I wouldn't at all wish to watch someone prepare for a competition or event as I don't see that having any impact on my life at all.

It's the same as with that student who uses AI for the "other" classes. He doesn't see the benefit or utility in them.

How did you turn "multiple choice tests are bad and university classes are often wildly hostile to learning" into "there should be no tests" ???

If you care about an indicator of whether someone has learned anything the current paradigm is objectively bad.
You were arguing that University lecturers should not try to to prevent cheating. The result of that is that evaluation, even if performed, does not distinguish between students who succeeded through cheating and those who did classwork as intended.

I was only replying to that point, not anything else you are claiming.
 
Upvote
2 (2 / 0)

richardbartonbrown

Wise, Aged Ars Veteran
112
Subscriptor++
We are sleepwalking into a world where people are so reliant on LLMs that society cannot function without them. The CEOs of these company know this...
Remember those people in the movie Wall-E? They were happy consumers who could barely stand on their own two feet. That's the ultimate goal!
 
Upvote
2 (2 / 0)

richardbartonbrown

Wise, Aged Ars Veteran
112
Subscriptor++
My day job is coding...<big snip>...

So, would I use an LLM in a mandatory class for which I have no passion? Hell yeah. All day, every day. Would I be more inclined to either discard the LLM, or try and do things I enjoy by hand for a subject which I'm passionate about? Yes, at least ... yes if I had the actual opportunity in my life to devote the required time.
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.

The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!
 
Upvote
11 (12 / -1)

WildGunman

Ars Scholae Palatinae
693
Subscriptor
If more courses were based entirely on a single end-of-term proctored exam taken in the classroom there'd be a lot less chat gpt use in college. Busy work is treated as such.
Yes and no. Time-inconsistent preferences are real. Part of the art of teaching is reeling in the rope students use to hang themselves.

This would work, but it would also result it a shit-ton of Fs. Then again, if the students are ready and willing to sign up for this, a lot of professors are more than ready to oblige. Say the word.
 
Upvote
1 (1 / 0)
I just read several pages of these comments, and I’m amazed, stunned even, that no one has identified the actual root cause of this behavior.

The college education system is fundamentally broken - the quote of the student who only uses AI for the classes he cares about is exactly what should be focused on. We have to ask, why would he do that?

The cost of college tuition has risen dramatically and is effectively out of reach for many more people now than it ever has been. Why? What are we paying for?

We require students to take a minimum number of “units” across a variety of topics and subjects with the assumption that they will “learn something useful” by forcing them to take classes that they don’t care about, and from their perspective are in the way of doing the thing they actually do care about - e.g their major. Why?

This article is simply more evidence that our higher education system is fundamentally broken.

I, for one, support using AI for things that are burdensome and uninteresting. Taxes. Laundry. 18th century poetry. The correct formatting of citations. Downvote me if you must, but it’s clear that most college students also agree. We have better things to do.
It sounds like you should have gone to a trade school or technically certification program, not a liberal arts University. In general, there probably are too many students going to 4 year universities and getting well-rounded but expensive educations instead of going to more focused technical and professional certification programs instead.
 
Upvote
11 (11 / 0)
This isn’t actually new. Privileged students pretty much had this access if their parents’ hired a tutor. A teacher didn’t know who wrote the essay or did the math homework. Now poor kids have that access.

As a high school teacher myself, it’s on us to give secure assessments in class. If students cheat on the homework (which they’ve always done) and do bad on the in class paper assessment, that’s on them.

But I don’t know how to deal with online courses like the author mentions.
 
Upvote
2 (3 / -1)
This is a really difficult issue to solve, because when policies are put in place to address the genuine needs of disadvantaged students, those policies are often actually exploited by students who are most advantaged.

Here in NSW, we have some robust and generous policies to assist disadvantaged and disabled students. Scribes, stress breaks, additional writing time, and so on.

However, this assistance is disproportionately used by students in wealthy areas and by private schools. One elite, exclusive private school had 26.3 per cent of students claiming assistance, while in Sydney’s disadvantaged south west, only 2.5 per cent claimed similar provisions.

Wealthy, advantaged students have the means to clear the hurdles to be granted assistance. Genuinely disadvantaged students simply can’t manage to get the paperwork in order or secure appointments with sympathetic specialists to gain recommendations for assistance. It’s become a tool that actually exacerbates the gap between advantaged and disadvantaged students.
The same thing happens in the US too.
 
Upvote
1 (1 / 0)

doubleyewdee

Ars Scholae Palatinae
841
Subscriptor++
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.

The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!

My life experience has taught me that mandatory, passionless work neither enriches me nor endears me to that subject matter. You can sit a person down in front of a task which they do not enjoy, and tell them their livelihood depends on them completing it, and they will indeed usually complete it to whatever term of satisfaction you give. Will they be enriched for it? I'm not so sure.

The very best argument for these kinds of things is that life does involve rote, dull, rigorous work to get through, so one must learn to grit their teeth and bear these things. I "learned" that a lot in school, in fact. What I did not learn, and not due to any cheating (my motto was more "I'm just not going to do this" or "I will invest in getting exactly a passing grade and nothing more"), was that it was somehow good to have a big stack of these experiences, and that it would somehow be better for my mental health to browbeat myself for hours or days on end completing tasks which held no purpose or value for me.

Very, very few people have the luxury of being able to focus exclusively on their education, or to focus solely on making it through these sorts of classes or work that they find uninteresting, uninspiring, or actively unpleasant to be in. They also need to work. Or they need to do the "pre-work" that is a lot of schooling in the US such as "extracurriculars" which young people are told are simply mandatory for them to "make it in the world." Forget whether they enjoy it or not, forget whether these things are rewarding, you're told up and down that you simply must do these things whether or not you might find them somewhere between dull and loathsome because otherwise you're ... not going to eat. You will be a pariah, a millstone dragging down your peers, a loser.

This is a big part of why people will gravitate to these shortcuts. Because the thing is, while it sounds nice to say "time is the great equalizer," it's not actually true. Time which is genuinely usable for secondary pursuits (such as deep diving into tangential educational subjects which you're required to participate in on some level but in no way enjoy) varies heavily from person to person. The time I had to focus on school work at home was less than many of my peers due to abuse issues and living in an unsafe home. That said, the time I had was also more than some of my other peers. If I could accomplish the work with a tool in less time than without, yes I would do that. I skimmed books that I didn't enjoy in order to produce book reports. I did math the way it worked for me and then went back and "showed my work" in the way the instructor demanded if that's what it took. People, it turns out, quite like tools, and quite like convenience. Getting people to voluntarily discard tool use for their own betterment requires genuine work on the part of the entity pushing for that. It requires something to be given in return, in the case of a class someone would rather LLM their way through, that's either more actual time to do the work "correctly" or more engaging material or something else.

Painting all of this with a binary and broad brush is doing a great disservice to the people on the other end of the education experience from the teachers. "It's too bad you're willing to be lazy," which is more or less how your reply came across to me, is quite lacking in empathy, ungrounded in the reality many people experience. There are many factors involved in why someone chooses to not do the assignments precisely as instructed. Sure, you can just say "that's cheating," but to me that's tantamount to levying identical scorn to a person who steals food for sustenance as to someone who has all they need and more and still robs those in less fortunate situations.
 
Upvote
-5 (1 / -6)

doubleyewdee

Ars Scholae Palatinae
841
Subscriptor++
Here’s the deal. You don’t need to go to college. Nobody does. Whilst it’s true that a lot of university and public policy types have forgotten that, I haven’t.
This is not at all what young people are told. Acting like it is doesn't make it true, nor does it help young people who could never have the worldly experience to reach this conclusion on their own when making life-altering, potentially financially ruinous decisions about their future.

I'm happy to do my part of the work to change the social contract, but where we are right now is far removed from a common acceptance (in the US) that college is this wholly optional pursuit of pure higher education rather than a gambit to gain social status.

College is supposed to be hard. It’s supposed to separate the people who want to think from the people who don’t. That was the original social contract and I’m perfectly fine re-enforcing that. If it turns out that certain swaths of the employer class who long ago found out sorting these people to the top of the hiring pool was useful still find it useful, I’m also fine with that. If they don’t, welp, I can’t stop them. But I kinda suspect they will.

Ah yes, count me in the "people who do not want to think" cohort since I did not attend college, I suppose. Certainly nothing else came in to play there! If you elect not to attend college you are simply an unthinking dullard upon whom scorn should be heaped by your intellectual betters, I suppose.

The "social contract" for university attendance has never been exclusively about this bright-line divide you're portraying. People have, for centuries, attended universities through cause of familial mandate, avoidance of military service, and myriad other reasons entirely divorced from unencumbered pursuit of thought for the sake of it. Certainly some people are there purely for the thinking, but I'm skeptical that this has hewed anywhere near a uniform population of students with that cause at any time in the last several centuries.

You yourself have laid out a case for ongoing financial incentives for those in this supposed "wanting to think" segment of humanity to attend college which undermines the more noble ideal of purely academic pursuit. Which means an ongoing system in which you incentivize attendees who do or do not particularly "want to think," but rather simply conclude that getting through a period of mandatory thinking of a certain type would enrich them monetarily and so elect to embark on this work in the pursuit of having to think less later in life by virtue of having greater financial flexibility. A kind of thought, I suppose, though I'm not sure it rises to filtering out those true thinkers from the chaff that you're envisioning.
 
Upvote
3 (3 / 0)

JohnDeL

Ars Tribunus Angusticlavius
8,744
Subscriptor
Apologies - I misread that trying to respond quickly between classes (the tuition costs at our institution are also a good bit lower than that even before you play the "net of waivers/scholarships" game, so it looked odd).

Fair enough. It isn't as if I've never misread something.

The other paragraphs remain true - especially the massive state disinvestments around the 2008 financial crisis.

Oh, it goes much further back than that. Take a look at the chart. Notice the inflection point at 1980? Suddenly, instead of keeping pace with inflation, tuition is skyrocketing. Gee, I wonder what (or "who", Reagan, cough cough) could have changed the rules so that college education started to become more expensive?
 
Upvote
0 (1 / -1)
I dont really recall getting that many assignments in university: the majority was learning principles and then applying them in an examination. If I had a time travelling AI helper back then I’d only be able to influence about 30% of my total score. Not insignificant but it would be clearly suspect if I aced assignments but failed exams consistently. And I seem to recall that you had to get a passing grade on exams as well as overall to get that module.

one possible defense against excessive AI dependency would be to require each assignment to have a reference back to a previous assignment, I.e. “consider how this differs from the area reviewed week 3”. That is much harder to feed into a machine to write as it doesn’t know what week 3 was, unless you pay for a lot of tokens.
Isn't this just two sets of inputs rather than one? Or maybe three, if you also upload what you turned in for assignment 3?

My NYU alumni amount gives me access to a Gemini pro account that I assume would handle the extra steps not problem, at no cost.
 
Upvote
0 (0 / 0)
the damage was done decades ago anyway, college degrees have been more about who you know, and more accurately, who your parents know, for decades now as entry level turned into 5+ years minimum experience or just stopped existing at all. HR managers have been completely brainless and outsourcing their work to LLMs to write jobs and suppress wage and talent for decades now, something well reported on, and lamented by everybody yet nothing has ever been done to fix it.

so ya it makes sense students who are either going to cruise into employment post graduation or are screwed if they aren't 4.0 scholars, are using AI. the risk vs reward math is entirely skewed towards getting the grade and figuring out how later.
 
Upvote
-5 (1 / -6)
The problem of Fake It till you Make It, is that if you never have ever learned how to actually make it, your always just going to be a fake, and people will eventually figure that out.

I feel Hollywood Sci-Fi has set a, while story expedient, comically tropey bad example where the supposively very intelligent character simply asks the computer for the answer to a difficult question. Star Trek/Galaxy Quest
either you have a job already lined up wherever your parents work, or you're going to be screwed by the system anyway and should maximize your GPA. I'm 10 years out of school and back then half my class either had jobs at their parents companies lined up, or stark unemployment and a career of under employment in their future.

its our society thats moved away from merit thats removed all incentives and rewards for people to actually be good at their jobs.
 
Upvote
-4 (0 / -4)
I get and sympathize with the sentiment, but you probably meant "make me want to launch myself into the Sun".

The problem with LLMs is that they take the old "they wouldn't print it if it weren't true" argument and amplify it by a million. Not only are they untrustworthy plagiarism machines, they also tend to lead people to abandon critical thinking and assume that the LLM output is correct. (It often isn't, as you well know.)

About the only way to kill this is to have the students ask the LLM an in-depth question about something that they know very well, just so they can see how wrong the LLM often is. And I fear that even that isn't enough.

ETA: In many subjects, there are misconceptions that can act as a block to learning. I am willing to bet that LLMs would get the answers to those misconceptions wrong in many cases, which suggests basing homework assignments and tests on misconceptions (which is where they really should be, honestly).

The problem is that for most students, anything they know sufficiently well to actually ask in-depth questions on is probably going to be something so generic and widespread that LLMs have good training data on it and thus provide good enough output to still fool these students. You can't ask them to ask deep questions on Dynamics engineering or viral propagation or something because students won't yet have the knowledge to judge the output by.
 
Upvote
2 (2 / 0)
Yes and no. Time-inconsistent preferences are real. Part of the art of teaching is reeling in the rope students use to hang themselves.

This would work, but it would also result it a shit-ton of Fs. Then again, if the students are ready and willing to sign up for this, a lot of professors are more than ready to oblige. Say the word.

Imho, there's too much effort and energy being expended by teachers and professors nowadays to drag unwilling students kicking and screaming to anything other than an F. IMHO providing the motivation for a student to put in the most basic of work is not a teachers job. It's the students (or maybe their parents if they're a minor). Don't do the work, get an F. I've had plenty of subjects in school and university that I had little interest in, but the motivation to still get a decent grade was entirely mine. Not that I always succeeded...
 
Upvote
0 (0 / 0)

LtLoLz

Wise, Aged Ars Veteran
112
I'm not a teacher, I'm a mentor for high school and college sysadmin internships at my company. I have the privilege of having just 1 to 3 students at a time and breaks inbetween. I've made a few progressively harder assignments for them using old hardware.

So far some were really smart and built stuff and experimented by themselves beyond what they were told to do while reading documentation without any LLMs. Just didn't need it.

Another was very smart and driven, yet lacked the basics and relied only on chatGPT. After repeatedly reminding him about the instructions were in documentation he managed to complete his assignments.

Some just don't want to think and outsource that to LLMs. They don't achieve much and they can be a pain to deal with. One had the gonads to ask us what he should write in his report after doing nothing for 2 months.

I also visit the professors and new students back at my high school every year and we talked about this last time. The better students already understand that you need some basic knownledge to make a good question/promt to actualy get usable information.

I'm pessimistic about LLMs, but I'm optimistic that there are still some smart people in the near future. There always were and this hasn't changed. The world is still turning and this won't stop it. They'll just do things differently. We should focus on those, as teachers always have for good reason.
 
Upvote
2 (2 / 0)
You were arguing that University lecturers should not try to to prevent cheating. The result of that is that evaluation, even if performed, does not distinguish between students who succeeded through cheating and those who did classwork as intended.

I was only replying to that point, not anything else you are claiming.
No, I argue the current university culture obsesses over cheating to the detriment of learning and that many aspects of modern class design incentivize gaming the system in various ways.

It's not a binary choice between total anarchy and wildly hostile systems that treat students like criminals and it's a simple reality that some of the desperate measures taken can meaningfully impact the quality of learning for everyone else.
 
Upvote
-5 (0 / -5)
I've mentioned it elsewhere: my daughter is studying CS at a junior college, transferring to a CSU school after 2 years. Right now, they're learning C++, and she seems to have a decent grasp of it. She said their prof had to make the assignments count for 0 because too many kids were using AI and leaving the AI's obvious comments / prompts IN the final submission. I.e. 0 effort. And these are a "CS cohort" - i.e. students specializing in CS. It's brutal.

In hindsight, the "2 years at junior college, transfer to a CSU" that both my daughters did (they tested out of high school after 2 years) has been somewhat of a disappointment - both the students at the junior college and now the 3rd/4th year students at CSU have been horribly unmotivated, unreliable, and basically just a shitshow - my autistic daughter has to constantly hunt down the other people in the group to get them to do the basics, and they can't even understand basic instructions or due dates. We expected 3rd/4th year students to at least be reasonably competent at a state school, but it feels like they wouldn't know how to walk to class without their phone.

AI is very, VERY bad for our intelligence levels. It's working amazingly well to create little robots that only know how to cut/paste what their magic hand-box tells them. Our daughters want more than the basics, and it's unbelievable how impossible writing 3 paragraphs seems for 4th year students.

To be fair, it looks like the issues with the students long predated AI, although phones may likely bear some responsibility.
 
Upvote
1 (1 / 0)
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.

The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!

We have the whole internet out there for anyone who wants to educate themselves on any academic subject, and has at least some talent for auto-didacticism. LLMs will likely help with that. Maybe university is destined to devolve into remedial education and credentialism.
 
Last edited:
Upvote
0 (0 / 0)
Except, you know, they're not.

People who would willingly rely on AI to get good grades and do their coursework are the types of people who will, by virtue of their academic record, subsequently occupy bureaucracies (e.g. all levels of government, insurance companies, etc.) in administrative positions (typically quite well compensated) and the rest of us -- the teachers, the technologists, the "doers" of the world -- will have to rely on them for anything to do with those organizations.

What a sad future awaits us.

Very thought-provoking article (and very well-written), @ScottJohnson. Thank you for putting in the real effort to produce this.
I would call that an "other effect."
 
Upvote
0 (0 / 0)

WildGunman

Ars Scholae Palatinae
693
Subscriptor
Imho, there's too much effort and energy being expended by teachers and professors nowadays to drag unwilling students kicking and screaming to anything other than an F. IMHO providing the motivation for a student to put in the most basic of work is not a teachers job. It's the students (or maybe their parents if they're a minor). Don't do the work, get an F. I've had plenty of subjects in school and university that I had little interest in, but the motivation to still get a decent grade was entirely mine. Not that I always succeeded...
is your screen name indicative of where you grew up or just your ethnic background? My dad, funny enough, grew up in the Netherlands and I also have a lot of colleagues with ubdergrad degrees from European universities. They have no problem in most of those university systems handing out Fs and making people retake classes.

The pressure on US professors to not fail their students is immense. An NYU basically canned an Organic Chemistry professor over this. (While there is nuance here, the basic facts on the ground are that the students bitched and he got the hammer brought down on him. It’s a real thing.)

Maybe university is destined to devolve into remedial education and credentialism.
It has been devolving into that for decades. No LLMs were required.

That said, I’m a little annoyed by the defeatism shown by a lot of professors. If the system needs to radically change, it can do that. But it requires a serious and thorough understanding of the problem, the landscape of available tools, the economic model, and above all an act of will. The people who don’t want to engage in that should quit. Resigning in protest is also an act of will that puts pressure on the system.
 
Last edited:
Upvote
-1 (1 / -2)
Life's too short to think of how you spend the time we have here that way.

YMMV, of course.
That's a rich man's perspective. A person making six figures and blowing it all on nonsense, then working overtime to try and earn even more? Sure, tell that to them, that's good advice for them.

A poor man knows life is a lot shorter when one full time job isn't enough to cover your housing, food, healthcare, etc. There's no time to relax and enjoy it when you're one step ahead of the bill collector. And you're not on a path to satisfaction if your two jobs (or one with lots of overtime) still make you choose between having kids and having enough savings to ever have a chance at retiring before your health is too poor to enjoy life. A few years in school to make sure you can afford the american dream is exactly what we've been selling people. If you want someplace for the elites to socialize, become more well rounded, etc without any piece of paper to show for it at the end, that's a whole separate category from people who come in wanting something they can use to get a job that only wastes part of their life instead of all of it.

Edit: On reflection, I admit there were a couple classes where the professor was legitimately a more interesting person to learn from and discuss topics with than I might have found if I was studying piecemeal online or from cheaper colleges with the same subject. So I did have access to prominent people in various fields, at the big school. Still expensive, still something where if I didn't need the degree to say what it did, I'd have been free to make different choices which could have involved meeting a greater number of people of less prominence, perhaps.
 
Upvote
-2 (1 / -3)
1) Students are really bad at determining what is irrelevant for them in the future. Something that seems unrelated now becomes the edge that lands you the better job later

2) an education is different than training (though the distinction has been blurred by politics and school administrations). Training prepares you to do one particular kind of job; an education broadens your flexibility and adaptability because we all know the world and job markets are not nearly as certain as they were in the 50s and 60s where a high school degree and some job training set you up for life.
I think about this a lot.

I am a dentist. At uni there was a 'easy' weekly session where you moulded teeth out of wax.
It wasn't run by a professor but someone with some junior title I don't remember and so no one really took it seriously.
Fillings are possibly the most common treatment we do, and my morphology is terrible.
I often wonder if I had taken those sessions more seriously would I be better at it.

Another anecdote was a bunch of students complaining to the biosciences prof that they didn't see the point of all the chemistry and biology we had to learn to be a dentist. This heavily german accented prof apparently told them that dentistry is a science and if the wanted to do a purely practical course to do plumbing.
 
Upvote
1 (1 / 0)
Test students inside a faraday cage.
Have you ever worked inside a Faraday cage built to keep out cell phone and WiFi frequencies? I have, and it was very uncomfortable. I can't imagine building one big enough to accommodate more than a couple of students at a time, never mind 30 HS students or the 2k taking first year life science prerequisites.
 
Upvote
0 (0 / 0)

graylshaped

Ars Legatus Legionis
67,885
Subscriptor++
I appreciate your up-front admission of willingness to use LLMs to cheat in a mandatory, passionless class -- I used the word "cheat" to emphasize the downside of cheating yourself of education in that class's subject. You do not want or need that subject because it's not your career or passion. I suppose that is typical of most students, especially young ones who are forced to be in school.

The point of these non-mandatory classes is broadening your view of the world. It's likely that AI will continue to undercut this forced broadening, and students will exclusively train towards their career. CompSci majors will know coding and architecture but not literature. Physicists will know physics and math but not botany. Designers will know art and design history but not anthropology. This type of focus is already common, but those who might have broadened their world will be intellectually smaller and more boring. Given that executives have time and again said they prefer to hire people with broad horizons and experiences, the business world will be less creative. It will be sad for civilization but those young students will be happy to not take those broadening subjects or to cheat their way thru them!
The tendency of smart professionals whose field requires substantial investment of focus and learning to assume that expertise extends into areas they couldn't be bothered to care about in school is widespread, to the extent of infecting many of us in these forums, in fact. I don't consider myself immune to it, and do try to remind myself periodically the line between ignorance and knowing enough to be dangerous is super-duper thin and nigh invisible.

For the record, if anyone suspects I might be talking about you and your fellow [insert profession], you are probably correct—as always, of course :)
 
Upvote
2 (2 / 0)