To teach in the time of ChatGPT is to know pain

LLM use is the most demoralizing problem I’ve faced as a college instructor.

See full article...

 

[vernonblack]

Why not just set offline, written, anonymous, exams? The habit of having professors grade their own students using term papers rather than exams has always been a rod for teachers’ backs.

 

[DCStone]

Exam proctoring software that records the screen and webcam effectively nips "open ChatGPT and copy and paste" in the bud. The problem is that you can use a phone while keeping it out of the view of the webcam, which would still be possible there. I've been advocating for replacing the webcam with a phone camera positioned from the side so you can see hands, which I'm seeing a little movement on from the software companies.

It's not an equitable solution*, either, and can raise some PPI concerns (in addition to the point you raise of whether or not it's sufficient for purpose)

For example, the software typically has to be installed on the student's computer, and it's not unlike video game anticheat/piracy stuff in that it needs deep access (security risk) and can slow things down on older hardware. The camera includes a view of the room, which may be shared. And there needs to be stable internet connection with adequate bandwidth. Finally, note that in many families it's probably a shared computer, not one solely used by the student taking the test or exam.

Some systems go as far as using eye-tracking to determine if a student may be using off-camera cheats, but that is also problematic for multiple reasons, and I've seen credible reports of false positives due to background activity in the home or just the consequence of ADHD.

I know that this varies across institutions, and even between different faculties within the same institution. I operate under the rule that we can't require the use of such software - and most of my colleagues are in agreement that we wouldn't be comfortable doing so anyway, for the reasons mentioned.

Footnote:
* I am very often struck by the disparities in computer hardware used by both students and TAs in my courses. Some of them really struggle not because they don't understand the content, but because their hardware is an obstacle to timely completion of course work. But such students typically are not in a financial position to upgrade to something newer and more reliable.

 

[wxfisch]

I recall for my undergrad that most of my core classes had very little actual course work, most of our grade was tests. I was in person, so tests were all written or multiple choice which would make an LLM somewhat unhelpful. If we did have an assignment, it was usually document review or synthesis of information as a longer essay which I suppose an LLM is helpful for, but as has pointed out would make things obvious when you are getting As on assignment and Ds on tests.

For my online grad degree though it was asynchronous just like is being talked about in the article. We had a reading assignment each week (usually read chapters a-n then complete either specific exercises from those chapters or answer a writing prompt), and a discussion requirement (post a response to the topic by Weds at 5pm, respond to at least 2 other students posts by the end of the week). The exercises I am sure I could get an LLM to do, but the discussion would be harder. You could likely get an initial topic easily enough, but the responses would be nearly as much work to get an LLM to create as just responding yourself.

I am curious though how group projects fare in the LLM age though. It was always an issue of one person doing more work than others, but how does that change when some students phone it in with LLMs and others actually put in the effort?

 

[JMTronicHobbyist]

Who's "we all"? Don't lump us into your cheaters circle.

I'd imagine in an MBA program you'd get good grades for cheating - but only if the professor knows you're cheating. How well the student lies about the cheating would be part of the grade also.

 

[pj88]

When OpenAI run out of venture capital money and have to realistically charge their 900 million users (current estimates of computer sit anywhere from $3-25 to $1 subscription) just to stay afloat whether this problem actually disappears on its own?
Would this generation of students be willing to subscribe to ChatGPT for $25 per prompt or would they just not bother as its no longer free?

 

[Anton Longshot]

I think it's likely that at some point there will be a hard split between people with actual skills and knowledge and the useless distracted that exclusively go through motions because that's all they can do.

It's going to hurt all of us (if it doesn't already) but especially those in category #2.
And fark once one gets labeled as cat #2 how to change that label?
I'm not sure #1 will want to help #2 to join #1.

 

[oluseyi]

Students often carry misconceptions about coursework. They may view an instructor as an opponent standing in the way of the grade they want. And they see “getting the right answers” as the goal of education because that’s how you secure that grade.

… The process of doing the work was what you needed to walk away with something.

Sadly, this isn’t true.

Economically, a degree has conferred better employment opportunities and consequent better pay and quality of life—even better partner selection possibilities. Given this reality, education for its own sake is not the median student’s objective; the goal is an impressive degree as a door wedge to greater economic outcomes.

Given this reality, it is wholly unsurprising that students are adopting LLMs wholesale to accelerate the process, and from a personal profit maximization perspective, they’re not even wrong to. The problem is macro, in our systemic incentives and rewards, and we just leave schools and teachers to be buffeted in the middle.

Because I think the issue is systemic, I also don’t think the solution lies in changing the nature of assessments or classwork. No amount of oral examination will resolve this, in the same way that ready electronic reference and calculation machines made rote memorization less valuable than integrative thinking and a mental map of the domain.

 

[Cart_catalog]

Time to stop giving assignments that can be answered with text.

Give them a diagram, ask them to label all of the features in the diagram.
Give them 'multiple guess' sheets with lots of wordy potential answers to select from.
Ask them to submit drawings, with the items depicted explained in a word or two.

Earth Sciences would seem to be a good topic to ask for non-text homework submissions.

 

[Elektriktoad]

I'd imagine in an MBA program you'd get good grades for cheating - but only if the professor knows you're cheating. How well the student lies about the cheating would be part of the grade also.

Not an MBA, but in a music history class I had a professor who would always throw a question on the test that we were completely unprepared for and expected to not know in order to assess our bullshitting skill, one he thought was important.

 

[thadco]

This was one of the very best, most insightful articles I have ever read. Here or anywhere. I would absolutely take any class you taught. Major Kudos.

 

[denemo]

Amen!

As for how to get back to basics. Covid hit in the middle of my studies in computer science and maths. Since we got sent home what we had to do for assignments and exams were to write it out all by hand, like if we were doing in classroom exam, then scan the pages with our phones, create a pdf and upload it to the course webpage.

Later the same day or the following day we would have a one-on-one zoom-meeting with the lecturer or TA where they would quiz you on your answers and try to gauge how well did you actually understand what you wrote.

It might not be perfect but that introduces some friction at least.

And during all the math-courses I took we weren't allowed to use calculators except for one course which I think was Statistics.

 

[t3rminus]

Students often carry misconceptions about coursework. They may view an instructor as an opponent standing in the way of the grade they want. And they see “getting the right answers” as the goal of education because that’s how you secure that grade.

Unfortunately, this is how much of human society has decided to structure education.

I would've had an amazing time in school if I had been evaluated based on my understanding or my engagement with the course material. I was a pretty good student with a love of learning and I often went beyond the requirements and narrow topics.

I definitely had some teachers that encouraged this and sympathized when my grades didn't match my effort. It didn't matter that I absorbed related material, engaged in higher-level concepts, or shortcut the repetition and memorization; I had to regurgitate the material we'd been given, exactly as it had been given, or we weren't getting the grade. I wasn't the only one like this in my class either.

The education system we've somehow accepted as the standard in modern society doesn't allow for free thought, outside-the-box thinking, creativity, or forging new paths. The standards are set based on the lowest common denominator, and repeatable, testable metrics. All students must perform the same, and all students must repeat what was "taught", no exceptions.

In this sense, instructors are often made into the opponent by a flawed education system, and "getting the right answers" becomes the only way to succeed.

As much as I hate them, LLMs are simply proving how flawed the system has always been.

 

[The Lurker Beneath]

Not an MBA, but in a music history class I had a professor who would always throw a question on the test that we were completely unprepared for and expected to not know in order to assess our bullshitting skill, one he thought was important.

I hope he also awarded at least a modicum of points for honesty!

 

[nononsense]

I think we should just look back to how we did this 25 years ago. We all used Sparknotes, and most problems asked by professors were answered in the spark notes. You could literally copy/paste whole sections and just move sentences around and change some words and get a good grade. Fraternities and academic societies have maintained “test banks” for years.

Which is all to say, this is probably not as big a problem as it seems.

I thought that the article was pretty clear as to why LLMs are far superior at cheating than methods of the past. Of course you could always cheat. That wasn’t the point of this article.

 

[randomuser42]

I operate under the rule that we can't require the use of such software - and most of my colleagues are in agreement that we wouldn't be comfortable doing so anyway, for the reasons mentioned.

So what do yall do about this problem? The current situation isn't equitable either because it puts the students who don't want to cheat at a disadvantage.

 

[cloudseer]

My hope is that the initial use of LLM was simply due to people doing one prompt searches (transferring google searches) , as we are already seeing LLM natives using the technology on a more interesting way: upload your manual into it and query it on the friction. I can see this working very well on subjects like maths and chemistry for example, where the student can interrogate the text book on the topics that are challenging to understand and then ask it to generate exercises similar to ones they failed. Or talk them through how they got it wrong.

Yes basically every student can have their own teacher, standardised by the quality of the textbooks.

 

[wxfisch]

I’ve long been skeptical of long-distance learning.

Primarily I feel that way as: an occasional student, because my motivation flags when I have to spend more time in front of screens. (Though I do recognize that, for some students, long-distance learning is the only possibility.)

But also as a past, and potential future instructor, I’m skeptical because of the cheating plague that has existed long before the rise of AI. LLMs, and other forms of AI are just ‘weaponizing’ that phenomenon, alas…

At this point the only way that I’d agree to teach would be: in-person only; and a student’s grade would rely solely (or at least largely) on their proctored work (e.g., in-class quizzes, exams, etc. ). However I’m not sure if typical schools and administrators would allow that approach…?

This is the same (flawed) thinking that upper managers apply to return-to-work edicts when some people do genuinly work better remote. "It doesn't work for me so no one could benefit from it".

Just because you learn better in person does not make that true for everyone. Some students do better with self-paced courses or asynchronous courses for a variety of reasons. Perhaps they can better focus on the topics they don't understand and breeze past ones that they find intutitve? Maybe it allows them to more easily pull in additional resources to add context or explain a topic in a different way? Or perhaps they are easily distracted in group settings or have social anxity that makes larger in-person classes less effective or more stressful for them? Maybe they just learn better through reading than lectures? There are a million reasons someone may prefer remote learning to in-person that have nothing to do with access to education.

At the end of the day the education is the responsibility of the student. No teacher can force a student to learn, the best they can do is create an environment that fosters learning for their students and be clear and up-front that they students need to put in the effort, especially in post-secondary settings (and even more so in graduate settings). If a student really sees their education as a check box to a future job, no amount of effort from a teacher is really going to solve that.

 

[Emotion_ology]

I clearly don't have all the answers, and you raise many legitimate points. I still think "grades" as we consider them today function to continue to degrade education quality in the united states.

Goodhart's law would be better written as "when a proxy measure becomes a target, it ceases to be a good measure", which both gets to the point but also helps consider that sometimes proxies are the best we've got. Sometimes one just needs to measure better.

Yes, and grades are functioning as a proxy for far too many things to be a good measure of nearly any of them at this point.

Have we? Or have we allowed grades to get diluted due to perverse incentives? Is the appropriate response to surrender on that or to attempt to realign via better tests? It's pretty important to figure out what the foundational demand is or else we'll just end up reinventing the wheel with new labels. Like with grades, yes the true goal is a complex combination of "how can we motivate students to learn, and measure how well they're learning in a dynamically granular manner, and also measure how well they handle meta tasks related to learning that matter for real world unsupervised activities including continued learning and learning application, and then also convey all that in a high density form that decision makers in different domains can make use of before the student in question has more track record, and also we need to feed that back into how we ourselves are doing with our teaching/methods". That's not an easy problem space, but it's key to recognize that grades became and have continued to be a thing because they've been useful.

Here you point out that grades have a crazy number of competing uses that create perverse incentives and dilute their meaning. Despite these things they've been a simple, intuitive, and useful enough proxy to reach the point they're at today. My argument is that they have been becoming less useful over time as more and more people have the capacity and incentive to game them. LLM's are, I believe, the tipping point where they no longer do what we need them to do well enough to keep using them.

But students very much actively want to be able to convey their achievements outside of the class. The whole point for most students is linked to the real world, to getting good jobs or other opportunities they desire. "Consequences" are therefore fundamental, touching on various zero-sum games. Of course people are going to care, and some people are going to do better relative to others, and the ones who do worse are going to have incentives to cheat. You can't just side step all that. Like if some colleges stop providing grades as you suggest, game theory out a bit the results of that. Students exit college, and go to apply to some job, and naturally the company wants to know how they did. How do they convey this in a rapid objective fashion? Will companies then preferentially recruit from colleges that do still provide grades? What would the result when it comes to where future students apply to, and the consequences for the colleges without grades they don't? Etc.

Not having the consequence be a direct impact on opportunities and learning potential is not the same as there being no consequences at all. It's about re-aligning the consequences to better match actual learning/effort, and then finding a different (related) measure that better matches capacity for success.

I'm not saying there should be no way to differentiate relevant student skills and achievements, just that grades in their current form are an increasingly bad proxy. If the solution were clear we wouldn't be having this debate. Any alternative would need to be trialed, tested, and refined tremendously, and there hasn't been the will/sufficient incentive to find an alternative to grades yet. Perhaps we're there now thought. Maybe we'll default to more rigorous high stakes proctored testing like the SAT. Perhaps we'll need to fall into more subjective qualitative assessments from teachers throughout schooling that are then processed into standardized themes by a specially trained LLM. The lack of an obvious alternative should not prevent us from noticing that we need to really start looking for one.

Teachers need some form of grade within a class to track individual understanding and development in order to adequately lesson plan and target interventions, and ultimately in order to do a good job teaching. When we use the same grades outside of class as a proxy for potential for success, we create an incentive to interfere with the accuracy of grades as a teaching tool and consequently decrease their value overall. That used to be tolerable when a small group of wealthy folks were the ones capable of effectively exploiting this issue. LLM's offering that capacity to everyone adds just too much bad data to the mix. Garbage in, garbage out, and LLM's introduce too much error for grades to be a meaningful measure now.

 

[justsomebytes]

I do hope these problems with LLMs will force change into the education system (at least in the US). The issue of cheating is more systemic than can be fixed by changing some examination methods or assignment types; there is a fundamental misalignment between the education system and education outcomes.

For example, I had several people in my graduating major class doing significantly better than me, they were even TAs for several of the more important fundamentals classes. In private, they admitted their success was being good at cheating, googling assignments, finding the test answers, etc. This was an engineering major, yet they and 50% of the rest of my class were hired out of university as financial analysts and other non-engineering financial roles at large banks. Did they learn what they were supposed to? From their own admission they didn't. Did it matter? I'm not sure it does, they pursued the career they wanted; the networking, some math classes, and being able to talk about being educated in engineering is the credentials they wanted.

If anything, even before AI, employees and universities seemed to be rewarding cheating and shortcutting with tangible benefits: high paying jobs, lucrative careers even if it hasn't aligned with what teachers want for student's educations.

 

[Green-PEAs]

Seriously, friction does matter. Assuming the goal of education is to "get the grade" is like assuming that when I run 3 miles for exercise, it's because I needed to be where ever the end of that 3 miles was.

This is a remnant from the original “liberal education” purposes of college. You round out a student into a complete thinker and equip them for life, not a career, etc

The problem is that this is completely out of touch with modern priorities and I have become a fatalist about it.

We have to accept that this is not at all the reason the majority of people go to college.

The purpose of college is now to get good grades so you can get into a good career and avoid being mired in poverty and hardship. Almost no one is attending school with the hopes of becoming more well-rounded and more intellectually robust. It is viewed as a necessary step on the ladder to “success” (or the ladder out of blue collar work, at a minimum).

Of course the kids are going to cheat on their philosophy class, their English class, and even the classes relevant to their career goal. Everything is in pursuit of the credential, the END, there is almost zero interest in the means or the journey.

And to be fair to the kids that’s probably valid. I had perhaps two faculty members ever that seemed to care. I got nothing from undergrad except my credential and my grades that I could not have gotten from the library.

We can talk about the fun and games of intellectual growth and how the grades shouldn’t matter, their LEARNING should, when attending college is not a life-altering debt acquisition gamble that is now required for numerous careers that have zero need for a degree requirement.

If students are going to be saddled with non-dischargable debt that will alter their lives forever - to pay for an education they may have zero interest in, that is required for a $40k sales job now - then how can we cry about their lack of intellectual curiosity or absence of effort?

They’re going through the motions to try to not be the last rats drowning on a sinking ship. I’ve read so many articles from teachers like this guy. Great intentions, valid complaints, completely out of touch with the brutality of modern economic reality.

 

[Jim Salter]

I haven’t encountered any students who think they’re learning when they let LLMs do their work.

This is, far and away, the most upbeat and positive thing I've heard anybody say about (relatively) normal people's interactions with LLMs yet. It gives me a bit of hope.

 

[Emotion_ology]

I'm willing to accept that there may be a day where we don't have to worry about this because dedicated teachers are no longer necessary, but I think we're still at the point where we should take seriously a factor that drives people away from a profession dedicated to preparing high quality humans in the next generation. Can't back that up with data, however.

To be clear, I absolutely think teachers being miserable is a HUGE problem. I just mean that, rhetorically, he doesn't do a great job connecting his arguments to his central thesis.

 

[DCStone]

So what do yall do about this problem? The current situation isn't equitable either because it puts the students who don't want to cheat at a disadvantage.

Simple. We don't do remote on-line exams, even for courses that have an on-line component. We're not set up as a distance education institution or department. If we were in that business, we'd have to come up with something, but we're not in that situation at this time.

I know some institutions that are primarily on-line distance education focussed will partner to set up exam centres in multiple locations, maybe even timing it to coincide with times when the student residences in the partner locations are available for booking. It's not a perfect approach by any means, but I'd personally prefer that to on-line proctoring solutions as either an instructor or as a student. (They will do the same thing for lab courses, too - offer a summer intensive version.)

 

[randomuser42]

FWIW, the thoughts expressed by the author can be confirmed by one horrifying trip to the Teachers subreddit.

Weep for the future.

Between a covid generation and LLMs it's definitely been tough, but that subreddit is extreme. It rewards the more extreme stories with more engagement and upvotes so there's a feedback loop where you get the wildest stories.

I used to read that sub but it was just so divorced from the reality we were experiencing at our son's school, one of the lowest ranked, low-income area elementary schools in our county which nevertheless has altogether been a positive experience for our son. I've chaperoned plenty of field trips and events during his time there and the kids are regular kids.

 

[Jim Salter]

This is a remnant from the original “liberal education” purposes of college. You round out a student into a complete thinker and equip them for life, not a career, etc

The problem is that this is completely out of touch with modern priorities and I have become a fatalist about it.

That's because you have to be EXTREMELY secure in the world before spending four to ten years of your prime and enough money to buy one or more perfectly nice homes without a VERY explicit goal at the end.

Essentially, you have to be so certain that the world is a safe and welcoming place that will be good for and to you, that you're willing to just "let the universe take the wheel" and gradually show you a path.

Does that sound like the world we currently live in?

Bonus question: does that sound like it has ever applied universally to all classes of any given society?

 

[Zoc]

So as an adjunct instructor myself, I disagree. I don't see my job as being "detective and prosecutor," making sure students don't cheat. First of all, that's impossible. There's no foolproof tool that can tell you a submission is "50% AI," or 75%, or 100%, or whatever. I've certainly read some papers that I know were partially or entirely AI-generated. But I can't PROVE it beyond a shadow of a doubt to a disciplinary committee, so I don't go down that road.

This isn't like when I was in college, where if a faculty member found you with a cheat sheet or "crib notes" written on your arm, that was indisputable. Cheating with AI isn't like that.

But in the second place -- I don't feel that my university is paying me to be a detective, or a prosecutor. My job is to deliver education on a specific topic to a group of students each semester, and devise ways to evaluate their learning. I'm delivering the education, and I'm evaluating their learning in reasonable ways. If students choose to fake those evaluations by using LLMs? That's their choice. It's a poor one, but that's their choice.

This is exactly what I was going to say. What's the point of trying to stamp out the LLM cheaters? You can't force people to learn (that's part of what makes education different from training). Time wasted on a futile crusade against LLM cheaters would be better spent on supporting honest students.

 

[Sehven]

I don't disagree at all about the author's take on LLM use. But I do have some thoughts on the students seeing the instructor as an adversary and the goal being the grade. This is simply a result of how we have chosen to do education.

It's been several years since my school days but I can tell you that I always had two goals in school - to obtain knowledge/skills and to obtain good grades and let me tell you the two have almost nothing to do with each other. Well, that's not completely fair - the author is right about the friction and that the process of doing the schoolwork well enough to obtain the grade would sometimes result in acquiring or cementing knowledge or skills. Emphasis on "sometimes."

The problem is a bad grade has consequences. You fail the class or your record looks bad in a world where you need it to look good. Instead of an opportunity to learn from mistakes and improve, a bad grade is a millstone that follows you throughout your academic career and potentially even into your actual career. Some instructors will give you the opportunity to redo the work consequence-free to obtain a better grade and this is wonderful but it's still burdensome because the class doesn't stop to wait for to catch up and you still have multiple other classes as well as any life responsibilities to keep up with. There simply isn't time to do every assignment multiple times.

So yeah, I cheated. I cheated ferociously. Any chance I got to cheat, I took it. I'm not defending that as the correct moral choice or the responsible one, merely the pragmatic. It's almost certain that this had deliterious consequences on my learning. I'd like to think I wouldn't use an LLM to do my work for me if I were going to school today but I probably would. I should qualify that to say that I probably would if I were taking a full time student course load with the goal of obtaining a piece of paper that states I'm qualified to work in my chosen field. If I were taking classes for no other purpose than to learn and develop skills, I would keep my course load light and I wouldn't feel the need to cheat.

 

[Totally Radical Liberal]

Who's "we all"? Don't lump us into your cheaters circle.

Unfortunately, cheaters think everyone cheats. Especially cheaters at the top (management). Part of the reason is so they can think they're better than people doing objectively harder work--they assume they're just cheating too. And somehow in the pre-LLM era, they never thought how skilled the people all the cheating was farmed out to must be. It's like the idea of turtles all the way down, but with cheaters.

 

[wxfisch]

Your argument has holes...so we started using skidsteers in factories to increase efficiency, as a result those who wanted had time to go to the gym and workout as a result of the weight yhey gained from using a skidsteer in the factory....even better before the loom everyone had to know how to make shirts, due to the loom we had more time to do other things. Your argument of using a skid steer to lift weights at the gym is backwards, as we used it to make work easier....instead lets look at school lessons. Now we could make a student go to a library and regurgitate a book onto paper....or we can change the goal...instead of writing a paper using the library and pretending the internet doesn't exist, or not using ai....Instead of:

“Write this essay yourself”

It becomes:

“Use AI to generate 5 perspectives, then critique them”

“Find flaws in this AI-generated argument”

“Design a system using AI and explain the tradeoffs”

That’s way harder cognitively than doing grunt work.....just my opinion

I think this is missing the point of education though, especially post-secondary education. The goal is to learn not just the process to get to end result, but also to learn core concepts about that subject (espcially in science and math courses). I won't say that LLMs are unhelpful in day-to-day work in the real world, they certainly can be, and we should teach taht skill as well. But the whole point of the article is that teachers don't have any good tools to combat how easy LLMs have made short-cutting the work required to learn something unrelated to LLMs, and lazy students (or overworked students) are using them to their own detriment either because they don't understand why they need to put in the work, or because they don't care.

The forklift in the gym example is, IMO, perfect here because the entire point of the gym is to do the work, for it to be hard. Making that easier goes exactly against what the end objective is (to be more fit, stronger, look better, etc). The objective was never to pick things up and put them down again, just like the objective in school is not to write essays, it's to learn a topic (and often learn how to learn new things). The exception to that is maybe trade schools where the goal is to learn how to accomplish specific tasks, but even there you still need to understand the why behind a lot of the skills, not just the mechanics of how to do them.

TFA points out that all of your examples can (and are), today, being responded to very convincingly by LLM tools. That is the problem that is being highlighted here.

 

[memphisbob]

In the past instructors could conflate students with insatiable curiosity with those simply trying to get their ticket punched and treat them the same. LLM's have ruined that approach. IDK, maybe students shouldn't be required to take classes for which they have no interest and employers should carefully read transcripts. Classes for which a passing grade can be obtained by the use of LLM's should be eliminated altogether.

 

[jhodge]

That's because you have to be EXTREMELY secure in the world before spending four to ten years of your prime and enough money to buy one or more perfectly nice homes without a VERY explicit goal at the end.

Essentially, you have to be so certain that the world is a safe and welcoming place that will be good for and to you, that you're willing to just "let the universe take the wheel" and gradually show you a path.

Does that sound like the world we currently live in?

Bonus question: does that sound like it has ever applied universally to all classes of any given society?

Of course it never applied to everyone, nor was it intended to. Colleges & universities grew out of seminaries and were not, until very, very recently, targeted at teaching career skills. That was the realm of professional schools (law, medicine), trade schools, military academies, apprenticeships, etc. - all perfectly respectable forms of education suited to different purposes. The idea of "college for everyone" meant trying to wedge many different goals in to the same institutional framework and, shocker, it's not working very well.

 

[DCStone]

That's because you have to be EXTREMELY secure in the world before spending four to ten years of your prime and enough money to buy one or more perfectly nice homes without a VERY explicit goal at the end.

The liberal arts ideal is very much a product of a time when only a small fraction of the population would even be heading to university anyway. That changed dramatically, especially once the boomer generation started graduating out of high school and there was a mad scramble to expand college and universities to accommodate them.

For a local example, back in 1948 only 4% of students province-wide completing grade 8 would go to university. For 2015, that figure was: university 44.1%, college 27.5%, and apprenticeship programs 1.7%.

I suspect that the numbers would be even more exaggerated if you went back to the Victorian era.

 

[Green-PEAs]

I think we should just look back to how we did this 25 years ago. We all used Sparknotes, and most problems asked by professors were answered in the spark notes. You could literally copy/paste whole sections and just move sentences around and change some words and get a good grade. Fraternities and academic societies have maintained “test banks” for years.

Which is all to say, this is probably not as big a problem as it seems.

One of the most devastating academic disillusionments I experienced came from those test banks.

I was in grad school where ostensibly all of this material is relevant to our ability to perform in our career. I put so much effort into studying for an exam. I’m reading the text, doing practice problems, reviewing lectures, etc etc.

I show up on exam day and there is a little pod of guys I’m friends with reviewing a printout. “What’s that?” “It’s last years exam!”

Every single one of those guys got a 100%. Not a question had been altered. They didn’t study at all, just brute memorization the morning of the test. I got an 85%. And our grades and class rank directly affected our ability to specialize.

One of them had a test bank that basically got the entire group through the first two years of the program at the top of the class. Was it wrong of them to use it? Probably. I consider it “more wrong” that the faculty was repeating the exams verbatim for years. They surely had to notice a disproportionate amount of 100% results!

 

[supercrisp]

I am am English professor with a nerdy bent. I'm one of the people who played with stuff like Mark V. Shaney back in the day.

I see people in these comments telling faculty to get with the LLM, learn how to use it, etc. In fact, one of the posts is almost identical to one I saw on r/professors a while back.

Here's the deal: most of my students do not want to use LLMs to learn. They want to use them to cheat. Period. That's it. Most students are at my institution for two reasons: a credential and parties.

Saying it so bluntly sounds horrible, but it's true. The sad fact is that education has historically relied on coercion, and now the cheating technology is good enough to evade punishment.

Now, that wasn't me. But I am weird. I built my own telescopes, mirror and all. I still take things apart to see how they work. If I had three wishes, one would be to speak more languages. I am not normal. Most of my students are, and they have better things to do than homework.

That said, I was from an impoverishered rural area, was a troubled youth, and so on. I got into college and then in my doctoral studies by the skin of my teeth. I benefitted from liberal admissions policies.

I don't know how this will all play out. Thank spagmunster I'm nearing retirement. But I'm about to walk into a classroom with a stack of photocopies, and at the midway point of the class, students will put away their devices, get out pen and paper, and write about the piece they just read in class. That's where I went this semester, and it's the first time in years I see actual evidence of learning and any interest in the material.

 

[Zoc]

So yeah, I cheated. I cheated ferociously. Any chance I got to cheat, I took it. I'm not defending that as the correct moral choice or the responsible one, merely the pragmatic. It's almost certain that this had deliterious consequences on my learning. I'd like to think I wouldn't use an LLM to do my work for me if I were going to school today but I probably would. I should qualify that to say that I probably would if I were taking a full time student course load with the goal of obtaining a piece of paper that states I'm qualified to work in my chosen field. If I were taking classes for no other purpose than to learn and develop skills, I would keep my course load light and I wouldn't feel the need to cheat.

How did that work out for you? Were you able to get a good job after graduation? Has the missing knowledge ever been a problem for you? Note: zero judgment is meant to be implied by these questions, I'm legitimately just interested to know whether this approach to school worked out in the long run.

 

[etr]

There's evidence that this is not correct. The act of writing seems to enhance learning in and of itself.

That hand-writing could enhance learning in and of itself is not a new idea or one I'm inclined to argue.

However, learning and demonstrating or applying learning are not the same thing. While in theory the thinking one does in an exam can be a bit of a learning activity, the primary goal is usually to demonstrate learning and advance to the next topic. Handwriting may not be a net win there.

I also sympathize with the plea against handwring for other reasons. I had a teacher who literally refused to accept handwritten work from me as illegible while still in high school and handwriting regularly. On one hand, reading garbage handwriting like mine is lousy, but on the other you hate to see handwriting get in the way of assessing learning.

 

[DCStone]

One of the most devastating academic disillusionments I experienced came from those test banks.

I was in grad school where ostensibly all of this material is relevant to our ability to perform in our career. I put so much effort into studying for an exam. I’m reading the text, doing practice problems, reviewing lectures, etc etc.

I show up on exam day and there is a little pod of guys I’m friends with reviewing a printout. “What’s that?” “It’s last years exam!”

Every single one of those guys got a 100%. Not a question had been altered. They didn’t study at all, just brute memorization the morning of the test. I got an 85%. And our grades and class rank directly affected our ability to specialize.

One of them had a test bank that basically got the entire group through the first two years of the program at the top of the class. Was it wrong of them to use it? Probably. I consider it “more wrong” that the faculty was repeating the exams verbatim for years. They surely had to notice a disproportionate amount of 100% results!

You'd have thought that it would set off a bunch of red flags, for sure. Then again I saw exactly the same thing in some of the undergraduate courses I had to take. I'm not against reusing some questions, and I'm certainly not against students having access to copies of past exams. But recognizing that and taking it into account should be a key part of any instructor's job.

 

[siliconaddict]

My hope is that the initial use of LLM was simply due to people doing one prompt searches (transferring google searches) , as we are already seeing LLM natives using the technology on a more interesting way: upload your manual into it and query it on the friction. I can see this working very well on subjects like maths and chemistry for example, where the student can interrogate the text book on the topics that are challenging to understand and then ask it to generate exercises similar to ones they failed. Or talk them through how they got it wrong.

Yes basically every student can have their own teacher, standardised by the quality of the textbooks.

The issue is you are making the assumption that your LLM teacher is providing a factional and valid response to your queries. The student is reliant that the SME is providing correct guidance. LLM are at the stage where they are good, they are not perfect. Now admittedly when it is a text book and usually dealing with a narrow set of data, that branch prediction it does is far better than more nebulous questions like asking a LLM something where it gets its data from the internet in general.
I'm still in the stage with LLM's of trust but verify. But then a good LLM that is giving you info from a textbook should easily be able to cite the page and passage it is using.