Tag: pedagogy

Categories
Teaching and learning

I taught my students how to cheat using ChatGPT and they hated it

If you’re on any teaching-related social media, you’ve been inundated by posts related to ChatGPT. This week, I decided to jump on the bandwagon and try a few lessons using the new tool.

What is ChatGPT?

ChatGPT is an artificial intelligence chatbot. The premise is that you can ask ChatGPT a question, and it will answer it. It was created by the OpenAI initiative, and a short run-down if its design can be found here. The key takeaway for teachers is,

ChatGPT was built using text found on the internet. That means it’s essentially as smart as your average internet poster.

For teaching, this raises several things you should be aware of:

  • It will give the most likely internet answer for a question. That means that any past-years exam questions will almost certainly be answered correctly, because students will likely have asked how to answer a question on internet forums and somebody will have posted the mark-schemes.
  • It will also magnify common misconceptions. Because common misconceptions are, by definition, common, they will be repeated often and weighted more highly in the AIs attempt at answering questions.

It doesn’t know when it’s wrong

Take a look at the exchange below:

Can you spot the problem? Neither can ChatGPT!

This is a function of how the AI works. It doesn’t actually have a human sense of understanding. Instead, it’s writing a sentence and then working out what, statistically speaking, would likely be said next. That means that it can’t answer any of these types of questions, because while the words are extremely formulaic, the numbers themselves are highly specific to each question. As a result, the AI happily accepts it’s wrong, and moves on to the next most likely answer, which is still wrong.

Using this in lessons

My lesson idea was to try to harness this to enhance students ability to spot answers. I took a high-risk approach, and suggested that they start cheating. There have been widespread reports of schools blocking ChatGPT to prevent cheating, and accounts on TikTok promoting it as a way of getting easy answers to homework questions. I decided to tell my students they would be using ChatGPTs answers on a worksheet, but with one caveat – they must check the answers were correct!

I used this Google Slides presentation, along side this worksheet.

The core task was to ask ChatGPT about a type of energy resource, and then rate the quality of its answer. Here’s my first attempt:

I asked chatCPT: Please explain simply how a coal power station works The chatGPT answer is below. The following codes have been used: Wrong information Correct information Not completely wrong, but not fully explained or correct to IGCSE standard. Chat GPT answer Comments A coal-fired power station is a type of power station that generates electricity by burning coal to produce steam, which is then used to generate electricity. Here's a simple explanation of how a coal power station works: Coal is delivered to the power station and is stored in a coal yard. The coal is then transported to a coal crusher, where it is crushed into a fine powder. The coal powder is fed into a boiler, where it is burned to heat water and produce steam. The steam is sent through a network of pipes to a steam turbine. The steam turbine is connected to a generator, which is a machine that converts mechanical energy into electrical energy. As the steam passes through the turbine, it causes the turbine blades to spin, which in turn generates electricity. The electricity is then sent to a transformer, where it is converted to a high voltage and sent through power lines to homes and businesses. I hope this helps! Let me know if you have any questions. 5 isn’t too clear. It should say that the steam turns a turbine, and then the turbine turns the generator. 6 and 2 aren’t needed for IGCSE.

As we can see, it wasn’t too bad an answer. However, things got a lot worse when I asked for the advantages and disadvantages, from an IGCSE Physics perspective:

The energy change involved I asked chatCPT: Thanks! What are the energy transfers at each stage? The chatGPT answer is below. The following codes have been used: Wrong information Correct information Not completely wrong, but not fully explained or correct to IGCSE standard. Chat GPT answer Comments In a coal-fired power station, energy is transferred from the coal to the steam to the mechanical energy of the spinning turbine blades, and finally to the electrical energy of the generator. The first sentence mixes up stores of energy with types of energy; it starts by saying the energy in the coal, but then says the mechanical energy which isn’t an IGCSE energy type. In the coal yard, the chemical energy stored in the coal is transferred to the surroundings as heat energy when the coal is burned. This energy change takes place in the boiler; in the coal yard, the coal isn’t burned so the energy is still chemical energy in the coal. In the coal crusher, the mechanical energy of the crusher's moving parts is used to crush the coal, transferring some of the energy to the coal in the form of kinetic energy. This would only be true if the coal continued to move after being crushed. In the boiler, the chemical energy in the coal is transferred to the water, heating it and turning it into steam. This energy transfer is due to the combustion of the coal, which releases heat energy. The energy change is chemical into heat energy. This answer again mixes up types of heat transfer and physical locations where the energy is present. In the steam turbine, the energy of the steam is transferred to the turbine blades, causing them to spin and creating mechanical energy. Firstly energy can never be created, so ‘creating mechanical energy’ is completely wrong. Secondly, at IGCSE we say kinetic energy. The energy change is heat into kinetic as the hot steam moves and spins the turbines. In the generator, the mechanical energy of the spinning turbine blades is transferred to the generator's rotating coil, producing a fluctuating magnetic field. This magnetic field then produces an electric current in the generator's stationary coil, transferring the energy to the electrical energy of the current. It should still read kinetic energy, but that kinetic energy is transferred into electrical energy. We don’t need to know hot it happens until Y11. In the transformer, the electrical energy of the current is transferred to the transformer's coil, causing a change in the voltage of the electricity. This energy transfer is due to the electromagnetic induction occurring in the transformer. This isn’t required until Y11.

You can read more on the link above, but essentially most of what the AI said wouldn’t be correct in an exam. What this shows is that while ChatGPT could be a good starting point, there’s a lot of work that students would need to do in order to get their answers into an acceptable form for their exams.

The students reaction

It was fascinating to watch students as they went through the exercise. At first, they were amazed by it, asking all sorts of questions from subject-specific ones like ‘describe the causes of the second world war’, to the esoteric like ‘should I eat avocado?’. However, as they started the work, they quickly became dissillusioned. Some of the comments I heard were:

  • What’s the point? Why don’t we just use the textbook?
  • Urgh, it takes so long to check everything, why can’t they just use the core knowledge document?
  • I keep thinking it’s find and then somebody points out another mistake.

Now, I’m luck that my class is extremely high-attaining and have strong subject knowledge. The average grade is a strong A*, with no students working below a B. I’m fairly confident that most students could get themselves to a grade B using the tool, but to gain anything higher than that they would have to spend more time on checking each answer against their textbook than if they just went to the textbook in the first place.

In summary, ChatGPT is an interesting tool and provides a nice set of starting answers, but it’s not going to revolutionise education. Because it doesn’t understand what it’s saying, none of its answers will be nuanced or explain the subtle differences in language that make good feedback important. I can see a role for for those with absurdly high class sizes, where individual feedback isn’t possible, but I don’t think it’s going to replace 1:1 conversations any time soon. Additionally, it can get a student currently working around a grade E to a B, but can’t really help the understanding of any students at a B or above, since its answers just aren’t specific enough.