In-Depth

Can teachers tell if you use AI text generators?

Henry Manning and Ella Furuichi|May 31, 2023

Archie Williams English and AP Language and Composition teacher Mathew Leffel finds comparison of ChatGPT to student-written essays to be similar to the Turing test.

The Pitch created a study to find out

As the use of technology continues to infiltrate modern education, some educators are turning to language models like ChatGPT to help teach writing skills. But, critics argue that this reliance on AI technology in the classroom may have negative consequences for student learning and writing skill. While ChatGPT can assist with tasks like generating ideas and improving grammar, it may also stifle creativity, critical thinking, and human connection in the writing process.

Guess what? ChatGPT wrote the paragraph above.

It’s undeniable that the AI language model is capable of putting out quality writing and original work. Its ease of access to students creates the perfect way to finish a late assignment without putting in effort. As an increasing number of students use ChatGPT for school, it is clear that the effects of assisted learning cause an absence of real learning pathways in a student’s education.

Programs similar to ChatGPT, created by the San Francisco-based company OpenAI, have erupted across many platforms, from Google to Snapchat. These new AI programs affect English and writing classrooms most of all. By inserting essay prompts and specifics, students can have an A-quality essay in seconds. In combating the issue, many teachers use AI-checking platforms.

The Pitch conducted a study with three AP Language and Composition (AP Lang) teachers from Archie Williams. The Pitch took an old AP Lang prompt, “What, if anything, is the value or function of polite speech? Develop your position on the value or function of polite speech in a culture or community with which you are familiar,” and asked ChatGPT to respond with an essay.

The three teachers received the ChatGPT essay along with a student-written one, both without labels. The teachers evaluated both essays, compared them, and later spoke on their views about the evolution of AI in schools. The three teachers had varied opinions on the essay responses.

AP Language and Composition teacher Greg Doherty believes that AI language models can follow writing formulas and deliver a solid essay or paragraph, but the program may not be able to demonstrate writing personality.
(Elliot Smith)

Greg Doherty, AP Lang and World Lit and Humanities teacher, correctly guessed which essay was AI-produced and student-produced, due in part to the formality of the AI essay.

“The [AI-written essay] showed a really good understanding of the task, in the sense that it really structurally followed the traditional formula…and it never really strayed from that structure,” Doherty said. “The [student-written essay] did not have a clear thesis statement anywhere, although it did answer the question… just structurally, it was a little unorthodox.”

Doherty believes that AI language models can follow writing formulas and deliver a solid essay or paragraph, but the program may not be able to demonstrate writing personality. The first essay, which he thought to be AI-written, did not connect its ideas to larger social trends separate from the prompt.

AI has a perfect structural system. This means certain trends can appear in the text and with few flaws. Despite this, assuming whether a student’s work is AI-generated or not can prove to be difficult.

Archie Williams English and AP Language and Composition teacher Mathew Leffel finds comparison of ChatGPT to student-written essays to be similar to the Turing test, or the Imitation Game, a test designed to show if a human could tell the difference between computer and human behavior. Leffel believes that determining whether text is human or AI-generated is difficult, pointing out that the AI detectors are somewhat ineffective.

“I didn’t detect a lot of distinctive markers between the two; I could not tell one from the other,” Leffel said. “[AI is] a dynamic system. It’s a kind of feedback loop. So if you set up the tool to detect certain elements, it will learn from that and it will advance. So at this point, I’m not relying on any type of technology.”

Chizzie Brown, Archie Williams English and former AP Language and Composition teacher, noticed that certain trends in the text arise from the Chat GPT language model. At the beginning of each paragraph, the AI uses run-on sentences and transitional expressions. The human-written essay barely utilized transitional expressions.

According to Government Technology, 30 percent of college students have reportedly used ChatGPT to complete a written assignment. Of that 30 percent, 60 percent use it on more than half of their assignments. With many students using this program to cheat, questions arise surrounding its possible effects on students’ quality of education and their ability to perform in their future careers.

“If a student has relied on AI to do the thinking for them, it could diminish their ability to to have original thought. And I do see for a lot of students a sort of creative loss,” Brown said. “If we stop building our imagination and creativity, that’s where students will show up in careers feeling somewhat limited in their abilities.”

Doherty agreed with Brown’s perspective, seeing students’ work ethic decline when immediately turning to the help of technology or AI.

“[Students will] avoid practicing learning how to think and hand that over to a computer, and then their brains don’t ever develop those neural pathways and people shortchange themselves in the interest of a grade,” Doherty said.

Both Doherty and Brown were correct in their belief that the first essay was ChatGPT produced, but all teachers agreed it was a difficult task to decipher the differences between them.

While it’s inevitable that students will use AI for school, it’s possible that implementing AI into class curriculum could become a reality. Using AI written examples as a tool for learning structure and conventions could become part of a curriculum. Doherty has already begun using it in his classroom for debate topics and class discussions.

“You can’t stick your head in the sand and pretend that it doesn’t exist. We have to work with the world we have,” Doherty said, “So if this is out there, then the question becomes, how do we use it best? How do we want to use it, because it’s going to change how people do everything.”

Sample Essays: Can you tell which one is from Chat GPT?

Essay #1

Polite speech is a crucial aspect of communication in any culture or community. It is a way to show respect, build relationships, and establish trust. Although some might argue that polite speech is insincere and merely a social convention, I contend that it serves a valuable function in our daily lives.

Growing up in an Asian-American household, I have learned firsthand the importance of polite speech. We are taught to address our elders and authority figures with honorifics, such as “Auntie” and “Uncle,” to show respect. We are also expected to use polite language in formal settings, such as job interviews and meetings. Polite speech is seen as a way to demonstrate our upbringing, education, and social status.

In addition to demonstrating respect, polite speech can also help build relationships. When meeting someone new, using friendly phrases like “Nice to meet you” or “How are you?” can help establish a rapport and put the other person at ease. In this way, polite speech serves as a social lubricant that helps us connect with others.

Furthermore, polite speech can help us avoid conflict and maintain harmony in our relationships. For example, when we disagree with someone, using polite language can help us express our views without offending the other person. By using phrases like “I understand your point, but I respectfully disagree,” we can maintain a civil and respectful conversation. Polite speech can also help us apologize and make amends when we have hurt someone’s feelings. By using phrases like “I’m sorry” and “Please forgive me,” we can show that we are taking responsibility for our actions and seeking to repair the relationship.

Some might argue that polite speech is insincere and superficial, merely a way to maintain social norms and avoid confrontation. However, I believe that polite speech can be genuine and authentic.

When we use polite language, we are expressing our desire to be respectful and kind to others. Even if we don’t necessarily feel like saying “Nice to meet you,” we can still use these phrases as a way to show that we value the other person and want to make a positive impression.

In conclusion, polite speech serves a valuable function in our daily lives. It helps us show respect, build relationships, and maintain harmony in our interactions with others. While some might view it as insincere or superficial, I believe that polite speech can be genuine and authentic, and that it reflects our desire to be kind and respectful to others. As such, we should continue to value and use polite speech as an important aspect of our communication in our culture and community.

Essay #2

In society there are certain standards that are expected to be met. It is expected that when meeting new people or old friends, you communicate with age old phrases. It is not required necessarily, it is just the norm that is expected. But what function does typical, everyday politeness serve? Why do we never come up with new phrases?

As a worker in customer service, I find myself obligated to inquire how my customers are feeling. It may not be much more than socially expected, but a polite, “How are you?” can be more than a colloquial phrase. When asked, “How are you?”, one has a sense of being cared for. The speaker may not mean anything by the phrase, aside from standard, but knowing that someone cares enough to inquire after their well-being can mean a lot to a person. Sometimes I will fail in asking this, but when I do, the customers, especially the older generation, seem genuinely surprised. Politeness seems to be a thing of the past, but when one is polite or principal, the mood of a room can change with a simple phrase.

As a teenager in today’s society, it is hard to break the social stereotypes placed on us. Media and social networking have made my generation an egotistical, self helping, arrogant generation with no respect for themselves, peers or elders. Politeness and chivalry are dying concepts, and with the media only encouraging their destruction, it is difficult for teenagers to stand out from stereotypes. Polite speech is also a showing of respect. When asking someone how their day is going, one is saying that whether they want it or not, they care about the other person. Most often, people respond with the expected, “I am doing well, how are you?”, but sometimes the question may begin a conversation that has the potential to bloom into the beginnings of a new friendship.

Politeness is expected, rudeness is socially unacceptable. There are things one is supposed to say, and things they are not. Then there are things that people say that go beyond expectations. Inquiring after a person’s well-being is a kind gesture, but offering a compliment is far more appreciated. People expect normal expected phrases but when one goes beyond the expectation great things happen.

Polite speech serves a purpose and we can take that out of the bare minimums, but when we go the distance ,and are kind as well as polite, we present answers in a newer, friendlier light. Don’t fit the typical mold, go beyond expectation and offer a little more of yourself. Sometimes you get a response, other times you don’t. But no matter the response, you will leave the situation feeling a better person.