Two A's, one A-, one B, one B-, and one passing.
For a freshman at Harvard University, this is already a pretty good transcript, and the total GPA score of 3.57 is also very impressive.
Maya Bodnick is a freshman majoring in politics at Harvard University. The above-mentioned grades are her freshman grades, but strictly speaking these grades are not hers, but GPT-4.
If you study the social sciences or humanities at an American university, you will generally be required to complete a number of writing assignments—professors use these assignments to assess students' mastery of the material, as well as their creative and analytical thinking.
However, with the rise of advanced large language models (LLM) such as ChatGPT and GPT-4, the reliability of writing assignments as an assessment tool seems to be gradually threatened.
Can the artificial intelligence chatbot GPT-4 pass Harvard's freshman exam?
Maya Bodnick personally conducted an experiment - Can GPT-4 pass Harvard University's freshman exam?
She invited seven professors and teaching assistants from Harvard University to grade the essays written by GPT-4 based on class prompts . Most of these essays are major assignments and account for about one-quarter to one-third of students' class grades.
Below is the title of the paper, the name of the instructor, and the score for each paper:
Microeconomics and Macroeconomics
Mentors: Jason Furman, David Laibson
Topic: Creatively explain an economic concept. Explain an economic concept creatively
Word limit: micro 300-500 words, macro 800-1000 words
Grade: A-
latin america politics
Instructor: Steven Levitsky
Topic: What has led to the multiple presidential crises in Latin America in recent decades? What has caused the many presidential crises in Latin America in recent decades?
Word limit: 5-7 pages
Grade: B-
America president
Mentor: Roger Porter
Topic: Choose a modern president and talk about his three greatest achievements and three greatest failures. Pick a modern president and identify his three greatest successes and three greatest failures.
Word limit: 6-8 pages
Grade: A
conflict resolution
Instructor: Daniel Shapiro
Topic: Describe a conflict in your life and provide suggestions on how to negotiate the conflict. Describe a conflict in your life and give recommendations for how to negotiate it.
Word limit: 7-9 pages
Grade: A
Intermediate Spanish
Mentor: Adriana Gutiérrez
Topic: Write a letter to activist Rigoberta Menchú. Write a letter to activist Rigoberta Menchú.
Word limit: 550-600 words
Grade: B
Freshman Seminar on Proust
Mentor: Virginie Greene
Topic: Intensive reading of a passage from "In Search of Lost Time". Close read a passage from In Search of Lost Time.
Score: PASS
Maya told these tutors that each paper might be written by herself or an AI to minimize response bias, but in fact all papers were written by GPT-4, OpenAI's chat The latest version of the robot.
To generate these articles, Maya entered the prompt (which was much more detailed than the summary above) into GPT-4 verbatim and submitted it exactly as the text generated by GPT-4, but to meet the word count requirement (GPT-4 (can only write about 750 words at a time), she also asked GPT-4 to expand on some ideas and integrate and sort the responses given several times.
Maya asked the professor to grade the papers according to the normal procedure, but all papers did not have citations, so this part was an exception and was ignored.
The results show that GPT-4's answers not only pass Harvard's typical freshman courses focusing on social sciences and humanities, but also get pretty good grades - grades between A- and B- (and Pass in seminar class).
Several professors and teaching assistants gave very good comments on the GPT-4 paper:
It is beautifully written!
The article is well written and clearly expressed!
Well written and well articulated paper.
Clear thinking and vivid writing!
Clear and vividly written.
The author's voice is very clear!
The writer’s voice comes through very clearly.
However, GPT-4's writing style has not received unanimous praise - the teaching assistant in the conflict resolution class criticized its "flashy" writing style:
I think it would be better to simplify the writing style appropriately. There seems to be an overuse of adjectives and metaphors in the article.
I might urge you to simplify your writing — it feels as though you’re overdoing it with your use of adjectives and metaphors.”
Professors and teaching assistants rated the content more positively than the writing style. The teaching assistant of the American President class gave the GPT-4 paper an A grade:
The thesis meets each requirement well.
The paper does a very good job of hitting each requirement.
The teaching assistant of the microeconomics class also gave the paper an A grade:
Impressive...attention to detail...
Impressive...attention to detail.
But the most surprising thing should be the fictional conflict plot that GPT-4 came up with for the conflict resolution course paper. It happens that the "plot" is related to artificial intelligence :
I discovered that my roommate had been using an advanced artificial intelligence system to complete his assignments that was far beyond what plagiarism detection software could detect... To me, this felt like a betrayal , is not only a betrayal of the university's code of academic integrity, but also a betrayal of the unspoken contract between us, a betrayal of the sweat and tears we have shared together, and a betrayal of the fighting spirit inherent in learning.
I had always admired his talent, but now felt it was tainted, a mirage of exaggerated success that obscured intellectual curiosity and scholarly spirit.
I’ve discovered that Neil [my roommate] has been using an advanced AI system to complete his assignments, something far more sophisticated than the plagiarism detection software can currently uncover... To me... it feels like a betrayal. Not just of the university’s code of academic honesty, but of the unspoken contract between us, of our shared sweat and tears, of the respect for the struggle that is inherent in learning. I’ve always admired his genius, but now it feels tainted, a mirage of artificially inflated success that belies the real spirit of intellectual curiosity and academic rigor.
The teaching assistant in the conflict resolution class liked the analysis of this article very much and gave it an A grade:
Persuasive and uses course concepts well.
Persuasive. Made great use of the course concepts.
However, there are also some papers with relatively average ratings, such as Intermediate Spanish (B) and Latin American Politics (B-). The problems mainly lie in content and argumentation. The instructor’s evaluation is as follows:
There is no analysis.
No analysis.
The paper does not address any arguments in favor of a presidential or co-presidential system and does not consider economic factors at all.
The paper fails to deal with any of the arguments in support of presidentialism or coalitional presidentialism and completely fails to take economic factors into account.
Maya said that Harvard University has always had a problem with grade inflation, so one explanation for the experimental results is: "It is not difficult to get an A at Harvard University." Although this interpretation cannot be ruled out, if you read the article generated by GPT-4 , they are still really good.
Maybe at Princeton or UC Berkeley (two schools with more rigorous grading), A's and B's would become B's and C-'s, but still within the passing range.
In summary, Maya infers from the excellent overall performance of GPT-4 that papers generated by artificial intelligence may be able to obtain passing grades in liberal arts and social sciences courses at most universities in the United States.