The results are in: AI tutoring works

Feb 2, 2024

Lulu Gao | lulu@flintk12.com

Student looking up to AI hologram robot that is acting as an effective personalized tutor

Illustrated by AI, written by humans.

What if we could create a classroom with a 1:1 teacher-to-student ratio? What if students could get more frequent and detailed feedback on their writing without generating boatloads of extra work for teachers? Two studies have illuminated how both of these hypotheticals can become reality with the proper use of AI.

Harvard study: AI works as a 1:1 tutor

We’ve talked about the positive effects of 1:1 tutoring in a previous article, and a recent study from Harvard University has shown how AI can effectively approximate a 1:1 learning experience. Upon reviewing their research paper, our team noticed that the study’s motivations and goals coincided with our own:

Provide a “pedagogically-minded subject-matter expert” and “continuous, customized support” for students in course CS50, an introduction to computer science class
“Guide students toward solutions rather than offer them outright”
“Enable human educators to address more complex pedagogical issues”

During 2023, the consensus around AI changed from one of fear to one of inspiration. This study is one of the first to document the process and results of designing AI specifically for teaching purposes—it evaluated AI’s ability to act as a 1:1 tutor for students. The Harvard researchers developed AI tools that employed pedagogical principles to both empower student learning and enable teachers to refocus on bigger issues. Flint’s vision for AI’s role in education adds a few amendments to the Harvard experiment’s list:

Provide a “pedagogically-minded subject-matter expert” and “continuous, customized support” for all students regardless of curriculum difficulty, subject type, or age level
“Guide students toward solutions rather than offer them outright” and not only respond to student prompting but also prompt students through new forms of learning activities
“Enable human educators to address more complex pedagogical issues” including both focusing on the human elements of teaching (e.g. mentorship, communication, etc.) and experimentation with technology to discover new teaching methods

AI is now scientifically proven to be beneficial to learning because of how personalized and flexible it can be as a tutor. The next few sections of this blog with dive into what the Harvard team built, how their work relates to Flint, and feedback both the Harvard team and our team have received about AI’s impact on education.

Provide a “pedagogically-minded subject-matter expert” and “continuous, customized support” for students in course CS50, an introduction to computer science class
“Guide students toward solutions rather than offer them outright”
“Enable human educators to address more complex pedagogical issues”

Provide a “pedagogically-minded subject-matter expert” and “continuous, customized support” for all students regardless of curriculum difficulty, subject type, or age level
“Guide students toward solutions rather than offer them outright” and not only respond to student prompting but also prompt students through new forms of learning activities
“Enable human educators to address more complex pedagogical issues” including both focusing on the human elements of teaching (e.g. mentorship, communication, etc.) and experimentation with technology to discover new teaching methods

Provide a “pedagogically-minded subject-matter expert” and “continuous, customized support” for students in course CS50, an introduction to computer science class
“Guide students toward solutions rather than offer them outright”
“Enable human educators to address more complex pedagogical issues”

Provide a “pedagogically-minded subject-matter expert” and “continuous, customized support” for all students regardless of curriculum difficulty, subject type, or age level
“Guide students toward solutions rather than offer them outright” and not only respond to student prompting but also prompt students through new forms of learning activities
“Enable human educators to address more complex pedagogical issues” including both focusing on the human elements of teaching (e.g. mentorship, communication, etc.) and experimentation with technology to discover new teaching methods

Types of AI tools that are benefiting learning

The Harvard research team built three solutions for the CS50 students, each to fulfill different purposes:

check50: for explaining highlighted code, providing “instant code clarification”
style50: for suggesting improvements to code style
CS50 Duck: for conversationally answering questions posed by students

Aiming to serve K-12 independent schools without any set curriculum, Flint takes the basics of what the Harvard study has proven (that GPT-4 can be effective for tutoring) and builds upon it. Similar to the tools in the study, Flint’s essay feature has a highlighting clarification and suggestion function for coaching student writing and both the essay and chat assignments leverage a conversational approach to guiding students in their learning.

Flint also has built-in text-to-speech and speech-to-text capabilities to make AI tutoring accessible to more students and more immersive for different subjects. Lower school teachers like Meghan, a 4th grade teacher from Wesleyan, have been using text-to-speech and speech-to-text to make assignments more accessible for students still learning to type. World language teachers like Sebastian at St. Stephen’s and St. Agnes are using the same technology to give students more 1:1 conversation practice.

Examples of how real teachers are using Flint in class, including for time travel conversations, interactive storytelling, and career exploration dialogues.

Harvard’s study showed the benefits of incorporating AI into one specific course. The best way to scale this 1:1 teaching effect is to enable all teachers to leverage AI in each of their unique classrooms. Flint empowers teachers to build assignments that transform the common process of the student prompting the AI for answers into the AI prompting the student for explanations. Flint uses AI to creatively provide suggestions for assignment types that go well beyond the generic “tutor”—e.g. a presentation to Shakespeare about modern-day slang, a debate with JFK about Cold War strategies, a podcast about traveling in Latin America in Spanish, an interview for a software company to demonstrate coding skills, etc. Teachers and students alike can use Flint not only to personalize learning but also embrace their creativity.

To complete the feedback loop, Flint takes the power of AI a step beyond student use and leverages GPT-4 to summarize insights for the teacher about each student’s performance and the class as a whole. Students therefore get immediate, personalized feedback from the AI and teachers get distilled and actionable advice on where students have done well and where they can improve.

Screenshot of Flint's immediate AI-powered feedback for a student.

Example strengths, weaknesses, and follow-up summaries from the AI showing how teachers can not only view each student’s full transcript but also get class-wide analytics.

Student feedback on AI tutoring has been starkly positive

Of the students surveyed in the Harvard study, 73% deemed the tools to be “helpful” or “very helpful.” One student was even quoted as saying “Love love loved the duck. We’re friends now.” The accessibility and helpfulness of the 1:1 AI tutoring turned out to be not only supportive academically but emotionally as well, with student quotes demonstrating a boost in comfortability in asking questions and excitement about how AI will impact their future careers. Some other great feedback quotes included:

“AI bots will answer questions without ego and without judgment… it has an … inhuman level of patience”
“Gave me enough hints to try on my own”
“It was nice to already get acclimated to working alongside these tools”

The feedback we’ve received about Flint has echoed a lot of the same sentiments. Students enjoy the extra practice, are engaged by the novelty of assignments, and are learning more actively in their classes. With Flint, students who are normally quiet or not paying attention are given the space to respond to prompts at their own pace and knowledge level. It’s teaching them to advocate for themselves and to be engaged in their own education. Recently, a few members of our team visited a 4th grade class at Wesleyan and all the students enthusiastically cheered that they loved Flint. Teachers have also commented on how Flint is revolutionizing how they teach:

"Flint has really made my job a lot more fun. The kids love Flint, and are asking me to use it more!” - William Heyler, history teacher at ‘Iolani School
"What had once been a frustrating experience [learning complex biochemical pathways] became student-driven content formation.” - Cheryl Cossel, biology teacher at Episcopal Academy
"This is such a revolutionary tool for language learning!” - Sebastian Bartis, Spanish teacher at St. Stephens and St. Agnes
"Flint's impact doesn't stop at teaching. It has become an indispensable part of my daily routine, optimizing my workflow and acting as a personal assistant.” - Matthew Davis, computer science teacher at Episcopal Academy

Human teachers are still invaluable, as IJETHE study shows

Something I personally wondered about while reading the Harvard study was how the professors and teaching assistants felt about teaching alongside AI. As we’ve presented Flint to school leaders, we’ve sometimes been asked what the role of the teacher becomes if AI can be just as effective at teaching and the technology is only going to get better. Truth is, there’s a lot that we may see automated and powered by AI, but our team firmly believes there are uniquely human aspects of teaching that AI can enhance. If AI can automate a lot of the menial, manual work, teachers will be free to focus even more on interpersonal communication, social-emotional learning, and building empathy and trust with their students.

AI-generated image of a student looking up at and up to a teacher helping them with an assignment.

A study from the International Journal of Educational Technology in Higher Education explored the performance and preferences of students who got feedback from human teachers, from AI, and from both. The students studied were writing essays in English and had English as a new language. The conclusion was that “one method of feedback was not better than another in terms of scores” and each method had its benefits:

Benefits of AI feedback:

Time-saving for teachers so that they can focus on other tasks
Responses were deemed to be more clear and specific than those from an instructor

Benefits of teacher feedback:

Face-to-face interaction was more engaging
Opportunity for students to practice their speaking abilities

The study’s final recommendation was to use a blend of both human and AI feedback to provide the best of both worlds for students. Human teachers won’t be replaced, but if AI is proven useful for learning, then it’s time that teachers embrace using it to augment their teaching. Flint is dedicated to working with schools to find out what that future looks like.