Chemistry Solutions

©Shutterstock.com/Paulm1993

Back in 2023, the widespread coverage of artificial intelligence (AI), and more specifically ChatGPT, generated a lot of excitement. However, it also sparked fears about the potential jeopardy to our jobs as educators. One of the most widespread beliefs circulating was that ChatGPT could become a student’s private tutor.

As mentioned in our September article, we (a team of two co-teachers) have used this school year as an opportunity to explore how ChatGPT might be used in our chemistry classroom. We hoped to use AI to support student learning and for our own instructional planning and assessment. In this article, we’ll share how our experiences has unfolded.

Using ChatGPT as an educational tool

ChatGPT is indeed a very useful tool in the educator’s toolkit. In the past year, we have used it successfully in various situations, including:

• Generating a lesson plan with specific phenomena and activities, student learning objectives, pacing, lesson procedures, and differentiation strategies.
• Differentiating by converting an article to a suitable reading level for individual students.
• Formulating a 5E lesson plan.
• Writing in a lesson plan submission portal (which some teachers are required to do).
• Writing weekly summary emails to students and parents highlighting what we did in class and upcoming activities/topics.

One of our most valuable uses of ChatGPT was to generate performance task questions for certain topics. In our experience we found that, in order to generate a “workable” question, the initial prompt we provide with a sample question was very important. For instance, at the end of our Chemical Reactions unit, we provided each team of students with a performance task generated by ChatGPT. The initial prompt we used is shown in Figure 1, the results we received are shown in Figure 2, and some student answers to performance task questions are shown in Figure 3.

After getting the results, we made some simple edits and tweaks to tailor the questions to our needs. Sometimes we would just change the question layout ourselves, and other times we would add more prompts.

Student interaction with ChatGPT

Unfortunately, our students’ experience using the tool independently did not work as well as we hoped. As mentioned in our previous article, we had planned to use ChatGPT to help our students self-monitor and review concepts where they needed the most help. Our hope was, if we could help students use the formative assessments we create at the end of each topic to identify where they needed the most help, they could use ChatGPT as a tutor to create flash cards, outline notes, or more examples related to a given practice problem. Figure 4 shows an example of a reflection assignment that we created and provided to students after a formative assessment.

We encountered multiple problems with this plan. One of the biggest issues we found was that many students who were struggling with content also had difficulty self-regulating, when asked to use ChatGPT. For example, they needed more help with typing the prompt correctly, so it ended up being easier for us to show them our notes, or work out a quick example on the whiteboard.

Another major problem with using ChatGPT for this purpose was that it often provided wrong answers to the students. For example, during our review of the gas laws, we assigned a set of calculation problems. If a student entered a prompt to ChatGPT for help with a calculation using Charles’s Law (volume versus temperature), then followed with a second prompt related to Boyle’s Law (pressure versus temperature), ChatGPT assumed that the relationship between the variables was the same in both instances and solved the problem incorrectly.

Out of curiosity, when we tried a few combinations of these problems and other questions, we found a lot of erroneous results. This experience made us change our plans for how we use ChatGPT.

On the other hand, we’ve also had successes. For example, when we see students struggling in a specific area, we can quickly use ChatGPT to generate more practice questions for them. In one case, one of our students had a difficult time in balancing chemical equations and asked for an extra set of practice problems to work on with a tutor. It took us just a few minutes to create, then transfer the generated questions from ChatGPT to a printed document. ChatGPT can then be used to create an answer key for the worksheet generated; however, based on our experience with the AI tool’s inaccuracies, we know it is an essential practice to check the generated answers.

Looking to the future

Based on our few months of experience using ChatGPT in the classroom, here are our major take-aways:

• Accuracy and reliability of responses: One of the biggest challenges we encountered is the inconsistency and inaccuracy of responses generated by ChatGPT, particularly when students use it independently.
• Difficulty in self-regulation: Students who are struggling with concepts may have difficulty self-regulating their use of ChatGPT effectively. This could be due to challenges in typing prompts correctly or difficulty in interpreting and applying the generated responses.
• Lack of contextual understanding and complexity of subject matter: Improving the contextual understanding of AI systems by enriching the training data with more nuanced and domain-specific examples could help address this challenge.
• Need for professional development: Educators may require more guidance and professional development to effectively integrate ChatGPT into their teaching practices. This may also improve the training of datasets within ChatGPT. As noted below, the authors have become active in advocating for professional development, on AI in general and ChatGPT in particular.

We are definitely looking forward to receiving more guidance in terms of how educators and students can use AI in school systems, and at the district and state levels. Only 2% of the state edtech leaders say their state has an AI initiative.¹ The field of education is beginning to transform how it uses AI, and a lot depends on the dataset these models are trained on. AI systems in education often rely on vast datasets to train machine learning models, which then analyze patterns and make predictions or recommendations based on the data.

Chemistry and math are complex subjects, with intricate concepts and principles. If the training data does not adequately represent the complexity and nuances of these subjects, ChatGPT itself may struggle to comprehend them and generate coherent responses for queries that require a deep understanding of advanced topics. These discussions will not arise if educators themselves don’t explore these tools and become more familiar with them and their capabilities.

In our opinion, training should be provided at both the district and state levels to allow educators to fully engage with these resources. Increased usage by educators may enhance the proficiency of these tools and contribute to a deeper contextual understanding of AI systems. This, in turn, may prompt further training of the models, facilitated by increased educator involvement.

While our experience has demonstrated the potential for ChatGPT as a helpful aid for teachers in generating lesson plans, creating practice worksheets, and developing different versions of assessments, we have also encountered limitations in students’ usage of it. It is crucial for educators to ensure that the training dataset encompasses a wide range of accurate and representative examples from the subject domain. Moving forward, we must leverage the advantages of using ChatGPT as a tool, while also addressing its limitations.

One of our ideas was inspired by a conversation between students that we overheard in the classroom. They mentioned that when they ask ChatGPT to explain a concept, they always include a prompt: “Explain it as you would to a fourth grader.” This advice resonated with us and could be beneficial, especially before final exams if students need a refresher on a concept.

We are actively engaged in advocating for professional development and sharing our experiences with educators within our district. Anjana has been involved in Turnkey Technology Training for Teachers, specifically focusing on the use of AI tools like ChatGPT. This effort aims to create a network of educators who can support each other and foster collaboration and knowledge sharing among educators, administrators, and edtech leaders.

Additionally, we plan to advocate for the implementation of AI initiatives at the district level to support educators and students in effectively leveraging AI technologies. As we continue to navigate the evolving landscape of AI in education, we remain committed to open-mindedness, thoughtful experimentation, and reflective practice to thrive in this digital world.

References

1. Kuykendall, K. Cybersecurity, AI Policy Top List of State Ed Tech Priorities. Blog post on The Journal website. Available at https://thejournal.com/articles/2023/09/18/cybersecurity-tops-list-of-state-ed-tech-priorities.aspx (accessed Apr 8, 2024).