How one college has brought AI into the classroom

Most colleges and universities are employing artificial intelligence for administrative affairs, but Rensselaer Polytechnic Institute is putting this technology into the hands of professors
By: | Issue: May/June 2020
May 21, 2020
RECOGNIZE AND RESPOND—At Rensselaer Polytechnic Institute, the Situation Room captures human locations, movements and speech patterns using advanced cameras, microphones and sensors. Cognitive computing technologies are then applied so that digital avatars can recognize and respond to what humans are doing.

After stepping into the Situation Room, Rensselaer Polytechnic Institute students are “transported” to a restaurant in China. A computer-generated waiter walks up to them and asks, in Mandarin, what they would like to eat. These students then respond, to which the virtual waiter reacts through the use of artificial intelligence.

Later, a Beijing airport materializes around these students who must now retrieve their luggage and find transportation to their next destination with the help of AI-powered digital avatars.

“Or there might be an object that students have to find in Beijing, which would involve negotiating their way through the city and interacting with digital individuals along the way,” says Shirley Ann Jackson of the New York-based institution.

Called the Mandarin Project, these experiences at RPI’s Cognitive and Immersive Systems Lab combine gamification, immersion and AI to improve the ways in which students learn Mandarin.

Beyond RPI, higher ed mostly employs AI outside of the classroom for research and development. In addition, AI is being used for large-scale data collection and reporting to, for example, drive down the cost of instruction and assessments, for example, says Brian Fleming, executive director of the Sandbox
ColLABorative at Southern New Hampshire University. Chatbots are used to answer student questions virtually and preserve staff resources.

To some extent, AI powers digital courseware for personalized learning and large learning platforms, such as YouTube. “There are also applications of AI or, at least, machine learning that accelerate the speed of learning platforms,” says Fleming, a member of Harvesting Academic Innovation for Learners (HAIL) Storm, a network of higher ed leaders who pursue transformational change.

“The important part of education is helping students think, and most computers are only programmed with AI algorithms to answer questions, not pose questions,” says Stefan Popenici, co-author of the research paper “Exploring the impact of artificial intelligence on teaching and learning in higher education” (UBmag.me/popenici).

Who controls the data?

As AI solutions continue to enter the market, faculty and campus administrators need to limit or at least understand where data gathered by AI goes.

Higher ed chatbots everyone is chatting about

• Arizona State University: Chatbot Sunny
has sent over 3 million texts to students on various subjects, such as deadlines, requirements and financial aid.

• Winston-Salem State University (N.C): Chatbot Winston increased enrollment by 2%, decreased incoming phone calls by 36% and increased on-time bill payments by 74% in 2017-18.

• Georgia Institute of Technology: Jill Watson, built on IBM’s Watson platform, is a teaching assistant that helps students enrolled in the Knowledge-Based Artificial Intelligence course.

• The University of Southern Denmark: Chatbot Kitt receives all IT service department inquiries from employees and students.

• Deakin University (Australia): IBM Watson received 1,600 questions a week to learn the ins and outs of campus life and how to study in the cloud in 2015.

• University of Adelaide (South Australia): An international eligibility assessment chatbot that uses Oracle Digital Assistant has participated in more than 9,000 sessions, since June 2019, with prospective students without needing to involve university staff.

Source: Stefan Popenici, co-author of the research paper “Exploring the impact of artificial intelligence on teaching and learning in higher education”

“Even though a vast majority of researchers don’t believe artificial intelligence can read or engage with student emotions, schools need to realize that students will gradually realize how much of their data AI can collect and will hold universities accountable,” says Popenici, also a senior lecturer in higher education and development at Charles Darwin University in Australia. “What kind of messaging do we want to give students? That we have complete control of their data?”

Campus leaders and faculty, therefore, need to ask difficult questions when looking to adopt new technology, such as who is going to manage the data and what the long-term costs are. Another option would be for schools to develop their own AI solutions, though this involves a great deal of database management and protection.

“The most significant advancements in tech started in academia,” says Popenici. “But the more complex these types of AI interactions are, the more technical support and financial resources you will need.”

Higher ed institutions also need to avoid technology that will drive up the cost of education, such as platforms that require payment from students. “If higher ed is not responsible for this technology, then who will be?” says Fleming.

Providing cultural familiarity and sensitivity

At RPI, the Mandarin Project aims to ensure students will make a difference in the world when traveling abroad. This requires mastering the languages and cultures of the places they visit.

“As a technological institution, our foreign language [offering] was not large enough to equip our students with the necessary skills to successfully navigate the world,” says Jackson. “We were interested in providing experiences that would give them cultural sensitivity and the sense that, when visiting a country for the first time, they had been there before.”

The institute started the project by equipping a classroom space at RPI, which was already outfitted with technology, to create a human-scale immersive environment, with Watson, a data-driven and knowledge-based IT services platform from IBM.

“We chose to teach Mandarin in this lab because it is a level-four language, which means it’s not only an important language to learn, but challenging to master,” says Jackson.

CISL Mandarin Project Scene Students Cognitive and Immersive Systems Lab

Accelerating instruction

Before incorporating artificial intelligence, the Mandarin Project used gamification to accelerate the language acquisition and immersion to help students gain cultural competence and a sense of familiarity with where they could later go.

Originally, students only interacted with human professors who spoke Mandarin in this immersive environment. “Using actual teachers slowed down the process of accelerated learning, so we decided to replace humans with a digital being,” says Jackson. “But to make interactions with these digital avatars more natural, we needed artificial intelligence.”

The first level involved a team of computer engineers, game developers and artists from the institute adding responsive and verbal queries, so the avatars could answer questions and point to objects.

“However, we wanted to go beyond a Q&A-type of interaction,” says Jackson. “To make it more real, we began adding complexity and nuance.”

This involved coding facial recognition and motion capture capabilities into these avatars. Additional coding was later inputted to allow avatars to interact with more than one student and then more coding was added so multiple digital avatars could interact with each other.

Meanwhile, all of these changes needed to be integrated into natural language processing, a branch of AI that deals with the interaction between computers and humans using the natural language.

“Even though we started with gamification and immersion,” says Jackson, “our use of artificial intelligence was what helped us achieve our pedagogical intent to accelerate and increase student cognitive learning.”

Steven Blackburn is an associate editor at UB.


Interested in technology? Keep up with the UB Tech® conference.