Guest Post: Student Researcher Cracks Open the Case for Classroom AR

As the Learning Lab continues to explore applications for augmented reality (AR) in higher ed – specifically, in the business-school setting – we are eager to give voice to fresh perspectives and innovative experimentation with the technology. This week we’re excited to hand our blog space over to Wharton student Jesse Cui, who recently served as an AR research assistant here in the Lab, so he can share his findings with you directly. (Take it away, Jesse!) 

image of mobile phoneAugmented reality (AR) is a potentially great game-changer in the classroom setting. It provides two new aspects to education that can greatly boost the learning experience: namely, control and visualization. As an undergraduate student-researcher in the Learning Lab, I worked on developing a proof of concept mobile app that is able to demonstrate the benefits of AR when studying more complex, difficult material.

For this app, I demonstrated how simple usage of AR can help students learn statistical concepts such as multivariate regression, which involves 3D visual aspects of visualization. This app allows users to visualize 3D data space with control over the visuals (unlike current 2D static images) by holding their phones up to a PowerPoint image. Users can also toggle regression on and off to see how it would appear in 3D space versus 2D.

Many people would imagine building an AR app would be difficult but, in fact, with the numerous amounts of software packages and libraries – including marker-based image recognition software and a variety of 3D objects to use in apps – developing AR apps has never been easier. For example, Vuforia is a company that provides software that allows developers to create model and image targets that can be easily identified and tracked in an AR app. Combining this technology with Unity, a game engine for developing apps and games that involves graphics, and a couple lines of C# scripts, I was able to create a fully functioning mobile app in no time. Another bonus with the combination of these technologies for AR app development is that you can easily build apps in both IOS and Android.

Tweet-Mining for AR Sentiment in the Twitterverse

Image result for twitter dataAnother part of my research under the Learning Lab was mining social-media feeds for performing sentiment analysis to gain a better understanding of the perception of augmented reality in the public sphere. Using GWU’s Social Feed Manager, an open-source software library that allows researchers to mine tweets from Twitter on a large-scale basis, I was able to collect a database of over 50,000 recent tweets surrounding AR and VR (virtual reality), and specific products surrounding these technologies. To dive deeper on the latter, I was able to pull tweets surrounding Microsoft’s HoloLens and the startup Magic Leap’s One AR glasses. Pulling tweets on these products can allow researchers like me to better understand market perception of different products.

However, in order to perform sentiment analysis on tweets, I trained a machine-learning classifier model –specifically, a deep Recurrent Neural Network (RNN) – that uses Long Short-Term Memory (LSTM) units with PyTorch. This type of model is often used for text analysis and natural language processing, since it keeps memory of information received previously. I opted against using a pre-trained model, since I wanted to use a model that was specifically trained on tweets. Then I pulled Sentiment140’s tweet dataset, which has 1.6 million tweets and sentiment labels for each tweet, and performed data cleaning and preprocessing, including stemming and tokenization of the tweets. Lastly, I trained the RNN on the tweets. Overall, the trained model had a testing accuracy of around 84%, which is decent for natural language processing.

Below are a couple of results I received from running the model on the AR/VR tweets.

  • Random tweets have .6403 (out of 1) positive sentiment (11122 total tweets) – used as a control
  • “Augmented Reality” tweets have .738 positive sentiment (16339 total tweets)
  • “Virtual Reality” tweets have .722 positive sentiment (25990 total tweets)
  • “HoloLens” tweets have .701 positive sentiment (6924 total tweets)
  • “Magic Leap One” tweets have .886 positive sentiment (266 total tweets)

Take from this what you will, but it’s great to see that AR/VR has overwhelmingly positive sentiments compared to the control, and I have no doubt that, as we continue to perfect the technologies, we will also continue to realize the benefits of AR as an education technology.

Jesse Cui is a dual-degree Wharton undergrad in the Jerome Fisher program in Management and Technology, who studies computer and cognitive science at the Penn School of Engineering and Applied Science, as well as operations, information, and decisions.