In this talk my project teammates and I present our work developing a plug-in cloud based inference engine to allow VR developers the ability to query an object recognition model for items around the player. We discuss how giving developers this ability allows for more immersive and embedded VR applications and show demonstrations of building such applications using our framework. Finally, we discuss some of the design decisions made during the project (such as hosting on a cloud server and keeping track of permanent world state information) that allow us to achieve close to zero reduction in frame rate when using our detection software – showing the viability of our method to also be used in resource-constrained applications.
This talk was given on May 3rd 2019 in the Wu and Chen Auditorium at the University of Pennsylvania in Philadelphia PA as part of the School of Engineering Senior Design Competition. Our team was one of three teams selected from the computer science department to present at the event.