- AI and XR — The Advent of Intelligent Reality for Clinical Medicine and Healthcare, Anthony Chang
- Impossible outside Virtual Reality, Mar Gonzalez Franco
- Towards pervasive, immersive, and augmented health management – where do we stand?, Ilkka Korhonen
- Homogenization and Racial Biases in the Beauty-verse, Nuria Oliver
- Reinforcing and Eliciting Behaviour in Virtual Reality Using Closed Loop Learning, Mel Slater
- Intelligent XR: Fusing Extended Reality and Artificial Intelligence, Frank Steinicke
Session Details and Keynote Bios
AI and XR — The Advent of Intelligent Reality for Clinical Medicine and Healthcare
Anthony Chang, Chief Intelligence and Innovation Officer (CIIO) and Medical Director of the Heart Failure Program at Children’s Hospital of Orange County, AIMed Founder, USA
Abstract: The combined resources of artificial intelligence and extended reality can leverage this synergistic technological dyad for a myriad of areas in clinical medicine and healthcare.
In medical education, many topics in biomedicine can be much better taught with this combined resource that can provide a gaming dimension. Many tools with extended reality and artificial intelligence can be deployed for clinical training, including procedural training. Developing a virtual twin is the precursor to precision medicine for all patients including preoperative planning and rehabilitation. Lastly, tools that leverage both extended reality and artificial intelligence can also provide an effective platform for patient care and experience.
Bio-sketch: Dr. Chang attended Johns Hopkins University for his B.A. in molecular biology prior to entering Georgetown University School of Medicine for his M.D. He then completed his pediatric residency at Children’s Hospital National Medical Center and his pediatric cardiology fellowship at the Children’s Hospital of Philadelphia. He then accepted a position as attending cardiologist in the cardiovascular intensive care unit of Boston Children’s Hospital and as assistant professor at Harvard Medical School.
He has been the medical director of several pediatric cardiac intensive care programs (including Children’s Hospital of Los Angeles, Miami Children’s Hospital, and Texas Children’s Hospital). He served as the medical director of the Heart Institute at Children’s Hospital of Orange County.
He is currently the Chief Intelligence and Innovation Officer (CIIO) and Medical Director of the Heart Failure Program at Children’s Hospital of Orange County. He has also been named a Physician of Excellence by the Orange County Medical Association and Top Cardiologist, Top Doctor for many years as well as one of the nation’s Top Innovators in Healthcare.
He has completed a Masters in Business Administration (MBA) in Health Care Administration at the University of Miami School of Business and graduated with the McCaw Award of Academic Excellence. He also completed a Masters in Public Health (MPH) in Health Care Policy at the Jonathan Fielding School of Public Health of the University of California, Los Angeles and graduated with the Dean’s Award for Academic Excellence. Finally, he graduated with his Masters of Science (MS) in Biomedical Data Science with a subarea focus in artificial intelligence from Stanford School of Medicine and has completed a certification on artificial Intelligence from MIT. He is a computer scientist-in-residence and a member of the Dean’s Scientific Council at Chapman University.
He has helped to build a successful cardiology practice as a startup company and was able to complete a deal on Wall Street. He is known for several innovations in pediatric cardiac care, including introducing the cardiac drug milrinone and co-designing (with Dr. Michael DeBakey) an axial-type ventricular assist device in children. He is a committee member of the National Institute of Health pediatric grant review committee. He is the editor of several textbooks in pediatric cardiology, including Pediatric Cardiac Intensive Care, Heart Failure in Children and Young Adults, and Pediatric Cardiology Board Review.
*Click here to learn more about Dr. Anthony Chang and view his extended bio.
Impossible Outside Virtual Reality
Mar Gonzalez Franco, Principal Researcher, Microsoft, USA
Abstract: For immersive technologies to become a primary form of interaction with digital content, we need to understand what types of things we can do in them that would be impossible with 2d screens. I will be talking about my technical work on avatars, locomotion and haptics. And some of our findings related to the challenges on the new interaction needs and accessibility.
Bio-sketch: Mar Gonzalez Franco is a researcher in the EPIC (Extended Perception Interaction and Cognition) team at Microsoft Research. In her research, she tries to advance Spatial Computing by building new devices and experiences. All while studying human behavior, perception, and neuroscience. (Find out more: https://margonzalezfranco.github.io/)
Mar Gonzalez Franco graduated top of her class in Computer Science from URL, Barcelona 2008, and completed a Master in Biomedical Engineering in a joint program between Universitat de Barcelona and Tsinghua University in China 2010. In 2014 Dr. Gonzalez Franco earned her PhD in Immersive Virtual Reality and Clinical Psychology under the supervision of Prof. Mel Slater at the EVENT-Lab, affiliated as a visiting student at the Massachusetts Institute of Technology, MediaLab. During her PhD, she produced anarchic hand experiences into healthy humans (link), as well as stabbed their virtual hands (link). Altogether to explore how our brain perceives our body and search the underlying mechanisms that determine what is real and what is not. Technologically, Mar was using real-time computer graphics, HMDs, and body tracking systems.
Over the years her work has evolved in three ways: (1) to continue exploring the wonders of VR and spatial computing; (2) to create new devices and explore what these technologies do to our perceptual systems, most prominent is her work exploring new haptic devices and spatial audio; and (3) to understand how people behave at large when presented with technology. In that context, Mar have an ongoing collaboration with the Soundscape Team, and completed a post-doc at the University College London’s Virtual Environments and Computer Graphics (UCL-VECG) group where she explored obedience to authority paradigms, and whether people would electroshock an avatar (Scientific American).
Dr. Gonzalez Franco’s interest in impacting larger audiences inspired her to pivot from academia into industrial laboratoriesl, and after having been at MIT, Tsinghua, UB, and UCL, she created and led an immersive tech lab at Airbus Group; joined the Startup, Traity, working on the creation of reputation standards for a user base of 4.5M. Since 2016, Mar has been with Microsoft Research.
Despite the shift into industry, Mar is still deeply involved in the scientific community where she often acts as an expert advisor to governments (US NSF, Canada NSERC, European Commission). In addition, she has published, served as program committee, chair, and reviewed in multiple venues (IEEE & ACM conferences and transactions, Nature Publishing group, Science Robotics). She is also very keen on disseminating her views on how technology companies and industrial labs should run, as well as the implications of technologies for the general public. In that role, she has been invited to write several commentaries on her work for Scientific American, and has done keynotes and interviews with multiple Media outside of the VR world (Bloomberg, GEN summit). Mar Gonzalez Franco also gained some awards as junior technology leader to follow (Business Insider ES 2019 MAS Technology Award 2019 and Next Generation Scientist 2021). Since 2020, she is also the Ethics and Diversity Chair of IEEE VGTC.
*Click here to learn more about Mar Gonzalez Franco and view her extended bio.
Towards pervasive, immersive, and augmented health management – where do we stand?
Ilkka Korhonen, Adjunct Professor, Faculty of Biomedicine and Health, Tampere University, Finland
Featuring Isaac Galazer-Levy, NYU Grossman School of Medicine, New York, USA
Abstract: Health is a combination of physical, mental and social health. Health outcomes are to a large extent related to health-related behaviours, such as diet, physical activity, sleep, stress, smoking, substance abuse, medication adherence, and, in case of communicable diseases, to adherence to personal hygiene, use of face masks, and social distancing. Today, wearable and ambient health sensors offer pervasive monitoring of physical health status, and also quantify increasingly accurately also health related behaviours. The development is towards even more ubiquitous, continuous and accurate sensing technologies, and in the near future we will be able to seamlessly monitor most important elements of our physical health. However, the data alone does not transform into better health outcomes, and physical health parameters represent only part of our overall health status. We make health outcome affecting decisions – consciously and unconsciously – hundreds of times per day.
These decisions, and health behaviours, are affected by social, mental, contextual and physical factors, as well as intrinsic motivations, preferences, and biases. To improve outcomes, we have to better understand the factors affecting these decisions and be able to design immersive interventions which seamlessly integrate to our daily lifestyles, and elevate our very natural need for social support and affect from our peers. In this keynote, some concrete examples of such modern technologies and future visions are presented.
Bio-sketch: Professor Ilkka Korhonen is currently an Adjunct Professor in Faculty of Biomedicine and Health, Tampere University, Finland. He has more than 30 years of expertise in HealthTech, wearables, algorithms and sensing technology, research and translating research into products, partner relations, and business development. Korhonen’s current research focuses on efforts to make the consumer a central user of health information and to use health tech to transform our behaviors to promote health and wellness. During his career, he has studied personal health systems, eHealth/mHealth, biosignal interpretation, health monitoring, and wearable health sensors, and even decision making based on sensor data in critical care and anesthesia. He has made several patents and inventions, contributed to several commercial products, businesses, and concept designs. He has over 200 international scientific publications, and he is a founding member of IEEE EMBS TC on Wearable Biomedical Sensors and Systems, member of IEEE EMBS TC for Biomedical and Health Informatics, and Senior Member in IEEE EMBS.
Homogenization and Racial Biases in the Beauty-verse
Nuria Oliver, Chief Data Scientist at DataPop Alliance, USA
Abstract: The Metaverse may be conceived as the culmination of the digitization of our lives and our society, leveraging key technological advances in fields, such as Augmented, Virtual and Mixed Reality and Artificial Intelligence. From a societal perspective, the Metaverse is considered to be the next stage in the development of current social media platforms. In this regard and similarly to what happens on social media, the broad adoption and use of the Metaverse by potentially billions of users poses significant ethical and societal challenges, including the need to develop an inclusive environment, respecting the diversity of its users. Thus, it is of paramount importance to ensure that the enabling technologies of the Metaverse do not create, replicate or even exacerbate patterns of discrimination and disadvantage towards specific groups of users: fairness and diversity should be at the foundation of its development.
In my talk, I will focus on diversity in self-representation in the Metaverse. Specifically, the impact that Augmented-reality (AR)-based selfie beautification algorithms have on diversity and the implicit racial biases that might be present in such filters. I will highlight the importance of coupling the proposal of novel technical contributions for the Metaverse with a comprehensive, multidisciplinary study of their societal implications. This is work led by PhD student Piera Riccio.
Bio-sketch: Nuria Oliver, is Chief Data Scientist at Data-Pop Alliance, Chief Scientific Advisor at the Vodafone Institute and co-founder of ELLIS (The European Laboratory for Learning and Intelligent Systems) and co-founder of the Alicante ELLIS Unit, devoted to research on “Human(ity)-centric Artificial Intelligence”. She is a Telecommunications Engineer from the UPM and holds a PhD in Artificial Intelligence from the Massachusetts Institute of Technology (MIT). Since March 2020, she was named Commissioner for the President of the Valencian Region on AI Strategy and Data Science to fight COVID-19. Since then, she has led at team with ~20 data scientists and co-lead the winning team of the XPRIZE Pandemic Response Challenge.
She has over 25 years of research experience in the areas of human behavior modeling and prediction from data and human-computer interaction. She has been a researcher at Microsoft Research (Redmond, WA), the first female Scientific Director at Telefonica R&D for over 8 years and the first Director of Research in Data Science at Vodafone globally (2017-2019).
Her work in the computational modeling of human behavior using Artificial Intelligence techniques, human-computer interaction, mobile computing and Big Data analysis – especially for the Social Good is well known with over 160 scientific publications that have received more than 18000 citations and a ten best paper award nominations and awards. She is co-inventor of over 40 filed patents and she is a regular keynote speaker at international conferences. Her work has contributed to the improvement of services, the creation of new services, the definition of business strategies and the creation of new companies.
Nuria is the only Spanish researcher recognized by the ACM as Distinguished Scientist (2015) and Fellow (2017) at the same time. She is also a Fellow of the IEEE (2017) and the European Association for Artificial Intelligence (2016). She has received an Honorary Doctorate from the Miguel Hernandez University (2018). Dr. Oliver is the youngest and fourth female member of the Spanish Royal Academy of Engineering (2018) and an elected member of the Academia Europaea (2016) and CHI Academy (2018), where she is the only Spanish scientist.
As an advisor, Dr. Oliver is a member of the scientific advisory committee of several European universities, the Gadea Ciencia Foundation, Mahindra Comviva and the Future Digital Society, among others. In addition, she advises the Government of the Valencia Community, Spain and the European Commission on issues related to Artificial Intelligence. She is a member of a Global Future Council at the World Economic Forum.
She has been named one of the top 11 Artificial Intelligence influencers worldwide by Pioneering Minds (2017), one of Spanish wonderful minds in technology by EL PAIS newspaper (2017), “an outstanding female director in technology” (El PAIS, 2012), one of “100 leaders for the future” (Capital, 2009) and one of the “40 youngsters who will mark the next millennium” (El PAIS, 1999).
*Click here to learn more about Nuria Oliver and view her extended bio.
Reinforcing and Eliciting Behaviour in Virtual Reality Using Closed Loop Learning
Mel Slater, Distinguished Investigator at the University of Barcelona, and Co-Director of the Event Lab
Abstract: The use of AI in VR is typically to improve some aspect of the functioning of the environment, such as endowing virtual characters with some intelligence. However, another possibility is to use AI techniques to either gather information from, or influence the human participants towards particular actions. In this talk I will describe a number of applications where we have used reinforcement for these purposes. I will describe the European project GuestXR where these ideas based on AI are central to the research.
Bio-sketch: Mel Slater is a Distinguished Investigator at the University of Barcelona in the Institute of Neurosciences, and co-Director of the Event Lab (Experimental Virtual Environments for Neuroscience and Technology). He was previously Professor of Virtual Environments at University College London in the Department of Computer Science. He has been involved in research in virtual reality since the early 1990s, and has been first supervisor of 40 PhDs in graphics and virtual reality since 1989. He held a European Research Council Advanced Grant TRAVERSE 2009-2015 and has now a second Advanced Grant MoTIVE 2018-2023. He is a Research Award Winner of the Alexander von Humboldt Foundation in 2021, and was elected to the IEEE VGTC Virtual Reality Academy in 2022. He is Field Editor of Frontiers in Virtual Reality, and Chief Editor of the Human Behaviour in Virtual Reality section. His publications can be seen on http://publicationslist.org/melslater.
Intelligent XR: Fusing Extended Reality and Artificial Intelligence
Frank Steinicke, Professor for Human-Computer Interaction at the Department of Informatics at the Universität Hamburg
Abstract: The fusion of extended reality (XR) and artificial intelligence (AI) will revolutionize human-computer interaction. XR/AI technologies and methods will enable scenarios with seamless transitions, interactions and transformations between real and virtual objects along the reality-virtuality continuum indistinguishable from corresponding real-world interactions. Yet, today’s immersive technology is still decades away from the ultimate display. However, imperfections of the human perceptual, cognitive and motor system can be exploited to bend reality in such a way that compelling immersive experiences can be achieved. In this talk, we will review some mixed reality illusions, which bring us closer to the ultimate blended reality.
Bio-sketch: Frank Steinicke is professor for Human-Computer Interaction at the Department of Informatics at the Universität Hamburg. His research is driven by understanding the human perceptual, cognitive and motor abilities and limitations in order to reform the interaction as well as the experience in computer-mediated realities.
He studied Mathematics with a minor in Computer Science at the University of Münster, from which he received his Ph.D. in 2006, and the Venia Legendi in 2010, both in Computer Science. He published about 300 peer-reviewed scientific publications and served as program chair for several XR and HCI-related conferences. Furthermore, he is chair of the steering committee of the ACM SUI Symposium, and member of the steering committee of GI SIG VR/AR. Furthermore, he is a member of the editorial boards of EEE Transactions on Visualization and Computer Graphics (TVCG) as well as Frontiers Section on Virtual Reality and Human Behaviour.