Dr Jacqueline Bailey
Associate Lecturer
School of Information and Physical Sciences (Computing and Information Technology)
- Email:jacqueline.d.bailey@newcastle.edu.au
- Phone: (02) 491 38780
Using avatars and VR/AR to shape a functional tomorrow
The use of digital user representations and immersive technologies extends beyond the gaming context. By exploring and applying them in serious gaming, Dr Jacqueline Bailey is improving learning outcomes and the lives of human users.
With a PhD in Information Technology (IT), Jacqueline’s core area of research is human-centred computing (HCC), focusing on the development and use of virtual humans, known as avatars, in serious gaming, simulation, and training.
Serious gaming refers to using video games or game-like interactive experiences for purposes beyond entertainment—and Jacqueline’s key industries of interest are healthcare, military, and education where serious gaming, avatars, and simulation training are common.
Her research also focuses on developing and using immersive technologies such as virtual or augmented/mixed reality for the same application.
“My overarching aim is to create novel and interactive learning experiences through role-play scenarios with ‘realistic-enough’ virtual humans,” explains Jacqueline.
“The global virtual reality (VR) market size was estimated at USD 59.96 billion in 2022 and is expected to grow at a compound annual growth rate (CAGR) of 27.5 per cent from 2023 to 2030. It’s therefore important to understand how best to maximise the use of these technologies for serious purposes.”
The benefits of gamified learning
Jacqueline was drawn to the field of HCC because of its boundless potential to use cutting-edge technology that offers real-world impacts and its ability to shape a better, more functional tomorrow.
“Avatars and immersive realities used in a serious context have the potential to revolutionise the way we train in industry. We can directly engage learners through role-play scenarios to develop skills vital for future career success.”
Developing immersive reality scenarios in the medical, military, and education fields is not only less risky than real-life training, but gamified learning has also been shown to help professionals achieve a deeper understanding.
Some examples include simulations for medical students to practice surgeries, for emergency responders to rehearse disaster scenarios, for soldiers to navigate combat and hostage situations, and for teachers to make abstract concepts tangible through interactive visuals.
Realistic enough virtual humans
However, Jacqueline says there are several ongoing challenges in the field of human-avatar interactions, including realism, uncanniness, expressions, and user acceptance.
This includes how real the avatar looks and how we can prevent a ‘too real’ looking avatar from negatively affecting the end-users.
Because of this, her research focuses on the quantitative analysis of human perceptions of avatars. In particular, the human startle reflex is used as a measure of affective processing.
“Previously, under Professor Karen Blackmore from the School of Design, Communication and IT and i3 Lab, I worked collaboratively with the Australian Defence College Wargaming and Simulation Centre to evaluate how emotional expressions in avatars’ of differing levels of realism and fidelity are perceived by human users.
Jacqueline’s 2016 research, Exploring Avatar Facial Fidelity and Emotional Expressions on Observer Perception of the Uncanny Valley, won her the best paper award at the Australasian Simulation Forum. She was also invited to speak at a ‘Best from around the world’ Session in the USA.
The findings from this research, particularly relating to the role of avatar gender, have helped inform the development of systems for avatar-augmented role play within the Australian Defence College.
More recently, in 2023, she was part of a grant funded by Meta (formerly Facebook) for $93K about highly realistic avatars in a virtual work setting.
Increasing education, jobs, and inclusivity
While her research career is still in its early stages, Jacqueline’s work has the potential to improve people’s lives by offering insights into human-centred technologies, paving the way for more informed decisions and better outcomes for humans using avatar-based and immersive technologies.
At the national level, her findings have been recognised through the publication of several conference papers presented locally, nationally, and internationally, offering a fresh perspective on avatar realism and uncanniness.
Beyond this, she says that the serious gaming industry has opened new job opportunities for game designers, educators, and tech professionals.
“Avatars can also be used in various ways, such as increasing empathy and perspective. Users can step into the shoes of someone from a different background, culture, or even historical period, fostering empathy and cultural understanding.”
“In addition, creating diverse avatars enables representation of various ethnicities, genders, and orientations, fostering inclusivity in the digital realm.”
From avatars to smartphone interfaces
In addition to her work on avatars and immersive realities, Jacqueline has recently branched out as an early career researcher (ECR) to undertake a study on the use of smartphones by older people.
For this, she has put together a team of researchers from all three University of Newcastle colleges to gain insights into the design development of the Older Persons Mode (OPM) — a new, more useable interface for this demographic.
As well as drawing on perspectives from IT, she’s also working with the Creative Industries to help with user experience and design guidelines and the School of Nursing and Midwifery to ensure they consider the physical and cognitive needs of older people who may struggle to use digital technology.
“These insights have been extremely useful as we work towards a solution that works with the older populations in a way that makes them feel safe and confident when using these devices,” she shares.
The work aims to help non-digital natives (60+ years old) feel more connected with the wider world via their phones. This further benefits older people, their families and carers with an easier and more trusted system that allows them to be less dependent on their immediate circle.
Considerations and endless possibilities
Through her work, Jacqueline aims to design solutions with the end-users, not against them. “No technology solution should ever be developed without the discussion and involvement of its target audience”, she shares, “to do so, wastes the potential the good a tech solution can have”.
Things like cost, use, and ethical and inclusive design principles are also always front of mind. Plus, she’s very aware of the need to balance novelty with usefulness. This is where her partnerships with industry prove invaluable.
Ultimately, her goal is to positively impact the lives of human users who engage with human-computer interaction (HCI) and HCC products.
What keeps her motivated in her work are the possibilities. She shares that there are so many applications in the world of avatars and immersive realities, and she’s excited to continue to find, build, and test tech solutions.
To this end, she has formed an international collaboration team called Interactive Intelligent Virtual Agents (I2VA), comprising ECRs in the human-avatar/agent field from colleges in Australia, Belgium, Japan, and Portugal.
She’s also currently in the early stages of developing a NUGamesTech team (similar to NUBots) for virtual reality competitions. This student-led team will create VR solutions for national and international competitions and run as a part of Professor Karen Blackmore’s i3Lab.
Using avatars and VR/AR to shape a functional tomorrow
Dr Jacqueline Bailey’s core area of research is human-centred computing (HCC), focusing on the development and use of virtual humans, known as avatars, in serious gaming, simulation, and training.
Career Summary
Biography
My research focuses on the quantitative analysis of human perceptions of avatars, and in particular, the use of the human startle reflex as a measure of affective processing. In my previous research, I have worked collaboratively with the Australian Defence College Simulation Centre to evaluate how emotional expressions in avatars’ of differing levels of realism and fidelity are perceived by human users.
The findings from this research, particularly relating to the role of avatar gender, have helped inform the development of systems for avatar augmented role play within the Australian Defence College. My current work extends this research to consider the underlying cognitive gender differences that can affect an end users’ perception of avatar realism and uncanniness.
Qualifications
- Doctor of Philosophy in Information Technology, University of Newcastle
- Bachelor of Information Technology (Honours), University of Newcastle
Keywords
- affective computing
- avatar
- human-avatar interaction
- realism
- uncanny valley
Fields of Research
Code | Description | Percentage |
---|---|---|
460702 | Computer graphics | 20 |
460806 | Human-computer interaction | 50 |
460802 | Affective computing | 30 |
Professional Experience
UON Appointment
Title | Organisation / Department |
---|---|
Associate Lecturer | University of Newcastle School of Information and Physical Sciences Australia |
Academic appointment
Dates | Title | Organisation / Department |
---|---|---|
1/1/2019 - 31/12/2021 | Casual Academic | The University of Newcastle Engineering, Science and Environment Australia |
Professional appointment
Dates | Title | Organisation / Department |
---|---|---|
1/1/2018 - 31/12/2021 | HunterWiSE Mentor | HunterWise HunterWiSE Australia |
25/2/2016 - 31/12/2019 | Casual Project Assistant | The University of Newcastle Strategy, Planning, & Performance Australia |
Awards
Award
Year | Award |
---|---|
2016 |
Best Paper Award Australasian Simulation Congress |
Recognition
Year | Award |
---|---|
2012 |
Faculty Commendation List 2012 - Faculty of Science and Information Technology University of Newcastle |
Teaching
Code | Course | Role | Duration |
---|---|---|---|
INFT3050 |
Web Programming College of Engineering, Science and Environment, University of Newcastle |
Course Coordinator | 21/2/2022 - 29/10/2022 |
Publications
For publications that are currently unpublished or in-press, details are shown in italics.
Chapter (1 outputs)
Year | Citation | Altmetrics | Link | |||||
---|---|---|---|---|---|---|---|---|
2023 |
Bailey JD, Blackmore KL, King R, 'Insights from the Uncanny Valley: Gender(Sex) Differences in Avatar Realism and Uncanniness Perceptions', Computer-Human Interaction Research and Applications, Springer, Cham, Switzerland 51-70 (2023) [B1]
|
Nova |
Journal article (2 outputs)
Year | Citation | Altmetrics | Link | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
2024 |
Blackmore KL, Smith SP, Bailey JD, Krynski B, 'Integrating Biofeedback and Artificial Intelligence into eXtended Reality Training Scenarios: A Systematic Literature Review', Simulation & Gaming,
|
||||||||||
2022 |
Bailey JD, Blackmore KL, 'A Novel Method of Exploring the Uncanny Valley in Avatar Gender(Sex) and Realism Using Electromyography', BIG DATA AND COGNITIVE COMPUTING, 6 (2022) [C1]
|
Nova |
Conference (9 outputs)
Year | Citation | Altmetrics | Link | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
2023 |
Deffrennes A, Vincent L, Pivette M, El Haddad K, Bailey JD, Perusquia-Hernandez M, et al., 'The Limitations of Current Similarity-Based Objective Metrics in the Context of Human-Agent Interaction Applications', ICMI 23 Companion Companion Publication of the 25th International Conference on Multimodal Interaction, Paris, France (2023) [E1]
|
Nova | |||||||||
2022 |
Bailey JD, Blackmore KL, King R, 'Observing the Uncanny Valley: Gender Differences in Perceptions of Avatar Uncanniness', Proceedings of the 6th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2022), Valletta, Malta (2022) [E1]
|
Nova | |||||||||
2021 |
Brown A, Bailey J, Blackmore K, Boogaard S, Nesbitt K, 'Gender Differences when Adopting Avatars for Educational Games', ACSW '21: 2021 Australasian Computer Science Week Multiconference, Virtual (2021) [E1]
|
Nova | |||||||||
2018 |
Bailey J, Blackmore K, Robinson G, 'Exploring avatar facial fidelity and emotional expressions on observer perception of the uncanny valley', Intersections in Simulation and Gaming. Lecture Notes in Computer Science, Melbourne, Australia (2018) [E1]
|
Nova | |||||||||
2017 |
Bailey JD, Blackmore KL, 'Gender and the Perception of Emotions in Avatars', ACSW'17 Proceedings of the Australasian Computer Science Week Multiconference, Geelong, Australia (2017) [E1]
|
Nova | |||||||||
2017 |
Bailey J, Sutton K, Blackmore K, 'Gender Differences in Spatial Ability and Affective Processing in Avatar Perception', Simulation Australasia, Sydney, Australia (2017) [E1]
|
Nova | |||||||||
2017 |
Bailey J, 'Avatar and Participant Gender Differences in the Perception of Uncanniness of Virtual Humans', Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), San Antonio, Texas (2017) [E1]
|
Nova | |||||||||
2016 |
Bailey JD, Blackmore KL, Robinson G, 'Exploring avatar facial fidelity and emotional expressions on observer perception of the Uncanny Valley', Congress Proceedings: 2016 Australasian Simulation Congress, Melbourne, Victoria (2016) [E1]
|
Nova | |||||||||
Show 6 more conferences |
Report (1 outputs)
Year | Citation | Altmetrics | Link | ||
---|---|---|---|---|---|
2022 |
Blackmore K, Bailey JD, Smith S, 'Artificial Intelligence Integration for XR Simulation/Training and Learning', Real Response Pty Ltd (2022)
|
Grants and Funding
Summary
Number of grants | 3 |
---|---|
Total funding | $101,686 |
Click on a grant title below to expand the full details for that specific grant.
20221 grants / $97,036
Accounting for Differences when Visualising Uncertainty of Avatar Emotions$97,036
Funding body: Meta
Funding body | Meta |
---|---|
Project Team | Professor Karen Blackmore, Doctor Jacqueline Bailey, Associate Professor Shamus Smith |
Scheme | Controls That Matter and Considering Everyone |
Role | Investigator |
Funding Start | 2022 |
Funding Finish | 2022 |
GNo | G2200325 |
Type Of Funding | C3600 - International Philanthropy |
Category | 3600 |
UON | Y |
20171 grants / $650
CORE (Australia) Student Travel Grant$650
Funding body: CORE
Funding body | CORE |
---|---|
Scheme | CORE (Australia) Student Travel Grant |
Role | Lead |
Funding Start | 2017 |
Funding Finish | 2017 |
GNo | |
Type Of Funding | External |
Category | EXTE |
UON | N |
20161 grants / $4,000
Best Paper Award Grant $4,000
Funding body: Simulation Australasian
Funding body | Simulation Australasian |
---|---|
Scheme | Best Paper Award Grant |
Role | Lead |
Funding Start | 2016 |
Funding Finish | 2016 |
GNo | |
Type Of Funding | External |
Category | EXTE |
UON | N |
Research Supervision
Number of supervisions
Current Supervision
Commenced | Level of Study | Research Title | Program | Supervisor Type |
---|---|---|---|---|
2020 | Masters | Data Modelling and Visualisation Techniques to Enhance Student and Staff Understanding of Course Learning Outcomes Across a Program of Study | M Philosophy (InformationTech), College of Engineering, Science and Environment, The University of Newcastle | Co-Supervisor |
Dr Jacqueline Bailey
Position
Associate Lecturer
School of Information and Physical Sciences
College of Engineering, Science and Environment
Focus area
Computing and Information Technology
Contact Details
jacqueline.d.bailey@newcastle.edu.au | |
Phone | (02) 491 38780 |
Link | Research Networks |
Office
Room | ES221 |
---|---|
Building | Engineering Science (ES) Building |
Location | Callaghan & Ourimbah University Drive Callaghan, NSW 2308 Australia |