AVA

Augmented
Auscultation Simulation

AVA

Augmented
Auscultation
Simulation

UX Design | Interaction Design
Master thesis 2020
In collaboration with Laerdal Medical

AVA

Augmented Auscultation Simulation

Interaction Design
Master thesis 2020
In collaboration with Laerdal Medical

Making auscultation education more
accessible and adaptable.

Together with Laerdal Medical I’ve been looking into how auscultation education can be made more accessible to students and allow for more self directed and repeatable training. 

With my project titled “augmented auscultation simulation”, I wanted to create an example as to how designing for a situation like this can be approached, leveraging modern technology and closely investigating users’ real needs and pain points.

“AVA” is a study of how dematerialization and the introduction of augmented reality can fill gaps in the current learning process, increase independence of students, reduce costs for educational institutions and create a seamless user experience.

Together with Laerdal Medical I’ve been looking into how auscultation education can be made more accessible to students and allow for more self directed and repeatable training. 

With my project titled “augmented auscultation simulation”, I wanted to create an example as to how designing for a situation like this can be approached, leveraging modern technology and closely investigating users’ real needs and pain points.

“AVA” is a study of how dematerialization and the introduction of augmented reality can fill gaps in the current learning process, increase independence of students, reduce costs for educational institutions and create a seamless user experience.

Making auscultation education more
accessible and adaptable.

Home

Home

Personalized training.

Students can see their progress and monthly goals, set by themselves or educators, friends can be added as well.

Successfully completing exercises is rewarded with experience points. Completing and more complex tasks leads to a higher reward.

In case of the user being a teacher, the students can be added as friends, giving an overview over their progress and allowing to issue mandatory exercises or schedule group trainings.

Personalized training.

Students can see their progress and monthly goals, set by themselves or educators, friends can be added as well.

Successfully completing exercises is rewarded with experience points. Completing and more complex tasks leads to a higher reward.

In case of the user being a teacher, the students can be added as friends, giving an overview over their progress and allowing to issue mandatory exercises or schedule group trainings.

Free Practice

Free Practice

Custom, on-demand experiences.

Replacing the physical simulation dummy with a digital one through augmented reality, allows the user to alter parameters that were previously impossible to change, such as age, sex or bodytype.
Their impact on sound characteristics is now immediately tangible through realtime feedback.

The digital simulator also allows for a multitude of optional visual, audible and haptic feedback, that can support the user making the right diagnose, if needed.

Custom, on-demand experiences.

Replacing the physical simulation dummy with a digital one through augmented reality, allows the user to alter parameters that were previously impossible to change, such as age, sex or bodytype.
Their impact on sound characteristics is now immediately tangible through realtime feedback.

The digital simulator also allows for a multitude of optional visual, audible and haptic feedback, that can support the user making the right diagnose, if needed.

In free practice mode, the user has complete control over the manikin and the possibility for exploration.

The manikin can be placed anywhere and anytime in the users environment.
The central ring is the “stethoscope”. Pointing it an area on the upper body, plays back an area specific sound.

   Settings

Toggle audibles (heart sounds – lung sounds – environment noise)
Toggle visuals  (landmarks – organs – ECG – Soundwaves)
Toggle haptics (pulse)

   Position

Reposition
Rotate
Change posture (standing – sitting – lying)

   Patient information

Change heart and lung condition
Change age, sex, bodytype
Add medical background

In free practice mode, the user has complete control over the manikin and the possibility for exploration.

The manikin can be placed anywhere and anytime in the users environment.
The central ring is the “stethoscope”. Pointing it an area on the upper body, plays back an area specific sound.

   Settings

Toggle audibles (heart sounds – lung sounds – environment noise)
Toggle visuals  (landmarks – organs – ECG – Soundwaves)
Toggle haptics (pulse)

   Position

Reposition
Rotate
Change posture (standing – sitting – lying)

   Patient information

Change heart and lung condition
Change age, sex, bodytype
Add medical background

Exercises

Exercises

Scalable practice.

AVA offers the possibility to scale exercises according to the users’ needs.

Individuals can practice through exploration or a set of premade exercises, such as multiple choice or match-making

Immersive scenarios.

Peer to peer training is also possible through role playing partner exercises, which increase immersion and allow for more detailed patient cases.

 

Partner training: A friend takes the role of a patient. He can also set the conditions and which tools can be used for examination. For increased realism, the patient can also give extended feedback for example through coughing, or expressing discomfort which can be helpful for making a diagnose.

Remote teaching.

In group settings, such as remote teaching, spectators can join an examination through partner sessions with live feedback from educators or compete against each other in a more casual setting.

Library

Library

Comprehensive sound library.

The expandable sound library contains detailed information not only about specific sounds, but also how certain factors impact sound characteristics. 

Interesting cases from personal exercises can be saved here as well.

Comprehensive sound library.

The expandable sound library contains detailed information not only about specific sounds, but also how certain factors impact sound characteristics. 

Interesting cases from personal exercises can be saved here as well.

Process — Insights

01_Research

User interviews – pain points

My research led me to three different countries where I had the chance to talk to a total of eleven educators and researchers from three different Universities (UiS Stavanger, Umeå University, Gävle University) in paramedic, nursing and medical schools and explore their training facilities.
I also had the chance to try some of the simulators myself.

Here are some of the most important learnings summed up:

Students always need a supervisor to use the simulators
Students rely on availability of patients and teachers for training
I feel bad leaving 1 mio. SEK in equipment unattended
Simulators can only produce “textbook” sounds
Repeatability isn‘t what it should be
It is impossible to reproduce the immersion of a real scenario

01_Research

User interviews – pain points

My research led me to three different countries where I had the chance to talk to a total of eleven educators and researchers from three different Universities (UiS Stavanger, Umeå University, Gävle University) in paramedic, nursing and medical schools and explore their training facilities.
I also had the chance to try some of the simulators myself.

Here are some of the most important learnings summed up:

Students always need a supervisor to use the simulators
Students rely on availability of patients and teachers for training
I feel bad leaving 1 mio. SEK in equipment unattended
Simulators can only produce “textbook” sounds
Repeatability isn‘t what it should be
It is impossible to reproduce the immersion of a real scenario
Curriculum analysis workshop

In this workshop I analyzed, together with two teachers from the Umeå University, the curriculum of a medicine student in relevant semesters, to find a context for a new solution and where it could make the most impact.
The result showed a suboptimal spread of the workload, which was mostly due to restrictive equipment and lack of availability of patients and staff (see pain-points from previous interviews). The opportunity I saw here was to create a solution that allows for high quality simulation training, without being dependent on supervisors, location or patients.

Spread the learning hours out more, not just once for many hours in one semester!

– Medicine Student, Umeå University

Technology in education

I also had the chance to talk with an associate professor and researcher for emerging tech and teaching practices at the Umeå University about the future of education and the role of technological advancements. These are the main take aways from these interviews:

– Haptics and sound are important for learning
– Scalability with group, plug-in into bigger system
– Collaborative learning is important for the learning progress
– Big classroom teaching is the worst case scenario
– AR/VR are making their way into schools to explain phenomena that are hard to understand
– Quality feedback on demand (not  just reward) + added value => important for learning process
– Coordination training = might take some time to get used to  visuals + haptic + sounds

Making connections between visuals, haptics and audio already in the application instead of having to draw the connections yourself in your mind will free up headspace for other things”
Effective learning

As a central part of my project is the learning process, I also tried to understand how to achieve effective learning.
According to research, the best indicators for effective learning can be measured in performance, knowledge, motivation and satisfaction.
From there I tried to find “enablers”, that allow the learner to achieve these 4 components.
With that and the findings from the field research, I could then set those in the context of auscultation training, resulting in a whole bunch of factors, that can be responsible for effective learning in my case.

Simulation training

During my research I learned that simulation based training has seen great success and is proven to be more efficient than traditional training.
Therefore, the more time students are engaged in learning simulation-based examination the better the learning outcome will be.

Simulation training provides these major benefits:

Increasing availability

In academic centers, trainees must often compete to examine an ever-dwindling number of suitable patients. Having an “artificial patient” leads to increased accessibility and saves the patients’ and teachers’ time.

Repeatable training

To become competent in cardiac physical examination, a clinician must examine on a repetitive basis. Simulation training enables controllable environments and therefore allows to recreate certain scenarios as often as needed.

Improved skilltransfer

Simulation based medical education (SBME) could improve the transfer of clinical skills from the teaching setting to real patients.

Effective skill acquisition

Direct contact (hands-on practice) with the simulator appears to increase the effectiveness of cardiac skills acquisition.

Learning methods

Alternatively to the traditional classroom teaching, I also investigated learning methods, that allow for a better integration into the students’ learning routines, which could then be combined with a simulation tool.
Considering the three models of e-learning, distance learning and blended learning, I thought to see the biggest potential to fit the needs of this project would be blended learning. Blended learning allows for a variety of learning methods, some of which are the following:

– Flipped classroom
– Integration of learning with entertainment / gamification
– Adaptive learning
– Low dose, high frequency learning
– Rapid feedback loops (e.g. duolingo)
– Just-in-time refreshers
– Scalable solutions

Blended learning methods can enable features and uses of high-fidelity medical simulations that lead to effective learning.
Learning methods

Alternatively to the traditional classroom teaching, I also investigated learning methods, that allow for a better integration into the students’ learning routines, which could then be combined with a simulation tool.
Considering the three models of e-learning, distance learning and blended learning, I thought to see the biggest potential to fit the needs of this project would be blended learning. Blended learning allows for a variety of learning methods, some of which are the following:

– Flipped classroom
– Integration of learning with entertainment / gamification
– Adaptive learning
– Low dose, high frequency learning
– Rapid feedback loops (e.g. duolingo)
– Just-in-time refreshers
– Scalable solutions

Blended learning methods can enable features and uses of high-fidelity medical simulations that lead to effective learning.
Opportunities

Considering all the findings from the research, I concluded that there are design opportunities especially in the simulation training for the advanced level of training auscultation. Simulation training offers a great foundation to embed blended learning methods into the learners’ routines. That in mind I could formulate some central “how might we” questions, adressing some of the major pain points found during the research.

How might we make auscultation training more …

… accessible
… scalable
… immersive
… reassuring
… repeatable
… adaptive

02_Ideation

Minimum product requirements

In order to find out what the most needed functions for a simulation trainer are, I sent out a survey to several professionals containing a list of features (with the opportunity to add more) and let them rate these according what they think was necessary on a scale from one to five, where a score of one represents an unnecessary feature and five would be a must have.

I divided the questions into three categories: Cardiac, pulmonary and bowel simulation. To make judgment from the answers, I visualized the results by giving each answer one point (visualized by a square on the following pages). From the distribution of those points I could quickly draw conclusions for the importance of each feature.

These scores should however not be seen as an absolute value but rather serve as an orientation for decision making later.

Creative workshops
01 “How Might We…?” workshop

The idea for this workshop was to do a “trial” run to prepare for an upcoming larger workshop. The main goal was to generate some broad concept ideas.
The main takeaway was that the how might we (HMW) questions were too generic. I took this as an inspiration to formulate more specific questions with a clear intention for the next session.
Participants were seven of my classmates who had 20 minutes to ideate around three HMW questions.

02 Scenario workshop

This workshop was held with 12 participants (UID students) that were split into groups of four.
Three scenarios with three sub-scenarios were presented to them, so that in total nine scenarios could be ideated around in the end.
The time-frame was ~20 minutes per scenario.

Goals

– Generate quantity
– Find low cost, accessible solutions
– Ideate on learning mechanics, integration into daily routine
– Ideate around quality of simulation

03 Feature cards workshop

From the results of the previous workshop I could generate so called feature cards, that I would take to professionals and use them as a foundation to discuss the different concepts. Each card contains a rough visualization of a concept and rating system. You can see an example here.
I had two participants for this 1,5h workshop, both teachers from the Umeå University.

Goals

– Generate quality
– Discuss / improve ideas
– Prioritize features according to usefulness

Concepts & functionality

Before making a decision on which route to choose, I revisited some of the most essential findings from my research.
According to these the final product should be:

– Cost effective
– Allowing independent training
– Flexible in terms of location
– Scaling according to user needs and group sizes
– Offering customizability in various ways
– Adapting to user needs
– Providing qualitative feedback
– Motivating to learn

These are some of the concepts that resulted from the previous ideation sessions.

01 “Inflate”

Inflatable manikin
Lightweight, inflatable manikin that can display projected information

App
To control the manikin, prepare patient cases and progress tracking

Stethoscope
Standard personal stethoscope

02 “Stetho-phone”

App
AR app tracks body and projects information and guidance. Preparing of patient cases and progress tracking

Headphones
Augmented audio, replaces analog stethoscope

03 “X-ray token”

App
AR app tracks body and token position

Token
Acts as wireless stethoscope, integrated display can show information and give „x-ray“ into body

Headphones
Augmented audio, replaces analog stethoscope

04 “Wearable”

Token
Acts as wireless stethoscope

Wearable
AR headset portrays additional information and guidance, tracks token position

05 “AR simulator”

App
AR app projects a virtual manikin in 3D space, allowing full costumization of every aspect. Preparing of patient cases and progress tracking

Headphones
Augmented audio, replaces analog stethoscope

03_Concept development

Prototyping & validation

Exploring functions in figma and creating conversation starters for review sessions with professionals.

 

I’ve also considered several forms of implementing augmentation, from enhancing real scenarios to creating a fully digital world.

This demo video was created to send out to educators and students for feedback and validation.

10/10 consulted professionals thought digitalizing the process makes sense and provides valuable benefits.
Visual direction
Smooth
Friendly
Fresh
Professional
Clean, but not clinical
Character

The AR manikin is a crucial component and its character design therefore very important. The goal was to create characters that avoid the “uncanny valley” situation while still retaining enough realism.

Pink dot = Location of desired character:

Realistic enough for simulation,
yet achievable in visualization
Character

The AR manikin is a crucial component and its character design therefore very important. The goal was to create characters that avoid the “uncanny valley” situation while still retaining enough realism.

Pink dot = Location of desired character:

Realistic enough for simulation,
yet achievable in visualization
Wireframes
Need more insights? Download the full report here.
Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google