All posts by HEATHER TAYLOR

VR Application – Interaction Diagram

A rough diagram showing how one might interact with the program. Note that there’s a lot of overlap due to the fact that treatments intersect – both anxiety and depression can make use of the diary application, and as such, both will lead there. Similarly, calming music is spread across all main modules apart from ‘videos of family’, due to it being used in multiple areas.

VR Application: Tablet Connection

Any carer using CareReality may want to keep an eye on their patient, and this is why a tablet connection has been added, modelled after SolisVR’s. It allows them to control the general aspects of what the patient sees, using a similar options menu to said patient.

 An example of what this might look like.

The connection settings for a care device need to be locked down, so the Carer panel on the Vive’s display will show this in its Connection Settings – a way to register users, specific devices that are allowed (using their MAC addresses), the device’s password (for accessing it over the internet and for accessing the carer menu) and of course, the Bluetooth and WiFi passwords.

The tablet itself will have a fairly simplistic display in a layout similar to above, except displaying what the user’s seeing, what music they have playing, et cetera. The tablet will also possess a way to upload onto the VR if on an internet connection (i.e. not Bluetooth).

References:

Dreamstime. 2017. Senior Having Fun Using VR Headset. [Online]. [26 May 2017]. Available from: https://thumbs.dreamstime.com/z/senior-having-fun-using-vr-headset-gentleman-cane-isolated-white-background-67465962.jpg

123rf. 2017. Young Smiling Woman Using Tablet Computer Stock Photo. [Online]. [26 May 2017]. Available from: https://previews.123rf.com/images/elenathewise/elenathewise1012/elenathewise101200079/8436664-Young-smiling-black-woman-using-tablet-computer-Stock-Photo.jpg

Exemplar: Videos for Dementia patients

The Hebrew Home in Riverdale, Canada has been testing videos on patients – specifically, videos showing their family and friends doing things such as greeting them in the morning and talking them through morning routines. Dementia patients are easiest to handle when in a routine, and the videos seem to assist with this.

CareReality wants to emulate their success with its video technology, allowing the patients to be up even closer to their loved ones without the barrier of a tablet or monitor – instead letting them walk around and see them as if they were standing before them.

‘Relatives who take part are urged to say good morning, use memory-triggering personal anecdotes and remind the residents that attendants will be helping them get dressed and ready for the day.’ states the CTV Canada article on the topic, speaking on how the Hebrew Home are attempting to use the videos as a way to both jog memory and to set a ‘positive tone’ for the day.

However, the article does go on to state that ‘[the] program at the Hebrew Home is limited to residents in the early and moderate stages of dementia who are likely to recognize the people in the video and understand what they say.’ This means that whilst CareReality’s videos may help those in the early stages, only the more immersive experiences without faces or voices would help later on, as they wouldn’t add to the confusion of hearing voices and seeing faces that the user doesn’t recognise any longer.

Reference: Fitzgerald, J. 2015. Families make videos to calm dementia patients. [Online]. [26 May 2017]. Available from: http://www.ctvnews.ca/health/families-make-videos-to-calm-dementia-patients-1.2331748

VR Application – Main Menus and a Name

Every application needs to come with a name, and this one is no different. I chose CareReality as the name for the application in the end – it’s a reality that’s designed around care, after all.

When the user initially turns on the headset after the calibrations, they will be given a screen selection menu. This allows for them to select what their needs are, so that the program can automatically send them into an appropriate scene.

It’d look something like this – the labels are big enough to easily hit, can be selected with either the handheld controllers or with a mouse or gamepad, and are easily understood. They’re also set back in the screen so that the user isn’t overwhelmed with buttons in front of their vision.

The Dementia button sends the user into the settings for accessibility first, forcing them to set it up to their liking. From there, it sends them into a pleasant scenery experience that they can calmly explore to the sound of relaxing music, with the option of watching videos from family or friends that have been uploaded, as previously mentioned.

The Depression option sends the user straight to the diary, with calming music playing. If they’re in the midst of a depressive episode, they’re going to be able to vent about it immediately.

The Anxiety option sends the user to the space scene, allowing them to take in the stars and relax, to hopefully break any panic attacks with distraction and relaxation.

The Carer option, meanwhile, takes the user to another screen where they can upload videos, images, sounds and 3D files containing more scenery, and alter the settings device-wide. This section is able to have a passcode placed on it to protect the patients. This is also the section able to connect to a tablet via bluetooth, so that the Carer can use a mobile application to keep watch over what the patient is doing and adjust the experience if necessary.

VR Application – Helping Depression & Anxiety

One of the first forms of CBT involves letting the reader reflect. The application does this by giving the user a virtual diary: somewhere to put all of their thoughts down on paper and relax, depicted below.The diary can be written in using the pen as a normal pen, or the keyboard can be raised at the press of a button and the on-screen keyboard can be used, or, alternatively, the computer’s keyboard can be used. This is designed to be able to used with or without the headset on just in case the person is in a crisis and needs to vent to their diary, but doesn’t have access to their headset.

Another form of CBT is through meditation and reflection – here, this is represented by a space scene. The space will be able to play calming music through a playlist (the > arrow should be replaced with a playlist icon) that has been preuploaded, with some set calming tracks for different moods already uploaded. For example, some people may prefer piano, some may like violin, and some may just enjoy simple ambience of a forest, space or the beach. This allows some level of personalisation. There will also be an option to enable meditation guidance, to have text walk the user through the basic steps of meditation. An audio track featuring similar could also be included.Like the other VR spaces, the user will be able to see the handheld controllers in the Space scene. However, they will not be able to move around – just float in the space and relax to calming music and the stars around them. Providing a distraction is one of the best ways to calm down from a panic attack or anxiety attack, and the relaxing space should thus help anxiety significantly. As CBT and relaxation therapy also assist depression, it will also be of use there.

References & resources used:
Storywarren. 2017. Space. [Online]. [25 May 2017]. Available from: http://www.storywarren.com/wp-content/uploads/2016/09/space-1.jpg

Public Domain Pictures. 2017. Old Wood Texture. [Online]. [25 May 2017]. Available from: http://www.publicdomainpictures.net/pictures/70000/velka/old-wood-texture.jpg

Amazon. 2017. Image. [Online]. [25 May 2017]. Available from: https://images-na.ssl-images-amazon.com/images/I/51hki9kuTFL._SL1024_.jpg

Highend. 2017. Keyboard. [Online]. [25 May 2017]. Available from: https://www.highend.com/pub/support/controllers/documents/html/images/keyboard.png

What Problems do Depression & Anxiety Patients Face?

Anxiety and depression, whilst not the same as dementia, can still be treated using the VR application this portfolio is developing. They, however, focus on different things – and the treatment for them is similar.

Whilst anyone dealing with anxiety and/or depression should really see a psychologist and discuss the best ways forward for them personally, one of the recommended ways to help overcome both disorders is through Cognitive Behaviour Therapy (CBT), which ‘works to change your thoughts and behaviour by teaching you to think rationally about common difficulties, helping you to shift negative or unhelpful thought patterns and reactions to a more realistic, positive and problem-solving approach’ (BeyondBlue, c2016). Usually, CBT would be done over a long period with a therapist, but it might be possible to utilise some CBT techniques in the VR space by giving the patient a place to write out their troubles and provide them with a few steps to follow to analyse each day and move forward.

Another CBT technique is to ‘[teach] relaxation and breathing techniques, particularly muscle relaxation, to control anxiety and the physical symptoms of tension’ (BeyondBlue, c2016) – this, again, could be done through the app, with a neutral and relaxing space being provided and providing instruction on breathing techniques and relaxation through text overlays, akin to the subtitles on the dementia recordings.

Whilst anxiety can be assisted through exposure therapy, this is going to be intentionally avoided. Whilst the menus should be separated, with different sub-applications being selected depending on the user’s needs, there is still a chance that someone not ready for exposure therapy would activate it, or that the patient would have an unpredictable reaction such as a panic attack in response to the VR scenario.

References:

Beyondblue. c2016. Psychological Treatments for Depression. [Online]. [25 May 2017]. Available from: https://www.beyondblue.org.au/the-facts/depression/treatments-for-depression/psychological-treatments-for-depression

Beyondblue. c2016. Psychological Treatments for Anxiety. [Online]. [25 May 2017]. Available from: https://www.beyondblue.org.au/the-facts/anxiety/treatments-for-anxiety/psychological-treatments-for-anxiety

What problems do Alzheimer’s & Dementia patients face?

To develop an application properly, some of our own research must be done on what Alzheimer’s and Dementia patients need. My mother works in aged care, and my grandfather is diagnosed with a form of dementia, so I have derived some information from them alone. The SolisVR and VirtuCare solutions both focus on familiarity and jogging memories with relaxing scenarios, in addition to that personal experience.

The FightDementia website has a list of ways to help cope with someone who has dementia, with the first two being very relevant to developing a VR experience for them:

  • ‘A calm, unstressed environment in which the person with dementia follows a familiar routine can help to avoid some difficult behaviours
  • Try to keep the environment familiar. People with dementia can become upset if they find themselves in a strange situation or among a group of unfamiliar people where they feel confused and unable to cope’ (FightDementia, c2017)

The first of these can be addressed by gradually introducing the VR device to their routine and providing them with the calm and stress-free scenery walkthroughs, as well as using the voices and videos of friends and family to welcome them to the virtual world. The second is again addressed by the scenery walkthroughs – if kept generic, they can trigger memories and help the patient feel relaxed. A lot of more specific dementia advice, such as preventing hoarding, will not easily be helped using this technology – instead it can be used to assist them in remaining calm and to help them retain memory.

Fightdementia. 2017. BEHAVIOUR CHANGES. [Online]. [25 May 2017]. Available from: https://www.fightdementia.org.au/about-dementia-and-memory-loss/about-dementia/behaviour/behaviour-changes

VR Application – Using External Sources

Something other VR therapy softwares such as VirtuCare and SolisVR tend to have is a way of interacting with the external world. SolisVR uses tablets, and VirtuCare comes with flashcards. This can be extended by allowing video and audio input to be uploaded into the application.

This can come in the form of a live feed from the front-facing camera transmitting an image into a more understandable construct (for instance, green-screening family members into a former living room) or in the form of pre-uploaded videos displayed either in a similar manner or on a television screen. Audio can be overlayed onto scenery and played from a specific direction so that it sounds like the person is speaking to them as they walk around, played with subtitles, or simply shown alongside video and pictures.  

An example of how this might be presented is shown below.

The screen is curved to mimic that of VR – note that the person would not end up being inside a black box with a screen in front of them, but the scenery around them would be a 360 view. If possible, recording specific 360 degree areas and uploading them into the application would make therapy easier.

The top UI remains standard with the previous post, though the options menu will have additional options for this setting: the ability to change scene, upload new videos, switch recording type and the like. A way to manually subtitle videos would also be coded in.

Something more like this, with a more professional look and some transparency over whatever was behind it, is what the Options screen should end up looking like, then.

VR Application – Walk-through Scenes & Beginnings of UI

Using the example of VirtuCare’s VR scenery, having sets of walk-through scenes would be a good thing for treating dementia (by allowing patients to relax and connect the virtual world with past experiences) and anxiety (by allowing the patient to focus on something else and forget the real world for a time). 

Forest background source: http://wallpapersafari.com/w/pBoWvJ/

The Vive’s refresh rate is 90Hz, so this means ideally, we’ll need 60 – 90FPS to keep the user from feeling motion sick due to screen flickering or visual lag. As they’ll be completely immersed in the environment, any lag will start to cause motion sickness or dizziness – both would be highly unpleasant and would turn off any patients using the program for therapy.

In the above graphic, you can see the Vive controllers. This is to make it easy for the users to tell what they’re doing instead of confusing them with hands that don’t completely accurately mimic their own. As they may require stabilisation/sensitivity changes to adapt to their conditions, these allow the user to see exactly where their hands are moving instead of making them guess. It should also help them keep one foot firmly in reality.

You can see at the top that there are three indicators for options – if the user presses either certain keys or points the red indicator at those, then it will bring up the options menu, home screen or send them to the next scene. If they’re having issues with the usual controller and thus have to use the keyboard or a gamepad controller, the bar at the top allows them to navigate and is very unobtrusive.

Here’s an example of what the Options UI could look like. The light blue background would be whatever they were doing before pressing the Options button: for instance, it could show the previous forest scene. It should have a few extra options in addition to these, namely transparency of menus, language and the like – the image is simply to illustrate it. Each option is either a checkbox or can be navigated via drag or arrowkeys. The Vive’s controllers have a disc that can function both as a touchpad and as arrowkeys – these can be used for this purpose.

The options listed are mainly for accessibility. Sensitivity and stabilisation are two major ones: sensitivity is about modulating input, whilst stabilising will actively try to correct the balance. Both may be needed due to the older patients or any possible mobility issues. Another option that should be added would be ‘handedness’ – switching controls based on whether the user is left or right handed.

Volume control is obviously needed as a general option – some users need it louder, and some quieter. Those who are undergoing therapy in a public space may want the volume louder to drown it out, or may need it quieter at home to listen to people talk. Subtitles are also there for any who are hearing impaired. The font will also assist in accessibility, as certain fonts are easier to read: someone who’s dyslexic may have issues with Calibri and may need a special font formatted for dyslexia, for instance.

HTC Vive – The Base of a VR Experience

Every VR experience is going to need some sort of headset or specialised equipment to put it into practice, and the HTC Vive is a good choice for the application I wish to develop. It needs to be paired with a computer that meets the Vive’s minimum specifications, which are:

Processor: Intel™ Core™ i5-4590 or AMD FX™ 8350, equivalent or better
Graphics: NVIDIA GeForce™ GTX 1060 or AMD Radeon™ RX 480, equivalent or better. 
Memory: 4 GB RAM or more
Video output: 1x HDMI 1.4 port, or DisplayPort 1.2 or newer
USB: 1x USB 2.0 port or newer
Operating system: Windows™ 7 SP1, Windows™ 8.1 or later or Windows™ 10

…according to the HTC Vive website.  A computer with these specifications is likely going to be around $1000, but as computer technology improves, this cost will go down. The HTC Vive is around $800, at the moment, which is a cost that has to be accounted for.

But why choose the Vive over the Oculus or another VR headset? It’s a slightly cheaper pricepoint, and the controllers are, in my personal opinion, more intuitive to use.

Source: https://img2.cgtrader.com/items/696810/0dcf157847/large/htc-vive-handheld-controller-3d-model-max-obj-fbx.jpg

The button works as a touchpad as well as a mouse button, so it could be pretty effective for UI navigation. The actual motion of the controllers would be a good way to move around, taking inspiration from the SteamVR Demo’s way of moving.

The Vive also features front-facing cameras – these could be used to integrate live video of people in real life into applications, presenting them in a context more understandable to a dementia patient, or giving a way for depressive patients to remain grounded by giving them something else to focus on.

Reference: HTC. c2017. HTC Vive. [Online]. [25 May 2017]. Available from: https://www.vive.com/anz/product/