Welcome – Page 16 – Nature does not hurry, yet everything is accomplished.

by Marie Kunzmann - July 18, 2021July 18, 2021

06 | Understanding the user part 1

Interview with a high school teacher

In my previous blog entry (“05 // Online interaction scenario: Experience Map”) I wrote about the protopersona Sophie which is based on a real person, her experiences mixed with my observation. In order to widen my perspective and not only examining the students/university view I am really happy to had the chance to interview Damaris about her online teaching experiences a german high school teacher.

The first question I asked was how she experienced the transition from presence to online lessons. She described that teaching in the first lockdown (around march 2020) was really hard because of the missing software and also missing equipment of teachers and students. Teachers mostly had to hand out printed homework packages and wait for them to be handed in later on. These experiences lead to a better preparation for the second lockdown (december 2020 – may 2021). The school then provided tablet for teachers and rental laptop for students without a device. They could held their lesson via video conferences in the software ‘Jitsi’ and used the platform ‘DiLer’ (= Digitale Lernplattform) for communication and data exchange. Teachers took their time to learn the software by themselves, then teach the students and let them practice the tool to handle it properly. The teachers also learned how to use a visualiser which is a document camera for digitally recording printed media.

_{The learning platform ‘DiLer’ – Developed by a german highschool}

Next I asked her if she had to change something of her presence lesson curriculum for the online teaching. Damaris told me, that she had to digitize most of her teaching material in the first place to make online teaching possible. This included not only scanning printed material but also rework existing material to make it suitable. Furthermore she had to reorganise some exercises because group or partner work were difficult to implement in the first lockdown (without online support) but also in the online environment. The communication with students and parents changed from mainly verbal to mostly written communication which took a lot of time.

I also asked Damaris, if online teaching made something easier or harder for her as a teacher. She answered that sadly there was nothing that online lessons made teaching for her easy. She explained that it was really hard for her to get the control back she needs as an educator. She didn’t know if the student actually work and couldn’t properly evaluate their performances – especially things like oral grades. Her own workload was extended due to the fact that she had to check each student’s (home)work instead of just discussing results orally in the classroom.

In regard of the class Damaris observed that the online lesson environment worked well for students who already have been very structured and good in the presence lessons. The ones who need more attention from teachers in presence lessons were mostly even more behind in the online teaching environment because they couldn’t handle their self-management. One really interesting fact was that one of the students who was a rather quiet person in presence class started to become more outgoing in the online lectures. Maybe the online environment gave this person kind of a ‘safe place’ to express herself.

The fifth question I asked was if Damaris could imagine a continuation of online lessons or parts of it in the corona-free future. She answered that online teaching/communication could have some advantages in the future. One examples she mentioned was the advantage for getting more easily in touch with parents and having the opportunity to provide parents consultations late in the evening. She also observed that the students were able to develop a lot of new media literacy through the online lessons, which should definitely be encouraged in the future.

I was really surprised when she told me that the teachers only had a software introduction of online teaching but no coaching for the didactic part of it like for example how to compensate/replace group work. The digitization of high school lessons during the pandemic and also in general times seems to me a bit neglected by the government and lies in the responsibility of the educators.

By researching about the software ‘DiLer’ I came across the article “5 Fragen – 5 Antworten” with Mirko Sigloch on the platform ‘wissensschule.de’. In this article the authors explains the approach of his school to cope with digitizing of/and education now and in the future. He is sure that the current way of teaching will be insufficient to prepare the students for complex problems in the future. By developing the platform DiLer he and his colleagues wanted to create an open source platform that combines good usability and flexibility for an ideal online school environment. After their launch and testing phase they recognized how many school have been in need for such a platform. They presented the software to the ministry of culture of the federal state Baden-Württemberg but they wanted to hold on to the old structures. In the course of the article, he finally gets very emotional about the current status of digitalisation in school that seems to be rather regressive. His call for a hybrid teaching structure makes sense from my point of view when reading, but I am sure that the advantages of the present teaching structure should not be neglected. This discussion definitely needs more research from my side and I don’t see myself in the responsibility to take a position in it (but I am still curious about the different voices about this boundary topic). I already had a quick look into the theses of Lisa Rosa which I want to examine in another blog post.

_Links

^{https://www.digitale-lernumgebung.de/}

_{https://www.wissensschule.de/5-fragen-5-antworten-schule_digital-mit-mirko-sigloch/}

_{https://shiftingschool.wordpress.com/about/}

by Johannes Lechner - July 16, 2021July 16, 2021

ML Sample Generator Project | Phase 2 pt3

Convolutional Networks

Convolutional networks include one or more convolutional layers. These layers are typically used for feature extraction. Stacking multiple on top of each other often can extract very detailed features. Depending on the input shape of the data, convolutional layers can be one- or multidimensional, but are usually 2D as they are mainly used for working with images. The feature extraction can be achieved by applying filters to the input data. The image below shows a very simple black and white (or pink & white) image with a size 3 filter that can detect vertical left-sided edges. The resulting image can then be shrinked down without losing as much data as reducing the original’s dimensions would.

2D convolution with filter size 3 detecting vertical left edges

In this project, all models containing convolutional layers are based off of WavGAN. For this cutting the samples down to a length of 16384 was necessary, as WavGAN only works with windows of this size. In detail, the two models consist of five convolutional layers, each followed by a leaky rectified linear unit activation function and one final dense layer afterwards. Both models were again trained for 700 epochs.

Convolutional Autoencoder

The convolutional autoencoder produces samples only in the general shape of a snare drum. There is an impact and a tail but like the small autoencoders, it is clicky. In contrast to the normal autoencoders, the whole sound is not noisy though but rather a ringing sound. The latent vector does change the sound but playing the sound to a third party would not result in them guessing that this should be a snare drum.

Ringy conv ae sample

GAN

The generative adversarial network worked much better than the autoencoder. While still being far from a snare drum sound, it produced a continuous latent space with samples resembling the shape of a snare drum. The sound itself however very closely resembles a bitcrushed version of the original samples. It would be interesting to develop this further as the current results suggest that there is just something wrong with the layers, but the network takes very long to train which might be due to the need of a custom implementation of the train function.

Bitcrushed sounding GAN sample

Variational Autoencoder

Variational autoencoders are a sub-type of autoencoders. Their big difference to a vanilla autoencoder is the encoder’s last layer, the sampling layer. With this, variational autoencoders always provide a continuous latent space, which is much better for generative models than just to sample from what has been provided. This is achieved by having the encoder output two different vectors instead of one: one for standard deviation and one for the mean. This provides a distribution rather than a single point, leading to the decoder learning that an area is responsible for a feature and not a single sample.

Training the variational autoencoder was especially troublesome as it required a custom class with it’s own train step function. The difficulty with this type of model is that the right mix between reconstruction loss and kl loss has to be found, otherwise the model produces unhelpful results. The currently trained models all have a ramp up time of 30,000 batches until full effect of the kl loss. This value gets multiplied by a different actor depending on the model. The trained versions are with a factor of 0.01 (A), 0.001(B), as well as 0.0001(C). Model A produces a snare drum like sound, but is very metallic. Additionally instead of having a continuous latent space, the sample does not change at all. Model B produces a much better sample but still does not include much changes. The main changes are the volume of the sample as well as it getting a little bit more clicky towards the edges of the y axis. Model C has much more different sounds, but the continuity is more or less not present. In some areas the sample seems to get slightly filtered over one third of the vector’s axis but then rapidly changes the sound multiple times over the next 10%. But still, out of the three variational autoencoders model C produced the best results.

VAE with 0.01 contribution (A) sample

VAE with 0.001 contribution (B) sample

VAE with 0.0001 contribution (C) sample

Next Steps

As I briefly mentioned before, this project will ultimately run on a web server which means the next steps will be deciding how to run this app. Since all of the project has been written in python so far Django would be a good solution. But since TensorFlow offers a JavaScript Library as well this is not the only possible way to go. You will find out more about this in the next semester.

by Johannes Lechner - July 16, 2021July 16, 2021

ML Sample Generator Project | Phase 2 pt2

Autoencoder Results

As mentioned in the post before I have trained nine autoencoders to (re)produce snare drum samples. For easier comparison I have visualized the results below. Each image shows the location of all ~7500 input samples.

Rectified Linear Unit

All three graphics portray how the samples are mostly close together but some are very far out. A continuous representation is with all three models not possible. Reducing the latent vector’s maximum on both axes definitely helps, but even then the resulting samples are not too pleasing to hear. The small network has clicks in the beginning and generates very silent but noisy tails after the initial impact. The medium network includes some quite okay samples but moving around in the latent space often produces similar but less pronounced issues as the small network. And the big network produces the best sounding samples but has no continuous changes.

Clicky small relu sample

Noisy medium relu sample

Quite good big relu sample

Hyperbolic Tangent

These three networks each produce different patterns with a cluster at (0|0). The similarities between the medium and the big network lead me to believe that there is a smooth transition between random noise, to forming small clusters, to turning 45° clockwise and refining the clusters when increasing the number of trainable parameters. Just like the relu version, the reproduced audio samples of the small network contain clicks. The samples are however much better. The medium sized network is the best one out of all the trained models. It produces mostly good samples and has a continuous latent space. One issue is however that there are still some clicky areas in the latent space. The big network is the second best overall as it mostly lacks a continuous latent space as well. The produced audio samples are however very pleasing to hear and resemble the originals quite well.

Clicky small tanh sample

Close-to-original medium tanh sample

Close-to-original big tanh sample

Sigmoid

This group shows a clear tendency to cluster up the more trainable parameters exist. While in the above two groups the medium and the big network produced better results, in this case the small network is by far the best. The big network delivers primarily noisy audio samples and the medium network very noisy ones as well but they are better identifiable as snare drum sounds. The small network has by far the closest sounds to the originals but produces clicks at the beginning as well.

Clicky small sigmoid sample

Noisy medium sigmoid sample

Super noisy big sigmoid sample

In the third part of this series we will take a closer look at the other models.

by Johannes Lechner - July 16, 2021July 16, 2021

ML Sample Generator Project | Phase 2 pt1

A few months ago I already explained a little bit about machine learning. This was because I started working on a project involving machine learning. Here’s a quick refresh on what I want to do and why:

Electronic music production often requires gathering audio samples from different libraries, which, depending on the library and on the platform, can be quite costly as well as time consuming. The core idea of this project was to create a simple application with as few as possible parameters, that will generate a drum sample for the end user via unsupervised machine learning. The interface’s editable parameters enable the user to control the sound of the generated sample and a drag-and-drop space could map a dragged sample’s properties to the parameters. To simplify interaction with the program as much as possible, the dataset should only be learned once and not by the end user. Thus, the application would work with the models rather than the whole algorithm. This would be a benefit as the end result should be a web application where this project is run. Taking a closer look at the machine learning process, the idea was to train the network in the experimentation phase with snare drum samples from the library noiiz. With as many different networks as possible, this would then create a decently sized batch of models from which the best one could be selected for phase 3.

So far I have worked with four different models in different variations to gather some knowledge on what works and what does not. To evaluate them I created a custom GUI.

The GUI

Producing a GUI for testing purposes was pretty simple and straight-forward. Implementing a Loop Play option required the use of threads, which was a little bit of a challenge but working on the Interface was possible without any major problems thanks to the library PySimpleGUI. The application worked mostly bug free and enabled extensive testing of models and also already saving some great samples. However, as it can be seen below, this GUI is only usable for testing purposes and does not meet the specifications developed in the first phase of this project. For the final product a much simpler app should exist and instead of being standalone it should run on a web server.

Autoencoders

An autoencoder is an unsupervised learning method where input data is encoded into a latent vector (therefore the name autoencoder). To get from the input to the latent vector multiple dense layers reduce the dimensionality of the data, creating a bottleneck layer and forcing the encoder to get rid of less important information. This results in data loss but also in a much smaller representation of input data. The latent vector can then be decoded back to produce a similar data sample to the original. While training an autoencoder, the weights and biases of individual neurons are modified to reduce data loss as much as possible.

In this project autoencoders seemed to be a valuable tool as audio samples, even though as short as only 2 seconds, can add up to a huge size. Training with an autoencoder would reduce this information down to only a latent vector with a few dimensions and the trained model itself, which seems perfect for a web application. The past semester resulted in nine different autoencoders, each containing dense layers only. All autoencoders differ from each other by either the amounts of trainable parameters, or the activation functions, or both. The chosen activation functions are rectified linear unit, hyperbolic tangent and sigmoid. These are used in all of the layers of the encoder as well as all layers of the decoder except for the last one to get back to an audio sample (where individual data points are positive and negative).

Additionally, the autoencoders’ size (as in the amount of trainable parameters) is one of the following three:

Two dense layers with units 9 and 2 (encoder) or 9 and sample length (decoder) with trainable parameters
Three dense layers with units 96, 24 and 2 (encoder) or 24, 96 and sample length (decoder) with trainable parameters
Four dense layers with units 384, 96, 24 and 2 (encoder) or 24, 96, 384 and sample length (decoder) with trainable parameters

Combining these two attributes results in nine unique models, better understandable as a 3×3 matrix as follows:

	Small (2 layers)	Medium (3 layers)	Big (4 layers)
Rectified linear unit	Ae small relu	Ae med relu	Ae big relu
Hyperbolic tangent	Ae small tanh	Ae med tanh	Ae big tanh
Sigmoid	Ae small sig	Ae med sig	Ae big sig

All nine of the autoencoders above have been trained on the same dataset for 700 epochs. We will take a closer look on the results in the next post.

by Valentina Riedisser - July 15, 2021August 5, 2021

Pt. 1: Wer sind meine User?

Im Moment ist mein Instagram ein Haufen aus verschiedenen Design Disziplinen und verschiedenen Styles. Anfangs dachte ich, dass meine Followerschaft sich an der Vielfalt von verschiedenen Design Methoden und Designdisziplinen erfreut, jedoch merke ich, dass das Wachsen meiner Followerzahlen nur schleichend voran geht und ich wohl etwas an meinem Inhalt ändern muss.

Obwohl ich am Anfang meiner Instagram-Reise User Research gemacht habe (ein bisschen). Muss ich wohl etwas tiefer graben, um die wirklichen Bedürfnisse meiner Follower zu entdecken. Dazu gehört nicht nur, was sie mögen oder was in ihrer Freizeit gerne machen, sondern auch was ihre Pain Points sind und wie ich ihnen mit meinem Post helfen kann.

Dieter Rams hat mal gesagt „You cannot understand good design if you do not understand people; design is made for people.”

Das selbst ich, als Design-Studentin mit Interesse für UX, die Wichtigkeit von guter User Research unterschätzt habe, zeigt auf, wie wichtig diese wirklich ist. Um also meine Follower und noch-nicht-Follower auf Instagram verstehen zu können, recherchiere ich zu aller erst nochmal welche Methoden man anwendet um user-zentriert planen und designen zu können.

Mental Models

Diese spiegeln die Gedanken(-Prozesse) und Assoziationen über ein gewissen Thema wieder. Diese beinhalten Erfahrungen, bestehendes Wissen und intuitive Wahrnehmungen und Ideen. Diese Erfahrungen und dieses Vorwissen beeinflussen wie eine Person denkt oder handelt. Deshalb können auch Rückschlüsse gezogen werden, wie Personen in gewissen Situationen auf zum Beispiel Probleme reagieren. Am Ende kann bei ‚Mental Models‘ auch eine Art Skript für die Vorgehensweise des Users entstehen. So weiß bzw. nimmt man an, welche Schritte der User als nächstes macht.

Personas

Personas sind Profile, die anhand Informationen von mehreren Individuen aus der Zielgruppe erstellt werden und in einer fiktiven Person widergespiegelt werden. Eine Persona gibt immer die Kernpunkte einer Zielgruppe und sozio-demografische Daten wider. Anhand eines Profilbildes und einer kurzen Einführung in den Alltag der Persona, erzählt eine Geschichte und lässt uns leichter in die Zielgruppe einführen.

Personenbezogenen Daten: Foto, Name
Sozio-demografische Daten: Alter, Geschlecht, Beruf, Beziehungsstatus, …
Psychografische Daten: Wünsche, Werte, Lebenstil, Hobbies
Technografische Daten: Geräte im Besitz des Users, User Verhalten, …
Geografische Daten: Stadt, Land, Kultur, …

Eine Zielgruppe besteht meist aus mehreren Untergruppen, die im Gesamten die Vielfältigkeit der Zielgruppe wiedergeben.

Sozio-demografische Daten

Lifestyle

Herkunft
Job
Gehalt
Alter
Geschlecht
Interessen
Werte

Verhalten

Aufgabe des Users
Kontext der Situation
User Lifestyle
Anforderungen & Wünsche
Vorteile & Benefits
Nutzung von Medien
Häufigkeit der Nutzung
Nutzungs-Muster
User Experience
Verbindung mit der Marke

Sinus Mileus

Wurde von dem Sinus Institut in Deutschland gegründet und dient dazu, via Lifestyles oder Soziale Milieus, User in Gruppen einzuteilen. Die Segmentierung kann von mehreren Kriterien abhängen.

User, what’s in your bag

Was jemand in seiner Tasche hat, sagt viel über eine Person aus. Deshalb kann es helfen, darüber nachzudenken, was der User in seiner Tasche mit sich trägt. Diese Dinge sind oft sehr persönlich, wie zum Beispiel Bilder der Liebsten. Diese Methode dient als Art Moodboard um den User besser zu verstehen und leichter Empathie entwickeln zu können.

by Nicolas Saavedra Rueda - July 13, 2021July 13, 2021

Meditation for Mental Health

“According to the National Center for Complementary and Integrative Health, meditation can help reduce stress, chronic pain (such as headaches), and blood pressure, as well as help you quit smoking and better navigate a variety of mental health conditions” (CANNING, 2020).

The new direction of my thesis topic brought with it some things that I had to thought about. As I decide on meditation for helping people improve their mental health by preventing anxiety, stress, and depression. The first thing that I decided to do was user research for preventing that there is no interest in users from using something like this. As a second step, I look for what is available in the market, and what people liked about the existing app. As the last step for this semester, I thought about the app structure. With all these steps after a hard semester, I can say that I am on a good way to continuing with the main part of the project that is developing.

After choosing the new topic for the project the first thing I decide to do was user research to find out if it was or not a good idea. For these, I decided to interview different people that are into the meditation practice, most of them gave me positive feedback on an app. The first thing that I was interested to ask about was if they think that is helpful to use an app for meditation, circa 85% of the interviewers told that yes and that some of them already use an app for it. The user explained to me that they do not want an app that teaches but more an app that has different content for different purposes. The main reason for this is that most of the user doesn’t have much time spend it on the learning process. User expectancies of the app are guided to information capsules for different purposes like sleep, focus, anxiety, stress, depression, mindfulness, and more all these capsules must be thought to improve mental health or to keep it at good levels. The research also shows that having time, difficulties level, and guides meditations can be useful for the app. Another result of the research was that most of the users find the app they use confused and not friendly for using, this means that here I can have an advantage in developing the app. This gave me the impulse to look for what is available on the actual market.

As soon as you look for the meditations app in your smartphone store, this one will give you a recommendation about which one can be the best for you. These recommendations are based on the number of downloads and reviews, so most users expect that the app they choose is good. In the meditation and mental health market, there are a few apps that show app first when you look for meditation. It is hard to give each app a specific position so I will just mention the app and its main purpose. Calm app lets you choose between your meditation practice. After all, the app provides guided sessions ranging in time from 3 to 25 minutes. And with topics from calming anxiety to gratitude to mindfulness at work—as well as sleep sounds, nature sounds, and breathing exercises—you can choose your focus. Experts across the board agree that Insight Timer is primo when it comes to choosing a meditation app.

“This app has many of the most experienced mindfulness teachers on it, and allows you the freedom to pick and choose depending on how long you have to practice, what style you’d like (e.g. body scan, loving-kindness, anxiety/stress-reducing, etc.), or just set a timer and sit without guidance,” Tandon says. On the other side, the Headspace app offers the widest variety of meditations, with the best-guided sessions for beginners, as well as less-structured programming for pros. Its easy-to-use interface was also the most streamlined.

Many different sites recommended different structures and base things that this kind of app should have. Now I am still investigating which structure combination can be the best for a rough prototype for trying out with users. In many sites they recommend different structures and organizations but, I look through it and I find things that can be improved to something better. Even due for the moment I can not says clearly what will be or how it will be organized.

Source:

https://www.mindful.org/how-to-meditate

https://www.mayoclinic.org/tests-procedures/meditation/in-depth/meditation/art-20045858

https://www.healthline.com/health/mental-health/top-meditation-iphone-android-apps

by Alexander Moser - July 12, 2021December 22, 2021

Aktuatoren

ERM – Eccentric rotating mass

Die älteste Methode zur Erzeugung um ein haptisches Feedback zu erzeugen, basiert auf einem Motor mit einem kleinen Gewicht, das exzentrisch auf der Welle. Die Kraft wird in zwei Achsen erzeugt senkrecht zur Welle des Motors erzeugt. Sie waren früher in allen Smartphones zu finden und erzeugen die charakteristischen Vibrationen. Die Amplitude (Stärke) wird bestimmt durch die Frequenz (Drehzahl) des Motors. ERMs sind gleichstrombetrieben.

LRA – Linear resonance actuators

In LRAs drückt eine Schwingspule einen Magneten gegen eine Feder. Durch Anlegen einer Wechselspannung führt diese oszillierende Bewegung zu einer vibrotaktile Rückmeldung in einer einzigen Achse. Die Rückmeldung von einem LRA ist gezielter und sauberer im Vergleich zu ERMs, weshalb sie den größten Teil des Marktes erobert haben. Ein LRA ist auf eine bestimmte Frequenz abgestimmt basierend auf der internen Feder [Resonanz Frequenz]. Dies ermöglicht die Steuerung der Schwingungsamplitude, ohne die Frequenz Frequenz bis zu einem gewissen Grad zu beeinflussen.

Oberflächenwandler

Wenn man die Membran eines Lautsprechers entfernt, erhält man die nackte Schwingspule. Auf einer Oberfläche angebracht, wandelt sie ein Eingangssignal in eine akustische oder in diesem Fall taktile Rückmeldung. Sie sind sehr ähnlich zu LRAs und benötigen ebenfalls eine Wechselspannung.

Hubmagnete

Ein naher Verwandter des LRA ist der Hubmagnet. Er ist im Vergleich nicht auf eine oszillierende Bewegung, sondern beschleunigt eine Masse, bis sie einen mechanischen Anschlag erreicht Anschlag erreicht. Eine Feder drückt die Masse zurück zu ihrem Ursprung. Magnete sind gleichstrombetrieben und erzeugen, je nach Größe, eine sehr hohe Stoßkraft.

Beschleunigter Stößel

Eine Mischung aus einem LRA und einem Elektromagneten sind beschleunigte Stößel oder lineare Wandler. Sie sind größer dimensioniert und erzeugen einzelne Impulse und Schwingungen durch Beschleunigen und Anhalten einer internen Masse durch ein elektromagnetisches Feld in zwei Richtungen beschleunigt und gestoppt wird. Unter Fall des Tac-Hammers (Nanoport) eine Seite mit einem mechanischen Anschlag für ein klickendes Gefühl, während die gegenüberliegende Seite einen magnetischen Anschlag für eine sanfte Rückmeldung. Sie werden meist mit Wechselspannung betrieben.

https://blog.piezo.com/hapticactuators-comparing-piezo-ermlra

https://www.precisionmicrodrives.com/content/ab-020-understandinglinear-resonant-actuatorcharacteristics/

Alexander Moser
https://www.alexander-moser.at/

by Petra Vymazalova - July 11, 2021

Interview – sign language student

As part of the methodology, I held an interview with a student who is studying sign language in Austria for three years to become a interpreter. The 21-year-old female became interested in sign language six years ago. After watching videos via YouTube from deaf persons signing various vocabulary, she realized she wanted to learn the language even though there was no one in here nearer surroundings who was hard-hearing or deaf. She was especially fascinated by the movements which looked in her eyes like an artistic dance, the variety of the language by facial expression and posture and that it was so incredibly different from the vocal one she knew. To become very good at signing, she attended a course at Urania in Graz. As time went on the course came to an end but she practiced further on with a private teacher so that she finally could take part in the selection process to study sign language in her bachelors which she is doing now for six semesters.

Very interesting was how she described the own learning process from her studies. In the beginning of her beginner course, she had to write down the vocabulary which the lecturer was signing to learn them later at home. First of all she did not know what to focus on and which movements are more common than others so she just wrote down the movements as she thought she could later understand and how she has seen like “left hand does this and the right one does that while the thumb of the left one does this …“ but could not remember the exact movements at home. As some of the lecturers do not provide the students with prerecorded videos of the vocabularies, the students must record them themselves to learn them. After some time, she learned that there are a few ways to transcribe the gestures more comprehensible. By classifying the movements by dividing the movement into components with answering questions like “in which direction do the palms point?”, “is the movement of the hands symmetrical?”, “which form has the hand?”, “what do the fingers do?”, “are the shoulders involved in the movement?” and more.

In the beginning the students vocally spoke out the words they were signing and structured the sentences in the vocal language grammar approach. After some time they stopped speaking next to the signing and started to structure the signs in the correct sign language grammar way. But she also knows that the current study beginners do not vocalize the signs anymore but do it right away without it and structure sentences correctly as well from the beginning.

own image: she did it like on the right side but should have done it like on the left side

She explained that when she must do homework or tasks for her studies, the students record themselves while gesturing sentences and vocabulary. To catch every detail, they have to set up the camera in a proper way that their whole upper body is visible. If there are movements outside the visible area they either have to turn themselves to enable a view from the side or turn the view of the camera itself. The lecturers who provide their students videos record themselves from the front and if needed from the side as well when a part of the movement is hidden between or behind hands or other parts of the body.

One of the lecturers once advised her after the examination of one of her tasks that she should show the sign for timeline from the other perspective as when she tried to explain when exactly in the course of the day she has eaten or taken a break, it was not possible to see the space between the hands which showed the timeline as she had them hidden behind the hands. This showed, that it is also possible to change up the sign itself and to show the sign from the other side.

For her it is common to borrow out and search for vocabulary with the help of physical dictionaries which are mainly consisting of verbatim descriptions, some of the digital comprise fotographs as well videos. A few examples are LedaSila https://ledasila.aau.at/, spreadthesign https://www.spreadthesign.com/de.at/search/, signdict https://signdict.org/about and more. If you want to search by giving parameters of the movements because there is no specific word or sentence to describe the vocabulary or meaning, the only help you will get for searching for these “special gestures” (“Spezialgebärden” like for example “not knowing how to start”) is over ögs Gänsefüßchen https://sites.google.com/site/oegsgaensefuesschen/ as it is the only side offering the possibility to search by descriptions of movements or gestures. Ögs states that “special gestures” are a category that does not exist from a linguistic point of view. These signs are described and highlighted as “untranslatable”.

source: https://sites.google.com/site/oegsgaensefuesschen/home/suchmoeglichkeiten: search by pictures, parameters or correspondences

Difficulties she recognized throughout the time are that there are so many dialects even throughout Austria itself and that signs in general can change quickly so that when she communicates with deaf in her age or visits a retirement home and signs with elderly people it is a completely different experience as the signs differ. Another point she mentioned is the lack of teaching materials as it is not easy to get material and self-study is rather difficult in terms of sign language. A study colleague of her quit studying sign language after one year because he could not manage to imitate the movements and it was not possible for him to train the spatial imagination for that. She stated that some lecturers describe the experience of online-teaching for study-beginners in the first semesters currently in corona times very difficult as it is only a two-dimensional experience.

As I told her about my ideas regarding the application, possible features and visualization in the end, she said that she liked the idea of an application for learning sign language and can especially imagine it in the phase of learning vocabulary with it as it is an absolute necessity in her eyes is the contact to native speakers to gain a good level in signing in the further stages of learning. Furthermore she stated that the feedback feature is very important if there is no person next to you telling you if the gesture is done wrong. Regarding the area that should be visible for the gesturing in general, she showed me a few gestures that were not restricted to the main area from the hips to above the head as she thinks that the system can not track a few of the gestures then. For example she showed me the sign for “eyebrow” where you actually slide with your fingers around your own eyebrows as it could be difficult to touch them when you wear the glasses as well the sign for “curious”(touch on the bridge of the nose) as well as “glasses” or everything around the eyes and in the face that you physically touch to relate to it, could be on one side hard to touch or reach under the glasses and on the other side hard to track by the system if they are over the head or under the glasses to give a feedback.

There are classifiers like for example spatial classifiers for “passing the house with the bicycle” where you show the position of yourself on your bicycle and how you move forward in the spatial area by pointing out the positions in the area in front of your own body and sign the house, the bicycle and the passing. The house then serves as a box in the spatial area after you signed the sign for house one time, you then later just relate to it as a point or box in the spatial area. In the sense of “where it is located and in which direction something goes”, one sometimes does not even show the sign for “house” at the beginning, because it occurred in the sentences before and from the context it is obvious that the house is meant.

Regarding tracking she can not imagine in different scenarios (like “Going to grandma’s house but on the way there still going to the bakery” or when talking about friends standing by, just pointing at them without signing their name again) and with other classifiers how this will be possible to track correctly and thinks it will need a lot of deaf to help evolve the tracking systems to bring them to a right level of accuracy so that testing is indispensable. As an example, she names that many deaf people can not even understand what avatars are signing or want to communicate because they can not relate to the movements and translate it to their own style and the mimic of the avatars is too difficult to read although it is decisive for the context. She and her study colleagues talked a lot about the facial expressions and naturalness of avatars. In their eyes it is necessary to test avatars with a lot of deaf people first to help developers to adjust the look of the avatars so deaf people can understand avatars because it would be useful to use avatars for announcements at the train station (delays) or other short notice information as these are currently not even communicated to deaf in most cases.

Conclusion

This interview was very helpful to reassure myself of the outcome of my previous research from the first and this semester as many points which occurred throughout it proved my outcome of the research from the literature and the internet. It helped me to get to know if there is a necessity for my idea of the application in the first place and which aspects I must think about when evolving it. As I did not have much knowledge about the process of the proper education trough universities of organizations which offer courses, I have got new insight into the phases of teaching and methodological approach. I can get more interview partners as she offered me to connect me with her study colleagues as well with especially one who is currently engaging in writing a bachelor thesis on the topic of avatar appearance and understanding which will be a great input for my own thesis as I imagined using avatars in the beginning but are now considering if another visualization would be more suitable and understandable after her explanation. In general, I plan to focus on the visual appearance of the application and how I will structure the application in the next semester. I could send possible visualizations to her study colleagues to let them evaluate them as I see the visualization as a key factor of willingness to use the application or not if it is not done understandable and aesthetic while being helpful.

Sources

https://sites.google.com/site/oegsgaensefuesschen/

https://ledasila.aau.at/

https://www.spreadthesign.com/de.at/search/

https://signdict.org/about

by Nicolas Saavedra Rueda - July 10, 2021

A new approach for Mental Health

By the disappointing results from the user research in VR therapy, I got into a crisis point in my research. Where I should decide which direction will be the best for my topic and what I want to have as a result of it. During this rethinking process, I got different approaches to get into a good path for developing the project. The first decision I took was to stay or not into the topic of mental health. The second point I thought was to decide how I want to help, and which way is the best to do it. The last point is to decide on a new starting point to start developing the project. All these processes took weeks to pass through and the result is something that I am happy about.

As I describe in the last topic of my thesis research I got not the best results during my first user research, this impulses me to think about reapproaching the topic. During the process, I realize that I do not want to get out of the field of mental health. But it was difficult to find something where I can help people without being involved directly in medicine. To be a part of medicine is tactics that prevent me to get into certified permissions, medical regulations, and for that moment the users that will manage the app. This decision put me in the position of prevention of mental illness in a state of treatment. At the same time, it opens the ways to alternative methods.

As I just mentioned before alternatives methods to prevent anxiety, stress and depression are often used by people to keep their mental state positive. There are various methods in the world that people use some like sports, hobbies, religion, and more. Each method has a different perspective of helping through the process and each of them has different efficiency and effect on our mental health. After a small research, I decided to focus on meditation because it shows the best results on preventing and improving the mental health of people. As it says in the article Effect of Transcendental Meditation on Employee Stress, Depression, and Burnout: A Randomized Controlled Study “Studies indicate that practice of TM reduces the psychological and physiologic response to stress factors, including decreased sympathetic nervous system and hypothalamic-pituitary-adrenal axis, and reductions in elevated cortisol (stress hormone) levels” (Perm, 2014).

Meditation is a kind of practice that has hundreds of years of existence and has a different cultural background. Each kind of meditation has a different purpose on our minds and bodies, which is important to be considered. A meditation process is simple to be taught and to learn what is hard about it is to keep yourself constant on the practice. The meditation practice brings multiple benefits for your health but, it is important to organize the way to do it to get the right things you are looking for. In the next part, I will focus to explain how I will look to achieve this through the use of an app.

by Nicolas Saavedra Rueda - July 9, 2021

Summary of the first semester and ideas changed comes and goes.

At the beginning of the Des&Res class the last semester, I decided to start research in mental health. The main topic of the research was VR for Mental Health, that is a methodology that has gained prestige in the last years, due to the advance in technology. During my research on this topic, I was centre on investigated how it works and what is their main field of application, for this, I read through different articles of medicine relate to these topics. In a second step I follow to get related to the history of VR and how long has it been in the medical field and for whom it was used. Finally, I focus on the design of a VR interface and prototype, with also a user research focus where I found something that changed my topic orientation.

For starting this research, I thought that VR therapy for mental health can be a key aspect during COVID-19. With this idea in my mind, I start research in VR for Psychotherapy focus on mental health, the focus was to get a better environment for treating special kinds of mental issues. During my research, I found out that VR in mental health can be useful for treating anxiety, stress, and depression. As Nigel Whittle said “VR offers the opportunity to develop more personalized therapeutics, especially in mental healthcare. It is already being used to treat PTSD, phobias, and psychiatric conditions such as conversion disorder and showing excellent results.” (WHITTLE, 2020). VR therapy had shown the last year more results during the pandemic, but even due some users do not trust this kind of technology. Also, VR therapy has been in the market for around 20 years.

VR therapy started around 1950 for treating different kinds of phobias around people. This treatment used multiple sensors to help the user experience the same sensation that provokes the phobia. But, it was not till the beginning of the 2000s that VR therapy was considered to be useful, in the article The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders the authors mentioned that “The first study to formally investigate the efficacy of VR-based exposure therapy (VRE) focused on the treatment of acrophobia and results suggested that VRE was effective” (Rothbaum, Bunnell, Sae-Jin, & Maples-Keller, 2017) this study was made in the early empowered the technology to continue into a development area in medicine. After this series of studies at this time, VR therapy was used in the military area to treat PTSD to help soldiers recover due to missions. In actuality, VR therapy is managed mainly by three big companies that are: Limbix, Psious, and OxfordVR. All these companies have a focus on VR therapy but, their UI/UX is in my opinion a little bit too old and hard to use. This brings us to the next step that is VR design and user research.

In the last part of the last semester when I finally got the focus on which kind of path I was getting in, I decided to research design for VR and make user research too. Design for VR is something that I thought will be hard. Surprisingly it was not, VR interfaces and design are still based on the same idea of a normal application for a smartphone or the computer. Not to miss, VR design has one important thing to be careful about and it is sound. VR sickness is one of the major issues due to the therapy and it is provoking usually for a misinterpretation of the sound and the environment, to prevent that to happen, sound designers use 8D sounds. This kind of sound helps the user to identifies themselves in the environment preventing the sickness to happen.

Into the user research, I have begun it was a disaster and it turns to be disappointing. I researched with 8 psychotherapeutics and most of them found this topic to be negative instead to be something useful. The main reason for them to think this was that they thought it will be more learning and work for them. Even, I told them about the studies and the results, the doctors did not change their minds. Also, they expect that these therapies will be really expensive. This put me into a crisis point for my research where I decided to changed my path to something else that is not out of mental health.