Illustration und Fotografie als Visual Language

In meiner Recherche werde ich Illustration und Fotografie als visuelle Sprachen analysieren und miteinander vergleichen, wobei der Fokus der Recherche auf dem Einsatz dieser Medien im Bereich des Communication Design liegt. Ich werde Illustration und Fotografie als Bildgebungsmittel erforschen, weil ich herausfinden will, ob sie als Transportmittel für Informationen austauschbar sind und ob sie in denselben Anwendungen verwendet werden können. 

Das Ziel meiner Recherche ist es, herauszufinden, wie Illustration in einem Anwendungsbereich, in dem traditionellerweise Fotografie verwendet wird, eingesetzt werden und möglicherweise sogar einen grafischen Mehrwert bieten kann. 

Dabei könnten Fragen wie diese bearbeitet und hoffentlich beantwortet werden: 

  • Welche Charakteristiken haben dieses visual languages und welche Signale senden sie?  
  • In welchen Bereichen ist welches Medium sinnvoll?
  • Was sind gängige Einsatzbereiche und gibt es Projekte, die es schaffen diese Grenzen erfolgreich zu überwinden? 
  • Was passiert wenn man Illustration in hauptsächlich der Fotografie vorbehaltenen Bereiche einsetzt und umgekehrt? Sind sie austauschbar?
  • Welche Wechselwirkung zwischen den beiden visuellen Darstellungsmedien gibt es und wie beeinflusst sich die Art der Darstellung gegenseitig? 
  • Welche Vorteile bzw. Nachteile bringen sie in Hinblick auf das Ergebnis einer grafischen Arbeit? 
  • Wie kann man die Nachteile einer visuellen Sprache gegenüber der anderen ausgleichen? 
  • Wie viel Menge an Information, Wahrheit, Emotion kann transportiert werden?  
  • Was sind die Creative Possibilities der beiden visuellen Sprachen?
  • Wie kann man Illustration und Grafik sinnvoll mit Fotografie kombinieren bzw. verbinden?  
  • Wie kann das Problem der Wiedergabe von Realität bzw. Ausdruck von Fiktivem gelöst werden?  

augmented reality – application for learning a language | 1

Augmented reality (AR) has become an attractive trend in the field of language learning. Publications throughout the years 2014 to 2019 show the popularity of mobile-based AR mainly for learning the vocabulary, reading and speaking. There are multiple choices when searching for apps to learn English, Spanish, Portuguese and many more but what if the language you want to get to know to communicate with our opponent is a visual language which depends on signing (forming words by gesticulation with hands)? The main reason for the chosen topic is to alleviate the communication to and with deaf people by making this visual language more accessible in a learning application by implementing augmented reality.

photo by google on https://www.bbc.com/news/technology-49410945

State of the art

The work on technologies, algorithms and language AI that tracks and translates hand gestures into speech developed a lot in the last years. In hope that developers will use them to make own translating apps, Google has published algorithms which can perceive the shape and motion of hands in 2019. Problems the technology is currently facing are hidden fingers and regionalisms in sings meaning that signs differ in specific local areas. Furthermore signing depends not only on hand signs but more importantly the meaning can vary through facial expressions or speed of signing.

To close the gap and enable communication to family members, innovators around the world have created own solutions and technologies like Roy Allela who made a pair of gloves for his hearing-impaired niece. His application reads the text aloud from the movement of the haptic gloves.

Current apps use pictures of hand guestures or videos and many do not include the upper body which is as already mentioned important for the signing context.

image: https://gebaerdenlernen.de/index.php?article_id=112

Goal

The app should include different possibilities for both sides, people which are hearing and which are not. For example features for learning the guestures/signs with the help of augmented realities to get in touch with the actual situation and speed of gesticulation. For communicating the app should translate the video information of recorded movements to text or audio.

Sources

https://onlinelibrary.wiley.com/doi/10.1111/jcal.12486

https://gebaerdenlernen.de/index.php?article_id=112

https://www.geo.de/geolino/mensch/1854-rtkl-gebaerden-wie-gebaerdensprache-funktioniert

https://www.gehoerlosen-bund.de/faq/deutsche%20geb%C3%A4rdensprache%20(dgs)

https://www.bbc.com/news/technology-49410945

Contemporary photography – Still life

Modern still life photographs benefit above all from the teaching of still life painting, which flourished in the 17th century. Still life has been used as a motif for centuries and is now being reinterpreted. Above all, the design direction is interesting, which adopts the compositional bases and the optical structure of the classic still lifes and refreshes them with modern elements. This modern direction mostly has a humorous, sarcastic or even socially critical character.

Examples for modern still life photography

Blow up – Ori Gersht

The photograph shows the moment when a bouquet of flowers explodes. The composition is based on a classic example by Henri Fantin-Latour. The technical development of photography enables the artist to capture a 6000th fraction of a second. The intensity of the explosion is deliberately emphasized in this work as a contrast to the calm, classic still life painting. According to Gersht, the message of impermanence should remain.

Natura Morte 2 – Cindy Wright

The work of the artist Cindy Wright has a consumer-critical aftertaste. In the past, dead animals were often used as a motif in still life in classic paintings. However, in our modern times, the artist would like to draw attention to the fact that meat and fish are considered food, but pay a high price for it, as the death of a living being is claimed. This moral responsibility is illustrated by the portrait with the gutted fish in an ornate fishbowl.

Broken Things – Livia Marin

This work is also about removing everyday objects from the ordinary image. In this way one raises the value of the object to something special, as it is intended in the whole ideology of a still life. With this depiction of the broken and then healed cup, the artist demands more attention for the works that are connected and created with life.

One third  – Klaus Pichler

The series of photographs by Klaus Pichler criticizes the wear and tear of food, which, according to a UN study, amounts to around a third of the food produced. The arranged motifs, which show moldy food, should also draw attention to the transport, production and resources used for the food.

Quellen:

https://www.fresko-magazin.de/stillleben-reloaded/

https://www.pixolum.com/blog/fotografie/stillleben

Virtual Reality in Healthcare

Virtual Reality has the possibilities to revolutionise the Healthcare

I am interested in the possibilities that virtual reality can bring into medicine. In the last years, VR has changed the healthcare industries. From developing new life-saving techniques to training the doctors of the future, VR has a multitude of applications for health and healthcare, from the clinical to the consumer.

Virtual Reality has the ability to transport you inside the human body – to access & view areas that otherwise would be impossible to reach. Currently, medical students learn on cadavers, which are difficult to get hold of and (obviously) do not react in the same way a live patient would. 

In VR however, you can view minute detail of any part of the body in stunning 360° CGI reconstruction & create training scenarios which replicate common surgical procedures.

Medical Realities is one of the companies pioneering the use of Virtual Reality to deliver high-quality surgical training. They film real-life surgery in 4K 360° video from multiple angles which is then combined with CGI models of the anatomy being operated on to provide an immersive & interactive training experience.

Fields of VR in Healthcare

  • Medical training
  • Patient treatment
  • Medical marketing
  • Disease awareness

Video Example

Articles

Virtual Reality for Health Care: a survey

Judi Moline

Embodied Labs

Carrie Shaw

Analog / Digital – a short introduction

The very early differentiation of analog and digital media came up with the development of the first computers and cybernetic approaches in the 1940s and since then became a major question in media theory. Especially with the invention of CDs in 1982 the difference between the digital and analog sound medium (vinyl records vs. CDs), practically displayed the divergence of the back then old and the new media and their featured characteristics. The question was not only if the medium carried digital or analog information, but also how the information was recorded (again either digital or analog) were indicators for the quality of the medium. In the specific example CDs have been said to have a more cold, but brilliant sound, while vinyl records purvey a much warmer and thus charming sound.

However, the core difference between analog and digital media seems to be determined via their representational qualities. While the (new) digital media are based on a binary system (0 & 1 = yes & no) leaving no space for coincidence or mistakes, analog media and their representation of information are perceivd as rather blurred, distorted and loss-making, hence the variety of their physical entities.

Furthermore one can sum it up that physical analog media are real things carrying information, while the digital counterpart acts in a hyper-real, virtual space, trying to simulate reality.

Another differentiation of various types of media after Heinz Pürer, extended by Marcus Burkhart also seems important for a contemprary view and disctinction of analog and digital media:

a) Primary Media
Media of human elementary contact, i.e. language, mimic, gestures used by humans to communicate directly.

b) Secondary Media
A part of the interacting parties uses technical tools to communicate. These tools can be smokesigns, but also newspapers, books and posters.

c) Tertiary Media
Both of the communicating parties use technical tools to send and receive information, e.g. telecommunication, TV, radio or computer.

d) Quarternary Media
Besides telecommunicative tools, both, sender and receiver need to have an internet connection. Unlike „classic“ media, there is no clear sender and receiver, communication happens interactive.

Keeping this differentiations in mind, new and interactive media are using electronic devices which don’t simply carry digital information virtually between human senders and receivers. The medium itself can be receiving and storing information and likewise not only forward this information but also artificially generate own contents based on the information supplied.

On the other side, analog media is defined by physical entities, that carry information. This kind of media has to be actively filled with information by the sender and actively received by the sender, using (various) human senses.

Analog vs. digital

With an extraordinary expansion of digital media within the last decade(s) analog media has often been said to lose it’s relevance and for some parts will be completely replaced by digital media.

While obviously new digital alternatives like online newspapers or music streaming services are conquering with „traditional“ media like printed news or vinyl records, analog media are defending their position and for some parts taking back lost shares of the market. Not only sales of vinyl records have been rising in the last decade, but also analog board games, real notebooks and printed books are getting more and more popular (again).

Even in the offices of new technological enterprises like Google, Facebook, or Adobe the value of analog design processes, like sketching and scribbling with pen and paper is highly appreaciated. Digital creators are aware that at some stages of the creative process analog approaches are way more effective to generate usable output then immediately starting to design on screen.

Besides that, the digital world needs the real world to represent it’s ideas and information. As Michael Meyer stated in 2013, the digital world is full of (analog) analogies. Just like the desktop of your computer is designed like your real desktop, featuring directories to store files and a trashcan to put in things you don’t use no more.

Sources

Analog/Digital – Opposition oder Kontinuum?, Jens Schröter, Alexander Böhnke (Hrsg.), 2004
Die Rache Des Analogen, David Sax, 2016
https://medienkindergarten.wien/medienpaedagogik/infothek/der-medienbegriff/
https://www.youtube.com/watch?v=grmZmibek70

Digital Truth

Introduction to the topic Digital Truth.

In times of new media and fake news it is hard to know which facts are actually true and which are not but why is this a problem for humanity?

You might think that some misinformation might not be harmful, but a workshop of Yale Law School and the Floyd Abrams Institute for Freedom of Expression showed that fake news can have a bad influence on our society. This problem exists not only in politics but also in our daily life. But what means bad influence and how can we make the online world more transparent?

Since the beginning of time humans were never exposed to such tons of data as we know today. At the beginning of the internet age people did not really use or understand the power of the world wide web. It started around the turn of the century that humans got connected and since then it increased exponentially. The devices got easier to use and the screen design improved as well. Originally most of the information online was reduced to fun articles and some early staged websites with mostly bad usability but that changed quickly. More and more humans became as we call them “Users” and at the same time the amount of misinformation rose and the transparency decreased. Nowadays it is hard to distinguish what information is correct and what is only there to get our emotions out of control. Bots and people who distribute false stories for profit or engage in ideological propaganda are now part of our everyday life as we spend around up to seven hours a day in front of a screen. Since the beginning of the pandemic our daily screen time might have increased even more. The positive or negative health effects of screen time are influenced by quality and content of exposure. The most salient danger associated with “fake news” is the fact that it devalues and delegitimizes voices of expertise, authoritative institutions, and the concept of objective data – all of which undermines society’s ability to engage in rational discourse based upon shared facts.

Reseach result of the American Press Institute

In 2014 some researches tried to cluster algorithms which have emerged as a powerful meta-learning tool to analyze massive volumes of data generated by time-based media. They developed a categorizing framework and highlighted the set of clustering algorithms which were best performing for big data. However, one of their major issues was that it caused confusion amongst practitioners because of the lack of consensus in the definition of their properties as well as a lack of formal categorization. Clustering data is the first step for finding patterns which may lead us to detecting misinformation, false stories, ideological propaganda or so-called fake news. It is also a method for unsupervised learning. Furthermore, it is a common technique for statistical data analysis used in many other fields of science and if used correctly it could be a game changer for our online and offline society.

PEW Research Center Internet and Technology

Why does Fake News exist?

A pretty important thing to know about social media, is that always the most recent published or shared content is the first you will see. That means if there is no reliable recent post on a topic, it leaves a so-called data void behind, which means as soon as somebody publishes something new on this topic, it will be shown first. This comes from the fact that we always long for “new” news, despite the fact that no one, no tool nor algorithm has ever screened these information verifying its accuracy.

Example of how data voids work

What about Twitter?

Since May 2020 Twitter is trying to make it easy or easier to find credible information and to limit the spread of potentially harmful and misleading content. They introduced new labels and warning messages that will provide additional context and information on some Tweets. These labels will link to a Twitter-curated page or external trusted sources containing additional information on the claims made within the Tweet. 

Twitter labels for false information about COVID-19
Twitter warnings for conflicting content

So Twitter is one of the major social media platforms actually labeling content, despite it being the Tweet of the current president of America alias Donald J. Trump. Also they are actively trying to decrease the spread of misinformation though introducing an extra notice before you can share conflicting content. Since content can take many different forms, they started clustering the false or misleading content into three broad categories:

categories of false or misleading content

Of course Twitter is not the only platform labeling false information or content going viral – Facebook and Instagram started doing that too. Instagram has been working with third-party fact checkers, but up until now the service was far less aggressive with misinformation than Facebook. Also qualitativ fact checking takes time, which can be problematic and there is still some catching up to do.

Instagram adds 'false information' labels to prevent fake news from going  viral
Facebook and Instagram – labeled content

Labeling or removing postings is a first approach in the right direction, but it does not solve all issues that come with false information and how we interact with it. This is why this topic is so important for the future and the wellbeing of our society.

Sources:

Fighting Fake News Workshop Report hosted by The Information Society Project & The Floyd Abrams Institute for Freedom of Expression

Links:

https://blog.twitter.com/en_us/topics/product/2020/updating-our-approach-to-misleading-information.html

https://towardsdatascience.com/the-5-clustering-algorithms-data-scientists-need-to-knowa36d136ef68

https://www.semanticscholar.org/paper/A-Survey-of-Clustering-Algorithms-for-Big-Data%3A-and-

https://researchguides.austincc.edu/c.php?g=612891&p=4258046/I

https://unesdoc.unesco.org/ark:/48223/pf0000265552

https://muckrack.com/blog/2017/02/27/fake-news-bubble

https://www.americanpressinstitute.org/publications/elections/trusted-elections-network/talking-about-misinformation-with-first-draft/

Audio & Machine Learning (pt 1)

Part 1: What is Machine Learning?

Machine Learning is essentially just a type of algorithm that improves over time. But instead of humans adjusting the algorithm the computer does it itself. In this process, computers discover how to do something without being programmed to do so. The benefits of such an approach to problem solving is that algorithms too complex for humans to develop can be learned by the machine. This leads to programmers being able to focus on what goes in to and what out of the algorithm rather than the algorithm itself.

Approaches

There are three broad categories of machine learning approaches:

  • Supervised Learning
  • Unsupervised Learning
  • Reinforcement Learning

Supervised learning is used for figuring out how to get from an input to an output. Inputs are classified meaning the dataset (or rather trainset, the part of the dataset used for training) is already split up into categories. The goal of using supervised learning is to generate a model that can map inputs to outputs. An example would be automatic audio file tagging – like either drum or guitar.

Unsupervised learning is used when the input data has not been labelled. The algorithm has to find out on its own how to describe a dataset. Common use cases are feature learning and discovering patterns in data (which might not have been visible without machine learning).

Reinforcement learning is probably what you have seen on YouTube. These are the algorithms that interact with something (like a human would do with a controller for example) and is either punished or rewarded for its behavior. Algorithms learning to play Super Mario World or Tesla’s Autopilot are trained with reinforcement learning.

Of course, there are other approaches as well, but these are a minority, and it is easier to just stick with the three categories above.

Models

The process of machine learning is to create an algorithm which can describe a set of data. This algorithm is called a model. A model exists from the beginning on and is trained. Trained models can then be used for example to categorize files. There are various approaches to machine learning:

  • Classifying
  • Regression
  • Clustering
  • Dimensionality reduction
  • Neural networks / deep learning

Classifying models are used to (you guessed it) classify data. They predict the type of data which can be several options (for example colors). One of the simplest classifying models is a decision tree which follows a flowchart-like concept of asking a question and getting either yes or no as an answer (or in more of a programmer’s terms: if and else statements). If you think of it as a tree (the way it is meant to be understood) you start at the root with one question, then get on to a branch where the next question is until you reach a leaf, which represents the class or tag you want to assign.

a very simple decision tree

Regression models come from statistical analysis. There are a multitude of regression models, the easiest of which is the linear regression. Linear regression tires to describe a dataset with just one liner function. The data is mapped on to a 2-dimensional space and then a linear function which “kind of” fits all the data is drawn. An example for regression analysis would be Microsoft Excel’s trendline tool.

non-linear regression | from not enough learning (left) to overfitting (right)

Clustering is used to group similar objects together. If you have unclassified data and want to make use of supervised learning, regression models can automatically classify the objects for you.

Dimensionality reduction models (again an aptronym) reduce dimensionality of the dataset. The dimensionality is the number of variables used to describe a dataset. As usually different variables do not contribute equally to the dataset, the dataset can still be reliably described by less variables. One example for dimensionality reduction is the principal component analysis. In 2D space the PCA generates a best fitting line, which is usually where the least squared distance from the points to the line is.

2D principal component analysis | the ideal state would be when the red lines are the smallest

Deep Learning will be covered in part 2 of this series as this is the main focus of this series.


Read more:

https://en.wikipedia.org/wiki/Glossary_of_artificial_intelligence
https://www.educba.com/machine-learning-models/
https://www.educba.com/machine-learning-algorithms/

The Environmental Film Activist Handbook |entry six

Just like in the last weeks, an audience watched an environmental documentary and answered a couple of questions after they finished watching. In this weeks entry I am also going to summarize the results of all four films and show data I had gathered in another questionnaire.

The audience is upwards from 20 years, from different countries in Europe, with different backgrounds and genders. The same audience watched all four films which I introduced in the second entry.

Below are the results of the fourth weeks film.

The results shall help documentary filmmakers reach their audience better and get greater results in spreading their message.

How did the audience perceive Kiss the ground?

After the participators watched the film they filled out a questionnaire. I did go through the answered questionnaires and summarized the answers, the results are shown below.

All viewers stated that they were interested in the topic and almost everybody had already informed themselves about this topic. 

What they liked about the film:

  • That they show a simple way to help with climate change
  • It’s a different and often unheard approach to combat climate change. It was quite optimistic
  • The approached was good, the topic of climate change without solely focusing on renewable energy sources. 
  • A lot of views on the topic, a lot of facts 
  • It gave an actual sound solution 

Everyone of the viewers thought that the topic was represented well.

65% of the viewers built a connection to somebody or something in the film.

Everybody stated that they want to research the film more.

What questions or thoughts came up?

  • Why do the states support such big farms?
  • It has raised thoughts, how to implement the new found knowledge without a garden
  • The most interesting point was the possibility of no till farming styles
  • It was less doomsday on the coming future and there might actually be a chance 

What was memorable?

  • The revegetation of parched farmland
  • The pesticides they used where also used in world war two in the gas chambers
  • The scale of the entire mass industrialized farming styles. It is difficult to grasp the size and scale of agriculture.
  • The satellite imagery of how landscapes are severely modified by fields and the process of farming. However on the contrary, the capability of nature to recover and rebound after such massive soil erosion, i.e. Loess Plateau, China. 
  • That cows and live stock cab actually help with regeneration and not build a part of the problem.

What message did they take from the film?

  • plant a lot of trees
  • Soil science is deeply a deeply overlooked part of the carbon cycle, which we as a species rely on still most people do not understand it.
  • Things can change for the better
  • Every field can be saved

Do you think about where your food comes from and how it is made when you buy it?

Everybody answered this questions with yes

If you are not already checking where your food comes from, will you now?

Everybody answered this questions with yes

https://youtu.be/KyQYYLsXhqc

Conclusion of the answers of the four films and second questionnaire:

The information I gathered from the question results showed, what the people still remember after watching the film, what they liked and disliked about them. All four films were about environmental issues. Still they were all different and also perceived differently.

The main protagonist of the first film Mission Blue was scientist Silvia Earle, the second weeks protagonist of an inconvenient sequel was politician Al Gore. The questionnaire revealed that the audience had a lot more trust in a scientist than in a politician. Scientific facts were overall wished in every film and a variety of scientific facts made the viewer trust the message more. 

A call to action and examples for resolutions are important, especially for films like this wich are about an environmental issue. Since the films are supposed to not only tell people about the problems but also make them a part of a positive change.

Shocking pictures seemed to stay in mind the best.  „A new study suggests that we recall bad memories more easily and in greater detail than good ones for perhaps evolutionary reasons.

Researchers say negative emotions like fear and sadness trigger increased activity in a part of the brain linked to memories. These emotionally charged memories are preserved in greater detail than happy or more neutral memories, but they may also be subject to distortion.“

“These benefits make sense within an evolutionary framework,” writes researcher Elizabeth Kensinger of Boston College in a review of research on the topic in Current Directions in Psychological Science. “It is logical that attention would be focused on potentially threatening information.”

source-https://www.webmd.com/brain/news/20070829/bad-memories-easier-to-remember

With each film I asked if the viewers did built an emotional connection.  „An investigation of autobiographical memories found that positive memories contained more sensorial and contextual details than neutral or negative memories (which didn’t significantly differ from each other in this regard). This was true regardless of individual’s personal coping styles.

  • Emotionally charged events are remembered better
  • Pleasant emotions are usually remembered better than unpleasant ones
  • Positive memories contain more contextual details (which in turn, helps memory)
  • Strong emotion can impair memory for less emotional events and information experienced at the same time –https://www.memory-key.com/memory/emotion

In fact did the results of the questionnaire show that people remembered shocking moments best and moments where they felt positive emotions and a connection to someone or something in the film.

The second questionnaire was answered by people in northern, central and south America, Australia, Asia and Europe, ages 20-65.

  • Where do people get most of their information from, where should films be promotet to get the needed attention?
  •   Why do people like to watch documentaries?

      To gain information about topics and learn something new.

  •   Have they watched a documentary which inspired them? Did it lead to an action?

From the answers I gathered from this question it turns out that documentaries can really inspire people to change something in their lives.

Some became activists because of documentaries they watched, others changed their lifestyle after watching a documentary and some were even inspired to choose their University major because of documentaries they watched on a specific topic before.

  •   What do people like to see in a documentary?
  •   Would they like further information about a topic they just watched?
  •   How long do they watch before they skip? How important is the beginning of a film?

The older generations 50+ stated that they need longer than two minutes, mostly 5-10min.

The younger generations, form early 20s to early 30s all stated that it takes them no longer than five minutes to decide if they want to watch a film, most of them not longer than two minutes.

  • Under which criteria is a film selected? What needs to be especially good to bring people to watch a film? 

Augmented Reality for industry

Augmented Reality (AR) is a growing and also promising field for industry. While companies like DHL or several automotive manufacturer make progress in the direction of Industry 4.0 with this new technology, there are many other possible use cases for AR which are not that evolved yet.

Selection of possible areas of applications:

  • trainings
  • maintenance and troubleshooting
  • visualization of data and work steps
  • marketing
  • brand experience on exhibitions and fairs

With AR workers can access digital information and overlay that information with the physical world. By combining real world environments with context-specific virtual information, manual operations are done more efficiently, also costly mistakes can be reduced and the mental effort for transferring information from an abstract repre sentation (e.g. monitor) to the physical world can be significantly reduced.

Example use case for troubleshooting:

Multiple studies proved that AR for industry increase productivity by an average of 32%. A big advantage of AR compared to Virtual Reality is the adaption with different devices like laptops, smartphones, tablets, smartwatches or data glasses. There is no external hardware needed and especially in today’s digital world of indispensable smartphone availability it is easy to implement AR in the daily life of employees.