AR technology
There are several types of AR in use today. For a better understanding of AR it is important to know the types and their advantages. There is marker-based and markerless technology.
Marker-based AR is:
- easy to produce
- most available technology (supports biggest variety of devices)
- commonly used for marketing and retail
Markers can be images or signs that trigger an augmented experience and act as anchors for the technology. A marker has to have enough unique visual points — especially images with corners or edges do well. Logos, packaging, QR-codes, products themself (engine, bottle, chocolate bars) can serve as markers. The technology uses natural feature tracking (NFT) and can share AR content like text, images, videos, audios, 2D/3D animations, objects, scenes, 360° videos and game mechanic. For image recognition there are license based solutions (software development kits) on the market like Vuforia, EasyAR, Wikitude and more.
Markerless AR is:
- more versatile
- not restricted to any surrounding
- allowing a multi-person interaction in virtual environment
Different styles and locations can be chosen by the user who can also rearrange his surrounding virtual objects. The user’s device collects information through the camera, GPS, digital compass and accelerometer to augment realities into the scene. It is restricted to some devices: for iOS it has to be version 11 or up and for Android 7.0+ or newer. For placing objects in the real world, it uses plane detection which means that horizontal and vertical planes are detected. When ARKit or ARCore (Apple’s iOS and Google’s Android AR framework) analyse the real surfaces in the natural environment and detect a flat surface, it places a virtual object on the detected surface so that it appears to rest on the real surface. This happens with new virtual coordinates which are related to the real coordinates in the environment.
Following types technically fall under markerless AR as well:
- Location-based AR ( Pokemon Go: characters in specific locations )
- Superimposition AR ( recreating or enhancing a object in the real world )
- Projection-based AR ( projectors instead of mobile devices: hologram-look )
- and Outlining AR ( outlining boundaries and lines, e.g. on the lane )
A combination of both technologies — marker-based and markerless — in one application is possible as well.
As reviewing the last blog ( number |4 ) about current AR applications, regarding the use of flashcards to learn the sign language, the cards themself serve as markers and marker-based technology triggers augmented realities.
Developement of marker-based app
As an example will be described how a mobile AR marker-based letter recognition application was developed to get to know the rough technicalities. Suman Deb, Suraksha and Paritosh Bhattacharya analysed how a AR application should be created and developed to improve deaf-mute students’ sign language skills. The used media-cards showed a specific Hindi letter and triggered the application to display 3D animated hand motions for each letter.
A quick description:
- Upload pictures of markers To make an app response to certain images ( markers ) every image has to be stored in a Library. The developers of the app uploaded them to the Vuforia Library. As described above, the Vuforia Engine is a software development kit (SDK) and allows developers to add advanced computer vision functionality to any application, allowing it to recognize images and objects, and interact with spaces in the real world
- Download Unity Package which was downloaded from Vuforia was imported. Also along with Vuforiaandroid and vuforiaimage targets-android to generate Augmented Reality application
- 3D hand model arrangement with markers To ensure that the right image is shown, you match and place them into an Image Target Hierarchy so that after scanning of the media-cards ( flashcards ) the corresponding animated hand 3D model is shown in the augmented reality interface
- Include Features The menu gives features like zooming in and out and rotation of the 3D hand. The scanned letters ( markers ) can be amassed into words
In use the camera feed will be analysed by the image capturing module when pointed over the marker. Then binary images are generated and processed by the image processing technique. Marker tracking modules tracks the location of the marker and displays the corresponding 3D hand animation over the marker itself. The Rendering module has two inputs at a time: calculation of the pose from the Tracking module and from the virtual object which will be augmented. The Rendering module also combines these inputs and renders the image on the screen of the mobile device.
Sources
https://www.springerprofessional.de/plane-detection/16253564
https://www.howtogeek.com/348445/what-are-the-arcore-and-arkit-augmented-reality-frameworks/
https://learn.g2.com/augmented-reality
https://www.youtube.com/watch?v=qAaUSmVfpaU
https://www.youtube.com/watch?v=16jT1_MtTXs
Suman Deb et al. / Procedia Computer Science 125 (2018), p. 492–500: “Augmented Sign Language Modeling(ASLM) with interaction design on smartphone – an assistive learning and communication tool for inclusive classroom”
