Designing with sensor data

FOSDEM 2021

by André Jaenisch on 6th February 2021

jaenis.ch/hobbies/speaking/fosdem2021-javascript-device-sensors/

Designing with sensor data by André Jaenisch is licensed under Creative Commons Attribution 4.0 International Attribution 4.0 International

Welcome. This is my first time I'm talking at FOSDEM. You can follow along with the slides via the link.

By the time this airs, I hope to be able to provide subtitles for download. The content will be close to speaker notes you can find behind the link.

My name is André Jaenisch. I'm currently working as Software engineer at Jambit. It's a medium-sized software provider from Germany. This talk has no relationship with my work.

Since this reveal.js presentation is pre-recorded using OBS Studio, please ask your questions at the end. In between you will see scenes taken with ScreenCam on Android. I would welcome if you would allow me to add your comments to the presentation slides afterwards. Please tell me if you want to opt-in.

Unless otherwise noted, the content is made available under the Creative Commons Attribution 4.0 International license.

Responsive Web Design

Demo!

What's the user doing?

Detectar

After the talk I decided to create a small demo site, which runs some primitive checks on your device capabilities.

Since then, I started to design tooltips based on a hover media query instead of guessing from the device width. I'm not quite sure what the colours media query does. I can think of use cases for optimising for eBook readers, though. Orientation can be used if you have certain criteria. For example, showing a sidebar. Device Motion and Orientation have we seen just now. They got replaced by a Permission-based API. Accelerometer measures the velocity. Ambient Light the light level around the devices. I have a small demo for you here, too. Web Bluetooth is available on Chrome behind a flag. I've also got something prepared here. Geolocation should be known from map apps. Gyroscope tells you something about the position in space of the device. We saw it in the user activity demo. For example, when you have indication that the user is driving, perhaps offer to read alout your blog post? Magnetometer is measuring the magnet field of the earth. I think, you should be able to measure the altitude roughly? NFC or Near-Field Communication can be used on slow distances. I have nothing similar available.

What Can Web Do Today?

Device APIs overview

What Can Web Do Today?

Demo!

Ambient Light Sensor

Demo!

Bluetooth

Demo!

Voice Recognition

Web Speech API - Speech Recognition

Project Fugu

Platform Adjacency Theory

Device Sensors

Fugu status page

Picture credits

Flickr: The Plan Company by Alvaro Rubio ( CC BY-NC-ND )

Flickr: 89 Morpheus by Felton Davis ( CC BY )

Flickr: Samsung Galaxy Note 3 by Kārlis Dambrāns ( CC BY )

GameDev.js Jam theme - taken from the announcement post

Flickr: Code World by David ( CC BY-NC-ND )

Flickr: Fugu by williamnyk ( CC BY-NC )

Flickr: Week 15 - Magic Hour by Morgan ( CC BY )

Questions?

Thanks for listening!