Designing Physical Experiences
The Web in the Age of Room-Scale Computing
MDN Roadshow - Kuala Lumpur, 2017
ZERO
* But are you innovating if you have zero users?
* or your startup fails?
* Google, half of US users install zero apps per month
* Many have app exhaustion, most use 7 per day
* For many people, apk size is a blocker
* App stores are a garbage fire
* It's a tough world out there for a baby unicorn
* right now room scale computing looks like this
"One download for every one thousand web views."
Ryan Sheffer,
Nobody Wants Your App.
Open or Install?
URL or App?
- easier to get someone to open a URL than to install an app
- instant content vs multiple steps
- ux and ix make the difference in access to information and services
* bluetooth
* good for multi-floor spaces
* configurable radio signal for distance
* google backported nearby to Android Kit Kat, 4.4
* get notification as users experience a space
* potential for notification storm, abuse
* mozilla festival
* 9 floor art school in london
* users find out what's going on around them
* leave with a digital archive of what they experienced
* instead of emails to self and whatnot
SPEECH
NOISE!
* a lot of talk about speech recognition
* but differences in the ambient decibel level are a signal
* you don't need fancy speech recognition to know if someone walked in the room
* or if someone is asleep vs awake
* but how do you know if there's a computer in the room?
* how do you know if something is listening?
* noise can be a signaller for presence/absence and whether to send notification, have the robot say "hi", etc
* proximity
* also explored privacy concerns
* signaling in public spaces
* but what to do once you've got people connected?
PATTERNS
& FLOW
* onboarding technology has led to new software inputs
* and requires new patterns and task flows in our designs
* now we've got an interaction flow going
* and with PWA we have things like push notifications
* so we can re-engage during app switching or whatever as someone moves through a space
➡️ 
➡️ URL
➡️ app
➡️
➡️ URL
➡️ 
➡️
➡️ URL
➡️ 
* complex topology in our interactions
* bluetooth mesh spec is coming soon
➡️
➡️ URL
➡️ ...
➡️ ???
➡️
➡️ URL
➡️ Payment Request API
➡️ 💰💰💰
PaymentRequest Example
var request = new PaymentRequest(
supportedPaymentMethods,
paymentDetails,
options
);
request.show();
➡️
➡️ URL
➡️ AR / VR
PROGRESSIVE
WEB APPS
(AKA PWA)
Homescreen Native
Offline & Sync
Push Notifications
getUserMedia()
navigator.mediaDevices.getUserMedia({
audio: true,
video: true
}).then(function (stream) {
// do something with the stream
})
A-Frame Example
<a-scene>
<a-sphere radius="1.25"></a-sphere>
<a-box width="1" height="1" depth="1"></a-box>
<a-cylinder radius="0.5" height="1.5"></a-cylinder>
<a-plane rotation="-90 0 0" width="4" height="4"></a-plane>
</a-scene>
INVISIBLE
COMPUTING
* too many apps
* not enough choice
* but we're moving into an era of invisible computing
* right now the apps are at least visible
* is this the roomscale endgame?
* it's not the future we want
* actually, that much choice would be great
* more often it looks like this
PRIVACY
SECURITY
* what does privacy look like in multi-entity topologies
* but what's really going on is this.
* this is the new web
* integrated multisensory physical+digital experiences
* bridging platforms
* frictionless physical interaction flows
* ubiquitous physical computing
* mobile
*
* the hardware isn't here yet. but it's not far off.
* native and proprietary tech is here to stay in some form.
* the web is ready to glue these things together
* to make really interesting things
* and these are things you can do *now*
*
* we need a responsive design for physical spaces
* we need a design language for physical interactions
* if we wait, amazon, google, apple will design this language for us
* using hardware to define the spaces we can design for
* but with the open web we have an opportunity now
* to get ready for these quickly coming technologies
* and fight the hegemony of the few big hardware providers
* and define the interactions we want to see.
Dietrich Ayala
@dietrich
