Touching the Future
The Web in the Age of Physical Computing
HKOSCON - Hong Kong, 2017
* why does mozilla want to talk about smart cities, at an event like this?
* you're making decisions now about privacy, security and what technologies to use
* and we want to see you succeed
* we want to see your innovations work for *everyone*
* the rooms are the computers
* walk into a room and it can react
* can say what's going on that day
* right now the concept is owned by amazon alexa, google home, siri coming this fall
* but when you can build these with commodity hardware, the cheap ones aren't far off
* and why do they even have to be visible really, as long as there's feedback
*
* when every *thing* is a computer or a bot, we can't have apps for all the things
* the champions of iot and bots haven't solved this
* we need new models of user engagement for physical technologies
*
* what's the role of a browser in a room-scale physical computing world?
* nobody knows yet
* computers ate into tv time
* smartphones ate into computer time
* maybe we divide time yet again
* right now room scale computing looks like this
* actually, that much choice would be great
* more often it looks like this
INVISIBLE
COMPUTING
* too many apps
* not enough choice
* but we're moving into an era of invisible computing
* right now the apps are at least visible
* is this the roomscale endgame?
* it's not the future we want
* so, the rooms are the computers
* but not every room will have a single "controller"
* won't be configured with your account
* how can we create seamless bridges to digital experiences from physical interactions?
ACTIVE
ONBOARDING
* onboarding sounds terrible, like the first day at a new job
* active onboarding methods require the user the initiate the digital experience
PASSIVE
ONBOARDING
* like not showing up for the first day on the job?!
* bluetooth
* good for multi-floor spaces
* configurable radio signal for distance
* google backported nearby to Android Kit Kat, 4.4
* get notification as users experience a space
* potential for notification storm, abuse
* mozilla festival
* 9 floor art school in london
* users find out what's going on around them
* leave with a digital archive of what they experienced
* instead of emails to self and whatnot
SPEECH
NOISE!
* a lot of talk about speech recognition
* but differences in the ambient decibel level are a signal
* you don't need fancy speech recognition to know if someone walked in the room
* or if someone is asleep vs awake
* but how do you know if there's a computer in the room?
* how do you know if something is listening?
* noise can be a signaller for presence/absence and whether to send notification, have the robot say "hi", etc
* proximity
* also explored privacy concerns
* signaling in public spaces
* but what to do once you've got people connected?
PATTERNS
& FLOW
* onboarding technology has led to new software inputs
* and requires new patterns and task flows in our designs
* now we've got an interaction flow going
* and with PWA we have things like push notifications
* so we can re-engage during app switching or whatever as someone moves through a space
➡️ 
➡️ URL
➡️ app
➡️
➡️ URL
➡️ 
➡️
➡️ URL
➡️ 
* complex topology in our interactions
* bluetooth mesh spec is coming soon
➡️
➡️ URL
➡️ ...
➡️ ???
➡️
➡️ URL
➡️ Payment Request API
➡️ 💰💰💰
CAMERA +
JAVASCRIPT +
3D +
NOISE
getUserMedia()
navigator.mediaDevices.getUserMedia({
audio: true,
video: true
}).then(function (stream) {
// do something with the stream
})
A-Frame Example
<a-scene>
<a-sphere radius="1.25"></a-sphere>
<a-box width="1" height="1" depth="1"></a-box>
<a-cylinder radius="0.5" height="1.5"></a-cylinder>
<a-plane rotation="-90 0 0" width="4" height="4"></a-plane>
</a-scene>
PRIVACY
SECURITY
* what does privacy look like in multi-entity topologies
* but what's really going on is this.
* this is the new web
* integrated multisensory physical+digital experiences
* bridging platforms
* frictionless physical interaction flows
* ubiquitous physical computing
* mobile
*
* the hardware isn't here yet. but it's not far off.
* native and proprietary tech is here to stay in some form.
* the web is ready to glue these things together
* to make really interesting things
* and these are things you can do *now*
*
* we need a responsive design for physical spaces
* we need a design language for physical interactions
* if we wait, amazon, google, apple will design this language for us
* using hardware to define the spaces we can design for
* but with the open web we have an opportunity now
* to get ready for these quickly coming technologies
* and fight the hegemony of the few big hardware providers
* and define the interactions we want to see.
Dietrich Ayala
@dietrich
