Multi-Screen Experiences?

What do you view as the single biggest shortcoming or challenge of the permission models deployed today?

• Many mainstream permissions UX patterns work the best on desktop browsers, since they were first invented in that era. Let's take Drag and Drop as the most obvious example. A DnD-based permissions UX should not be a normative requirement. There should be instead a higher-level permissions abstraction that enables different implementations (DnD, modal dialog, non-modal toast dialog ...).

Decomposing the Permissions UX in Multi-Screen Experiences for the Web

• Display Arrangement
  • OS
• Display Source Picker
  • getDisplayMedia()
  • HTMLMediaElement.captureStream()?
• Virtual Display Creation
  • OS
• Window Placement
  • OS
  • Multi-Screen Window Placement API?
    • (Virtual workspaces/desktops a non-goal currently)

Display Arrangement

Display Source Picker

(Chrome display source picker UI, https://www.webrtc-experiment.com/getDisplayMedia/)

Virtual Display Creation

(Deskreen Virtual Display Picker UI, https://deskreen.com/)

getDisplayMedia UI vs Deskreen Virtual Display UI

Window Placement

Implicit consent example: drag and drop

OSes have a variety of implicit consenting mechanisms

• Keyboard shortcuts in Windows: "Windows logo key + Shift + Left arrow or Right arrow to move an app or window in the desktop from one monitor to another."

• Drag and drop evolved in macOS: "Drag the app window you want to move to the edge of the screen [or] enter Mission Control, then drag the window up to the space you want to use."

Could such implicit permission grants work for web app window placement?

The traditional Web API approach: user's consent via browser prompt

OS-level: controls built into window manager

Browser prompt vs OS controls

Permissions challenge?

• Enable permissions UX innovation and protect the user's privacy

  • in a future-proof manner
  • also on non-desktop-browser contexts

• Things to do to get there

  • Does the permissions UX adapt to non-desktop-browser contexts? XR context? On a watch? TV with a remote? With voice control using a home assistant?
  • Define permissions abstractions that enable context-sensitive implementations and leave room for Quality of Implementation differentiation and UX innovation
  • Your idea goes here

Discussion