Discussion:
Towards 1.0
(too old to reply)
Kristian Høgsberg
2012-02-17 17:01:48 UTC
Permalink
Hello,

So with 0.85.0 Wayland and Weston released, it's a good time to figure
out what the plan is going forward. We had a two day workshop in
Brussels leading up to FOSDEM and we went through the Wayland protocol
to figure out what's missing and what to do about various loose ends.
We also worked our way through ICCCM, EWMH and Xdnd, to make sure we
have an answer for the various functionality defined there. I'll try
to summarize what we came up with below, but first a few practical
things.

We now have bugzilla set up in bugs.freedesktop.org. Nothing
surprising there, we have a wayland product and wayland and weston
components:

https://bugs.freedesktop.org/enter_bug.cgi?product=Wayland

Second, about branches, release plans and protocol/api breaking: we
start breaking things on master (both wayland and weston) now. The
master branch is going to break API and protocol and that's going to
make it much harder to track things. Of course, that's how it always
was (except for the brief freeze leading up to 0.85), but we do have a
lot of pent-up protocol changes that will start landing now.

The 0.85 branches are going to keep API and protocol stable, but I
don't see a problem in backporting bigger features like Pekkas
transformations or Jesses sprites. It may make sense for toolkits to
stick with the 0.85 branch for as long as they don't need features
from the new protocol work. For Gtk+, for example, we still have work
to do with dnd, client-side-decorations and popups and most of this
work can be done with 0.85.

I expect we'll take a few months to finish the protocol work and then
we'll start the 0.9x releases. From this point on, the protocol
should not change in backwards incompatible ways. The protocol
supports versioning and discovery of new interfaces, so we can extend
the protocol as we need, both as we move towards 1.0 and after 1.0.
Ideally we can incorporate the protocol changes pretty fast and then
sit on them for a while. Once we haven't made protocol changes in a
while, we should be in a good spot to start freezing it.

Anyway, for the immediate future, we have a lot of stuff that we need
to work on. I've tried to combine the TODO file from the wayland
repo, various notes of mine and the notes from FOSDEM. I'll update
and consolidate all this back into the TODO file.

- version/since attributes in the protocol xml. Let's us keep track
of when a request or event was added to an interface and we can check
this against the negotiated version at runtime.

- we need surface.enter/leave events to be emitted as the surface
enters and leaves outputs. This lets us track which output(s) a
surface is currently showing on, which affects how we render (subpixel
information, dpi, rotation). By using enter/leave events, a surface
can be on multiple output.

- We need rotation information in the output (multiples of 90
degress) and we'll need a way for a client to communicate that it has
rendered it's buffer according to the output rotation. The goal is to
be able to pageflip directly to the client buffer, and for that we
need the client to render correctly and the compositor needs to know
that it did.

- We need the fullscreen request. Pretty much what has already been
discussed on the list. We need to send a configure event to the
client as we go fullscreen, wait for the client to attach new buffer
before changing anything.

- Brief talk about multi-gpu. Not conclusive, but we have the option
of advertising multiple wl_drm objects, and let the client or EGL pick
which drm device to render to. Perhaps different arguments to
eglGetDisplay or maybe a eglCreateWindowSurface attribute.

- Atomicity. Currently a lot of the atomicity in Wayland relies on
how we batch up all requests in a protocol buffer and only flushes in
the "blockhandler" in the client. Concencus was that we need
something more reliable and explicit. The suggestion is that we make
surface.attach a synchronization point such that everything before
that is batched and applied atomically when the surface.attach request
comes in. For cases where we need atomicity beyond a surface.attach,
we can add an atomic grouping mechanism, that can group together
multiple surface.attach requests into a bigger atomic change. To be
researched a bit.

- We should make pointer sprites regular surfaces. Something like
input_device.set_sprite(surface). This also make client side animated
cursors simple/possible, since we now get a frame event that can drive
the animation.

- maybe try to make remote wayland actually happen, to see if there
is something in the protocol/architecute that makes it harder than it
should be.

- remove wl_buffer.damage. This is only used for wl_shm buffers and
is not a generic wl_buffer request. We move it to wl_shm, and we'll
have to figure out how to get swrast to call it.

- drop global coords in events. clients have no use for this.

- reconsider data types for coordinates in events. double, floats or
fixed point. Transformed and/or accelerated input generates sub-pixel
positions. 24.8 fixed point could work. Need to think about the
range of coordinates we need. Different from X problem, since we
don't have sub-windows, which is where X hits the 16 bit limitations.
Monitor and window sizes haven't yet out-grown the 16 bit range.

- Key events need a bit more work/thinking/redesign. As it is we need
a mechanism to change and synchronize keymaps, repeat rates. But as
we started talking, we decided that we needed to go back and research
what other systems do instead of just essentially copying the X model.
Sending out unicode events in addition to keycode events has a lot of
benefits (OSK can send out unicode events instead of fake keycode
events, apps become much simpler...) Move keymap handling and repeat
to the server? Needs more research.

- pointer axis events need modifiers (ctrl-scroll eg), but we either
need to send the modifier state with each axis/scroll event or send
keys down on pointer_focus and subsequent key events... or just key
events for modifier keys... or for the non-repeating subset?

- split pointer_focus into enter/leave event, split pointer events
into wl_pointer interface. split keyboard events into wl_keyboard
interface, touch events into wl_touch. Rename wl_input_device to
wl_seat. Add mechanism to let wl_seat announce whether it has
wl_pointer/wl_keyboard/wl_touch devices.

- add timestamp to touch_cancel, add touch id to touch_cancel (?)

- serial numbers. The wayland protocol, as X, uses timestamps to
match up certain requests with input events. The problem is that
sometimes an event happens that triggers a timestamped event. For
example, a surface goes away and a new surface receives a
pointer.enter event. These events are normally timestamped with the
evdev event timestamp, but in this case, we don't have a evdev
timestamp. So we have to go to gettimeofday (or clock_gettime()) and
then we don't know if it's coming from the same time source etc. And
we don't really need a real time timestamp, we just need a serial
number that encodes the order of events inside the server. So we need
to introduce a serial number mechanism (uint32_t, maintained in
libwayland-server.so) that we can use to order events, and have a look
at the events we send out and decide whether they need serial number
or timestamp or both. We still need real-time timestamps for actual
input device events (motion, buttons, keys, touch), to be able to
reason about double-click speed and movement speed. The serial number
will also give us a mechanism to key together events that are
"logically the same" such as a unicode event and a keycode event, or a
motion event and a relative event from a raw device.

- the output protocol needs to send all the ugly timing details for the modes.

- we need an input shape extension that lets the client set the input
shape of the window. We can start with just a region-lite api, that
lets applications specify the region as a set of rectangles. Should
we make a wl_region interface that encapsulates the region
building/updating api and then make a surface.set_input_region(region)
request? We'll need the same thing for opaqe region.

ICCCM

- clipboard manager interface? what's needed? just notification
that the selection has gone away. should the clipboard manager be
able to take over the selection "seamlessly", ie, with the same
timestamp etc? Doesn't seem like we need that, the clipboard will
have to set a new data_source anyway, with the subset of mimetypes it
offers (the clipboad manager may only offer a subset of the types
offered by the original data_source)

- mime-type guidelines for data_source (ie, both dnd and selection):
recommended types for text or images, types that a clipboard manager
must support, mime-types must be listed in preferred order

- TRANSIENT_FOR handled by wl_shell_surface, WM_CLASS replaced by
.desktop file filename (absolute path if non-standard location)
WM_CLASS used for grouping windows in one button in a panel, for
example. So we'll need a request to set that.

- WM_TAKE_FOCUS, a workaround for lack of timestamp in the focus
event, not a problem for us, but we may need a "no kb focus please"
mechanism. Or should this be implicit in a specific surface type?

- Session management is dbus now. The compsitor may talk to the dbus
session manager and communicate cookies to the session manager, that
allows the compositor to restore windows to their current state.

EWMH

- The EWMH standard has a lot of properties, window types and
language for standardizing out-of-process desktop components such as
pagers, panels and desktop backgrounds. We're not going to try to
standardize that in 1.0, the focus is going to be on the communication
between regular applications and the desktop environment. It
certainly possible to do with Wayland, and standardization could come
from different environments doing their own thing initially, and then
later try to consolidate. desktop-shell and tablet-shell in weston
are good examples of custom interfaces to split the desktop
environment into several processes.

- _NET_WM_NAME (shell_surface.set_title(utf8)), _NET_WM_ICON Is this
just another wl_surface? Do we need this if we have the .desktop file?
How to set multiple sizes?

- ping event, essentially the opposite of the display.sync request.
Compositor can ping clients to see if they're alive (typically when
sending input events to a client surface)

- Opaque region request. This is an optimization to let the
compositor skip repainting content below an opaque area. Share the
region interface with input region?

- WM_USER_TIME is needed by a WM to implement focus stealing
prevention since it can't know when a random window last received
input events. With the wm in the display server, this is not a
problem.

- Don't need wl_surface pid, we know the client and can get creds
from the socket (should implement this).

- move to workspace, keep on top, on all workspaces, minimize etc
requests for implementing client side window menu? or just make a
"show window menu" request?

- window move and resize functionality for kb and touch.

- dnd loose ends: drag icon; self-dnd: initiate dnd with a null
data-source, compositor will not offer to other clients, client has to
know internally what's offered and how to transfer data. no fd
passing.

- Protocol for specifying title bar rectangle (for moving
unresponsive apps). Rectangle for close button, so we can popup
force-close dialog if application doesn't respond to ping event when
user clicks there. We could use the region mechanism here too.

- popup placement protocol logic.

- subsurface mechanism. we need this for cases where we would use an
X subwindow for gl or video other different visual type.

- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces, rename wl_input_device to wl_seat. wl_seat has
zero or one of each, and will announce this at bind time. Raw devices
are also tied to a wl_seat, but we'll not do that for 1.0, we just
need to make sure wl_seat has a forward compatible way to announce
them.

EGL/gbm

- Land Anders gbm_surface patches.

- Don't wl_display_iterate in eglSwapBuffer, send an eventfd fd?

- Land Robert Braggs EGL extensions: start frame, frame age, swap with damage

- Make it possible to share buffers from compositor to clients.
Tricky part here is how to indicate to EGL on the server side that it
should make an EGLImage available to a client. We'll need a "create a
wl_buffer for this EGLImage for this client" kind of entry point.

- gbm get_format patch, Jesses sprite patches.

Weston

- dpms/backlight control. we need this to be a real display server.

- Framebased input event delivery. Only process input events at the
beginning of new frames instead of as they come in.

Anyway, that's the big list. There's no particular order to it and
there are huge tasks and trivial stuff in there, but it's a fairly
complete list. We have a lot of patches sitting on the mailing list
and I'm starting to review and merge them now, which should address a
lot of the issues here. If I miss something, feel free to resend that
patches and ideal also put the branch somewhere where I can just pull
it.

thanks,
Kristian
Pier Luigi
2012-02-17 21:35:37 UTC
Permalink
?- The EWMH standard has a lot of properties, window types and
language for standardizing out-of-process desktop components such as
pagers, panels and desktop backgrounds. ?We're not going to try to
standardize that in 1.0, the focus is going to be on the communication
between regular applications and the desktop environment. ?It
certainly possible to do with Wayland, and standardization could come
from different environments doing their own thing initially, and then
later try to consolidate. ?desktop-shell and tablet-shell in weston
are good examples of custom interfaces to split the desktop
environment into several processes.
Please standardize window types in Wayland 1.0, in order to avoid confusion.
With toolkits defining their own way to handle pagers, panels, etc...
what will happen with heterogeneous (let's say GTK+ in a
QtWayland-based compositor)?

I would also like to know what's the plan for a feature like X struts.
--
"Don't let the noise?of other's opinions drown out your own inner
voice." (Steve Jobs)
Chase Douglas
2012-02-20 17:23:28 UTC
Permalink
Post by Kristian Høgsberg
- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces
[snip]

So the client window will receive touch events without delay, but may
receive a cancel? I am in favor of this approach, but you need to add a
way to tell the window when the window manager has "rejected" a touch
sequence as well. Otherwise the client will never know when they can
perform destructive operations with it.

-- Chase
Kristian Høgsberg
2012-02-21 20:16:01 UTC
Permalink
Post by Chase Douglas
?- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces
[snip]
So the client window will receive touch events without delay, but may
receive a cancel? I am in favor of this approach, but you need to add a
way to tell the window when the window manager has "rejected" a touch
sequence as well. Otherwise the client will never know when they can
perform destructive operations with it.
No, we don't need that. Don't do destructive operations on something
that could be the first half on a globall gesture. If you need to
know for sure that nobody is going to take over the events, wait until
the touch session is over (all touch points up, none cancelled).

Kristian
Chase Douglas
2012-02-21 20:49:26 UTC
Permalink
Post by Kristian Høgsberg
Post by Chase Douglas
Post by Kristian Høgsberg
- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces
[snip]
So the client window will receive touch events without delay, but may
receive a cancel? I am in favor of this approach, but you need to add a
way to tell the window when the window manager has "rejected" a touch
sequence as well. Otherwise the client will never know when they can
perform destructive operations with it.
No, we don't need that. Don't do destructive operations on something
that could be the first half on a globall gesture. If you need to
know for sure that nobody is going to take over the events, wait until
the touch session is over (all touch points up, none cancelled).
That doesn't make any sense. What if I'm playing a game? Latency
matters, and I need to know that the touches are mine early on. In the
case of a game, the environment should leave touches alone and
immediately tell the game that the touches are owned by it.

Touches are used for much more than just button tapping, and waiting
until a touch is lifted to do anything won't work.

Another reason this won't work is if your environment wants to recognize
a double-tap sequence. The first tap will end, at which point the
wayland application will attempt to use it. But a second tap comes along
and the environment performs an action.

-- Chase
Thiago Macieira
2012-02-21 21:21:55 UTC
Permalink
Post by Chase Douglas
That doesn't make any sense. What if I'm playing a game? Latency
matters, and I need to know that the touches are mine early on. In the
case of a game, the environment should leave touches alone and
immediately tell the game that the touches are owned by it.
Touches are used for much more than just button tapping, and waiting
until a touch is lifted to do anything won't work.
Another reason this won't work is if your environment wants to recognize
a double-tap sequence. The first tap will end, at which point the
wayland application will attempt to use it. But a second tap comes along
and the environment performs an action.
There's one important difference with X here: the compositor can simply delay
sending the events if it wants to. That's the case of the double-tap above.

If latency matters, the actions cannot be delayed, not even to wait for
"you're not going to get cancelled from this point on".

You can also think that, in the Wayland world, the "cannot be cancelled" event
matches the "touch end" event. That constrains what the UX can do, sure.

If there is a real UX which cannot be met by this constraint, let us know.
It's been really hard to get UX requirements from anyone...
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
Intel Sweden AB - Registration Number: 556189-6027
Knarrarn?sgatan 15, 164 40 Kista, Stockholm, Sweden
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20120221/33f2b984/attachment-0001.pgp>
Kristian Høgsberg
2012-02-21 21:25:39 UTC
Permalink
Post by Chase Douglas
Post by Chase Douglas
?- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces
[snip]
So the client window will receive touch events without delay, but may
receive a cancel? I am in favor of this approach, but you need to add a
way to tell the window when the window manager has "rejected" a touch
sequence as well. Otherwise the client will never know when they can
perform destructive operations with it.
No, we don't need that. ?Don't do destructive operations on something
that could be the first half on a globall gesture. ?If you need to
know for sure that nobody is going to take over the events, wait until
the touch session is over (all touch points up, none cancelled).
That doesn't make any sense. What if I'm playing a game? Latency
matters, and I need to know that the touches are mine early on. In the
case of a game, the environment should leave touches alone and
immediately tell the game that the touches are owned by it.
I didn't say that applications always have to wait for touch end.
Just that if you're doing something irrevocable like, sending an email
or formatting your floppy, you don't want to trigger on something that
could be part of a global gesture. Like touch down, then move. You
need to wait for touch end in that case.

On the other hand, if you're scrolling in your web browser or moving
around in a game, you just assume you own the events you get and
scroll/move around. If later in that touch session, you get the
cancel event, you can just leave the web page/game where you
scrolled/moved to. Or you can undo what you did, for example, if you
scribbled in a paint app as part of the global gesture. Because you
have to be able to do that anyway.
Post by Chase Douglas
Touches are used for much more than just button tapping, and waiting
until a touch is lifted to do anything won't work.
Another reason this won't work is if your environment wants to recognize
a double-tap sequence. The first tap will end, at which point the
wayland application will attempt to use it. But a second tap comes along
and the environment performs an action.
This doesn't seem like a valid use case.

Kristian
Chase Douglas
2012-02-22 04:39:24 UTC
Permalink
Post by Kristian Høgsberg
Post by Chase Douglas
Post by Kristian Høgsberg
Post by Chase Douglas
Post by Kristian Høgsberg
- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces
[snip]
So the client window will receive touch events without delay, but may
receive a cancel? I am in favor of this approach, but you need to add a
way to tell the window when the window manager has "rejected" a touch
sequence as well. Otherwise the client will never know when they can
perform destructive operations with it.
No, we don't need that. Don't do destructive operations on something
that could be the first half on a globall gesture. If you need to
know for sure that nobody is going to take over the events, wait until
the touch session is over (all touch points up, none cancelled).
That doesn't make any sense. What if I'm playing a game? Latency
matters, and I need to know that the touches are mine early on. In the
case of a game, the environment should leave touches alone and
immediately tell the game that the touches are owned by it.
I didn't say that applications always have to wait for touch end.
Just that if you're doing something irrevocable like, sending an email
or formatting your floppy, you don't want to trigger on something that
could be part of a global gesture. Like touch down, then move. You
need to wait for touch end in that case.
On the other hand, if you're scrolling in your web browser or moving
around in a game, you just assume you own the events you get and
scroll/move around. If later in that touch session, you get the
cancel event, you can just leave the web page/game where you
scrolled/moved to. Or you can undo what you did, for example, if you
scribbled in a paint app as part of the global gesture. Because you
have to be able to do that anyway.
I don't see how this resolves the game use case. Usually in a game,
every action is destructive from the start. The application will either
have to assume that wayland won't do something with the touches, or will
have to wait until the end of a touch, which just isn't feasible.
Post by Kristian Høgsberg
Post by Chase Douglas
Touches are used for much more than just button tapping, and waiting
until a touch is lifted to do anything won't work.
Another reason this won't work is if your environment wants to recognize
a double-tap sequence. The first tap will end, at which point the
wayland application will attempt to use it. But a second tap comes along
and the environment performs an action.
This doesn't seem like a valid use case.
Why not? I can easily envision a four-touch double tap being an
environment gesture.

I can come up with many more use cases if you need. There is no way this
will work without a concept of touch ownership that is passed on to the
client applications.

-- Chase
Peter Hutterer
2012-02-23 05:41:07 UTC
Permalink
Post by Kristian Høgsberg
Post by Chase Douglas
Post by Chase Douglas
?- input protocol restructuring: break up events into wl_pointer
(enter/leave/motion/button/axis events, set_pointer_surface request),
wl_keyboard (enter/leave/key events... what else... unicode event,
set_map request? pending kb work), and wl_touch (down/up/motion/cancel
events) interfaces
[snip]
So the client window will receive touch events without delay, but may
receive a cancel? I am in favor of this approach, but you need to add a
way to tell the window when the window manager has "rejected" a touch
sequence as well. Otherwise the client will never know when they can
perform destructive operations with it.
No, we don't need that. ?Don't do destructive operations on something
that could be the first half on a globall gesture. ?If you need to
know for sure that nobody is going to take over the events, wait until
the touch session is over (all touch points up, none cancelled).
That doesn't make any sense. What if I'm playing a game? Latency
matters, and I need to know that the touches are mine early on. In the
case of a game, the environment should leave touches alone and
immediately tell the game that the touches are owned by it.
I didn't say that applications always have to wait for touch end.
Just that if you're doing something irrevocable like, sending an email
or formatting your floppy, you don't want to trigger on something that
could be part of a global gesture. Like touch down, then move. You
need to wait for touch end in that case.
On the other hand, if you're scrolling in your web browser or moving
around in a game, you just assume you own the events you get and
scroll/move around. If later in that touch session, you get the
cancel event, you can just leave the web page/game where you
scrolled/moved to. Or you can undo what you did, for example, if you
scribbled in a paint app as part of the global gesture. Because you
have to be able to do that anyway.
Post by Chase Douglas
Touches are used for much more than just button tapping, and waiting
until a touch is lifted to do anything won't work.
Another reason this won't work is if your environment wants to recognize
a double-tap sequence. The first tap will end, at which point the
wayland application will attempt to use it. But a second tap comes along
and the environment performs an action.
This doesn't seem like a valid use case.
Quote from above "you need to wait for a touch end in that case".

How will a client know what touch sequence _could_ be a global gesture? How
long after the touch end will I need to wait to make sure this wasn't a
gesture used by the environment?

Surface-local coordinates and the fat-finger problem may cause a double-tap
to happen in two different surfaces. Can the compositor cancel completed
sequences?

Cheers,
Peter
Bill Spitzak
2012-02-22 00:13:38 UTC
Permalink
It seems like it would be better if clients got the touch events first,
and the compositor only did things if the client said it was
uninterested in the events. For a lot of touch events this can be
decided immediately on the touch-down, since the client knows the user
is pushing nothing.

I don't know much about touch events, but this seems similar to global
shortcuts. It would help a lot if every keystroke was sent to clients
but the clients could say "I don't do anything with that" and then the
compositor/shell could use the keystroke as a global shortcut, and in
fact I thought this was what Wayland was going to do. It seems the same
ideas should apply to other events like touch.
Daniel Stone
2012-02-22 00:21:29 UTC
Permalink
Hi,
It seems like it would be better if clients got the touch events first, and
the compositor only did things if the client said it was uninterested in the
events. For a lot of touch events this can be decided immediately on the
touch-down, since the client knows the user is pushing nothing.
No, this just isn't true. You want global gestures (e.g. three finger
swipe to change desktop) to take precedence over local shortcuts. And
you cannot -- cannot -- know at touch-down time who's going to be
ultimately interested in it. Because for a three-finger swipe, there
will be a time when there's only one finger down, or two. At which
point the client will say, 'ah yes, I'm interested in this!'. And
then the third finger lands and you regret a terrible design decision
you made.

Cheers,
Daniel
Bill Spitzak
2012-02-22 05:15:57 UTC
Permalink
Post by Daniel Stone
Hi,
It seems like it would be better if clients got the touch events first, and
the compositor only did things if the client said it was uninterested in the
events. For a lot of touch events this can be decided immediately on the
touch-down, since the client knows the user is pushing nothing.
No, this just isn't true. You want global gestures (e.g. three finger
swipe to change desktop) to take precedence over local shortcuts. And
you cannot -- cannot -- know at touch-down time who's going to be
ultimately interested in it. Because for a three-finger swipe, there
will be a time when there's only one finger down, or two. At which
point the client will say, 'ah yes, I'm interested in this!'. And
then the third finger lands and you regret a terrible design decision
you made.
I would think the client could say "not interested" when it sees the
third finger.

If the clients can look at things first, this would allow the compositor
to do things like "one finger can be used to change desktops if the
underlying program does not use it".

Solutions like "three fingers are needed" are just like the solutions
for shortcuts where you have to hold Alt and Ctrl and the right-hand
Shift and push the key, in an attempt to not collide with keystrokes the
program wants.
Chase Douglas
2012-02-22 23:18:39 UTC
Permalink
Post by Bill Spitzak
Post by Daniel Stone
Hi,
It seems like it would be better if clients got the touch events first, and
the compositor only did things if the client said it was uninterested in the
events. For a lot of touch events this can be decided immediately on the
touch-down, since the client knows the user is pushing nothing.
No, this just isn't true. You want global gestures (e.g. three finger
swipe to change desktop) to take precedence over local shortcuts. And
you cannot -- cannot -- know at touch-down time who's going to be
ultimately interested in it. Because for a three-finger swipe, there
will be a time when there's only one finger down, or two. At which
point the client will say, 'ah yes, I'm interested in this!'. And
then the third finger lands and you regret a terrible design decision
you made.
I would think the client could say "not interested" when it sees the
third finger.
The client won't see the third finger if it touches outside its window.
In the wayland case, only the WM has all the info needed to determine if
a touch is part of a global gesture. The WM needs to make the decision,
not the client.
Post by Bill Spitzak
If the clients can look at things first, this would allow the compositor
to do things like "one finger can be used to change desktops if the
underlying program does not use it".
That would be bad UI design because then global gestures would fire only
sometimes. Further, it would break global gestures if touches occur over
a broken application.
Post by Bill Spitzak
Solutions like "three fingers are needed" are just like the solutions
for shortcuts where you have to hold Alt and Ctrl and the right-hand
Shift and push the key, in an attempt to not collide with keystrokes the
program wants.
Yes, system and application gestures may collide. There's not much one
can do about it other than provide guidelines for application
developers. Unity's guidelines are: three and four finger gestures are
reserved for global uses. Everything else is fair game.

I think we can look at other OSes as case studies. iOS and OS X employ
effective global gestures imo, and they take precedence over the
application receiving touch or gesture events.

-- Chase
Bill Spitzak
2012-02-23 20:22:19 UTC
Permalink
Post by Chase Douglas
The client won't see the third finger if it touches outside its window.
In the wayland case, only the WM has all the info needed to determine if
a touch is part of a global gesture. The WM needs to make the decision,
not the client.
I'm pretty certain all touch events *MUST* go to the same surface until
all touches are released. Otherwise it will be quite impossible to do
gestures reliably, like the user could not do them to objects near the
edge of a window.
Post by Chase Douglas
Post by Bill Spitzak
If the clients can look at things first, this would allow the compositor
to do things like "one finger can be used to change desktops if the
underlying program does not use it".
That would be bad UI design because then global gestures would fire only
sometimes. Further, it would break global gestures if touches occur over
a broken application.
I consider it bad design that global actions are "complex" (like needing
3 fingers) or global shortcuts require lots of shift keys held down,
just to avoid collisions with applications.

I also think you are making up "user confusion" that does not exist in
the real world to make an excuse for this. Users will find it pretty
obvious if the same action that scrolls a document also scrolls the
entire screen if you don't touch a document, IMHO.
Post by Chase Douglas
I think we can look at other OSes as case studies. iOS and OS X employ
effective global gestures imo, and they take precedence over the
application receiving touch or gesture events.
I think it is pretty clear that "what other OS's do" is not always what
Wayland wants to do. Most of them copied ideas from X.
Chase Douglas
2012-02-23 21:19:28 UTC
Permalink
Post by Bill Spitzak
Post by Chase Douglas
The client won't see the third finger if it touches outside its window.
In the wayland case, only the WM has all the info needed to determine if
a touch is part of a global gesture. The WM needs to make the decision,
not the client.
I'm pretty certain all touch events *MUST* go to the same surface until
all touches are released. Otherwise it will be quite impossible to do
gestures reliably, like the user could not do them to objects near the
edge of a window.
No? I'm not sure what to say other than this is plainly incorrect. For a
touchscreen device, the user may be interacting with two separate
applications at the same time. Imagine you have your web browser open on
the left side of the screen, and you have your spreadsheet open on the
right side. You then want to scroll both side by side as you compare
numbers. To do this, you need to send touch events to each client
separately.

I'm fairly certain that no window system in major use behaves as you
suggest.
Post by Bill Spitzak
Post by Chase Douglas
Post by Bill Spitzak
If the clients can look at things first, this would allow the compositor
to do things like "one finger can be used to change desktops if the
underlying program does not use it".
That would be bad UI design because then global gestures would fire only
sometimes. Further, it would break global gestures if touches occur over
a broken application.
I consider it bad design that global actions are "complex" (like needing
3 fingers) or global shortcuts require lots of shift keys held down,
just to avoid collisions with applications.
My belief is that it is considered, universally (meaning >95% of
people), that global gestures should behave consistently across
applications, and should not be inhibited by broken applications.

I also believe that 3 or more finger global gestures are not bad design,
especially since the iPad uses them. Maybe individuals like yourself
don't like them, but very many do.

I'm not trying to start an argument about what is the best design of
global gestures. However, I think it is a requirement that the window
system allows for consistent global gesture behavior, and that three or
more touch gestures are valid for global gestures.
Post by Bill Spitzak
I also think you are making up "user confusion" that does not exist in
the real world to make an excuse for this. Users will find it pretty
obvious if the same action that scrolls a document also scrolls the
entire screen if you don't touch a document, IMHO.
How would it not be confusing if you have a global "alt-tab" gesture
work when performed over your spreadsheet but not your browser?
Post by Bill Spitzak
Post by Chase Douglas
I think we can look at other OSes as case studies. iOS and OS X employ
effective global gestures imo, and they take precedence over the
application receiving touch or gesture events.
I think it is pretty clear that "what other OS's do" is not always what
Wayland wants to do. Most of them copied ideas from X.
That's why I said we should look at them as case studies...

-- Chase
Daniel Stone
2012-02-24 00:04:55 UTC
Permalink
Hi,
Post by Chase Douglas
The client won't see the third finger if it touches outside its window.
In the wayland case, only the WM has all the info needed to determine if
a touch is part of a global gesture. The WM needs to make the decision,
not the client.
I'm pretty certain all touch events *MUST* go to the same surface until all
touches are released. Otherwise it will be quite impossible to do gestures
reliably, like the user could not do them to objects near the edge of a
window.
On touchpads, yes. On touchscreens, no.
Post by Chase Douglas
That would be bad UI design because then global gestures would fire only
sometimes. Further, it would break global gestures if touches occur over
a broken application.
I consider it bad design that global actions are "complex" (like needing 3
fingers) or global shortcuts require lots of shift keys held down, just to
avoid collisions with applications.
Needing three fingers isn't complex. My grandparents can understand
it, and regularly use three-finger gestures.
Post by Chase Douglas
I think we can look at other OSes as case studies. iOS and OS X employ
effective global gestures imo, and they take precedence over the
application receiving touch or gesture events.
I think it is pretty clear that "what other OS's do" is not always what
Wayland wants to do. Most of them copied ideas from X.
Not really. X's input system is pretty unique (and pretty uniquely
broken), and I think it's safe to say that no-one -- least of all iOS
-- has copied it.

Cheers,
Daniel

Mohamed Ikbel Boulabiar
2012-02-22 08:22:49 UTC
Permalink
?Because for a three-finger swipe, there
will be a time when there's only one finger down, or two. ?At which
point the client will say, 'ah yes, I'm interested in this!'. ?And
then the third finger lands and you regret a terrible design decision
you made.
Some values about the time between fingers touching the surface (in
mt) can help designing the interaction. I don't have numbers now but
what I remember is that they are very small values (20ms-50ms) and
thresholded to devices timestamps.
Time between double-click using a mouse is 500ms. (which can be
configured later by the user)
And in terms of interactions, the maximum time between an input and a
feedback should be less than 100ms, otherwise the user will detect a
delay.

Using these values, it's not possible to model things using usual FSM ?

i
Kristian Hoegsberg
2012-02-22 18:16:12 UTC
Permalink
Post by Kristian Høgsberg
ICCCM
- clipboard manager interface? what's needed? just notification
that the selection has gone away. should the clipboard manager be
able to take over the selection "seamlessly", ie, with the same
timestamp etc? Doesn't seem like we need that, the clipboard will
have to set a new data_source anyway, with the subset of mimetypes it
offers (the clipboad manager may only offer a subset of the types
offered by the original data_source)
Clipboard managers in X interact badly with programs that copy sensitive
information to the clipboard (e.g. KeePassX). As long as there will be
a new spec, should it have a way for the source of a clipboard selection
to indicate that the contents are sensitive and should not be saved at
all or shown on the screen unless requested?
The clipboard will not be a secure communication channel. If you're
concerned about how your passwords are transmitted, use a different
mechanism.

Kristian
Continue reading on narkive:
Loading...