Discussion:
[PATCH] virtual-keyboard: Add new virtual keyboard protocol
(too old to reply)
Silvan Jegen
2018-05-18 18:31:08 UTC
Permalink
Hi

Just a typo I found below.
Provides the ability to emulate keyboards by
applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials,
and added seat binding.
---
This proposal is another one needed by Purism to support on screen
keyboards on a phone screen.
Virtual-keyboard is a protocol to let applications send arbitrary
keyboard events. The need for this protocol comes from the fact
that the input-method protocol combines two separate input
responsibilities. It currently deals both with text input and raw
keyboard events. I hope to split input-method along this line into
virtual-keyboard and the rest of input-method. I'm going to submit the
updated input-method for review soon.
Applications should be able to control both interfaces at the same
time. A screen keyboard supporting autocorrect (input-method) still
wants to send arrow keys (vityual-keyboard) correctly. Because of
this, both kinds of events at minimum must be sent to the same seat. I
made the seat binding explicitly done by the application, taking
inspiration from text-input protocol, which assumes per-seat binding
as well.
Input welcome.
Cheers,
Dorota
Makefile.am | 1 +
.../virtual-keyboard-unstable-v1.xml | 97 ++++++++++++++++++++++
2 files changed, 98 insertions(+)
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..51a47a2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
There is a 'u' missing in the beginning of the file location.


Cheers,

Silvan
R***@sony.com
2018-05-18 19:23:15 UTC
Permalink
Hi Dorota,

Thanks for sharing your proposal. Ourselves we have interest in this
kind of capability as well. Looking at our own use cases, I wonder if
this proposal goes far enough.

We are basically interested in the ability to inject keyboard, mouse,
touch (and gamepad). On regular X some of that worked through XTest
protocol. From my perspective any virtual keyboard proposal, would make
sense to be flexible enough to handle other input devices as well.

On the other hand some may bring up, why virtual device support should
be in Wayland as input devices can also be spoofed through uinput.

Thanks,

Roderick Colenbrander
Sr Manager Hardware & Systems Engineering
Sony Interactive Entertainment
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
Input welcome.
Cheers,
Dorota
Makefile.am | 1 +
.../virtual-keyboard-unstable-v1.xml | 97 ++++++++++++++++++++++
2 files changed, 98 insertions(+)
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..51a47a2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..18130e2
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,97 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+ </interface>
+</protocol>
Dorota Czaplejewicz
2018-05-21 12:26:07 UTC
Permalink
Hello Roderick,

On Fri, 18 May 2018 19:23:15 +0000
Post by R***@sony.com
Hi Dorota,
Thanks for sharing your proposal. Ourselves we have interest in this
kind of capability as well. Looking at our own use cases, I wonder if
this proposal goes far enough.
We are basically interested in the ability to inject keyboard, mouse,
touch (and gamepad). On regular X some of that worked through XTest
protocol. From my perspective any virtual keyboard proposal, would make
sense to be flexible enough to handle other input devices as well.
I thought about adding mouse or game controller support in the same protocol, but I came to the conclusion that they would be better off as dedicated protocols. We already have an input method protocol, which can be considered an input device, then the separate keyboard protocol already, and using a dedicated one for any other type of device would follow this trend.

The main reason to keep separate, however, is to keep it simple to develop. Personally, I don't have much of an idea about approaching a mouse or gamepad protocol, because they are not in the scope of what I'm doing. That puts a damper on my efforts regarding virtual keyboard. In the same way, I expect protocol designers of the future to only have experience in a subset of devices, and they will probably want to avoid worrying about possible changes to other devices when they update what they care about.
Post by R***@sony.com
On the other hand some may bring up, why virtual device support should
be in Wayland as input devices can also be spoofed through uinput.
Wayland can still be a good place for these kinds of protocols, due to how it handles seat assignment. I would not be able to ensure that an input method and a virtual keyboard is assigned the same seat if I tried to emulate the keyboard in a lower layer. I suspect that this is not the only possible issue in this area. The effect of a single standard interface is also important – applications don't have to rely on system-specific libraries any more, and make things like nested sessions much more palatable.
Post by R***@sony.com
Thanks,
Roderick Colenbrander
Sr Manager Hardware & Systems Engineering
Sony Interactive Entertainment
Have a nice day,
Dorota Czaplejewicz
Post by R***@sony.com
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
Input welcome.
Cheers,
Dorota
Makefile.am | 1 +
.../virtual-keyboard-unstable-v1.xml | 97 ++++++++++++++++++++++
2 files changed, 98 insertions(+)
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..51a47a2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..18130e2
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,97 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian HÞgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+ </interface>
+</protocol>
Markus Ongyerth
2018-05-21 13:09:06 UTC
Permalink
Post by R***@sony.com
Hi Dorota,
Thanks for sharing your proposal. Ourselves we have interest in this
kind of capability as well. Looking at our own use cases, I wonder if
this proposal goes far enough.
We are basically interested in the ability to inject keyboard, mouse,
touch (and gamepad). On regular X some of that worked through XTest
protocol. From my perspective any virtual keyboard proposal, would make
sense to be flexible enough to handle other input devices as well.
All of these devices are different enough, that they need a dedicated API in
some way (keys/analog axis/relative movements). This could be implemented as
different interfaces in the same protocol, or separate protocols.
IMO splitting it up will be nicer, so compositors and clients can update the
versions they support indipendently, and we aren't forced to bump a version on
virtual keyboards to add to virtual gamepads.
Post by R***@sony.com
On the other hand some may bring up, why virtual device support should
be in Wayland as input devices can also be spoofed through uinput.
Doing this as a wayland protocol has the (IMO huge) advantage that it doesn't
require root permissions, and can be well confined.
Adding an input device into a test session with uinput sounds like a hassle as
well, since the user session will pick it up as well by default. As wayland
protocol it can just be spawned into the correct session.

Cheers,
ongy
Post by R***@sony.com
Thanks,
Roderick Colenbrander
Sr Manager Hardware & Systems Engineering
Sony Interactive Entertainment
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This proposal is another one needed by Purism to support on screen keyboards on a phone screen.
Virtual-keyboard is a protocol to let applications send arbitrary keyboard events. The need for this protocol comes from the fact that the input-method protocol combines two separate input responsibilities. It currently deals both with text input and raw keyboard events. I hope to split input-method along this line into virtual-keyboard and the rest of input-method. I'm going to submit the updated input-method for review soon.
Applications should be able to control both interfaces at the same time. A screen keyboard supporting autocorrect (input-method) still wants to send arrow keys (vityual-keyboard) correctly. Because of this, both kinds of events at minimum must be sent to the same seat. I made the seat binding explicitly done by the application, taking inspiration from text-input protocol, which assumes per-seat binding as well.
Input welcome.
Cheers,
Dorota
Makefile.am | 1 +
.../virtual-keyboard-unstable-v1.xml | 97 ++++++++++++++++++++++
2 files changed, 98 insertions(+)
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..51a47a2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ nstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..18130e2
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,97 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian HÞgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="keymap_format" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="key_state" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+ </interface>
+</protocol>
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Simon Ser
2018-05-21 07:19:03 UTC
Permalink
Apart from the typo that Silvan spotted, I have also encountered the issue where
the .c/.h generator complained about undefined key_state and keymap_format enums
which are defined in wayland.xml. I'm not sure what the correct way to solve
this - should they be copied into this protocol?
This is a bug in wayland-scanner, see [1]. Fixing wayland-scanner has been on my
TODO-list for a while, but if you want to do it, go ahead. A (less than ideal)
workaround is to remove the "enum" attribute from the <arg> for now.

[1]: https://lists.freedesktop.org/archives/wayland-devel/2018-April/037960.html
Dorota Czaplejewicz
2018-05-24 18:27:29 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.

The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.

Changes done:
- fixed typos
- enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
- added errors

The test client is at https://code.puri.sm/Librem5/weston/src/virtual_keyboard - please use the weston-keyboard program:

./configure --enable-clients
./weston-keyboard

A working wlroots implementation is available here:

https://github.com/swaywm/wlroots/pull/999

Thanks for feedback,
Dorota Czaplejewicz


Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 113 +++++++++++++++++++++
3 files changed, 116 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml

diff --git a/Makefile.am b/Makefile.am
index 4b9a901..fcd4572 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)

stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..df4d01c
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,113 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+ Software is furnished to do so, subject to the following conditions:
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+
+ Format carries a value from the keymap_format enumeration.
+ </description>
+ <arg name="format" type="uint" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+
+ State carries a value from the key_state enumeration.
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
+ </enum>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+ </interface>
+</protocol>
--
2.14.3
Simon Ser
2018-05-29 17:42:21 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.
- fixed typos
- enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
- added errors
./configure --enable-clients
./weston-keyboard
https://github.com/swaywm/wlroots/pull/999
Thanks for feedback,
Dorota Czaplejewicz
Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 113 +++++++++++++++++++++
3 files changed, 116 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..fcd4572 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..df4d01c
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,113 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+
+ Format carries a value from the keymap_format enumeration.
Is it possible to specify the interface name too, to make it easier to find the
enum? ("wl_keyboard.keymap_format")

Anyway, it doesn't matter a lot, because a fix for references to foreign enums
will be merged soon (hopefully).
+ </description>
+ <arg name="format" type="uint" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+
+ State carries a value from the key_state enumeration.
(Ditto)
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
No need for `since="1"`, this is implicit.
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
+ </enum>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
Which error?

Maybe this could just be reworded as something among the lines of "it should
only allow trusted clients to use this request", implying it can either
disconnect clients or (and this is probably preferred) don't expose the global
by default.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
Is it possible to add a request to destroy the manager? Quoting pq from the
This interface is missing a destroy request. Interfaces must always have a
destroy request unless there is a very good reason to not have one. In any
case, every object must be destroyable somehow.
+ </interface>
+</protocol>
--
2.14.3
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Otherwise, this protocol looks pretty good to me.

---
Simon Ser
https://emersion.fr
Peter Hutterer
2018-05-30 04:28:12 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.
- fixed typos
- enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
- added errors
./configure --enable-clients
./weston-keyboard
https://github.com/swaywm/wlroots/pull/999
Thanks for feedback,
Dorota Czaplejewicz
just one nitpick over Simon's comments, the copyright list starting at 2008
seems excessive.

However, I wonder what you're really trying to achieve here. For virtual
keyboards the mapping to a normal keyboard's physical buttons, then mapped
to the glyph elsewhere seems strange and limiting. On a virtual keyboard,
I'm not hitting KEY_A, I'm clicking on the button currently labelled with
'A'. There's a million things we could do to virtual keyboard that make the
assumption of virtual keyboard == physical keyboards go boom quite quickly.
e.g. on my phone once the emoticons are selected I don't get the normal
qwerty layout anymore but just a row/column arrangement of smileys.

I read your v1 email about splitting input-method and virtual-keyboard but
it's still not quite clear to me, sorry.

Cheers,
Peter
Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 113 +++++++++++++++++++++
3 files changed, 116 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..fcd4572 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..df4d01c
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,113 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+
+ Format carries a value from the keymap_format enumeration.
+ </description>
+ <arg name="format" type="uint" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+
+ State carries a value from the key_state enumeration.
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
+ </enum>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+ </interface>
+</protocol>
--
2.14.3
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Dorota Czaplejewicz
2018-05-30 06:43:32 UTC
Permalink
On Wed, 30 May 2018 14:28:12 +1000
Post by Peter Hutterer
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
This is the slightly improved version of the virtual-keyboard-v1 protocol from Purism.
- fixed typos
- enum arguments don't have the "enum" key in them, in order not to trip up wayland-scanner
- added errors
./configure --enable-clients
./weston-keyboard
https://github.com/swaywm/wlroots/pull/999
Thanks for feedback,
Dorota Czaplejewicz
just one nitpick over Simon's comments, the copyright list starting at 2008
seems excessive.
However, I wonder what you're really trying to achieve here. For virtual
keyboards the mapping to a normal keyboard's physical buttons, then mapped
to the glyph elsewhere seems strange and limiting. On a virtual keyboard,
I'm not hitting KEY_A, I'm clicking on the button currently labelled with
'A'. There's a million things we could do to virtual keyboard that make the
assumption of virtual keyboard == physical keyboards go boom quite quickly.
e.g. on my phone once the emoticons are selected I don't get the normal
qwerty layout anymore but just a row/column arrangement of smileys.
I read your v1 email about splitting input-method and virtual-keyboard but
it's still not quite clear to me, sorry.
Cheers,
Peter
Hi Peter,

copyrights go back to 2008 because I used chunks of input-method and wayland.xml as the base.

This virtual keyboard protocol is just that - a way to provide an emulated input device. It's not expected to be a complete solution for what an actual on-screen keyboard is doing, but it's a necessary part. It allows for keyboard shortcuts to be used to interact with the compositor for example.

The second part of responsibilities of an on-screen keyboard is providing text composition capabilities, which covers the emojis you mentioned, but is also limited to text composition scenarios. As an example, it doesn't make sense to control the window manager by sending it text but they usually care about keyboard shortcuts.

These use cases both look like "a keyboard" as they were traditionally done with keyboards, but are really something completely different. To exaggerate the difference, the virtual-keyboard protocol is similar to using a game controller, and input-method is much closer to handwriting recognition than pressing buttons.

I hope that cleared it up a little. Either way, I will be submitting an input-method update soon to show the implementation.

Cheers,
Dorota
Post by Peter Hutterer
Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 113 +++++++++++++++++++++
3 files changed, 116 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 4b9a901..fcd4572 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -17,6 +17,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..df4d01c
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,113 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian HÞgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be
+ memory-mapped to provide a keyboard mapping description.
+
+ Format carries a value from the keymap_format enumeration.
+ </description>
+ <arg name="format" type="uint" summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+
+ State carries a value from the key_state enumeration.
+ </description>
+ <arg name="time" type="uint" summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ The client should use wl_keyboard.modifiers event to synchronize its
+ internal state with seat state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor" since="1">
+ <description summary="destroy the virtual keyboard keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0" summary="client not authorized to use the interface"/>
+ </enum>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should present an error when an untrusted client requests a new
+ keyboard.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+ </interface>
+</protocol>
--
2.14.3
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Jan Arne Petersen
2018-05-30 08:05:52 UTC
Permalink
Hey,
Provides the ability to emulate keyboards by applications.
Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials,
and added seat binding.
---
I do not really understand for what this protocol is useful, trying to map
a virtual keyboard to a physical keyboard seems to be extremely limiting
and I do not see really use-cases for it. If you want to be able to send
non-text from a virtual keyboard to an application it makes more sense to
have some protocol to send keysyms for example.

Regards
Jan Arne Petersen
Dorota Czaplejewicz
2018-05-31 14:25:41 UTC
Permalink
On Wed, 30 May 2018 10:05:52 +0200
Hey,
Provides the ability to emulate keyboards by applications.
Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials,
and added seat binding.
---
I do not really understand for what this protocol is useful, trying to map
a virtual keyboard to a physical keyboard seems to be extremely limiting
and I do not see really use-cases for it. If you want to be able to send
non-text from a virtual keyboard to an application it makes more sense to
have some protocol to send keysyms for example.
Regards
Jan Arne Petersen
Hi,

the immediate need is a virtual keyboard indeed. Keeping the protocol simple and general can make it useful for automating input for example, so I think it's worth to stick to the current form.

It's not a suboptimal form for a virtual keyboard either. I have considered keysyms, gathered opinions, and decided against them. The reason is simple. If we tried to send scan codes, then the compositor would have to do the scan code->keysym translation anyway, but with less information. Either that, or applications would have to support yet another protocol (in addition to wl_keyboard and text-input) to receive generic keyboard events, defeating the point of having a generic protocol.

On the other hand, raw scan codes can be trivially piped via wl_keyboard to any existing application that cares about keyboard input, even if it doesn't implement text-input.

Cheers,
Dorota Czaplejewicz
Dorota Czaplejewicz
2018-06-22 15:20:45 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.

The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,

this patch is another improvement to the previously sent virtual keyboard protocol. Changes compared to v2:

- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping

First off, the new version of wayland-scanner looks up enums found in other xml files, so the textual reference are gone and actual enum values are used.

Secondly, there was a suggestion that if the client wants to know global modifiers, it should listen to wl_keyboard.modifiers. In reality, this can only be done when the client has keyboard focus. I decided to remove this suggestion.

Clarified that create_virtual_keyboard must error out with unauthorized when an untrusted client connects. That doesn't preclude making the whole virtual_keyboard_manager interface unavailable when the client is determined as untrusted ahead of time.

Lastly, added a missing destructor as per Simon Ser's suggestion.

I hope that we're getting closer to perfect with this revision! As usual, feedback is welcome.

Cheers,
Dorota Czaplejewicz

Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 116 +++++++++++++++++++++
3 files changed, 119 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml

diff --git a/Makefile.am b/Makefile.am
index 1aa13cf..17cedc1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -19,6 +19,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)

stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..bde55e8
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+ Software is furnished to do so, subject to the following conditions:
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be memory-mapped
+ to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="keymap_format"
+ summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="time" type="uint"
+ summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="key_state"
+ summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
+
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If an untrusted client issues this request, it should receive the
+ unauthorized error in return.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object"/>
+ </request>
+ </interface>
+</protocol>
--
2.14.4
Simon Ser
2018-06-22 16:36:16 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,
- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping
First off, the new version of wayland-scanner looks up enums found in other xml files, so the textual reference are gone and actual enum values are used.
Secondly, there was a suggestion that if the client wants to know global modifiers, it should listen to wl_keyboard.modifiers. In reality, this can only be done when the client has keyboard focus. I decided to remove this suggestion.
Clarified that create_virtual_keyboard must error out with unauthorized when an untrusted client connects. That doesn't preclude making the whole virtual_keyboard_manager interface unavailable when the client is determined as untrusted ahead of time.
Lastly, added a missing destructor as per Simon Ser's suggestion.
I hope that we're getting closer to perfect with this revision! As usual, feedback is welcome.
Cheers,
Dorota Czaplejewicz
Hi,

Thanks for this updated version. A few comments below.
Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 116 +++++++++++++++++++++
3 files changed, 119 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 1aa13cf..17cedc1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -19,6 +19,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..bde55e8
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,116 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be memory-mapped
+ to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="keymap_format"
+ summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="time" type="uint"
+ summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="key_state"
+ summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
This is more of a general metaphorical question than anything else: I wonder how
we should handle unauthorized clients, and if adding an error for them is a good
idea. Generally errors are meant for protocol violations: it's clear from the
protocol spec that e.g. sending a request with a negative value will trigger a
protocol error. Also, errors are unrecoverable, they abort the whole client
(though this is being worked on).

So here we use an error, and the conditions in which the error is sent are
implementation-defined. I wonder if there's a better way to handle this
situation?
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If an untrusted client issues this request, it should receive the
+ unauthorized error in return.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object"/>
This description should probably specify what happens to keyboards created with
the manager when the manager is destroyed. A common practice is not to destroy
objects when the manager is destroyed, so that one can easily create the
manager, use the factory request, and immediately destroy the manager (see e.g.
wl_subcompositor).
+ </request>
+ </interface>
+</protocol>
--
2.14.4
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Dorota Czaplejewicz
2018-06-22 18:00:00 UTC
Permalink
On Fri, 22 Jun 2018 12:36:16 -0400
Post by Simon Ser
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,
- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping
First off, the new version of wayland-scanner looks up enums found in other xml files, so the textual reference are gone and actual enum values are used.
Secondly, there was a suggestion that if the client wants to know global modifiers, it should listen to wl_keyboard.modifiers. In reality, this can only be done when the client has keyboard focus. I decided to remove this suggestion.
Clarified that create_virtual_keyboard must error out with unauthorized when an untrusted client connects. That doesn't preclude making the whole virtual_keyboard_manager interface unavailable when the client is determined as untrusted ahead of time.
Lastly, added a missing destructor as per Simon Ser's suggestion.
I hope that we're getting closer to perfect with this revision! As usual, feedback is welcome.
Cheers,
Dorota Czaplejewicz
Hi,
Thanks for this updated version. A few comments below.
---- snip ----
Post by Simon Ser
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
This is more of a general metaphorical question than anything else: I wonder how
we should handle unauthorized clients, and if adding an error for them is a good
idea. Generally errors are meant for protocol violations: it's clear from the
protocol spec that e.g. sending a request with a negative value will trigger a
protocol error. Also, errors are unrecoverable, they abort the whole client
(though this is being worked on).
So here we use an error, and the conditions in which the error is sent are
implementation-defined. I wonder if there's a better way to handle this
situation?
Speaking from a position of someone without a lot of Wayland experience: are Wayland errors meant to be recoverable by a client? It's obvious that if an application doing its primary task and then trying to use a restricted protocol as a secondary action crashes, that's undesireable.

As a more concrete example, an automation application may use e.g. DBus API to automate tasks, and display a window to control it. It may request a virtual keyboard to extend its possibilities, but it shouldn't suddenly stop working if it's refused by the compositor.

That brings me to the question: should applications using restricted protocols create additional connections which may be broken and recovered from individually or should they rather use one connection? If the latter is required for some use cases, then authorization and probing/graceful rejection should be specified inside protocols. Unfortunately, I'm not sure where to look for examples. Perhaps chat applications where screen sharing may be decided ad hoc check the marks?
Post by Simon Ser
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If an untrusted client issues this request, it should receive the
+ unauthorized error in return.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object"/>
This description should probably specify what happens to keyboards created with
the manager when the manager is destroyed. A common practice is not to destroy
objects when the manager is destroyed, so that one can easily create the
manager, use the factory request, and immediately destroy the manager (see e.g.
wl_subcompositor).
This is an oversight and will be fixed.
Post by Simon Ser
+ </request>
+ </interface>
+</protocol>
--
2.14.4
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Markus Ongyerth
2018-06-24 09:46:50 UTC
Permalink
Post by Dorota Czaplejewicz
On Fri, 22 Jun 2018 12:36:16 -0400
Post by Simon Ser
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,
- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping
First off, the new version of wayland-scanner looks up enums found in other xml files, so the textual reference are gone and actual enum values are used.
Secondly, there was a suggestion that if the client wants to know global modifiers, it should listen to wl_keyboard.modifiers. In reality, this can only be done when the client has keyboard focus. I decided to remove this suggestion.
Clarified that create_virtual_keyboard must error out with unauthorized when an untrusted client connects. That doesn't preclude making the whole virtual_keyboard_manager interface unavailable when the client is determined as untrusted ahead of time.
Lastly, added a missing destructor as per Simon Ser's suggestion.
I hope that we're getting closer to perfect with this revision! As usual, feedback is welcome.
Cheers,
Dorota Czaplejewicz
Hi,
Thanks for this updated version. A few comments below.
---- snip ----
Post by Simon Ser
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
This is more of a general metaphorical question than anything else: I wonder how
we should handle unauthorized clients, and if adding an error for them is a good
idea. Generally errors are meant for protocol violations: it's clear from the
protocol spec that e.g. sending a request with a negative value will trigger a
protocol error. Also, errors are unrecoverable, they abort the whole client
(though this is being worked on).
So here we use an error, and the conditions in which the error is sent are
implementation-defined. I wonder if there's a better way to handle this
situation?
Speaking from a position of someone without a lot of Wayland experience: are Wayland errors meant to be recoverable by a client? It's obvious that if an application doing its primary task and then trying to use a restricted protocol as a secondary action crashes, that's undesireable.
As a more concrete example, an automation application may use e.g. DBus API to automate tasks, and display a window to control it. It may request a virtual keyboard to extend its possibilities, but it shouldn't suddenly stop working if it's refused by the compositor.
Currently the errors will result in an abort() by libwayland. There is a patch
series that changes this, but it will still be fatal on the wayland connection
as far as I'm aware. Which isn't quite as bad, but can still be a problem, if
an application uses the same wayland connection for more than one feature.
Post by Dorota Czaplejewicz
That brings me to the question: should applications using restricted protocols create additional connections which may be broken and recovered from individually or should they rather use one connection?
I'd personally prefer single fat connection. Mostly because it makes some
bookkeeping easier, and some of the things I want to try out from the
compositor side work better this way. But it's more robust (or rather will be)
to have multiple, especially if one feature is expected to result in errors
for some reason (the only expected errors should be design mistakes IMO, but
this might still happen)
Post by Dorota Czaplejewicz
If the latter is required for some use cases, then authorization and probing/graceful rejection should be specified inside protocols. Unfortunately, I'm not sure where to look for examples. Perhaps chat applications where screen sharing may be decided ad hoc check the marks?
This is a good usecase for defaulting to query the user IMO. The client would
try to access the interface at that time (either bind late, or have the first
real request) at which point the compositor shows the user a query window that
asks for permission to be granted for the client in question.
IFF the client uses the fat connection, it could display the window of the
client in the query/highlight it on screen.
In the split case, this isn't quite as easy.

If the application is trusted with screen contents in general, it can be
started by the compositor with a pre-prepared wl_client (with WAYLAND_SOCKET
and socketpair()) which is associated with permissions. Sadly this fails hard
with the multi-connection approach, since only the first client will be
recognised.

Another approach I want to try out (but haven't gotten around to yet, due to
lack of immediate usecase) would be to use dedicated WAYLAND_DISPLAY sockets
for some applications (protected by container mountpoints) and have
permissions bound to all clients accepted over that socket.


The socket/client based approach has the advantage, that we can use the
wl_display_set_global_filter to prevent clients binding to privileged
protocols in the first place. On the other hand, sending the query on demand
is quite a bit more flexible, but can result in odd cases, where a client
creates an object (which can't fail) but the compositor doesn't give
permissions to it.
Post by Dorota Czaplejewicz
Post by Simon Ser
+ <request name="create_virtual_keyboard">
+ <description summary="Create a new virtual keyboard">
+ Creates a new virtual keyboard associated to a seat.
+
+ If an untrusted client issues this request, it should receive the
+ unauthorized error in return.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object"/>
This description should probably specify what happens to keyboards created with
the manager when the manager is destroyed. A common practice is not to destroy
objects when the manager is destroyed, so that one can easily create the
manager, use the factory request, and immediately destroy the manager (see e.g.
wl_subcompositor).
This is an oversight and will be fixed.
Post by Simon Ser
+ </request>
+ </interface>
+</protocol>
--
2.14.4
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Simon Ser
2018-06-24 11:24:56 UTC
Permalink
Post by Dorota Czaplejewicz
On Fri, 22 Jun 2018 12:36:16 -0400
Post by Simon Ser
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,
- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping
First off, the new version of wayland-scanner looks up enums found in other xml files, so the textual reference are gone and actual enum values are used.
Secondly, there was a suggestion that if the client wants to know global modifiers, it should listen to wl_keyboard.modifiers. In reality, this can only be done when the client has keyboard focus. I decided to remove this suggestion.
Clarified that create_virtual_keyboard must error out with unauthorized when an untrusted client connects. That doesn't preclude making the whole virtual_keyboard_manager interface unavailable when the client is determined as untrusted ahead of time.
Lastly, added a missing destructor as per Simon Ser's suggestion.
I hope that we're getting closer to perfect with this revision! As usual, feedback is welcome.
Cheers,
Dorota Czaplejewicz
Hi,
Thanks for this updated version. A few comments below.
---- snip ----
Post by Simon Ser
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
This is more of a general metaphorical question than anything else: I wonder how
we should handle unauthorized clients, and if adding an error for them is a good
idea. Generally errors are meant for protocol violations: it's clear from the
protocol spec that e.g. sending a request with a negative value will trigger a
protocol error. Also, errors are unrecoverable, they abort the whole client
(though this is being worked on).
So here we use an error, and the conditions in which the error is sent are
implementation-defined. I wonder if there's a better way to handle this
situation?
Speaking from a position of someone without a lot of Wayland experience: are
Wayland errors meant to be recoverable by a client? It's obvious that if an
application doing its primary task and then trying to use a restricted
protocol as a secondary action crashes, that's undesireable.
Wayland protocol errors are meant to be fatal. They're meant to signal that the
client is doing something really wrong, against the protocol specification. They
indicate a bug in the client. As ongy said, they currently abort() the whole
client.
Post by Dorota Czaplejewicz
As a more concrete example, an automation application may use e.g. DBus API to
automate tasks, and display a window to control it. It may request a virtual
keyboard to extend its possibilities, but it shouldn't suddenly stop working
if it's refused by the compositor.
That's right, and that's a reason why I'd personally prefer not to use protocol
errors for security errors.
Post by Dorota Czaplejewicz
That brings me to the question: should applications using restricted protocols
create additional connections which may be broken and recovered from
individually or should they rather use one connection? If the latter is
required for some use cases, then authorization and probing/graceful rejection
should be specified inside protocols. Unfortunately, I'm not sure where to
look for examples. Perhaps chat applications where screen sharing may be
decided ad hoc check the marks?
Additional connections may not help for now because libwayland just aborts on
protocol errors (though this might change in the future). Also, using multiple
connections makes it very difficult to share objects (e.g. surfaces) between
the privileged connection and the unprivileged one.

Again as ongy said, the compositor can decide to ask the user for permission
either when the client binds or when it sends a particular request. This is
what GNOME does when a client tries to use the pointer-constraints protocol for
instance: the constraint isn't activated until the user accepts. This works
relatively well for protocols allowing the compositor to give feedback to the
client. Another example is our new export-dmabuf protocol [1], it has a
"cancel" event which allows the compositor to say "I don't want you to capture
this output".

This gets more complicated for protocols like yours which don't have such
feedback events. Since binding to an interface or sending a request that creates
a new object can never fail, the client has no way to know if the compositor
refuses his requests or not. If you're asking the user for permission when the
client binds to the global, it gets even more complicated, since you still need
to allow the client to use the global to create objects, but you want these
objects to be inert (you want to ignore all requests on `wp_virtual_keyboard`).
And when the user accepts, you need to iterate over all these objects, and make
them non-inert. What if the objects themselves have requests to create yet other
objects?

However, access to your protocol should be restricted to a few clients. That
makes it viable to require users to update their compositor config to allow
a client to use the interface. To paraphrase ongy again, it's possible to
create a Wayland socket for a particular client and add privileged globals to
this socket only. That way the global is invisible to regular clients but is
visible to your e.g. IME. This would allow a Android-like UI for instance, where
you choose your virtual keyboard in the system settings.

[1]: https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-export-dmabuf-unstable-v1.xml
Simon Ser
2018-06-25 10:04:11 UTC
Permalink
Post by Simon Ser
Post by Dorota Czaplejewicz
On Fri, 22 Jun 2018 12:36:16 -0400
Post by Simon Ser
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,
- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping
First off, the new version of wayland-scanner looks up enums found in other xml files, so the textual reference are gone and actual enum values are used.
Secondly, there was a suggestion that if the client wants to know global modifiers, it should listen to wl_keyboard.modifiers. In reality, this can only be done when the client has keyboard focus. I decided to remove this suggestion.
Clarified that create_virtual_keyboard must error out with unauthorized when an untrusted client connects. That doesn't preclude making the whole virtual_keyboard_manager interface unavailable when the client is determined as untrusted ahead of time.
Lastly, added a missing destructor as per Simon Ser's suggestion.
I hope that we're getting closer to perfect with this revision! As usual, feedback is welcome.
Cheers,
Dorota Czaplejewicz
Hi,
Thanks for this updated version. A few comments below.
---- snip ----
Post by Simon Ser
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
This is more of a general metaphorical question than anything else: I wonder how
we should handle unauthorized clients, and if adding an error for them is a good
idea. Generally errors are meant for protocol violations: it's clear from the
protocol spec that e.g. sending a request with a negative value will trigger a
protocol error. Also, errors are unrecoverable, they abort the whole client
(though this is being worked on).
So here we use an error, and the conditions in which the error is sent are
implementation-defined. I wonder if there's a better way to handle this
situation?
Speaking from a position of someone without a lot of Wayland experience: are
Wayland errors meant to be recoverable by a client? It's obvious that if an
application doing its primary task and then trying to use a restricted
protocol as a secondary action crashes, that's undesireable.
Wayland protocol errors are meant to be fatal. They're meant to signal that the
client is doing something really wrong, against the protocol specification. They
indicate a bug in the client. As ongy said, they currently abort() the whole
client.
It seems we've been misunderstanding this point. They don't abort the client,
they just kill the connection. After that, functions called with the same
wl_display will start returning errors.

There are functions to get details about the protocol error
(wl_display_get_error and wl_display_get_protocol_error) but those are more for
better logging than recovery.

Thanks Pekka for pointing that out!
Post by Simon Ser
Post by Dorota Czaplejewicz
As a more concrete example, an automation application may use e.g. DBus API to
automate tasks, and display a window to control it. It may request a virtual
keyboard to extend its possibilities, but it shouldn't suddenly stop working
if it's refused by the compositor.
That's right, and that's a reason why I'd personally prefer not to use protocol
errors for security errors.
Post by Dorota Czaplejewicz
That brings me to the question: should applications using restricted protocols
create additional connections which may be broken and recovered from
individually or should they rather use one connection? If the latter is
required for some use cases, then authorization and probing/graceful rejection
should be specified inside protocols. Unfortunately, I'm not sure where to
look for examples. Perhaps chat applications where screen sharing may be
decided ad hoc check the marks?
Additional connections may not help for now because libwayland just aborts on
protocol errors (though this might change in the future). Also, using multiple
connections makes it very difficult to share objects (e.g. surfaces) between
the privileged connection and the unprivileged one.
Again as ongy said, the compositor can decide to ask the user for permission
either when the client binds or when it sends a particular request. This is
what GNOME does when a client tries to use the pointer-constraints protocol for
instance: the constraint isn't activated until the user accepts. This works
relatively well for protocols allowing the compositor to give feedback to the
client. Another example is our new export-dmabuf protocol [1], it has a
"cancel" event which allows the compositor to say "I don't want you to capture
this output".
This gets more complicated for protocols like yours which don't have such
feedback events. Since binding to an interface or sending a request that creates
a new object can never fail, the client has no way to know if the compositor
refuses his requests or not. If you're asking the user for permission when the
client binds to the global, it gets even more complicated, since you still need
to allow the client to use the global to create objects, but you want these
objects to be inert (you want to ignore all requests on `wp_virtual_keyboard`).
And when the user accepts, you need to iterate over all these objects, and make
them non-inert. What if the objects themselves have requests to create yet other
objects?
However, access to your protocol should be restricted to a few clients. That
makes it viable to require users to update their compositor config to allow
a client to use the interface. To paraphrase ongy again, it's possible to
create a Wayland socket for a particular client and add privileged globals to
this socket only. That way the global is invisible to regular clients but is
visible to your e.g. IME. This would allow a Android-like UI for instance, where
you choose your virtual keyboard in the system settings.
[1]: https://github.com/swaywm/wlr-protocols/blob/master/unstable/wlr-export-dmabuf-unstable-v1.xml
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Pekka Paalanen
2018-06-25 10:17:22 UTC
Permalink
On Fri, 22 Jun 2018 20:00:00 +0200
Post by Dorota Czaplejewicz
On Fri, 22 Jun 2018 12:36:16 -0400
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hi,
this patch is another improvement to the previously sent virtual
- readded enum names
- removed suggestion to listen to modifiers
- clarified untrusted client behaviour
- added destructor on virtual_keyboard_manager
- fixed text wrapping
First off, the new version of wayland-scanner looks up enums
found in other xml files, so the textual reference are gone and
actual enum values are used.
Hi,

that is not exactly true. Now wayland-scanner just skips checking any
referenced enums that are not defined in the current XML file. It does
not make lookups in other XML files.
Post by Dorota Czaplejewicz
---- snip ----
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to
provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <enum name="error">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the interface"/>
+ </enum>
I wonder how we should handle unauthorized clients, and if adding
an error for them is a good idea. Generally errors are meant for
protocol violations: it's clear from the protocol spec that e.g.
sending a request with a negative value will trigger a protocol
error. Also, errors are unrecoverable, they abort the whole client
(though this is being worked on).
Errors will remain unrecoverable because they will always cause a
disconnection.

What is being worked on is avoiding the abort in the few yet unhandled
loew-level failure cases like implicit flush, so that the app can at
last clean up and save what it was doing, or maybe even reconnect if it
really wants to.

Protocol errors should not cause an abort() already.
Post by Dorota Czaplejewicz
So here we use an error, and the conditions in which the error is
sent are implementation-defined. I wonder if there's a better way
to handle this situation?
Speaking from a position of someone without a lot of Wayland
experience: are Wayland errors meant to be recoverable by a client?
No, protocol errors are always fatal to the Wayland connection, which
means the client gets disconnected and "destroyed". The application
process may continue though and even attemp to reconnect, but that
should not be relied upon in protocol design. So from protocol design
perspective, protocol errors are always unrecoverable.

The idea is that when a protocol error happens, it may invalidate
further requests made by the client and it almost always implies that
the server and the client no longer agree on the protocol state. Hence,
keeping the connection would be an equally good idea to continuing a
process after receiving a SIGSEGV.
Post by Dorota Czaplejewicz
It's obvious that if an application doing its primary task and then
trying to use a restricted protocol as a secondary action crashes,
that's undesireable.
As a more concrete example, an automation application may use e.g.
DBus API to automate tasks, and display a window to control it. It
may request a virtual keyboard to extend its possibilities, but it
shouldn't suddenly stop working if it's refused by the compositor.
Indeed, one cannot use protocol errors for graceful rejection.
Post by Dorota Czaplejewicz
That brings me to the question: should applications using restricted
protocols create additional connections which may be broken and
recovered from individually or should they rather use one connection?
If the latter is required for some use cases, then authorization and
probing/graceful rejection should be specified inside protocols.
Unfortunately, I'm not sure where to look for examples. Perhaps chat
applications where screen sharing may be decided ad hoc check the
marks?
The intention is for one application to use one connection, but nothing
forbids using multiple connections. Multiple connection are usually
impractical, though. If the client was connected with WAYLAND_SOCKET (as
opposed to WAYLAND_DISPLAY), the client probably won't have the means
to reconnect or open additional connections.

Wayland's isolation between clients means that one cannot refer to a
Wayland object of one connection in another connection. It does not
matter if it's the same process or thread, if it's a separate
connection it is isolated.

There is currently no agreed mechanism to see if an interface is
allowed for the client or if it is even a privileged interface.

Giulio Camuffo at least had a proposal for advertising privileged
interfaces with graceful deny. Authenticating clients OTOH continues to
be an unsolved issue in general, I believe.


Thanks,
pq
Dorota Czaplejewicz
2018-06-25 18:36:06 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to input-method protocol.

The interface is a mirror copy of wl_keyboard, with removed serials, and added seat binding.
---
Hello,

thank you for giving me a lot of useful feedback in the last round. I applied the suggestions and this is the new incarnation of the protocol. Changes to v3, in no particular order:

- added clarification to the destructor
- namespaced enum references
- made virtual keyboard creation non-fatal

The responses about keeping a virtual keyboard on a separate connection and about using protocol errors as part of the authentication mechanism were all in agreement, and they made sense to me too. As a result, there is a whole new mechanism for that.

The client now requests a creation of a virtual keyboard, but does not immediately get a valid instance. It must wait for one of two events in virtual_keyboard_manager: `virtual_keyboard_create_failed` for a failure or `virtual_keyboard_created` for success. I deliberately kept them inside the manager instead of virtual_keyboard, since I don't like the idea of creating an object and immediately sending a message that it's useless (in case of failure).

I also deliberately kept the names long in order not to forget what's being manipulated.

The serial number is there to match up requests on different seats with failures. I imagine that a compositor might spam the user(s) with a number of permission requests for different seats, and the user would deal with them in a random order. Serials prevent the answers from getting mixed up.

I am not happy only with one aspect of this: the amount of time between create request and response is indefinite, opening a loophole for the compositor never to reply. I decided that it doesn't warrant a protocol error however.

As usual, feedback is welcome.

Cheers,
Dorota Czaplejewicz

Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 154 +++++++++++++++++++++
3 files changed, 157 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml

diff --git a/Makefile.am b/Makefile.am
index 1aa13cf..17cedc1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -19,6 +19,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)

stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..173f3bc
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,154 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+ Software is furnished to do so, subject to the following conditions:
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be memory-mapped
+ to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="wl_keyboard.keymap_format"
+ summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="time" type="uint"
+ summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="wl_keyboard.key_state"
+ summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <request name="create_virtual_keyboard">
+ <description summary="create a new virtual keyboard">
+ Asks to create a new virtual keyboard associated to a seat.
+
+ If the client is found unauthorized to create a virtual keyboard, then
+ it must receive the virtual_keyboard_create_failed event, with the
+ reason field set to unauthorized.
+
+ Otherwise, the client must receive the virtual_keyboard_created event.
+
+ Serial numbers used in consecutive create_virtual_keyboard requests
+ must form a sequence without repetitions.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="serial" type="uint"/>
+ </request>
+
+ <enum name="virtual_keyboard_create_failed_reason">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the request"/>
+ </enum>
+
+ <event name="virtual_keyboard_create_failed">
+ <description summary="creation of a new virtual keyboard failed">
+ Indicates that a new virtual keyboard instance could not be created.
+
+ Reason contains the reason why creation did not succeed.
+
+ Serial is the serial number of the corresponding
+ create_virtual_keyboard request.
+ </description>
+ <arg name="reason" type="uint"
+ enum="virtual_keyboard_create_failed_reason"
+ summary="indicate the reason for this failure"/>
+ <arg name="serial" type="uint" description="serial number of request"/>
+ </event>
+
+ <event name="virtual_keyboard_created">
+ <description summary="successfully created a new virtual keyboard">
+ Provides the client with the zwp_virtual_keyboard_v1 which was
+ previously requested using create_virtual_keyboard.
+
+ Serial is the serial number of the corresponding
+ create_virtual_keyboard request.
+ </description>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"
+ summary="the id of the newly created virtual keyboard"/>
+ <arg name="serial" type="uint" description="serial number of request"/>
+ </event>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object">
+ Destroys the virtual keyboard manager.
+
+ Existing zwp_virtual_keyboard_v1 objects remain valid.
+ </description>
+ </request>
+ </interface>
+</protocol>
--
2.14.4
Simon Ser
2018-07-12 22:15:32 UTC
Permalink
Provides the ability to emulate keyboards by applications. Complementary to
input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added
seat binding.
---
Hello,
thank you for giving me a lot of useful feedback in the last round. I applied
the suggestions and this is the new incarnation of the protocol. Changes to v3,
- added clarification to the destructor namespaced enum references made virtual
- keyboard creation non-fatal
The responses about keeping a virtual keyboard on a separate connection and
about using protocol errors as part of the authentication mechanism were all in
agreement, and they made sense to me too. As a result, there is a whole new
mechanism for that.
The client now requests a creation of a virtual keyboard, but does not
immediately get a valid instance. It must wait for one of two events in
virtual_keyboard_manager: `virtual_keyboard_create_failed` for a failure or
`virtual_keyboard_created` for success. I deliberately kept them inside the
manager instead of virtual_keyboard, since I don't like the idea of creating an
object and immediately sending a message that it's useless (in case of failure).
I also deliberately kept the names long in order not to forget what's being manipulated.
The serial number is there to match up requests on different seats with
failures. I imagine that a compositor might spam the user(s) with a number of
permission requests for different seats, and the user would deal with them in a
random order. Serials prevent the answers from getting mixed up.
I am not happy only with one aspect of this: the amount of time between create
request and response is indefinite, opening a loophole for the compositor never
to reply. I decided that it doesn't warrant a protocol error however.
As usual, feedback is welcome.
Hi,

Sorry for the delay.

I'm not sure I like this new design.

First, this is using server-side created resources (the a `new_id` argument in
an event). One should be really careful when doing so, see Pekka's post on the
matter [1].

Then, this serial mechanism makes me uneasy. A more idiomatic approach would be
to use a proper object for this.

Finally, I'm not even sure this security mechanism belongs here. I think adding
this mechanism to potentially all privileged protocols will result in duplicated
code and "pollutes" protocols. Here are some other solutions:

* Have one keyboard daemon they spawn themselves. In this case they can only
expose this interface on the daemon's display. This is e.g. weston's approach
for its private protocols, and will be sway's approach for privileged
protocols.
* Allow other clients to use this interface too, but don't let them know if the
keyboard they created is active or is ignored. This could possibly lead to
bad UX.
* Allow other clients to use this interface too, and use another protocol to
manage authorizations. Basically the idea would be not to expose this
interface, and require the client to request access to this privileged
interface through an authorizer protocol. Someone already mentioned it, this
is Orbital's approach [2]. This allows the compositor to spawn a dialog asking
to the user "do you want to allow <client> to create a virtual keyboard?".

What do you think?

An event on the keyboard to signal when the keyboard is no longer active
(ie. when the compositor decides to destroy it) might still be useful, maybe.

Everything else looks good to me. Thanks for your work!

Simon

[1]: http://ppaalanen.blogspot.com/2014/07/wayland-protocol-design-object-lifespan.html
[2]: https://github.com/giucam/orbital/blob/master/protocol/orbital-authorizer.xml
Cheers,
Dorota Czaplejewicz
Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 154 +++++++++++++++++++++
3 files changed, 157 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 1aa13cf..17cedc1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -19,6 +19,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..173f3bc
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,154 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian Høgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be memory-mapped
+ to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="wl_keyboard.keymap_format"
+ summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="time" type="uint"
+ summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="wl_keyboard.key_state"
+ summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <request name="create_virtual_keyboard">
+ <description summary="create a new virtual keyboard">
+ Asks to create a new virtual keyboard associated to a seat.
+
+ If the client is found unauthorized to create a virtual keyboard, then
+ it must receive the virtual_keyboard_create_failed event, with the
+ reason field set to unauthorized.
+
+ Otherwise, the client must receive the virtual_keyboard_created event.
+
+ Serial numbers used in consecutive create_virtual_keyboard requests
+ must form a sequence without repetitions.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="serial" type="uint"/>
+ </request>
+
+ <enum name="virtual_keyboard_create_failed_reason">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the request"/>
+ </enum>
+
+ <event name="virtual_keyboard_create_failed">
+ <description summary="creation of a new virtual keyboard failed">
+ Indicates that a new virtual keyboard instance could not be created.
+
+ Reason contains the reason why creation did not succeed.
+
+ Serial is the serial number of the corresponding
+ create_virtual_keyboard request.
+ </description>
+ <arg name="reason" type="uint"
+ enum="virtual_keyboard_create_failed_reason"
+ summary="indicate the reason for this failure"/>
+ <arg name="serial" type="uint" description="serial number of request"/>
+ </event>
+
+ <event name="virtual_keyboard_created">
+ <description summary="successfully created a new virtual keyboard">
+ Provides the client with the zwp_virtual_keyboard_v1 which was
+ previously requested using create_virtual_keyboard.
+
+ Serial is the serial number of the corresponding
+ create_virtual_keyboard request.
+ </description>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"
+ summary="the id of the newly created virtual keyboard"/>
+ <arg name="serial" type="uint" description="serial number of request"/>
+ </event>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object">
+ Destroys the virtual keyboard manager.
+
+ Existing zwp_virtual_keyboard_v1 objects remain valid.
+ </description>
+ </request>
+ </interface>
+</protocol>
--
2.14.4
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Dorota Czaplejewicz
2018-07-24 13:10:29 UTC
Permalink
On Thu, 12 Jul 2018 18:15:32 -0400
Post by Simon Ser
Provides the ability to emulate keyboards by applications. Complementary to
input-method protocol.
The interface is a mirror copy of wl_keyboard, with removed serials, and added
seat binding.
---
Hello,
thank you for giving me a lot of useful feedback in the last round. I applied
the suggestions and this is the new incarnation of the protocol. Changes to v3,
- added clarification to the destructor namespaced enum references made virtual
- keyboard creation non-fatal
The responses about keeping a virtual keyboard on a separate connection and
about using protocol errors as part of the authentication mechanism were all in
agreement, and they made sense to me too. As a result, there is a whole new
mechanism for that.
The client now requests a creation of a virtual keyboard, but does not
immediately get a valid instance. It must wait for one of two events in
virtual_keyboard_manager: `virtual_keyboard_create_failed` for a failure or
`virtual_keyboard_created` for success. I deliberately kept them inside the
manager instead of virtual_keyboard, since I don't like the idea of creating an
object and immediately sending a message that it's useless (in case of failure).
I also deliberately kept the names long in order not to forget what's being
manipulated.
The serial number is there to match up requests on different seats with
failures. I imagine that a compositor might spam the user(s) with a number of
permission requests for different seats, and the user would deal with them in a
random order. Serials prevent the answers from getting mixed up.
I am not happy only with one aspect of this: the amount of time between create
request and response is indefinite, opening a loophole for the compositor never
to reply. I decided that it doesn't warrant a protocol error however.
As usual, feedback is welcome.
Hi,
Sorry for the delay.
I'm not sure I like this new design.
First, this is using server-side created resources (the a `new_id` argument in
an event). One should be really careful when doing so, see Pekka's post on the
matter [1].
Thanks, this was an insightful read. If not in this protocol, this kept me from making mistakes in my input-method revision.
Post by Simon Ser
Then, this serial mechanism makes me uneasy. A more idiomatic approach would be
to use a proper object for this.
Do you mean an object defining that an attempt was made, a sort of "creation_attempt" which would later receive the result of the attempt?
Post by Simon Ser
Finally, I'm not even sure this security mechanism belongs here. I think adding
this mechanism to potentially all privileged protocols will result in duplicated
* Have one keyboard daemon they spawn themselves. In this case they can only
expose this interface on the daemon's display. This is e.g. weston's approach
for its private protocols, and will be sway's approach for privileged
protocols.
* Allow other clients to use this interface too, but don't let them know if the
keyboard they created is active or is ignored. This could possibly lead to
bad UX.
* Allow other clients to use this interface too, and use another protocol to
manage authorizations. Basically the idea would be not to expose this
interface, and require the client to request access to this privileged
interface through an authorizer protocol. Someone already mentioned it, this
is Orbital's approach [2]. This allows the compositor to spawn a dialog asking
to the user "do you want to allow <client> to create a virtual keyboard?".
What do you think?
I'm in favor of option 3, which will be useful for other protocols as well, which are not necessarily in position to be kept in a permanent privileged process, e.g. screen recording.

It would also make me happier as a protocol author, because the protocol becomes more compact and easier to maintain without explicit managing.
Post by Simon Ser
An event on the keyboard to signal when the keyboard is no longer active
(ie. when the compositor decides to destroy it) might still be useful, maybe.
After chatting out of band (thanks Simon for the clarification!), we painted an image where the compositor doesn't want the keyboard to exist any more, e.g. because the user revoked permission, or because it was turned off by the user.

If it was only about permissions, it would seem like a good idea for the authorization protocol to handle disabling the virtual keyboard. Unfortunately, it seems to be only possible to render the global inert. That gets complicated because the global is a factory and its children need to be implicitly inert as well.

There are other protocols which implement an event that notifies the client that an object is inert, e.g. zwp_linux_buffer_params_v1.{created,failed} or zwp_confined_pointer_v1.unconfined. Some objects that sound like they should be force-destroyable don't have this, e.g. zxdg_toplevel_v6. It's not clear to me what the separation is and where virtual-keyboard falls.

I won't be adding such an event now, but I'm happy to get educated and then do the right thing.
Post by Simon Ser
Everything else looks good to me. Thanks for your work!
Thank you for reviewing!

--Dorota
Post by Simon Ser
Simon
[1]: http://ppaalanen.blogspot.com/2014/07/wayland-protocol-design-object-lifespan.html
[2]: https://github.com/giucam/orbital/blob/master/protocol/orbital-authorizer.xml
Cheers,
Dorota Czaplejewicz
Makefile.am | 1 +
unstable/virtual-keyboard/README | 2 +
.../virtual-keyboard-unstable-v1.xml | 154 +++++++++++++++++++++
3 files changed, 157 insertions(+)
create mode 100644 unstable/virtual-keyboard/README
create mode 100644 unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
diff --git a/Makefile.am b/Makefile.am
index 1aa13cf..17cedc1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -19,6 +19,7 @@ unstable_protocols = \
unstable/keyboard-shortcuts-inhibit/keyboard-shortcuts-inhibit-unstable-v1.xml \
unstable/xdg-output/xdg-output-unstable-v1.xml \
unstable/input-timestamps/input-timestamps-unstable-v1.xml \
+ unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml \
$(NULL)
stable_protocols = \
diff --git a/unstable/virtual-keyboard/README b/unstable/virtual-keyboard/README
new file mode 100644
index 0000000..a2c646d
--- /dev/null
+++ b/unstable/virtual-keyboard/README
@@ -0,0 +1,2 @@
+Virtual keyboard protocol
+
diff --git a/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
new file mode 100644
index 0000000..173f3bc
--- /dev/null
+++ b/unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml
@@ -0,0 +1,154 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<protocol name="virtual_keyboard_unstable_v1">
+ <copyright>
+ Copyright © 2008-2011 Kristian HÞgsberg
+ Copyright © 2010-2013 Intel Corporation
+ Copyright © 2012-2013 Collabora, Ltd.
+ Copyright © 2018 Purism SPC
+
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+
+ The above copyright notice and this permission notice (including the next
+ paragraph) shall be included in all copies or substantial portions of the
+ Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
+ </copyright>
+
+ <interface name="zwp_virtual_keyboard_v1" version="1">
+ <description summary="virtual keyboard">
+ The virtual keyboard provides an application with requests which emulate
+ the behaviour of a physical keyboard.
+
+ This interface can be used by clients on its own to provide raw input
+ events, or it can accompany the input method protocol.
+ </description>
+
+ <request name="keymap">
+ <description summary="keyboard mapping">
+ Provide a file descriptor to the compositor which can be memory-mapped
+ to provide a keyboard mapping description.
+ </description>
+ <arg name="format" type="uint" enum="wl_keyboard.keymap_format"
+ summary="keymap format"/>
+ <arg name="fd" type="fd" summary="keymap file descriptor"/>
+ <arg name="size" type="uint" summary="keymap size, in bytes"/>
+ </request>
+
+ <enum name="error">
+ <entry name="no_keymap" value="0" summary="No keymap was set"/>
+ </enum>
+
+ <request name="key">
+ <description summary="key event">
+ A key was pressed or released.
+ The time argument is a timestamp with millisecond granularity, with an
+ undefined base. All requests regarding a single object must share the
+ same clock.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="time" type="uint"
+ summary="timestamp with millisecond granularity"/>
+ <arg name="key" type="uint" summary="key that produced the event"/>
+ <arg name="state" type="uint" enum="wl_keyboard.key_state"
+ summary="physical state of the key"/>
+ </request>
+
+ <request name="modifiers">
+ <description summary="modifier and group state">
+ Notifies the compositor that the modifier and/or group state has
+ changed, and it should update state.
+
+ Keymap must be set before issuing this request.
+ </description>
+ <arg name="mods_depressed" type="uint" summary="depressed modifiers"/>
+ <arg name="mods_latched" type="uint" summary="latched modifiers"/>
+ <arg name="mods_locked" type="uint" summary="locked modifiers"/>
+ <arg name="group" type="uint" summary="keyboard layout"/>
+ </request>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard object"/>
+ </request>
+ </interface>
+
+ <interface name="zwp_virtual_keyboard_manager_v1" version="1">
+ <description summary="virtual keyboard manager">
+ A virtual keyboard manager allows an application to provide keyboard
+ input events as if they came from a physical keyboard.
+
+ If the compositor enables a keyboard to perform arbitrary actions, it
+ should prevent untrusted clients from using this interface.
+ </description>
+
+ <request name="create_virtual_keyboard">
+ <description summary="create a new virtual keyboard">
+ Asks to create a new virtual keyboard associated to a seat.
+
+ If the client is found unauthorized to create a virtual keyboard, then
+ it must receive the virtual_keyboard_create_failed event, with the
+ reason field set to unauthorized.
+
+ Otherwise, the client must receive the virtual_keyboard_created event.
+
+ Serial numbers used in consecutive create_virtual_keyboard requests
+ must form a sequence without repetitions.
+ </description>
+ <arg name="seat" type="object" interface="wl_seat"/>
+ <arg name="serial" type="uint"/>
+ </request>
+
+ <enum name="virtual_keyboard_create_failed_reason">
+ <entry name="unauthorized" value="0"
+ summary="client not authorized to use the request"/>
+ </enum>
+
+ <event name="virtual_keyboard_create_failed">
+ <description summary="creation of a new virtual keyboard failed">
+ Indicates that a new virtual keyboard instance could not be created.
+
+ Reason contains the reason why creation did not succeed.
+
+ Serial is the serial number of the corresponding
+ create_virtual_keyboard request.
+ </description>
+ <arg name="reason" type="uint"
+ enum="virtual_keyboard_create_failed_reason"
+ summary="indicate the reason for this failure"/>
+ <arg name="serial" type="uint" description="serial number of request"/>
+ </event>
+
+ <event name="virtual_keyboard_created">
+ <description summary="successfully created a new virtual keyboard">
+ Provides the client with the zwp_virtual_keyboard_v1 which was
+ previously requested using create_virtual_keyboard.
+
+ Serial is the serial number of the corresponding
+ create_virtual_keyboard request.
+ </description>
+ <arg name="id" type="new_id" interface="zwp_virtual_keyboard_v1"
+ summary="the id of the newly created virtual keyboard"/>
+ <arg name="serial" type="uint" description="serial number of request"/>
+ </event>
+
+ <request name="destroy" type="destructor">
+ <description summary="destroy the virtual keyboard manager object">
+ Destroys the virtual keyboard manager.
+
+ Existing zwp_virtual_keyboard_v1 objects remain valid.
+ </description>
+ </request>
+ </interface>
+</protocol>
--
2.14.4
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Dorota Czaplejewicz
2018-07-29 17:55:01 UTC
Permalink
On Tue, 24 Jul 2018 15:10:29 +0200
Post by Dorota Czaplejewicz
On Thu, 12 Jul 2018 18:15:32 -0400
Post by Simon Ser
Hi,
Sorry for the delay.
I'm not sure I like this new design.
Finally, I'm not even sure this security mechanism belongs here. I think adding
this mechanism to potentially all privileged protocols will result in duplicated
--- snip ---
Post by Dorota Czaplejewicz
Post by Simon Ser
* Allow other clients to use this interface too, and use another protocol to
manage authorizations. Basically the idea would be not to expose this
interface, and require the client to request access to this privileged
interface through an authorizer protocol. Someone already mentioned it, this
is Orbital's approach [2]. This allows the compositor to spawn a dialog asking
to the user "do you want to allow <client> to create a virtual keyboard?".
What do you think?
I'm in favor of option 3, which will be useful for other protocols as well, which are not necessarily in position to be kept in a permanent privileged process, e.g. screen recording.
It would also make me happier as a protocol author, because the protocol becomes more compact and easier to maintain without explicit managing.
Thinking a bit more about the topic, this patch with the authentication features removed would end up exactly the same as patch v3. Patch v3 is already implemented in wlroots too.

Hereby I withdraw this update. Please refer to 20180622152045.10563-1-***@puri.sm which can be read here: https://lists.freedesktop.org/archives/wayland-devel/2018-June/038629.html

Regards,
Dorota Czaplejewicz
Simon Ser
2018-07-30 08:13:15 UTC
Permalink
Post by Dorota Czaplejewicz
On Tue, 24 Jul 2018 15:10:29 +0200
Post by Dorota Czaplejewicz
On Thu, 12 Jul 2018 18:15:32 -0400
Post by Simon Ser
Hi,
Sorry for the delay.
I'm not sure I like this new design.
Finally, I'm not even sure this security mechanism belongs here. I think adding
this mechanism to potentially all privileged protocols will result in duplicated
--- snip ---
Post by Dorota Czaplejewicz
Post by Simon Ser
* Allow other clients to use this interface too, and use another protocol to
manage authorizations. Basically the idea would be not to expose this
interface, and require the client to request access to this privileged
interface through an authorizer protocol. Someone already mentioned it, this
is Orbital's approach [2]. This allows the compositor to spawn a dialog asking
to the user "do you want to allow <client> to create a virtual keyboard?".
What do you think?
I'm in favor of option 3, which will be useful for other protocols as well, which are not necessarily in position to be kept in a permanent privileged process, e.g. screen recording.
It would also make me happier as a protocol author, because the protocol becomes more compact and easier to maintain without explicit managing.
Thinking a bit more about the topic, this patch with the authentication features removed would end up exactly the same as patch v3. Patch v3 is already implemented in wlroots too.
Hi,

While the interfaces, requests and events are the same, there are some wording
changes from v3 to v4 and some enum reference fixes:

- added clarification to the destructor
- namespaced enum references

Is it possible to backport these changes to v3?

Thanks,

Simon
Post by Dorota Czaplejewicz
Regards,
Dorota Czaplejewicz
Continue reading on narkive:
Loading...