Discussion:
[GSoC] Some questions about wayland clients with no outputs present
(too old to reply)
Armin Krezović
2016-04-30 15:20:09 UTC
Permalink
Hi all,

Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).

For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.

For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.

I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.

Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.

As I've originally proposed, I see two different ways in achieving this:

First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.

Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?

The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.

The second question then might be: What should clients do in such case?

1. Do they only need to schedule redraw and toggle back fullscreen if it was on?

2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).

3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?

4. Is there anything special needed for XWayland or is it just another wayland client?

That would be all for now.

Armin.
Bryce Harrington
2016-04-30 20:14:13 UTC
Permalink
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
It happens that the x11 backend is able to run with 0 outputs. Perhaps
it would be simpler rather than defaulting to virtual outputs to just
have the drm backend also run with 0. I suppose in that case it would
also skip the rendering paths.

But virtual outputs is not a bad idea, it could be useful for testing or
perhaps to extend one's rendering beyond ordinary hardware capabilities
or something. But it should be user-controlled via commandline options
or config file options.

Keep in mind also potential security issues if we render clients to
outputs the user isn't aware of or able to actually see.
Post by Armin Krezović
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
I don't think so. When the screen blanking kicks in, the weston
compositor actually removes all the client surfaces. I would guess
something analogous could be done here. The client surfaces simply
become a state where rendering isn't happening.

I wrote up some notes on details of how the screen blanking functions in
detail in my quest to understand how that works. I can send it to you
if you'd like; I'm wondering if perhaps the 0-output situation could be
considered analogous - the clients are still connected but nothing is
actively rendering anywhere. I suppose in a way, at least from a client
perspective, powering a monitor off vs. actually unplugging isn't all
that totally different.

I also posted a set of patches to allow clients to request their monitor
not be powered off (e.g. if the client is displaying a presentation or
movie). These patches provide for allowing outputs to individually come
and go. They're implementing this at a higher level than you'll be
working, but might give a useful perspective. I can send you the link
if you're interested.
Post by Armin Krezović
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
I would think this is not something the clients need to take action on.
E.g. for the screen blanking, when an output becomes inactive the client
doesn't need to watch for that, the compositor simply stops directing
them to render.

The slight but crucial difference, however, is that when a screensaver
kicks on, you don't want the clients to rearrange or move around. You
want everything to come back exactly as it was; whereas with output
unplugging, you probably *do* want things rearranged. Unfortunately the
tricky thing there is that how to rearrange things is going to be a
policy thing. I should imagine different *shell* implementations could
desire setting different types of settings. Neither the backends nor
the clients should be making that policy, just the shells.

The fullscreen shell is simplest - there's just one client so it is just
resized to the new number of outputs, or if 0 outputs just stops
rendering, and restores when some output gets attached. The desktop
shell perhaps should force clients to be moved off of the removed
output(s), maybe just to some randomly chosen spot. Extra snazzy would
be if it could rearrange the clients to look nice on the remaining
outputs (but that's almost certainly beyond scope for your project).
Other shells like IVI would probably have their own desired behavioral
policies...

I would imagine that Weston would want to exhibit the same general
behavior when outputs are changed, for any backend. So if you're
running desktop shell on the drm, x11, headless or other backends and
remove one output, then whatever happens as a result should always
the same result regardless of backend.

So, the backends should indicate to the compositor (and thence to the
enabled shell) that the output change has occurred, and then the shell
has the responsibility to rearrange the client surfaces as it wishes.
Perhaps you'd like to focus your scope on the backend side of things,
make sure the communication path *to* the shell is all working, but
leave the layout logic as outside scope to be done as followup work.
Post by Armin Krezović
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
Hope my ramblings above stumbled into an answer for this.
Post by Armin Krezović
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
#2 and #3 are Weston shell policy, so there won't be a single answer;
potentially each shell could do it differently. Again, I suggest
focusing just on making the underpinnings work right, and maybe pick one
shell (likely the desktop shell but pq might prefer you focus on IVI or
something) as a demo/reference implementation.
Post by Armin Krezović
4. Is there anything special needed for XWayland or is it just another wayland client?
I would suggest leaving Xwayland outside scope for your project, I
think you're going to have plenty on your plate as it is.

Bryce
Armin Krezović
2016-05-02 10:49:04 UTC
Permalink
Post by Bryce Harrington
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
It happens that the x11 backend is able to run with 0 outputs. Perhaps
it would be simpler rather than defaulting to virtual outputs to just
have the drm backend also run with 0. I suppose in that case it would
also skip the rendering paths.
Thanks for the info, I'll take a look at the X11 backend.
Post by Bryce Harrington
But virtual outputs is not a bad idea, it could be useful for testing or
perhaps to extend one's rendering beyond ordinary hardware capabilities
or something. But it should be user-controlled via commandline options
or config file options.
I was thinking about using the virtual outputs when no physical outputs
are present, and keep them out of the way otherwise. Sure, a command
line switch/config option is useful, especially for the test suite (Pekka
proposed something similar).
Post by Bryce Harrington
Keep in mind also potential security issues if we render clients to
outputs the user isn't aware of or able to actually see.
See above.
Post by Bryce Harrington
Post by Armin Krezović
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
I don't think so. When the screen blanking kicks in, the weston
compositor actually removes all the client surfaces. I would guess
something analogous could be done here. The client surfaces simply
become a state where rendering isn't happening.
I wrote up some notes on details of how the screen blanking functions in
detail in my quest to understand how that works. I can send it to you
if you'd like; I'm wondering if perhaps the 0-output situation could be
considered analogous - the clients are still connected but nothing is
actively rendering anywhere. I suppose in a way, at least from a client
perspective, powering a monitor off vs. actually unplugging isn't all
that totally different.
Yes, please send me the notes you have. I welcome anything that may
help me achieve my goal.
Post by Bryce Harrington
I also posted a set of patches to allow clients to request their monitor
not be powered off (e.g. if the client is displaying a presentation or
movie). These patches provide for allowing outputs to individually come
and go. They're implementing this at a higher level than you'll be
working, but might give a useful perspective. I can send you the link
if you're interested.
I don't think it's related, but sure, why not. As noted above, anything
is welcome that helps me implement this feature.
Post by Bryce Harrington
Post by Armin Krezović
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
I would think this is not something the clients need to take action on.
E.g. for the screen blanking, when an output becomes inactive the client
doesn't need to watch for that, the compositor simply stops directing
them to render.
Hm, thanks for the info. I kinda got confused between desktop-shell/shell.c
and clients/desktop-shell.c here, which is what lead me to this conclusion.

Now, to confirm, the shell is the X window manager equivalent of the wayland
world? Is it responsible for placing windows and moving them around? If so,
my work has suddenly become way more easier.
Post by Bryce Harrington
The slight but crucial difference, however, is that when a screensaver
kicks on, you don't want the clients to rearrange or move around. You
want everything to come back exactly as it was; whereas with output
unplugging, you probably *do* want things rearranged. Unfortunately the
tricky thing there is that how to rearrange things is going to be a
policy thing. I should imagine different *shell* implementations could
desire setting different types of settings. Neither the backends nor
the clients should be making that policy, just the shells.
The fullscreen shell is simplest - there's just one client so it is just
resized to the new number of outputs, or if 0 outputs just stops
rendering, and restores when some output gets attached. The desktop
shell perhaps should force clients to be moved off of the removed
output(s), maybe just to some randomly chosen spot. Extra snazzy would
be if it could rearrange the clients to look nice on the remaining
outputs (but that's almost certainly beyond scope for your project).
Other shells like IVI would probably have their own desired behavioral
policies...
This is why I asked what should be done on output unplugging. I see that
window placement isn't that important, but I'd still like to try to
arrange the windows symetrical to the unplugged screen. I am unsure if
they should be scaled/resized in case of, ie, moving from a screen with
bigger resolution to a screen with smaller one. What do other
compositors/shells do?
Post by Bryce Harrington
I would imagine that Weston would want to exhibit the same general
behavior when outputs are changed, for any backend. So if you're
running desktop shell on the drm, x11, headless or other backends and
remove one output, then whatever happens as a result should always
the same result regardless of backend.
So, the backends should indicate to the compositor (and thence to the
enabled shell) that the output change has occurred, and then the shell
has the responsibility to rearrange the client surfaces as it wishes.
Perhaps you'd like to focus your scope on the backend side of things,
make sure the communication path *to* the shell is all working, but
leave the layout logic as outside scope to be done as followup work.
I think that desktop-shell already supports rendering on multiple outputs
(as confirmed by plugging in a second monitor) and stopping any rendering
from a certain output when it gets removed. It still needs to properly handle
screen blanking and screensaver when all outputs are gone. The things will
change when I start working on second part of my GSoC, which involves
extended screens.
Post by Bryce Harrington
Post by Armin Krezović
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
Hope my ramblings above stumbled into an answer for this.
Post by Armin Krezović
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
#2 and #3 are Weston shell policy, so there won't be a single answer;
potentially each shell could do it differently. Again, I suggest
focusing just on making the underpinnings work right, and maybe pick one
shell (likely the desktop shell but pq might prefer you focus on IVI or
something) as a demo/reference implementation.
As noted above, I'd like for clients to be on exactly similar spot on the
new screen, even after all outputs are gone. This might require preserving
client layout somewhere, so would it be okay?

As for the shell choice, I haven't worked on IVI shell yet, or even used it.
I'm more familar with the desktop shell, but I may need to ask more questions
about some terms used in the implementation that I don't yet understand.
Post by Bryce Harrington
Post by Armin Krezović
4. Is there anything special needed for XWayland or is it just another wayland client?
I would suggest leaving Xwayland outside scope for your project, I
think you're going to have plenty on your plate as it is.
Fair enough.
Post by Bryce Harrington
Bryce
Thanks for taking your time to respond. It means a lot.

Armin.
Pekka Paalanen
2016-05-02 13:42:52 UTC
Permalink
On Mon, 2 May 2016 12:49:04 +0200
Post by Armin Krezović
Post by Bryce Harrington
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
It happens that the x11 backend is able to run with 0 outputs. Perhaps
it would be simpler rather than defaulting to virtual outputs to just
have the drm backend also run with 0. I suppose in that case it would
also skip the rendering paths.
Hi,

yes, I would much prefer dealing with zero outputs.
Post by Armin Krezović
Thanks for the info, I'll take a look at the X11 backend.
Post by Bryce Harrington
But virtual outputs is not a bad idea, it could be useful for testing or
perhaps to extend one's rendering beyond ordinary hardware capabilities
or something. But it should be user-controlled via commandline options
or config file options.
I was thinking about using the virtual outputs when no physical outputs
are present, and keep them out of the way otherwise. Sure, a command
line switch/config option is useful, especially for the test suite (Pekka
proposed something similar).
Let's keep virtual/fake outputs contained in the headless backend for
the time being.
Post by Armin Krezović
Post by Bryce Harrington
Keep in mind also potential security issues if we render clients to
outputs the user isn't aware of or able to actually see.
See above.
Post by Bryce Harrington
Post by Armin Krezović
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
I don't think so. When the screen blanking kicks in, the weston
compositor actually removes all the client surfaces. I would guess
something analogous could be done here. The client surfaces simply
become a state where rendering isn't happening.
I wrote up some notes on details of how the screen blanking functions in
detail in my quest to understand how that works. I can send it to you
if you'd like; I'm wondering if perhaps the 0-output situation could be
considered analogous - the clients are still connected but nothing is
actively rendering anywhere. I suppose in a way, at least from a client
perspective, powering a monitor off vs. actually unplugging isn't all
that totally different.
Yes, please send me the notes you have. I welcome anything that may
help me achieve my goal.
Post by Bryce Harrington
I also posted a set of patches to allow clients to request their monitor
not be powered off (e.g. if the client is displaying a presentation or
movie). These patches provide for allowing outputs to individually come
and go. They're implementing this at a higher level than you'll be
working, but might give a useful perspective. I can send you the link
if you're interested.
I don't think it's related, but sure, why not. As noted above, anything
is welcome that helps me implement this feature.
Post by Bryce Harrington
Post by Armin Krezović
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
I would think this is not something the clients need to take action on.
E.g. for the screen blanking, when an output becomes inactive the client
doesn't need to watch for that, the compositor simply stops directing
them to render.
Hm, thanks for the info. I kinda got confused between desktop-shell/shell.c
and clients/desktop-shell.c here, which is what lead me to this conclusion.
Now, to confirm, the shell is the X window manager equivalent of the wayland
world? Is it responsible for placing windows and moving them around? If so,
my work has suddenly become way more easier.
The shell plugin part, yes. The shell plugin is the window manager. The
shell helper client (clients/desktop-shell.c) is an implementation
detail of the shell plugin, where we move all drawing etc. operations
into another process. We keep weston mostly clean of any drawing code
this way, we avoid adding API for compositor-internal drawing and
buffer allocation, and we avoid stalling the compositor in drawing
operations.
Post by Armin Krezović
Post by Bryce Harrington
The slight but crucial difference, however, is that when a screensaver
kicks on, you don't want the clients to rearrange or move around. You
want everything to come back exactly as it was; whereas with output
unplugging, you probably *do* want things rearranged. Unfortunately the
tricky thing there is that how to rearrange things is going to be a
policy thing. I should imagine different *shell* implementations could
desire setting different types of settings. Neither the backends nor
the clients should be making that policy, just the shells.
Exactly.
Post by Armin Krezović
Post by Bryce Harrington
The fullscreen shell is simplest - there's just one client so it is just
resized to the new number of outputs, or if 0 outputs just stops
rendering, and restores when some output gets attached. The desktop
shell perhaps should force clients to be moved off of the removed
output(s), maybe just to some randomly chosen spot. Extra snazzy would
be if it could rearrange the clients to look nice on the remaining
outputs (but that's almost certainly beyond scope for your project).
Other shells like IVI would probably have their own desired behavioral
policies...
This is why I asked what should be done on output unplugging. I see that
window placement isn't that important, but I'd still like to try to
arrange the windows symetrical to the unplugged screen. I am unsure if
they should be scaled/resized in case of, ie, moving from a screen with
bigger resolution to a screen with smaller one. What do other
compositors/shells do?
The first thing with Weston is to keep the code simple. shell.c might
be one of the messiest files in Weston, so there is a lot going on
anyway.

What to do is compositor policy. The important thing is to maintain
user's access to all windows, so the very least they would need moving.
That might even be implemented already. A nice bonus would be to resize
the windows down to have them completely visible.

Btw. don't do scaling. Just do a one-shot compositor-initiated resize,
and the clients will redraw in the new size. The shell protocol
specifications tell you what could be done and what might happen.
Post by Armin Krezović
Post by Bryce Harrington
I would imagine that Weston would want to exhibit the same general
behavior when outputs are changed, for any backend. So if you're
running desktop shell on the drm, x11, headless or other backends and
remove one output, then whatever happens as a result should always
the same result regardless of backend.
Yes, that is how it must be.
Post by Armin Krezović
Post by Bryce Harrington
So, the backends should indicate to the compositor (and thence to the
enabled shell) that the output change has occurred, and then the shell
has the responsibility to rearrange the client surfaces as it wishes.
Perhaps you'd like to focus your scope on the backend side of things,
make sure the communication path *to* the shell is all working, but
leave the layout logic as outside scope to be done as followup work.
I think that desktop-shell already supports rendering on multiple outputs
(as confirmed by plugging in a second monitor) and stopping any rendering
from a certain output when it gets removed. It still needs to properly handle
screen blanking and screensaver when all outputs are gone. The things will
change when I start working on second part of my GSoC, which involves
extended screens.
Seems like you are talking about weston-desktop-shell the client.
Indeed, fairly easy to confirm with e.g. the x11-backend by letting
Weston create more than one X11 window.

I forget if it was the Weston core or desktop shell plugin that
triggers blanking and powersave, and screensaver was removed some time
ago.
Post by Armin Krezović
Post by Bryce Harrington
Post by Armin Krezović
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
Hope my ramblings above stumbled into an answer for this.
Post by Armin Krezović
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
#2 and #3 are Weston shell policy, so there won't be a single answer;
potentially each shell could do it differently. Again, I suggest
focusing just on making the underpinnings work right, and maybe pick one
shell (likely the desktop shell but pq might prefer you focus on IVI or
something) as a demo/reference implementation.
Let us focus primarily on the desktop shell indeed. Fixing and testing
the other shells is completely secondary, apart from checking they are
not broken in the process.
Post by Armin Krezović
As noted above, I'd like for clients to be on exactly similar spot on the
new screen, even after all outputs are gone. This might require preserving
client layout somewhere, so would it be okay?
Yeah in the case where the last output is unplugged, and then later a
new first output is plugged in. In that case it would make very good
sense to maintain window positions if possible. Only if any window
would end up completely outside of the visible region, then it should
be moved.

Preserving the exact layout should be simple: do nothing.

Resizing is up to you, but don't make it too complicated. We need to
review and maintain that code, and desktop shell code is not exactly in
the prime focus of weston's goals. :-)
Post by Armin Krezović
As for the shell choice, I haven't worked on IVI shell yet, or even used it.
I'm more familar with the desktop shell, but I may need to ask more questions
about some terms used in the implementation that I don't yet understand.
If we would be putting the shells in order of importance for this
project, I would say:
1. desktop shell
2. fullscreen shell
3. ivi-shell

You are welcome to ignore all except desktop shell. However, if you
change Weston internal APIs, you are expected to migrate all users of
it.
Post by Armin Krezović
Post by Bryce Harrington
Post by Armin Krezović
4. Is there anything special needed for XWayland or is it just another wayland client?
I would suggest leaving Xwayland outside scope for your project, I
think you're going to have plenty on your plate as it is.
Fair enough.
Yup.


Thanks,
pq
Bryce Harrington
2016-05-03 21:11:07 UTC
Permalink
Post by Pekka Paalanen
On Mon, 2 May 2016 12:49:04 +0200
Post by Armin Krezović
Post by Bryce Harrington
So, the backends should indicate to the compositor (and thence to the
enabled shell) that the output change has occurred, and then the shell
has the responsibility to rearrange the client surfaces as it wishes.
Perhaps you'd like to focus your scope on the backend side of things,
make sure the communication path *to* the shell is all working, but
leave the layout logic as outside scope to be done as followup work.
I think that desktop-shell already supports rendering on multiple outputs
(as confirmed by plugging in a second monitor) and stopping any rendering
from a certain output when it gets removed. It still needs to properly handle
screen blanking and screensaver when all outputs are gone. The things will
change when I start working on second part of my GSoC, which involves
extended screens.
Seems like you are talking about weston-desktop-shell the client.
Indeed, fairly easy to confirm with e.g. the x11-backend by letting
Weston create more than one X11 window.
I forget if it was the Weston core or desktop shell plugin that
triggers blanking and powersave, and screensaver was removed some time
ago.
Blanking and powersave is all in Weston core. The fade in/out animation
stuff is in the desktop shell.

Bryce

Bryce Harrington
2016-05-03 00:33:14 UTC
Permalink
Post by Armin Krezović
Post by Bryce Harrington
But virtual outputs is not a bad idea, it could be useful for testing or
perhaps to extend one's rendering beyond ordinary hardware capabilities
or something. But it should be user-controlled via commandline options
or config file options.
I was thinking about using the virtual outputs when no physical outputs
are present, and keep them out of the way otherwise. Sure, a command
line switch/config option is useful, especially for the test suite (Pekka
proposed something similar).
I wonder what situation would arise where the user would want to use the
drm backend when there weren't any physical outputs. I can see that for
testing purposes but am drawing a blank for use cases other than that.
Post by Armin Krezović
Post by Bryce Harrington
Post by Armin Krezović
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
I would think this is not something the clients need to take action on.
E.g. for the screen blanking, when an output becomes inactive the client
doesn't need to watch for that, the compositor simply stops directing
them to render.
Hm, thanks for the info. I kinda got confused between desktop-shell/shell.c
and clients/desktop-shell.c here, which is what lead me to this conclusion.
Now, to confirm, the shell is the X window manager equivalent of the wayland
world? Is it responsible for placing windows and moving them around? If so,
my work has suddenly become way more easier.
Pretty much yeah. The core of Weston doesn't make assumptions about how
windows are laid out and stuff - it provides an abstract compositing
core. It has multiple different "shells" that sort of wrapper the core
and implement the GUI management policy particulars. I have a few notes
on this too which I'll shoot your way.
Post by Armin Krezović
Post by Bryce Harrington
The slight but crucial difference, however, is that when a screensaver
kicks on, you don't want the clients to rearrange or move around. You
want everything to come back exactly as it was; whereas with output
unplugging, you probably *do* want things rearranged. Unfortunately the
tricky thing there is that how to rearrange things is going to be a
policy thing. I should imagine different *shell* implementations could
desire setting different types of settings. Neither the backends nor
the clients should be making that policy, just the shells.
The fullscreen shell is simplest - there's just one client so it is just
resized to the new number of outputs, or if 0 outputs just stops
rendering, and restores when some output gets attached. The desktop
shell perhaps should force clients to be moved off of the removed
output(s), maybe just to some randomly chosen spot. Extra snazzy would
be if it could rearrange the clients to look nice on the remaining
outputs (but that's almost certainly beyond scope for your project).
Other shells like IVI would probably have their own desired behavioral
policies...
This is why I asked what should be done on output unplugging. I see that
window placement isn't that important, but I'd still like to try to
arrange the windows symetrical to the unplugged screen. I am unsure if
they should be scaled/resized in case of, ie, moving from a screen with
bigger resolution to a screen with smaller one. What do other
compositors/shells do?
For Weston I don't think anyone has spec'd that out; I couldn't speak
for other compositors - quite likely you can find some good guidance
there. You might check if there's a design doc for Gnome Shell or KDE.
I know a very detailed and thorough one was done for Canonical's Unity
but it no longer appears to be public (I can't find it anyway).

Your mentors may have more specific guidance on all of this.

Bryce
Post by Armin Krezović
Post by Bryce Harrington
I would imagine that Weston would want to exhibit the same general
behavior when outputs are changed, for any backend. So if you're
running desktop shell on the drm, x11, headless or other backends and
remove one output, then whatever happens as a result should always
the same result regardless of backend.
So, the backends should indicate to the compositor (and thence to the
enabled shell) that the output change has occurred, and then the shell
has the responsibility to rearrange the client surfaces as it wishes.
Perhaps you'd like to focus your scope on the backend side of things,
make sure the communication path *to* the shell is all working, but
leave the layout logic as outside scope to be done as followup work.
I think that desktop-shell already supports rendering on multiple outputs
(as confirmed by plugging in a second monitor) and stopping any rendering
from a certain output when it gets removed. It still needs to properly handle
screen blanking and screensaver when all outputs are gone. The things will
change when I start working on second part of my GSoC, which involves
extended screens.
Post by Bryce Harrington
Post by Armin Krezović
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
Hope my ramblings above stumbled into an answer for this.
Post by Armin Krezović
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
#2 and #3 are Weston shell policy, so there won't be a single answer;
potentially each shell could do it differently. Again, I suggest
focusing just on making the underpinnings work right, and maybe pick one
shell (likely the desktop shell but pq might prefer you focus on IVI or
something) as a demo/reference implementation.
As noted above, I'd like for clients to be on exactly similar spot on the
new screen, even after all outputs are gone. This might require preserving
client layout somewhere, so would it be okay?
As for the shell choice, I haven't worked on IVI shell yet, or even used it.
I'm more familar with the desktop shell, but I may need to ask more questions
about some terms used in the implementation that I don't yet understand.
Post by Bryce Harrington
Post by Armin Krezović
4. Is there anything special needed for XWayland or is it just another wayland client?
I would suggest leaving Xwayland outside scope for your project, I
think you're going to have plenty on your plate as it is.
Fair enough.
Post by Bryce Harrington
Bryce
Thanks for taking your time to respond. It means a lot.
Armin.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBCAAGBQJXJzCgAAoJEFFnYkKqnAm6r9kP/0j0VACKxVOYsZ8ucBS9k/p/
29vnB7XGb62ej88CsDlLBPJdydabUfBxNWtron6IyMBiIk4khWPqjKqu4ZUcvn59
P/gb0qaFOBgvC5XhAKCNnS87dDr9cNPCi0SBe6ntQ6Nw9RnsCPZ8K1AHeyjpKjsJ
8/rEPxJQXjuPtVFtpHNLBMHuo50AYgN9AbgIVBRNmwYuzKB0MkvQHr8aMcWDVCRG
8Ordg7kKD5AqMpKXXkIocwpuOmroXYDO7IedZdbO8JzDSynpTEYvBwTaH4WGIWSB
7Ob1AQQcbXzqvY0vfAFP0C9M03+rVs0DwRo2j83TVBZvOSRGSnV+6oVGdeTecRuh
QaFROr5wwvMLVPm3sWRDysavBkG5eTW28RxSajMKr6I1UUMlVP21itYcfFbujVer
FTP+2NCdvCroVj6Fo4bWPhUuggv3eGVkfkDAf/rN3NpvG4S2Uj8zOsofsJ7fZDdL
t/fwrYyECy8UjTURWiEBP+EfxMv6CpUd9NqJLVW/hUHKmDtVL+hdjfENzDV6dd+M
jPbWprXeyTgVpZdE/14AqeAaE986LiHsC5X0klvyMdiASJD2HdjTfTZZpITML5IO
SE2BlYzSs4P7gnTgTiz2+tvkSmSMz6/lU/6faOSN9qen/3dpwoH+cv1vHHhx0VNu
rRpOWUkwK2AnHsX15X35
=oZV6
-----END PGP SIGNATURE-----
Pekka Paalanen
2016-05-03 07:52:46 UTC
Permalink
On Mon, 2 May 2016 17:33:14 -0700
Post by Bryce Harrington
Post by Armin Krezović
Post by Bryce Harrington
But virtual outputs is not a bad idea, it could be useful for testing or
perhaps to extend one's rendering beyond ordinary hardware capabilities
or something. But it should be user-controlled via commandline options
or config file options.
I was thinking about using the virtual outputs when no physical outputs
are present, and keep them out of the way otherwise. Sure, a command
line switch/config option is useful, especially for the test suite (Pekka
proposed something similar).
I wonder what situation would arise where the user would want to use the
drm backend when there weren't any physical outputs. I can see that for
testing purposes but am drawing a blank for use cases other than that.
Say, you have your system running, and you want swap the monitor. Or
re-route the cable better. Or just close the laptop lid (I hear that
might cause the LVDS to disconnect). Or have a mechanical video switch
which causes the display to appear disconnected when not selected.

But none of that is a reason in itself to have a fake output.


Thanks,
pq
Hardening
2016-05-03 13:40:10 UTC
Permalink
Post by Pekka Paalanen
On Mon, 2 May 2016 17:33:14 -0700
Post by Bryce Harrington
Post by Armin Krezović
Post by Bryce Harrington
But virtual outputs is not a bad idea, it could be useful for testing or
perhaps to extend one's rendering beyond ordinary hardware capabilities
or something. But it should be user-controlled via commandline options
or config file options.
I was thinking about using the virtual outputs when no physical outputs
are present, and keep them out of the way otherwise. Sure, a command
line switch/config option is useful, especially for the test suite (Pekka
proposed something similar).
I wonder what situation would arise where the user would want to use the
drm backend when there weren't any physical outputs. I can see that for
testing purposes but am drawing a blank for use cases other than that.
Say, you have your system running, and you want swap the monitor. Or
re-route the cable better. Or just close the laptop lid (I hear that
might cause the LVDS to disconnect). Or have a mechanical video switch
which causes the display to appear disconnected when not selected.
But none of that is a reason in itself to have a fake output.
Funny, back 3 years ago I had the same questions with the RDP
compositor. My idea was to create the output (and the seat) when the
first client would connect (because the output would have the size
requested by the RDP peer). So if you manage to have 0 output working
everywhere else (I mean shell plugins), I'll probably consider have the
RDP compositor start with 0 outputs.

Best regards.
--
David FORT
website: http://www.hardening-consulting.com/
Eugen Friedrich
2016-04-30 20:19:45 UTC
Permalink
Hi Armin,

first of all I wish you existing and successful project experience!

About the missing output for clients, have just an idea: how about creating
some fake output object, maybe defined in weston.ini and send this to the
clients.
Client needs also the information that current output is a fake output in
order to decide what to do: just continue or wait for a real one.
Compositor could just pass the buffers that are arrived on the fake output
to the real one, ones its available.

Just an idea.
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
4. Is there anything special needed for XWayland or is it just another wayland client?
That would be all for now.
Armin.
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Armin Krezović
2016-05-02 10:53:00 UTC
Permalink
Post by Eugen Friedrich
Hi Armin,
Hi Eugen,
Post by Eugen Friedrich
first of all I wish you existing and successful project experience!
Thanks!
Post by Eugen Friedrich
About the missing output for clients, have just an idea: how about creating
some fake output object, maybe defined in weston.ini and send this to the
clients.
Client needs also the information that current output is a fake output in
order to decide what to do: just continue or wait for a real one.
Compositor could just pass the buffers that are arrived on the fake output
to the real one, ones its available.
As discussed with Bryce and Pekka, there could be a virtual output, no problems.

However, as Bryce explained in a post before yours, I don't think clients need
to know whether they're on virtual output or not. It's just the compositor/shell
that needs to know that and then tell clients to stop rendering to the fake
screen (either if it's the only one around or there are fake screen + one or more
physical outputs).
Post by Eugen Friedrich
Just an idea.
Much appreciated.

Armin.
Post by Eugen Friedrich
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
4. Is there anything special needed for XWayland or is it just another wayland client?
That would be all for now.
Armin.
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Pekka Paalanen
2016-05-02 11:55:00 UTC
Permalink
On Sat, 30 Apr 2016 22:19:45 +0200
Post by Eugen Friedrich
Hi Armin,
first of all I wish you existing and successful project experience!
About the missing output for clients, have just an idea: how about creating
some fake output object, maybe defined in weston.ini and send this to the
clients.
Client needs also the information that current output is a fake output in
order to decide what to do: just continue or wait for a real one.
Compositor could just pass the buffers that are arrived on the fake output
to the real one, ones its available.
Hi Eugen,

sorry, I don't understand what that would benefit.

Extending the Wayland core protocol to support and identify outputs as
fake, so that a client could... detect fake outputs, filter them
out from the output list, and then realize there are no real outputs
and act accordingly?

Clients (toolkits) already should deal with wl_output globals going
away or not existing in the first place. When the last output goes
away, it won't have any too radical effects: all wl_surfaces would
simply receive a leave event for the last output, saying the wl_surface
while still mapped is not actually on any output - it is off-screen. A
wl_surface can be off-screen for other reasons too.

A side-effect of a wl_surface being off-screen is that wl_surface.frame
callback won't be sent (so often), which automatically throttles
well-behaving clients' rendering to a halt (or reduced rate). Clients
that do not use the frame callback or Presentation feedback would be
better to observe the wl_surface.enter/leave events and stop rendering
when all outputs have been left.

Of course if a window was maximized on the output that went away or
something, the toolkit needs to respond somehow. Also compositor could
choose to unmaximize such windows first anyway and move then to
remaining outputs if any.

Buffers have no ties to an output. A wl_buffer is attached to a
wl_surface, and that wl_surface may be showing on any number of
outputs, including zero, at any number of copies.

wl_output also has no relation to a GPU (rendering device), and a
wl_buffer must be somehow usable (by the compositor) on all wl_output
that exist or might appear.

*If* there would be a fake output (a weston internal object), and *if*
that output is advertised to clients (wl_output), it would probably be
hot-plugged right before removing the last wl_output and removed right
after plugging the first real wl_output. But I don't think we should
have any fake outputs unless e.g. weston internals really can't work
without any, and fixing it would not be fitting the GSoC timescale.


Thanks,
pq
Post by Eugen Friedrich
Just an idea.
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
4. Is there anything special needed for XWayland or is it just another wayland client?
That would be all for now.
Armin.
_______________________________________________
wayland-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Pekka Paalanen
2016-05-02 12:50:57 UTC
Permalink
On Sat, 30 Apr 2016 17:20:09 +0200
Post by Armin Krezović
Hi all,
Now that I've analyzed the weston codebase a bit, I'm ready to
bombard you people with some questions :).
Hi Armin!
Post by Armin Krezović
For those that aren't yet aware, I'll be working on making weston
able to run with no outputs and survive with all outputs plugged
out at runtime.
For starters, I've simply commented out the output backend load code
in src/main.c to see what would happen. To my surprise, the compositor
didn't crash, but the desktop-shell did.
Heh, that was unexpected. :-)
Post by Armin Krezović
I'm currently guessing that it crashes because the clients that use
clients/window.c code look for outputs in the registry. Of course,
there are none because the backend that updates the global registry
isn't loaded.
Now, my conclusion is: In order to make weston work without any outputs
present is to make clients work without any outputs present. This will
probably require people to update their toolkits and apps.
Clients, that is toolkits really, should already be handling wl_output
global removals. It is possible they do not, but that is not your
concern. Let us focus on Weston particularly, and the demo clients in
weston repository. You should not care about any other toolkits for
now, they might make your GSoC project scope explode.

I explained some of the mechanisms that apps and toolkits should be
using already in my reply to Eugen. If they were implemented properly,
I would expect all clients to just work.
Post by Armin Krezović
First and easiest way is use of a virtual output. I could modify the
DRM backend to add a virtual output if no physical outputs are found
and remove the output when a physical output is connected. Add similar
behavior on phyisical screen unplug. If there are none left, add a virtual
output. If I remember correctly, Pekka once noted that DRM backend already
supports output hotplugging, more or less, but it doesn't support the
primary output unplugging.
I would say the DRM backend is the wrong place. Surely running with
zero outputs may apply to all backends, so the implementation should
live in Weston core.

However, there are some catches about manufacturing a weston_output
that is not created by the backend. I have seen some of that in my
experiments towards Weston IVI remoting.

I think we should keep the fake output design as plan B, and only in
case the plan A below is not possible to execute under the single GSoC
project.
Post by Armin Krezović
Now, the first question is: Even if this way is picked, would clients or
something else require much work to make them behave correctly when
the primary output is unplugged?
Considering it would never cause the number of wl_outputs to drop to
zero, they would be just fine if they expect output hot-unplugging to
occur at all. Dropping the number to zero might break a few more
assumptions, though I have hard time imagining what they could be.

The Wayland protocol implementation is safe anyway, wl_proxies cannot
suddenly disappear.
Post by Armin Krezović
The second and more harder way is to simply cope with no outputs present and
watch for output plugging/unplugging. This would require clients to watch for
output changes and take apropriate action.
This is the plan A and the approach to the tried first.
Post by Armin Krezović
The second question then might be: What should clients do in such case?
1. Do they only need to schedule redraw and toggle back fullscreen if it was on?
Maybe, though it doesn't really matter much. Also the compositor could
ask the window to un-fullscreen, at least with xdg-shell. wl_shell
might be a different case. The interesting question is what happens
when output comes back.

For a multi-output scenario, you could unplug an output where there are
windows, but it's not the last output. The case is the same from client
perspective.
Post by Armin Krezović
2. Do they need to resize/scale relative to the size of new screen? (eg, a maximized
app on 1920x1080 screen would not be entirely visible on 1600x900 one).
The compositor too could signal a resize.
Post by Armin Krezović
3. Is there something as "move an app to different screen" that can be issued
by compositor? Or do apps need to do that too?
There is nothing to be issued as such. It's only compositor-internal
state change, and clients will get notified via wl_surface.enter/leave
events.

What clients can do depends on the shell protocol extension, e.g.
xdg-shell. Normally only maximize and fullscreen window state can have
a designated output by the client. Otherwise it is all compositor's
discretion.
Post by Armin Krezović
4. Is there anything special needed for XWayland or is it just another wayland client?
XWayland probably cannot remove the last or primary output at all, but
you should not worry about that for now. If there is something to fix,
it must be fixed in XWayland (the X server - to keep a ghost output if
necessary).

It is good to investigate the client side problems too, but I think it
is getting too much weight for starters. When I wrote up on this
project, the main culprits I were thinking about in Weston were all the
code that does:

assert(!wl_list_empty(&compositor->output_list));
output = container_of(compositor->output_list.next,
struct weston_output, link);

or similar. That is, code that explodes if the output list is empty or
there cannot be a valid weston_output pointer. You can find it by
grepping for 'output_list'.

Some major issues are the functions weston_surface_is_mapped() and
weston_view_is_mapped(). I think we are finally going to need to add
booleans 'is_mapped' in weston_surface and weston_view structs. It is
possible that these are actually the source for almost all of the cases
where having no outputs would explode.


Thanks,
pq
Loading...