How to release the weston_buffer after weston_view is destroyed.

[Sichem Zhou]

Dear Weston devs:

Sorry for the bothering in when weston-5.0 is approaching, but I am stuck
in the project for three days I need some one's help very much.

I have a widget implementation of shell widgets that destroys the
weston_view
when pressed Esc. The widget is implemented in wayland EGL. Here is the
view_destroy code.

void
twshell_close_ui_surface(struct weston_surface *wd_surface)
{
struct weston_view *view, *next;
wd_surface->committed = NULL;
wd_surface->committed_private = NULL;
wl_list_for_each_safe(view, next, &wd_surface->views, surface_link)
{
weston_view_destroy(view);
}
}

The weston_view is destroyed but the the buffer in the last commit was not
released and the done event was not sent to the widget. When I triggered
the event to show widget again, the client was stuck in EGLSwapbuffer. I
found out about this by using WAYLAND_DEBUG env variable, I hope to the
way to release the buffer after umapping the weston view. Thanks very much.

Regards,
Sichem

[Sichem Zhou]

I actually found the reason, in the function `weston_output_repaint`, the
frame_callbacks
only send the done events for surface which has view on the compositor's
view_list,
but this poses a dilemma, since I cannot either destroy or unmap the view
before the
output repainted.

Sichem

Post by Sichem Zhou
Sorry for the bothering in when weston-5.0 is approaching, but I am stuck
in the project for three days I need some one's help very much.
I have a widget implementation of shell widgets that destroys the
weston_view
when pressed Esc. The widget is implemented in wayland EGL. Here is the
view_destroy code.
void
twshell_close_ui_surface(struct weston_surface *wd_surface)
{
struct weston_view *view, *next;
wd_surface->committed = NULL;
wd_surface->committed_private = NULL;
wl_list_for_each_safe(view, next, &wd_surface->views,
surface_link) {
weston_view_destroy(view);
}
}
The weston_view is destroyed but the the buffer in the last commit was not
released and the done event was not sent to the widget. When I triggered
the event to show widget again, the client was stuck in EGLSwapbuffer. I
found out about this by using WAYLAND_DEBUG env variable, I hope to the
way to release the buffer after umapping the weston view. Thanks very much.
Regards,
Sichem

[Sichem Zhou]

Dear all,

I solved it using the `frame_signal` in the weston_output. I referenced the
Weston shell `set_minimized` implementation, which works by moving view to
a hidden layer. It didn't work for me since I trigger immediately the next
frame once moving the view back to the original layer. The commit can
happen before receiving the last frame done. Which actually happened all
the time.

I hope this mail can also be helpful to others, I found a shell
implementation is really tricky.

Regards,
Sichem

Post by Sichem Zhou
I actually found the reason, in the function `weston_output_repaint`, the
frame_callbacks
only send the done events for surface which has view on the compositor's
view_list,
but this poses a dilemma, since I cannot either destroy or unmap the view
before the
output repainted.
Sichem

Post by Sichem Zhou
Sorry for the bothering in when weston-5.0 is approaching, but I am stuck
in the project for three days I need some one's help very much.
I have a widget implementation of shell widgets that destroys the
weston_view
when pressed Esc. The widget is implemented in wayland EGL. Here is the
view_destroy code.
void
twshell_close_ui_surface(struct weston_surface *wd_surface)
{
struct weston_view *view, *next;
wd_surface->committed = NULL;
wd_surface->committed_private = NULL;
wl_list_for_each_safe(view, next, &wd_surface->views,
surface_link) {
weston_view_destroy(view);
}
}
The weston_view is destroyed but the the buffer in the last commit was not
released and the done event was not sent to the widget. When I triggered
the event to show widget again, the client was stuck in EGLSwapbuffer. I
found out about this by using WAYLAND_DEBUG env variable, I hope to the
way to release the buffer after umapping the weston view. Thanks very much.
Regards,
Sichem

[Pekka Paalanen]

Post by Sichem Zhou

Post by Sichem Zhou
Sorry for the bothering in when weston-5.0 is approaching, but I am stuck
in the project for three days I need some one's help very much.
I have a widget implementation of shell widgets that destroys the
weston_view
when pressed Esc. The widget is implemented in wayland EGL. Here is the
view_destroy code.
void
twshell_close_ui_surface(struct weston_surface *wd_surface)
{
struct weston_view *view, *next;
wd_surface->committed = NULL;
wd_surface->committed_private = NULL;
wl_list_for_each_safe(view, next, &wd_surface->views,
surface_link) {
weston_view_destroy(view);
}
}
The weston_view is destroyed but the the buffer in the last commit was not
released and the done event was not sent to the widget. When I triggered
the event to show widget again, the client was stuck in EGLSwapbuffer. I
found out about this by using WAYLAND_DEBUG env variable, I hope to the
way to release the buffer after umapping the weston view. Thanks very much.

Hi,

destroying or unmapping a weston_view is not meant to release the
wl_surface's buffer. If the compositor spontaneously chooses to hide a
surface, it may also spontaneously choose to show it again. To show a
surface, it must have content, so if the renderer needs the buffer
around, hiding cannot release the buffer.

Views are Weston internal objects, clients are completely unaware of
them. However, clients are aware of the mappedness of a wl_surface
(weston_surface). Clients issue protocol requests and assume a specific
sequence will map the surface. A compositor must keep up that
appearance, even if it was a lie internally. That is, if a compositor
spontaneously unmaps the view, it will not make the surface unmapped.
This is also why we have 'bool is_mapped' in both weston_surface and
weston_view. The client still thinks the surface is mapped, so it will
not even expect the buffer to be released.

The only client-visible effect of unmapping all views is that the frame
callbacks will pause. This does not mean that the surface is unmapped,
it is just temporarily not visible. Frame callbacks are used to
throttle clients so that they don't draw frames that will not be shown.

If you don't forcefully release the buffer, re-creating the view should
let your app continue as usual while Weston is showing the last frame
it got.

It is also intentional that frame callback 'done' events are only ever
triggered from repaint. Repaint makes the frame visible. As long as the
frame is not processed yet, 'done' should not be sent.

'done' means that it is a good time for a continuously animating client
to start drawing the next frame. It is an advisory throttling
mechanism, and if your client really knows better, it can choose to
draw new frames regardless.

When you use EGL in a client, is it important to never let
eglSwapBuffers() block for you. You need to track frame callbacks
manually or set eglSwapInterval to zero, preferably both, so your
application code does not get blocked by EGL API that was designed for
applications far less smart than yours.


On Sun, 29 Jul 2018 20:49:24 -0400

Post by Sichem Zhou
I actually found the reason, in the function `weston_output_repaint`, the
frame_callbacks
only send the done events for surface which has view on the compositor's
view_list,
but this poses a dilemma, since I cannot either destroy or unmap the view
before the
output repainted.

I don't see a problem on the compositor side. You can unmap or destroy
the view, and the frame callbacks will stall until you re-map the view
again. The last buffer will not get released while unmapped, because
the compositor needs it on re-map. The pending frame callbacks will
resume once the view is re-mapped.

It sounds like your problem is in the app: do not let eglSwapBuffers()
block on a frame callback. Create a frame callback just before you call
eglSwapBuffers(), and do not call eglSwapBuffers() again until that
frame callback has signalled. That way eglSwapBuffers should never
block indefinitely. Additionally you can set eglSwapInterval to zero in
case there are situations where you really need to eglSwapBuffers()
before the frame callback has signalled.


Thanks,
pq

[Sichem Zhou]

Hi Pekka,

Thanks for your reply. Yes, preferably if I can hide the map and unmap from
client, in this case I cannot because it is part of a desktop shell. After
I unmap it and remap the view next time, it is not expected the shell to
continue from last draw. I would want the shell be drawing total different
things instead. If I simply remap, I shall see the last frame that did get
drawed.

It can be seen as a problem in the application, designing smarter clients
which is aware of such details, by not sending the last frame for example,
but I found an easier solution by unmap the view in the `frame_signal`. It
worked quite well.

Regards,
Sichem

Post by Sichem Zhou

Post by Sichem Zhou

Post by Sichem Zhou
Sorry for the bothering in when weston-5.0 is approaching, but I am

stuck

Post by Sichem Zhou

Post by Sichem Zhou
in the project for three days I need some one's help very much.
I have a widget implementation of shell widgets that destroys the
weston_view
when pressed Esc. The widget is implemented in wayland EGL. Here is the
view_destroy code.
void
twshell_close_ui_surface(struct weston_surface *wd_surface)
{
struct weston_view *view, *next;
wd_surface->committed = NULL;
wd_surface->committed_private = NULL;
wl_list_for_each_safe(view, next, &wd_surface->views,
surface_link) {
weston_view_destroy(view);
}
}
The weston_view is destroyed but the the buffer in the last commit was

not

Post by Sichem Zhou

Post by Sichem Zhou
released and the done event was not sent to the widget. When I

triggered

Post by Sichem Zhou

Post by Sichem Zhou
the event to show widget again, the client was stuck in EGLSwapbuffer.

I

Post by Sichem Zhou

Post by Sichem Zhou
found out about this by using WAYLAND_DEBUG env variable, I hope to the
way to release the buffer after umapping the weston view. Thanks very

much.
Hi,
destroying or unmapping a weston_view is not meant to release the
wl_surface's buffer. If the compositor spontaneously chooses to hide a
surface, it may also spontaneously choose to show it again. To show a
surface, it must have content, so if the renderer needs the buffer
around, hiding cannot release the buffer.
Views are Weston internal objects, clients are completely unaware of
them. However, clients are aware of the mappedness of a wl_surface
(weston_surface). Clients issue protocol requests and assume a specific
sequence will map the surface. A compositor must keep up that
appearance, even if it was a lie internally. That is, if a compositor
spontaneously unmaps the view, it will not make the surface unmapped.
This is also why we have 'bool is_mapped' in both weston_surface and
weston_view. The client still thinks the surface is mapped, so it will
not even expect the buffer to be released.
The only client-visible effect of unmapping all views is that the frame
callbacks will pause. This does not mean that the surface is unmapped,
it is just temporarily not visible. Frame callbacks are used to
throttle clients so that they don't draw frames that will not be shown.
If you don't forcefully release the buffer, re-creating the view should
let your app continue as usual while Weston is showing the last frame
it got.
It is also intentional that frame callback 'done' events are only ever
triggered from repaint. Repaint makes the frame visible. As long as the
frame is not processed yet, 'done' should not be sent.
'done' means that it is a good time for a continuously animating client
to start drawing the next frame. It is an advisory throttling
mechanism, and if your client really knows better, it can choose to
draw new frames regardless.
When you use EGL in a client, is it important to never let
eglSwapBuffers() block for you. You need to track frame callbacks
manually or set eglSwapInterval to zero, preferably both, so your
application code does not get blocked by EGL API that was designed for
applications far less smart than yours.
On Sun, 29 Jul 2018 20:49:24 -0400

Post by Sichem Zhou
I actually found the reason, in the function `weston_output_repaint`, the
frame_callbacks
only send the done events for surface which has view on the compositor's
view_list,
but this poses a dilemma, since I cannot either destroy or unmap the view
before the
output repainted.

I don't see a problem on the compositor side. You can unmap or destroy
the view, and the frame callbacks will stall until you re-map the view
again. The last buffer will not get released while unmapped, because
the compositor needs it on re-map. The pending frame callbacks will
resume once the view is re-mapped.
It sounds like your problem is in the app: do not let eglSwapBuffers()
block on a frame callback. Create a frame callback just before you call
eglSwapBuffers(), and do not call eglSwapBuffers() again until that
frame callback has signalled. That way eglSwapBuffers should never
block indefinitely. Additionally you can set eglSwapInterval to zero in
case there are situations where you really need to eglSwapBuffers()
before the frame callback has signalled.
Thanks,
pq

[Pekka Paalanen]

On Fri, 3 Aug 2018 18:04:36 -0400

Post by Sichem Zhou
Hi Pekka,
Thanks for your reply. Yes, preferably if I can hide the map and unmap from
client, in this case I cannot because it is part of a desktop shell. After
I unmap it and remap the view next time, it is not expected the shell to
continue from last draw. I would want the shell be drawing total different
things instead. If I simply remap, I shall see the last frame that did get
drawed.
It can be seen as a problem in the application, designing smarter clients
which is aware of such details, by not sending the last frame for example,
but I found an easier solution by unmap the view in the `frame_signal`. It
worked quite well.

Hi,

if you need the unmap to happen immediately without round-tripping to
the helper client, I think a good solution for your case would be for
the desktop-shell to unmap the surface and send an event to the helper
client telling that the surface is unmapped. That way the client can
destroy the wl_surface, which will make the compositor automatically
release buffers it might have held, and there is no chance of showing
outdated content on the next time.

In my opinion, the frame_signal solution is a hack. It is not really
meant for what you are doing it with it, and it does not allow the
compositor to release buffers it would not have released anyway.

It also does not guarantee that the client would not block inside
eglSwapBuffers, you have to prevent that in the client anyway. Usually
unmapping a window permanently is initiated by the client by e.g.
destroying the wl_surface, which naturally avoids an indefinite wait
for a frame callback. That could be a response to a specific event.

With the frame_signal hack, even if you unmap the view after sending
out the frame callbacks, what will guarantee that the client will not
post another frame as a response to the frame callback you just sent?
And then another frame after that would again hit the indefinite
blocking inside eglSwapBuffers(). I suspect your design is quite
fragile as is.


Thanks,
pq

[Sichem Zhou]

Hi Pekka,

Thanks for the reply. Sorry I forgot to say that `frame_callback` was a
hack for me. It has historical reason. Initially I designed the client
using EGL and found that I could only create the glprogram, creating
shaders after having wl_surface. Since I used the same shader I thought
that why creating new shaders every time when opening windows. So I decided
to reuse shaders, where it set up the potential trap for me. However I
found the `frame_callback` works really well for me, currently it is the
most natural way.

Regarding the second problem, the client doesn't really send another frame
immediately after receiving one. It renders based on input events, so it
doesn't really send any frame if nothing new to render. Again, I designed
it to save as much resources as I can. But thanks for your concern, I may
switch to destroying `wl_surface` approach at next refactoring.

Thanks,
Sichem

Post by Pekka Paalanen
On Fri, 3 Aug 2018 18:04:36 -0400

Post by Sichem Zhou
Hi Pekka,
Thanks for your reply. Yes, preferably if I can hide the map and unmap

from

Post by Sichem Zhou
client, in this case I cannot because it is part of a desktop shell.

After

Post by Sichem Zhou
I unmap it and remap the view next time, it is not expected the shell to
continue from last draw. I would want the shell be drawing total

different

Post by Sichem Zhou
things instead. If I simply remap, I shall see the last frame that did

get

Post by Sichem Zhou
drawed.
It can be seen as a problem in the application, designing smarter clients
which is aware of such details, by not sending the last frame for

example,

Post by Sichem Zhou
but I found an easier solution by unmap the view in the `frame_signal`.

It

Post by Sichem Zhou
worked quite well.

Hi,
if you need the unmap to happen immediately without round-tripping to
the helper client, I think a good solution for your case would be for
the desktop-shell to unmap the surface and send an event to the helper
client telling that the surface is unmapped. That way the client can
destroy the wl_surface, which will make the compositor automatically
release buffers it might have held, and there is no chance of showing
outdated content on the next time.
In my opinion, the frame_signal solution is a hack. It is not really
meant for what you are doing it with it, and it does not allow the
compositor to release buffers it would not have released anyway.
It also does not guarantee that the client would not block inside
eglSwapBuffers, you have to prevent that in the client anyway. Usually
unmapping a window permanently is initiated by the client by e.g.
destroying the wl_surface, which naturally avoids an indefinite wait
for a frame callback. That could be a response to a specific event.
With the frame_signal hack, even if you unmap the view after sending
out the frame callbacks, what will guarantee that the client will not
post another frame as a response to the frame callback you just sent?
And then another frame after that would again hit the indefinite
blocking inside eglSwapBuffers(). I suspect your design is quite
fragile as is.
Thanks,
pq

[Pekka Paalanen]

On Mon, 13 Aug 2018 09:33:53 -0400

Post by Sichem Zhou
Hi Pekka,
Thanks for the reply. Sorry I forgot to say that `frame_callback` was a
hack for me. It has historical reason. Initially I designed the client
using EGL and found that I could only create the glprogram, creating
shaders after having wl_surface. Since I used the same shader I thought
that why creating new shaders every time when opening windows. So I decided
to reuse shaders, where it set up the potential trap for me. However I
found the `frame_callback` works really well for me, currently it is the
most natural way.
Regarding the second problem, the client doesn't really send another frame
immediately after receiving one. It renders based on input events, so it
doesn't really send any frame if nothing new to render. Again, I designed
it to save as much resources as I can. But thanks for your concern, I may
switch to destroying `wl_surface` approach at next refactoring.

Hi,

ok, that explains. Did you have problems calling eglMakeCurrent()
without creating a wl_surface?

There is EGL_KHR_surfaceless_context extension that, if present, allows
calling eglMakeCurrent() with EGL_NO_SURFACE, so you can do all the GL
setup without a wl_surface.

You could also do the GL setup on the first show, and keep the shaders
etc. beyond the destruction of the wl_surface. I assume that should
work from EGL and GL point of view.


Thanks,
pq

Post by Sichem Zhou

Post by Pekka Paalanen
On Fri, 3 Aug 2018 18:04:36 -0400

Post by Sichem Zhou
Hi Pekka,
Thanks for your reply. Yes, preferably if I can hide the map and unmap

from

Post by Sichem Zhou
client, in this case I cannot because it is part of a desktop shell.

After

Post by Sichem Zhou
I unmap it and remap the view next time, it is not expected the shell to
continue from last draw. I would want the shell be drawing total

different

Post by Sichem Zhou
things instead. If I simply remap, I shall see the last frame that did

get

Post by Sichem Zhou
drawed.
It can be seen as a problem in the application, designing smarter clients
which is aware of such details, by not sending the last frame for

example,

Post by Sichem Zhou
but I found an easier solution by unmap the view in the `frame_signal`.

It

Post by Sichem Zhou
worked quite well.

Hi,
if you need the unmap to happen immediately without round-tripping to
the helper client, I think a good solution for your case would be for
the desktop-shell to unmap the surface and send an event to the helper
client telling that the surface is unmapped. That way the client can
destroy the wl_surface, which will make the compositor automatically
release buffers it might have held, and there is no chance of showing
outdated content on the next time.
In my opinion, the frame_signal solution is a hack. It is not really
meant for what you are doing it with it, and it does not allow the
compositor to release buffers it would not have released anyway.
It also does not guarantee that the client would not block inside
eglSwapBuffers, you have to prevent that in the client anyway. Usually
unmapping a window permanently is initiated by the client by e.g.
destroying the wl_surface, which naturally avoids an indefinite wait
for a frame callback. That could be a response to a specific event.
With the frame_signal hack, even if you unmap the view after sending
out the frame callbacks, what will guarantee that the client will not
post another frame as a response to the frame callback you just sent?
And then another frame after that would again hit the indefinite
blocking inside eglSwapBuffers(). I suspect your design is quite
fragile as is.
Thanks,
pq