17:37 riteo: Hi people, I'm in the process of adding "game embedding" to godot on wayland, which needs to hijack the running game's surface and slap it into the main editor window
17:38 riteo: from what I can tell I need to implement a mini compositor inside the editor and run the game through it
17:38 riteo: the question is, should I use wlroots for this? Is there a known path from there or should I figure out things as I go?
17:39 riteo: there's actually another very dumb idea I had which is to make a protocol proxy that intercepted new_id arguments and remapped them on-the-fly, all while running from the same connection of the editor
17:39 riteo: and then intercepting xdg-toplevel messages and "translating" them to subsurfaces
17:42 riteo: but I guess that the proper approach would be to implement a real compositor with some library, wouldn't it?
17:50 kchibisov: is it an arbitrary game? Generally one could just support subsurface or toplevel as main surface.
17:50 kchibisov: and when you embed you ask to draw on subsurface.
17:50 kchibisov: where toplevel and subsurface both wrapped.
17:51 riteo: they are arbitrary games yes, but they are guaranteed to run on godot engine, since it is a "preview" of the project you're modifying
17:51 kchibisov: I mean, then they are not arbitrary.
17:51 kchibisov: it's whatever you've created with godot.
17:51 riteo: yes
17:51 riteo: that's what I meant sorry
17:51 kchibisov: thus, you can just have enum { subsurface, toplevel } and pass an option when you embed.
17:52 riteo: thing is, how should I go forwards implementing this?
17:52 kchibisov: I mean, you have xdg_toplevel somewhere and drawing, you should wrap?
17:52 zamundaaa[m]: riteo: does the game run in the same process, or in a different one in this case?
17:52 emersion: riteo: wlroots has an "embedded" exemple fwiw
17:52 kchibisov: Like the end handle that will be passed to the graphics API will be the same.
17:52 riteo: zamundaaa: it's a different process
17:52 emersion: example*
17:52 kchibisov: ah, different process, then yeah, won't work.
17:52 zamundaaa[m]: Then you need to implement that nested compositor, no way around it
17:52 riteo: emersion: oh I missed that, it would help me considerably
17:53 riteo: zamundaaa: would proxying the protocol be too dumb of an approach?
17:53 riteo: I actually was playing around with the idea but it feels wrong while I wrote it
17:54 zamundaaa[m]: I think you'd just end up with a less reliable nested compositor if you do that
17:54 zamundaaa[m]: As the game may try to use protocols that only apply to xdg toplevels on the subsurface, and then get a protocol error for it
17:54 emersion: it uses a subsurface so that all interactions with the parent compositor are managed by wlroots - it's also possible to render to a buffer and then paint it yourself, but it's more work
17:54 kchibisov: I think it'll be fine actually?
17:54 kchibisov: The proxy approach.
17:55 riteo: I did not do any remapping but I could make a very dumb proxy in very little time and it seems to work surprisingly well, as long as it does not get protocol errors and put everything down :P
17:55 kchibisov: it works good for xwayland at least.
17:55 riteo: but yea I'm definitely gonna take a look at the embedded example anyways, thank you emersion :D
17:55 kchibisov: so don't see why proxing another game won't be good.
17:56 riteo: kchibisov: well xwayland stuff I think would need special treatment anyways
17:56 riteo: as in, I don't think I can intercept it
17:56 kchibisov: there's a wayland proxy to proxy xwayland.
17:56 riteo: uh
17:56 kchibisov: so you can have xwayland outside of compositor.
17:56 kchibisov: https://github.com/Supreeeme/xwayland-satellite
17:56 kchibisov: and not being rootful.
17:56 riteo: I see
17:57 riteo: I must admit, the whole reason I even thought of the proxy setup is that making a whole compositor sounds like a chore
17:58 kchibisov: I mean, if wlroots has a good API then probably will be fine.
17:58 riteo: but my rational part says that I should probably just really just bite the bullet
17:58 riteo: yea it has, looking at tinywl
17:58 riteo: it does _everything_ for you already
17:59 riteo: I just have to wire up some event handlers I think, and there's already an "embedded" example
17:59 kchibisov: but for this case, you need to hijack things a bit, which you know how they work.
17:59 kchibisov: since you embed only godot games.
18:00 riteo: sorry for the proxy case or the wlroots case
18:01 riteo: ok I'm sold, the embedded example i 244 lines long
18:02 riteo: biting the bullet it is
18:02 kchibisov: I mean, you pull entire compositor lib.
18:02 kennylevinsen: does Godot games use subsurfaces or just a single toplevel surface that is pre-composited?
18:02 riteo: kennylevinsen: they don't use subsurfaces
18:03 kchibisov: it's just a single surface, which is easy to proxy, from what I understood.
18:03 riteo: is the proxy solution really that viable
18:03 riteo: I mean I don't see why subsurfaces wouldn't be easy to proxy
18:03 riteo: I can just filter in-out interface I need to know about anyways to parse their signatures
18:04 kennylevinsen: then you just need to get a hold of the buffer - if you can integrate with the game you're previewing's Godot implementation, you could just have it send the buffer on whatever side channel you want
18:04 riteo: I mean I have a working simple proxy which does not do any remapping/embedding yet
18:04 riteo: kennylevinsen: yea but what about input and stuff?
18:04 kchibisov: just pass through.
18:04 riteo: I should've specified that other platforms cheat by teleporting the child window into the designated space
18:05 riteo: so there's no input passing mechanism already implemented
18:05 kchibisov: they are all in surface coords.
18:05 kennylevinsen: do you also want all input routed through the preview window? or is it fine that it is the game's main window for that?
18:05 riteo: I also need all input routed through there
18:05 riteo: it's basically like unity's game preview
18:05 riteo: where the game itself is spawned into a "preview" subwindow in the editor
18:05 kennylevinsen: so you don't expect there to be a "real" window, you only want the preview?
18:05 riteo: yes
18:06 riteo: that's why the proxy approach seems so alluring
18:07 riteo: I'm sorry if this became somewhat of a "please convince me that a proxy is a bad idea", there's very little previous art in regarding to wayland embedding compositors
18:08 kchibisov: cosmic-panel iirc
18:09 kchibisov: or was it something else, it's some cosmic element that does embedding iirc.
18:09 kennylevinsen: your options are: custom separate Godot backend (no Wayland), embedded compositor, or a stripped down embedded compositor that proxies a few things
18:10 kennylevinsen: the last one is a bit funky as you need to re-advertise globals, translate input coordinates, etc.
18:10 riteo: wait why do I have to translate input coordinates
18:10 riteo: it's all per-surface
18:11 kennylevinsen: hmm maybe it's fine if you use a subsurface
18:11 kennylevinsen: but you can't make a new display connection
18:11 riteo: that's the only bad thing with proxying I think
18:12 riteo: as if there's a protocol error from the game it all goes down
18:12 riteo: that's why I'm really 50/50 with option no.2 and option no.3
18:12 kennylevinsen: so you need to act as a Wayland server that implements all the relevant interfaces as calls to your own display connection
18:13 riteo: well I guess that's really proxying with more steps though, isn't it
18:13 kennylevinsen: and you'll have to mimick some xdg-shell stuff
18:13 kennylevinsen: yeah
18:14 riteo: so it's really either "make a whole compositor and blit the thing out" or "proxy everything and trust the client"...
18:14 riteo: I think there's no way around it
18:15 riteo: welp, the wlroots example opened another path, which is reassuring at least
18:15 kennylevinsen: well you can't proxy it directly - you don't have access to the connection buffer, and even if you did you'd have object ID collisions, and you want an xdg toplevel to become a subsurface
18:15 riteo: well yea I can remap ids
18:15 kennylevinsen: so it's "make a whole compositor" or "make a gutted compositor" - can't really say if 2 is easier
18:16 riteo: I actually have a broken prototype that parses the message signature from the generated headers, finds the `n` argument and then can do stuff with it
18:16 riteo: the plan is to make a wl_proxy from the editor in response to a proxied new id and then switch to its id instead
18:17 riteo: maybe i'm really making an extremely low level gutted compositor
18:17 riteo: i'm literally playing with the wire here
18:19 kchibisov: I mean, you need to write a proxy compositor.
18:19 kchibisov: and not e.g. directly deal with wire.
18:19 riteo: why not?
18:19 kchibisov: well, you can, but probably more error prone.
18:19 riteo: i'm doing wayland<->wayland stuff
18:19 riteo: the proxy interfaces would ideally do the same identical thing than relaying the wire format
18:19 kchibisov: yeah, but proxy compositor just means that you use wayland server and communicate forward with client api.
18:20 riteo: yes but that's a real chore
18:20 riteo: at that point I'd rather go with a full wlroots compositor, which does everything for me
18:20 kchibisov: you'd still need to hijack the toplevel.
18:21 riteo: well yea
18:21 kchibisov: to the point where you resize itc, etc.
18:21 riteo: I can't see why I can't do that with the wire either
18:21 kchibisov: I mean, wayland-server means that you parse the wire.
18:21 kchibisov: Though, I guess you need a custom parser.
18:21 riteo: I tried using wayland-server but unfortunately it did not give enough info
18:26 riteo: Anyways, thank you everybody for all the knowledge! You were lightning fast as usual to respond :DDDDD
18:28 riteo: I'll definitely keep the wlroots embedded example in mind, that's undeniably the most bullet-proof solution.
18:28 riteo: ... I can't stop wondering about the proxy approach though, it's terribly alluring.
18:29 kchibisov: people tried to load balance with proxies in wayland.
18:29 kchibisov: and iirc kennylevinsen has a debugger in go that introspects the wire.
18:29 riteo: I'm actually a bit confused on why the firefox folks chose to do the proxy thing
18:30 riteo: as in, why not do everything on another thread?
18:30 riteo: There's surely an architectural reason behind, no judgment obviously, just curiosity
18:31 riteo: the wire is extremely easy to parse BTW
18:31 riteo: especially when you're looking for a single thing
18:31 riteo: I think waypipe did a similar thing
18:31 kchibisov: just don't get cursor while at it.
18:32 kchibisov: though, maybe you don't create it in godot.
18:32 riteo: I'm sorry I having trouble parsing the phrase
18:32 riteo: get cursor as in wl_cursor?
18:32 kchibisov: wl_surface for cursor.
18:33 riteo: why not?
18:33 riteo: the hijacked surface is still a different surface, isn't it?
18:33 kchibisov: Yeah, but you need to hijack the right one.
18:33 riteo: tbh I've never worked with subsurfaces, maybe I'm missing something very important
18:34 riteo: kchibisov: well I think that's easy though, isn't it?
18:34 kchibisov: yeah, should be easy to track.
18:34 kchibisov: it'll be passed in set_cursor.
18:34 kchibisov: just need to track that.
18:34 riteo: or I could track directly get_xdg_surface and get_xdg_toplevel
18:34 riteo: and ignore everything else
18:34 kchibisov: yeah, that as well.
18:34 riteo: all right, fair :D
18:34 kchibisov: then you'd need to hijack `attach` and change to which surface you attach.
18:35 riteo: yea doesn't sound too hard either
18:35 riteo: it's the most cursed thing I ever thought
18:35 riteo: If the proxy works I'll feel forever dirty
18:35 riteo: lol
18:35 kchibisov: And not forward toplevel stuff up, only reply back.
18:36 riteo: yeah you can actually skip packets quite fine
18:36 kchibisov: some xdg_toplevel requests mandate a reply.
18:36 riteo: like, I tried skipping global events for interfaces I don't have signatures for
18:36 riteo: and it just worked
18:36 colinmarc: <
18:36 riteo: oh right
18:36 kchibisov: but it shouldn't be that hard.
18:37 riteo: yeah it's been surprisingly easy to do simple proxying and snooping with some hacked-up c code
18:37 riteo: I'm not even using any fancy library (godot has its own stuff so I'm keeping things terribly dumb)
18:37 riteo: the protocol is /clean/
18:38 riteo: there's also a whole elegance to it, don't you think? The only reason I can proxy so much stuff is that basically everything is surface-local
18:39 riteo: so all it takes is to swap ids and emulate a single interface, I think
18:40 riteo: welp, time to keep going ig, armed now with my wlroots parachute for when stuff gets serious
18:47 kennylevinsen: re: Firefox, they don't control their Wayland display connection - gtk3 does
18:48 kennylevinsen: which increases complexity quite a bit - but with dynamic connection buffers on the server side the need for their proxy hack is minimized
18:52 riteo: ohhh all right that explains everything
18:52 riteo: ty
18:53 kennylevinsen: they also have a lot of stuff that gets sent to the main thread for some reason, but it's been a while since I've been down there - it's also a lot easier to point out flaws than it is to fix them, so won't throw too much shade over it
18:54 riteo: that's perfectly fair yeah
18:54 riteo: as I said before, no judging, platform stuff is tricky every single time
18:54 kennylevinsen: btw if you keep going the proxy route, you could consider having your proxy expose a godot specific she'll protocol to make things easier with less xdg mimicry and such
18:55 kennylevinsen: could also be used as general side channel comms
18:55 riteo: that's a really nice idea
18:55 kennylevinsen: ... shell you dumb phone
18:55 riteo: lol
18:55 riteo: We'll see how it goes ig
18:56 riteo: I'm here in the first place following the path of least resistance after all ;)
18:56 riteo: I'm unfortuntately a really-hard headed sometimes and my curiosity is too strong to not keep going anyways
18:59 kennylevinsen: it's fine, as long as you're okay occasionally discovering dead ends :)
19:00 riteo: ye I'm ok with that :D
23:52 DemiMarie: Is a nested rootless compositor generally considered feasible to implement? I’m not aware of any implementations but it would be nice to know.
23:53 emersion: waypipe is such a compositor
23:56 DemiMarie: How big is the performance hit compared to a protocol-level proxy?
23:57 emersion: oh, waypipe is a proxy of sorts
23:57 DemiMarie: I know you take a hit from the extra blits and I know that is bad when one is pushing a system to its limits, but most users don’t push a system to its limits
23:57 emersion: you don't need to blit
23:57 emersion: gamescope doesn't blit, for instance
23:57 DemiMarie: what about subsurfaces & damage?
23:58 emersion: these can be forwarded as sub-surfaces to the parent compositor
23:59 DemiMarie: how complex is it to keep all of the coordinates in sync?