00:35 wlb: wayland-protocols Merge request !269 opened by Matthias Klumpp (mak) Draft: staging: Add ext-window-icon to allow windows to set dedicated icons https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/269
04:23 riteo: oh boy it's been a while and I only now notice that pq answered me, I'm very sorry
04:24 riteo: let's see
04:26 riteo: pq, so, re: the messages you said me (again, sorry a lot for the delay), your suggestions are really interesting and clear up most of the ideas behind wayland
04:28 riteo: so, thanks!
04:29 riteo: The messy thing is as usual the rendering though. I do agree that it's not a black or white situation. re: the timer, wouldn't one be inaccurate from the screen or somehow "shift out of phase"?
04:30 riteo: Also, since godot is used for many things, including UI tools (like the editor itself), I think that we'd like indeed to get rendering dictated by the frame event.
04:33 riteo: re: risky design, it indeed is unfortunately. Godot's rendering architecture wasn't really thought with Wayland's way of doing things sadly but I hope to have mitigated as much risky behaviour as possible with compromises such as the one described above your answer
04:40 riteo: still, I don't think that it's _that_ different scheduling a draw. There are some edge cases due to lazy rendering but they should be "contained" pretty reasonably rn, due to various things.
04:41 riteo: The real problem is _when_ to read that scheduled draw. Since we're don't have the whole graphics stack timing us anymore, we have to tick at a fixed frequency, and I found that ticking at something like 144 Hz gave some really wonky frame times
04:43 riteo: I bumped the tick rate to something like 2000 Hz and that improved things but that's obviously not ideal. I'm not even sure if that's possible to resolve, as we _must_ poll the draw schedule somehow and somewhen.
04:48 riteo: although, the wonky frame time might come from how we "schedule" frames, as we receive messages from the Wayland thread, do physics/input/whatever logic, draw and the cycle restarts. We might be "missing" very often frame schedules because we're not checking at the right time
04:50 riteo: still, to get back to discussion, that's the intrinsic issue I wanted to point out: we want to use frame events manually, so we have to somehow poll at an _undetermined_ frequency for schedules since our rendering code is main-loop dependent
04:51 riteo: I wonder if this is a real issue or if I'm misguided
16:25 DemiMarie: riteo: I think the problem is that you have other code depending on your rendering.
16:31 DemiMarie: The Wayland-native solution is to decouple rendering from all other operations. Networking, physics, and other events do not cause a synchronous redraw, and instead merely update a data structure in memory. Your rendering code then uses that data structure to draw frames at the rate the system requires them, which might even be zero.
16:32 DemiMarie: By separating rendering from other operations, you avoid drawing frames that will be discarded, saving CPU and GPU time and power consumption.
16:33 any1: I am running into a cleanup problem when the compositor goes away from under a client. The compositor often cleans up all proxies within destroy_queued_closure() which is called from wl_display_dispatch_queue_pending().
16:33 any1: I was under the impression that the client was responsible for cleaning up all resources, but this causes double free.
16:34 any1: Is this how it's supposed to be?
16:35 emersion: where is the UAF? client or compositor?
16:37 any1: UAF?
16:37 psykose: use after free
16:37 any1: client
16:38 any1: This is probably a rather poorly tested use-case. I added a "detached mode" to wayvnc, so that it can re-attach if the compositor goes away.
16:38 any1: I clean up all resources when the compositor goes away and the re-attach when it comes back up.
16:47 Arnavion: I heard Qt6 does that too, so you could check if their code has any caveats
16:48 Arnavion: https://blog.davidedmundson.co.uk/blog/qt6_wayland_robustness/
16:57 any1: Looks like a reference counting issue
18:21 riteo: DemiMarie: indeed. It's hard to do though, especially with an unmerged branch. I'll definitely put this issue to the rendering team somehow
18:22 riteo: My biggest concern is, though: rendering is just showing stuff, but that stuff has to be updated every so often
18:23 riteo: in a video game stuff is updated at the same pace of drawing (the `_process` method in godot parlance), but with this essential mechanism broken how often are we supposed to update the state that we're rendering now?
18:33 any1: Turns out I was dealing with improper cleanup in my own project...
18:50 DemiMarie: riteo: Do you have one or more separate rendering thread, or do you draw on the same thread that handles other events? If you have one or more separate rendering threads, I suspect you will already have the needed synchronization. Otherwise, event-driven rendering would mean that rendering happens in a separate callback from other event handling.
18:50 DemiMarie: So instead of a single method that does physics, drawing, etc, there would be separate callbacks for each event that comes in.
19:00 Guest9879: Hi? Does anybody know if wayland supports telling applications to render at a decimal scaling factor (example 1.75), instead of doing the integer scaling thing of x ?
19:02 kennylevinsen: Guest9879: yes, with the fractional scale protocol.
19:03 Guest9879: Thanks!
19:03 riteo: DemiMarie: I'm not really that experienced with the rendering architecture of godot
19:03 DemiMarie: I suspect the answer to “how often” is “however often your game requires”, which might or might not be the same as the frame rate.
19:04 riteo: Apparently we have an option for a rendering thread, but it actually just runs a new thread every invocation so that the process loop can work in the meantime
19:04 riteo: also, we have a separate Wayland thread altogether since we want to be able to handle events while the game is crunching data
19:04 riteo: otherwise we fill the whole wayland event loop and it crashes beatifully
19:04 riteo: s/loop/queue
19:08 kennylevinsen: So your issue is that your main thread was previously throttled by some vsync equivalent, and only handled events in batches before rendering?
19:08 riteo: pretty much
19:08 riteo: we have another ticking mechanism, but that's for suspended windows (e.g. minimized)
19:09 riteo: and it's pretty low and fixed, which is not ideal for our use case
19:09 kennylevinsen: If you have a Wayland thread, you could have it signal a condition on frame event. The main thread could then wait on that with a timeout (the timeout in case the window is hidden)
19:09 riteo: that could be done, but what would the timeout be?
19:10 riteo: I think that the current approach is pretty similar to that
19:10 kennylevinsen: Something equivalent to the background ticking mechanism, as it indicates a hidden window
19:10 riteo: basically we force the "suspended" tick rate to 2000 Hz or something and check every main loop iteration with a flag if we can draw, otherwise we just `_process` stuff
19:11 kennylevinsen: 2000 Hz o.O
19:11 riteo: yeah otherwise frame times would go all wonky
19:11 riteo: they still are, it's just less wonky this way
19:11 riteo: this is obviously wrong but it's the best thing I could figure out
19:11 riteo: probably having rendering completely separate would be a lot better
19:12 riteo: but still, finding a reasonable amount of time to tick will be hard, especially without breaking a lot of API or conventions
19:12 kennylevinsen: I was thinking more like 10. An average monitor would give you a tick of 60Hz, no reason to suddenly go 40 times faster than that when no one is looking at the output
19:12 riteo: it's going always like that xD
19:13 riteo: right now we tick a lot and if in the meanwhile the event polling logic got a frame request we draw in that single iteration
19:13 riteo: I've heard about the timeout heuristic but it looks pretty finnicky to me
19:14 riteo: like, this compromise is obviously wrong but it also makes a lot of sense, especially if we eventually get completely independent drawing
19:14 kennylevinsen: I understand game-esque logic where your process all pending events, then render, then block until vsync or similar - but busy-looping at 2000Hz sounds like a broken attempt at emulating a proper event loop :S
19:15 riteo: it is lol
19:15 riteo: that's just the best I could do without touching the actual main loop logic
19:15 riteo: godot is still multiplatform after all
19:15 riteo: after it gets merged we can hopefully find a better solution that might be a bit more invasive
19:16 kennylevinsen: Either way if your loop is used to blocking until vsync when a frame is drawn, you might be fine with 60Hz rather than 2000Hz
19:17 kennylevinsen: Maybe you can't go to 10 like I wanted, but if it can handle 60 sometimes it might be less wasteful to just do that all the time
19:17 riteo: Frame times got really wonky in my experience and what about 144 hz displays?
19:17 riteo: or 240?
19:17 kennylevinsen: If the timer timing out, frames are not visible
19:17 kennylevinsen: The moment the window is visible you'd get 240Hz or whatever
19:18 riteo: In my uneducated mind I wonder if we couldn't like write all of the drawing commands in something akin to a double-buffered queue and sync only that swap, so that when we want we can draw without waiting for a full physics update and do it all from the wl thred
19:18 riteo: kennylevinsen: what timer? That's the biggest issue with this approach
19:18 kennylevinsen: Any in that state is solely to make the window not super out of date when it is shown again later (e.g. what blocked it was removed)
19:18 kennylevinsen: Timeout on waiting on the condition
19:19 riteo: yeah, how much though
19:19 riteo: wait too little and it triggers that condition every time (e.g. throttled from the compositor), wait too much and it's annoying
19:20 riteo: and then there's the stalled situation where the game is crunching hard, a few seconds pass and then because of the execution order it could be thinking that it's suspended while it's actually not
19:20 kennylevinsen: As mentioned, as slow as does not break Godot - but at least like a slow monitor as Godot should be able to handle that. Inconsistent frame timing are fine for frames never shown - only the last frame before the window is revealed again might end up briefly seen.
19:21 riteo: mh, maybe we could use this heuristic just for rendering
19:22 kennylevinsen: (You would of course also use the suspended state to stop rendering when it's clearly not needed)
19:22 riteo: I'm still very concerned about the "as slow as does not break godot" for some reason
19:22 riteo: like, that value is pretty much "infinitely slow"
19:23 kennylevinsen: Well, then I see no reason for concern - go slow. Breaking in this case would be that expected background behavior stops working
19:23 kennylevinsen: Like audio playback glitching
19:23 riteo: busy looping, while a bit inefficient, is completely robust
19:23 kennylevinsen: If there are no such concerns, from my perspective you could stop ticking altogether
19:23 kennylevinsen: Why waste cycles on nothing of interest?
19:24 riteo: kennylevinsen: unfortunately `_process` is also used for networking
19:24 riteo: and an awful lot of other vital things, hence the alternative "suspended" ticking logic
19:24 kennylevinsen: But either way, if Godot is normally throttled to 60Hz, there should never be a reason to waste cycles going *faster* than that. 144Hz monitors do not matter when it's not on screen.
19:25 riteo: so, to recap
19:25 riteo: I'd wait for the frame event right before drawing?
19:26 riteo: and timeout
19:26 riteo: no wait, before processing anything else
19:27 riteo: mh, I think I actually tried this approach just yesterday and I didn't feel a difference. Maybe it's because of some other variable, but if that would improve cpu utilization...
19:27 riteo: I'll experiment a bit with your approach and see what happens, thanks
19:28 kennylevinsen: Using frame events keep you in sync with the monitor, not polling at a gajillion hertz saves you cpu
19:28 DemiMarie: riteo: so `_process` being used for networking needs to be fixed anyway
19:29 riteo: mh I think there's a problem though
19:29 riteo: we'd still have to manually poll the event thread
19:29 DemiMarie: why?
19:29 riteo: unless I use some threading primitive I suppose
19:29 DemiMarie: why not have a single event loop that includes everything?
19:29 riteo: but rn we just have `DisplayServer::process_events`
19:29 kennylevinsen: DemiMarie: well yeah, this thing needs a proper event loop, it's more about how to salvage it for now
19:29 riteo: and we'd have to call it while `!DisplayServer::get_singleton()->can_any_window_draw()`
19:29 riteo: so we'd still be burning the CPU
19:30 riteo: unless we throttle _that_ too, but then we're at square one
19:30 riteo: ughhhhhhhhhhhhhhhh
19:30 LaserEyess: is this yet another issue that would be solved with https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/199 ?
19:30 DemiMarie: riteo: Is adding a proper event loop an option?
19:31 riteo: DemiMarie: what do you mean by that?
19:31 riteo: LaserEyess: I'm not sure, maybe?
19:31 riteo: I discussed a bit there and the answer was "probably", but I honestly think that the proper approach would be better in the long run
19:32 DemiMarie: riteo: asynchronous networking, drawing, etc
19:32 DemiMarie: No busy polling anywhere
19:32 LaserEyess: well, if that's not the proper approach then what is? it basically lets you skip all of the guesswork of "when am I going to need to render next?"
19:33 LaserEyess: maybe there's a newer version of that particular protocol, the idea gets discussed and debated *constantly*
19:33 kennylevinsen: proper event loop: sleeping in the kernel until events occur, being waken up at exactly the right time with zero wasted CPU cycles - rather than busy-looping
19:33 DemiMarie: LaserEyess: The protocol would help but it would not solve Godot’s design problem
19:33 LaserEyess: I guess this is what I get for not fully reading the backlog
19:34 riteo: DemiMarie: mh, maybe
19:34 DemiMarie: In Godot, this would mean that networking/storage/rendering/etc are all based on callbacks
19:34 riteo: I'd have to discuss this with the various teams
19:35 riteo: the annoying part is compatibility
19:35 kennylevinsen: riteo: I think I might need a recap of hoe Godot does this on other platforms and why this doesn't work on Wayland before I'd be able to help more.
19:35 DemiMarie: riteo: compatibility with other platforms or with existing code?
19:35 kennylevinsen: well event loops with compatibility with everything already exist
19:35 riteo: existing games
19:35 DemiMarie: I see
19:35 riteo: I think that there is already _some_ infrastructure for that, the main loop is actually pretty opaque under that aspect
19:36 riteo: like, games already receive notifications, which are then wrapped into cutesy little methods for the most common ones AFAIK
19:36 DemiMarie: At the C++ level this means callbacks or C++20 coroutines
19:36 riteo: we have our own homebrew stuff
19:37 riteo: we have callables, fancy function pointers
19:37 riteo: I think that's what you're talking about?
19:37 DemiMarie: sort of
19:37 DemiMarie: With an event-driven main loop things can happen in any order
19:38 riteo: anyways, to give a very rough idea of how this stuff works
19:38 riteo: We have `OS` classes, one for LinuxBSD, one for Windows and so on
19:39 riteo: They implement `main`, like the actual entrypoint. Then, at least the LinuxBSD one processes events, joypad input and then calls the actual main loop logic, implemented in, well `MainLoop`
19:39 riteo: The process stuff is implemented into `DisplayServer`s, of which there can be multiple per platform to support multiple backends like X11 and Wayland in the same binary
19:39 riteo: so, DS -> joypad -> main and so on
19:40 riteo: the main loop then processes, renders, updates physics, VR and anything else (not in this order)
19:41 LaserEyess: do you have a link to the API?
19:41 riteo: during process we actually check if we can draw and either normally draw (and let the graphics stack wait for us) or we go into "low consumption" mode, where we basically loop at 144 hz (by default) without drawing (but still calling `_process`)
19:41 riteo: LaserEyess: the docs are available over at docs.godotengine.org
19:41 DemiMarie: _process is the problem here
19:41 riteo: most of this stuff I'm saying comes from playing around a lot with those specific areas of code
19:42 riteo: DemiMarie: yeah I think it's mostly that. All notifications are "manually" sent though
19:42 DemiMarie: With an event-driven main loop, the main loop handles not just input events, but also timers, network activity, and all other I/O
19:43 riteo: so there's also the issue at the root where we must still poll for when to update physics, and send various events such as input stuff
19:43 DemiMarie: Physics would be “whenever the game needs”
19:44 riteo: at least physics have a static tick rate
19:44 riteo: it has it's own notification, `_physics_process`
19:45 DemiMarie: Every main loop I know of has its own timer handling code
19:45 riteo: ours too
19:45 LaserEyess: riteo: it looks like the main loop is separate from your OS/DS classes, and also that games can provide their own
19:45 DemiMarie: But that timer handling is implemented via callback-on-completion at the API level
19:46 riteo: we actually just manually wait
19:46 riteo: like with sleep
19:46 DemiMarie: that’s an antipattern
19:46 riteo: either nothing IIRC or the low consumption mode tickrate
19:46 DemiMarie: The proper solution is to use epoll (Linux)/kqueue (BSD)/etc to wait until an event happens
19:47 DemiMarie: You then use an API to get a notification from the OS when something has happened
19:47 riteo: we also sync with the renderer though if that helps
19:47 DemiMarie: “Something” = I/O has completed, timer has expired, etc
19:48 DemiMarie: And pretty much your whole program runs in various event handlers
19:48 riteo: would all of those polls be into different threads?
19:48 DemiMarie: no
19:48 kennylevinsen: polling interfaces allow you to wait for any number of event sources all at once
19:48 riteo: oh all right
19:48 DemiMarie: exactly
19:48 DemiMarie: you can wait for tens of thousands of events at once without any performance problems
19:49 riteo: so we just poll for a lot of stuff at once and if any changes we check, update, rinse and repeat
19:49 riteo: that's definitely a pretty radical change
19:49 kennylevinsen: yup - you get informed of exactly which events have occurred (e.g., traffic on wayland socket, so you need to dispatch wayland, traffic on joystick event device so you need to read that, ...)
19:49 DemiMarie: yup
19:50 kennylevinsen: and event loops also allow timer, signal and idle events so you can do whatever "soft" events you want, or batch everything up to simulate existing behavior
19:50 riteo: not sure if that's doable in the current release cycle, but I think that it shouldn't be too hard to implement while keeping compatibility
19:50 kennylevinsen: yeah it would be a pretty big surgery, which is why I was suggesting more of a hack :P
19:50 kennylevinsen: definitely not a last-minute merge
19:51 riteo: yeah we're definitely in hack-land here
19:51 riteo: I'm pretty sure that the current approach and your proposed one are actually pretty similar
19:52 kennylevinsen: that might be - what do you do right now?
19:52 riteo: like, if we add a timeout we're actually enforcing a minimum amount of iterations
19:52 LaserEyess: honestly I'm pretty shocked to hear that godot is busywaiting instead of using a real event loop
19:52 LaserEyess: even on windows?
19:52 kennylevinsen: the timeout is mainly to avoid the unexpected stall as a window gets hidden just as you render
19:52 riteo: LaserEyess: boy, is it that weird? I might be missing something perhaps
19:52 riteo: I'm not _that_ experienced with Godot's core logic
19:52 DemiMarie: On Windows you will need a separate thread to handle I/O completion ports and post a message to the message loop
19:53 kennylevinsen: if you use the suspended state so you know when windows are e.g. minimized, you shouldn't hit the timeout too often on supported compositor - so it's just anti-stall
19:53 riteo: kennylevinsen: we're ticking at a defined rate and draw if between iterations we get a frame event
19:53 kennylevinsen: and with that in mind I think it's fine to make it slow
19:53 DemiMarie: Or the reverse: pump window messages on a separate thread and use PostQueuedCompletionStatus()
19:53 riteo: by adding a time out we're still enforcing a minimum tickrate I think
19:53 riteo: so we're doing pretty much the same thing I think
19:53 DemiMarie: riteo: It’s not really *weird*, more somewhat naive
19:54 kennylevinsen: yeah, that seems fine
19:54 kennylevinsen: well I don't like the ticks at all, but again the solution to that is an event loop
19:54 DemiMarie: Don’t try to rush event loop support in, though
19:54 riteo: kennylevinsen: I do agree that it's weird to tick at 2000 Hz though, but that makes it bullet-proof on a lot of setups
19:55 riteo: DemiMarie: AFAIK the main core logic hasn't changed ever, but again, there might be some more cleverness higher in the stack
19:55 kennylevinsen: the main option you have here is to slow down your tick rate to the monitor refresh if that's something that aligns with godot
19:55 LaserEyess: riteo: I am not a gamedev and I have no idea how major game engines work but I would imagine that you're hitting the CPU too hard and adding too much latency for e.g. games that need 200+ Hz
19:55 riteo: like, we got multithreaded scene tree updating and stuff liek that
19:55 LaserEyess: and yes I concur with switching to an event loop being a massive change likely worthy of a major version bump
19:55 DemiMarie: riteo: It’s going to be a big change indeed
19:55 riteo: LaserEyess: ironically it lowers the frametime from what I've experienced
19:56 DemiMarie: riteo: Higher frame rate is not always good
19:56 LaserEyess: well if you poll at 2000 Hz then I mean yeah you can brute force it
19:56 riteo: because we're not updating physics every 2000ths of a second, but still at the fixed rate
19:56 LaserEyess: oh
19:56 riteo: DemiMarie: I'm not talking about higher frame rate, but less noisy frame times
19:56 DemiMarie: riteo: I see
19:56 riteo: it sohuldn't be too hard to like poll the event loop at a different rate than the rest of the stuff though if that's annoying
19:56 kennylevinsen: I definitely wouldn't want an application I use to poll at 2000Hz, but if I'm playing a game I probably won't care too much as long as it works
19:57 riteo: but that's the problem, we still have to poll the event thread somehow from the main thread, so there's always a tickrate
19:57 DemiMarie: nope
19:57 DemiMarie: not with a fully event driven design
19:57 riteo: yeah, I know that
19:57 DemiMarie: but for now, yeah
19:57 riteo: but from the main thread-oriented design that's how it works
19:58 riteo: in a way, that looks by design
19:58 kennylevinsen: riteo: if you wanted to be "smart", you could average the frame times and use that after a while
19:58 riteo: that could be done
19:58 riteo: we already have some delta smoothing after all
19:58 LaserEyess: well you can use multiple threads with an event loop, and even an event loop per thread, but in addition to not being a gamedev I am also not an expert on multithreaded and/or lockless programming so I would probably not do that
20:00 riteo: I think that this all boils down to a sever architectural difference between what a traditional game engine like Godot expects and Wayland
20:00 riteo: which isn't a bad thing per se mind you
20:00 riteo: that's just how stuff is rn
20:00 kennylevinsen: well you usually don't have a reason to have more than one event loop thread. If needed, you can dispatch to other threads.
20:00 riteo: rn we have a WaylandThread to always handle wayland things even when the engine is stalled
20:01 kennylevinsen: riteo: it is not an uncommon mismatch, SDL2 has a similarly wonky architecture
20:01 riteo: and I'm pretty sure that we want to keep a wl thread due to the heavy "handle quickly or else" logic behind libwayland
20:01 kennylevinsen: but the solution is a significant design and performance/latency improvement, so it's a win-win when you get there
20:02 riteo: oh yeah definitely
20:02 riteo: maybe one day, in godot 5...
20:02 riteo: or maybe earlier, who knows
20:02 kennylevinsen: use as many threads as you want to get it to work, you're the one that has to juggle the synchronization as the end of the day
20:03 riteo: I'm not a core or rendering person, so take everything I've said with a grain of salt
20:03 riteo: or as we say over there, TIWAGOS
20:04 riteo: thanks a lot folks for all your thoughts, this has been really interesting!
20:05 LaserEyess: good luck, I'm glad to see godot getting good funding and promoting FOSS
20:06 LaserEyess: and thanks for not ignoring wayland because "too hard" :')
20:08 riteo: this is all somehow very gratifying for some reason lol
20:08 riteo: I really like this platform stuff for some reason
20:12 emersion: i too like banging my head against the wall for some reason
20:13 emersion: something something WSI
20:14 kennylevinsen: it can be therapeutic to slam your head against the wall. Reshuffles the brain cells a bit.
20:18 DemiMarie: emersion: should the WSI have been asynchronous?
20:18 emersion: not sure what you mean
21:04 DemiMarie: It seems that people complain a lot about the Vulkan WSI, so how could it have been done better?
21:06 kennylevinsen:complains about WSI's in general, not vulkan specifically