10:21wlb: wayland Merge request !370 opened by Simon Ser (emersion) protocol: document that color channels provide electrical values https://gitlab.freedesktop.org/wayland/wayland/-/merge_requests/370 [Protocol]
11:27pq: emersion, since you mentioned; how does that Vulkan doing TF decoding work with pre-mult-in-electrical alpha? Can you get it right, or did you just decide it's close enough?
11:27emersion: so…
11:27emersion: we don't use the _SRGB vulkan format for buffers with an alpha channel
11:28emersion: so yeah, i suppose global alpha is kind-of the same situation
11:28pq: what do you with alpha-ful formats?
11:28emersion: we use _UNORM, and manually do the decoding
11:28pq: alright
11:29emersion: still don't think it's a good idea to encode the exact behavior in the spec, but oh well
11:30emersion: like, if you spec that the protocol multiplies encoded values, then it's impossible to do gamma-correct blending
11:30pq: yes, I'm about to reply something like that
11:30emersion: i don't really want to encourage clients to do gamma-incorrect stuff
12:37pq: emersion, is it so in libwayland that the compiler will not warn about unhandled values for switch (char), but it does for switch (enum)?
12:38emersion: -Wswitch is only about enums
12:38pq: thanks
12:39emersion: i don't know of an option to warn about missing default case for non-enums
12:39emersion: (https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wswitch)
12:40emersion: there is -Wswitch-default, but it would also warn if there is no default case for enums, even if all enum members are covered
12:41pq: ok
13:24emersion: > Premultiplying into electrical color channels (here alpha does not count as a color channel!) would have an issue with DRM_FORMAT_AYUV...
13:24emersion: pq, what do you mean here? YUV count as "color channel" no?
13:24pq: you never multiply A into Y, U or V
13:25pq: hmm, or do you...
13:25emersion: hm, so that is more of an issue with the existing wording i suppose?
13:25pq: yeah
13:28pq: I've never heard of pre-multiplied YUV, but as long as YUV->RGB is a simple matrix operation, I guess pre-mult could... work... hmm, but the neutral value for U,V is not zero
13:28emersion: yeah…
13:29pq: but I think from linear math perspective it should be equivalent?
13:29emersion: applying transparency to YUV should probably leave UV as-is and only affect Y?
13:29emersion: hrm, or would it…
13:30pq: we need to recall that luma (Y) is not luminance
13:30pq: since we're talking about non-constant luminance YCbCr encodings
13:31emersion: does it even make sense to premult YUV channels?
13:31emersion: does it help when compositing?
13:32emersion: since no-one uses YUV with alpha channel, we can probably decide to define it to whatever we want?
13:32pq: emersion, something strange with https://gitlab.freedesktop.org/wayland/wayland/-/merge_requests/367/diffs?diff_id=3782998&start_sha=cc01720c33dbe5ed4c73e6aa7a74bfccee2576c2 - like the sha1 changed, but code change is missing.
13:33pq: or did it just get removed by a later patch in the series?
13:34emersion: pq, it's because the line with the bad indent is removed in a later commit replacing a switch with a if
13:34pq: cool
13:34wlb: wayland/main: Simon Ser * Introduce enum wl_arg_type https://gitlab.freedesktop.org/wayland/wayland/commit/155dd63b58b8 src/ connection.c wayland-client.c wayland-private.h wayland-server.c
13:34wlb: wayland/main: Simon Ser * connection: simplify wl_closure_lookup_objects() loop https://gitlab.freedesktop.org/wayland/wayland/commit/830883e5b2d3 src/connection.c
13:34wlb: wayland/main: Simon Ser * client: simplify create_outgoing_proxy() loop https://gitlab.freedesktop.org/wayland/wayland/commit/7a1e7dd54963 src/wayland-client.c
13:34wlb: wayland/main: Simon Ser * client: simplify create_proxies() loop https://gitlab.freedesktop.org/wayland/wayland/commit/440defbd2ba3 src/wayland-client.c
13:34wlb: wayland Merge request !367 merged \o/ (Introduce enum wl_arg_type https://gitlab.freedesktop.org/wayland/wayland/-/merge_requests/367)
13:35pq: about AYUV, let's look at H.273
13:37pq: right, so premultiplying YCbCr does not work, because the offset needed to encode the negative side of Pb,Pr would get reduced as well.
13:39pq: premultiplying YPbPr would work but you cannot encode with any UNORM format, you'd need SNORM
13:42JEEB: so YPbPr, is that just analog stuff VS digital being YCbCr? never really understood what YPbPr was supposed to be.
13:42JEEB: some places seemed to utilize them interchange'ably
13:42JEEB: just like the old way of calling YCbCr "YUV" :D
13:44pq: yes, or in this case, YCbCr uses unsigned integer encoding, while YPbPr is using real-valued numbers, [0.0, 1.0] or [-0.5, 0.5] IIRC
13:45emersion: with "darkness", i suppose alpha premult wouldn't really make sense either would it?
13:45pq: I'm basing this on the H.273 terminology
13:45JEEB: oh, H.273 referred to YPbBr? missed completely :o
13:46emersion: the darkness channel would need to use alpha pre-division rather than alpha pre-multiplication, no?
13:46pq: it's E'_PB etc. there
13:47pq: emersion, good question. What is "transparent darkness" to begin with?
13:47pq: well, not transparent, but semi-coverage more like
13:48emersion: i'm seeing it as d = 1 - r
13:48emersion: darkness vs. red
13:49pq: straight: (a, d), needs to convert to (a, rgb = 1 - d) for blending in rgb, presumably.
13:49emersion: so hm dividing would not be the right thing to do
13:49emersion: yeah
13:50pq: rgb = 1 - d, we want to find a*rgb, so multiply both sides by a, get a*rgb = a(1 - d) = a - a*d.
13:51JEEB: this alpha discussion reminds me that Apple actually added a flag in QTFF for whether the alpha is premultiplied or not
13:52pq: JEEB, what's QTFF and what about the other premult mode?
13:52JEEB: QTFF aka MOV
13:52JEEB: the container :D
13:52pq: ah
13:52JEEB: (from which ISOBMFF was forked off of)
13:54JEEB: unless that info is actually in prores bitstream itself which would be less fun :P
13:54JEEB: since QTFF specification is at least relatively open
13:54JEEB: while prores... lol
13:55emersion: pq, yeah. can this still be called "alpha premult"?
13:55emersion: maybe we can say that alpha premult is only for rgb formats?
13:56pq: 1 - a*rgb = 1 - a(1 - d) = 1 - a + ad = ad - a + 1 = a(d - 1) + 1... it doesn't seem like this would be anything like premult
13:56JEEB: ahh, probably this https://developer.apple.com/documentation/quicktime-file-format/graphics_modes
14:00pq: emersion, what about TF encoding on darkness formats? The usual TFs give more precision near zero, but human vision is more sensitive in the dark, so darkness formats would need more precision near 1.0. :-)
14:00emersion: ow :<
14:01emersion: i also have no idea how TFs work with YUV tbh
14:01pq: H.273 can tell you
14:02pq: just look for the ()' operator, that's the TF. Electrical values are marked with ', mostly.
14:03pq: the most common case is that TF is applied on RGB, and afterwards there is a matrix converting to YCbCr + quantization range encoding
14:04pq: that is the non-constant luminance way
14:04pq: i.e. the simplest form
14:09emersion: that makes sense
14:09emersion: to decode YUV into RGB and you get TF-encoded values
14:10emersion: so decode*
14:10emersion: if we wanted to, we could specify the same for darkness
14:11emersion: D' = 1 - R'
14:11emersion: so just like YUV, if one decodes D into R, they get TF-encoded values
14:11pq: sure
14:12pq: that just doesn't make premult-in-raw-pixel-value work
14:12emersion: yeah
14:12pq: like it doesn't work for YCbCr
14:12emersion: hm, why is that?
14:13pq: the encoding to u8 or what you have needs an offset, and you need to keep the offset intact
14:14emersion: i'm not following i think
14:15pq: Pb, Pr range is [-0.5, 0.5], and that then gets offsetted to be able to encode it in UNORM
14:16emersion: i mean, premult-in-raw-pixel-value only works for RGB, no?
14:16pq: multiplying RGB matches multiplying YPbPr (all electrical here), but that does not translate to simple multiplication of YCbCr
14:16pq: yes
14:17pq: maybe should even say, only for full range raw RGB, that is, UNORM.
14:38emersion: zamundaaa[m]: so your point is that for a compositor with gamma-incorrect blending, optical and electrical values are the same?
14:40emersion: that sounds like a weird definition to me, but i'm no expert…
14:42emersion: to me, a compositor with gamma-incorrect blending never converts to optical values
14:42emersion: but the meaning of optical values remains the same as for gamma-correct blending
14:51pq: I agree with emersion here
15:00zamundaaa[m]: emersion: in KWin we have the issue that we can't default to linear blending yet because some Intel and ARM iGPUs have surprisingly severe performance issues with it
15:00zamundaaa[m]: The shaders that do color management still work and get used when conversions are needed, even without linear blending though
15:00emersion: that's fine
15:01emersion: i don't plan to do gamma-correct blending for GLES2 or Pixman
15:01zamundaaa[m]: If you want to not have to do conversions to optical values in a compositor with gamma-incorrect blending, you can just not support the protocol in there, right?
15:01emersion: i'd still want to support the protocol with today's KMS and with my GLES2 renderer
15:03zamundaaa[m]: Then the only reasonable way forward I see is to leave it entirely undefined. Because requiring pre-mult-in-electrical to also make this alpha value be multiplied in electrical space doesn't sound useful to me
15:04pq: yes
15:04emersion: my preference would be to leave it undefined, and then make it defined if color-representation alpha_mode is set
15:05emersion: (or advertised?)
15:05pq: would it follow alpha-mode like I suggested, or?
15:06emersion: also keep in mind that some users may prefer gamma-incorrect blending
15:06emersion: so even if gamma-correct blending would be inexpensive with old Intel and ARM, some compositors may still have a reason to keep their gamma-incorrect mode
15:07zamundaaa[m]: Sure, and that's fine, but that doesn't mean these compositors have to support this protocol
15:07zamundaaa[m]: The point is for clients to offload their own rendering, if we can't define the exact behavior at least eventually, then it's not that useful
15:08zamundaaa[m]: Maybe we should just add an event to the global, saying which behavior the compositor applies?
15:08zamundaaa[m]: Then the client can decide whether or not it's acceptable for its use case
15:08emersion: there are many things that are "undefined" when it comes to sub-surface offloading and such
15:08emersion: and pq said it may not be desirable to fully define everything
15:10emersion: zamundaaa[m]: gamma-correct vs gamma-incorrect is almost like a user preference, it seems very weird to me to gate protocol support behind that
15:10zamundaaa[m]: Sure, defining everything may not make sense, but this protocol is purely an optimization, not a user visible feature
15:11pq: I think the whole debate on alpha-multiplier is a testament to defining too much being a bad idea for extension that are intended to gain wide adoption.
15:11pq: zamundaaa[m], Julian was promoting the optimization aspect, but you turned that down.
15:12pq: so what's the actual goal here? Off-load mathematics or semantics to the compositor?
15:14zamundaaa[m]: Can you link where I turned it down? Pretty sure I didn't; offloading to the compositor as an optimization is the sole purpose of the merge request
15:15pq: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/287#note_2313448
15:16zamundaaa[m]: I don't see how you could interpret that in this way
15:16pq: You'll see in my reply.
15:16zamundaaa[m]: What I'm saying there is that this protocol doesn't have to define everything by itself. It's fine if the exact blending behavior is specified somewhere else later
15:17emersion: pq, i agree with your proposal wrt. alpha-mode interactions
15:33pq: zamundaaa[m], what is it in optical blending that makes those old GPUs so slow? Is it half-float render target?
16:02zamundaaa[m]: I think so, yes
16:03zamundaaa[m]: Rendering to dmabufs instead of a GL_RGBA16F texture did seem to help on Intel at least, so it could also be a driver bug. I haven't spent too much time looking at it yet though
18:10wlb: wayland Merge request !371 opened by Simon Ser (emersion) connection: use enum wl_arg_type in wl_message_count_arrays() https://gitlab.freedesktop.org/wayland/wayland/-/merge_requests/371 [IPC library]
22:47zamundaaa[m]: TIL amdvlk still uses wl_drm for creating buffers :/
22:47emersion: yeah, they know about this
22:48emersion: (i've reported this issue to them)
22:51zamundaaa[m]: I found out about this from a user reporting mvp crashing on Plasma 6 because of that. I hope not a lot of people have amdvlk installed...
22:53emersion: mpv?
22:54emersion: yeah, i hope so too. next wlroots will drop wl_drm completely
22:54emersion: there is also https://github.com/intel/libva/pull/790
22:59zamundaaa[m]: Yes, mvp, with -- gpu-api=vulkan