Maybe I'm wrong because I'm not a Linux developer, but I think that the future for Canonical is Mir and not Wayland.

@iz0eyj right!

The main problem with Mir and Wayland is that both require DRM/GL. Right now, WSL does not expose graphics hardware, and rendering/compositing is done over remote X, which partially uses AIGLX and partially uses software rendering due to missing plugins in vcXsrv.

In order for DRM/GL to work, Microsoft would need to have an API for driver makers to expose their devices inside of WSL, which is not yet planned. Wayland and Mir do not support remote indirect opengl rendering, and they won't for the foreseeable future, although perhaps it might be possible in the future (I have a thread on this called something like "WSL should add a DRM api for Windows driver makers" or something like that).

In theory, I also don't see a reason why MS couldn't also provide a translation layer for DRM openGL calls to the corresponding Windows openGL driver, since I don't think the syscalls would even have to be translated.

This is what the Usermode side of Linux is looking for:
https://en.wikipedia.org/wiki/Direct_Rendering_Manager#API

The LXSS and LXCore might support just forwarding this on to the Graphics drivers, since I know that the win32 graphics system was moved to the kernelmode in like 1995 (see below)

From Wikipedia's article on the native NT api:

user32.dll and gdi32.dll include several other calls that trap into kernel mode. These were not part of the original Windows NT design, as can be seen in Windows NT 3.5. However, due to performance issues of hardware of that age, it was decided to move the graphics subsystem into kernel mode. As such, system call in the range of 0x1000-0x1FFF are satisfied by win32k.sys (instead of ntoskrnl.exe as done for 0-0x0FFF), and are declared in user32.dll and gdi32.dll. These functions have the NtUser and NtGdi prefix (e.g. NtUserLockWorkStation and NtGdiEnableEudc).

However, I'm not positive how this works, since I am pretty sure that both OpenGL and DirectX on Windows bypass GDI entirely, and I'm not sure how the Win32 subsystem interfaces at the kernel level with Graphics drivers. I have some speculation and research that I've done below:

Usermode to kernelmode for hw-accelerated OpenGL in Windows

Apparently the situation with openGL on Windows is that the built-in library opengl32.dll is an ancient library used as a loader to get to the graphics card's openGL interface with basic functionality (sort of like how VGA used to work), which then loads a series of OpenGL extensions provided by GLEW in a special header in order to extend the functionality to a modern version of OpenGL.

This means that the "magic" happens inside of opengl32.dll, which opens the tunnel to the graphics hardware, which is then quickly widened in scope to the full modern OpenGL4.5 API by loading OpenGL extensions.

Maybe someone who has experience writing graphics drivers for Windows could explain this at a high level?

Usermode to kernelmode for hw-accelerated OpenGL in Linux

Presumably, however OpenGL32.dll works, the WSL team understands it, and the difficulty would be either mapping the usermode functionality of Linux's Direct Rendering Manager to a corresponding Windows native component, or emulate it in lxss.sys. Since NT understands its own drivers, this would probably actually avoid driver makers having to support LXSS directly. I'm not exactly sure how this would work, but that is probably what you would need to do.

@fpqc would be possible to build a layer OVER WSL that intercepts the graphic calls and transmit it to the Win side?

@iz0eyj Not with Wayland or Mir. Both of them, I believe, do not have server-side rendering functionality. All rendering is done at the client side via libdrm or perhaps with a fallback to software rendering.

The compatibility layer would have to be part of WSL, although conceptually it actually doesn't seem that difficult to imagine MS being able to emulate the direct rendering manager itself (it would be part of the WSL layer sitting between libdrm and libgl on the Linux usermode side and whatever kernel-mode interface Windows uses to provide hardware-accelerated rendering in the kernelmode).

The main problem with DRM is that emulating it would be much harder than emulating the Linux Standard Base API, since DRM is much newer and much less well documented, and potentially less stable. It also happens to be one of the most active areas of Linux Kernel development over the past few years, and could potentially be much more difficult to emulate, especially if there is no corresponding native NT component that provides the same functionality as DRM. It could entail essentially rewriting 5-10 years of recent (and therefore less-well-documented) work by Linux Kernel devs from scratch, which seems like a pretty daunting task.

@iz0eyj Oh yeah Mir... (I wonder how long Canonical will keep that project going, they do like going out on there own).

@fpqc Server side rendering is almost dead these days. Even with X11 modern tool kits do lots of rendering client side and just give X11 the pixmap to draw. Also note that X11 has security issues which Wayland has fixed.

@fpqc From what I have seen the Linux DRM has lots of documentation, which can always be improved if that is necessary.

I'm sure if Microsoft in interesting it adding DRM they can talk to the developers from Intel, AMD etc that wrote the specs and implementation.

I have not looked but I expect that Mir sits on top of DRM as well.

All the graphics cards these day have Windows and Linux driver. The graphic card vendors could easily expose there Linux API for WSL if Microsoft specs the interface.

@barry-scott Not exactly how it would work. They would have to be Windows drivers that attached to some kind of DRM translation interface (or Windows could manage access). Using the Linux driver directly (at least the kernel-mode part) is totally out of the question.

@fpqc I must not have been clear.

Assuming that MS wish to do this they provide a Windows Kernal API that allows the DRM interface into the graphics driver.

The vendors have DRM code. Its not beyond the wit of man to put that code into the same Windows driver with the Windows Kernel interface code.

The Linux Kernal DRM API can then be exposed to a WSL user process.

That is how it could work.

Now as Mir is out of the picture, implementing Wayland support is the only hurdle in supporting GUI apps in wsl.

Are you sure DRM is required at all? The way I understood it DRM would only be a requirement for running weston or gnome-shell on the Linux side. But that is not what you're want to do.

Wayland doesn't specify how the compositor does the job. C.f. the weston branch using raspberry pi's dedicated hardware or Sailfish's compositor, which uses proprietary android drivers.

Theoretically, one could implement a Direct X12 based compositor.

If the client is using hardware accelerated drawing, that's of course a different story. OpenGL calls would need to be for to the host operating system. Maybe VMWare's gallium driver could be adapted. But for the time being software rendering would do.

Theoretically one could implement a DX12 based compositor but it would be a lot of work and would not be packaged with ordinary Linux distros.

I'm not saying one should use DX12. I just wanted to point out, that the Linux guest doesn't have to know details about the compositor just to get the basics working. It would definitely make sense to have some means of doing hardware accelerated rendering on the client side and only implement a basic compositor on Windows side supporting client compositors like gnome-shell or kwin. AFAIR this is what login managers like gdm do. But for the rootless case, I assume a full featured compositor on the windows side would be required.
I'm not sure if libweston is tied to drm. If not, that would make implementing a Windows native compositor a lot easier, I guess.

@oleid I don't think it's that simple. The libdrm part is just the kernel-facing part. Then there's also libGL which is the entrypoint for userspace OpenGL implementation. Both have to work as expected so you get accelerated graphics. Both GL drivers and libdrm are very vendor-specific so I reckon a lot of tuits getting things to work. If we had Vulkan compositors, things might be simpler since it should in theory be possible to translate Vulkan into DX12 for rendering. (the other way is under work already) Alas, Vulkan under Linux is still too new for Wayland compositors to have picked up support

Wayland doesn't require drm, weston works, on fbdev, drm, headless, and rdp (raspberry pi I think is being removed). (Weston can also use X11 or another wayland compositor as a backend).

Drm is used to implement OpenGL and Vulkan on Linux for most drivers (basically all except Nvdia's official drivers although they've added some drm support).

So you can get CPU rendering by just implement some Wayland compositor that passes on the frame buffers to Windows somehow. (I assume X11 has some way of doing this).

On Linux+Wayland the EGL library (interally) talks to the wayland compositor to determine what driver's in use and loads the appropriate OpenGL implementation, which then uses DRM to talk directly to the Graphics Driver. (and sends PRIME-FD's to share objects with the compositor).

A simpler solution to "Intercepting the DRM driver and direct it to the windows driver" (fun fact the Intel graphics drivers for Linux and Windows are complete separate from each other).
Would be: to implement an EGL that loads GL libraries that make calls to Windows GL (and does some sync with wayland). (Something similar for Vulkan).

(Although then you have the fun fact that programs that might work with Intels' Linux OpenGL don't work with Windows)

@hornetblack ifyou're ok with cpu rendering, might as well use X

May as well ask AMD to do some conversion between cpu rendering and graphics rendering on their APUs (comes with both) if that's the route hornetblack wants to take. I personally would rather utilize my 290x AMD GPUs because of performance boost. At work, computer specs aren't as great so cpu rendering could possibly be acceptable.

@DarthSpock There is no kernelmode API right now for driver-writers to add GPU support to WSL. Ben Hillis mentioned a while ago that an approach they are considering is a generic compatibility layer lying over the Windows driver stack and a WPF application for display.

If you are familiar with the history of WSL/LXSS, it originated as something called ADSS (an android compatibility layer for Windows Phone), which did have graphical application support, so the team does likely have some of this work done in some deactivated parts of the codebase.

Eh, there already is native X server on Windows called xming but it requires boilerplate (eithet X forwarding over SSH or opening up TCP sockets which is a vuln basically) and you cannot have Wayland apps with it

I have to disagree. The only difference with the two is packaging. If Microsoft packaged xming so you could install it, would it make it more native? Anyhow, it's not the preferred solution. It'd be better that Linux' graphics servers could render using Windows' driver facilities. It's more about not breaking WSL isolation layers than nativeness

The problem is not X, it's lack of hardware rendering support. When you render out to X on Linux, there is a usermode driver called "libgl-Vendor" that calls into the Linux kernelmode driver for rendering. Since there is no graphical hardware available in theWSL environment and no proper libgl to drive the hardware, everything is rendered in software.

Is libglencompassed by the UserVoice for OPENGL/CUDA? If libgl is so integral then should a UserVoice be created for that specifically?

There was here. It was locked because FAQ.

Can I run ALL Linux apps in WSL?
No! WSL is a tool aimed at enabling users who need them to run Bash and core Linux command-line tools on Windows.

WSL does not aim to support GUI desktops or applications (e.g. Gnome, KDE, etc.)

The CUDA UserVoice is a CLI development use case scenario. The closed issue is not. No, CUDA will not get you DRM. Someone very clever could implement OpenGL with CUDA. But even if you did, you still need to paint the pixels with something like #230 or a network transparent protocol to a Windows server.

So again, a new one for libgl specifically?

If it will make this issue# go quiet again, yes. Be sure to cite your use case.

The problem is that without kernel support there is no window to draw on.
This is not a software vs. OpenGL problem. But once you have the window to draw on
you wil l have the API to use OpenGL on that Window.

Actually a DRM driver is not mandatory for this goal. NVIDIA is trying to avoid this with EGLStreams and EGLStreams is at least supported at least by Gnome

We can have a guest EGL driver that manages resources, as well as a guest OpenGL driver that translates OpenGL calls to some intermediate representation (e.g. Vulkan commands). Both drivers are talking with the counter parts on the Windows subsystem.

The whole architecture would be similar to hardware accelerated OpenGL in VMWare Workstation, with the kernel driver part in the guest os merged into the hypervisior.

There was a huge religious war between nvidia and the Wayland team over EGLstreams, and they have agreed to develop a replacement interface in the future. I doubt that MS is going to get involved in developing a driver for something that Wayland has said they will not support.

Oh and also...

Someone very clever could implement OpenGL with CUDA

http://www.nvidia.com/content/gtc/documents/1055_gtc09.pdf

You get to X11 by using a running the wayland X11 server.
MS would be crazy to attempt to support X11 at this point in time.

It does not matter if its drm or egl streams without a kernel interface nothing can be shown.
CUDA needs a kernel interface asd well.

Will it be possible to run a Wayland WM like Sway this way? Similarly to how nowadays you can use an X client in Windows and run a window manager in WSL.

Will it be possible to run a Wayland WM like Sway this way? Similarly to how nowadays you can use an X client in Windows and run a window manager in WSL.

A nested compositor would work, but that isn't going to be what you want as it would be in its own window. You'd likely have to modify Sway to function.

Will it be possible to run a Wayland WM like Sway this way? Similarly to how nowadays you can use an X client in Windows and run a window manager in WSL.

A nested compositor would work, but that isn't going to be what you want as it would be in its own window. You'd likely have to modify Sway to function.

Thanks. That actually is what I want and do right now. I mostly spend time in my WSL, so I just run the full (tiling) WM in one fullscreen window on my main display.

For my development needs I'd like to have full support:

• Wayland
• Vulkan
• PipeWire

I hope to get a great and stable multimedia platform and to be able to target it from Visual Studio 2019 (Code), so I can seamlessly switch between Windows/Linux and other targets.

For my development needs I'd like to have full support

What does prevent you to use an actual installation of a GNU/Linux distribution or any other unix-like OSes?

@Biswa96 Technically nothing. It's a matter of convenience.

Will it be possible to run a Wayland WM like Sway this way? Similarly to how nowadays you can use an X client in Windows and run a window manager in WSL.

A nested compositor would work, but that isn't going to be what you want as it would be in its own window. You'd likely have to modify Sway to function.

@valorl I would love having it as well. Either in its own windows or being able to stream apps independently of the desktop os window. Up to give a hand in this adaptation. I will definitely appreciate my arch + sway setup at the end!

We released the first preview of this work today!
Please see the blog post here for all the details: https://devblogs.microsoft.com/commandline/the-initial-preview-of-gui-app-support-is-now-available-for-the-windows-subsystem-for-linux-2/

Build, test and use Linux applications that use audio or the microphone with built in audio support

I like that :)

Bonus: Leverage WSL’s GPU access to run Linux applications with 3D acceleration

Why is that considered a bonus? 3D isn't a bonus since 20 years?

In that blog post, the diagram of the architecture does not explain the role of WSLGd binary. More info will be helpful.

@Biswa96 you're probably looking for the lower-level sister blog post: https://devblogs.microsoft.com/commandline/wslg-architecture/ 😄

@craigloewen-msft Thanks so much! I was really impressed with your presentations at XDC last year. This has been so well architected so congratulations on getting it out the door.

For the time being though, unlike most users, I'm actually more interested in RAIL than VAIL. Are there any instructions on connecting remotely? I imagine you can just fire up Weston and connect, much like I was doing before, only it should work much better now? I've found Microsoft's Weston fork so I could build this or maybe I could run your existing image with Docker somehow?

@Zingam I put 'Bonus' since it's a pretty exciting feature and as of right now requires preview versions of software but will soon be fully available on every distro, I get your point though!

@chewi that's an interesting point. We haven't explicitly built WSLg to do this, but it is open sourced as you mentioned so you could take a look and see if you could get it working.

Hey guys, wonderful news. At the moment I saw this update, I've put my hands on it.
Using an Arch distro here, installed the beta Nvidia Cuda drivers, installed gedit on my distro (which installed automatically Mesa-21.0 release) and I can run gui apps which looks like to take into account gpu acceleration.

My problem is it looks like to take the wrong GPU as I have a Optimus system (so an intel card + a Nvidia discrete GPU) + in addition an external more powerful NVidia GPU through Thunderbolt3). I would like to select this last (tb3) GPU.

Any way to do that?

ps: I've made some tests using Gimp, glxgears and a linux game made with Unity3D Engine (ie. https://www.quantex-online-entertainment.net/ ), and it should be on the wrong GPU... as display is jerky.

Thanks in advance for further info.

Well I got RAIL to work. I built replacement Weston and FreeRDP .deb packages for Ubuntu 20.04. I was able to connect using mstsc on Windows (locally with a non-Insider build) and FreeRDP on Linux. With mstsc, I was even able to use "seamless" mode with individual applications. I couldn't get this to work with FreeRDP, even though I'd also patched the client. The server complains with:

HiDef-RAIL is required for RAIL.

That aside, I don't think the performance is any better than it was when I ran the vanilla stack last year. It's not even that good locally. Xpra's performance is much better. I know this is why you've created VAIL but I had hoped it would still be a bit better than it was. On the plus side, the clipboard works now!

NVidia GPU through Thunderbolt3). I would like to select this last (tb3) GPU.

External GPU was raised at #5604. It would probably suffice as a landing zone for the GUI use case as well.

Ok, cool to see there's an issue about it. What about switching to the discrete gpu in the meantime (which is internal to the laptop)? What's the way to do it?I will be a lot better than the intel GPU. I'm up to read some docs.

glxinfo do not give more then that currently:

[nox@NoX NoX]$ glxinfo|egrep "OpenGL vendor|OpenGL renderer"
OpenGL vendor string: Mesa/X.org
OpenGL renderer string: llvmpipe (LLVM 11.1.0, 256 bits)

Thanks anyway

Ah I guess this mail explains the error I saw about "HiDef".

FreeRDP does support the capabilities and messages but the rendering code of xfreerdp does not.

@3kynox: Maybe I misunderstood, but from the blog post and following into the Mesa 21 release notes, shoudn't the OpenGL renderer string be "d3d12", not "llvmpipe", if it's using the new Wayland setup?

Is it possible that the 3D acceleration is not working at all for you, rather than using the wrong GPU?

Edit: Yeah, the Arch Mesa 21 build does not enable the d3d12 driver. It looks like d3d12 needs to be added to the gallium-drivers list.

Ok, so a mesa21 build that includes d3d12 is required. You are probably right, I could not have GPU acceleration at all.
Thanks @TBBle for direction to take and research.

For a d3d12 enabled Mesa on Ubuntu I recommend the kisak PPA.

sudo add-apt-repository ppa:kisak/kisak-mesa
sudo apt-get update

I can vouch for @kisak-valve as I've frequently worked with him on Steam-related issues.

Ok, cool to see there's an issue about it. What about switching to the discrete gpu in the meantime (which is internal to the laptop)? What's the way to do it?I will be a lot better than the intel GPU. I'm up to read some docs.

glxinfo do not give more then that currently:

[nox@NoX NoX]$ glxinfo|egrep "OpenGL vendor|OpenGL renderer"
OpenGL vendor string: Mesa/X.org
OpenGL renderer string: llvmpipe (LLVM 11.1.0, 256 bits)

Thanks anyway

Just for the record (I know this was already clarified). llvmpipe is a software rasterizer - https://docs.mesa3d.org/drivers/llvmpipe.html.

Closing this issue. For all wslg issues, please file issues in https://github.com/microsoft/wslg/issues.