WebRender and OpenGL

The general concept of interacting between OpenGL and webrender is fairly simple - you draw to a texture,hand it to webrender and webrender draws it when necessary. What can be quite confusing however, is theway that the process of drawing to an OpenGL texture is structured.

The problem, however, is that if we'd allow directly pushing a Texture into the DOM, there would beno way of knowing how large that texture needs to be, since the size can depend on the size and numberof its sibling DOM nodes.

We need to know the size of the texture for aspect ratio correction and preventing stretched / blurrytextures. Since the size of the rectangle that the texture should cover isn't known until it is time tolayout the frame, we have to "delay" the rendering of the texture. In azuls case, the DOM-building stepsimply pushes a GlTextureCallback instead of the texture itself, i.e. a function that will renderthe texture in the future (after the layout step).

The definition of a GlTextureCallback is the following:

  1. pub struct GlTextureCallback<T: Layout>(pub fn(&StackCheckedPointer<T>, LayoutInfo<T>, HidpiAdjustedBounds) -> Option<Texture>);

The HidpiAdjustedBounds contains the width, height of the desired texture as well as HiDPI informationthat you might need to scale the content of your texture correctly. If the callback returns None, theresult is simply a white square. The LayoutInfo allows you to create an OpenGL texture like this:

  1. let mut texture = window_info.window.create_texture(
  2.     hi_dpi_bounds.physical_size.width as usize, 
  3.     hi_dpi_bounds.physical_size.height as usize);

This creates an empty, uninitialized GPU texture. Note that we use the physical_size instead ofthe logical_size - the "logical size" is the HiDPI-adjusted version (which is usually what you wantto calculate UI metrics), but the physical size is necessary in this case to provide the actual size ofthe texture, without a HiDPI scaling factor.

Next, we clear the texture with the color red (255, 0, 0) and return it:

  1. texture.as_surface().clear_color(1.0, 0.0, 0.0, 1.0);
  2. return Some(texture);

Here, the .as_surface() activates the texture as the current FBO and draws to it. In this case weonly clear the texture and return it, but of course, you can do much more here - upload and drawvertices and textures, activate and bind shaders, etc. See the Surface trait for more details.

Azul requires at least OpenGL 3.1, which is checked at startup. You can rely on any function ofOpenGL 3.1 being available at the time the callback is called.

Here is a simple, full example:

  1. impl Layout for OpenGlAppState {
  2.     fn layout(&self, _info: LayoutInfo) -> Dom<Self> {
  3.         // See below for the meaning of StackCheckedPointer::new(self)
  4.         Dom::new(NodeType::GlTexture(GlTextureCallback(render_my_texture), StackCheckedPointer::new(self)))
  5.     }
  6. }
  7.  
  8. fn render_my_texture(
  9.     state: &StackCheckedPointer<OpenGlAppState>, 
  10.     info: LayoutInfo<OpenGlAppState>, 
  11.     hi_dpi_bounds: HidpiAdjustedBounds) 
  12. -> Option<Texture> 
  13. {
  14.     let mut texture = info.window.create_texture(
  15.         hi_dpi_bounds.physical_size.width as usize, 
  16.         hi_dpi_bounds.physical_size.height as usize);
  17.  
  18.     texture.as_surface().clear_color(0.0, 1.0, 0.0, 1.0);
  19.     Some(texture)
  20. }

This should give you a window with a red texture spanning the entire window. Remember than if aDiv isn't limited in width / height, it will try to fill its parent, in this case the entire window.Try adding another Div in the layout() function and laying them out horizontally via CSS. You candecorate, skew, etc. your texture with CSS as you like. You can even use clipping and borders - OpenGLtextures get treated the same way as regular images.

Using and updating the components state in OpenGL textures

Let's say you have a component that draws a cube to an OpenGL texture. You want to update thecube's rotation, translation and scale from another UI component. How would you implement such a widget?

By now you have probably noticed the StackCheckedPointer<T> that gets passed into the callback.This allows you to build a stack-allocated "component" that takes care of rendering itself, withoutthe user calling any rendering code. As always, be careful to cast the pointer back to the type youcreated it with (see the [https://github.com/maps4print/azul/wiki/Two-way-data-binding] chapter onwhy StackCheckedPointer is unsafe and how to migitate this problem to build a type-safe API).

For example, we want to draw a cube, and control its rotation and scaling from another UI element.So we build a CubeControl that renders the cube and exposes an API to control the cubes rotationfrom any other UI element:

  1. // This is your "API" that other UI elements will mess with. For example, a user could hook up a button
  2. // to increase the scale by 0.1 every time a button is clicked.
  3. //
  4. // The point is that the CubeControl is just a dumb struct, it doesn't know about any other component.
  5. // The CubeControl contains all the "state" necessary for your renderer, the renderer itself doesn't know
  6. // about the state itself
  7. pub struct CubeControl {
  8.     pub translation: Vector3,
  9.     pub rotation: Quaternion,
  10.     pub scaling: Vector3,
  11. }
  12.  
  13. // This is your "rendering component", i.e. the thing that generates the DOM.
  14. // The procedure is similar to how you'd use a regular StackCheckedPointer
  15. #[derive(Default)]
  16. pub struct CubeRenderer { /* no state! */ }
  17.  
  18. impl CubeRenderer {
  19.     // The DOM stores the pointer to the state of the renderer. This state may be modified 
  20.     // by other UI controls before the GlTextureCallback is invoked.
  21.     pub fn dom<T: Layout>(data: &CubeControl) -> Dom<T> {
  22.          // Regular two-way data binding. Yes, you should use unwrap() here, since StackCheckedPointer
  23.          // will only fail if the data is not on the stack.
  24.          //
  25.          // Think of this as a StackCheckedPointer<CubeControl> - internally the `<CubeControl>` type is erased
  26.          // and you need to cast it back manually in the rendering callback
  27.          let ptr = StackCheckedPointer::new(data); 
  28.         Dom::new(NodeType::GlTexture(GlTextureCallback(Self::render), ptr))
  29.     }
  30.  
  31.     // Private rendering function. External code doesn't need to know or care how the 
  32.     // `CubeRenderer` renders itself.
  33.     fn render<T: Layout>(
  34.         state: &StackCheckedPointer<T>, 
  35.         info: LayoutInfo<T>, 
  36.         bounds: HidpiAdjustedBounds) 
  37.     {
  38.         // Important: The type of the StackCheckedPointer has been erased
  39.         // Casting the pointer back to anything else than a &mut CubeControl will invoke undefined behaviour.
  40.         // HOWEVER: This function is (and should be) private. Only you, the **creator** of this component 
  41.         // can invoke UB, not the user.
  42.         // 
  43.         // The way that the pointer is casted back is by giving it a function that has the same signature
  44.         // as the render() function, but with a `&mut CubeControl` instead of a `&StackCheckedPointer<T>`.
  45.         // You do not have to worry about aliasing the pointer or race conditions, that is what the 
  46.         // StackCheckedPointer takes care of.
  47.         fn render_inner(component_state: &mut CubeControl, info: LayoutInfo<T>, bounds: HidpiAdjustedBounds) -> Texture {
  48.              let texture = info.window.create_texture(width as u32, height as u32);
  49.              // render_cube_to_surface (not included in this example for brevity) takes the texture
  50.              // **and the current state of the component** and draws the cube on the surface according to the state
  51.              render_cube_to_surface(texture.as_surface(), &component_state);
  52.              // You could update your component_state here, if you'd like.
  53.              texture
  54.         }
  55.  
  56.         // Cast the StackCheckedPointer to a CubeControl, then invoke the render_inner function on it.
  57.         Some(unsafe { state.invoke_mut_texture(render_inner, state, info, bounds) })
  58.     }
  59. }

Now, why is this so complicated? The answer is that now the API for the user of this component is very easy:

  1. // The data model of the final program
  2. struct DataModel {
  3.     // Stores the state of the cubes rotation, scaling and translation.
  4.     // The user doesn't need to know about any other details
  5.     cube_control: CubeControl,
  6. }
  7.  
  8. impl Layout for DataModel {
  9.     fn layout(&self, _info: LayoutInfo<Self>) -> Dom<Self> {
  10.         CubeRenderer::default().dom(&self.cube_control)
  11.     }
  12. }

That's it! Now the user can hook up other components or custom callbacks that modify self.cube_control -but the application data is cleanly seperated from its view or other components.

The user does also not need to care about how the component (in this case our CubeRenderer) rendersitself, it is done "magically" by the framework - the framework determines when to call theGlTextureCallback and does so behind the back of the user. There is no code that the userhas to write in order to render a CubeRenderer. The only limitation is that the CubeControlhas to be stack-allocated, it can't be stored in a Vec or similar (because then, azul can'treason about the lifetime of the component, to make sure it's not dereferencing a dangling pointer).

What is StackCheckedPointer::new(self) ?

If you followed closely, you can probably already see what this is doing: StackCheckedPointer takes a reference to something on the stack that is contained in T. However, it can also take a reference to the entire data model (i.e. T itself). So StackCheckedPointer::new(self) essentially builds a StackCheckedPointer<OpenGlAppState> - the pointer can be safely casted back to a OpenGlAppState,at which point the callback has full control over the entire data model. Usually this is onlysomething you'd want to do for prototyping, it's better for maintentance to build a customcomponent as shown above, for type safety reasons.

Raw OpenGL

For ease of use, azul exposes primitives of the glium library, which provide functions, such asfor example .clear_color() - usually it's easier to work with that than with raw OpenGL - gliumprovides primitives for GLSL shader compilation and linking.

Right now OpenGL is the only supported backend and that will probably stay this way in the future -since webrender is only portable to platforms that can target OpenGL, it wouldn't make sense tosupport other rendering backends, since webrender, the main appeal of this entire library,wouldn't run on them. There are experiments of porting webrender to Vulkan / DirectX or Metal,however these are, as the name implies, experimental indeed.

Notes

  • It is not possible to render directly to the screen (for example, to use the built-in MSAA).