NVList internals 02: Rendering

Previous parts — 01 Image Loading

Rendering in NVList is handled entirely through OpenGL (ES on Android). There’s no software fallback renderer, but there’s not really a need for one either. I’ve heard terrible tales of the OpenGL implementations on Intel graphics cards, but other than awful performance I haven’t found any problems. Heck, even the "GDI generic" OpenGL 1.1 implementation built into Windows seems to work fine. The capabilities of NVList’s renderer attempt to closely mirror that of the underlying graphics hardware. Some parts (GLSL shaders) require 2004-era hardware, which is why they’re optional. I’d love to be able to count on fragment shader support, but unfortunately for anyone building a VN engine, your users will expect things to work even on their decade-old toaster.

Rendering 2D graphics with OpenGL can feel a little clumsy. What you want to do is "render sprites", what you actually end up implementing is "render alpha-blended textured quads on a camera-aligned plane". The sprites have to be rendered in order (back to front) for alpha blending to work properly. Traditional depth-buffering doesn’t work for semi-transparent pixels.

layers

The primary way of determining a draw order for sprites is by grouping them into layers. Each layer contains a number of drawables (layers are also drawables), each with an associated Z-value. Drawables with higher Z-values are rendered before (and thus underneath) ones with a lower Z-value. Each layer’s bounds are used as a clipping rectangle while rendering their contents.

Draw Buffer

Drawables don’t directly issue rendering calls to OpenGL. Instead they issue draw calls on a draw buffer object which then stores the draw calls per-layer and sorts them by Z-coordinate prior to rendering. The draw buffer also groups similar draw calls together in order to avoid unnecessary OpenGL state changes (color/texture/shader) during rendering. Grouping draw calls together like this makes it possible (and rather easy) to batch-render hundreds/thousands of sprites at once. Especially for particle effects with sometimes thousands of small sprites, batch rendering is critical to achieve optimal performance.

draw-buffer

The draw calls issued to the draw buffer are all objects inheriting from the RenderCommand class. The full list of draw call types is rather short:

Quad

By far the most common type. Renders a single sprite (or more accurately a textured quad).

BlendQuad

Used by the CrossFadeTween to blend between two different textures based on a weight factor.

FadeQuad

The shutter and wipe effects use this one to render a quad which gradually fades to transparent in one direction.

DistortQuad

Implements the ‘distort’ effect — the equivalent of running a vertex shader on a subdivided grid, but without requiring the hardware support.

Screenshot

Copies a rectangle of pixels from the render buffer into a texture. Depending on a boolean flag it may also download the pixels to main memory to be able to recreate the generated texture in case the OpenGL context is lost (which may happen at any time).

Custom

Allows arbitrary code to run inside the renderer. Not currently used and the only thing preventing the rendering code from running in its own thread. I’ll probably get rid of it if I ever get around to making the renderer properly asynchronous.

Layer

Makes a recursive call to renderLayer.

Multi-resolution support

The (virtual) screen resolution specified in game.ini may not match that of the physical screen. In this case, NVList has to perform scaling in some way. The preferred way of achieving this is to render everything to an off-screen buffer with a physical size exactly matching the virtual size, then scale the result and render it to the screen. It’s possible to render directly to the screen without the off-screen buffer, scaling down on the fly as each sprite is rendered (gives slightly better performance), but doing so may produce some small scaling artifacts.

vsize

For low-end devices, rendering at the full virtual resolution may be too costly. For a 1024×600 netbook, rendering everything at 1920×1080 doesn’t make a whole lot of sense. NVList supports pre-scaled image sets (for example 1920×1080/1280×720/1024×576), switching between sets based on the current resolution of the render target (which may change at runtime by resizing the window). The netbook mentioned earlier will then switch to 1024×576 images to match its 1024×600 resolution, lowering the strain on the system dramatically. Pre-scaled image sets are even more important on Android, where screen sizes vary wildly and system memory is tight.

The changing texture sizes are hidden from most of the code. NVList simply pretends the texture sizes stay the same regardless of which image set is selected. Only some internal code directly accessing pixels needs to take the image scale into account (stuff in imagefx mostly: blur/composite/etc). The renderer doesn’t care either, OpenGL texture coordinates are independent of texture size anyway.

3 thoughts on “NVList internals 02: Rendering

  1. I was wondering, if NVList works with different layers, how difficult could be to render a fixed/slighly-animated 3D model on one of those layers?
    I know performance is really ugly with some graphic card configurations, but it would be awesome to create 3D models for backgrounds, using a blur or bloom effect and rendering sprites on top of those.

    • The layers don’t even matter, you could just implement a ThreeDeeDrawable and use a custom render command to draw it. That gives you direct access to all OpenGL functionality and you can render whatever you want. The main problem would be writing code for loading/animating/displaying a 3D model — integration into NVList would be pretty easy.

      That said, you quickly reach a point where adding VN functionality to an existing 3D game engine is more convenient.

      • I’d like to try my hand on that aptly-named ThreeDeeDrawable based on some examples from JOGL.
        Maybe you’re right with your assertion that adding VN functionality to an existing 3D engine is easier, but for some weeks I’ve been trying my hand at Unity, and if I had any kind of experience with Mono/.NET as I have with Java, I’d be all over it.

Leave a comment

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>