docs

Asynchronous GPU Driver

Record GPU work on the Renderer thread and execute it on your render thread.

On this page

GPUDriverAsync is an optional implementation of GPUDriver that records drawing calls into self-contained frames (GPUFrame). Your render thread executes these frames with your graphics API while the Renderer thread continues running without waiting.

This driver is designed for game engines where the graphics device lives on a dedicated render thread. The driver source is located in the SDK platform folder (#include <Platform/GPUDriverAsync.h>, under namespace ultralight::platform).

The Renderer thread is whichever thread calls Renderer::Render(). It does not have to be your application's main thread.

Each GPUFrame holds the drawing commands of one command list together with every texture, render buffer, and geometry change made before it. A frame represents a batch of GPU work rather than a display frame.

📘 AppCore Applications

AppCore apps never create this driver because AppCore manages its own driver. Use Settings::use_async_gpu_driver to control whether AppCore runs GPU work on a dedicated thread. It defaults to true on macOS and false on Windows— see App Lifecycle and Settings.

Creating the Driver

Configure GPUDriverAsync::Options and register the instance with Platform::set_gpu_driver() before creating the Renderer:

C++
#include <Ultralight/Ultralight.h>
#include <Platform/GPUDriverAsync.h>
#include <memory>

using namespace ultralight;
using namespace ultralight::platform;

std::unique_ptr<GPUDriverAsync> gpu_driver;
RefPtr<Renderer> renderer;
RefPtr<View> view;

void InitRenderer() {
  GPUDriverAsync::Options options;
  options.caps.supports_partial_redraw = 1;  // our targets keep their pixels

  // The driver must outlive the Renderer.
  gpu_driver = std::make_unique<GPUDriverAsync>(options);
  Platform::instance().set_gpu_driver(gpu_driver.get());
  renderer = Renderer::Create();
}

Set configuration options in GPUDriverAsync::Options to configure limits, capabilities, and payload allocation:

Field Default Effect
caps All zero Device capabilities that GetDeviceCaps() reports, fixed for the lifetime of the driver— see GPU Device Capabilities.
max_frames_in_flight 2 Number of frames queued or taken and not yet destroyed at which ready_for_frame() turns false.
memory_budget 0 (none) Memory in use, in bytes, above which ready_for_frame() turns false.
supports_readback false Whether your render thread handles readback requests.
readback_timeout_ms 1000 Maximum time in milliseconds that ReadbackTexture() waits, or GPUDriverAsync::kNoTimeout for no limit.
allocator nullptr Allocator for staging payloads; when null, the driver allocates from CPU memory.

Rendering Each Frame

On the Renderer thread, check ready_for_frame() before rendering to avoid queuing work when your render thread is busy:

C++
void Tick() {
  renderer->Update();
  renderer->RefreshDisplay(0);

  ///
  /// Skip Render() while the render thread is behind. The next Render()
  /// paints everything that changed in the meantime.
  ///
  if (gpu_driver->ready_for_frame())
    renderer->Render();
}

Skipping Renderer::Render() loses no visual updates. The library accumulates changes and renders everything that changed on the next call.

The ready_for_frame() method returns false once the number of in-flight frames reaches max_frames_in_flight, or when memory in use exceeds memory_budget. It always returns true when no frames are in flight, and is safe to call from any thread.

Executing Frames

On your render thread, call TryDequeue() in a loop to take queued frames and process their operations:

C++
void ExecuteFrames(GPUDriverAsync& driver) {
  GPUFrame frame;
  while (driver.TryDequeue(frame)) {
    for (const GPUTextureOp& t : frame.textures())
      ApplyTexture(t);  // Create and Update operations
    for (const GPURenderBufferOp& rb : frame.render_buffers())
      ApplyRenderBuffer(rb);
    for (const GPUGeometryOp& g : frame.geometry())
      ApplyGeometry(g);
    for (const Command& cmd : frame.commands())
      ExecuteCommand(cmd);  // as in Implementing a GPUDriver
    ApplyDestroys(frame);   // Destroy operations run after the commands
    for (const GPUReadbackOp& r : frame.readbacks())
      ServeReadback(driver, r);
  }
}

TryDequeue() takes the oldest frame from the queue and returns true. If the queue is empty, it returns false without blocking.

You must call TryDequeue() from only one render thread at a time. Passing a GPUFrame variable into the call automatically releases whatever frame it previously held.

Process each frame by applying its textures, render buffers, geometry, drawing commands, and readbacks in that order. Within each array, execute operations in the order they appear.

Run GPUResourceAction::Destroy operations after commands(), since the frame's drawing commands can still reference those resources. This ordering applies only to the sequence within the frame— you don't need to wait for the GPU to finish before processing them.

The drawing commands in commands() match the commands received by a synchronous driver's UpdateCommandList() method— see Implementing a GPUDriver.

🚧 Execute Every Frame in Order

You must execute every frame in the order it was queued. Never drop, merge, or reorder frames, since the library updates geometry in place and redraws content incrementally. To reduce GPU workload, skip Renderer::Render() on the Renderer thread instead.

Handling Resource Operations

Each resource operation has a unique resource id and a GPUResourceAction of Create, Update, or Destroy. The fields you use to create and update resources depend on the resource type:

Operation Create Action Update Action
Texture (GPUTextureOp) Create the texture using width, height, format, and flags. Upload initial data from pixels with row_pitch bytes per row. The pixels payload is empty for render targets and empty bitmaps. Textures with kGPUTextureFlag_Immutable in flags receive data only at creation and are never updated. Write pixels into rect. For block-compressed formats, rect is block-aligned and row_pitch is the distance between block rows. Block-compressed textures created without kGPUTextureFlag_Immutable can receive updates.
Render Buffer (GPURenderBufferOp) Create the render buffer from desc. None— render buffers are never updated.
Geometry (GPUGeometryOp) Allocate vertex_capacity and index_capacity bytes, then write the initial vertex and index data. Write vertices at byte offset vertex_offset and indices at byte offset index_offset. Either payload can be empty.

🚧 Preserve Existing Geometry Buffers

A geometry update operation modifies only the byte range at its offsets. Bytes outside that range keep their previous contents— never clear or reallocate an existing geometry buffer on update.

Waking Your Render Thread

Subclass GPUDriverAsync and override OnFrameQueued() to wake your render thread when new work arrives:

C++
class RenderThreadDriver : public GPUDriverAsync {
 public:
  using GPUDriverAsync::GPUDriverAsync;

 protected:
  void OnFrameQueued(uint64_t frame_id) override {
    // Pseudo-code, signal the render thread to call ExecuteFrames() here.
  }
};

The library calls OnFrameQueued() on the Renderer thread immediately after pushing a new frame into the queue. Use this callback to signal a condition variable or post a task to your render thread.

Managing Payload Memory

During Renderer::Render(), the driver copies changing pixel rows, geometry buffers, and commands into internal memory. You do not need to stage or copy this data yourself.

Immutable bitmaps (Bitmap::is_immutable()) are an exception— the driver retains references to their memory instead of copying pixels. For details on bitmap lifecycles, see Working with Bitmaps.

Payload data arrives in GPUMemory, which wraps a Buffer reference along with an offset, byte size, and usage flags. Call GPUMemory::data() to obtain a pointer to the start of the block.

Destroying a GPUFrame releases its payload memory. Frames can be destroyed on any thread. If your graphics pipeline needs to keep a memory block alive after destroying its frame, retain a copy of GPUMemory::buffer.

Using Custom Upload Memory

Modern graphics APIs like Direct3D 12 and Vulkan can read upload buffers directly on the GPU. You can provide a custom GPUAllocator through Options::allocator so payloads land directly in mapped memory that your GPU reads in place.

Ultralight provides GPUChunkAllocator, which implements GPUAllocator over a custom GPUChunkSource. Your allocator must outlive the driver.

Implement GPUChunkSource::CreateChunk() to allocate mapped memory chunks of at least the requested size. Store your graphics resource handle in Buffer::user_data(). The chunk allocator calls GPUChunkSource::DestroyChunk() whenever it releases an idle or retired chunk.

Track GPU synchronization through pending_fence() and completed_fence(). The pending_fence() method must return the fence value that your next fence signal will write, covering all recorded GPU work— including commands recorded but not yet submitted. The completed_fence() method returns the fence value that the GPU has reached. Because the library reads both fence values from any thread, implement them using atomic variables.

Override texture_row_alignment() and texture_offset_alignment() to match your graphics API's texture copy alignment requirements. If Allocate() returns an empty block, the driver automatically falls back to CPU memory.

Releasing Idle Allocator Memory

Calling GPUDriverAsync::PurgeMemory() releases the driver's idle CPU memory, but leaves any custom allocator untouched— the driver calls OnPurgeMemory() on the Renderer thread at the end of the purge so you can reclaim allocator memory too.

If you use GPUChunkAllocator, calling its PurgeMemory() method returns idle chunks to your GPUChunkSource through DestroyChunk() while keeping a few ready for the next frame.

Override OnPurgeMemory() in your driver subclass to forward the purge request:

C++
class UploadDriver : public GPUDriverAsync {
 public:
  UploadDriver(const Options& options, GPUChunkAllocator* allocator)
      : GPUDriverAsync(options), allocator_(allocator) {}

 protected:
  void OnPurgeMemory() override { allocator_->PurgeMemory(); }

 private:
  GPUChunkAllocator* allocator_;  // also set as Options::allocator
};

On the Renderer thread, run both purges together to reclaim all idle memory:

C++
void ReleaseIdleMemory() {
  renderer->PurgeMemory();
  gpu_driver->PurgeMemory();
}

Using the Metal Driver

Ultralight provides ready-made GPUDriverAsync subclasses in the Platform module for specific graphics APIs, including Direct3D 11, Direct3D 12, and Metal. These drivers record commands like GPUDriverAsync and execute them directly, so you don't need to write a dequeue loop or handle resource operations by hand.

Creating the Metal Driver

Instantiate GPUDriverAsyncMetal with an initialized GPUContextMetal and register it before creating the renderer:

C++
#include <Platform/Factories.h>
#include <Platform/gpu/metal/GPUContextMetal.h>
#include <Platform/gpu/metal/GPUDriverAsyncMetal.h>

std::unique_ptr<GPUContextMetal> gpu_context;
std::unique_ptr<GPUDriverAsyncMetal> metal_driver;

void InitRenderer() {
  gpu_context = CreateGPUContextMetal(MTLCreateSystemDefaultDevice(), false);

  // The context outlives the driver; the driver outlives the Renderer.
  metal_driver = std::make_unique<GPUDriverAsyncMetal>(gpu_context.get());
  Platform::instance().set_gpu_driver(metal_driver.get());
  renderer = Renderer::Create();
}

Compile this setup file as Objective-C++ (.mm), since CreateGPUContextMetal() is declared only there.

The driver checks the Metal device directly, filling in Options::caps and Options::supports_readback from hardware capabilities. When Options::allocator is null, the driver allocates staging memory from internal Metal upload buffers based on your configuration's memory profile.

Executing and Presenting

On your render thread, call DrainFrames() to process every queued frame in order. You can also pass a GPUDrainBudget to limit execution by time or byte size— if the budget stops execution early, DrainFrames() returns false and resumes from that operation on your next call.

Executing a frame encodes commands into the GPUContextMetal command buffer without waiting for the GPU. The recorded work reaches the GPU once that command buffer is committed.

Drain queued frames and commit your drawing during each window frame:

C++
void PresentFrame() {
  metal_driver->DrainFrames();
  gpu_context->BeginDrawing();
  // Pseudo-code, draw your composite through metal_driver->sync_driver().
  gpu_context->EndDrawing();
}

Allocate render buffer identifiers for your MetalDrawableTarget instances through sync_driver() so they share this driver's identifier space. A frame that draws into a target must execute while that target's drawable is current.

If all windows are hidden or inactive, commit the encoded command buffer directly:

C++
void ExecuteWithoutPresenting() {
  metal_driver->DrainFrames(GPUDrainBudget{});
  gpu_context->CommitCommandBuffer();
}

🚧 Commit Every Command Buffer

If you drain frames without presenting a window or calling GPUContextMetal::CommitCommandBuffer(), the GPU never runs the commands. The driver cannot reuse its upload memory until those commands finish executing.

You must call DrainFrames(), ExecuteFrame(), and sync_driver() from one render thread at a time. State queries such as last_executed_frame_id() are safe to call from any thread.

Stop your render thread before destroying the driver, because its destructor waits for the GPU to finish all pending work.

Using Metal Upload Chunks

UploadChunkSourceMetal implements GPUChunkSource using Metal shared buffers (MTLStorageModeShared). The CPU writes texture and geometry data into memory that the GPU reads directly during copy operations.

GPUDriverAsyncMetal creates a chunk source internally, but you can also pair one with your own GPUChunkAllocator to manage custom upload memory.

Set up the chunk source with a GPUChunkAllocator, and tear them down once the GPU finishes pending work:

C++
std::unique_ptr<UploadChunkSourceMetal> upload_source;
std::unique_ptr<GPUChunkAllocator> upload_allocator;

void InitUploads() {
  upload_source = std::make_unique<UploadChunkSourceMetal>(gpu_context.get());
  upload_allocator = std::make_unique<GPUChunkAllocator>(
      upload_source.get(), Platform::instance().config().memory_profile);
}

void ShutdownUploads() {
  // Wait for every command buffer committed so far.
  gpu_context->WaitForFenceValue(gpu_context->pending_fence_value() - 1);
  upload_source->MarkGPUIdle();
  upload_allocator.reset();
  upload_source.reset();
}

The chunk source reports frame fence values from its GPUContextMetal. Be sure to commit every command buffer that reads these chunks through GPUContextMetal::Commit() or a method that calls it.

🚧 Signal GPU Idle Before Teardown

Always wait for the GPU to idle and call MarkGPUIdle() before destroying the GPUChunkAllocator. The allocator retires chunks at the pending fence value during destruction— if nothing signals that fence, its destructor waits forever.

Reading Pixels Back

The ReadbackTexture() method copies pixel data from a render-target texture into a Bitmap after all earlier command lists finish executing.

To enable readbacks, set Options::supports_readback to true. Call ReadbackTexture() on the Renderer thread, but never between BeginSynchronize() and EndSynchronize(). The destination bitmap must use BitmapFormat::BGRA8_UNORM_SRGB and be at least as large as rect.

The call blocks until the pixels arrive and returns true on success. It returns false if readbacks are disabled, arguments are invalid, an error occurs, or the wait exceeds Options::readback_timeout_ms.

On your render thread, serve each GPUReadbackOp in frame.readbacks() by copying rect of texture_id into destination. Place the top row of rect at row 0, spacing each row row_pitch bytes apart. When the copy finishes or fails, call CompleteReadback() with the operation's request_id and the success flag— you can call this from any thread.

Threading Rules

Different driver operations must run on specific threads to maintain thread safety:

Thread Allowed Calls
Renderer thread All GPUDriver methods that the library calls, along with ReadbackTexture() and PurgeMemory(). Driver callbacks OnFrameQueued(), OnPurgeMemory(), and OnReadbackWait(). Custom allocator methods GPUAllocator::Allocate(), GPUChunkSource::CreateChunk(), and GPUChunkSource::DestroyChunk().
Your render thread (one thread at a time) TryDequeue().
Any thread Moving or destroying a GPUFrame, CompleteReadback(), ready_for_frame() and other state getters, and NextTextureId(), NextRenderBufferId(), or NextGeometryId().

The role of your render thread can move to another thread only if the handoff is explicitly synchronized, such as joining the previous thread or synchronizing across a mutex.

Shutting Down

During application shutdown, stop your render thread and drain remaining frames before destroying the driver:

C++
void Shutdown() {
  view = nullptr;
  renderer = nullptr;

  ///
  /// The render thread is stopped. Run what is left, or just drop it.
  ///
  ExecuteFrames(*gpu_driver);
  // Pseudo-code, free every native texture, render buffer, and geometry
  // buffer here.

  gpu_driver.reset();
}

Tear down components in this sequence:

  1. Release all View instances and destroy the Renderer.
  2. Stop your render thread.
  3. Destroy all remaining GPUFrame instances and release any retained GPUMemory buffers, whether you execute the queued work or discard it.
  4. Free your graphics resources. The library emits no Destroy operations during shutdown.
  5. Destroy the GPUDriverAsync instance.
  6. Destroy your custom allocator, if you used one.

🚧 Lifetime Ordering During Teardown

You must destroy every GPUFrame and release all retained GPUMemory buffers before destroying the driver. Releasing a frame after destroying the driver attempts to access freed memory.

Related Pages