Optimizing memory (VRAM) usage for GPU rendering

Introduction

For a scene (incl. geometry, textures, frame buffer, etc.) to be rendered in V-Ray GPU, it must fully fit into GPU memory (VRAM) in either CUDA or RTX engines. Not having enough GPU memory might slow down rendering considerably or crash with or without an out-of-memory error.

Warnings / Error messages due to insufficient memory

• Hanging or crashing
• Slower rendering warning

Num samples per thread reduced to XXXXX, rendering might be slower

• Out-of-memory errors

[GPU Light Cache] Could not allocate buffer of size XXXX MB!

What can I do to optimize VRAM usage?

• Using the Progressive image sampler

  The Progressive image sampler is generally faster than Bucket mode and consumes less memory.

• Switching to the V-Ray GPU CUDA engine

  Swapping engines from RTX to CUDA will reduce GPU memory usage by around 2-3 GB, depending on the scene.

• Using Compressed textures (V-Ray 6.1 and later)

  The Compressed Mode uses special compression for your bitmaps to reduce memory usage without sacrificing quality. It could be very useful for scenes and assets using many high-resolution bitmaps. 

  Not all textures are compressed using this option; exceptions are made based on the bitmap usage. For example:

  • Bitmaps used for both normal/bump and color maps
  • Bitmaps used for displacement
  • Bitmaps used for lens effects.

The texture compression happens in a pre-pass before the rendering process starts. This mode affects rendering performance slightly for machines with a single GPU. Compressed Textures mode also works well for animation rendering, reducing flickering.

•  Using the Out-Of-Core feature (V-Ray 6.2 and later)

  V-Ray GPU can offload textures to system memory to lower GPU memory usage significantly with little impact on performance. You can enable OOC in the UI by enabling the "Use System Memory for Textures" option.

• Lowering GPU Light Cache Subdivs

  Higher values will use more GPU memory. Ideally, the Light Cache Subdivs should be between 1000 to 3000 based on rendering resolution. For 4K images or animations, a value of 3000 is generally sufficient.

• Using fewer Render Elements ​

  Using a lot of render elements or VRayLightMix will increase GPU memory usage. Limiting the amount of Render Elements to what's needed for the final frame will help reduce the utilized VRAM.

• Rendering with V-Ray Standalone

  Host applications, just like V-Ray GPU, need some VRAM to keep the scene open. Depending on scene complexity, the host application may occupy a few gigabytes of VRAM, which could be very valuable for V-Ray GPU. Exporting the scene to a .vrscene file and rendering it with V-Ray Standalone will minimize VRAM consumption by excluding the Host Application.

Here's how:

1. Go to the V-Ray shelf menu and choose .vrscene exporter

2. Export a .vrscene

   

3. Open a new text document (f.e, Notepad)
4. Paste the following code:

"C:\ProgramData\Autodesk\ApplicationPlugins\VRay3dsMax20XX\bin\vray.exe" -sceneFile=""""C:\Users\muham\Desktop\exported.vrscene"""" -rtEngine=7 -display=1 -verboseLevel=3 -showProgress=1

Learn more about the V-Ray Standalone flags.

5. Replace the vray.exe path according to your DCC and V-Ray version:
    

• V-Ray for Rhino: C:\Program Files\Chaos\V-Ray\V-Ray for Rhinoceros\vrayappsdk\bin
• V-Ray for SketchUp: C:\Program Files\Chaos\V-Ray\V-Ray for SketchUp\extension\vrayappsdk\bin
• V-Ray for Revit: C:\Program Files\Chaos\V-Ray\V-Ray for Revit\Libraries\vrayappsdk\bin
• V-Ray for 3dsMax: C:\ProgramData\Autodesk\ApplicationPlugins\VRay3dsMax20XX\bin
• V-Ray for Maya: C:\Program Files\Chaos\V-Ray\Maya 20XX for x64\maya_vray\bin
• V-Ray for Houdini:  C:\Program Files\Chaos\V-Ray\Houdini_XX.X\appsdk\bin
• V-Ray for Cinema4D: C:\Program Files\Maxon Cinema 4D <version>\plugins\V-Ray\res\libs\win64

• V-Ray for Blender: C:\Users\<username>\AppData\Roaming\Blender Foundation\Blender\<Blender version>\scripts\addons\vray_blender\bin\VRayZmqServer\appsdk\bin

   

6. Replace the -sceneFile path with the freshly exported .vrscene's path
7. Use -rtEngine=7 is for the RTX engine, and -rtEngine=5 is for the CUDA engine
8. Save the text document as .bat
9. Double-click on the .bat file to start rendering in V-Ray Standalone

Use the Device Selector tool to set your rendering devices for V-Ray Standalone. Learn how to use the Device Selector.

• Having a separate device for the Viewport

Adding a mid-range GPU like the RTX 3060 12 GB to your workstation could be helpful for monitor connections. This GPU will be highlighted by an asterisk symbol in the Device Selector tool. This way, host DCC and any other open applications will not use the GPU memory on your rendering devices.

If you have more than one GPU, you can set one of your GPUs to be used for Viewport Performance. Open NVIDIA Control Panel, and set the OpenGL rendering GPU to this device. The host DCC will use this device for viewport performance.

• Optimizing Displacement

Displacement and Subdivision of objects tend to consume a lot of GPU memory. A few things to consider:

1. Lowering the Edge Length in the Global Displacement Defaults / VRayDisplacementMod

2. Enabling the View-Dependent mode in the Global Displacement Defaults / VRayDisplacementMod.

   This option causes the Edge Length to be calculated in Screen pixels. For higher resolutions, the Edge Length should be lowered accordingly.

   

For quick troubleshooting, you can disable/enable displacement from Global Swatches in the Settings tab to see its impact on GPU memory usage.

• Using Chaos Cloud

Chaos Cloud is a brilliant option to render projects that can't fit into a local machine's memory. It's a one-click solution that will help you render the scene without investing in additional hardware or losing time to optimize the scene. Currently, Chaos Cloud uses Nvidia Tesla T4 16 GB GPUs, so it is limited to 16 GB of GPU memory. Learn more about Submitting Render Jobs to Chaos Cloud in Host Applications.

• Closing other applications that use GPU memory

It's not unusual to have multiple applications open when working on a project. Although it's very handy to have all the needed apps up and running, this will also reduce the amount of GPU memory available for the V-Ray GPU. Applications like Substance Painter or Nuke utilize a lot of memory. This approach will reduce VRAM usage from the GPU devices connected to monitors, as already mentioned above.

• Using the V-Ray GPU CUDA-x86 engine (CPU)

Although this method is not an effective approach in terms of render times, since V-Ray GPU won't benefit from using your fast GPUs, it is still possible for a scene to be rendered with V-Ray GPU using the V-Ray GPU CUDA-x86 engine (CPU). This method uses system memory (paging is supported). Learn more in the article about Hybrid Rendering.