Issue with VkRenderPass load operation for Skybox rendering - intel

Question

I seem to have another issue when it comes to renderpasses in Vulkan.
Drawing my scene, I first submit a commandbuffer to render a sky using atmospheric scattering onto a cubemap, to which I then use for my forward pass to draw out the sky and sun.
The renderpass used when drawing the skybox and storing into a cubemap for sampling:
m_pFrameBuffer = rhi->CreateFrameBuffer();
VkImageView attachment = m_RenderTexture->View();
VkAttachmentDescription attachDesc = CreateAttachmentDescription(
m_RenderTexture->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_CLEAR,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
m_RenderTexture->Samples()
);
VkAttachmentReference colorRef = { 0, VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL };
std::array<VkSubpassDependency, 2> dependencies;
dependencies[0] = CreateSubPassDependency(
VK_SUBPASS_EXTERNAL,
VK_ACCESS_MEMORY_READ_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
0,
VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_DEPENDENCY_BY_REGION_BIT
);
dependencies[1] = CreateSubPassDependency(
0,
VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_SUBPASS_EXTERNAL,
VK_ACCESS_MEMORY_READ_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
VK_DEPENDENCY_BY_REGION_BIT
);
VkSubpassDescription subpassDesc = { };
subpassDesc.colorAttachmentCount = 1;
subpassDesc.pColorAttachments = &colorRef;
subpassDesc.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
VkRenderPassCreateInfo renderpassCi = { };
renderpassCi.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO;
renderpassCi.attachmentCount = 1;
renderpassCi.pAttachments = &attachDesc;
renderpassCi.dependencyCount = static_cast<u32>(dependencies.size());
renderpassCi.pDependencies = dependencies.data();
renderpassCi.subpassCount = 1;
renderpassCi.pSubpasses = &subpassDesc;
VkFramebufferCreateInfo framebufferCi = { };
framebufferCi.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO;
framebufferCi.height = kTextureSize;
framebufferCi.width = kTextureSize;
framebufferCi.attachmentCount = 1;
framebufferCi.layers = 1;
framebufferCi.pAttachments = &attachment;
m_pFrameBuffer->Finalize(framebufferCi, renderpassCi);
After rendering the skybox, and storing it into a cubemap, I used the following renderpass to sample the sky onto the rendered scene. This pass uses VK_LOAD_OP_LOAD so as to not clear the rendered scene when drawing the skybox onto it:
// Create a renderpass for the pbr overlay.
Texture* pbrColor = gResources().GetRenderTexture(PBRColorAttachStr);
Texture* pbrNormal = gResources().GetRenderTexture(PBRNormalAttachStr);
Texture* pbrPosition = gResources().GetRenderTexture(PBRPositionAttachStr);
Texture* pbrRoughMetal = gResources().GetRenderTexture(PBRRoughMetalAttachStr);
Texture* pbrDepth = gResources().GetRenderTexture(PBRDepthAttachStr);
Texture* RTBright = gResources().GetRenderTexture(RenderTargetBrightStr);
std::array<VkAttachmentDescription, 6> attachmentDescriptions;
VkSubpassDependency dependenciesNative[2];
attachmentDescriptions[0] = CreateAttachmentDescription(
pbrColor->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_LOAD,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
pbrColor->Samples()
);
attachmentDescriptions[1] = CreateAttachmentDescription(
pbrNormal->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_LOAD,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
pbrNormal->Samples()
);
attachmentDescriptions[2] = CreateAttachmentDescription(
RTBright->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_LOAD,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
RTBright->Samples()
);
attachmentDescriptions[3] = CreateAttachmentDescription(
pbrPosition->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_LOAD,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
pbrPosition->Samples()
);
attachmentDescriptions[4] = CreateAttachmentDescription(
pbrRoughMetal->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_LOAD,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
pbrRoughMetal->Samples()
);
attachmentDescriptions[5] = CreateAttachmentDescription(
pbrDepth->Format(),
VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL,
VK_ATTACHMENT_LOAD_OP_LOAD,
VK_ATTACHMENT_STORE_OP_STORE,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,
pbrDepth->Samples()
);
dependenciesNative[0] = CreateSubPassDependency(
VK_SUBPASS_EXTERNAL,
VK_ACCESS_MEMORY_READ_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
0,
VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_DEPENDENCY_BY_REGION_BIT
);
dependenciesNative[1] = CreateSubPassDependency(
0,
VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_SUBPASS_EXTERNAL,
VK_ACCESS_MEMORY_READ_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
VK_DEPENDENCY_BY_REGION_BIT
);
std::array<VkAttachmentReference, 5> attachmentColors;
VkAttachmentReference attachmentDepthRef = { static_cast<u32>(attachmentColors.size()), VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL };
attachmentColors[0].attachment = 0;
attachmentColors[0].layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
attachmentColors[1].attachment = 1;
attachmentColors[1].layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
attachmentColors[2].attachment = 2;
attachmentColors[2].layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
attachmentColors[3].attachment = 3;
attachmentColors[3].layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
attachmentColors[4].attachment = 4;
attachmentColors[4].layout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
VkSubpassDescription subpass = {};
subpass.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
subpass.colorAttachmentCount = static_cast<u32>(attachmentColors.size());
subpass.pColorAttachments = attachmentColors.data();
subpass.pDepthStencilAttachment = &attachmentDepthRef;
VkRenderPassCreateInfo renderpassCI = CreateRenderPassInfo(
static_cast<u32>(attachmentDescriptions.size()),
attachmentDescriptions.data(),
2,
dependenciesNative,
1,
&subpass
);
VkResult result =
vkCreateRenderPass(rhi->LogicDevice()->Native(), &renderpassCI, nullptr, &m_SkyboxRenderPass);
This is the command buffer for rendering the sky onto my scene. I submit this commandbuffer after rendering the scene to take advantage of early z rejection:
if (m_pSkyboxCmdBuffer) {
m_pRhi->DeviceWaitIdle();
m_pSkyboxCmdBuffer->Reset(VK_COMMAND_BUFFER_RESET_RELEASE_RESOURCES_BIT);
}
VkCommandBufferBeginInfo beginInfo = { };
beginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
CommandBuffer* buf = m_pSkyboxCmdBuffer;
FrameBuffer* skyFrameBuffer = gResources().GetFrameBuffer(PBRFrameBufferStr);
GraphicsPipeline* skyPipeline = gResources().GetGraphicsPipeline(SkyboxPipelineStr);
DescriptorSet* global = m_pGlobal->Set();
DescriptorSet* skybox = gResources().GetDescriptorSet(SkyboxDescriptorSetStr);
VkDescriptorSet descriptorSets[] = {
global->Handle(),
skybox->Handle()
};
buf->Begin(beginInfo);
std::array<VkClearValue, 6> clearValues;
clearValues[0].color = { 0.0f, 0.0f, 0.0f, 1.0f };
clearValues[1].color = { 0.0f, 0.0f, 0.0f, 1.0f };
clearValues[2].color = { 0.0f, 0.0f, 0.0f, 1.0f };
clearValues[3].color = { 0.0f, 0.0f, 0.0f, 1.0f };
clearValues[4].color = { 0.0f, 0.0f, 0.0f, 1.0f };
clearValues[5].depthStencil = { 1.0f, 0 };
VkViewport viewport = {};
viewport.height = (r32)m_pWindow->Height();
viewport.width = (r32)m_pWindow->Width();
viewport.minDepth = 0.0f;
viewport.maxDepth = 1.0f;
viewport.y = 0.0f;
viewport.x = 0.0f;
VkRenderPassBeginInfo renderBegin = { };
renderBegin.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO;
renderBegin.framebuffer = skyFrameBuffer->Handle();
renderBegin.renderPass = m_pSky->GetSkyboxRenderPass();
renderBegin.clearValueCount = static_cast<u32>(clearValues.size());
renderBegin.pClearValues = clearValues.data();
renderBegin.renderArea.offset = { 0, 0 };
renderBegin.renderArea.extent = m_pRhi->SwapchainObject()->SwapchainExtent();
// Start the renderpass.
buf->BeginRenderPass(renderBegin, VK_SUBPASS_CONTENTS_INLINE);
buf->SetViewPorts(0, 1, &viewport);
buf->BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, skyPipeline->Pipeline());
buf->BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, skyPipeline->Layout(), 0, 2, descriptorSets, 0, nullptr);
VertexBuffer* vertexbuffer = m_pSky->GetSkyboxVertexBuffer();
IndexBuffer* idxBuffer = m_pSky->GetSkyboxIndexBuffer();
VkDeviceSize offsets[] = { 0 };
VkBuffer vert = vertexbuffer->Handle()->NativeBuffer();
VkBuffer ind = idxBuffer->Handle()->NativeBuffer();
buf->BindVertexBuffers(0 , 1, &vert, offsets);
buf->BindIndexBuffer(ind, 0, VK_INDEX_TYPE_UINT32);
buf->DrawIndexed(idxBuffer->IndexCount(), 1, 0, 0, 0);
buf->EndRenderPass();
buf->End();
Finally, I submit it inside my rendering function:
// TODO(): Need to clean this up.
VkCommandBuffer offscreenCmd = m_Offscreen._CmdBuffers[m_Offscreen._CurrCmdBufferIndex]->Handle();
VkCommandBuffer skyBuffers[] = { m_Offscreen._CmdBuffers[m_Offscreen._CurrCmdBufferIndex]->Handle(), m_pSky->CmdBuffer()->Handle() };
VkSemaphore skyWaits[] = { m_Offscreen._Semaphore->Handle(), m_pSky->SignalSemaphore()->Handle() };
VkSemaphore waitSemas[] = { m_pRhi->SwapchainObject()->ImageAvailableSemaphore() };
VkSemaphore signalSemas[] = { m_Offscreen._Semaphore->Handle() };
VkSemaphore shadowSignal[] = { m_Offscreen._ShadowSema->Handle() };
VkPipelineStageFlags waitFlags[] = { VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT };
VkSubmitInfo offscreenSI = {};
offscreenSI.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO;
offscreenSI.pCommandBuffers = &offscreenCmd;
offscreenSI.commandBufferCount = 1;
offscreenSI.signalSemaphoreCount = 1;
offscreenSI.pSignalSemaphores = signalSemas;
offscreenSI.waitSemaphoreCount = 1;
offscreenSI.pWaitSemaphores = waitSemas;
offscreenSI.pWaitDstStageMask = waitFlags;
VkSubmitInfo skyboxSI = offscreenSI;
VkSemaphore skyboxWaits[] = { m_Offscreen._Semaphore->Handle() };
VkSemaphore skyboxSignal[] = { m_SkyboxFinished->Handle() };
VkCommandBuffer skyboxCmd = m_pSkyboxCmdBuffer->Handle();
skyboxSI.commandBufferCount = 1;
skyboxSI.pCommandBuffers = &skyboxCmd;
skyboxSI.pSignalSemaphores = skyboxSignal;
skyboxSI.pWaitSemaphores = skyboxWaits;
VkSubmitInfo hdrSI = offscreenSI;
VkSemaphore hdrWaits[] = { m_SkyboxFinished->Handle() };
VkSemaphore hdrSignal[] = { m_HDR._Semaphore->Handle() };
VkCommandBuffer hdrCmd = m_HDR._CmdBuffers[m_HDR._CurrCmdBufferIndex]->Handle();
hdrSI.pCommandBuffers = &hdrCmd;
hdrSI.pSignalSemaphores = hdrSignal;
hdrSI.pWaitSemaphores = hdrWaits;
VkSemaphore waitSemaphores = m_HDR._Semaphore->Handle();
if (!m_HDR._Enabled) waitSemaphores = m_Offscreen._Semaphore->Handle();
// Update materials before rendering the frame.
UpdateMaterials();
// begin frame. This is where we start our render process per frame.
BeginFrame();
while (m_Offscreen._CmdBuffers[m_HDR._CurrCmdBufferIndex]->Recording() || !m_pRhi->CmdBuffersComplete()) {}
// Render shadow map here. Primary shadow map is our concern.
if (m_pLights->PrimaryShadowEnabled()) {
VkCommandBuffer shadowbuf[] = { m_Offscreen._ShadowCmdBuffers[m_Offscreen._CurrCmdBufferIndex]->Handle() };
VkSubmitInfo shadowSubmit = { };
shadowSubmit.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO;
shadowSubmit.pCommandBuffers = shadowbuf;
shadowSubmit.commandBufferCount = 1;
shadowSubmit.signalSemaphoreCount = 1;
shadowSubmit.waitSemaphoreCount = 1;
shadowSubmit.pWaitSemaphores = waitSemas;
shadowSubmit.pSignalSemaphores = shadowSignal;
shadowSubmit.pWaitDstStageMask = waitFlags;
// Submit shadow rendering.
m_pRhi->GraphicsSubmit(shadowSubmit);
offscreenSI.pWaitSemaphores = shadowSignal;
}
// Check if sky needs to update it's cubemap.
if (m_pSky->NeedsRendering()) {
skyboxSI.waitSemaphoreCount = 2;
skyboxSI.pWaitSemaphores = skyWaits;
offscreenSI.commandBufferCount = 2;
offscreenSI.signalSemaphoreCount = 2;
offscreenSI.pSignalSemaphores = skyWaits;
offscreenSI.pCommandBuffers = skyBuffers;
m_pSky->MarkClean();
}
// Offscreen PBR Forward Rendering Pass.
m_pRhi->GraphicsSubmit(offscreenSI);
// Render Sky onto our render textures.
m_pRhi->GraphicsSubmit(skyboxSI);
// High Dynamic Range and Gamma Pass.
if (m_HDR._Enabled) m_pRhi->GraphicsSubmit(hdrSI);
// Before calling this cmd buffer, we want to submit our offscreen buffer first, then
// sent our signal to our swapchain cmd buffers.
// TODO(): We want to hold off on signalling GraphicsFinished Semaphore, and instead
// have it signal the SignalUI semaphore instead. UI Overlay will be the one to use
// GraphicsFinished Semaphore to signal end of frame rendering.
VkSemaphore signal = m_pRhi->GraphicsFinishedSemaphore();
VkSemaphore uiSig = m_pUI->Signal()->Handle();
m_pRhi->SubmitCurrSwapchainCmdBuffer(1, &waitSemaphores, 1, &signal);
// Render the Overlay.
RenderOverlay();
EndFrame();
On an Nvidia GTX 870M, the results seem to work as expected,
However, using Intel HD Graphics 620, I get this screenshot, unfortunately I can't display here because it's too big: https://github.com/CheezBoiger/Recluse-Game/blob/master/Regression/Shaders/ForwardPass.png
It seems as though the scene from previous frames are left un-cleared onto the color attachment, as if it was rendering onto a separate surface and using that instead, but it should be cleared every frame at the beginning of rendering...
Removing VK_LOAD_OP_LOAD and replacing with VK_LOAD_OP_CLEAR, the situation clears, however, only the skybox is rendered... I am wondering if my render pass is not doing something that it needs to be doing on Intel hardware, or am I going about drawing the skybox onto my rendered scene all wrong?
Much appreciated on the help.
* Update *
Problem fixed, with solution by #Ekzuzy below.
Final Image on Intel Hardware after fix:

Answer 1

You always provide UNDEFINED layout for the initial layout in all Your render passes and for all attachments. Layout transition from UNDEFINED layout to any other layout doesn't guarantee image contents to be preserved. So if You create a render pass with LOAD value for the load op, You need to provide an actual layout given image has just before the render pass starts. This applies to other layout transitions as well (through memory barriers).
As for clears, some images should be cleared at the beginning of a frame or render pass. So for them You can leave UNDEFINED as the initial layout but You should change the load op to clear.
As to why this works on Nvidia and doesn't work on Intel - layout transitions don't have any effect on Nvidia's hardware, but they are important on Intel's platforms (and on AMD's too). So skipping (or setting improper) layout transitions, even though it violates the specification, it still should work on Nvidia. But don't do that just because it works. Such approach is invalid. And future platforms, even from the same vendor, may behave differently.