--- title: XR拍摄角色没有阴影的解决方案 date: 2022-12-05 14:01:39 excerpt: tags: XR rating: ⭐ --- # 解决方案 在不修改引擎的情况下，解决思路有： 1. [[#手动渲染阴影]] 2. [[#使用Composure进行合成]] 修改引擎的解决思路有： 1. ## 手动渲染阴影 - 使用ShadowMapMaterialFunction插件渲染渲染阴影并且贴在一个面片上。 ![](https://cdn.jsdelivr.net/gh/blueroseslol/ImageBag@latest/ImageBag/XR/XR_ShadowMap_Material.png) 缺点： 1. 如果快速旋转视角，阴影会有残影效果。**原因：因为这个阴影贴图不是在同一帧渲染出来的关系。** ## 使用Composure进行合成 - 使用Composure渲染 Floor（地面模型）、Shadow Layer（地面+角色模型），再通过Shader实现阴影+角色抠像方法。 - 使用Composure渲染 Floor（地面模型）、Reflection Layer（地面+角色模型），再通过Shader实现反射+角色抠像方法。 ![](https://cdn.jsdelivr.net/gh/blueroseslol/ImageBag@latest/ImageBag/XR/XR_UEComposure_ShadowReflection.png) ![](https://cdn.jsdelivr.net/gh/blueroseslol/ImageBag@latest/ImageBag/XR/XR_UEComposure_Material1.png) ![](https://cdn.jsdelivr.net/gh/blueroseslol/ImageBag@latest/ImageBag/XR/XR_UEComposure_Material2.png) 经过测试捕捉反射可以使用绿色面片作为背景，最后使用Chroma进行抠像。 ![](https://cdn.jsdelivr.net/gh/blueroseslol/ImageBag@latest/ImageBag/XR/XR_UEComposure_Reflection_Env.png) 注意：使用Composure捕捉的结果都是经过ToneMapping，如果把RT再次贴到场景中进行渲染会有2次Tonemaping只是结果“发白”，所以需要在几个Layer中添加Tonemap选项，进行反ToneMapping。 ![](https://cdn.jsdelivr.net/gh/blueroseslol/ImageBag@latest/ImageBag/XR/XR_UEComposure_Tonemap.png) 缺点： 1. 需要使用多次SceneCapture，会严重降低帧数。即使只捕捉单个物体，渲染消耗也和原始的渲染消耗一样。**建议：把角色与Composure相关的东西，放在一个子关卡中，场景放在一个子关卡中，然后2台机器使用SVN同步完工程之后，启动对应的关卡进行推流。** ## ~~UPlanarReflectionComponent~~ 核心函数： - UPlanarReflectionComponent（主要用于维护Scene的PlanarReflections数组以及数据更新标记bRegisteredReflectionCapturesHasChanged） - CreateRenderState_Concurrent()：创建FPlanarReflectionSceneProxy，并且调用Scene->AddPlanarReflection。 - SendRenderTransform_Concurrent()：调用Scene->UpdatePlanarReflectionTransform。 - DestroyRenderState_Concurrent()：调用Scene->RemovePlanarReflection(this)，并且删除渲染线程的FPlanarReflectionSceneProxy。 - PostEditChangeProperty()：更新可见性。 - FScene - UpdatePlanarReflectionContents()： - 清理RT并重新初始化RT - 计算反射平面与FSceneCaptureViewInfo（主要是View与Projaction矩阵） - 构建FSceneViewFamilyContext，可以理解为Capture的画布与相关View变量 - 调用FSceneRenderer::CreateSceneRenderer()创建渲染器 - 在渲染线程调用UpdatePlanarReflectionContents_RenderThread()渲染反射结果 ```c++ // Reflection view late update if (SceneRenderer->Views.Num() > 1) { const FMirrorMatrix MirrorMatrix(MirrorPlane); for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex) { FViewInfo& ReflectionViewToUpdate = SceneRenderer->Views[ViewIndex]; const FViewInfo& UpdatedParentView = MainSceneRenderer->Views[ViewIndex]; ReflectionViewToUpdate.UpdatePlanarReflectionViewMatrix(UpdatedParentView, MirrorMatrix); } } // Render the scene normally { RDG_RHI_EVENT_SCOPE(GraphBuilder, RenderScene); SceneRenderer->Render(GraphBuilder); } SceneProxy->RenderTarget = RenderTarget; // Update the view rects into the planar reflection proxy. for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex) { // Make sure screen percentage has correctly been set on render thread. check(SceneRenderer->Views[ViewIndex].ViewRect.Area() > 0); SceneProxy->ViewRect[ViewIndex] = SceneRenderer->Views[ViewIndex].ViewRect; } FRDGTextureRef ReflectionOutputTexture = GraphBuilder.RegisterExternalTexture(CreateRenderTarget(RenderTarget->TextureRHI, TEXT("ReflectionOutputTexture"))); GraphBuilder.SetTextureAccessFinal(ReflectionOutputTexture, ERHIAccess::SRVGraphics); FSceneTextureShaderParameters SceneTextureParameters = CreateSceneTextureShaderParameters(GraphBuilder, &SceneRenderer->GetActiveSceneTextures(), SceneRenderer->FeatureLevel, ESceneTextureSetupMode::SceneDepth); const FMinimalSceneTextures& SceneTextures = SceneRenderer->GetActiveSceneTextures(); for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex) { FViewInfo& View = SceneRenderer->Views[ViewIndex]; RDG_GPU_MASK_SCOPE(GraphBuilder, View.GPUMask); if (MainSceneRenderer->Scene->GetShadingPath() == EShadingPath::Deferred) { PrefilterPlanarReflection(GraphBuilder, View, SceneTextureParameters, SceneProxy, SceneTextures.Color.Resolve, ReflectionOutputTexture); }else { PrefilterPlanarReflection(GraphBuilder, View, SceneTextureParameters, SceneProxy, SceneTextures.Color.Resolve, ReflectionOutputTexture); } } ``` 可行的实现思路： 1. 将`class FPlanarReflectionRenderTarget* RenderTarget`的结果覆盖到对应的`UTextureRenderTarget2D* `中。 2. 使用`UTextureRenderTarget2D`数据来对`FPlanarReflectionRenderTarget`进行初始化。 1. 扩展FPlanarReflectionRenderTarget类，添加新的构造函数，将UTextureRenderTarget2D的数据来扩充内部的RT。进行手动的初始化。 2. 扩展UPlanarReflectionComponent类，增加UTextureRenderTarget2D选项。 但因为RenderTarget是Private，所以不修改源码的情况无法实现。 ## 使用SceneCapture2D模拟UPlanarReflectionComponent 位于PlanarReflectionRendering.cpp的UpdatePlanarReflectionContents_RenderThread() 计算视锥与反射平面的代码： ```c++ const FMatrix ComponentTransform = CaptureComponent->GetComponentTransform().ToMatrixWithScale(); FPlane MirrorPlane = FPlane(ComponentTransform.TransformPosition(FVector::ZeroVector), ComponentTransform.TransformVector(FVector(0, 0, 1))); // Normalize the plane to remove component scaling bool bNormalized = MirrorPlane.Normalize(); if (!bNormalized) { MirrorPlane = FPlane(FVector(0, 0, 1), 0); } for (int32 ViewIndex = 0; ViewIndex < MainSceneRenderer.Views.Num() && ViewIndex < GMaxPlanarReflectionViews; ++ViewIndex) { const FViewInfo& View = MainSceneRenderer.Views[ViewIndex]; FSceneCaptureViewInfo NewView; FVector2D ViewRectMin = FVector2D(View.UnscaledViewRect.Min.X, View.UnscaledViewRect.Min.Y); FVector2D ViewRectMax = FVector2D(View.UnscaledViewRect.Max.X, View.UnscaledViewRect.Max.Y); ViewRectMin *= FMath::Clamp(CaptureComponent->ScreenPercentage / 100.f, 0.25f, 1.f); ViewRectMax *= FMath::Clamp(CaptureComponent->ScreenPercentage / 100.f, 0.25f, 1.f); NewView.ViewRect.Min.X = FMath::TruncToInt(ViewRectMin.X); NewView.ViewRect.Min.Y = FMath::TruncToInt(ViewRectMin.Y); NewView.ViewRect.Max.X = FMath::CeilToInt(ViewRectMax.X); NewView.ViewRect.Max.Y = FMath::CeilToInt(ViewRectMax.Y); // Create a mirror matrix and premultiply the view transform by it const FMirrorMatrix MirrorMatrix(MirrorPlane); const FMatrix ViewMatrix(MirrorMatrix * View.ViewMatrices.GetViewMatrix()); const FVector ViewLocation = ViewMatrix.InverseTransformPosition(FVector::ZeroVector); const FMatrix ViewRotationMatrix = ViewMatrix.RemoveTranslation(); const float HalfFOV = FMath::Atan(1.0f / View.ViewMatrices.GetProjectionMatrix().M[0][0]); FMatrix ProjectionMatrix; BuildProjectionMatrix(View.UnscaledViewRect.Size(), HalfFOV + FMath::DegreesToRadians(CaptureComponent->ExtraFOV), GNearClippingPlane, ProjectionMatrix); NewView.ViewLocation = ViewLocation; NewView.ViewRotationMatrix = ViewRotationMatrix; NewView.ProjectionMatrix = ProjectionMatrix; NewView.StereoPass = View.StereoPass; NewView.StereoViewIndex = View.StereoViewIndex; SceneCaptureViewInfo.Add(NewView); } ``` ```c++ void BuildProjectionMatrix(FIntPoint InRenderTargetSize, float InFOV, float InNearClippingPlane, FMatrix& OutProjectionMatrix) { float const XAxisMultiplier = 1.0f; float const YAxisMultiplier = InRenderTargetSize.X / float(InRenderTargetSize.Y); if ((int32)ERHIZBuffer::IsInverted) { OutProjectionMatrix = FReversedZPerspectiveMatrix( InFOV, InFOV, XAxisMultiplier, YAxisMultiplier, InNearClippingPlane, InNearClippingPlane ); } else { OutProjectionMatrix = FPerspectiveMatrix( InFOV, InFOV, XAxisMultiplier, YAxisMultiplier, InNearClippingPlane, InNearClippingPlane ); }} ``` 其他计算方式： ```c++ bool IsInFrustum( AActor* Actor) { ULocalPlayer* LocalPlayer = GetWorld()->GetFirstLocalPlayerFromController(); if (LocalPlayer != nullptr && LocalPlayer->ViewportClient != nullptr && LocalPlayer->ViewportClient->Viewport) { FSceneViewFamilyContext ViewFamily(FSceneViewFamily::ConstructionValues( LocalPlayer->ViewportClient->Viewport, GetWorld()->Scene, LocalPlayer->ViewportClient->EngineShowFlags) .SetRealtimeUpdate(true)); FVector ViewLocation; FRotator ViewRotation; FSceneView* SceneView = LocalPlayer->CalcSceneView(&ViewFamily, ViewLocation, ViewRotation, LocalPlayer->ViewportClient->Viewport); if (SceneView != nullptr) { return SceneView->ViewFrustum.IntersectSphere( Actor->GetActorLocation(), Actor->GetSimpleCollisionRadius()); } } return false } ``` ### SceneCapture2D 位于SceneCaptureRendering.cpp # 其他 ## Disguise 支持的输入方式： - sdi传输（采集卡走这个协议） - ndi网络传输 （通过网卡网线） - smpte2110协议传输（显卡直接插大屏幕） ## Composure相关问题 - Transforms - Custom Material Pass - Post Process Pass Set - Tonemap - Multi Pass Chroma Keyer - Multi Pass Despill - Outputs - Media Capture：走采集卡推流模式。 - Image Sequence - Player Viewport：可以通过在UE中实现虚拟摄像机，再通过OBS推流。 - Render Target Asset 值得注意的是Transforms的Tonemap，这是一个反ToneMapping效果。