227 lines
10 KiB
Markdown
227 lines
10 KiB
Markdown
|
---
|
|||
|
|
title: XR拍摄角色没有阴影的解决方案
|
|||
|
|
date: 2022-12-05 14:01:39
|
|||
|
|
excerpt:
|
|||
|
|
tags: XR
|
|||
|
|
rating: ⭐
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# 解决方案
|
|||
|
|
在不修改引擎的情况下，解决思路有：
|
|||
|
|
1. [[#手动渲染阴影]]
|
|||
|
|
2. [[#使用Composure进行合成]]
|
|||
|
|
|
|||
|
|
修改引擎的解决思路有：
|
|||
|
|
1.
|
|||
|
|
|
|||
|
|
## 手动渲染阴影
|
|||
|
|
- 使用ShadowMapMaterialFunction插件渲染渲染阴影并且贴在一个面片上。
|
|||
|
|
|
|||
|
|
|
|||
|
|
缺点：
|
|||
|
|
1. 如果快速旋转视角，阴影会有残影效果。**原因：因为这个阴影贴图不是在同一帧渲染出来的关系。**
|
|||
|
|
|
|||
|
|
## 使用Composure进行合成
|
|||
|
|
- 使用Composure渲染 Floor（地面模型）、Shadow Layer（地面+角色模型），再通过Shader实现阴影+角色抠像方法。
|
|||
|
|
- 使用Composure渲染 Floor（地面模型）、Reflection Layer（地面+角色模型），再通过Shader实现反射+角色抠像方法。
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
经过测试捕捉反射可以使用绿色面片作为背景，最后使用Chroma进行抠像。
|
|||
|
|
|
|||
|
|
|
|||
|
|
注意：使用Composure捕捉的结果都是经过ToneMapping，如果把RT再次贴到场景中进行渲染会有2次Tonemaping只是结果“发白”，所以需要在几个Layer中添加Tonemap选项，进行反ToneMapping。
|
|||
|
|
|
|||
|
|
|
|||
|
|
缺点：
|
|||
|
|
1. 需要使用多次SceneCapture，会严重降低帧数。即使只捕捉单个物体，渲染消耗也和原始的渲染消耗一样。**建议：把角色与Composure相关的东西，放在一个子关卡中，场景放在一个子关卡中，然后2台机器使用SVN同步完工程之后，启动对应的关卡进行推流。**
|
|||
|
|
|
|||
|
|
## ~~UPlanarReflectionComponent~~
|
|||
|
|
核心函数：
|
|||
|
|
- UPlanarReflectionComponent（主要用于维护Scene的PlanarReflections数组以及数据更新标记bRegisteredReflectionCapturesHasChanged）
|
|||
|
|
- CreateRenderState_Concurrent()：创建FPlanarReflectionSceneProxy，并且调用Scene->AddPlanarReflection。
|
|||
|
|
- SendRenderTransform_Concurrent()：调用Scene->UpdatePlanarReflectionTransform。
|
|||
|
|
- DestroyRenderState_Concurrent()：调用Scene->RemovePlanarReflection(this)，并且删除渲染线程的FPlanarReflectionSceneProxy。
|
|||
|
|
- PostEditChangeProperty()：更新可见性。
|
|||
|
|
- FScene
|
|||
|
|
- UpdatePlanarReflectionContents()：
|
|||
|
|
- 清理RT并重新初始化RT
|
|||
|
|
- 计算反射平面与FSceneCaptureViewInfo（主要是View与Projaction矩阵）
|
|||
|
|
- 构建FSceneViewFamilyContext，可以理解为Capture的画布与相关View变量
|
|||
|
|
- 调用FSceneRenderer::CreateSceneRenderer()创建渲染器
|
|||
|
|
- 在渲染线程调用UpdatePlanarReflectionContents_RenderThread()渲染反射结果
|
|||
|
|
|
|||
|
|
```c++
|
|||
|
|
// Reflection view late update
|
|||
|
|
if (SceneRenderer->Views.Num() > 1)
|
|||
|
|
{ const FMirrorMatrix MirrorMatrix(MirrorPlane);
|
|||
|
|
for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex)
|
|||
|
|
{ FViewInfo& ReflectionViewToUpdate = SceneRenderer->Views[ViewIndex];
|
|||
|
|
const FViewInfo& UpdatedParentView = MainSceneRenderer->Views[ViewIndex];
|
|||
|
|
|
|||
|
|
ReflectionViewToUpdate.UpdatePlanarReflectionViewMatrix(UpdatedParentView, MirrorMatrix);
|
|||
|
|
} }
|
|||
|
|
// Render the scene normally
|
|||
|
|
{
|
|||
|
|
RDG_RHI_EVENT_SCOPE(GraphBuilder, RenderScene);
|
|||
|
|
SceneRenderer->Render(GraphBuilder);
|
|||
|
|
}
|
|||
|
|
SceneProxy->RenderTarget = RenderTarget;
|
|||
|
|
|
|||
|
|
// Update the view rects into the planar reflection proxy.
|
|||
|
|
for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex)
|
|||
|
|
{ // Make sure screen percentage has correctly been set on render thread.
|
|||
|
|
check(SceneRenderer->Views[ViewIndex].ViewRect.Area() > 0);
|
|||
|
|
SceneProxy->ViewRect[ViewIndex] = SceneRenderer->Views[ViewIndex].ViewRect;
|
|||
|
|
}
|
|||
|
|
FRDGTextureRef ReflectionOutputTexture = GraphBuilder.RegisterExternalTexture(CreateRenderTarget(RenderTarget->TextureRHI, TEXT("ReflectionOutputTexture")));
|
|||
|
|
GraphBuilder.SetTextureAccessFinal(ReflectionOutputTexture, ERHIAccess::SRVGraphics);
|
|||
|
|
|
|||
|
|
FSceneTextureShaderParameters SceneTextureParameters = CreateSceneTextureShaderParameters(GraphBuilder, &SceneRenderer->GetActiveSceneTextures(), SceneRenderer->FeatureLevel, ESceneTextureSetupMode::SceneDepth);
|
|||
|
|
const FMinimalSceneTextures& SceneTextures = SceneRenderer->GetActiveSceneTextures();
|
|||
|
|
|
|||
|
|
for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex)
|
|||
|
|
{ FViewInfo& View = SceneRenderer->Views[ViewIndex];
|
|||
|
|
RDG_GPU_MASK_SCOPE(GraphBuilder, View.GPUMask);
|
|||
|
|
if (MainSceneRenderer->Scene->GetShadingPath() == EShadingPath::Deferred)
|
|||
|
|
{
|
|||
|
|
PrefilterPlanarReflection<true>(GraphBuilder, View, SceneTextureParameters, SceneProxy, SceneTextures.Color.Resolve, ReflectionOutputTexture);
|
|||
|
|
}else
|
|||
|
|
{
|
|||
|
|
PrefilterPlanarReflection<false>(GraphBuilder, View, SceneTextureParameters, SceneProxy, SceneTextures.Color.Resolve, ReflectionOutputTexture);
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
可行的实现思路：
|
|||
|
|
1. 将`class FPlanarReflectionRenderTarget* RenderTarget`的结果覆盖到对应的`UTextureRenderTarget2D* `中。
|
|||
|
|
2. 使用`UTextureRenderTarget2D`数据来对`FPlanarReflectionRenderTarget`进行初始化。
|
|||
|
|
1. 扩展FPlanarReflectionRenderTarget类，添加新的构造函数，将UTextureRenderTarget2D的数据来扩充内部的RT。进行手动的初始化。
|
|||
|
|
2. 扩展UPlanarReflectionComponent类，增加UTextureRenderTarget2D选项。
|
|||
|
|
|
|||
|
|
但因为RenderTarget是Private，所以不修改源码的情况无法实现。
|
|||
|
|
|
|||
|
|
## 使用SceneCapture2D模拟UPlanarReflectionComponent
|
|||
|
|
位于PlanarReflectionRendering.cpp的UpdatePlanarReflectionContents_RenderThread()
|
|||
|
|
|
|||
|
|
计算视锥与反射平面的代码：
|
|||
|
|
```c++
|
|||
|
|
const FMatrix ComponentTransform = CaptureComponent->GetComponentTransform().ToMatrixWithScale();
|
|||
|
|
FPlane MirrorPlane = FPlane(ComponentTransform.TransformPosition(FVector::ZeroVector), ComponentTransform.TransformVector(FVector(0, 0, 1)));
|
|||
|
|
|
|||
|
|
// Normalize the plane to remove component scaling
|
|||
|
|
bool bNormalized = MirrorPlane.Normalize();
|
|||
|
|
|
|||
|
|
if (!bNormalized)
|
|||
|
|
{
|
|||
|
|
MirrorPlane = FPlane(FVector(0, 0, 1), 0);
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
for (int32 ViewIndex = 0; ViewIndex < MainSceneRenderer.Views.Num() && ViewIndex < GMaxPlanarReflectionViews; ++ViewIndex)
|
|||
|
|
{
|
|||
|
|
const FViewInfo& View = MainSceneRenderer.Views[ViewIndex];
|
|||
|
|
FSceneCaptureViewInfo NewView;
|
|||
|
|
|
|||
|
|
FVector2D ViewRectMin = FVector2D(View.UnscaledViewRect.Min.X, View.UnscaledViewRect.Min.Y);
|
|||
|
|
FVector2D ViewRectMax = FVector2D(View.UnscaledViewRect.Max.X, View.UnscaledViewRect.Max.Y);
|
|||
|
|
ViewRectMin *= FMath::Clamp(CaptureComponent->ScreenPercentage / 100.f, 0.25f, 1.f);
|
|||
|
|
ViewRectMax *= FMath::Clamp(CaptureComponent->ScreenPercentage / 100.f, 0.25f, 1.f);
|
|||
|
|
|
|||
|
|
NewView.ViewRect.Min.X = FMath::TruncToInt(ViewRectMin.X);
|
|||
|
|
NewView.ViewRect.Min.Y = FMath::TruncToInt(ViewRectMin.Y);
|
|||
|
|
NewView.ViewRect.Max.X = FMath::CeilToInt(ViewRectMax.X);
|
|||
|
|
NewView.ViewRect.Max.Y = FMath::CeilToInt(ViewRectMax.Y);
|
|||
|
|
|
|||
|
|
// Create a mirror matrix and premultiply the view transform by it
|
|||
|
|
const FMirrorMatrix MirrorMatrix(MirrorPlane);
|
|||
|
|
const FMatrix ViewMatrix(MirrorMatrix * View.ViewMatrices.GetViewMatrix());
|
|||
|
|
const FVector ViewLocation = ViewMatrix.InverseTransformPosition(FVector::ZeroVector);
|
|||
|
|
const FMatrix ViewRotationMatrix = ViewMatrix.RemoveTranslation();
|
|||
|
|
const float HalfFOV = FMath::Atan(1.0f / View.ViewMatrices.GetProjectionMatrix().M[0][0]);
|
|||
|
|
|
|||
|
|
FMatrix ProjectionMatrix;
|
|||
|
|
BuildProjectionMatrix(View.UnscaledViewRect.Size(), HalfFOV + FMath::DegreesToRadians(CaptureComponent->ExtraFOV), GNearClippingPlane, ProjectionMatrix);
|
|||
|
|
|
|||
|
|
NewView.ViewLocation = ViewLocation;
|
|||
|
|
NewView.ViewRotationMatrix = ViewRotationMatrix;
|
|||
|
|
NewView.ProjectionMatrix = ProjectionMatrix;
|
|||
|
|
NewView.StereoPass = View.StereoPass;
|
|||
|
|
NewView.StereoViewIndex = View.StereoViewIndex;
|
|||
|
|
|
|||
|
|
SceneCaptureViewInfo.Add(NewView);
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```c++
|
|||
|
|
void BuildProjectionMatrix(FIntPoint InRenderTargetSize, float InFOV, float InNearClippingPlane, FMatrix& OutProjectionMatrix)
|
|||
|
|
{
|
|||
|
|
float const XAxisMultiplier = 1.0f;
|
|||
|
|
float const YAxisMultiplier = InRenderTargetSize.X / float(InRenderTargetSize.Y);
|
|||
|
|
|
|||
|
|
if ((int32)ERHIZBuffer::IsInverted)
|
|||
|
|
{ OutProjectionMatrix = FReversedZPerspectiveMatrix(
|
|||
|
|
InFOV,
|
|||
|
|
InFOV,
|
|||
|
|
XAxisMultiplier,
|
|||
|
|
YAxisMultiplier,
|
|||
|
|
InNearClippingPlane,
|
|||
|
|
InNearClippingPlane
|
|||
|
|
);
|
|||
|
|
} else
|
|||
|
|
{
|
|||
|
|
OutProjectionMatrix = FPerspectiveMatrix( InFOV, InFOV, XAxisMultiplier, YAxisMultiplier, InNearClippingPlane, InNearClippingPlane ); }}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
其他计算方式：
|
|||
|
|
```c++
|
|||
|
|
bool IsInFrustum( AActor* Actor)
|
|||
|
|
{
|
|||
|
|
ULocalPlayer* LocalPlayer = GetWorld()->GetFirstLocalPlayerFromController();
|
|||
|
|
if (LocalPlayer != nullptr && LocalPlayer->ViewportClient != nullptr && LocalPlayer->ViewportClient->Viewport)
|
|||
|
|
{
|
|||
|
|
FSceneViewFamilyContext ViewFamily(FSceneViewFamily::ConstructionValues(
|
|||
|
|
LocalPlayer->ViewportClient->Viewport,
|
|||
|
|
GetWorld()->Scene,
|
|||
|
|
LocalPlayer->ViewportClient->EngineShowFlags)
|
|||
|
|
.SetRealtimeUpdate(true));
|
|||
|
|
|
|||
|
|
FVector ViewLocation;
|
|||
|
|
FRotator ViewRotation;
|
|||
|
|
FSceneView* SceneView = LocalPlayer->CalcSceneView(&ViewFamily, ViewLocation, ViewRotation, LocalPlayer->ViewportClient->Viewport);
|
|||
|
|
if (SceneView != nullptr)
|
|||
|
|
{
|
|||
|
|
return SceneView->ViewFrustum.IntersectSphere(
|
|||
|
|
Actor->GetActorLocation(), Actor->GetSimpleCollisionRadius());
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
return false
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### SceneCapture2D
|
|||
|
|
位于SceneCaptureRendering.cpp
|
|||
|
|
|
|||
|
|
|
|||
|
|
# 其他
|
|||
|
|
## Disguise
|
|||
|
|
支持的输入方式：
|
|||
|
|
- sdi传输（采集卡走这个协议）
|
|||
|
|
- ndi网络传输 （通过网卡网线）
|
|||
|
|
- smpte2110协议传输（显卡直接插大屏幕）
|
|||
|
|
|
|||
|
|
## Composure相关问题
|
|||
|
|
- Transforms
|
|||
|
|
- Custom Material Pass
|
|||
|
|
- Post Process Pass Set
|
|||
|
|
- Tonemap
|
|||
|
|
- Multi Pass Chroma Keyer
|
|||
|
|
- Multi Pass Despill
|
|||
|
|
- Outputs
|
|||
|
|
- Media Capture：走采集卡推流模式。
|
|||
|
|
- Image Sequence
|
|||
|
|
- Player Viewport：可以通过在UE中实现虚拟摄像机，再通过OBS推流。
|
|||
|
|
- Render Target Asset
|
|||
|
|
|
|||
|
|
值得注意的是Transforms的Tonemap，这是一个反ToneMapping效果。
|