2023-06-29 11:55:02 +08:00

10 KiB

Raw Blame History

title, date, excerpt, tags, rating

title	date	excerpt	tags	rating
XR拍摄角色没有阴影的解决方案	2022-12-05 14:01:39		XR	⭐

解决方案

在不修改引擎的情况下，解决思路有：

修改引擎的解决思路有： 1.

手动渲染阴影

使用ShadowMapMaterialFunction插件渲染渲染阴影并且贴在一个面片上。

缺点：

如果快速旋转视角，阴影会有残影效果。原因：因为这个阴影贴图不是在同一帧渲染出来的关系。

使用Composure进行合成

使用Composure渲染 Floor（地面模型）、Shadow Layer（地面+角色模型），再通过Shader实现阴影+角色抠像方法。
使用Composure渲染 Floor（地面模型）、Reflection Layer（地面+角色模型），再通过Shader实现反射+角色抠像方法。

经过测试捕捉反射可以使用绿色面片作为背景，最后使用Chroma进行抠像。

注意：使用Composure捕捉的结果都是经过ToneMapping，如果把RT再次贴到场景中进行渲染会有2次Tonemaping只是结果“发白”，所以需要在几个Layer中添加Tonemap选项，进行反ToneMapping。

缺点：

需要使用多次SceneCapture，会严重降低帧数。即使只捕捉单个物体，渲染消耗也和原始的渲染消耗一样。建议：把角色与Composure相关的东西，放在一个子关卡中，场景放在一个子关卡中，然后2台机器使用SVN同步完工程之后，启动对应的关卡进行推流。

UPlanarReflectionComponent

核心函数：

UPlanarReflectionComponent（主要用于维护Scene的PlanarReflections数组以及数据更新标记bRegisteredReflectionCapturesHasChanged）
- CreateRenderState_Concurrent()：创建FPlanarReflectionSceneProxy，并且调用Scene->AddPlanarReflection。
- SendRenderTransform_Concurrent()：调用Scene->UpdatePlanarReflectionTransform。
- DestroyRenderState_Concurrent()：调用Scene->RemovePlanarReflection(this)，并且删除渲染线程的FPlanarReflectionSceneProxy。
- PostEditChangeProperty()：更新可见性。
FScene
- UpdatePlanarReflectionContents()：
  - 清理RT并重新初始化RT
  - 计算反射平面与FSceneCaptureViewInfo（主要是View与Projaction矩阵）
  - 构建FSceneViewFamilyContext，可以理解为Capture的画布与相关View变量
  - 调用FSceneRenderer::CreateSceneRenderer()创建渲染器
  - 在渲染线程调用UpdatePlanarReflectionContents_RenderThread()渲染反射结果

    // Reflection view late update  
   if (SceneRenderer->Views.Num() > 1)  
   {      const FMirrorMatrix MirrorMatrix(MirrorPlane);  
      for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex)  
      {         FViewInfo& ReflectionViewToUpdate = SceneRenderer->Views[ViewIndex];  
         const FViewInfo& UpdatedParentView = MainSceneRenderer->Views[ViewIndex];  
  
         ReflectionViewToUpdate.UpdatePlanarReflectionViewMatrix(UpdatedParentView, MirrorMatrix);  
      }   }  
   // Render the scene normally  
   {  
      RDG_RHI_EVENT_SCOPE(GraphBuilder, RenderScene);  
      SceneRenderer->Render(GraphBuilder);  
   }  
   SceneProxy->RenderTarget = RenderTarget;  
  
   // Update the view rects into the planar reflection proxy.  
   for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex)  
   {      // Make sure screen percentage has correctly been set on render thread.  
      check(SceneRenderer->Views[ViewIndex].ViewRect.Area() > 0);  
      SceneProxy->ViewRect[ViewIndex] = SceneRenderer->Views[ViewIndex].ViewRect;  
   }  
   FRDGTextureRef ReflectionOutputTexture = GraphBuilder.RegisterExternalTexture(CreateRenderTarget(RenderTarget->TextureRHI, TEXT("ReflectionOutputTexture")));  
   GraphBuilder.SetTextureAccessFinal(ReflectionOutputTexture, ERHIAccess::SRVGraphics);  
  
   FSceneTextureShaderParameters SceneTextureParameters = CreateSceneTextureShaderParameters(GraphBuilder, &SceneRenderer->GetActiveSceneTextures(), SceneRenderer->FeatureLevel, ESceneTextureSetupMode::SceneDepth);  
   const FMinimalSceneTextures& SceneTextures = SceneRenderer->GetActiveSceneTextures();  
  
   for (int32 ViewIndex = 0; ViewIndex < SceneRenderer->Views.Num(); ++ViewIndex)  
   {      FViewInfo& View = SceneRenderer->Views[ViewIndex];  
      RDG_GPU_MASK_SCOPE(GraphBuilder, View.GPUMask);  
      if (MainSceneRenderer->Scene->GetShadingPath() == EShadingPath::Deferred)  
      {         
	      PrefilterPlanarReflection<true>(GraphBuilder, View, SceneTextureParameters, SceneProxy, SceneTextures.Color.Resolve, ReflectionOutputTexture);  
      }else  
      {  
         PrefilterPlanarReflection<false>(GraphBuilder, View, SceneTextureParameters, SceneProxy, SceneTextures.Color.Resolve, ReflectionOutputTexture);  
      }   
	}

可行的实现思路：

将class FPlanarReflectionRenderTarget* RenderTarget的结果覆盖到对应的UTextureRenderTarget2D* 中。
使用UTextureRenderTarget2D数据来对FPlanarReflectionRenderTarget进行初始化。
1. 扩展FPlanarReflectionRenderTarget类，添加新的构造函数，将UTextureRenderTarget2D的数据来扩充内部的RT。进行手动的初始化。
2. 扩展UPlanarReflectionComponent类，增加UTextureRenderTarget2D选项。

但因为RenderTarget是Private，所以不修改源码的情况无法实现。

使用SceneCapture2D模拟UPlanarReflectionComponent

位于PlanarReflectionRendering.cpp的UpdatePlanarReflectionContents_RenderThread()

计算视锥与反射平面的代码：

const FMatrix ComponentTransform = CaptureComponent->GetComponentTransform().ToMatrixWithScale();  
FPlane MirrorPlane = FPlane(ComponentTransform.TransformPosition(FVector::ZeroVector), ComponentTransform.TransformVector(FVector(0, 0, 1)));

// Normalize the plane to remove component scaling  
bool bNormalized = MirrorPlane.Normalize();  

if (!bNormalized)  
{  
   MirrorPlane = FPlane(FVector(0, 0, 1), 0);  
}

for (int32 ViewIndex = 0; ViewIndex < MainSceneRenderer.Views.Num() && ViewIndex < GMaxPlanarReflectionViews; ++ViewIndex)  
{  
   const FViewInfo& View = MainSceneRenderer.Views[ViewIndex];  
   FSceneCaptureViewInfo NewView;  
  
   FVector2D ViewRectMin = FVector2D(View.UnscaledViewRect.Min.X, View.UnscaledViewRect.Min.Y);  
   FVector2D ViewRectMax = FVector2D(View.UnscaledViewRect.Max.X, View.UnscaledViewRect.Max.Y);  
   ViewRectMin *= FMath::Clamp(CaptureComponent->ScreenPercentage / 100.f, 0.25f, 1.f);  
   ViewRectMax *= FMath::Clamp(CaptureComponent->ScreenPercentage / 100.f, 0.25f, 1.f);  
  
   NewView.ViewRect.Min.X = FMath::TruncToInt(ViewRectMin.X);  
   NewView.ViewRect.Min.Y = FMath::TruncToInt(ViewRectMin.Y);  
   NewView.ViewRect.Max.X = FMath::CeilToInt(ViewRectMax.X);  
   NewView.ViewRect.Max.Y = FMath::CeilToInt(ViewRectMax.Y);  
  
   // Create a mirror matrix and premultiply the view transform by it  
   const FMirrorMatrix MirrorMatrix(MirrorPlane);  
   const FMatrix ViewMatrix(MirrorMatrix * View.ViewMatrices.GetViewMatrix());  
   const FVector ViewLocation = ViewMatrix.InverseTransformPosition(FVector::ZeroVector);  
   const FMatrix ViewRotationMatrix = ViewMatrix.RemoveTranslation();  
   const float HalfFOV = FMath::Atan(1.0f / View.ViewMatrices.GetProjectionMatrix().M[0][0]);  
  
   FMatrix ProjectionMatrix;  
   BuildProjectionMatrix(View.UnscaledViewRect.Size(), HalfFOV + FMath::DegreesToRadians(CaptureComponent->ExtraFOV), GNearClippingPlane, ProjectionMatrix);  
  
   NewView.ViewLocation = ViewLocation;  
   NewView.ViewRotationMatrix = ViewRotationMatrix;  
   NewView.ProjectionMatrix = ProjectionMatrix;  
   NewView.StereoPass = View.StereoPass;  
   NewView.StereoViewIndex = View.StereoViewIndex;  
  
   SceneCaptureViewInfo.Add(NewView);  
}

void BuildProjectionMatrix(FIntPoint InRenderTargetSize, float InFOV, float InNearClippingPlane, FMatrix& OutProjectionMatrix)  
{  
   float const XAxisMultiplier = 1.0f;  
   float const YAxisMultiplier = InRenderTargetSize.X / float(InRenderTargetSize.Y);  
  
   if ((int32)ERHIZBuffer::IsInverted)  
   {      OutProjectionMatrix = FReversedZPerspectiveMatrix(  
         InFOV,  
         InFOV,  
         XAxisMultiplier,  
         YAxisMultiplier,  
         InNearClippingPlane,  
         InNearClippingPlane  
         );  
   }   else  
   {  
      OutProjectionMatrix = FPerspectiveMatrix(         InFOV,         InFOV,         XAxisMultiplier,         YAxisMultiplier,         InNearClippingPlane,         InNearClippingPlane         );   }}

其他计算方式：

bool IsInFrustum( AActor* Actor)
{
	ULocalPlayer* LocalPlayer = GetWorld()->GetFirstLocalPlayerFromController();
	if (LocalPlayer != nullptr && LocalPlayer->ViewportClient != nullptr && LocalPlayer->ViewportClient->Viewport)
	{
		FSceneViewFamilyContext ViewFamily(FSceneViewFamily::ConstructionValues(
			LocalPlayer->ViewportClient->Viewport,
			GetWorld()->Scene,
			LocalPlayer->ViewportClient->EngineShowFlags)
			.SetRealtimeUpdate(true));

		FVector ViewLocation;
		FRotator ViewRotation;
		FSceneView* SceneView = LocalPlayer->CalcSceneView(&ViewFamily, ViewLocation, ViewRotation, LocalPlayer->ViewportClient->Viewport);
		if (SceneView != nullptr)
		{
			return SceneView->ViewFrustum.IntersectSphere(
						Actor->GetActorLocation(), Actor->GetSimpleCollisionRadius());
		}			 
	}

    return false
}

SceneCapture2D

位于SceneCaptureRendering.cpp

其他

Disguise

支持的输入方式：

sdi传输（采集卡走这个协议）
ndi网络传输（通过网卡网线）
smpte2110协议传输（显卡直接插大屏幕）

Composure相关问题

Transforms
- Custom Material Pass
- Post Process Pass Set
- Tonemap
- Multi Pass Chroma Keyer
- Multi Pass Despill
Outputs
- Media Capture：走采集卡推流模式。
- Image Sequence
- Player Viewport：可以通过在UE中实现虚拟摄像机，再通过OBS推流。
- Render Target Asset

值得注意的是Transforms的Tonemap，这是一个反ToneMapping效果。

10 KiB Raw Blame History Unescape Escape