BlueRose/BlueRoseNote
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
BlueRoseNote/02-Note/DAWA/ASoul/动画相关/动捕逻辑.md

9.4 KiB
Raw Blame History

相关类

  • TsArkitDataReceiver(ArkitDataReceiver)
  • TsChingmuMocapReceiverActor(ChingmuMocapReceiverActor)
  • TsMotionReceiverActor(MotionReceiverActor) => BP_MotionReceiver：定义了MotionNetConfig.ini。
  • TsMotionSenderActor(MotionSenderActor)

TsChingmuMocapReceiverActor

地图里只会有一个生成的TsChingmuMocapReceiverActor来管理动捕数据接收

  1. Init()：在Server才会Spawn TsChingmuMocapReceiverActor。
  2. ConnectChingMu()：ChingmuComp.StartConnectServer()
  3. Multicast_AligmMotionTime()：寻找场景中的BP_MotionReceiver，并且调用Receiver.AlignTimeStamp()。

ChingmuMocapReceiverActor

核心逻辑：

  • FChingmuThread::Run()
  • AChingmuMocapReceiverActor::Tick()
  • AChingmuMocapReceiverActor::DoSample()
void AChingmuMocapReceiverActor::BeginPlay()
{
	Super::BeginPlay();
	MaxHumanCount = 10;
	MaxRigidBodyCount = 10;
	CacheLimit = 240;
	SampledHumanData = NewObject<UMocapFrameData>();
	ThreadInterval = 0.002;
	BackIndexCount = int64(UMotionUtils::BackSampleTime / (1000.0 / CHINGMU_SERVER_FPS));//BackSampleTime = 100ms  CHINGMU_SERVER_FPS =120ms
	ChingmuComp = Cast<UChingMUComponent>(GetComponentByClass(UChingMUComponent::StaticClass()));
	if (ChingmuComp == nullptr)
	{
		UE_LOG(LogTemp, Error, TEXT("Chingmu Component is missing!!"));
	}
	Thread = new FChingmuThread("Chingmu Data Thread", this);
	Sender = GetMotionSender();
}

void AChingmuMocapReceiverActor::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);

	if(!Sender)
	{
		Sender = GetMotionSender();
	}
	const auto CurTime = ULiveDirectorStatics::GetUnixTime();
	if(UseThread)
	{
		// 线程方式
		// 在数据队列中获取青瞳数据
		while (!FrameQueue.IsEmpty())
		{
			ST_MocapFrameData* Frame;
			if (FrameQueue.Dequeue(Frame))
			{
				PutMocapDataIntoFrameList(Frame);
			}
		}
	}

	DoSample(AllHumanFrames);
	DoSample(AllRigidBodyFrames);

	// 每隔1s计算一次平均包间隔
	if (CurTime - LastCheckIntervalTime > 1000)
	{
		if (AllHumanFrames.Num() > 0)
		{
			AllHumanFrames[0]->CalculatePackageAverageInterval(this->PackageAverageInterval);
			LastCheckIntervalTime = CurTime;
		}
	}
}

void AChingmuMocapReceiverActor::DoSample(TArray<MocapFrames*>& Frames)
{
	for (auto i = 0; i < Frames.Num(); i++)
	{
		Frames[i]->CheckSize(CacheLimit);
		if (SampleByTimeStamp(Frames[i]->Frames))
		{
			SendFrameToCharacter();
		}
	}
}

FChingmuThread

用途为：

  • 获取当前系统时间。
  • 使用异步Task的方式，通过调用UChingMUComponent::FullBodyMotionCapBaseBonesLocalSpaceRotation() 来更新每个演员的动捕数据。动捕数据存储在ChingMUComponent中的LocalRotationList、GlobalLocationList中。
  • 管理HumanToLastReceiveTime，以此管理每个动捕演员的动画数据时长。
  • OwnerActor->OnGetHumanData_NotInGameThread()，
    • 根据当前时间与当前Frames，从UChingMUComponent中将数据复制到#ST_MocapFrameData中。
    • 将#ST_MocapFrameData转换成JSON后，使用AMotionSenderActor::OnGetRawMocapData_NotInGameThread()发送。
    • 将当前帧数据加入FrameQueue队列。
  • 线程睡眠0.001s。
uint32 FChingmuThread::Run()
{
	FTransform Tmp;
	while (bRun)
	{
		if (OwnerActor && OwnerActor->UseThread && OwnerActor->ChingmuComp && OwnerActor->ChingmuComp->IsConnected())
		{
			CurTime = ULiveDirectorStatics::GetUnixTime();
			// Human
			for (auto HumanIndex = 0; HumanIndex < OwnerActor->MaxHumanCount; HumanIndex++)
			{
				const auto bRes = OwnerActor->ChingmuComp->FullBodyMotionCapBaseBonesLocalSpaceRotation(
					OwnerActor->ChingmuFullAddress, HumanIndex, TmpTimeCode);
				if (bRes)
				{
					if (!HumanToLastReceiveTime.Contains(HumanIndex))//空数据处理。
					{
						HumanToLastReceiveTime.Add(HumanIndex, 0);
					}
					if (HumanToLastReceiveTime[HumanIndex] != TmpTimeCode.Frames)//判断是否收到新的Frame数据
					{
						HumanToLastReceiveTime[HumanIndex] = TmpTimeCode.Frames;
						OwnerActor->OnGetHumanData_NotInGameThread(HumanIndex, CurTime, TmpTimeCode.Frames);
					}
					else
					{
						// get same frame, skip
						break;
					}
				}
			}

			// Rigidbody

			for (auto RigidBodyIndex = OwnerActor->RigidBodyStartIndex; RigidBodyIndex < OwnerActor->RigidBodyStartIndex
			     + OwnerActor->MaxRigidBodyCount; RigidBodyIndex++)
			{
				OwnerActor->ChingmuComp->GetTrackerPoseTC(OwnerActor->ChingmuFullAddress, RigidBodyIndex, Tmp,
				                                          TmpTimeCode);

				if (!RigidBodyToLastReceiveTransform.Contains(RigidBodyIndex))
				{
					RigidBodyToLastReceiveTransform.Add(RigidBodyIndex, FTransform::Identity);
				}
				// 道具的TmpTimeCode.Frames永远为0，所以无法用帧数判断
				// 改为transform判断
				if (!RigidBodyToLastReceiveTransform[RigidBodyIndex].Equals(Tmp))
				{
					RigidBodyToLastReceiveTransform[RigidBodyIndex] = Tmp;
					OwnerActor->OnGetRigidBodyData_NotInGameThread(RigidBodyIndex, Tmp, CurTime, TmpTimeCode.Frames);
				}
			}
		}
		if (bRun)
		{
			FPlatformProcess::Sleep(OwnerActor ? OwnerActor->ThreadInterval : 0.004);
		}
		else
		{
			break;
		}
	}
	UE_LOG(LogTemp, Warning, TEXT("%s finish work."), *ThreadName)
	return 0;
}

ST_MocapFrameData

#define MOCAP_BONE_COUNT 23  
  
enum E_MotionType  
{  
    Human,  
    RigidBody  
};  
  
enum E_SourceType  
{  
    Mocap,  
    CMR,  
    Replay  
};

struct ST_MocapFrameData  
{  
    int ID;  
    int64 TimeStamp;  
    int FrameIndex;  
    E_MotionType MotionType;  
    E_SourceType SourceType;  
    FVector BonesWorldPos[MOCAP_BONE_COUNT];  
    FQuat BonesLocalRot[MOCAP_BONE_COUNT];
};

class LIVEDIRECTOR_API UMocapFrameData : public UObject
{
	GENERATED_BODY()

public:
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	int ID;
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	TArray<FVector> BonesWorldPos;
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	TArray<FQuat> BonesLocalRot;
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	int64 TimeStamp;
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	int FrameIndex;
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	int MotionType; // 0 human; 1 rigidbody
	UPROPERTY(BlueprintReadWrite, EditAnywhere)
	int SourceType; // 0 mocap, 1 cmr
public:
	void CopyFrom(const ST_MocapFrameData* Other)
	{
		ID = Other->ID;
		TimeStamp = Other->TimeStamp;
		FrameIndex = Other->FrameIndex;
		MotionType = Other->MotionType;
		SourceType = Other->SourceType;
		for (auto Index = 0; Index < 23; Index++)
		{
			BonesWorldPos[Index] = Other->BonesWorldPos[Index];
			BonesLocalRot[Index] = Other->BonesLocalRot[Index];
		}
	}
};

class MocapFrames
{
public:
	int ID;
	std::vector<ST_MocapFrameData*> Frames = {};
	
	void CalculatePackageAverageInterval(float& Res)
	{
		if(Frames.size() > 0)
		{
			auto First = Frames[0];
			auto Last = Frames[Frames.size() - 1];
			if(Last->FrameIndex > First->FrameIndex)
			{
				Res = 1.0 * (Last->TimeStamp - First->TimeStamp) / (Last->FrameIndex - First->FrameIndex);
			}
		}
	}
};

MotionCapture(青瞳插件)

实现1个组件与3个动画节点：

ChingMUComponent

  1. Init
    1. BeginPlay()：取得ini文件中的配置信息；取得当前角色的SkeletonMesh => CharacterSkinMesh;取得BoneName=>BoneIndex Map、TPose状态下骨骼的旋转值、TposeParentBonesRotation。
  2. Connect
    1. StartConnectServer()：motionCapturePlugin->ConnectCommand = "ConnectServer"。具体逻辑会在FMotionCapture::Tick()处理。
    2. DisConnectServer()：motionCapturePlugin->ConnectCommand = "DisConnect"。
  3. #CalculateBoneCSRotation()
  4. #FullBodyMotionCapBaseBonesLocalSpaceRotation

CalculateBoneCSRotation

Get Human Fullbody Tracker data ,including of 23joints localRotation and root joint world Position

  1. m_motioncap->CMHuman()：调用DLL的CMHumanExtern()，获取一个Double数组，前3个是RootLocation，后面全是Rotation。
  2. 计算最终的四元数旋转值。
  3. 返回的形参 FQuat* BonesComponentSpaceRotation，数组指针。

FullBodyMotionCapBaseBonesLocalSpaceRotation

相比CalculateBoneCSRotation，增加了时间码以及GlobalLocation的动捕数据获取。

  1. m_motioncap->CMHuman()：调用DLL的CMHumanExtern()，获取一个Double数组，前3个是RootLocation，后面全是Rotation。
  2. motionCapturePlugin->CMHumanGlobalRTTC()：调用DLL的CMHumanGlobalRTTC()，1-24 New Features。计算VrpnTimeCode以及GlobalLocationList。

数据存在ChingMUComponent中的LocalRotationList、GlobalLocationList。

FAnimNode_ChingMUPose

  1. Initialize_AnyThread()：取得ChingMUComponent。
  2. Update_AnyThread()：调用ChingMUComponent->CalculateBoneCSRotation()
  3. Evaluate_AnyThread()：对23根骨骼进行遍历；取得RefPose后，将从Update_AnyThread()获得动捕数据（Rotation）覆盖到上面（ComponentSpace），根骨骼需要额外添加Location数据。最后将数据从ComponentSpace => LocalSpace。

AnimNode_ChingMURetargetPose

  1. Initialize_AnyThread()：创建曲线逻辑（TCHour、TCMinute、TCSecond、TCFrame）。
  2. Update_AnyThread()：
  3. Evaluate_AnyThread()：相关逻辑都实现在这里。

AnimNode_ChingMURetargetPose::Evaluate_AnyThread()

TsMotionReceiverActor

只在BeginPlay()中调用了this.MarkAsClientSeamlessTravel(); 具体逻辑在AMotionReceiverActor

MotionReceiverActor