BlueRoseNote/03-UnrealEngine/Sequence/Sequence Runtime Binding.md

---
title: Sequence Runtime Binding
date: 2024-05-16 17:55:58
excerpt: 
tags: 
rating: ⭐
---
# 前言
***MovieSceneTextTrack*** 引擎插件可以作为参考。
***UMovieSceneCVarTrackInstance***

参考：
1. [sequencer mute tracks at runtime from c](https://forums.unrealengine.com/t/sequencer-mute-tracks-at-runtime-from-c/476278/3)
2. 其他参考
	1. [Actor Rebinding in Blueprints with Sequencer(官方文档)]https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/Sequencer/HowTo/AnimateDynamicObjects/
	2. [Level Sequence actor rebinding in C++](https://www.reddit.com/r/unrealengine/comments/btk1uz/level_sequence_actor_rebinding_in_c/)
3. Sequence ECS资料
	1. [大规模内容的性能保障：虚幻引擎4.26中的Sequencer(官方文档)](https://www.unrealengine.com/zh-CN/tech-blog/performance-at-scale-sequencer-in-unreal-engine-4-26)
	2. [UE4.26(5.0)后的Sequence系统](https://zhuanlan.zhihu.com/p/589465561)
	3. [UE5 Sequence 浅-浅-析](https://zhuanlan.zhihu.com/p/676690043)
4. ECS资料 
	1. [一文看懂ECS架构]https://zhuanlan.zhihu.com/p/618971664

可行思路：
1. 给导播创建一个快捷键（Hotkey），之后在触发根据Tag切换对应角色动画蓝图中的逻辑。
2. [x] 使用Sequence的EventTrack设置若干事件。
3. 模改UMovieSceneSkeletalAnimationTrack，实现一个可以自动根据Tag捕获对应角色并且播发动画的功能。

# 自定义Track
- Runtime修改绑定
	- UE4：扩展Sequencer的BakeTransform有关Camera的参数:https://zhuanlan.zhihu.com/p/554484287
	- **LevelSequence对象绑定分析**:https://zhuanlan.zhihu.com/p/548155341
- 自定义Track
	- https://zhuanlan.zhihu.com/p/396353973
	- https://zhuanlan.zhihu.com/p/414179358
	- https://zhuanlan.zhihu.com/p/413151867
- LevelSequence分析
	- https://zhuanlan.zhihu.com/p/157892605

## UMovieSceneTrackInstance
大部分Track的instance类。

## 继承关系
UMovieSceneSignedObject -> UObject。一个UMovieScene由若干个MasterTrack(UMovieSceneTrack)组成，UMovieScene相当于是UMovieSceneTracks的容器，那么UMovieSceneTrack又由UMovieSceneSections组成，UMovieSceneSection就是一个Track中间的某一段。

### UMovieSceneSignedObject
子类有UMovieScene、UMovieSceneSequence、UMovieSceneTrack、UMovieSceneSection。
定义： 
- FGuid Signature：用于确定Object身份，以便于绑定。
- FOnSignatureChanged OnSignatureChangedEvent：ChangedEvent。

### UMovieSceneSection
#### UMovieSceneSkeletalAnimationSection


### UMovieSceneTrack
#### UMovieSceneSkeletalAnimationTrack
- `TArray<TObjectPtr<UMovieSceneSection>> AnimationSections`：存储所有动画Section。
- FMovieSceneSkeletalAnimRootMotionTrackParams RootMotionParams：RootMotion控制。

编辑器代码：**FSkeletalAnimationTrackEditor**

## Sequence绑定机制笔记
1. _SpawnableObject_(自动Spawn的Object)  
    表示该Object在Sequence被Evaluate的时候自动Spawn，并且由Sequence来管理生命周期
2. _PossessableObject_(可以被赋予的Object)  
    表示该Object可以被外部赋予，比如我手动赋予一个Actor给某个Track
### UMovieSceneSequence
UMovieSceneSequence::CreatePossessable()
UMovieSceneSequence::BindPossessableObject()

绑定过程可以参考：
ULevelSequenceEditorSubsystem::AddActorsToBinding(const TArray<AActor*>& Actors, const FMovieSceneBindingProxy& ObjectBinding)


### ActorToSequencer
菜单出现逻辑位于：FLevelSequenceEditorActorBinding::BuildSequencerAddMenu(FMenuBuilder& MenuBuilder)。

`FLevelSequenceEditorActorBinding::AddPossessActorMenuExtensions(FMenuBuilder& MenuBuilder)`
=>
`FLevelSequenceEditorActorBinding::AddActorsToSequencer(AActor*const* InActors, int32 NumActors)`
=>
`TArray<FGuid> FSequencer::AddActors(const TArray<TWeakObjectPtr<AActor> >& InActors, bool bSelectActors)；`
=>
`TArray<FGuid> FSequencerUtilities::AddActors(TSharedRef<ISequencer> Sequencer, const TArray<TWeakObjectPtr<AActor> >& InActors)`

```c++
TArray<FGuid> FSequencerUtilities::AddActors(TSharedRef<ISequencer> Sequencer, const TArray<TWeakObjectPtr<AActor> >& InActors)
{
	TArray<FGuid> PossessableGuids;

	UMovieSceneSequence* Sequence = Sequencer->GetFocusedMovieSceneSequence();
	if (!Sequence)
	{
		return PossessableGuids;
	}

	UMovieScene* MovieScene = Sequence->GetMovieScene();
	if (!MovieScene)
	{
		return PossessableGuids;
	}

	if (MovieScene->IsReadOnly())
	{
		ShowReadOnlyError();
		return PossessableGuids;
	}
	
	const FScopedTransaction Transaction(LOCTEXT("AddActors", "Add Actors"));
	Sequence->Modify();

	for (TWeakObjectPtr<AActor> WeakActor : InActors)
	{
		if (AActor* Actor = WeakActor.Get())
		{
			FGuid ExistingGuid = Sequencer->FindObjectId(*Actor, Sequencer->GetFocusedTemplateID());
			if (!ExistingGuid.IsValid())
			{
				FGuid PossessableGuid = CreateBinding(Sequencer, *Actor, Actor->GetActorLabel());
				PossessableGuids.Add(PossessableGuid);

				if (ACameraActor* CameraActor = Cast<ACameraActor>(Actor))
				{
					NewCameraAdded(Sequencer, CameraActor, PossessableGuid);
				}

				Sequencer->OnActorAddedToSequencer().Broadcast(Actor, PossessableGuid);
			}
		}
	}

	return PossessableGuids;
}
```

其他参考
```c++
const FGuid PossessableGuid = OwnerMovieScene->AddPossessable(InName, InObject.GetClass());

if (!OwnerMovieScene->FindPossessable(PossessableGuid)->BindSpawnableObject(Sequencer->GetFocusedTemplateID(), &InObject, &Sequencer.Get()))
{
	OwnerSequence->BindPossessableObject(PossessableGuid, InObject, BindingContext);
}
```

```c++
FGuid FSequencerUtilities::CreateBinding(TSharedRef<ISequencer> Sequencer, UObject& InObject, const FString& InName)
{
	const FScopedTransaction Transaction(LOCTEXT("CreateBinding", "Create New Binding"));

	UMovieSceneSequence* OwnerSequence = Sequencer->GetFocusedMovieSceneSequence();
	UMovieScene* OwnerMovieScene = OwnerSequence->GetMovieScene();

	OwnerSequence->Modify();
	OwnerMovieScene->Modify();

	const FGuid PossessableGuid = OwnerMovieScene->AddPossessable(InName, InObject.GetClass());

	// Attempt to use the parent as a context if necessary
	UObject* ParentObject = OwnerSequence->GetParentObject(&InObject);
	UObject* BindingContext = Sequencer->GetPlaybackContext();

	AActor* ParentActorAdded = nullptr;
	FGuid ParentGuid;

	if (ParentObject)
	{
		// Ensure we have possessed the outer object, if necessary
		ParentGuid = Sequencer->GetHandleToObject(ParentObject, false);
		if (!ParentGuid.IsValid())
		{
			ParentGuid = Sequencer->GetHandleToObject(ParentObject);
			ParentActorAdded = Cast<AActor>(ParentObject);
		}

		if (OwnerSequence->AreParentContextsSignificant())
		{
			BindingContext = ParentObject;
		}

		// Set up parent/child guids for possessables within spawnables
		if (ParentGuid.IsValid())
		{
			FMovieScenePossessable* ChildPossessable = OwnerMovieScene->FindPossessable(PossessableGuid);
			if (ensure(ChildPossessable))
			{
				ChildPossessable->SetParent(ParentGuid, OwnerMovieScene);
			}

			FMovieSceneSpawnable* ParentSpawnable = OwnerMovieScene->FindSpawnable(ParentGuid);
			if (ParentSpawnable)
			{
				ParentSpawnable->AddChildPossessable(PossessableGuid);
			}
		}
	}

	if (!OwnerMovieScene->FindPossessable(PossessableGuid)->BindSpawnableObject(Sequencer->GetFocusedTemplateID(), &InObject, &Sequencer.Get()))
	{
		OwnerSequence->BindPossessableObject(PossessableGuid, InObject, BindingContext);
	}

	// Broadcast if a parent actor was added as a result of adding this object
	if (ParentActorAdded && ParentGuid.IsValid())
	{
		Sequencer->OnActorAddedToSequencer().Broadcast(ParentActorAdded, ParentGuid);
	}

	return PossessableGuid;
}
```

### FAudioTrackEditor
FAudioTrackEditor::HandleAssetAdded(UObject* Asset, const FGuid& TargetObjectGuid) => FAudioTrackEditor::AddNewSound() => UMovieSceneAudioTrack::AddNewSoundOnRow(USoundBase* Sound, FFrameNumber Time, int32 RowIndex)

## MovieSceneSkeletalAnimationTrack
- UMovieSceneSkeletalAnimationTrack
- UMovieSceneSkeletalAnimationSection
	- FMovieSceneSkeletalAnimationParams
		- `TObjectPtr<UAnimSequenceBase> Animation`
- FSkeletalAnimationTrackEditor
- UMovieSceneSkeletalAnimationSystem
	- 里面有介绍System相关的逻辑 https://zhuanlan.zhihu.com/p/413151867
- FSkeletalAnimationTrackEditMode

Track的编辑器注册位于MovieSceneToolsModule.cpp：
```c++
AnimationTrackCreateEditorHandle = SequencerModule.RegisterTrackEditor( FOnCreateTrackEditor::CreateStatic( &FSkeletalAnimationTrackEditor::CreateTrackEditor ) );
```

只对UMovieSceneSkeletalAnimationTrack生效：
```c++
bool FSkeletalAnimationTrackEditor::SupportsSequence(UMovieSceneSequence* InSequence) const  
{  
    ETrackSupport TrackSupported = InSequence ? InSequence->IsTrackSupported(UMovieSceneSkeletalAnimationTrack::StaticClass()) : ETrackSupport::NotSupported;  
    return TrackSupported == ETrackSupport::Supported;  
}

FEditorModeRegistry::Get().RegisterMode<FSkeletalAnimationTrackEditMode>(  
    FSkeletalAnimationTrackEditMode::ModeName,  
    NSLOCTEXT("SkeletalAnimationTrackEditorMode", "SkelAnimTrackEditMode", "Skeletal Anim Track Mode"),  
    FSlateIcon(),  
    false);
```

### FSkeletalAnimationTrackEditor
FSkeletalAnimationTrackEditor::BuildTrackContextMenu()：在MovieSceneSkeletalAnimationTrack的Animation上右键出现的菜单。
FSkeletalAnimationTrackEditor::BuildOutlinerEditWidget：MovieSceneSkeletalAnimationTrack下面生成Animtion那一列。

##  播放逻辑
```c++
ALevelSequenceActor::InitializePlayer() 
ALevelSequenceActor->SequencePlayer->Play(); 
ALevelSequenceActor->SequencePlayer->Update(DeltaSeconds);
```

具体的逻辑位于UMovieSceneSequencePlayer::PlayInternal()：
```c++

void UMovieSceneSequencePlayer::PlayInternal()
{
	if (NeedsQueueLatentAction())
	{
		QueueLatentAction(FMovieSceneSequenceLatentActionDelegate::CreateUObject(this, &UMovieSceneSequencePlayer::PlayInternal));
		return;
	}

	if (!IsPlaying() && Sequence && CanPlay())
	{
		const FString SequenceName = GetSequenceName(true);
		UE_LOG(LogMovieScene, Verbose, TEXT("PlayInternal - %s (current status: %s)"), *SequenceName, *UEnum::GetValueAsString(Status));

		// Set playback status to playing before any calls to update the position
		Status = EMovieScenePlayerStatus::Playing;

		float PlayRate = bReversePlayback ? -PlaybackSettings.PlayRate : PlaybackSettings.PlayRate;

		// If at the end and playing forwards, rewind to beginning
		if (GetCurrentTime().Time == GetLastValidTime())
		{
			if (PlayRate > 0.f)
			{
				SetPlaybackPosition(FMovieSceneSequencePlaybackParams(FFrameTime(StartTime), EUpdatePositionMethod::Jump));
			}
		}
		else if (GetCurrentTime().Time == FFrameTime(StartTime))
		{
			if (PlayRate < 0.f)
			{
				SetPlaybackPosition(FMovieSceneSequencePlaybackParams(GetLastValidTime(), EUpdatePositionMethod::Jump));
			}
		}

		// Update now
		if (PlaybackSettings.bRestoreState)
		{
			RootTemplateInstance.EnableGlobalPreAnimatedStateCapture();
		}

		bPendingOnStartedPlaying = true;
		Status = EMovieScenePlayerStatus::Playing;
		TimeController->StartPlaying(GetCurrentTime());
		
		if (PlayPosition.GetEvaluationType() == EMovieSceneEvaluationType::FrameLocked)
		{
			if (!OldMaxTickRate.IsSet())
			{
				OldMaxTickRate = GEngine->GetMaxFPS();
			}

			GEngine->SetMaxFPS(1.f / PlayPosition.GetInputRate().AsInterval());
		}

		//播放的核心逻辑估计在这里。
		if (!PlayPosition.GetLastPlayEvalPostition().IsSet() || PlayPosition.GetLastPlayEvalPostition() != PlayPosition.GetCurrentPosition())
		{
			UpdateMovieSceneInstance(PlayPosition.PlayTo(PlayPosition.GetCurrentPosition()), EMovieScenePlayerStatus::Playing);
		}

		RunLatentActions();
		UpdateNetworkSyncProperties();
		
		if (bReversePlayback)
		{
			if (OnPlayReverse.IsBound())
			{
				OnPlayReverse.Broadcast();
			}
		}
		else
		{
			if (OnPlay.IsBound())
			{
				OnPlay.Broadcast();
			}
		}
	}
}

void UMovieSceneSequencePlayer::UpdateMovieSceneInstance(FMovieSceneEvaluationRange InRange, EMovieScenePlayerStatus::Type PlayerStatus, bool bHasJumped)
{
	FMovieSceneUpdateArgs Args;
	Args.bHasJumped = bHasJumped;
	UpdateMovieSceneInstance(InRange, PlayerStatus, Args);
}

void UMovieSceneSequencePlayer::UpdateMovieSceneInstance(FMovieSceneEvaluationRange InRange, EMovieScenePlayerStatus::Type PlayerStatus, const FMovieSceneUpdateArgs& Args)
{
	if (Observer && !Observer->CanObserveSequence())
	{
		return;
	}

	UMovieSceneSequence* MovieSceneSequence = RootTemplateInstance.GetSequence(MovieSceneSequenceID::Root);
	if (!MovieSceneSequence)
	{
		return;
	}

#if !NO_LOGGING
	if (UE_LOG_ACTIVE(LogMovieScene, VeryVerbose))
	{
		const FQualifiedFrameTime CurrentTime = GetCurrentTime();
		const FString SequenceName = GetSequenceName(true);
		UE_LOG(LogMovieScene, VeryVerbose, TEXT("Evaluating sequence %s at frame %d, subframe %f (%f fps)."), *SequenceName, CurrentTime.Time.FrameNumber.Value, CurrentTime.Time.GetSubFrame(), CurrentTime.Rate.AsDecimal());
	}
#endif

	if (PlaybackClient)
	{
		PlaybackClient->WarpEvaluationRange(InRange);
	}

	// Once we have updated we must no longer skip updates
	bSkipNextUpdate = false;

	// We shouldn't be asked to run an async update if we have a blocking sequence.
	check(!Args.bIsAsync || !EnumHasAnyFlags(MovieSceneSequence->GetFlags(), EMovieSceneSequenceFlags::BlockingEvaluation));
	// We shouldn't be asked to run an async update if we don't have a tick manager.
	check(!Args.bIsAsync || TickManager != nullptr);

	FMovieSceneContext Context(InRange, PlayerStatus);
	Context.SetHasJumped(Args.bHasJumped);

	TSharedPtr<FMovieSceneEntitySystemRunner> Runner = RootTemplateInstance.GetRunner();
	if (Runner)
	{
		Runner->QueueUpdate(Context, RootTemplateInstance.GetRootInstanceHandle());
		if (Runner == SynchronousRunner || !Args.bIsAsync)
		{
			Runner->Flush();
		}
	}
}
```

这段解析来自于：https://zhuanlan.zhihu.com/p/413151867
```c++
UMovieSceneSequencePlayer::Play()           //蓝图中调用
UMovieSceneSequencePlayer::PlayInternal()
UMovieSceneSequencePlayer::UpdateMovieSceneInstance
RootTemplateInstance.Evaluate(Context, *this);

// FMovieSceneRootEvaluationTemplateInstance::Evaluate
EntitySystemRunner.Update(Context, RootInstanceHandle);

FMovieSceneEntitySystemRunner::Flush()
FMovieSceneEntitySystemRunner::DoFlushUpdateQueueOnce()
FMovieSceneEntitySystemRunner::GameThread_ProcessQueue()
FMovieSceneEntitySystemRunner::GameThread_SpawnPhase()

// FMovieSceneEntitySystemRunner::GameThread_SpawnPhase()
/// System Linker->GetInstanceRegistry();
FInstanceRegistry* InstanceRegistry = GetInstanceRegistry(); 
FSequenceInstance& Instance = InstanceRegistry->MutateInstance(Update.InstanceHandle);
Instance.Update(Linker, Update.Context);

// FSequenceInstance::Update
SequenceUpdater->Update(Linker, Params, ComponentField, EntitiesScratch);

// FSequenceUpdater_Flat::Update
FSequenceInstance& SequenceInstance = Linker->GetInstanceRegistry()->MutateInstance(InstanceHandle);
SequenceInstance.Ledger.UpdateEntities

// FEntityLedger::UpdateEntities
FInstanceRegistry* InstanceRegistry = Linker->GetInstanceRegistry();
ImportEntity(Linker, ImportParams, EntityField, Query);
}
```


暂停的核心逻辑：
```c++
FMovieSceneEvaluationRange CurrentTimeRange = PlayPosition.GetCurrentPositionAsRange();
const FMovieSceneContext Context(CurrentTimeRange, EMovieScenePlayerStatus::Stopped);
Runner->QueueUpdate(Context, RootTemplateInstance.GetRootInstanceHandle(), FSimpleDelegate::CreateWeakLambda(this, FinishPause));
```

## SequenceRecorder
SequenceRecorder模块的基类是ISequenceRecorder。

# Sequencer ECS
## 使用ECS的原因
**可伸缩性**  
如果使用新的运行时，应该能够编写包含成百上千个轨道或序列的内容，并且将这些内容作为一个整体来优化求值逻辑。这包括：
- 分配和组织数据，使性能不会随着活动Sequencer轨道的增加而快速恶化。
- 能够完全删除已知不再需要的逻辑和分支，不必在每一帧都为它们付出成本。没有代码就是最快的代码。这条原则应该适用于Sequencer求值代码的所有方面，而不仅限于最高级。
- 求值逻辑应该能够直接、高效且不受阻碍地批量访问必要数据，而不必通过复杂或低效的抽象与内存交互。

**并发性**  
写入求值逻辑应该很简单，而且天然就能安全而高效地扩展到多个核心，包括上游/下游依赖性的表达定义（例如，并行地对所有曲线求值）。由于具备仅设置一次管线的综合优点，这不仅能让许多轻量级的小型动画受益，也有益于非常大的序列。  
  
**可扩展性**  
应该可以在内置功能的基础上构建逻辑，而不必重新实现核心系统。添加与核心系统交互的上游或下游功能应该是可以合理实现的。这包括：
- 在管线中的任何位置，当前帧的所有数据都应该是透明且可改变的。
- 可靠的依赖性管理。

这些设计目标，再加上Sequencer本身的各种问题，使得采用[数据导向的设计](https://en.wikipedia.org/wiki/Data-oriented_design)原则成为自然而然的选择，理由如下：
- 大部分Sequencer数据是同质的，可以循序布局。
- 每种数据变换的逻辑通常具有非常高的独立性，与情境无关（即运行函数f(x)来得到曲线）。
- 控制和数据流是非常线性的，没有循环或递归的依赖性（也就是说，没有一种逻辑会要求重新计算已经计算出的数值）。
- 只有初始设置和最终属性设置器才有线程限制。

可参考Tracker实现：
- https://www.unrealengine.com/marketplace/zh-CN/product/blueprintsequencertrack
- GeometryCacheTracks & GeometryCacheSequencer
- MovieSceneEventTrack
	- MovieSceneEventSection
	- UMovieSceneEventTriggerSection -> UMovieSceneEventSectionBase -> UMovieSceneSection
	- UMovieSceneEventRepeaterSection  -> UMovieSceneEventSectionBase -> UMovieSceneSection
	- MovieSceneEventSystem
	- MovieSceneEventChannel
## Sequencer 组件相关概念