BlueRoseNote/03-UnrealEngine/Sequence/Sequence Runtime Binding.md

---
title: Sequence Runtime Binding
date: 2024-05-16 17:55:58
excerpt: 
tags: 
rating: ⭐
---
# 前言
***MovieSceneTextTrack*** 引擎插件可以作为参考。
***UMovieSceneCVarTrackInstance***

参考：
1. [sequencer mute tracks at runtime from c](https://forums.unrealengine.com/t/sequencer-mute-tracks-at-runtime-from-c/476278/3)
2. 其他参考
	1. [Actor Rebinding in Blueprints with Sequencer(官方文档)]https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/Sequencer/HowTo/AnimateDynamicObjects/
	2. [Level Sequence actor rebinding in C++](https://www.reddit.com/r/unrealengine/comments/btk1uz/level_sequence_actor_rebinding_in_c/)
3. Sequence ECS资料
	1. [大规模内容的性能保障：虚幻引擎4.26中的Sequencer(官方文档)](https://www.unrealengine.com/zh-CN/tech-blog/performance-at-scale-sequencer-in-unreal-engine-4-26)
	2. [UE4.26(5.0)后的Sequence系统](https://zhuanlan.zhihu.com/p/589465561)
	3. [UE5 Sequence 浅-浅-析](https://zhuanlan.zhihu.com/p/676690043)
4. ECS资料 
	1. [一文看懂ECS架构]https://zhuanlan.zhihu.com/p/618971664

可行思路：
1. 给导播创建一个快捷键（Hotkey），之后在触发根据Tag切换对应角色动画蓝图中的逻辑。
2. [x] 使用Sequence的EventTrack设置若干事件。
3. 模改UMovieSceneSkeletalAnimationTrack，实现一个可以自动根据Tag捕获对应角色并且播发动画的功能。

# 方案一
- UMovieSceneTrack::SetEvalDisabled(bool)
- ULevelSequence::MarkAsChanged()
- MovieSceneCompiledDataManager::GetPrecompiledData()->Compile(ULevelSequence*)
```c++
// You first have to set the unwanted track as disabled via 
void YourDisableTrackFunction(UMovieSceneTrack* MovieSceneTrack, bool Value)
{
	if (MovieSceneTrack)
	{
		MovieSceneTrack->SetEvalDisabled(Value);
	}
}


// Then notify the sequencer that the tracks have been modified using MarkAsChanged and UMovieSceneCompiledDataManager::GetPrecompiledData()->Compile.
void YourMarkAsChangedFunction(ULevelSequence* Sequence)
{
	if (Sequence)
	{
		Sequence->MarkAsChanged();
		if (UMovieSceneCompiledDataManager::GetPrecompiledData())
		{
			UMovieSceneCompiledDataManager::GetPrecompiledData()->Compile(Sequence);
		}
	}
}
```

# 方案二
UMovieSceneSequenceExtensions methods are not exported. They don’t have the PLUGIN_API macro in the class, so you can’t link to them. Someone on discord said you can copy the plugin into your own project and add your own API macro and rebuild, but seems like a bit of work.
```c++
#include "LevelSequencePlayer.h" #include "MovieSceneSequencePlaybackSettings.h" #include "DefaultLevelSequenceInstanceData.h" #include "LevelSequenceActor.h" void UMyAnimInstance::StartSequence_Implementation() { FMovieSceneSequencePlaybackSettings Settings; ALevelSequenceActor *SequenceActor = nullptr; ULevelSequencePlayer* player = ULevelSequencePlayer::CreateLevelSequencePlayer(this, this->MySequence, Settings, SequenceActor); check(player); check(SequenceActor); SequenceActor->bOverrideInstanceData = 1; UDefaultLevelSequenceInstanceData* InstanceData = Cast<UDefaultLevelSequenceInstanceData>(SequenceActor->DefaultInstanceData.Get()); InstanceData->TransformOrigin = this->GetOwningActor()->GetActorTransform(); UMovieScene* MovieScene = this->MySequence->GetMovieScene(); check(MovieScene); const FMovieSceneBinding* Binding = Algo::FindBy(MovieScene->GetBindings(), FString("SequencerActor"), &FMovieSceneBinding::GetName); check(Binding); EMovieSceneObjectBindingSpace Space = EMovieSceneObjectBindingSpace::Root; FMovieSceneObjectBindingID BindingID = UE::MovieScene::FRelativeObjectBindingID(Binding->GetObjectGuid()); // Not sure about this if (Space == EMovieSceneObjectBindingSpace::Root) { BindingID.ReinterpretAsFixed(); } SequenceActor->AddBinding(BindingID, this->GetOwningActor()); player->Play(); // Save actor and player in properties this->SequencerActor = SequenceActor; this->SequencerPlayer = player; }
```

# 自定义Track
- Runtime修改绑定
	- UE4：扩展Sequencer的BakeTransform有关Camera的参数:https://zhuanlan.zhihu.com/p/554484287
	- **LevelSequence对象绑定分析**:https://zhuanlan.zhihu.com/p/548155341
- 自定义Track
	- https://zhuanlan.zhihu.com/p/396353973
	- https://zhuanlan.zhihu.com/p/414179358
	- https://zhuanlan.zhihu.com/p/413151867
- LevelSequence分析
	- https://zhuanlan.zhihu.com/p/157892605

## UMovieSceneTrackInstance
大部分Track的instance类。

## 继承关系
UMovieSceneSignedObject -> UObject。一个UMovieScene由若干个MasterTrack(UMovieSceneTrack)组成，UMovieScene相当于是UMovieSceneTracks的容器，那么UMovieSceneTrack又由UMovieSceneSections组成，UMovieSceneSection就是一个Track中间的某一段。

### UMovieSceneSignedObject
子类有UMovieScene、UMovieSceneSequence、UMovieSceneTrack、UMovieSceneSection。
定义： 
- FGuid Signature：用于确定Object身份，以便于绑定。
- FOnSignatureChanged OnSignatureChangedEvent：ChangedEvent。

### UMovieSceneSection
#### UMovieSceneSkeletalAnimationSection


### UMovieSceneTrack
#### UMovieSceneSkeletalAnimationTrack
- `TArray<TObjectPtr<UMovieSceneSection>> AnimationSections`：存储所有动画Section。
- FMovieSceneSkeletalAnimRootMotionTrackParams RootMotionParams：RootMotion控制。

编辑器代码：**FSkeletalAnimationTrackEditor**

## Sequence绑定机制笔记
1. _SpawnableObject_(自动Spawn的Object)  
    表示该Object在Sequence被Evaluate的时候自动Spawn，并且由Sequence来管理生命周期
2. _PossessableObject_(可以被赋予的Object)  
    表示该Object可以被外部赋予，比如我手动赋予一个Actor给某个Track
### UMovieSceneSequence
UMovieSceneSequence::CreatePossessable()
UMovieSceneSequence::BindPossessableObject()

绑定过程可以参考：
ULevelSequenceEditorSubsystem::AddActorsToBinding(const TArray<AActor*>& Actors, const FMovieSceneBindingProxy& ObjectBinding)


### ActorToSequencer
菜单出现逻辑位于：FLevelSequenceEditorActorBinding::BuildSequencerAddMenu(FMenuBuilder& MenuBuilder)。

`FLevelSequenceEditorActorBinding::AddPossessActorMenuExtensions(FMenuBuilder& MenuBuilder)`
=>
`FLevelSequenceEditorActorBinding::AddActorsToSequencer(AActor*const* InActors, int32 NumActors)`
=>
`TArray<FGuid> FSequencer::AddActors(const TArray<TWeakObjectPtr<AActor> >& InActors, bool bSelectActors)；`
=>
`TArray<FGuid> FSequencerUtilities::AddActors(TSharedRef<ISequencer> Sequencer, const TArray<TWeakObjectPtr<AActor> >& InActors)`

```c++
TArray<FGuid> FSequencerUtilities::AddActors(TSharedRef<ISequencer> Sequencer, const TArray<TWeakObjectPtr<AActor> >& InActors)
{
	TArray<FGuid> PossessableGuids;

	UMovieSceneSequence* Sequence = Sequencer->GetFocusedMovieSceneSequence();
	if (!Sequence)
	{
		return PossessableGuids;
	}

	UMovieScene* MovieScene = Sequence->GetMovieScene();
	if (!MovieScene)
	{
		return PossessableGuids;
	}

	if (MovieScene->IsReadOnly())
	{
		ShowReadOnlyError();
		return PossessableGuids;
	}
	
	const FScopedTransaction Transaction(LOCTEXT("AddActors", "Add Actors"));
	Sequence->Modify();

	for (TWeakObjectPtr<AActor> WeakActor : InActors)
	{
		if (AActor* Actor = WeakActor.Get())
		{
			FGuid ExistingGuid = Sequencer->FindObjectId(*Actor, Sequencer->GetFocusedTemplateID());
			if (!ExistingGuid.IsValid())
			{
				FGuid PossessableGuid = CreateBinding(Sequencer, *Actor, Actor->GetActorLabel());
				PossessableGuids.Add(PossessableGuid);

				if (ACameraActor* CameraActor = Cast<ACameraActor>(Actor))
				{
					NewCameraAdded(Sequencer, CameraActor, PossessableGuid);
				}

				Sequencer->OnActorAddedToSequencer().Broadcast(Actor, PossessableGuid);
			}
		}
	}

	return PossessableGuids;
}
```

其他参考
```c++
const FGuid PossessableGuid = OwnerMovieScene->AddPossessable(InName, InObject.GetClass());

if (!OwnerMovieScene->FindPossessable(PossessableGuid)->BindSpawnableObject(Sequencer->GetFocusedTemplateID(), &InObject, &Sequencer.Get()))
{
	OwnerSequence->BindPossessableObject(PossessableGuid, InObject, BindingContext);
}
```

```c++
FGuid FSequencerUtilities::CreateBinding(TSharedRef<ISequencer> Sequencer, UObject& InObject, const FString& InName)
{
	const FScopedTransaction Transaction(LOCTEXT("CreateBinding", "Create New Binding"));

	UMovieSceneSequence* OwnerSequence = Sequencer->GetFocusedMovieSceneSequence();
	UMovieScene* OwnerMovieScene = OwnerSequence->GetMovieScene();

	OwnerSequence->Modify();
	OwnerMovieScene->Modify();

	const FGuid PossessableGuid = OwnerMovieScene->AddPossessable(InName, InObject.GetClass());

	// Attempt to use the parent as a context if necessary
	UObject* ParentObject = OwnerSequence->GetParentObject(&InObject);
	UObject* BindingContext = Sequencer->GetPlaybackContext();

	AActor* ParentActorAdded = nullptr;
	FGuid ParentGuid;

	if (ParentObject)
	{
		// Ensure we have possessed the outer object, if necessary
		ParentGuid = Sequencer->GetHandleToObject(ParentObject, false);
		if (!ParentGuid.IsValid())
		{
			ParentGuid = Sequencer->GetHandleToObject(ParentObject);
			ParentActorAdded = Cast<AActor>(ParentObject);
		}

		if (OwnerSequence->AreParentContextsSignificant())
		{
			BindingContext = ParentObject;
		}

		// Set up parent/child guids for possessables within spawnables
		if (ParentGuid.IsValid())
		{
			FMovieScenePossessable* ChildPossessable = OwnerMovieScene->FindPossessable(PossessableGuid);
			if (ensure(ChildPossessable))
			{
				ChildPossessable->SetParent(ParentGuid, OwnerMovieScene);
			}

			FMovieSceneSpawnable* ParentSpawnable = OwnerMovieScene->FindSpawnable(ParentGuid);
			if (ParentSpawnable)
			{
				ParentSpawnable->AddChildPossessable(PossessableGuid);
			}
		}
	}

	if (!OwnerMovieScene->FindPossessable(PossessableGuid)->BindSpawnableObject(Sequencer->GetFocusedTemplateID(), &InObject, &Sequencer.Get()))
	{
		OwnerSequence->BindPossessableObject(PossessableGuid, InObject, BindingContext);
	}

	// Broadcast if a parent actor was added as a result of adding this object
	if (ParentActorAdded && ParentGuid.IsValid())
	{
		Sequencer->OnActorAddedToSequencer().Broadcast(ParentActorAdded, ParentGuid);
	}

	return PossessableGuid;
}
```

### FAudioTrackEditor
FAudioTrackEditor::HandleAssetAdded(UObject* Asset, const FGuid& TargetObjectGuid) => FAudioTrackEditor::AddNewSound() => UMovieSceneAudioTrack::AddNewSoundOnRow(USoundBase* Sound, FFrameNumber Time, int32 RowIndex)

## MovieSceneSkeletalAnimationTrack
- UMovieSceneSkeletalAnimationTrack
- UMovieSceneSkeletalAnimationSection
	- FMovieSceneSkeletalAnimationParams
		- `TObjectPtr<UAnimSequenceBase> Animation`
- FSkeletalAnimationTrackEditor
- UMovieSceneSkeletalAnimationSystem
	- 里面有介绍System相关的逻辑 https://zhuanlan.zhihu.com/p/413151867
- FSkeletalAnimationTrackEditMode

Track的编辑器注册位于MovieSceneToolsModule.cpp：
```c++
AnimationTrackCreateEditorHandle = SequencerModule.RegisterTrackEditor( FOnCreateTrackEditor::CreateStatic( &FSkeletalAnimationTrackEditor::CreateTrackEditor ) );
```

只对UMovieSceneSkeletalAnimationTrack生效：
```c++
bool FSkeletalAnimationTrackEditor::SupportsSequence(UMovieSceneSequence* InSequence) const  
{  
    ETrackSupport TrackSupported = InSequence ? InSequence->IsTrackSupported(UMovieSceneSkeletalAnimationTrack::StaticClass()) : ETrackSupport::NotSupported;  
    return TrackSupported == ETrackSupport::Supported;  
}

FEditorModeRegistry::Get().RegisterMode<FSkeletalAnimationTrackEditMode>(  
    FSkeletalAnimationTrackEditMode::ModeName,  
    NSLOCTEXT("SkeletalAnimationTrackEditorMode", "SkelAnimTrackEditMode", "Skeletal Anim Track Mode"),  
    FSlateIcon(),  
    false);
```

### FSkeletalAnimationTrackEditor
FSkeletalAnimationTrackEditor::BuildTrackContextMenu()：在MovieSceneSkeletalAnimationTrack的Animation上右键出现的菜单。
FSkeletalAnimationTrackEditor::BuildOutlinerEditWidget：MovieSceneSkeletalAnimationTrack下面生成Animtion那一列。

##  播放逻辑
```c++
ALevelSequenceActor::InitializePlayer() 
ALevelSequenceActor->SequencePlayer->Play(); 
ALevelSequenceActor->SequencePlayer->Update(DeltaSeconds);
```

具体的逻辑位于UMovieSceneSequencePlayer::PlayInternal()：
```c++

void UMovieSceneSequencePlayer::PlayInternal()
{
	if (NeedsQueueLatentAction())
	{
		QueueLatentAction(FMovieSceneSequenceLatentActionDelegate::CreateUObject(this, &UMovieSceneSequencePlayer::PlayInternal));
		return;
	}

	if (!IsPlaying() && Sequence && CanPlay())
	{
		const FString SequenceName = GetSequenceName(true);
		UE_LOG(LogMovieScene, Verbose, TEXT("PlayInternal - %s (current status: %s)"), *SequenceName, *UEnum::GetValueAsString(Status));

		// Set playback status to playing before any calls to update the position
		Status = EMovieScenePlayerStatus::Playing;

		float PlayRate = bReversePlayback ? -PlaybackSettings.PlayRate : PlaybackSettings.PlayRate;

		// If at the end and playing forwards, rewind to beginning
		if (GetCurrentTime().Time == GetLastValidTime())
		{
			if (PlayRate > 0.f)
			{
				SetPlaybackPosition(FMovieSceneSequencePlaybackParams(FFrameTime(StartTime), EUpdatePositionMethod::Jump));
			}
		}
		else if (GetCurrentTime().Time == FFrameTime(StartTime))
		{
			if (PlayRate < 0.f)
			{
				SetPlaybackPosition(FMovieSceneSequencePlaybackParams(GetLastValidTime(), EUpdatePositionMethod::Jump));
			}
		}

		// Update now
		if (PlaybackSettings.bRestoreState)
		{
			RootTemplateInstance.EnableGlobalPreAnimatedStateCapture();
		}

		bPendingOnStartedPlaying = true;
		Status = EMovieScenePlayerStatus::Playing;
		TimeController->StartPlaying(GetCurrentTime());
		
		if (PlayPosition.GetEvaluationType() == EMovieSceneEvaluationType::FrameLocked)
		{
			if (!OldMaxTickRate.IsSet())
			{
				OldMaxTickRate = GEngine->GetMaxFPS();
			}

			GEngine->SetMaxFPS(1.f / PlayPosition.GetInputRate().AsInterval());
		}

		//播放的核心逻辑估计在这里。
		if (!PlayPosition.GetLastPlayEvalPostition().IsSet() || PlayPosition.GetLastPlayEvalPostition() != PlayPosition.GetCurrentPosition())
		{
			UpdateMovieSceneInstance(PlayPosition.PlayTo(PlayPosition.GetCurrentPosition()), EMovieScenePlayerStatus::Playing);
		}

		RunLatentActions();
		UpdateNetworkSyncProperties();
		
		if (bReversePlayback)
		{
			if (OnPlayReverse.IsBound())
			{
				OnPlayReverse.Broadcast();
			}
		}
		else
		{
			if (OnPlay.IsBound())
			{
				OnPlay.Broadcast();
			}
		}
	}
}

void UMovieSceneSequencePlayer::UpdateMovieSceneInstance(FMovieSceneEvaluationRange InRange, EMovieScenePlayerStatus::Type PlayerStatus, bool bHasJumped)
{
	FMovieSceneUpdateArgs Args;
	Args.bHasJumped = bHasJumped;
	UpdateMovieSceneInstance(InRange, PlayerStatus, Args);
}

void UMovieSceneSequencePlayer::UpdateMovieSceneInstance(FMovieSceneEvaluationRange InRange, EMovieScenePlayerStatus::Type PlayerStatus, const FMovieSceneUpdateArgs& Args)
{
	if (Observer && !Observer->CanObserveSequence())
	{
		return;
	}

	UMovieSceneSequence* MovieSceneSequence = RootTemplateInstance.GetSequence(MovieSceneSequenceID::Root);
	if (!MovieSceneSequence)
	{
		return;
	}

#if !NO_LOGGING
	if (UE_LOG_ACTIVE(LogMovieScene, VeryVerbose))
	{
		const FQualifiedFrameTime CurrentTime = GetCurrentTime();
		const FString SequenceName = GetSequenceName(true);
		UE_LOG(LogMovieScene, VeryVerbose, TEXT("Evaluating sequence %s at frame %d, subframe %f (%f fps)."), *SequenceName, CurrentTime.Time.FrameNumber.Value, CurrentTime.Time.GetSubFrame(), CurrentTime.Rate.AsDecimal());
	}
#endif

	if (PlaybackClient)
	{
		PlaybackClient->WarpEvaluationRange(InRange);
	}

	// Once we have updated we must no longer skip updates
	bSkipNextUpdate = false;

	// We shouldn't be asked to run an async update if we have a blocking sequence.
	check(!Args.bIsAsync || !EnumHasAnyFlags(MovieSceneSequence->GetFlags(), EMovieSceneSequenceFlags::BlockingEvaluation));
	// We shouldn't be asked to run an async update if we don't have a tick manager.
	check(!Args.bIsAsync || TickManager != nullptr);

	FMovieSceneContext Context(InRange, PlayerStatus);
	Context.SetHasJumped(Args.bHasJumped);

	TSharedPtr<FMovieSceneEntitySystemRunner> Runner = RootTemplateInstance.GetRunner();
	if (Runner)
	{
		Runner->QueueUpdate(Context, RootTemplateInstance.GetRootInstanceHandle());
		if (Runner == SynchronousRunner || !Args.bIsAsync)
		{
			Runner->Flush();
		}
	}
}
```

这段解析来自于：https://zhuanlan.zhihu.com/p/413151867
```c++
UMovieSceneSequencePlayer::Play()           //蓝图中调用
UMovieSceneSequencePlayer::PlayInternal()
UMovieSceneSequencePlayer::UpdateMovieSceneInstance
RootTemplateInstance.Evaluate(Context, *this);

// FMovieSceneRootEvaluationTemplateInstance::Evaluate
EntitySystemRunner.Update(Context, RootInstanceHandle);

FMovieSceneEntitySystemRunner::Flush()
FMovieSceneEntitySystemRunner::DoFlushUpdateQueueOnce()
FMovieSceneEntitySystemRunner::GameThread_ProcessQueue()
FMovieSceneEntitySystemRunner::GameThread_SpawnPhase()

// FMovieSceneEntitySystemRunner::GameThread_SpawnPhase()
/// System Linker->GetInstanceRegistry();
FInstanceRegistry* InstanceRegistry = GetInstanceRegistry(); 
FSequenceInstance& Instance = InstanceRegistry->MutateInstance(Update.InstanceHandle);
Instance.Update(Linker, Update.Context);

// FSequenceInstance::Update
SequenceUpdater->Update(Linker, Params, ComponentField, EntitiesScratch);

// FSequenceUpdater_Flat::Update
FSequenceInstance& SequenceInstance = Linker->GetInstanceRegistry()->MutateInstance(InstanceHandle);
SequenceInstance.Ledger.UpdateEntities

// FEntityLedger::UpdateEntities
FInstanceRegistry* InstanceRegistry = Linker->GetInstanceRegistry();
ImportEntity(Linker, ImportParams, EntityField, Query);
}
```


暂停的核心逻辑：
```c++
FMovieSceneEvaluationRange CurrentTimeRange = PlayPosition.GetCurrentPositionAsRange();
const FMovieSceneContext Context(CurrentTimeRange, EMovieScenePlayerStatus::Stopped);
Runner->QueueUpdate(Context, RootTemplateInstance.GetRootInstanceHandle(), FSimpleDelegate::CreateWeakLambda(this, FinishPause));
```

## SequenceRecorder
SequenceRecorder模块的基类是ISequenceRecorder。

# Sequencer ECS
## 使用ECS的原因
**可伸缩性**  
如果使用新的运行时，应该能够编写包含成百上千个轨道或序列的内容，并且将这些内容作为一个整体来优化求值逻辑。这包括：
- 分配和组织数据，使性能不会随着活动Sequencer轨道的增加而快速恶化。
- 能够完全删除已知不再需要的逻辑和分支，不必在每一帧都为它们付出成本。没有代码就是最快的代码。这条原则应该适用于Sequencer求值代码的所有方面，而不仅限于最高级。
- 求值逻辑应该能够直接、高效且不受阻碍地批量访问必要数据，而不必通过复杂或低效的抽象与内存交互。

**并发性**  
写入求值逻辑应该很简单，而且天然就能安全而高效地扩展到多个核心，包括上游/下游依赖性的表达定义（例如，并行地对所有曲线求值）。由于具备仅设置一次管线的综合优点，这不仅能让许多轻量级的小型动画受益，也有益于非常大的序列。  
  
**可扩展性**  
应该可以在内置功能的基础上构建逻辑，而不必重新实现核心系统。添加与核心系统交互的上游或下游功能应该是可以合理实现的。这包括：
- 在管线中的任何位置，当前帧的所有数据都应该是透明且可改变的。
- 可靠的依赖性管理。

这些设计目标，再加上Sequencer本身的各种问题，使得采用[数据导向的设计](https://en.wikipedia.org/wiki/Data-oriented_design)原则成为自然而然的选择，理由如下：
- 大部分Sequencer数据是同质的，可以循序布局。
- 每种数据变换的逻辑通常具有非常高的独立性，与情境无关（即运行函数f(x)来得到曲线）。
- 控制和数据流是非常线性的，没有循环或递归的依赖性（也就是说，没有一种逻辑会要求重新计算已经计算出的数值）。
- 只有初始设置和最终属性设置器才有线程限制。


TODO：
1. https://www.unrealengine.com/marketplace/zh-CN/product/blueprintsequencertrack
2. 

可参考Tracker实现：
- GeometryCacheTracks & GeometryCacheSequencer
- MovieSceneEventTrack
	- MovieSceneEventSection
	- UMovieSceneEventTriggerSection -> UMovieSceneEventSectionBase -> UMovieSceneSection
	- UMovieSceneEventRepeaterSection  -> UMovieSceneEventSectionBase -> UMovieSceneSection
	- MovieSceneEventSystem
	- MovieSceneEventChannel
## Sequencer 组件相关概念