--- title: Sequence Runtime Binding date: 2024-05-16 17:55:58 excerpt: tags: rating: ⭐ --- # 前言 ***MovieSceneTextTrack*** 引擎插件可以作为参考。 ***UMovieSceneCVarTrackInstance*** 参考： 1. [sequencer mute tracks at runtime from c](https://forums.unrealengine.com/t/sequencer-mute-tracks-at-runtime-from-c/476278/3) 2. 其他参考 1. [Actor Rebinding in Blueprints with Sequencer(官方文档)]https://docs.unrealengine.com/4.27/en-US/AnimatingObjects/Sequencer/HowTo/AnimateDynamicObjects/ 2. [Level Sequence actor rebinding in C++](https://www.reddit.com/r/unrealengine/comments/btk1uz/level_sequence_actor_rebinding_in_c/) 3. Sequence ECS资料 1. [大规模内容的性能保障：虚幻引擎4.26中的Sequencer(官方文档)](https://www.unrealengine.com/zh-CN/tech-blog/performance-at-scale-sequencer-in-unreal-engine-4-26) 2. [UE4.26(5.0)后的Sequence系统](https://zhuanlan.zhihu.com/p/589465561) 3. [UE5 Sequence 浅-浅-析](https://zhuanlan.zhihu.com/p/676690043) 4. ECS资料 1. [一文看懂ECS架构]https://zhuanlan.zhihu.com/p/618971664 可行思路： 1. 给导播创建一个快捷键（Hotkey），之后在触发根据Tag切换对应角色动画蓝图中的逻辑。 2. [x] 使用Sequence的EventTrack设置若干事件。 3. 模改UMovieSceneSkeletalAnimationTrack，实现一个可以自动根据Tag捕获对应角色并且播发动画的功能。 # 自定义Track - Runtime修改绑定 - UE4：扩展Sequencer的BakeTransform有关Camera的参数:https://zhuanlan.zhihu.com/p/554484287 - **LevelSequence对象绑定分析**:https://zhuanlan.zhihu.com/p/548155341 - 自定义Track - https://zhuanlan.zhihu.com/p/396353973 - https://zhuanlan.zhihu.com/p/414179358 - https://zhuanlan.zhihu.com/p/413151867 - LevelSequence分析 - https://zhuanlan.zhihu.com/p/157892605 ## UMovieSceneTrackInstance 大部分Track的instance类。 ## 继承关系 UMovieSceneSignedObject -> UObject。一个UMovieScene由若干个MasterTrack(UMovieSceneTrack)组成，UMovieScene相当于是UMovieSceneTracks的容器，那么UMovieSceneTrack又由UMovieSceneSections组成，UMovieSceneSection就是一个Track中间的某一段。 ### UMovieSceneSignedObject 子类有UMovieScene、UMovieSceneSequence、UMovieSceneTrack、UMovieSceneSection。 定义： - FGuid Signature：用于确定Object身份，以便于绑定。 - FOnSignatureChanged OnSignatureChangedEvent：ChangedEvent。 ### UMovieSceneSection #### UMovieSceneSkeletalAnimationSection ### UMovieSceneTrack #### UMovieSceneSkeletalAnimationTrack - `TArray> AnimationSections`：存储所有动画Section。 - FMovieSceneSkeletalAnimRootMotionTrackParams RootMotionParams：RootMotion控制。 编辑器代码：**FSkeletalAnimationTrackEditor** ## Sequence绑定机制笔记 1. _SpawnableObject_(自动Spawn的Object) 表示该Object在Sequence被Evaluate的时候自动Spawn，并且由Sequence来管理生命周期 2. _PossessableObject_(可以被赋予的Object) 表示该Object可以被外部赋予，比如我手动赋予一个Actor给某个Track ### UMovieSceneSequence UMovieSceneSequence::CreatePossessable() UMovieSceneSequence::BindPossessableObject() 绑定过程可以参考： ULevelSequenceEditorSubsystem::AddActorsToBinding(const TArray& Actors, const FMovieSceneBindingProxy& ObjectBinding) ### ActorToSequencer 菜单出现逻辑位于：FLevelSequenceEditorActorBinding::BuildSequencerAddMenu(FMenuBuilder& MenuBuilder)。 `FLevelSequenceEditorActorBinding::AddPossessActorMenuExtensions(FMenuBuilder& MenuBuilder)` => `FLevelSequenceEditorActorBinding::AddActorsToSequencer(AActor*const* InActors, int32 NumActors)` => `TArray FSequencer::AddActors(const TArray >& InActors, bool bSelectActors)；` => `TArray FSequencerUtilities::AddActors(TSharedRef Sequencer, const TArray >& InActors)` ```c++ TArray FSequencerUtilities::AddActors(TSharedRef Sequencer, const TArray >& InActors) { TArray PossessableGuids; UMovieSceneSequence* Sequence = Sequencer->GetFocusedMovieSceneSequence(); if (!Sequence) { return PossessableGuids; } UMovieScene* MovieScene = Sequence->GetMovieScene(); if (!MovieScene) { return PossessableGuids; } if (MovieScene->IsReadOnly()) { ShowReadOnlyError(); return PossessableGuids; } const FScopedTransaction Transaction(LOCTEXT("AddActors", "Add Actors")); Sequence->Modify(); for (TWeakObjectPtr WeakActor : InActors) { if (AActor* Actor = WeakActor.Get()) { FGuid ExistingGuid = Sequencer->FindObjectId(*Actor, Sequencer->GetFocusedTemplateID()); if (!ExistingGuid.IsValid()) { FGuid PossessableGuid = CreateBinding(Sequencer, *Actor, Actor->GetActorLabel()); PossessableGuids.Add(PossessableGuid); if (ACameraActor* CameraActor = Cast(Actor)) { NewCameraAdded(Sequencer, CameraActor, PossessableGuid); } Sequencer->OnActorAddedToSequencer().Broadcast(Actor, PossessableGuid); } } } return PossessableGuids; } ``` 其他参考 ```c++ const FGuid PossessableGuid = OwnerMovieScene->AddPossessable(InName, InObject.GetClass()); if (!OwnerMovieScene->FindPossessable(PossessableGuid)->BindSpawnableObject(Sequencer->GetFocusedTemplateID(), &InObject, &Sequencer.Get())) { OwnerSequence->BindPossessableObject(PossessableGuid, InObject, BindingContext); } ``` ```c++ FGuid FSequencerUtilities::CreateBinding(TSharedRef Sequencer, UObject& InObject, const FString& InName) { const FScopedTransaction Transaction(LOCTEXT("CreateBinding", "Create New Binding")); UMovieSceneSequence* OwnerSequence = Sequencer->GetFocusedMovieSceneSequence(); UMovieScene* OwnerMovieScene = OwnerSequence->GetMovieScene(); OwnerSequence->Modify(); OwnerMovieScene->Modify(); const FGuid PossessableGuid = OwnerMovieScene->AddPossessable(InName, InObject.GetClass()); // Attempt to use the parent as a context if necessary UObject* ParentObject = OwnerSequence->GetParentObject(&InObject); UObject* BindingContext = Sequencer->GetPlaybackContext(); AActor* ParentActorAdded = nullptr; FGuid ParentGuid; if (ParentObject) { // Ensure we have possessed the outer object, if necessary ParentGuid = Sequencer->GetHandleToObject(ParentObject, false); if (!ParentGuid.IsValid()) { ParentGuid = Sequencer->GetHandleToObject(ParentObject); ParentActorAdded = Cast(ParentObject); } if (OwnerSequence->AreParentContextsSignificant()) { BindingContext = ParentObject; } // Set up parent/child guids for possessables within spawnables if (ParentGuid.IsValid()) { FMovieScenePossessable* ChildPossessable = OwnerMovieScene->FindPossessable(PossessableGuid); if (ensure(ChildPossessable)) { ChildPossessable->SetParent(ParentGuid, OwnerMovieScene); } FMovieSceneSpawnable* ParentSpawnable = OwnerMovieScene->FindSpawnable(ParentGuid); if (ParentSpawnable) { ParentSpawnable->AddChildPossessable(PossessableGuid); } } } if (!OwnerMovieScene->FindPossessable(PossessableGuid)->BindSpawnableObject(Sequencer->GetFocusedTemplateID(), &InObject, &Sequencer.Get())) { OwnerSequence->BindPossessableObject(PossessableGuid, InObject, BindingContext); } // Broadcast if a parent actor was added as a result of adding this object if (ParentActorAdded && ParentGuid.IsValid()) { Sequencer->OnActorAddedToSequencer().Broadcast(ParentActorAdded, ParentGuid); } return PossessableGuid; } ``` ### FAudioTrackEditor FAudioTrackEditor::HandleAssetAdded(UObject* Asset, const FGuid& TargetObjectGuid) => FAudioTrackEditor::AddNewSound() => UMovieSceneAudioTrack::AddNewSoundOnRow(USoundBase* Sound, FFrameNumber Time, int32 RowIndex) ## MovieSceneSkeletalAnimationTrack - UMovieSceneSkeletalAnimationTrack - UMovieSceneSkeletalAnimationSection - FMovieSceneSkeletalAnimationParams - `TObjectPtr Animation` - FSkeletalAnimationTrackEditor - UMovieSceneSkeletalAnimationSystem - 里面有介绍System相关的逻辑 https://zhuanlan.zhihu.com/p/413151867 - FSkeletalAnimationTrackEditMode Track的编辑器注册位于MovieSceneToolsModule.cpp： ```c++ AnimationTrackCreateEditorHandle = SequencerModule.RegisterTrackEditor( FOnCreateTrackEditor::CreateStatic( &FSkeletalAnimationTrackEditor::CreateTrackEditor ) ); ``` 只对UMovieSceneSkeletalAnimationTrack生效： ```c++ bool FSkeletalAnimationTrackEditor::SupportsSequence(UMovieSceneSequence* InSequence) const { ETrackSupport TrackSupported = InSequence ? InSequence->IsTrackSupported(UMovieSceneSkeletalAnimationTrack::StaticClass()) : ETrackSupport::NotSupported; return TrackSupported == ETrackSupport::Supported; } FEditorModeRegistry::Get().RegisterMode( FSkeletalAnimationTrackEditMode::ModeName, NSLOCTEXT("SkeletalAnimationTrackEditorMode", "SkelAnimTrackEditMode", "Skeletal Anim Track Mode"), FSlateIcon(), false); ``` ### FSkeletalAnimationTrackEditor FSkeletalAnimationTrackEditor::BuildTrackContextMenu()：在MovieSceneSkeletalAnimationTrack的Animation上右键出现的菜单。 FSkeletalAnimationTrackEditor::BuildOutlinerEditWidget：MovieSceneSkeletalAnimationTrack下面生成Animtion那一列。 ## 播放逻辑 ```c++ ALevelSequenceActor::InitializePlayer() ALevelSequenceActor->SequencePlayer->Play(); ALevelSequenceActor->SequencePlayer->Update(DeltaSeconds); ``` 具体的逻辑位于UMovieSceneSequencePlayer::PlayInternal()： ```c++ void UMovieSceneSequencePlayer::PlayInternal() { if (NeedsQueueLatentAction()) { QueueLatentAction(FMovieSceneSequenceLatentActionDelegate::CreateUObject(this, &UMovieSceneSequencePlayer::PlayInternal)); return; } if (!IsPlaying() && Sequence && CanPlay()) { const FString SequenceName = GetSequenceName(true); UE_LOG(LogMovieScene, Verbose, TEXT("PlayInternal - %s (current status: %s)"), *SequenceName, *UEnum::GetValueAsString(Status)); // Set playback status to playing before any calls to update the position Status = EMovieScenePlayerStatus::Playing; float PlayRate = bReversePlayback ? -PlaybackSettings.PlayRate : PlaybackSettings.PlayRate; // If at the end and playing forwards, rewind to beginning if (GetCurrentTime().Time == GetLastValidTime()) { if (PlayRate > 0.f) { SetPlaybackPosition(FMovieSceneSequencePlaybackParams(FFrameTime(StartTime), EUpdatePositionMethod::Jump)); } } else if (GetCurrentTime().Time == FFrameTime(StartTime)) { if (PlayRate < 0.f) { SetPlaybackPosition(FMovieSceneSequencePlaybackParams(GetLastValidTime(), EUpdatePositionMethod::Jump)); } } // Update now if (PlaybackSettings.bRestoreState) { RootTemplateInstance.EnableGlobalPreAnimatedStateCapture(); } bPendingOnStartedPlaying = true; Status = EMovieScenePlayerStatus::Playing; TimeController->StartPlaying(GetCurrentTime()); if (PlayPosition.GetEvaluationType() == EMovieSceneEvaluationType::FrameLocked) { if (!OldMaxTickRate.IsSet()) { OldMaxTickRate = GEngine->GetMaxFPS(); } GEngine->SetMaxFPS(1.f / PlayPosition.GetInputRate().AsInterval()); } //播放的核心逻辑估计在这里。 if (!PlayPosition.GetLastPlayEvalPostition().IsSet() || PlayPosition.GetLastPlayEvalPostition() != PlayPosition.GetCurrentPosition()) { UpdateMovieSceneInstance(PlayPosition.PlayTo(PlayPosition.GetCurrentPosition()), EMovieScenePlayerStatus::Playing); } RunLatentActions(); UpdateNetworkSyncProperties(); if (bReversePlayback) { if (OnPlayReverse.IsBound()) { OnPlayReverse.Broadcast(); } } else { if (OnPlay.IsBound()) { OnPlay.Broadcast(); } } } } void UMovieSceneSequencePlayer::UpdateMovieSceneInstance(FMovieSceneEvaluationRange InRange, EMovieScenePlayerStatus::Type PlayerStatus, bool bHasJumped) { FMovieSceneUpdateArgs Args; Args.bHasJumped = bHasJumped; UpdateMovieSceneInstance(InRange, PlayerStatus, Args); } void UMovieSceneSequencePlayer::UpdateMovieSceneInstance(FMovieSceneEvaluationRange InRange, EMovieScenePlayerStatus::Type PlayerStatus, const FMovieSceneUpdateArgs& Args) { if (Observer && !Observer->CanObserveSequence()) { return; } UMovieSceneSequence* MovieSceneSequence = RootTemplateInstance.GetSequence(MovieSceneSequenceID::Root); if (!MovieSceneSequence) { return; } #if !NO_LOGGING if (UE_LOG_ACTIVE(LogMovieScene, VeryVerbose)) { const FQualifiedFrameTime CurrentTime = GetCurrentTime(); const FString SequenceName = GetSequenceName(true); UE_LOG(LogMovieScene, VeryVerbose, TEXT("Evaluating sequence %s at frame %d, subframe %f (%f fps)."), *SequenceName, CurrentTime.Time.FrameNumber.Value, CurrentTime.Time.GetSubFrame(), CurrentTime.Rate.AsDecimal()); } #endif if (PlaybackClient) { PlaybackClient->WarpEvaluationRange(InRange); } // Once we have updated we must no longer skip updates bSkipNextUpdate = false; // We shouldn't be asked to run an async update if we have a blocking sequence. check(!Args.bIsAsync || !EnumHasAnyFlags(MovieSceneSequence->GetFlags(), EMovieSceneSequenceFlags::BlockingEvaluation)); // We shouldn't be asked to run an async update if we don't have a tick manager. check(!Args.bIsAsync || TickManager != nullptr); FMovieSceneContext Context(InRange, PlayerStatus); Context.SetHasJumped(Args.bHasJumped); TSharedPtr Runner = RootTemplateInstance.GetRunner(); if (Runner) { Runner->QueueUpdate(Context, RootTemplateInstance.GetRootInstanceHandle()); if (Runner == SynchronousRunner || !Args.bIsAsync) { Runner->Flush(); } } } ``` 这段解析来自于：https://zhuanlan.zhihu.com/p/413151867 ```c++ UMovieSceneSequencePlayer::Play() //蓝图中调用 UMovieSceneSequencePlayer::PlayInternal() UMovieSceneSequencePlayer::UpdateMovieSceneInstance RootTemplateInstance.Evaluate(Context, *this); // FMovieSceneRootEvaluationTemplateInstance::Evaluate EntitySystemRunner.Update(Context, RootInstanceHandle); FMovieSceneEntitySystemRunner::Flush() FMovieSceneEntitySystemRunner::DoFlushUpdateQueueOnce() FMovieSceneEntitySystemRunner::GameThread_ProcessQueue() FMovieSceneEntitySystemRunner::GameThread_SpawnPhase() // FMovieSceneEntitySystemRunner::GameThread_SpawnPhase() /// System Linker->GetInstanceRegistry(); FInstanceRegistry* InstanceRegistry = GetInstanceRegistry(); FSequenceInstance& Instance = InstanceRegistry->MutateInstance(Update.InstanceHandle); Instance.Update(Linker, Update.Context); // FSequenceInstance::Update SequenceUpdater->Update(Linker, Params, ComponentField, EntitiesScratch); // FSequenceUpdater_Flat::Update FSequenceInstance& SequenceInstance = Linker->GetInstanceRegistry()->MutateInstance(InstanceHandle); SequenceInstance.Ledger.UpdateEntities // FEntityLedger::UpdateEntities FInstanceRegistry* InstanceRegistry = Linker->GetInstanceRegistry(); ImportEntity(Linker, ImportParams, EntityField, Query); } ``` 暂停的核心逻辑： ```c++ FMovieSceneEvaluationRange CurrentTimeRange = PlayPosition.GetCurrentPositionAsRange(); const FMovieSceneContext Context(CurrentTimeRange, EMovieScenePlayerStatus::Stopped); Runner->QueueUpdate(Context, RootTemplateInstance.GetRootInstanceHandle(), FSimpleDelegate::CreateWeakLambda(this, FinishPause)); ``` ## SequenceRecorder SequenceRecorder模块的基类是ISequenceRecorder。 # Sequencer ECS ## 使用ECS的原因 **可伸缩性** 如果使用新的运行时，应该能够编写包含成百上千个轨道或序列的内容，并且将这些内容作为一个整体来优化求值逻辑。这包括： - 分配和组织数据，使性能不会随着活动Sequencer轨道的增加而快速恶化。 - 能够完全删除已知不再需要的逻辑和分支，不必在每一帧都为它们付出成本。没有代码就是最快的代码。这条原则应该适用于Sequencer求值代码的所有方面，而不仅限于最高级。 - 求值逻辑应该能够直接、高效且不受阻碍地批量访问必要数据，而不必通过复杂或低效的抽象与内存交互。 **并发性** 写入求值逻辑应该很简单，而且天然就能安全而高效地扩展到多个核心，包括上游/下游依赖性的表达定义（例如，并行地对所有曲线求值）。由于具备仅设置一次管线的综合优点，这不仅能让许多轻量级的小型动画受益，也有益于非常大的序列。 **可扩展性** 应该可以在内置功能的基础上构建逻辑，而不必重新实现核心系统。添加与核心系统交互的上游或下游功能应该是可以合理实现的。这包括： - 在管线中的任何位置，当前帧的所有数据都应该是透明且可改变的。 - 可靠的依赖性管理。 这些设计目标，再加上Sequencer本身的各种问题，使得采用[数据导向的设计](https://en.wikipedia.org/wiki/Data-oriented_design)原则成为自然而然的选择，理由如下： - 大部分Sequencer数据是同质的，可以循序布局。 - 每种数据变换的逻辑通常具有非常高的独立性，与情境无关（即运行函数f(x)来得到曲线）。 - 控制和数据流是非常线性的，没有循环或递归的依赖性（也就是说，没有一种逻辑会要求重新计算已经计算出的数值）。 - 只有初始设置和最终属性设置器才有线程限制。 可参考Tracker实现： - https://www.unrealengine.com/marketplace/zh-CN/product/blueprintsequencertrack - GeometryCacheTracks & GeometryCacheSequencer - MovieSceneEventTrack - MovieSceneEventSection - UMovieSceneEventTriggerSection -> UMovieSceneEventSectionBase -> UMovieSceneSection - UMovieSceneEventRepeaterSection -> UMovieSceneEventSectionBase -> UMovieSceneSection - MovieSceneEventSystem - MovieSceneEventChannel ## Sequencer 组件相关概念