Improve local eye tracking: rotation space, axis calibration, offline bone actuation#663
Open
kidkwazine wants to merge 3 commits intoBasisVR:developerfrom
Open
Improve local eye tracking: rotation space, axis calibration, offline bone actuation#663kidkwazine wants to merge 3 commits intoBasisVR:developerfrom
kidkwazine wants to merge 3 commits intoBasisVR:developerfrom
Conversation
- initial eye rotation capture from .rotation (world) to .localRotation (local) - Replace Quaternion.Euler axis assumption with BasisLocalEyeDriver's per-eye calibration - Expose calLeft/calRight for use by EyeTrackingBoneActuation - Align leftEyeInitialRotation type
Move leftEyeInitialRotation/rightEyeInitialRotation from separate static fields on BasisLocalEyeDriver into the EyeCalibration struct, captured during CalibrateOneEye()
Gate local eye tracking state management on `_eyeFollowDriverApplicable` (set on avatar ready) instead of `IsLocal` (set only after network ready). `BlendshapeActuation` already works offline because it gates on `_isWearer`. `EyeTrackingBoneActuation` used `IsLocal` for the equiv checks. (Network methods are unaffected)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BasisLocalEyeDriver.Initalize()captured initial eye rotations using.rotation(world-space) but they were being used for.localRotationwrites for eye tracking. This would bake the parent chain into a parent-relative value. I'm not fully sure how / if this manifested in practice, but I presume it wasn't too noticeable because the avatar is in T-pose when it's captured?SetEyeRotation()assumed a specific bone orientation. Hai flagged this in a comment, but was waiting on a WIP normalized muscle system. I integratedBasisLocalEyeDriver's existing per-eye calibration (since that already accounts for eye bone axis orientation during init) as a stopgap for the local path.EyeTrackingBoneActuationgated local eye tracking onIsLocalwhich is only set duringOnHVRReadyBothAvatarAndNetworkso bone actuation didn't work unless you were connected / in Host Mode. I switched it to_eyeFollowDriverApplicable(set onOnHVRAvatarReady) to match howBlendshapeActuationalready handles this with_isWearer.Changes
BasisLocalEyeDriver.cs:leftEyeInitialRotation/rightEyeInitialRotationinto theEyeCalibrationstructcalLeft/calRightaspublic staticfor use byEyeTrackingBoneActuation1EyeTrackingBoneActuation.cs:Quaternion.Euleron the local path with axis-aware rotation viaBasisLocalEyeDriver_eyeFollowDriverApplicableinstead ofIsLocalso it works without a network connectionI smoke tested with a few avis and an iPhone app that sends face tracking data since I don't have any fancy eye tracking HMDs, so may benefit from someone with eye trackers giving this a review pass!
Footnotes
I exposed
calLeft/calRightas barepublic staticfields to be consistent with how other fields are exposed there. I presume they're never meant to be set outside ofCalibrateEyes()though, so might be better to encapsulate them? Can change if you'd prefer that ↩