Update

Added parameters to the preferences for adjusting the horizontal resolution and the smoothness of textures.

Max Width Of Waveform Texture On Timeline

Texture Smoothness On Timeline

Update

Added a new preference option to enable or disable waveform display on the timeline. This setting was introduced due to reports of performance degradation when waveforms are displayed.

Screenshots

Update

• set maximum value for BlendShapes #63

Bug Fix

• fix mouth remaining open when audio is not playing on WebGL #57

Update

• WebGL Support
  • Works out-of-the-box when building
  • Microphone support is not yet available

Bug Fix

• #43 @liudger
• #48 @mkc1370
• #49 @mkc1370
• #56 @mkc1370

Bug Fixes

• Fixed an issue causing an error when the VRM package is not included
• Fixed errors occurring when using .NET Standard 2.0

Updates

• Added samples for VRM 0.X (The 04. VRM includes samples for both VRM 1.0 and VRM 0.X)

Screenshot

Transition from v2 to v3

Starting from version 3.0.0, the MFCC values have been corrected to be more accurate. Therefore, if you are upgrading from version 2 to version 3, you will need to recalibrate or create a new Profile.

Update

• Improved calculation method for MFCC (to get values closer to commonly used values).
  • Article: https://tips.hecomi.com/entry/2023/03/31/022324 (written in Japanese)
• Added multiple comparison methods for MFCC in Profile (L1 / L2 / CosineSimilarity and Use Standardization).
• Added VRM 1.0 resources to samples.
  • Created using VRoid
• Added a sample that displays UI at runtime to allow editing and calibration of profiles.
• Improved editor performance (by creating MFCC textures with Job + Burst).

Bug Fix

• Fix microphone desynchronization issues #30
• The first parameter set in uLipSyncAnimator does not work. #37

Screen Shot

Comparison Method

Previously, L1 without standardization was used.

VRM 1.0

Runtime UI

Bug Fix

• fix Timeline Setup Helper to take clipIn and timescale into account

Update

• Add 10. Runtime Setup example
• Add Timeline Setup Helper
• Add an API to get phoneme names Profile.GetPhonemeNames()

Timeline Setup Helper

Window > uLipSync > Timeline Setup Helper

This tool automatically creates BakedData corresponding to clips registered in AudioTrack and registers them in uLipSync Track.

Runtime Setup

1. Attach uLipSyncBlendShape
2. Register SkinneMeshRenderer there
3. Bind the phoneme name registered in Profile and the corresponding SkinnedMeshRenderer BlendShape with AddBlendShape(string phoneme, string blendShape)
4. Attach uLipSync
5. Register Profile
6. Register the callback for uLipSyncBlendShape there

using UnityEngine;
using System.Collections.Generic;
public class uLipSyncBlendShapeRuntimeSetupExample : MonoBehaviour
{
    // GameObject to add uLipSync (that has an AudioSource component)
    public GameObject target;
    // Profile to bind
    public uLipSync.Profile profile;
    // SkinnedMeshRenderer containing BlendShapes
    public string skinnedMeshRendererName = "MTH_DEF";
    // Assuming that some arbitrary data exists
    [System.Serializable]
    public class PhonemeBlendShapeInfo
    {
        public string phoneme;
        public string blendShape;
    }
    public List<PhonemeBlendShapeInfo> phonemeBlendShapeTable 
        = new List<PhonemeBlendShapeInfo>();
    uLipSync.uLipSync _lipsync;
    uLipSync.uLipSyncBlendShape _blendShape;
    void Start()
    {
        if (!target) return;
        SetupBlendShpae();
        SetupLipSync();
    }
    void SetupBlendShpae()
    {
        // Search for GameObject with specified name
        var targetTform = uLipSync.Util.FindChildRecursively(
            target.transform, 
            skinnedMeshRendererName);
        if (!targetTform) 
        {
            Debug.LogWarning(
                $"There is no GameObject named \"{skinnedMeshRendererName}\"");
            return;
        }
        // Get SkinnedMeshRenderer
        var smr = targetTform.GetComponent<SkinnedMeshRenderer>();
        if (!smr) 
        {
            Debug.LogWarning(
                $"\"{skinnedMeshRendererName}\" does not have SkinnedMeshRenderer.");
            return;
        }
        // Attach uLipSyncBlendShape
        _blendShape = target.AddComponent<uLipSync.uLipSyncBlendShape>();
        _blendShape.skinnedMeshRenderer = smr;
        // Bind Phoneme and BlendShape name using some data
        foreach (var info in phonemeBlendShapeTable)
        {
            _blendShape.AddBlendShape(info.phoneme, info.blendShape);
        }
    }
    void SetupLipSync()
    {
        if (!_blendShape) return;
        // Attach uLipSync
        _lipsync = target.AddComponent<uLipSync.uLipSync>();
        // Specify Profile
        _lipsync.profile = profile;
        // Register callback for uLipSyncBlendShape
        _lipsync.onLipSyncUpdate.AddListener(_blendShape.OnLipSyncUpdate);
    }
}

Update

• Add uLipSyncAnimator #24 (@mkc1370)
• Add uLipSyncExpressionVRM vrm expression sample #27 (@liudger)

Article

https://tips.hecomi.com/entry/2022/12/28/233803

Update

• Added uLipSyncTexture

Article

• https://tips.hecomi.com/entry/2022/08/31/232553