pixi-live2d-display

English | 中文

Live2D integration for PixiJS v7.

This project aims to be a universal Live2D framework on the web platform. While the official Live2D framework is just complex and problematic, this project has rewritten it to unify and simplify the APIs, which allows you to control the Live2D models on a high level without the need to learn how the internal system works.

Feel free to support the Maintainer:

Features

Supports all versions of Live2D models
Supports PIXI.RenderTexture and PIXI.Filter
Pixi-style transform APIs: position, scale, rotation, skew, anchor
Automatic interactions: focusing, hit-testing
Enhanced motion reserving logic compared to the official framework
Loading from uploaded files / zip files (experimental)
Fully typed - we all love types!
Live Lipsync

Requirements

PixiJS: 7.x
Cubism core: 2.1 or 4
Browser: WebGL, ES6

Demos

Basic demo
Interaction demo
Render texture & filter demo
Live2D Viewer Online

Documentations

Documentation
API Documentation
Development Notes

Cubism

Cubism is the name of Live2D SDK. There are so far three versions of it: Cubism 2.1, Cubism 3 and Cubism 4; where Cubism 4 is backward-compatible with Cubism 3 models.

This plugin supports all variants of Live2D models by using Cubism 2.1 and Cubism 4.

Cubism Core

Before using the plugin, you'll need to include the Cubism runtime library, aka Cubism Core.

For Cubism 4, you need live2dcubismcore.min.js that can be extracted from the Cubism 4 SDK, or be referred by a direct link (however the direct link is quite unreliable, don't use it in production!).

For Cubism 2.1, you need live2d.min.js. It's no longer downloadable from the official site since 2019/9/4, but can be found here, and with a CDN link that you'll probably need.

Individual Bundles

The plugin provides individual bundles for each Cubism version to reduce your app's size when you just want to use one of the versions.

Specifically, there are cubism2.js and cubism4.js for respective runtime, along with an index.js that includes both of them.

Note that if you want both the Cubism 2.1 and Cubism 4 support, use index.js, but not the combination of cubism2.js and cubism4.js.

To make it clear, here's how you would use these files:

Use cubism2.js+live2d.min.js to support Cubism 2.1 models
Use cubism4.js+live2dcubismcore.min.js to support Cubism 3 and Cubism 4 models
Use index.js+live2d.min.js+live2dcubismcore.min.js to support all versions of models

Installation

Via npm

npm install pixi-live2d-display-lipsyncpatch

import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch';
// or import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch'; // i didn't test this

// if only Cubism 2.1
import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch/cubism2';
// or import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch/cubism2';

// if only Cubism 4
import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch/cubism4';
// or import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch/cubism4';

Via CDN (lipsync patched)

<!-- Load Cubism and PixiJS -->
<script src="https://cubism.live2d.com/sdk-web/cubismcore/live2dcubismcore.min.js"></script>
<script src="https://cdn.jsdelivr.net/gh/dylanNew/live2d/webgl/Live2D/lib/live2d.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/pixi.js@7.x/dist/pixi.min.js"></script>


<!-- if support for both Cubism 2.1 and 4 -->
<script src="https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v0.5.0-ls-7/dist/index.min.js"></script>

<!-- if only Cubism 2.1 support-->
<script src="https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v0.5.0-ls-7/dist/cubism2.min.js"></script>

<!-- if only Cubism 4 support-->
<script src="https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v0.5.0-ls-7/dist/cubism4.min.js"></script>

In this way, all the exported members are available under PIXI.live2d namespace, such as PIXI.live2d.Live2DModel.

Basic usage

On demand import (Recommend)

import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch';
import { Application, Ticker } from 'pixi.js';

(async function () {
  const app = Application({
    view: document.getElementById('canvas'),
  });

  const model = await Live2DModel.from('shizuku.model.json', {
    // register Ticker for model
    ticker: Ticker.shared,
  });

  app.stage.addChild(model);

  // transforms
  model.x = 100;
  model.y = 100;
  model.rotation = Math.PI;
  model.skew.x = Math.PI;
  model.scale.set(2, 2);
  model.anchor.set(0.5, 0.5);

  // interaction
  model.on('hit', (hitAreas) => {
    if (hitAreas.includes('body')) {
      model.motion('tap_body');
    }
  });
})();

Full import PIXI

import * as PIXI from 'pixi.js';
import { Live2DModel } from 'pixi-live2d-display-lipsyncpatch';

// expose PIXI to window so that this plugin is able to
// reference window.PIXI.Ticker to automatically update Live2D models
window.PIXI = PIXI;

(async function () {
    const app = new PIXI.Application({
        view: document.getElementById('canvas'),
    });

    const model = await Live2DModel.from('shizuku.model.json');

    app.stage.addChild(model);

    // transforms
    model.x = 100;
    model.y = 100;
    model.rotation = Math.PI;
    model.skew.x = Math.PI;
    model.scale.set(2, 2);
    model.anchor.set(0.5, 0.5);

    // interaction
    model.on('hit', (hitAreas) => {
        if (hitAreas.includes('body')) {
            model.motion('tap_body');
        }
    });
})();

Do some motion manually

First either you need to load your model on Live2d viewer app, or the Website by guansss here
Check for motion category names (like "idle", "" (blank) etc)
- Screenshot will be added soon
Under those motion categories, each motions are used by their index
Motion Priority table:
- 0: no priority
- 1: maybe [for idle animation]
- 2: normal [default when normal action]
- 3: Just do it! Do id now! [Forced] [default when using audio]
Time to code

const category_name = "Idle" // name of the morion category
const animation_index = 0 // index of animation under that motion category [null => random]
const priority_number = 3 // if you want to keep the current animation going or move to new animation by force [0: no priority, 1: idle, 2: normal, 3: forced]
const audio_link = "https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v1.0.3/playground/test.mp3" //[Optional arg, can be null or empty] [relative or full url path] [mp3 or wav file]
const volume = 1; //[Optional arg, can be null or empty] [0.0 - 1.0]
const expression = 4; //[Optional arg, can be null or empty] [index|name of expression]
const resetExpression = true; //[Optional arg, can be null or empty] [true|false] [default: true] [if true, expression will be reset to default after animation is over]

model.motion(category_name, animation_index, priority_number, {
  sound: audio_link,
  volume: volume,
  expression: expression,
  resetExpression: resetExpression
});

// if you dont want voice, just ignore the option
model.motion(category_name, animation_index, priority_number);
model.motion(category_name, animation_index, priority_number, { expression: expression, resetExpression: resetExpression });
model.motion(category_name, animation_index, priority_number, { expression: expression, resetExpression: false });

Lipsync Only

You can do sound lipsync even without triggering any motion
This supports expressions arg too (if you have/need any)
Demo code

const audio_link = "https://cdn.jsdelivr.net/gh/RaSan147/pixi-live2d-display@v1.0.3/playground/test.mp3" // [relative or full url path] [mp3 or wav file]
const volume = 1; // [Optional arg, can be null or empty] [0.0 - 1.0]
const expression = 4; // [Optional arg, can be null or empty] [index|name of expression]
const resetExpression = true; // [Optional arg, can be null or empty] [true|false] [default: true] [if true, expression will be reset to default after animation is over]
const crossOrigin = "anonymous"; // [Optional arg, to use not same-origin audios] [DEFAULT: null]

model.speak(audio_link, {
  volume: volume,
  expression: expression,
  resetExpression: resetExpression,
  crossOrigin: crossOrigin
});

// Or if you want to keep some things default
model.speak(audio_link);
model.speak(audio_link, { volume: volume });
model.speak(audio_link, { expression: expression, resetExpression: resetExpression });

Fix "MediaElementAudioSource outputs zeroes due to CORS access restrictions for"

Both functions have crossOrigin param. Setting that to crossOrigin : "anonymous" will fix it.

model.speak(audio_link, { expression: expression, crossOrigin: "anonymous" });

model.motion(category_name, animation_index, priority_number, { sound: audio_link, volume: volume, crossOrigin: "anonymous" });

Suddenly stop audio and lipsync

Demo code

model.stopSpeaking();

Reset motions as well as audio and lipsync

Demo code

model.stopMotions();

Use callback function after a voiceline is over

Demo code

model.speak(audio_link, {
  volume: volume, 
  onFinish: () => { console.log("Voiceline is over") },
  onError: (err) => { console.log("Error: ", err) } // [if any error occurs]
});

model.motion(category_name, animation_index, priority_number, {
  sound: audio_link,
  volume: volume,
  expression: expression,
  onFinish: () => { console.log("Voiceline and Animation is over") },
  onError: (err) => { console.log("Error: ", err) } // [if any error occurs]
});

Totally destroy the model

This will also stop the motion and audio from running and hide the model
Demo code

model.destroy();

Result

20230408191713_1.mp4

See here for more Documentation: Documentation

Package importing

When importing Pixi packages on-demand, you may need to manually register some plugins to enable optional features.

import { Application } from '@pixi/app';
import { Ticker, TickerPlugin } from '@pixi/ticker';
import { InteractionManager } from '@pixi/interaction';
import { Live2DModel } from 'pixi-live2d-display';

// register Ticker for Live2DModel
Live2DModel.registerTicker(Ticker);

// register Ticker for Application
Application.registerPlugin(TickerPlugin);

// register InteractionManager to make Live2D models interactive
Renderer.registerPlugin('interaction', InteractionManager);

(async function () {
  const app = new Application({
    view: document.getElementById('canvas'),
  });

  const model = await Live2DModel.from('shizuku.model.json');

  app.stage.addChild(model);
})();

The example Live2D models, Shizuku (Cubism 2.1) and Haru (Cubism 4), are redistributed under Live2D's Free Material License.

Extra credit:

fatalgoth for making lipsync possible.
windjackz for creating all the cool features

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

pixi-live2d-display

Feel free to support the Maintainer:

Features

Requirements

Demos

Documentations

Cubism

Cubism Core

Individual Bundles

Installation

Via npm

Via CDN (lipsync patched)

Basic usage

On demand import (Recommend)

Full import PIXI

Do some motion manually

Lipsync Only

Fix "MediaElementAudioSource outputs zeroes due to CORS access restrictions for"

Suddenly stop audio and lipsync

Reset motions as well as audio and lipsync

Use callback function after a voiceline is over

Totally destroy the model

Result

See here for more Documentation: Documentation

Package importing

Extra credit:

Files

README.md

Latest commit

History

README.md

File metadata and controls

pixi-live2d-display

Feel free to support the Maintainer:

Features

Requirements

Demos

Documentations

Cubism

Cubism Core

Individual Bundles

Installation

Via npm

Via CDN (lipsync patched)

Basic usage

On demand import (Recommend)

Full import PIXI

Do some motion manually

Lipsync Only

Fix "MediaElementAudioSource outputs zeroes due to CORS access restrictions for"

Suddenly stop audio and lipsync

Reset motions as well as audio and lipsync

Use callback function after a voiceline is over

Totally destroy the model

Result

See here for more Documentation: Documentation

Package importing

Extra credit: