Skip to content

This is an official PyTorch implementation of "Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation" (IROS 2022).

License

Notifications You must be signed in to change notification settings

pjyazdian/Gesture2Vec

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation

The Best Paper Award Winner in Cognitive Robotics at IROS2022

This is an official PyTorch implementation of Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation (IROS 2022). In this paper, we present an automatic gesture generation model that uses a vector-quantized variational autoencoder structure as well as training techniques to learn a rigorous representation of gesture sequences. We then translate input text into a discrete sequence of associated gesture chunks in the learned gesture space. Subjective and objective evaluations confirm the success of our approach in terms of appropriateness, human-likeness, and diversity. We also introduce new objective metrics using the quantized gesture representation.

OVERVIEW

Demo Video

Demo Video

Presentation

IROS2022 Presentation

Instructions

TODO

License

This code is distributed under an MIT LICENSE.

Note that our code uses datasets inluding Trinity and Talk With Hand (TWH) that each have their own respective licenses that must also be followed.

Please feel free to contact us ([email protected]) with any question or concerns.

About

This is an official PyTorch implementation of "Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation" (IROS 2022).

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published