PhD Researcher in Learning-based control algorithms for avatar animation in dyadic interactions

Last application date Dec 31, 2024 00:00

Department TW05 - Department of Information Technology

Contract Limited duration

Degree European master's degree in computer science, Artificial Intelligence, or equivalent

Occupancy rate 100%

Vacancy type Research staff

Job description

PhD Researcher in Learning-based control algorithms for avatar animation in dyadic interactions

Job description
Ghent University is a world of its own. Employing more than 15.000 people, it is actively involved in education and research, management and administration, as well as technical and social service provision on a daily basis. It is one of the largest, most exciting employers in the area and offers great career opportunities.
With its 11 faculties and more than 85 departments offering state-of-the-art study programmes grounded in research in a wide range of academic fields, Ghent University is a logical choice for its staff and students.

The IDLab Ghent research group is seeking a highly motivated and talented PhD student to join the distributed machine learning team. Our team focuses on developing efficient machine learning algorithms for perception and control; with diverse applications in industry, robotics, remote sensing and agriculture.

Topic
With the advent of novel VR/XR devices, we envision an increasing number of social interactions with AI-animated avatar. Compared to the limitations of hardware humanoid robots, the bodily and facial expressions of contemporary avatars can be rendered with much more detail, making them a fertile platform for research on human-AI interaction. Meaningful interaction with avatars requires autonomous generation of verbal as well as non-verbal communication. Non-verbal communication involves not only co-gestures that support speech, but also the emission of feedback signals while listening. Whereas Large Language Models provide a mature technology to autonomously generate verbal responses, research on AI-driven generation of non-verbal communication cues is still in its infancy, especially in situations where the avatar is actively listening or where it is providing non-verbal feedback on the performance of the conversation partner.

In this PhD, we want to investigate learning-based control algorithms for animation of non-verbal avatar behavior during a dyadic interaction, in particular a teaching scenario. The control algorithm will be designed in the framework of active inference, an emerging computational principle stating that embodied behavior is the result from variational minimization of free energy.

The research will be contextualized in an industry relevant use case scenario. As an illustrative example, one could consider an avatar is teaching an assembly operation to an operator. Social studies have indicated how the social signals exchanged are indicative for tracking the learning progress of the student. The exact use case will be determined at the start of the project.

Role and responsibilities
This is a PhD project that will be executed in close cooperation with researchers from imec-mict-UGent, a research group focused on unraveling human interactions through technology. In this project, imec-mict-UGent provides expertise in VR environment design, cognitive and experimental psychology, as well as data-driven metrics for communication quality. At a larger scale, this project is situated within the context of the Flanders AI Research Program, allowing the candidate to interact with fellow researchers throughout Flanders.

Your main tasks include:
• Reviewing literature on control frameworks in the domain of reinforcement learning, active inference and others
• Researching and prototyping control algorithms for generating non-verbal communication, for instance feedback signals
• Researching and prototyping algorithms for knowledge tracking and belief modelling, components needed for the teaching scenario
• Set-up and conduct user experiments
• Co-developing with the MICT research group an experimental VR environment for assembly instructions
• Writing high quality publications, targeting top journals and international conferences.

In addition to your primary research responsibilities, you will actively contribute to the educational mission of our institution by providing support for various courses in areas such as machine learning, reinforcement learning and algorithm design. In addition, you can take on a mentoring role by supervising bachelor and master theses related to the subject of this PhD.

Job profile

Job profile

We are looking for a highly creative and motivated PhD student with the following qualifications and skills:
• You have (or will obtain in the next months) a European master's degree in computer science, Artificial Intelligence, or equivalent, with excellent ('honors'-level or above) grades.
• You have a strong background in deep learning.
• You are willing to work in a multidisciplinary context. In particular, you are willing to acquaint yourself with concepts of communication theory to the extent needed and are willing to set-up and execute user experiments to collect necessary data.
• Previous experience with active inference, reinforcement learning or other ML-based techniques for control is considered a plus but is not a necessity
• Previous experience with VR/AR is considered a plus but is not a necessity
• You have excellent computer science skills (python, git, linux, etc.) .
• You have hands-on experience with machine learning frameworks such as PyTorch.
• You have strong analytical skills to interpret the obtained research results.
• You are a team player and have strong communication skills.
• Your English is fluent (C1 CEFR level), both speaking and writing.

Our offer

We offer the opportunity to do this research in an international and stimulating environment. The research will be conducted at the premises of IDLab, located in Ghent. However, there will be frequent interactions with researchers of MICT, another research group in Ghent.

Ghent University consistently ranks among the best 100 universities in the world. Located in the heart of Europe, Ghent is a beautiful and welcoming city (https://www.lonelyplanet.com/articles/ghent-belgiums-best-kept-secret) with plenty of cultural and leisure activities.

The selected candidate will be offered a full-time position as a doctoral fellow, consisting of an initial period of 12 months, which - after a positive evaluation, will be extended to a total maximum of 48 months.
The fellowship amount is 100% of the net salary of an AAP member in equal family circumstances. The individual fellowship amount is determined by the Department of Personnel and Organization based on family status and seniority. A grant that meets the conditions and criteria of the regulations for doctoral fellowships is considered free of personal income tax. Click here for more information about our salary scales

All Ghent University staff members enjoy a number of benefits, such as a wide range of training and education opportunities, 36 days of holiday leave (on an annual basis for a full-time job) supplemented by annual fixed bridge days, bicycle allowance and eco vouchers. Click here for a complete overview of all the staff benefits (in Dutch).

How to apply

How to apply
Send your application by email to Prof. Pieter Simoens (pieter.simoens@ugent.be). Applications should include:
• An academic/professional resume
• Transcripts of study results
• A short overview describing your earlier research or technical work (e.g., scientific papers, link to GitHub repository, master thesis, report on project work, etc.). These documents need not be on the topic of the advertised position.

After a first screening, selected candidates will be invited for an interview (also possible via Teams). The selection process will involve multiple steps.

Application deadline: continuous evaluation until the vacancy is filled.