Funded by the European Commission Horizon 2020 programme with grant nr. 762021, the ultimate project’s objective is the development of an automated photorealistic 3D human avatar for live sign language interpretation in a specific broadcasting domain i.e. Weather Forecast, through the employment of novel technologies and AI algorithms. Fincons Group plays the key role of project coordinator for the consortium of companies, universities and research centers working on the project, which include the University of Surrey (UK), the Fraunhofer Heinrich Hertz Institute – HHI (DE), Human Factors Consult (DE), SwissTXT (CH) and Vlaamse Radio en Televisieomroep – VRT (BE). Fincons, leveraging its experience working with major European broadcasters, is also the main ICT integrator partner.
CONTENT4ALL provides broadcasters with an affordable and sustainable solution to increase accessibility and services to the deaf community. By reducing production costs broadcasters are thus enabled to make more content available in sign language, reaching a bigger audience and making viewing more inclusive as well as engaging. The solution also helps meet increasing regulatory such as compliance with Art.30, 21 of the UN Convention on the Rights of Persons with Disabilities.
This 3D photorealistic avatar, named in the project “realatar”, reproduces in real-time a sign language interpreter as a realistic virtual human. It is achieved by enabling an innovative remote studio where real person movements are captured (body, hands an even facial expressions) and the processed thanks to advanced Deep Learning and Machine Learning algorithms.
The realatar can then be distributed via a video stream that is fully compatible with latest Next Gen TV innovation thank to the most advanced standards HbbTV 2.0.1 and ATSC 3.0.
Specifically, some of the project’s key objectives are:
- Remote live realatar animation to provide live signing with a 3D virtual human
- Develop technologies and algorithms to explore automatic sign generation capabilities
- Extract sign language interpretation parameters, real-time development of 3D models and animation.
- Evaluate of trial performance with the end users local associations e.g. Swiss Federation of the Deaf (Switzerland) and Doof Vlaanderen and affiliated subcontractors (Flanders/Belgium).
- Development of sustainable business models
Gartner’s 2018 "Market Insight: Three Key Areas in Which AI Will Disrupt the Video Entertainment Ecosystem” report describes CONTENT4ALL as one of the AI based projects that is most likely to radically revolutionise the creative video industry and CONTENT4ALL reception at IBC 2019 Future Zone and NAB 2020 Futures Park is confirming this role.
CONTENT4ALL leverages latest AI technologies such as:
- Facial expressions reproduction via RGB video and Facial parameters
- Body movements reproduction as skeleton via depth sensors and AI algorithms
- Gestures reproduction via hands poses interpolations and fingers modelling thanks to advanced Deep Learning algorithms