PhD Thesis: Propostas

Multimedia
Representation

Fernando Pereira
Março 2021
 TÓPICO: Deep Learning-based Multimedia Representation
Imagens e Realidade
vídeo virtual e
aumentada

Nuvens de Veículos
pontos 3D autónomos
TEMAS: Deep Learning-based Multimedia Representation
• Deep Learning-based Image Coding – Codificação de imagens usando técnicas de aprendizagem
profunda
• Deep Learning-based Video Coding – Codificação de vídeo usando técnicas de aprendizagem
profunda
• Deep Learning-based Point Cloud Geometry + Color Coding - Codificação de nuvens de pontos
(geometria e cor) usando técnicas de aprendizagem profunda
• Deep Learning-based Point Cloud Geometry + Color Scalable Coding - Codificação escalável de
nuvens de pontos usando técnicas de aprendizagem profunda
• Deep Learning-based Point Cloud Denoising – Remoção de ruído em nuvens de pontos usando
técnicas de aprendizagem profunda
• Deep Learning-based Point Cloud Super-resolution – Super-resolução (aumento da resolução) de
nuvens de pontos usando técnicas de aprendizagem profunda
SAÍDAS PROFISSIONAIS
Deep learning e multimedia representation são tecnologias com muitas saídas
profissionais, quer na academia, quer na indústria, nomeadamente:
• Operadoras de telecomunicações e televisão
• OTT providers, e.g. Netflix, HBO
• Produtoras de conteúdos
• Redes sociais, e.g. Facebook, Instagram
• Veículos autónomos, e.g. Tesla
• Jogos online
• Sistemas de informação geográfica
• Museus e entidades culturais
• …
DESCRIÇÕES: Deep Learning-based Multimedia
Representation (1)
• Deep Learning-based Image Coding
While some promising image coding performance has already been achieved using different types
of deep neural networks, such as autoencoders, this technology is still not mature. The main goal of
this Thesis would be to design, implement and assess an advanced deep learning-based image
coding solution, leveraging on the recent advances in deep neural networks and deep learning-
based coding, in general.
• Deep Learning-based Video Coding
Promising image coding performance has already been achieved using different types of deep
neural networks, such as autoencoders. Some first solutions have also emerged for deep learning-
based video coding but this field is still emerging. The main goal of this Thesis would be to design,
implement and assess a deep learning-based video coding solution leveraging on the recent
advances in deep neural networks and deep learning-based coding, in general.
DESCRIÇÕES: Deep Learning-based Multimedia
Representation (2)
• Deep Learning-based Point Cloud Geometry + Color Coding
Most deep learning-based point cloud coding solutions in the literature focus only on geometry
coding, likely because geometry coding is a more novel challenge, or simply because color is an
optional attribute for point clouds on top of geometry. However, point cloud color plays a
fundamental role in many applications, notably those targeting human visualization. In this context,
this Thesis shall design, implement and assess an efficient point cloud coding solution for both
geometry and color considering both a single color per 3D point as well as plenoptic color.
• Deep Learning-based Point Cloud Geometry + Color Scalable Coding
Scalability is a critical requirement for applications where the receivers may have very different
capabilities in terms of resolution, available bandwidth or computational power. In these cases, the
ability to access a lower resolution or quality PC is decisive, notably by partially decoding a
bitstream structured in multiple layers, even if at the cost of reduced compression efficiency. In this
context, this Thesis shall design, implement and assess an efficient scalable point cloud coding
solution, considering both geometry and color.
DESCRIÇÕES: Deep Learning-based Multimedia
Representation (3)
• Deep Learning-based Point Cloud Denoising
Since point cloud acquisition solutions are prone to noise, denoising is critical to improve the user
experience and the success of point cloud-based applications. In this context, this Thesis shall
design, implement and assess voxel-domain and compressed-domain point cloud denoising
techniques which may be applied independently to the geometry and color, or simultaneously to
the geometry and color.
• Deep Learning-based Point Cloud Super-resolution
Super-resolution techniques are essential to appropriately increase point cloud density for
rendering, targeting increasing the total number of points and quality of user experience. In this
context, this Thesis shall design, implement and assess voxel-domain and compressed-domain point
cloud super-resolution techniques which may be applied only to the geometry (if there is no color
data available) or simultaneously to the geometry and color.