On May 04th 2022 at 15:30, I graduated with my PhD at the University of Amsterdam. My thesis is titled Information Sharing Methods for Multi-Task Learning and contains a bunch of papers from my research from 2017 until 2022. Yay!
You can watch my defence in the video below:
Your browser does not support the video tag. or if you enjoy light reading, you can download my thesis here.
Thesis Abstract This thesis investigates information sharing for Multi-Task Learning (MTL) in the multimedia and computer vision domains.
We present TindART - a comprehensive visual arts recommender system. TindART leverages real time user input to build a usercentric preference model based on content and demographic features. Our system is coupled with visual analytics controls that allow users to gain a deeper understanding of their art taste and further refine their personal recommendation model. The content based features in TindART are extracted using a multi-task learning deep neural network which accounts for a link between multiple descriptive attributes and the content they represent.
Our work on generative modelling of artistically painted eyes got accepted at MMM 2020 in Daejon, South Korea.
Faces in artistic paintings most often contain the same elements (eyes, nose, mouth…) as faces in the real world, however they are not a photo-realistic transfer of physical visual content. These creative nuances the artists introduce in their work act as interference when facial detection models are used in the artistic domain. In this work we introduce models that can accurately detect, classify and conditionally generate artistically painted eyes in portrait paintings.
Color is everywhere, and whether we perceive it consciously or not each color we encounter provides an emotional experience. Our work for exploring and navigating emotion in art through color named ACE: Art, Color & Emotion has a technical demo slot at ACM Multimedia 2019 in Nice, France. We are happy to demonstrate the functionalities and invite you to explore OmniArt together through ACE.
ACE: Art, Color and Emotion
Gjorgji Strezoski, Arumoy Shome, Riccardo Bianchi, Shruti Rao, Marcel Worring
I am excited to announce that our work on routing data flows per task in Multi-Task Learning models in order to improve the task count scalability is accepted for an Oral presentation at ICCV 2019 in Seoul, South Korea. This paper introduces Task Routing our method for routing data within convolutional neural networks and the PyTorch layer enabling this functionality.
As with any combinatorial problem, in MTL there exists an optimal combination of tasks and shared resources which is unknown.
This week I will talk about my research at the 16th International Conference on Informatics and Information Technologies. We will cover computer vision and multi-task learning fundamentals as well as state of the art approaches in the these fields. You can find more details about the talk and complete conference programme on the CIIT [ website ] (http://ciit.finki.ukim.mk/).
Our work on exploting secondary latent features for task grouping got accepted for oral presentation in ICMR 2019 in Ottawa, Canada. This paper introduces Selective Sharing, a method using the factorized gradients per task as a signal that helps in grouping tasks that benefit eachother’s learning process. The grouping is conditioned on a predefined metric so different strategies can be explored. We are preparing the repo for the code release and the site will be updated with a link to the official proceedings.
Over the past two years my research interests have revolved arround Multi-Task Learning (MTL) as a learning paradigm. It is a vast field of diverse research in all domains of computer science from NLP and Signal Processing to Computer Vision and Multimedia. In what follows I will motivate, describe and discuss an approach to MTL we developed called Task Routing.
Multi-Task and Many-Task Learning By definition (Carruana 1997), multi-task learning is a learning paradigm that seeks to improve the generalization performance of machine learning models by optimizing for more than one task simultaneously.
Our favorite artistic dataset is published in ACM TOMM V.14 Issue 4, November 2018.
Baselines are the starting point of any quantitative multimedia research, and benchmarks are essential for pushing those baselines further. In this article, we present baselines for the artistic domain with a new benchmark dataset featuring over 2 million images with rich structured metadata dubbed OmniArt. OmniArt contains annotations for dozens of attribute types and features semantic context information through concepts, IconClass labels, color information, and (limited) object-level bounding boxes.
The VISTORY Project on the cover of I/O Magazine in a featured artcile The Science of Art.
Behind the façade of the majestic Ateliergebouw in Amsterdam you can find a research institute that is unique in the world. At this Netherlands Institute for Conservation+Art+Science+ (NICAS), art historians, conservators, physicists, chemists, mathematicians and ICT researchers work together to better understand, access and preserve cultural heritage.
Check out the full article on I/O Magazine’s website or order a printed copy.