Ilija Ilievski

Visual question answering as recently proposed multimodal learning task has enjoyed wide attention from the deep learning community. Lately, the focus was on developing new representation fusion methods and attention mechanisms to achieve superior performance. On the other hand, very little focus has been put on the models' loss function, arguably one of the most important aspects of training deep learning models. The prevailing practice is to use cross entropy loss function that penalizes the probability given to all the answers in the vocabulary except the single most common answer for the particular question. However, the VQA evaluation function compares the predicted answer with all the ground-truth answers for the given question and if there is a matching, a partial point is given. This causes a discrepancy between the model's cross entropy loss and the model's accuracy as calculated by the VQA evaluation function. In this work, we propose a novel loss, termed as soft cross entropy, that considers all ground-truth answers and thus reduces the loss -- accuracy discrepancy. The proposed loss leads to an improved training convergence of VQA models and an increase in accuracy as much as 1.6%.

Ilija Ilievski, and Jiashi Feng

CVPR Visual Question Answering  PDF

Recent studies show that LSTM-based neural optimizers are competitive with state-of-the-art hand-designed optimization methods for short horizons. Existing neural optimizers learn how to update the optimizee parameters, namely, predicting the product of learning rates and gradients directly and we suspect it is the reason why the training task becomes unnecessarily difficult. Instead, we train a neural optimizer to only control the learning rates of another optimizer using gradients of the training loss with respect to the learning rates. Furthermore, with the assumption that learning rates tend to remain unchanged over a certain number of iterations, the neural optimizer is only allowed to propose learning rates every S iterations where the learning rates are fixed during these S iterations and this enables it to generalize to longer horizons. The optimizee is trained by Adam on MNIST, and our neural optimizer learns to tune the learning rates for the Adam. After 5 meta-iterations, another optimizee trained by Adam whose learning rates are tuned by the learned but frozen neural optimizer, outperforms those trained by existing hand-designed and learned neural optimizers in terms of convergence rate and final accuracy for long horizons across several datasets.

Jie Fu, Ritchie Ng, Danlu Chen, Ilija Ilievski, Christopher Pal, Tat-Seng Chua

ICML AutoML  PDF

Capturing the semantic interaction of pairs of words across arguments and proper argument representation are both crucial issues in implicit discourse relation recognition. The current state-of-the-art represents arguments as distributional vectors that are computed via bi-directional Long Short-Term Memory networks (BiLSTMs), known to have significant model complexity. In contrast, we demonstrate that word-weighted averaging can encode argument representation which can be incorporated with word pair information efficiently. By saving an order of magnitude in parameters and eschewing the recurrent structure, our proposed model achieves equivalent performance, but trains seven times faster.

Wenqiang Lei, Xuancong Wang, Meichun Liu, Ilija Ilievski, Xiangnan He , Min-Yen Kan

IJCAI PDF

Visual question answering (VQA) is arguably one of the most challenging multimodal understanding problems as it requires reasoning and deep understanding of the image, the question, and their semantic relationship. Existing VQA methods heavily rely on attention mechanisms to semantically relate the question words with the image contents for answering the related questions. However, most of the attention models are simplified as a linear transformation, over the multimodal representation, which we argue is insufficient for capturing the complex nature of the multimodal data. In this paper we propose a novel generative attention model obtained by adversarial self-learning. The proposed adversarial attention produces more diverse visual attention maps and it is able to generalize the attention better to new questions. The experiments show the proposed adversarial attention leads to a state-of-the-art VQA model on the two VQA benchmark datasets, VQA v1.0 and v2.0.

Ilija Ilievski, and Jiashi Feng

ACM Multimedia Understanding  PDF

Automatically searching for optimal hyperparameter configurations is of crucial importance for applying deep learning algorithms in practice. Recently, Bayesian optimization has been proposed for optimizing hyperparameters of various machine learning algorithms. Those methods adopt probabilistic surrogate models like Gaussian processes to approximate and minimize the validation error function of hyperparameter values. However, probabilistic surrogates require accurate estimates of sufficient statistics (e.g., covariance) of the error distribution and thus need many function evaluations with a sizeable number of hyperparameters. This makes them inefficient for optimizing hyperparameters of deep learning algorithms, which are highly expensive to evaluate. In this work, we propose a new deterministic and efficient hyperparameter optimization method that employs radial basis functions as error surrogates. The proposed mixed integer algorithm, called HORD, searches the surrogate for the most promising hyperparameter values through dynamic coordinate search and requires many fewer function evaluations. HORD does well in low dimensions but it is exceptionally better in higher dimensions. Extensive evaluations on MNIST and CIFAR-10 for four deep neural networks demonstrate HORD significantly outperforms the well-established Bayesian optimization methods such as GP, SMAC, and TPE. For instance, on average, HORD is more than 6 times faster than GP-EI in obtaining the best configuration of 19 hyperparameters.

Ilija Ilievski, Taimoor Akhtar, Jiashi Feng, and Christine Annette Shoemaker

AAAI-17 PDF Supplement Poster Code BibTeX

Recently, several optimization methods have been successfully applied to the hyperparameter optimization of deep neural networks (DNNs). The methods work by modeling the joint distribution of hyperparameter values and corresponding error. Those methods become less practical when applied to modern DNNs whose training may take a few days and thus one cannot collect sufficient observations to accurately model the distribution. To address this challenging issue, we propose a method that learns to transfer optimal hyperparameter values for a small source dataset to hyperparameter values with comparable performance on a dataset of interest. As opposed to existing transfer learning methods, our proposed method does not use hand-designed features. Instead, it uses surrogates to model the hyperparameter-error distributions of the two datasets and trains a neural network to learn the transfer function. Extensive experiments on three CV benchmark datasets clearly demonstrate the efficiency of our method.

Ilija Ilievski and Jiashi Feng

PDF

Visual Question and Answering (VQA) problems are attracting increasing interest from multiple research disciplines. Solving VQA problems require techniques from both computer vision for understanding the visual contents of a presented image or video, as well as the ones from natural language processing for understanding the semantics of the question and generating the answers. Regarding visual content modeling, most of existing VQA methods adopt the strategy of extracting global features from the image or video, which inevitably fails in capturing fine-grained information such as the spatial configuration of multiple objects. Extracting features from auto-generated regions — as some region-based image recognition methods do — cannot essentially address this problem and may introduce some overwhelming irrelevant features with the question. In this work, we propose a novel Focused Dynamic Attention (FDA) model to provide better-aligned image content representation with proposed questions. Being aware of the key words in the question, FDA employs off-the-shelf object detector to identify important regions and fuse the information from the regions and global features via an LSTM unit. Such question-driven representations are then combined with question representation and fed into a reasoning unit for generating the answers. Extensive evaluation on a large-scale benchmark dataset, VQA, clearly demonstrates the superior performance of FDA over well-established baselines.

Ilija Ilievski, Shuicheng Yan and Jiashi Feng

PDF       BibTeX

This master thesis presents a tool that employs intelligent technologies to capture the patterns of urban change driven by a diverse set of context factors. Data mining provides opportunities, which complement and extend the knowledge previously obtained with other approaches. A series of exploratory case studies were done to showcase the modeling and predictive capabilities of the tool. A number of simulations have revealed distinctive local patterns of urban change in the city of Skopje, shaped by local urban spatial and institutional structures. This thesis shows the importance of intelligent technologies in the interpretation of the historical evidence of urban development.

Ilija Ilievski, Master Thesis, Ss. Cyril & Methodius University, Macedonia.

PDF (in Macedonian)

This paper presents a personalized news recommendation system that combines effective ways of understanding new articles with novel ways of modeling evolving user interest profiles to deliver relevant news articles to a user. A news article is represented as a taxonomy of hierarchical abstractions that capture different semantic facets of the news story. A user's interest profile is modeled as an evolving interest over these facets. User's interest in individual articles is determined using a novel SWL (select-watch-leave) interest modeling framework that leverages on a detailed analysis of his usage history. Initial performance comparisons with state-of-the-art personalized ranking approaches are promising.

Ilija Ilievski and Sujoy Roy, ACM RecSys.

PDF       BibTeX

The goal of this research is to develop a tool that employs intelligent technologies to capture the patterns of urban change driven by a diverse set of context factors. Data mining provides opportunities, which complement and extend the knowledge previously obtained with other approaches. A series of exploratory case studies are in place to investigate the modeling and predictive capabilities of the tool. A number of simulations have revealed distinctive local patterns of urban change in the city of Skopje, shaped by local urban spatial and institutional structures. This study shows the importance of intelligent technologies in the interpretation of the historical evidence of urban development.

Ilija Ilievski, Sonja Gievska and Ognen Marina, ICT Innovations.

PDF       BibTex

This research attempts to test a series of hypotheses on how the mechanisms of social and economic stratification have manifested in urban space and whether population dynamics has reconfigured the spatiality of the city landscape. This paper reflects upon the challenges surrounding the efforts in recognizing and interpreting the patterns of urban change. Our efforts are directed towards correlating real-word emergent patterns of urban change to contextual knowledge and incorporating them into model's predicting capabilities. Suitability of an extensive set of machine learning algorithms for simulation and prediction of urban development is investigated. Our long-term goal has been to lay out a foundation in terms of a knowledge base and a tool that will accommodate future exploration of different research scenarios related to urban dynamics. The specific goal of this research is to develop a tool that employs intelligent technologies to capture the patterns of urban change driven by a diverse set of context factors. Data mining provides opportunities, which complement and extend the knowledge previously obtained with other approaches.

Sonja Gievska, Ognen Marina and Ilija Ilievski, 9th Congress "Virtual City and Territory".

PDF       BibTex