Hybrid texture synthesis approaches in improving the stability of the domain mappings and loss definitions in a GAN-based style transfer - ON-154

Preferred Disciplines: Computer Science, Computational Linguistics, Computer Engineering (PhD or Post-Doc)
Company: Crater Labs Inc.
Project Length: 8-12 months (2 units)
Desired start date: As soon as possible
Location: Toronto, ON
No. of Positions: 1
Preferences: University in the Greater Toronto Area or Southern Ontario.

About the Company: 

We are a Toronto-based studio specializing in the use of computer vision, predictive analytics and natural language processing to build intelligence into business applications.

Project Description:

Based on the hypothesis that current instabilities might arise from lack of correspondence between image domain (i.e. intensity-based features) and feature space (higher order features), we aim to augment the objective function to include hybrid texture synthesis. However, such augmentation is not trivial as we need to embed this within the model to ensure a non-zero gradient between iterations while preventing image noise from deriving the optimization process.

Research Objectives:

We propose to develop a scheme to examine the augmented loss function at multiple resolutions in an effort to maintain a balance between preserving hybrid texture synthesis incorporation and ensuring model convergence. If successful, this technique will provide for a scalable means of style transfer based on unannotated data.

Methodology:

  • Implement and train generative adversarial network model as per Gatys et al, Image Style Transfer Using Convolutional Neural Networks as baseline
  • Implement augmented loss functions as per Risser et al, Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
  • Conduct unsupervised training evaluation to determine if the hypothesis is a feasible approach to reduce instabilities caused by a lack of correspondence in the image  domain.

Expertise and Skills Needed:

    • Familiarity with neural networks (e.g. CNN, RNN) and generative adversarial networks.
    • Familiarity with deep learning libraries such as PyTorch, TensorFlow
    • Knowledge of Python, matplotlib, numpy, pandas and Jupyter Notebooks

    For more info or to apply to this applied research position, please

    1. Check your eligibility and find more information about open projects
    2. Interested students need to get the approval from their supervisor and send their CV along with a link to their supervisor’s university webpage by applying through the webform.
    Program: