Multiple Source Adaptation and the Rényi Divergence

By Yishay Mansour, Mehryar Mohri, Afshin Rostamizadeh. 2009

156 citations recorded by the source corpus.

This paper presents a novel theoretical study of the general problem of multiple source adaptation using the notion of Renyi divergence. Our results build on our previous work [12], but significantly broaden the scope of that work in several directions. We extend previous multiple source loss guarantees based on distribution weighted combinations to arbitrary target distributions P, not necessarily mixtures of the source distributions, analyze both known and unknown target distribution cases, and prove a lower bound. We further extend our bounds to deal with the case where the learner receives an approximate distribution for each source instead of the exact one, and show that similar loss guarantees can be achieved depending on the divergence between the approximate and true distributions. We also analyze the case where the labeling functions of the source domains are somewhat different. Finally, we report the results of experiments with both an artificial data set and a sentiment analysis task, showing the performance benefits of the distribution weighted combinations and the quality of our bounds based on the Renyi divergence.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Information Theory, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Mingsheng Long
  3. Michael I. Jordan
  4. Qiang Yang
  5. Sergey Levine
  6. Percy Liang
  7. Dacheng Tao
  8. Trevor Darrell

In Representation & Self-Supervised Learning

  1. Mingsheng Long
  2. Mehryar Mohri
  3. Jianmin Wang
  4. Qiang Yang
  5. Michael I. Jordan
  6. Kate Saenko
  7. Judy Hoffman
  8. Corinna Cortes

In Information Theory

  1. Mehryar Mohri
  2. Ningshan Zhang
  3. Jayadev Acharya
  4. Himanshu Tyagi
  5. Judy Hoffman
  6. Ananda Theertha Suresh
  7. Jieping Ye
  8. Lei Zhang

In Signal Processing

  1. Mehryar Mohri
  2. Fei Sha
  3. Boqing Gong
  4. Bernhard Schölkopf
  5. Ningshan Zhang
  6. Judy Hoffman
  7. Mingsheng Long
  8. Shiliang Sun

Citations in the corpus