For a long time, I wanted to write a summary of the conferences I attend — not just to share new ideas and trends, but to keep track of thoughts during the event. Writing has been one of my weaknesses. I am too lazy to write things down, and because writing concretizes an idea, it loses some of its charm.
EMNLP 2018 in Brussels was a huge conference — close to 2,500 attendees and around 550 papers, making it the largest NLP conference ever at that time. It was by far one of the best conferences I attended.
Broad Trends
- Almost all papers reported that ELMo (Deep Contextualized Word Representations, best paper at NAACL) performs substantially better than GloVe embeddings. The NLP community had clearly moved to language models.
- Bi-LSTM + Attention was the dominant architecture.
- Good number of papers on low-resource languages (Spanish, Tamil, etc.) using transfer learning for representation, generation, and translation.
- Blackbox NLP was a new workshop that year and featured some of the best talks of the conference.
- Keynote by Yoav Goldberg discussed what RNNs can and cannot encode.
Papers I Found Interesting
Interpretable Structure Induction via Sparse Attention
Sparse attention was discussed heavily at WMT, WASSA, and Blackbox NLP. Worth reading if you're interested in attention interpretability.
Understanding CNNs for Text Classification
Ablation study on what CNNs focus on during sentiment analysis. Surprising finding: most of the time CNNs focus on seemingly random portions. Data augmentation with negation and label flipping gave interesting insights.
Keynote: The Moment of Meaning and the Future of Computational Semantics
Johan Bos
Argued that BLEU score is insufficient — simple negation can yield a 0.9 BLEU score despite the sentence having the opposite meaning. Metrics should be rooted in semantics and what sentences actually mean.
Best Papers
- MultiWOZ — A large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue. Useful for anyone working on chatbots with multiple conversational domains.
- Linguistically-Informed Self-Attention for Semantic Role Labeling — Multi-headed attention with a very deep network for SRL. Impressive talk. Code on GitHub.
- Phrase-Based & Neural Unsupervised Machine Translation — Neural stylistic transfer applied to text: transferring style across age, gender, etc. Cool concept closely related to image style transfer.
Conference Notes
- EMNLP 2019 to be co-located with IJCNLP in Hong Kong.
- Visa issues affected several attendees in Brussels — one PhD student was detained and deported, and a keynote speaker gave their talk over Skype.
- All talks were recorded and made available online.