EMNLP 2020 - from an Industry Perspective

EMNLP 2020 (The 2020 Conference on Empirical Methods in Natural Language Processing) took place between November 16 - 20, virtually. Virtual conferences are one of the nice perks of the current pandemic situation, making it easy to keep up to date with the current research and get in touch with the academia community, without long-distance travel.

On Navigating Virtual EMNLP 2020

The schedule was packed with many Zoom sessions and Gather.town sessions (a game-like environment where your avatar can walk around the virtual conference venue and interact with other participants via audio and video).

The accepted papers (751 main papers and another 400+ findings papers) were the topic of the first three days while workshops and tutorials took place during the last two days. Another perk of 2020, pre-recorded video summaries were available for each paper! The papers are organized by track and there are tools on the website to search, to navigate by keywords or to visualize in a clustered way.

There were chat rooms available for the papers and the different sessions. Twitter’s #emnlp2020 tag was also useful to see what participants are talking about.

I found it very hard to decide what to read or watch first. There was no ranking of the papers so it was hard to prioritize. Then again, having a ranking based on views might turn it into a popularity contest on Twitter and might incentivize people to focus on the most popular topics to the detriment of diversity so I can see reasons why not to include one.

On Live Sessions and Community

I loved it that the live sessions were organized throughout the day (sometimes with a mix of live and recorded videos), giving participants from all timezones a chance to interact.

I attended the following workshops and tutorials (some partially):

• Evaluation and Comparison of NLP Systems
• Fifth Conference on Machine Translation (WMT20)
• Interpreting Predictions of NLP Models
• Deep Learning Inside Out (DeeLIO): The First Workshop on Knowledge Extraction and Integration for Deep Learning Architectures
• BlackboxNLP 2020: Analyzing and interpreting neural networks for NLP
• T7: The Amazing World of Neural Language Generation

I missed others like SustaiNLP: Workshop on Simple and Efficient Natural Language Processing.

The organizers of these sessions went out of their way to make it interactive and answer lots of questions. It was a lovely feeling of a community interested in the same research topics, sharing results, respectfully asking questions, kindly answering. I found these sessions heart-warming, the organizers did an amazing job! I followed many people on Twitter :)

On Poster Sessions

I did not attend them. As a software engineer working in industry, I feel that the video summaries bring me more value than approaching people I don’t know in front of their (virtual) posters, checking out a few slides and trying to figure out what questions to ask.

This is mainly because I have less familiarity with the research topics and developments as I’m not working as a researcher in my day-to-day. While watching the videos I would many times hear terms for the first time and look them up (e.g. specific datasets) or because I’m working at the same company for a long time I would have to reconcile terms used in academia to what we call some tasks internally.

I can see how poster sessions may be more efficient for people actively involved in the research areas to navigate the content. However, I hope they keep the videos format for future conferences, I think it helps bring industry and academia together.

On Exciting Areas and a few Highlights

I’m excited about research that is trying to understand what the huge models dominating NLP leaderboards are actually doing. It’s a very interesting and active area.

Language Generation and Machine Translation are two other areas that fascinate me (but I’m pretty curious in general).

Highlighting a few papers that caught my eye:

• With Little Power Comes Great Responsibility (pdf): The importance of power analysis in NLP.

• Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models (pdf): Transfer learning for studying how models learn. e.g. there is transfer from music, code to language. Recursiveness may play a role but also ability to link different parts of input and look back.

• Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models (pdf): Checking if language models learn sane facts about numbers by masking the numbers and looking at predictions.

• Digital Voicing of Silent Speech (pdf): Using electrodes to learn silent speech and convert it to speech. This got the best paper award.

• If beam search is the answer, what was the question? (pdf): Why beam search is useful in text generation: people don’t like surprises in text and beam search helps to have a uniform information distribution.

---

EMNLP 2021 is scheduled to be hopefully in Punta Cana, Dominican Republic, but possibly online depending on the development of the covid situation. Hoping for the best for everyone and looking forward!