Review of “Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input”
This is the paper that Marco Baroni used to explain the emergence of languages in his talk “Is Composonality over rated?”.
In (Lazaridou et al. 2018) the authors look emergence of language using a deep reinforcement learning approach. They train reinforcement-learning neural network agents on referential communication games. They extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. They find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured.
The goal of the paper is to investigate the properties of communication protocols that emerge when reinforcement learning agents are trained on referential communication games. The study aims to explore how agents learn to communicate in scenarios with structured and disentangled input data, as well as in more challenging scenarios with raw pixel input, resembling the complexity of real-world environments.
The training of agents just described was successful and the researchers found that agents can produce structured and compositional communication protocols when presented with disentangled inputs, but struggle to do so when presented with entangled raw pixel input. The emergent protocols were found to be unstable and highly grounded in the specific game situation, leading to specialized ad-hoc naming conventions.
Abstract
The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by us ing contemporary deep learning methods and by training reinforcement-learning neural network agents on referential communication games. We extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. We find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured
Outline
So I went through the paper and outlined most of the methodolgy etc. Its a bit long but I think it is worth it. Here is a quick outline. I might come back and add more material later. But I think the video above though not dierectly on this paper is a good explainer to understand this more easily.
Introduction
- Explores the emergence of linguistic communication through referential games with symbolic and pixel inputs.
- Motivated by understanding the role of environmental conditions on emergent communication.
- Introduces the use of deep reinforcement learning agents to scale up traditional studies of language emergence.
Referential Games Framework
- Based on multi-agent cooperative reinforcement learning, inspired by the Lewis signaling game.
- Involves a speaker communicating a target object to a listener, who identifies it among distractors.
- Differentiates between symbolic data (structured and disentangled) and pixel data (entangled).
Study 1: Referential Game with Symbolic Data
- Uses disentangled input from the Visual Attributes for Concepts Dataset.
- Demonstrates that agents can learn compositional protocols when input is structured.
- Explores the effects of message length, showing improved communicative success and reduced ambiguity with longer messages.
- Investigates how context-dependent distractors impact language emergence and object confusability.
Study 2: Referential Game with Raw Pixel Data
- Employs synthetic scenes of geometric objects generated using the MuJoCo engine.
- Agents learn to process raw pixel input without pre-training, achieving significant communicative success.
- Highlights environmental pressures’ role in shaping emergent protocols, leading to overfitting and ad-hoc conventions.
Structural Properties of Emergent Protocols
- Examines the topographic similarity metric, correlating object similarity with message similarity.
- Observes compositional signals in structured environments but instability and environmental overfitting with pixel input.
Probe Models
- Analyzes the speaker’s visual representations using linear classifiers.
- Finds that disentanglement is necessary for encoding object properties and effective communication.
Conclusion
- Demonstrates that structured input aids compositionality, while raw pixel input challenges protocol stability.
- Highlights the scalability of emergent communication studies with realistic data and deep learning techniques.
- Suggests future work to mitigate overfitting and promote generalization across diverse environments.
The paper
Citation
@online{bochman2025,
author = {Bochman, Oren},
title = {Emergence of {Linguistic} {Communication} from {Referential}
{Games} with {Symbolic} and {Pixel} {Input}},
date = {2025-01-01},
url = {https://orenbochman.github.io/posts/2024/2024-10-10-marco-baoni-composionality/paper3.html},
langid = {en}
}
Comments
What are the main research questions of the paper?
Looking over this paper I did not see any outrageous claims not much that I thought wrong. Although Lazaridou has a number of criticism on research in this area this paper seems sound work.
The referential game
Although the referential game isn’t a novelty and the authors give a number of prior works that use it, I do suspect that using the referential game has some possible pitfalls. Let’s consider for a second how the referential game differs from the vanilla Lewis Signaling game and if these differences should be significant.
In a vanilla Lewis signaling game the sender encodes the pre-linguistic object into a message and the receiver has to pick one state from all states. In this game a good sender should be able to pick a unique message per state (assuming there are sufficient1 signals and it does not make use of homonyms).
1 for a simple system one per state is enough. For complex signaling systems this depends on how the atomic signals are aggreaged into complex ones. If the are assembled with replacement into a sequence of length k there are |S|^k complex symbols possible. If additional structure is imposed there may be less possible states. If partial sequences are allowed we may have almost twice as many states.
The receiver needs to match the signal with a state. It can pick one from the undecoded states. This is initially a task with en expectation of 1/|S|. Once it solves a messages it should eliminate its states thus increase its expectation of success.
In the referential games I abstract to a two round extensive form game. In the first round the sender and receiver play a classification game. Sender looks at the pre-linguistic object and classifies it. It then encodes it into a sequence of symbols. The encoder has an error rate and should perform poorly as it has no pretraining.
In the referential game we can imagine two rounds. In the first round the agent
Let’s assume that the sender encodes each input into a unique message or at least unique up to
If there are S states the
In the referential game the receiver need to solve a multiple choice question with one answer and several distractors by decoding the message from the sender.
The researchers call the language that emerges a lexicon or a protocol rather than a language or a signaling system.
A complex signaling system
One term I don’t know if i like is pre-linguistic concepts, usually we call this as the states. However I think that this term isn’t bad at all. It suggests that we arn’t looking just at states but at an item we want to talk about. This makes more sense particularly when we think about bitmaps of states - they are less like states.
One more point is that by adding the vision learning we are adding a second game. Call it a classification game. The agent needs to succeed at classification game otherwise they are just guessing. It worth while to consider though that just guessing with a good memory is enough to develop a signaling system.
There is a massive asymmetry between the sender and the receiver that is not extant in the original game. The sender can learn all images via a ground truth while the receiver can only learn about the correct ones.
So that as a framing game this needs to be reconsidered. What I mean is that the sender’s vision should be evaluated compared in a scale between an agent with a perfect vision and perfect blindness as baselines. And the same for receiver.
The vision capability should be factored in to the evaluation of the agent’s learning of the signaling system.
The paper does have many interesting ideas and shows methods, for achieving them. In a number of areas I think one could do better, but I doubt the results should be very different.
One area that seems wort further investigation is CONCEPTUAL ALIGNMENT in appendix A. This seems to be related to semantic grounding — getting the agents language concepts/semantics to align with the world or with a second set of semantics like say a human language.
What they consider here is much more specific - does the visual capacity learned by the agents provide them with a disentangled view of the world that is in line with the compositional structure of the state space they are observing (called pre-linguistic concepts).
It seems that either the methodology is inadequate or that there is a problem with alignment.
What might be done -
If they do we might consider that the agents vision are not seeing things in the same way. But consider that the sender always knows the state the receivers might not know the state most of the time. So thier vision might be less developed