Research Questions

Talking with some people today at BIU I get to realize that

1. I have made lots of progress recently but I have not been able to articulate it well.
2. It’s not clear what direction I am going with my research as there are a number of related questions I am trying to answer
3. My research questions are not clear enough
4. A number of them are such that it’s unlikely that anyone will be able to say “I am an expert in this area”. This is an issue if I want to write this up as a thesis or a paper with someone else as an advisor or co-author.

So let’s try to clarify them a bit more.

Research Questions

0. How can agents use a language to facilitate transfer learning across curriculums of multi-agent environment.

1. What are some (efficient) ways for a language to emerge in a MGP with a number of interacting agents?

   • working paper on scenarios

2. What are the desirable properties of such emergent languages?

   • working paper with a desiderata

3. Given that a language can emerge by spontaneous symmetry breaking, how can an agent plan a better language for such a collective? If the over arching concern is learnability, is there a clear way to measure the learnability of a language and to

4. Paradigms for language emergence ?

   • Can we consider the language emergence as independent of the other agents other behavior?

     • If we so wish what assumptions must we make?
     • e.g. if we have MGP i.e. a MARL environment and with Lewis a asymmetrical viewed state, send/receive actions and a cooperative reward signal for the receiver who decodes the state correctly, then wouldn’t the agents with an exploration strategy eventually learn a signaling system even if the greedy actions is to zero sum action?

   • is this a game theoretic question ?

   • should we treat language emergence as Lewis step in an extended form game where payoffs and strategies are intimately entangled?

   • should we treat language emergence as an iterated sequence of games where in each step decision is made independently of the previous step?

5. If we consider wide classes of complex lewis signaling games can we characterize types of equilibria that are more likely to emerge. If some equilibria more stable or stronger attractors than others

6. Grounding:

   • The goals of the grounding is specific if not rather broad. But should be considered as an MVP, i.e. the top goals are to be realized first the others as future work.
     1. map a GPL general purpose language encompassing many different MDPs and the experiences gained within them onto the current mdp using a subset of the GPL that is isomorphic to the current DSL.
   • The grounding is a set of symbols that can be used to communicate about the state of the world and the actions of the agent. The DSL or Domain Specific Language is a set of symbols and an aggregation rule that can for agent to communicate and some additional symbols that can let the agent communicate about its models with an LLM or some other agents. The LLM here might be a gateway to a RAG system with access to its past experiences, dynamic models, values, policies, options, general value functions, etc.
     
   • There seems to be a rather trivial way to do grounding for an MDP/MGP for each action in the MDP/MGP we can assign a unique verb symbol for each state in the MDP/MGP we can assign a unique noun symbol
   • To handle case where state is structured e.g. in two parts we can use a noun phrase with two nouns e.g. cat and dog
   • We thus need a set of primitive symbols to capture the state + a syntax to combine them into a phrase.
   • This would allow us to describe even an image as a list of pixels….

Some other questions

1. how much of a natural language vs synthetic language is overhead ? what are good metrics for this?
2. is this still true when resources are as severely restricted as in the case of a human agent?