Bayesian Agents

I want to create a baysian and updating scheme for Lewis signaling games that supports fast learning of signaling systems.

One direction is to use hierachial model. First I wanted to draw the intial wights from a prior is there a prior one can use for hierarchical learning in the Lewis signaling game.

the name of a prior for an distribution that is like an identity matrix?

Some thought on modeling games with agents.

1. Idealy one should be able to plug in a minimal amount of information about the agents and then be able to simulate the game and identify the optimal strategies for the agents.
2. One should be able to simulate the game with different solution concepts and see how the agents behave - like making mistakes or introducing private information.

For example for two player games we can provide a payoff matrix. then we can simulate the players playing the game in turn or at the same time once and repeatedly (with memory) and see how the agents behave.

This could cover a wide range of games from the prisoner’s dilemma, stag hunt, battle of the sexes, Lewis signaling game with two signals. A simple bandit algorithm could be used to simulate the agents playing the game and identify the optimal strategies for the agents.

prisoners dilemma

the payoff matrix for the prisoners dilemma is:

import numpy as np

prisoners_dillema_payoff_matrix = np.array([[(-1, -1), (-3, 0)], [(0, -3), (-2, -2)]])
stag_hunt_payoff_matrix = np.array([[(-1, -1), (-3, 0)], [(0, -3), (-2, -2)]])
lewis_signaling_game_payoff_matrix = np.array([[(1, 1), (0, 0)], [(0, 0), (1, 1)]])
battle_of_the_sexes_payoff_matrix = np.array([[(2, 1), (0, 0)], [(0, 0), (1, 2)]])
dove_hawk_payoff_matrix = np.array([[(3, 3), (0, 4)], [(4, 0), (1, 1)]])
suppot_oppose_evade_payoff_matrix = np.array([[(6, 4), (2, 8),(8,2)],
                                              [(8, 2), (25, 7.5),(7.5,2.5)], 
                                              [(3.5, 6.5), (3, 7),(4,6)]])
chicken_payoff_matrix = np.array([[(0, 0), (-1, 1)], [(1, -1), (-10, -10)]])
a=100
b=10
robber_guards_payoff_matrix = np.array([[(0, 0), (a, -1*a)], [(b, -1*b), (0, 0)]])  # mixed stategy

For games with incomplete information we can provide a prior distribution over the possible payoffs and then update the distribution based on the agents actions.

ion exploring the space of possible games and strategies one should be able to identify the optimal strategies for the agents. 2.

Some thoughts on developing the Bayesian agents:

Pareto improvement

In welfare economics, a Pareto improvement formalizes the idea of an outcome being “better in every possible way”. A change is called a Pareto improvement if it leaves everyone in a society better-off (or at least as well-off as they were before).

Pareto efficient or Pareto optimality

A situation is called Pareto efficient or Pareto optimal if all possible Pareto improvements have already been made; in other words, there are no longer any ways left to make one person better-off, unless we are willing to make some other person worse-off

Multi-objective optimization or Pareto optimization

is an area of multiple-criteria decision making that is concerned with mathematical optimization problems involving more than one objective function to be optimized simultaneously.

Admissible decision rule

In statistical decision theory, an admissible decision rule is a rule for making a decision such that there is no other rule that is always “better” than it, in the precise sense of “better” defined below. This concept is analogous to Pareto efficiency.

e.g. The James–Stein estimator is a nonlinear estimator of the mean of Gaussian random vectors and can be shown to dominate the ordinary least squares technique with respect to a mean-squared-error loss function. Therefore in this context the James–Stein estimator is admissible, while the ordinary least squares estimator is inadmissible.

Hiererchy of solution concepts

1. What is the hierarchy of solution concepts - in the sense that one solution concept can provide better solutions for a broader class of games than another?

One of the tricky aspects is that games can seem very different at first yet when we work out the optimal strategies, it turns out that the crucial aspects of the games are the same.

Solution concepts typically apply to a given class of games and these classes can be used to provide a multidimensional hierarchy of solution concepts.

strict dominance, weak dominance, iterated dominance, Nash equilibrium, correlated equilibrium, subgame perfect equilibrium, Bayesian Nash equilibrium, trembling hand perfect equilibrium, sequential equilibrium, perfect Bayesian equilibrium,

pareto optimality, ESS, backward induction, minimax, maxmin, risk dominance, quantal response equilibrium, level-k reasoning, cognitive hierarchy, iterated elimination of dominated strategies, rationalizability, sequential equilibrium, trembling hand perfect equilibrium, proper equilibrium, sequential equilibrium, perfect Bayesian equilibrium, core, Shapley value, nucleolus, kernel, bargaining set, von Neumann-Morgenstern solution, Nash bargaining solution, Kalai-Smorodinsky solution, egalitarian solution, competitive equilibrium, Walrasian equilibrium, Arrow-Debreu equilibrium, Radner,

for non-coopertaive game: Mertens stable equilibrium > forward induction, backward induction

1. Given a set of agent, with a schedule, action and payoff - can we define a ‘formal models’ for game in extensive and normal form.
2. For the formal game can we identifying all the different equlibria for a game is specified?
3. Implementing different solution concepts for game theoretic agents.
4. For games with incomplete information, can we implement a bayesian updating scheme for agents.
5. Can we implement a learning scheme for agents in a game.

import warnings
from warnings import simplefilter
warnings.filterwarnings('ignore', message='The AgentSet is experimental*')

# Import necessary modules
from mesa import Agent, Model
from mesa.time import RandomActivation
from mesa.space import MultiGrid
from mesa.datacollection import DataCollector
import numpy as np

class BayesianUpdater:
    def __init__(self, prior=None):
        if prior is None:
            # Default prior: uniform distribution over actions 'A' and 'B'
            prior = {'A': 0.5, 'B': 0.5}
        self.prior = prior
        self.belief = prior.copy()

    def update_belief(self, observation, likelihoods):
        # Update belief using Bayesian updating for each action
        for action in self.belief:
            self.belief[action] *= likelihoods[action]
        
        # Normalize to get new belief
        total = sum(self.belief.values())
        for action in self.belief:
            self.belief[action] /= total

    def make_decision(self):
        # Example decision rule: choose action with highest belief
        return max(self.belief, key=self.belief.get)

class BayesianAgent(Agent):
    def __init__(self, unique_id, model, prior=None):
        super().__init__(unique_id, model)
        self.bayesian_updater = BayesianUpdater(prior)
        self.observed_actions = []
        self.action = None

    def step(self):
        # Make a decision based on current belief
        self.action = self.bayesian_updater.make_decision()
        
        # Update belief based on the observed outcome
        observation = self.model.observe(self)
        likelihoods = {action: self.model.likelihood(observation, action) for action in self.bayesian_updater.prior}
        self.bayesian_updater.update_belief(observation, likelihoods)
        
        # Observe actions of all other agents
        self.observe_other_agents()
        
        # Print detailed output
        print(f"Agent {self.unique_id} action: {self.action}")
        print(f"Agent {self.unique_id} belief: {self.bayesian_updater.belief}")

    def observe_other_agents(self):
        # Observe actions of all other agents in the model
        self.observed_actions = [agent.action for agent in self.model.schedule.agents if agent != self]
        print(f"Agent {self.unique_id} observed actions: {self.observed_actions}")

    def update_belief_about_others(self):
        # Update belief about the world based on observed actions
        for action in self.observed_actions:
            likelihoods = {'A': self.model.likelihood(True, 'A'), 'B': self.model.likelihood(True, 'B')}
            self.bayesian_updater.update_belief(True, likelihoods)

class BayesianModel(Model):
    def __init__(self, N):
        super().__init__()
        self.num_agents = N
        self.schedule = RandomActivation(self)
        self.grid = MultiGrid(10, 10, True)
        
        # Define priors for three types of agents
        prior_type_1 = {'A': 0.8, 'B': 0.2}
        prior_type_2 = {'A': 0.5, 'B': 0.5}
        prior_type_3 = {'A': 0.2, 'B': 0.8}

        # Create agents with different priors
        for i in range(self.num_agents):
            if i % 3 == 0:
                prior = prior_type_1
            elif i % 3 == 1:
                prior = prior_type_2
            else:
                prior = prior_type_3

            agent = BayesianAgent(i, self, prior)
            self.schedule.add(agent)
            x = self.random.randrange(self.grid.width)
            y = self.random.randrange(self.grid.height)
            self.grid.place_agent(agent, (x, y))
        
        self.datacollector = DataCollector(
            agent_reporters={"Belief": lambda a: a.bayesian_updater.belief}
        )

    def step(self):
        self.datacollector.collect(self)
        self.schedule.step()
        for agent in self.schedule.agents:
            agent.update_belief_about_others()

    def observe(self, agent):
        # Simulate an observation based on the agent's action
        if agent.action == 'A':
            return self.random.random() < 0.7  # 70% chance of success
        else:
            return self.random.random() < 0.3  # 30% chance of success

    def likelihood(self, observation, action):
        # Return likelihood of observation given action
        if action == 'A':
            return 0.7 if observation else 0.3
        else:
            return 0.3 if observation else 0.7

# Run the model
if __name__ == "__main__":
    model = BayesianModel(10)
    for i in range(10):  # Reduced the number of steps for brevity
        print(f"\n--- Step {i + 1} ---")
        model.step()
    
    # Extract and print data
    data = model.datacollector.get_agent_vars_dataframe()
    print(data.tail())

--- Step 1 ---
Agent 7 observed actions: [None, None, None, None, None, None, None, None, None]
Agent 7 action: A
Agent 7 belief: {'A': 0.3, 'B': 0.7}
Agent 8 observed actions: ['A', None, None, None, None, None, None, None, None]
Agent 8 action: B
Agent 8 belief: {'A': 0.09677419354838711, 'B': 0.903225806451613}
Agent 3 observed actions: ['A', 'B', None, None, None, None, None, None, None]
Agent 3 action: A
Agent 3 belief: {'A': 0.903225806451613, 'B': 0.09677419354838711}
Agent 0 observed actions: ['A', 'B', 'A', None, None, None, None, None, None]
Agent 0 action: A
Agent 0 belief: {'A': 0.903225806451613, 'B': 0.09677419354838711}
Agent 5 observed actions: ['A', 'B', 'A', 'A', None, None, None, None, None]
Agent 5 action: B
Agent 5 belief: {'A': 0.09677419354838711, 'B': 0.903225806451613}
Agent 4 observed actions: ['A', 'B', 'A', 'A', 'B', None, None, None, None]
Agent 4 action: A
Agent 4 belief: {'A': 0.7, 'B': 0.3}
Agent 6 observed actions: ['A', 'B', 'A', 'A', 'B', 'A', None, None, None]
Agent 6 action: A
Agent 6 belief: {'A': 0.631578947368421, 'B': 0.3684210526315789}
Agent 1 observed actions: ['A', 'B', 'A', 'A', 'B', 'A', 'A', None, None]
Agent 1 action: A
Agent 1 belief: {'A': 0.3, 'B': 0.7}
Agent 9 observed actions: ['A', 'B', 'A', 'A', 'B', 'A', 'A', 'A', None]
Agent 9 action: A
Agent 9 belief: {'A': 0.903225806451613, 'B': 0.09677419354838711}
Agent 2 observed actions: ['A', 'B', 'A', 'A', 'B', 'A', 'A', 'A', 'A']
Agent 2 action: B
Agent 2 belief: {'A': 0.09677419354838711, 'B': 0.903225806451613}

--- Step 2 ---
Agent 1 observed actions: ['A', 'A', 'B', 'B', 'A', 'A', 'B', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 0.997351434488271, 'B': 0.0026485655117290162}
Agent 6 observed actions: ['A', 'A', 'B', 'B', 'A', 'A', 'B', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 0.9998780740955051, 'B': 0.00012192590449486855}
Agent 4 observed actions: ['A', 'A', 'B', 'B', 'A', 'A', 'B', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 0.9999104188867107, 'B': 8.958111328923861e-05}
Agent 5 observed actions: ['A', 'A', 'A', 'B', 'A', 'A', 'B', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 0.9980527468371296, 'B': 0.0019472531628704443}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'B', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 0.9980527468371296, 'B': 0.0019472531628704443}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'B', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 0.9998780740955052, 'B': 0.00012192590449486855}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'B', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 0.9999776032169311, 'B': 2.239678306888071e-05}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 0.9980527468371296, 'B': 0.0019472531628704443}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 0.997351434488271, 'B': 0.0026485655117290162}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 0.9999776032169311, 'B': 2.239678306888071e-05}

--- Step 3 ---
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 0.9999994448703694, 'B': 5.551296306621704e-07}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 0.999999592149599, 'B': 4.078504009625729e-07}
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 0.9999999812721682, 'B': 1.8727831821417877e-08}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 0.999999995318042, 'B': 4.6819580211166604e-09}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 0.9999998612175346, 'B': 1.3878246544723665e-07}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 0.9999999745093402, 'B': 2.5490659806763155e-08}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 0.9999999745093402, 'B': 2.5490659806763158e-08}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 0.999999592149599, 'B': 4.078504009625729e-07}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 0.9999994448703694, 'B': 5.551296306621704e-07}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 0.9999977794851753, 'B': 2.220514824627957e-06}

--- Step 4 ---
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 0.9999999999946714, 'B': 5.328601407091876e-12}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 0.9999999999147423, 'B': 8.525762250665542e-11}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 0.9999999999946715, 'B': 5.328601407091877e-12}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 0.9999999999709888, 'B': 2.9011274326813147e-11}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 0.9999999998839549, 'B': 1.1604509729715271e-10}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 0.9999999974727957, 'B': 2.5272043350445003e-09}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 0.9999999998839549, 'B': 1.1604509729715271e-10}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 0.9999999999147423, 'B': 8.525762250665542e-11}
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 0.999999999996085, 'B': 3.914890829705688e-12}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 0.9999999999946714, 'B': 5.328601407091876e-12}

--- Step 5 ---
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 0.9999999999999956, 'B': 4.455591569214526e-15}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 0.9999999999999031, 'B': 9.703288306288511e-14}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 0.9999999999999757, 'B': 2.425822076572305e-14}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 0.9999999999998679, 'B': 1.3207253528003347e-13}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 0.9999999999999939, 'B': 6.064555191430872e-15}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 0.9999999999999939, 'B': 6.064555191430872e-15}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 0.9999999999999822, 'B': 1.7822366276857863e-14}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 0.9999999999994716, 'B': 5.282901411199244e-13}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 0.999999999999967, 'B': 3.301813382001164e-14}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 0.9999999999999989, 'B': 1.1138978923036352e-15}

--- Step 6 ---
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 1.0, 'B': 2.3285069001793296e-19}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 1.0, 'B': 2.760861761644727e-17}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 1.0, 'B': 2.028388233045105e-17}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 1.0, 'B': 2.7608617616447268e-17}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 1.0, 'B': 1.2677426456531907e-18}
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 1.0, 'B': 5.0709705826127634e-18}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 1.0, 'B': 6.9021544041118146e-18}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 1.0, 'B': 3.725611040286927e-18}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 0.9999999999999994, 'B': 6.012543392026289e-16}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 1.0, 'B': 6.9021544041118146e-18}

--- Step 7 ---
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 1.0, 'B': 5.771341975641892e-21}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 1.0, 'B': 3.1421750756272515e-20}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 1.0, 'B': 3.1421750756272515e-20}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 1.0, 'B': 1.4428354939104723e-21}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 1.0, 'B': 7.7880666393501565e-22}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 1.0, 'B': 1.4428354939104723e-21}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 1.0, 'B': 1.2568700302509001e-19}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 1.0, 'B': 7.855437689068126e-21}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 1.0, 'B': 4.240169614757307e-21}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 1.0, 'B': 4.867541649593848e-23}

--- Step 8 ---
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 1.0, 'B': 8.863706695300737e-25}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 1.0, 'B': 3.0161224171509453e-25}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 1.0, 'B': 1.4304612194724779e-22}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 1.0, 'B': 1.01751735022585e-26}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 1.0, 'B': 1.6421110937821813e-24}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 1.0, 'B': 3.0161224171509453e-25}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 1.0, 'B': 6.568444375128727e-24}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 1.0, 'B': 6.568444375128727e-24}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 1.0, 'B': 4.8257958674415124e-24}
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 1.0, 'B': 1.2064489668603783e-24}

--- Step 9 ---
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 1.0, 'B': 2.127031739105971e-30}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 1.0, 'B': 1.8528809816212014e-28}
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 1.0, 'B': 1.373076307676699e-27}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 1.0, 'B': 2.990255070051478e-26}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 1.0, 'B': 1.373076307676699e-27}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 1.0, 'B': 6.304942229127699e-29}
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 1.0, 'B': 1.0087907566604319e-27}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 1.0, 'B': 3.4326907691917473e-28}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 1.0, 'B': 1.373076307676699e-27}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 1.0, 'B': 6.304942229127699e-29}

--- Step 10 ---
Agent 8 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 8 action: A
Agent 8 belief: {'A': 1.0, 'B': 2.10878955239161e-31}
Agent 2 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 2 action: A
Agent 2 belief: {'A': 1.0, 'B': 6.2508687843114266e-30}
Agent 3 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 3 action: A
Agent 3 belief: {'A': 1.0, 'B': 1.3179934702447563e-32}
Agent 6 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 6 action: A
Agent 6 belief: {'A': 1.0, 'B': 1.3179934702447563e-32}
Agent 7 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 7 action: A
Agent 7 belief: {'A': 1.0, 'B': 2.8702968907552474e-31}
Agent 1 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 1 action: A
Agent 1 belief: {'A': 1.0, 'B': 2.8702968907552474e-31}
Agent 4 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 4 action: A
Agent 4 belief: {'A': 1.0, 'B': 2.8702968907552474e-31}
Agent 9 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 9 action: A
Agent 9 belief: {'A': 1.0, 'B': 2.4208043331026137e-33}
Agent 0 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 0 action: A
Agent 0 belief: {'A': 1.0, 'B': 7.175742226888119e-32}
Agent 5 observed actions: ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A']
Agent 5 action: A
Agent 5 belief: {'A': 1.0, 'B': 3.873286932964182e-32}
                                               Belief
Step AgentID                                         
9    6         {'A': 1.0, 'B': 6.428685662431004e-36}
     8        {'A': 1.0, 'B': 1.0285897059889607e-34}
     0         {'A': 1.0, 'B': 3.500062193990214e-35}
     7        {'A': 1.0, 'B': 1.4000248775960855e-34}
     3         {'A': 1.0, 'B': 6.428685662431004e-36}