A sequential Bayesian model for
learning and memory in
multi-context environments

Dave F. Kleinchmidt Rutgers University/Princeton Neuroscience Institute
Pernille Hemmer Rutgers University

24 July 2018 // MathPsych // osf.io/dqz73

in the real world judgements are made in

context

which provides useful information

arena()

x, y = -0.3, 0.4
arena([x], [y], color="black")

plot(Gray.(rand(Bool, 100,100)), axis=false, lims=(0,100), aspect_ratio=:equal)

arena([x], [y], color=:white, markerstrokecolor=:black, markersize=5)
annotate!(0,0, text("?", 32))

Now you need to reconstruct the location of this dot from your memory, which has some uncertainty associated with it.

arena(randn(200).*0.2, randn(200).*0.2, color=:black, markeralpha=0.25)
scatter!([x], [y], color=:white, markerstrokecolor=:black, markersize=5)

quiver!([x], [y], quiver=(-[x*0.3], -[y.*0.3]), color=:black)

srand(2)
θ = rand(200) * 2π
ρ = randn(200) * .05 + 0.85

arena(cos.(θ).*ρ, sin.(θ).*ρ, markeralpha=0.25, color=:black)
scatter!([x], [y], color=:white, markerstrokecolor=:black, markersize=5)


quiver!([x], [y], quiver=([x*.3], [y*.3]), color=:black)

but what is a

context?

and how do you know?

begs the question, what constitutes a context? and how is an agent supposed to know?

there are of course a lot of different sources of information about the context. recall Jennifer Truebloods work, where context comes from available alternatives in a decision making task. another important source of information about context is....history

history provides context

While it might look like there's a just a single context here, and teh dots appear randomly around the arena...

r1_shuffled = view(recall, randperm(180) .+ 20, :)

p = arena([], [], markeralpha=0.25, color=:black, lims=(-1.1,1.1))
anim = @animate for (x,y) in @_ zip(r1_shuffled[:x], r1_shuffled[:y])
    push!(p, x,y)
end

gif(anim, "figures/shuffled.gif", fps=5)

INFO: Saved animation to /home/dave/.dropbox-raizadalab/Dropbox (Raizada Lab)/work/dots-location-memory-pernille/figures/shuffled.gif

history provides context

...if the same dots appear in a different order it's clear that there's actually a number of different contexts, or clusters.

p = arena([], [], markeralpha=0.25, color=:black, lims=(-1.1,1.1))
anim = @animate for (x,y) in @_ zip(recall[:x], recall[:y]) |> It.drop(_, 20) |> It.take(_, 180)
    push!(p, x,y)
end

gif(anim, "figures/clustered.gif", fps=5)

INFO: Saved animation to /home/dave/.dropbox-raizadalab/Dropbox (Raizada Lab)/work/dots-location-memory-pernille/figures/clustered.gif

Behavior

In a structured environment recall is biased towards clusters [Robbins, Hemmer, and Tang, CogSci2014]

...and in fact, in such structured environments, recall is biased towards clusters! above and beyond the overall context

arena(lims=(-1,1))
@_ known_recalled1 |>
    @by(_, :block, x_clus = mean(:x), y_clus = mean(:y)) |>
    @df(_, scatter!(:x_clus, :y_clus, color=:red, seriestype=:scatter, markerstrokecolor=:white))
@df recall1 quiver!(:x, :y, quiver=(:x_resp.-:x, :y_resp.-:y), color=:black, seriestype=:quiver, lims=(-1,1))

Behavior + cluster bias

In a structured environment recall is biased towards clusters [Robbins, Hemmer, and Tang, CogSci2014]

arena(lims=(-1,1))
@_ known_recalled1 |>
    @by(_, :block, x_clus = mean(:x), y_clus = mean(:y)) |>
    @df(_, scatter!(:x_clus, :y_clus, color=:red, seriestype=:scatter, markerstrokecolor=:white))
@df known_recalled1 quiver!(:x, :y, quiver=(:x_resp.-:x, :y_resp.-:y), color=GrayA(0.0, 0.5))
@df known_recalled1 quiver!(:x, :y, quiver=(:x_mod.-:x, :y_mod.-:y), color=RGBA(1, 0, 0, 0.5))

approach: bounded rationality

Computational-level: Dirichlet Process mixture model

• Infer how points $x_t$ are assigned $z_t$
  • $p(z_1, \ldots, z_T | x_1, \ldots, x_T) \propto p(x_1, \ldots, x_T | z_1, \ldots, z_T) p(z_1, \ldots, z_T)$
• Prior: "sticky" CRP $p(z_t = j | z_{1\ldots t-1}) \propto N_j (\times \frac{\rho}{1-\rho}$ if $z_{t-1}=j)$
  • $N_j = \alpha$ for all new $j$.
  • Prefer small number of contexts
  • Allow for up to $T$ (one per point)
• Likelihood: $p(x_t | z_t, z_{1:t-1}, x_{1:t-1}) = p(x_t | \{x_i \ \mathrm{if}\ z_i = z_t\})$
  • Prefer compact clusters

Algorithmic-level

approach: bounded rationality

Computational-level

Algorithmic-level: Sequential Monte Carlo

• online (not batch)
• finite uncertainty
• particle filter:
  • Each particle is one hypothetical clustering $z_{1\ldots t}$
  • Update particles in parallel following new data point
  • Re-sample when particles become too homogenous

Does it work?

Learning clusters

Recall

Prediction

Learning clusters

@df recalled1 plot(arena(:x, :y, group=assignments(first(particles(rf))), title="Inferred clusters"),
                   arena(:x, :y, group=:block, title="True blocks"),
                   layout=(1,2), size=(800, 400))

Learning clusters

plot(plot(show_assignment_similarity(rf), title="Inferred clusters"),
     plot(Gray.(@with(recall1, :block .== :block')), title="True blocks"),
     axis=false, aspect_ratio=:equal, layout=(1,2), size=(800,400))

Recall

Task

• Immediate recall with mask

Model

• Bayesian cue combination (after e.g., Huttenlocher)
• two cues: thing you saw, and inferred context
• weighted average (by inverse-variance)

• works best for high stickiness, low clustering (fewer clusters). but all work about the same

Recall: model

size2 = (900,400)

p1 = @df recalled1 arena(:x, :y, quiver=(:x_mod.-:x, :y_mod.-:y), seriestype=:quiver, label="Model",
                         layout=@layout([a{0.5w} _]), size=size2)

Recall: model + behavior

@df recalled1 quiver!(:x, :y, quiver=(:x_resp.-:x, :y_mod.-:y), color=GrayA(0.0, 0.3), label="Behavior", subplot=1)