Variables

Consider presenting a light to a dog before food.

Light \(\to\) Food

\(V_L\) is the association strength between the light and the food.

\(\alpha_L\) is the salience of the light.

\(\beta\) is the learning rate (or step size).

\(\lambda\) is the maximum associability of any event with the food.

In this scenario, we update associations according to the following equation:

\(\Delta V_L = \alpha_L\beta(\lambda-\sum_y V_y)\)

Learning Example

Light \(\to\) Food

\(V_L = 0.1\)

\(\alpha_L = 1\)

\(\beta = 0.5\)

\(\lambda = 1\)

w = data.frame(Object='light', V=0.1, Time=0)
beta = 0.5

# Time 1 
w = observe(w, 'light', 1, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.1) = 0.45
## V_light = 0.55

# Time 2 
w = observe(w, 'light', 2, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.55) = 0.225
## V_light = 0.775

# Time 3 
w = observe(w, 'light', 3, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.775) = 0.1125
## V_light = 0.8875

# Time 4 
w = observe(w, 'light', 4, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.8875) = 0.05625
## V_light = 0.94375

# Time 5 
w = observe(w, 'light', 5, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.94375) = 0.028125
## V_light = 0.971875

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Change in maximum Associability?

\(V_L = 0.1\)

\(\alpha_L = 1\)

\(\beta = 0.5\)

\(\lambda = 0.5\)

w = data.frame(Object='light', V=0.1, Time=0)
beta = 0.5

# Time 1 
w = observe(w, 'light', 1, 1, beta)

## delta V = 1 * 0.5 * (0.5 - 0.1) = 0.2
## V_light = 0.3

# Time 2 
w = observe(w, 'light', 2, 1, beta)

## delta V = 1 * 0.5 * (0.5 - 0.3) = 0.1
## V_light = 0.4

# Time 3 
w = observe(w, 'light', 3, 1, beta)

## delta V = 1 * 0.5 * (0.5 - 0.4) = 0.05
## V_light = 0.45

# Time 4 
w = observe(w, 'light', 4, 1, beta)

## delta V = 1 * 0.5 * (0.5 - 0.45) = 0.025
## V_light = 0.475

# Time 5 
w = observe(w, 'light', 5, 1, beta)

## delta V = 1 * 0.5 * (0.5 - 0.475) = 0.0125
## V_light = 0.4875

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Change in salience?

\(V_L = 0.1\)

\(\alpha_L = 0.5\)

\(\beta = 0.5\)

\(\lambda = 1\)

w = data.frame(Object='light', V=0.1, Time=0)
beta = 0.5

# Time 1 
w = observe(w, 'light', 1, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.1) = 0.225
## V_light = 0.325

# Time 2 
w = observe(w, 'light', 2, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.325) = 0.16875
## V_light = 0.49375

# Time 3 
w = observe(w, 'light', 3, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.49375) = 0.1265625
## V_light = 0.6203125

# Time 4 
w = observe(w, 'light', 4, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.6203125) = 0.094921875
## V_light = 0.715234375

# Time 5 
w = observe(w, 'light', 5, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.715234375) = 0.07119140625
## V_light = 0.78642578125

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Change in learning rate

\(V_L = 0.1\)

\(\alpha_L = 1\)

\(\beta = 0.6\)

\(\lambda = 1\)

w = data.frame(Object='light', V=0.1, Time=0)
beta = 0.6

# Time 1 
w = observe(w, 'light', 1, 1, beta)

## delta V = 1 * 0.6 * (1 - 0.1) = 0.54
## V_light = 0.64

# Time 2 
w = observe(w, 'light', 2, 1, beta)

## delta V = 1 * 0.6 * (1 - 0.64) = 0.216
## V_light = 0.856

# Time 3 
w = observe(w, 'light', 3, 1, beta)

## delta V = 1 * 0.6 * (1 - 0.856) = 0.0864
## V_light = 0.9424

# Time 4 
w = observe(w, 'light', 4, 1, beta)

## delta V = 1 * 0.6 * (1 - 0.9424) = 0.03456
## V_light = 0.97696

# Time 5 
w = observe(w, 'light', 5, 1, beta)

## delta V = 1 * 0.6 * (1 - 0.97696) = 0.013824
## V_light = 0.990784

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Overshadowing

Light + Tone \(\to\) Food

\(V_L = 0.1\)

\(V_T = 0.1\)

\(\alpha_L = 1\)

\(\alpha_T = 0.5\)

\(\beta = 0.5\)

\(\lambda = 1\)

w = data.frame(Object=c('light', 'tone'), V=0.1, Time=0)
beta = 0.5


# Time 1 
w = observe(w, 'light', 1, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.2) = 0.4
## V_light = 0.5

w = observe(w, 'tone', 1, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.2) = 0.2
## V_tone = 0.3

# Time 2 
w = observe(w, 'light', 2, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.8) = 0.1
## V_light = 0.6

w = observe(w, 'tone', 2, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.8) = 0.05
## V_tone = 0.35

# Time 3 
w = observe(w, 'light', 3, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.95) = 0.025
## V_light = 0.625

w = observe(w, 'tone', 3, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.95) = 0.0125
## V_tone = 0.3625

# Time 4 
w = observe(w, 'light', 4, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.9875) = 0.00624999999999998
## V_light = 0.63125

w = observe(w, 'tone', 4, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.9875) = 0.00312499999999999
## V_tone = 0.365625

# Time 5 
w = observe(w, 'light', 5, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.996875) = 0.00156250000000002
## V_light = 0.6328125

w = observe(w, 'tone', 5, 0.5, beta)

## delta V = 0.5 * 0.5 * (1 - 0.996875) = 0.000781250000000011
## V_tone = 0.36640625

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Blocking

Light + Tone \(\to\) Food

\(V_L = 0.6\)

\(V_T = 0.1\)

\(\alpha_L = 1\)

\(\alpha_T = 1\)

\(\beta = 0.5\)

\(\lambda = 1\)

w = data.frame(Object=c('light', 'tone'), V=c(0.6, 0.1), Time=0)
beta = 0.5

# Time 1 
w = observe(w, 'light', 1, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.7) = 0.15
## V_light = 0.75

w = observe(w, 'tone', 1, 1, beta)

## delta V = 1 * 0.5 * (1 - 0.7) = 0.15
## V_tone = 0.25

# Time 2 
w = observe(w, 'light', 2, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_light = 0.75

w = observe(w, 'tone', 2, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_tone = 0.25

# Time 3 
w = observe(w, 'light', 3, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_light = 0.75

w = observe(w, 'tone', 3, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_tone = 0.25

# Time 4 
w = observe(w, 'light', 4, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_light = 0.75

w = observe(w, 'tone', 4, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_tone = 0.25

# Time 5 
w = observe(w, 'light', 5, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_light = 0.75

w = observe(w, 'tone', 5, 1, beta)

## delta V = 1 * 0.5 * (1 - 1) = 0
## V_tone = 0.25

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Extinction

In extinction, the conditioned stimulus is presented without the unconditioned stimulus. The main difference in the math is that the maximum amount you could associate light to food in a trial is zero because there is no food in extinction.

Light \(\to\) ~Food

\(V_L = 1\)

\(\alpha_L = 1\)

\(\beta = 0.5\)

\(\lambda = 0\)

w = data.frame(Object='light', V=1, Time=0)
beta = 0.5

# Time 1 
w = observe(w, 'light', 1, 1, beta)

## delta V = 1 * 0.5 * (0 - 1) = -0.5
## V_light = 0.5

# Time 2 
w = observe(w, 'light', 2, 1, beta)

## delta V = 1 * 0.5 * (0 - 0.5) = -0.25
## V_light = 0.25

# Time 3 
w = observe(w, 'light', 3, 1, beta)

## delta V = 1 * 0.5 * (0 - 0.25) = -0.125
## V_light = 0.125

# Time 4 
w = observe(w, 'light', 4, 1, beta)

## delta V = 1 * 0.5 * (0 - 0.125) = -0.0625
## V_light = 0.0625

# Time 5 
w = observe(w, 'light', 5, 1, beta)

## delta V = 1 * 0.5 * (0 - 0.0625) = -0.03125
## V_light = 0.03125

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Change in salience?

\(\alpha_L = 0.5\)

w = data.frame(Object='light', V=1, Time=0)
beta = 0.5

# Time 1 
w = observe(w, 'light', 1, 0.5, beta)

## delta V = 0.5 * 0.5 * (0 - 1) = -0.25
## V_light = 0.75

# Time 2 
w = observe(w, 'light', 2, 0.5, beta)

## delta V = 0.5 * 0.5 * (0 - 0.75) = -0.1875
## V_light = 0.5625

# Time 3 
w = observe(w, 'light', 3, 0.5, beta)

## delta V = 0.5 * 0.5 * (0 - 0.5625) = -0.140625
## V_light = 0.421875

# Time 4 
w = observe(w, 'light', 4, 0.5, beta)

## delta V = 0.5 * 0.5 * (0 - 0.421875) = -0.10546875
## V_light = 0.31640625

# Time 5 
w = observe(w, 'light', 5, 0.5, beta)

## delta V = 0.5 * 0.5 * (0 - 0.31640625) = -0.0791015625
## V_light = 0.2373046875

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Change in learning rate

\(\beta = 0.6\)

w = data.frame(Object='light', V=1, Time=0)
beta = 0.6

# Time 1 
w = observe(w, 'light', 1, 0.5, beta)

## delta V = 0.5 * 0.6 * (0 - 1) = -0.3
## V_light = 0.7

# Time 2 
w = observe(w, 'light', 2, 0.5, beta)

## delta V = 0.5 * 0.6 * (0 - 0.7) = -0.21
## V_light = 0.49

# Time 3 
w = observe(w, 'light', 3, 0.5, beta)

## delta V = 0.5 * 0.6 * (0 - 0.49) = -0.147
## V_light = 0.343

# Time 4 
w = observe(w, 'light', 4, 0.5, beta)

## delta V = 0.5 * 0.6 * (0 - 0.343) = -0.1029
## V_light = 0.2401

# Time 5 
w = observe(w, 'light', 5, 0.5, beta)

## delta V = 0.5 * 0.6 * (0 - 0.2401) = -0.07203
## V_light = 0.16807

ggplot(w[w$Object != '',], aes(Time, V, shape=Object, color=Object)) +
  geom_line() +
  geom_point() +
  theme_bw() + ylim(0, 1)

Rescorla-Wagner Model

Frank Mollica & Shirlene Wade

Variables

Learning Example

Change in maximum Associability?

Change in salience?

Change in learning rate

Overshadowing

Blocking

Extinction

Change in salience?

Change in learning rate