[FIXED] Is there a vectorized way to sample multiples times with np.random.choice() with differents p?

Issue

I'm trying to implement a variation ratio, and I need T samples from an array C, but each sample has different weights p_t.

I'm using this:

import numpy as np
from scipy import stats

batch_size = 1
T = 3
C = np.array(['A', 'B', 'C'])
# p_batch_T dimensions: (batch, sample, class)
p_batch_T = np.array([[[0.01, 0.98, 0.01],
                       [0.3,  0.15, 0.55],
                       [0.85, 0.1,  0.05]]])

def variation_ratio(C, p_T):
  # This function works only with one sample from the batch.
  Y_T = np.array([np.random.choice(C, size=1, p=p_t) for p_t in p_T]) # vectorize this
  C_mode, frecuency =  stats.mode(Y_T)
  T = len(Y_T)
  return 1.0 - (f/T)

def variation_ratio_batch(C, p_batch_T):
  return np.array([variation_ratio(C, p_T) for p_T in p_batch_T]) # and vectorize this

Is there a way to implement these functions with any for?

Solution

In stead of sampling with the given distribution p_T, we can sample uniformly between [0,1] and compare that to the cumulative distribution:

Let's start with Y_T, say for p_T = p_batch_T[0]

cum_dist = p_batch_T.cumsum(axis=-1)

idx_T = (np.random.rand(len(C),1) < cum_dist[0]).argmax(-1)
Y_T = C[idx_T[...,None]]
_, f = stats.mode(Y_T) # here axis=0 is default

Now let take that to the variation_ratio_batch:

idx_T = (np.random.rand(len(p_batch_T), len(C),1) < cum_dist).argmax(-1)

Y = C[idx_T[...,None]]

f = stats.mode(Y, axis=1)   # notice axis 0 is batch

out = 1 - (f/T)

Answered By - Quang Hoang