Problem implementing Softmax Regression (Multinomial Logistic) · Issue #1004 · pymc-devs/pymc (original) (raw)

I am trying to implement a logistic multinomial regression (AKA softmax regression). In this example I am trying to classify the iris dataset.

I have a problem specifying the model, I get an optimization error with find_MAP(). If I avoid using find_MAP() I get a "sample” of all zero vectors for alfa and beta and a vector of 0.33 for p if I use a Categorical for the likelihood. if I use Mutinomial(n=1, p=p) the posterior is exactly the same as the priors.

import pymc3 as pm import numpy as np import matplotlib.pyplot as plt import seaborn as sns import pandas as pd

iris = sns.load_dataset("iris") y_2 = pd.Categorical(iris['species']).labels x_n = iris.columns[:-1] x_2 = iris[x_n].values x_2 = (x_2 - x_2.mean(axis=0))/x_2.std(axis=0) indice = list(set(y_2))

with pm.Model() as modelo_s: alfa = pm.Normal('alfa', mu=0, sd=100, shape=3) beta = pm.Normal('beta', mu=0, sd=100, shape=(4,3))

mu = (alfa[indice] + pm.dot(x_2, beta[:,indice])).T
p = pm.exp(mu)/pm.sum(pm.exp(mu), axis=0)

yl = pm.Categorical('yl', p=p, observed=y_2)
#yl = pm.Multinomial('yl', n=1, p=p, observed=y_2)

#start = {'alfa': np.random.rand(3), 'beta':np.random.rand(4,3)}
#start = pm.find_MAP(start)
#step = pm.NUTS(scaling=start)
start = pm.find_MAP()
step = pm.Metropolis()
trace_s = pm.sample(1000, step, start)

This is also an open question on stackoverflow