[FIXED] Implementing AIC score for multiple linear regression manually

Issue

I have implemented a multiple linear regression class by hand and right now I am working on the metrics methods. I have tried to calculate the AIC and BIC scores manually, but the results weren't correct. The reason is that I didn't use the Log Likelihood function but went with the SSE approach. Could you please suggest me how to change the implementation to compute the full AIC and BIC scores?

Here is how my method looks like right now:

  def AIC_BIC(self, actual = None, pred = None):
    if actual is None:
      actual = self.response
    if pred is None:
      pred = self.response_pred

    n = len(actual)
    k = self.num_features

    residual = np.subtract(pred, actual)
    RSS = np.sum(np.power(residual, 2))

    AIC = n * np.log(RSS / n) + 2 * k
    BIC = n * np.log(RSS / n) + k * np.log(n)

    return (AIC, BIC)

And please try to give me a manual approach and not a library call. Thanks!

Solution

I managed to solve the issue with the help of @James. Here is how my new implementation looks like:

 def LogLikelihood(self):
     n = self.num_obs
     k = self.num_features
     residuals = self.residuals

     ll = -(n * 1/2) * (1 + np.log(2 * np.pi)) - (n / 2) * np.log(residuals.dot(residuals) / n)

     return ll

  def AIC_BIC(self):
    ll = self.LogLikelihood()
    n = self.num_obs
    k = self.num_features + 1

    AIC = (-2 * ll) + (2 * k)
    BIC = (-2 * ll) + (k * np.log(n))

    return AIC, BIC

I implemented a log-likelihood calculation and used it in the formula which can be found on Wikipedia.

Answered By - OwlOnMoon