Comparison of Methods for Estimating Correctly Specified and Misspecified Linear Regression Models

Abstract

If a probability model of observed data is misspecified, then the interpretation of its Parameter estimates May be invalid, leading to incomplete or incorrect conclusions. The comparison of Weighted Maximum Likelihood (WLE) and Conditional Maximum Likelihood Estimates (CMLE) on their predictive performance when applied to correctly specified and misspecified linear regression was extensively studied. To carry out this analysis, we utilized simulated data that effectively mimicked various scenarios, allowing us to compare the performance of these estimators under small and large sample sizes. Specifically, we focused on the Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) as our primary metrics for evaluating the efficiency of the estimated models, as these statistics provide a clear understanding of the average prediction error associated with each method. The results of our study revealed a significant trend: the WLE consistently outperformed the CMLE across all tested conditions. Notably, it exhibited the least MSE for linear regression models characterized by one dependent and one independent variable, irrespective of whether smaller sample sizes or larger ones were utilized for the comparison. Moreover, when we extended our analysis to multiple linear regression scenarios, the superiority of the WLE remained evident, reinforcing its position as a more efficient estimator than its counterpart. Furthermore, we took a step further by applying both methodologies to real-life data sets. The findings demonstrated that the advantages of the WLE were not constrained to theoretical or simulated environments but also persisted in practical applications, further validating its efficiency and robustness.