Log-transformation and its implications for data analysis

By C. Feng, Hongyue Wang, N. Lu, Tian Chen, Hua He, Ying Lu, Xin M. Tu. Shanghai Archives of Psychiatry, 2014

843 citations recorded by the source corpus.

Summary The log-transformation is widely used in biomedical and psychosocial research to deal with skewed data. This paper highlights serious problems in this classic approach for dealing with skewed data. Despite the common belief that the log transformation can decrease the variability of data and make data conform more closely to the normal distribution, this is usually not the case. Moreover, the results of standard statistical tests performed on log-transformed data are often not relevant for the original, non-transformed data.We demonstrate these problems by presenting examples that use simulated data. We conclude that if used at all, data transformations must be applied very cautiously. We recommend that in most circumstances researchers abandon these traditional methods of dealing with skewed data and, instead, use newer analytic methods that are not dependent on the distribution the data, such as generalized estimating equations (GEE).

Subfields: Management Science and Operations Research, Plant Science, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/76cffd7e71817d42b1bd001d97657358b8fca935

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

1. J. P. Ekwaru
2. P. Veugelers
3. Guangchuang Yu

In Plant Science

1. Rasha Alsaadawi
2. K. Kheirallah
3. Roy T. Sabo
4. Guangchuang Yu
5. Hanan M. Hammouri

In Statistics and Probability

1. Li Zhan
2. Tingze Feng
3. Meijun Chen
4. Lang Zhou
5. Rasha Alsaadawi
6. K. Kheirallah
7. Roy T. Sabo
8. Shuangbing Xu

Citations in the corpus

• The Overlooked Importance of Constants Added in Log Transformation of Independent Variables with Zero Values: A Proposed Approach for Determining an Optimal Constant (2018, background): “However, research has shown that the value of the constant that is added affects model parameters (Johnson and Rausser 1971) and statistical inference (Feng et al. 2013, 2014).”
• COVID-19: fear appeal favoring purchase behavior towards personal protective equipment (2020, methodology): “We stated the continuous variables in terms of logarithmic functions to avoid data misrepresentations, typical about continuous and community behavior datasets (Bründl & Thomas, 2006; Feng et al., 2014), and also to smooth-edge high values (Feng et al., 2014).”
• Handling Skewed Data: A Comparison of Two Popular Methods (2020, methodology): “In addition, the use of log transformations can lead to mathematical errors and misinterpretation of results [6,8].”
• Use ggbreak to Effectively Utilize Plotting Space to Deal With Large Datasets and Outliers (2021, background): “problems that are even more difficult to deal with (Feng et al., 2014).”