Evidence-Based Technical Analysis: Part 2 - Statistical Foundations of Technical Analysis and Data Mining

The episode offers an extensive examination of statistical inference as the foundation for evaluating Technical Analysis (TA) rules, arguing that only rigorous statistical methods can distinguish genuinely predictive TA techniques from those based merely on chance or luck . It introduces key statistical concepts such as the null hypothesis , which assumes a TA rule has no predictive power, and the use of the sampling distribution to quantify uncertainty and determine a rule's statistical significance through its p-value . A significant portion of the text addresses the pervasive problem of data-mining bias , showing through experimental examples that systematically searching many rules inflates observed performance, and explores methods like walk-forward testing and Monte Carlo permutation to mitigate this bias. Finally, the text explores theories of nonrandom price motion , challenging the assumptions of the Efficient Market Hypothesis (EMH) with evidence from behavioral finance, such as investor cognitive biases (e.g., anchoring and conservatism), and suggests that TA profits may be understood as compensation for risk, or a risk premium , within market structures like futures