Selection bias in research

Selection bias can arise when the examples you inspect differ systematically from the population you want to understand. Choosing only memorable moves is one way to create it. Repeating a calculation exactly does not fix that choice.

Start with the full question

State the markets, dates, setup, outcome horizon and baseline. Keep a record of the alternatives you tried. A condition chosen after hundreds of threshold changes has a different evidential status from one specified before examining outcomes.

What the product protects

Setup-first queries distinguish conditions at an anchor from outcomes measured afterward. Counts carry eligible denominators, and missing measurements remain absent. The query grammar documents those boundaries.

These protections do not make all selection bias impossible. You can still choose a favorable market, time range, threshold or report to publish. An exact receipt makes that selected calculation inspectable; it cannot certify the selection was unbiased.

Starting from a move is discovery

Investigating a move or comparing chosen moments is useful for generating hypotheses. The examples were selected with knowledge of what happened. A subsequent scan over the same record is broader evidence, but is not an independent validation of the hypothesis that record suggested.

Test on a separate time period or population where practical. Inspect dependence between nearby events and markets, realistic trading costs, uncertainty and contradictory examples before drawing conclusions.

What to report

Keep the full result, denominator, exclusions, baseline and receipt. Distinguish exploratory choices from choices fixed before evaluation. A roughly even up/down split alone does not prove a condition has no information: outcome magnitude and comparison with the baseline also matter.

Read an outcome distribution or open the Research Workbench to formulate an explicit question.