intra-scientist irreproducibility missing: systematic approach to reduction/analysis e.g., paper trail for analysis train, especially for bugs what we don't yet require of ourselves and each other: keep master script of analysis results keep intermediate forms of data keep notebook of what was done version control for scripts and notebook ways to visualize what's changed in analysis results after re-run (recently funded meta-repository project will try to faciliate the cultivation of a culture of calibration. long road.)