Criminal justice policies have been changing rapidly in a number of Australian jurisdictions, often driven by political rhetoric and expediency. In some cases, these policies have been implemented in the absence of robust evidence as to their effectiveness. But in this era of tough-on-crime political rhetoric and ever increasing (and ever more costly) prison populations, it is critical to have a clear understanding of what works in preventing crime. This paper presents an analysis of data from NSW to examine a number of methodological approaches to measuring reoffending, comparing and contrasting their results. It aims to show how different methodologies affect results, and to identify the implications for policy-makers. The paper concludes with a discussion of the importance of robust research to encourage the development of evidence-based policy rather than policy-based evidence.