Is it possible to predict malfeasance in public procurement? With the proliferation of e-procurement systems in the public sector, anti-corruption agencies and watchdog organizations have access to valuable sources of information with which to identify transactions that are likely to become troublesome and why. In this article, we discuss the promises and challenges of using machine learning models to predict inefficiency and corruption in public procurement. We illustrate this approach with a dataset with more than two million public procurement contracts in Colombia. We trained machine learning models to predict which of them will result in corruption investigations, a breach of contract, or implementation inefficiencies. We then discuss how our models can help practitioners better understand the drivers of corruption and inefficiency in public procurement. Our approach will be useful to governments interested in exploiting large administrative datasets to improve the provision of public goods, and it highlights some of the tradeoffs and challenges that they might face throughout this process.