Wednesday, June 3, 2015

The Problem with Curation

Recently I received a question from a user asking about "cabal hell" when installing one of my packages. The scenario in question worked fine for us, but for some reason it wasn't working for the user. When users report problems like this they usually do not provide enough information for us to solve it. So then we begin the sometimes arduous back and forth process of gathering the information we need to diagnose the problem and suggest a workaround or implement a fix.

In this particular case luck was on our side and the user's second message just happened to include the key piece of information. The problem in this case was that they were using stackage instead of the normal hackage build that people usually use. Using stackage locks down your dependency bounds to a single version. The user reporting the problem was trying to add additional dependencies to his project and those dependencies required different versions. Stackage was taking away degrees of freedom from the dependency solver (demoting it from the driver seat to the passenger seat). Fortunately in this case the fix was simple: stop freezing down versions with stackage. As soon as the user did that it worked fine.

This highlights the core problem with package curation: it is based on a closed-world assumption. I think that this makes it not a viable answer to the general question of how to solve the package dependency problem. The world that many users will encounter is not closed. People are constantly creating new packages. Curation resources are finite and trying to keep up with the world is a losing battle. Also, even if we had infinite curation resources and zero delay between the creation of a package and its inclusion in the curated repository, that would still not be good enough. There are many people working with code that is not public and therefore cannot be curated. We need a more general solution to the problem that doesn't require a curator.