Sunday, July 13, 2014

Haskell Best Practices for Avoiding "Cabal Hell"

I posted this as a reddit comment and it was really well received, so I thought I'd post it here so it would be more linkable.  A lot of people complain about "cabal hell" and ask what they can do to solve it.  There are definitely things about the cabal/hackage ecosystem that can be improved, but on the whole it serves me quite well.  I think a significant amount of the difficulty is a result of how fast things move in the Haskell community and how much more reusable Haskell is than other languages.

With that preface, here are my best practices that seem to make Cabal work pretty well for me in my development.

1. I make sure that I have no more than the absolute minimum number of packages installed as --global.  This means that I don't use the Haskell Platform or any OS haskell packages.  I install GHC directly.  Some might think this casts too much of a negative light on the Haskell Platform.  But everyone will agree that having multiple versions of a package installed at the same time is a significant cause of build problems.  And that is exactly what the Haskell Platform does for you--it installs specific versions of packages.  If you use Haskell heavily enough, you will invariably encounter a situation where you want to use a different version of a package than the one the Haskell Platform gives you.

2. Make sure ~/.cabal/bin is at the front of your path.  Hopefully you already knew this, but I see this problem a lot, so it's worth mentioning for completeness.

3. Install happy and alex manually.  These two packages generate binary executables that you need to have in ~/.cabal/bin.  They don't get picked up automatically because they are executables and not package dependencies.

4. Make sure you have the most recent version of cabal-install.  There is a lot of work going on to improve these tools.  The latest version is significantly better than it used to be, so you should definitely be using it.

5. Become friends with "rm -fr ~/.ghc".  This command cleans out your --user repository, which is where you should install packages if you're not using a sandbox.  It sounds bad, but right now this is simply a fact of life.  The Haskell ecosystem is moving so fast that packages you install today will be out of date in a few months if not weeks or days.  We don't have purely functional nix-style package management yet, so removing the old ones is the pragmatic approach.  Note that sandboxes accomplish effectively the same thing for you.  Creating a new sandbox is the same as "rm -fr ~/.ghc" and then installing to --user, but has the benefit of not deleting everything else you had in --user.

6. If you're not working on a single project with one harmonious dependency tree, then use sandboxes for separate projects or one-off package compiles.

7. Learn to use --allow-newer.  Again, things move fast in Haskell land.  If a package gives you dependency errors, then try --allow-newer and see if the package will just work with newer versions of dependencies.

8. Don't be afraid to dive into other people's packages.  "cabal unpack" makes it trivial to download the code for any package.  From there it's often trivial to make manual changes to version bounds or even small code changes.  If you make local changes to a package, then you can either install it to --user so other packages use it, or you can do "cabal sandbox add-source /path/to/project" to ensure that your other projects use the locally modified version.  If you've made code changes, then help out the community by sending a pull request to the package maintainer.  Edit: bergmark mentions that unpack is now "cabal get" and "cabal get -s" lets you clone the project's source repository.

9. If you can't make any progress from the build messages cabal gives you, then try building with -v3.  I have encountered situations where cabal's normal dependency errors are not helpful.  Using -v3 usually gives me a much better picture of what's going on and I can usually figure out the root of the problem pretty quickly.

12 comments:

Pse said...

Thank you. I was recently put off from a simple project using Haskell because of "cabal hell" when preparing for deployment. Hopefully, with your tips, this won't happen again in the future :)

Ian Zimmerman said...

Hi, I am not sure what you mean with points 1 and 3. WDYM "I install
ghc directly"? Do you just download and untar the binary blob? Do you
know that the libc version problem doesn't apply to you?

About happy and alex, again what does "manually install" mean? Download
binaries, compile from source, or cabal install?

mightybyte said...

Ian,

Yes, I mean download the binary from here and install it using the built-in installer. I have never encountered the libc version problem in practice. If you have a problem there, then you could still download and build from source.

I'm talking about the two packages happy and alex on hackage. You manually install them by doing "cabal install happy" and "cabal install alex".

Ian Zimmerman said...

"built-in installer" - hmm, let me guess, you're on MacOS.

I'm on Debian, and the officially packaged ghc version there is the very
bleeding-edge 7.4.1 :-p

I don't want to diss you by any means, you mean well and are trying to
help - thank you for that. But for quite some time I have had the
feeling that the Haskell (or rather ghc) model is deeply at odds with
binary package distros like Debian, and most instances of Cabal Hell are
just a symptom of this dissonance. Which is really sad for me because
I love Haskell the language and would like to use it, but you'll only
drag my cold dead body from Debian.

mightybyte said...

I don't think it's just GHC or "the Haskell model" that's at odds with binary packages. I think it's the speed that the ecosystem moves. It's a young community and there has been a LOT of activity over the last few years. I never use OS-packaged Haskell packages because that's simply too slow. Sometimes I'm actually waiting on the maintainer to merge my patches, and when they merge and upload to hackage I don't want to be waiting on someone else get that into the OS packages.

I switched to Arch Linux solely because of Haskell. I guess we all have our own priorities...

Ian Zimmerman said...

I also code patches and provide them to Debian. In such cases, I simply
build my own binary package with the patch. It is easy and quick
enough, _for a couple of packages_. But I get covered with cold sweat
at the thought of having to do that for _all_ packages, or even just all
Haskell libraries.


mightybyte said...

Of course, and understandably so. That just means that the binary distro model is not suitable for a fast moving ecosystem like Haskell. This isn't a problem with Debian or a problem with Haskell. It's a fundamental incompatibility.

Ian Zimmerman said...

Ok, let's suppose I am convinced and I want to try your way, at least to
see how long before I get disgusted. I downloaded the generic ghc
tarball and I unpacked it. ldd on the ghc binary seems to succeed, so
at least I don't expect any immediate ld.so hell :-) However, there is
no cabal binary. I assume I need it before I do anything else, so where
do I get it?

mightybyte said...

Googling "haskell cabal download" and clicking the first link takes you here. As you can see, there are three binary download links there, one for Mac, Windows, and Linux. If those don't work, you can download the source and run the bootstrap script that comes with it.

Paolo Giarrusso said...

I just tried out --allow-newer for a package that depended on pandoc-1.9 (yes, I know), and it has the problem I expected: it's recursive, and at least in this case it's not what you want (arguably, you want to allow-newer only at the top-level if just one package is broken). Cabal selected pandoc-1.12.4.2 but --allow-newer broke it, because it applied also to dependencies with perfectly fine settings — this pandoc wants texmath-0.6.7, cabal picked texmath-0.8, pandoc failed to compile. Relevant messages follow.

The error:

src/Text/Pandoc/Parsing.hs:174:8:
Could not find module ‘Text.TeXMath.Macros’
Perhaps you meant Text.TeXMath.Types (from texmath-0.8)
Use -v to see a list of the files searched for.
[...]

In this state, here's what cabal-install proposed:

$ cabal install -v --dry pandoc-1.12.4.2
Reading available packages...
Choosing modular solver.
Resolving dependencies...
In order, the following would be installed:
texmath-0.6.7 (latest: 0.8) (new version)
pandoc-1.12.4.2 (latest: 1.13.1) (new package)

$ cabal install pandoc-1.12.4.2 --dry --allow-newer
Resolving dependencies...
In order, the following would be installed (use -v for more details):
pandoc-1.12.4.2 (latest: 1.13.1)

mightybyte said...

Paolo,

You can narrow the behavior to specific packages with --allow-newer=text.

Paolo Giarrusso said...

Thanks for the extra tip!