Haskell Scripting: Log Analysis

The other day I wanted to analyze some of my website log files to get a better idea of how many active users I have. I've been meaning to do this for quite some time, but have kept putting it off. I decided to see what I could accomplish by just doing some experimenting in GHCI. It was so easy and convenient that I decided to do a screencast demonstrating what I did and how easy it was. The conciseness of Haskell combined with the instant feedback of an interpreter make a very powerful combination. Here's the screencast on vimeo.

Comments

Anonymous said…
This would be equally easy with a "popular OO language" like Ruby. Never tried Python, but I'm confident it would be the same.

(Compiled OO languages are another story obviously... C++, Java etc.)

But it's not the kind of thing I'd do in any interpreter, to be honest; I'll stick with Webalizer.
June 7, 2010 at 10:22 AM
T_S_ said…
The utility of Data.Function, groupBy, nubBy were the takeaways for me. Thanks!. Maybe say a few words about them next time.
June 8, 2010 at 10:15 AM
Mike Machenry said…
I agree with T_S_, I'd never seen groupBy and nubBy before. Anyway, nice video. Thanks.

-mike
June 8, 2010 at 11:46 AM
Louis said…
Obviously this analysis is pretty easy to do it with any scripting language, but I think the point is haskell is able to easily do it too (and so to break some "prejugés")

it fits well the title of the book: real world haskell
June 8, 2010 at 6:32 PM
joe said…
I love this vid. Great intro to usable simple powerful haskell without all the type business and complexity. How does one take it to the next level without bringing in a lot of complexity: simple program to find most frequent users, most common users to generate error pages, etc.

Great vid! Thanx.
March 16, 2011 at 1:50 PM
mightybyte said…
joe,

Glad you liked it. In my experience the best way to learn more is to just write more Haskell code. Learning more of the functionality provided by the standard libraries will be helpful. Data.List is the obvious place to start.

I would suggest just trying to solve whatever problems jump out at you. You can play around in ghci, but I think you'll quickly find that you want to load code from an external file so you can more easily define your own functions.

If you're having trouble with something and it just seems like there should be a better way, there probably is. When that happens try searching on Google or ask on the #haskell IRC channel. I've found the real-time nature of IRC and the extreme friendliness of the people in #haskell very helpful.
March 16, 2011 at 2:33 PM
Anonymous said…
I really liked a lot your screencast.

I'm a total noob on it. But decided to get into it just to do something like that you showed us in the screencast.

It's great to see it in action. Thanks.

Where could I find some more screencasts like this?
April 22, 2011 at 1:31 PM
mightybyte said…
Anonymous,

Thanks. It's always nice to hear that other people have found it useful. I don't really know of much else in the way of Haskell screencasts, so Google is pretty much the best place I can point you to.
April 22, 2011 at 1:42 PM
Post a Comment

Popular posts from this blog

A Hopefully Fair and Useful Comparison of Haskell Web Frameworks

Recently there has been a lot of discussion and questions about the differences between the big three Haskell web frameworks: Happstack, Yesod, and Snap. Different pieces of the answer have been discussed in a number of places. In this post, I'd like to try to give a more complete comparison. Hopefully it will be relatively unbiased, but without being so watered down that it is useless. I've succeeded if you can't tell which framework I'm a major contributor to based solely on the text of this post. Happstack First, up is Happstack , the oldest of the three with original commit history as early as February, 2005. Happstack used to be called HAppS . It was primarily composed of a web server based on lazy IO, a fairly radical (for the time) in-memory persistence approach for storing and versioning native Haskell data types, and a library called IxSet for making it easier to use this state system with relational data access patterns. I've heard a number of pe
Read more

Setting Up A Private Nix Cache

I recently went through the process of setting up a private Nix binary cache. It was not obvious to me how to go about it at first, so I thought I would document what I did here. There are a few different ways of going about this that might be appropriate in one situation or another, but I’ll just describe the one I ended up using. I need to serve a cache for proprietary code, so I ended up using a cache served via SSH. Setting up the server For my cache server I’m using an Amazon EC2 instance with NixOS. It’s pretty easy to create these using the public NixOS 18.03 AMI. I ended up using a t2.medium with 1 TB of storage, but the jury is still out on the ideal tradeoff of specs and cost for our purposes. YMMV. The NixOS AMI puts the SSH credentials in the root user, so log in like this: ssh -i /path/to/your/key.pem root@nixcache.example.com To get your new NixOS machine working as an SSH binary cache there are two things you need to do: generate a signing key and turn on cache serv
Read more

Dependent Types are a Runtime Maybe

Awhile back I was discussing dependent types with someone and we ended up concluding that dependent types can always be replaced by a runtime Maybe.  This seemed to me then (and still does today) as a fairly surprising and provocative conclusion.  So I thought I'd put the idea out there and see what people think. Let's look at a few examples that are commonly used to illustrate dependent types: Vectors of length N Matrices of size m x n Sorted lists Height-balanced trees (trees where the height of subtrees differ by at most one) The argument is roughly as follows.  All of these examples ultimately boil down to enforcing some kind of constraint on some data types.  The more powerful your dependent type system, the more rich and expressive will be the constraints that you can enforce.  If we take this thought experiment to its logical conclusion, we end up with a dependent type system that allows us to enforce any constraint that can be computed. The important realization here is
Read more