Django in 24 hours

Last Saturday afternoon I decided to learn Django. It was 2pm on the first day of SiCamp 2008 in London and being the only developer in the room at that point I decided that I should do whatever I felt would be the best option to get an application running by 2pm the next day.

Previously I have done some Google App Engine and the experience convinced me to give Django a go after I found myself, by intuition, creating a GAE project structure of handlers.py (views), models.py and a directory called templates that contained templates. I was then disappointed to find the whole world had got there before me.

So, Django in 24 hours, baptism of fire. What do I think now looking back on the experience?

Everyone has told me that the Django documentation is good and I think I have to concur. Not everything is so clear that when you’re speed reading in one window and typing in another it works first time but importantly nothing in the documentation is actually wrong. When stuff is not working a second, careful look at the documentation got me back on track.

Importantly Django’s core model of web development is sound and intuitive. My editor had around ten files open for the project and the flow of adding something to the application did naturally flow from url to handler to view to model. Maybe the only quibble I have is that the views.py file is deceptively named in MVC terms.

The core of the framework is amazingly concise, I spent the majority of my time thinking about the problem not about the framework API. Binding a URL to function made sense, having to specify a template instead of having one inferred from the method name was maybe my one criticism in the method handler but on the plus side it does allow for flexibility in handling requests. Handing off from one request handler to another was very easy.

Django templates are both amazing and annoying. The syntax and principles are amazing, it was easy to play around with the pages and the template inheritance was really powerful for avoiding duplication. However when I transferred the application from self-serving to mod_python the template generation was very wobbly when compiling changes from the file system. Of course this could also have been mod_python but it was the latest 3 series stable source compiled for the machine. I’ve used Jinja2 previously when generating HTML in Python and might be tempted to stick with it in future.

Django models are great, I hate ORM but I really liked syntax for defining persistence properties and I liked the way that you don’t have the fact that you are really dealing with SQL hidden away from you. It genuinely seemed a more convenient way of expressing the data model rather than an OO wallpaper over relational data storage. I didn’t feel the need to add domain logic to the models but I felt like it wasn’t really polluting the model to do that either.

One thing that didn’t work at all was changing the relations between models; it took me two or three attempts to finally model the relationships between the data concepts. Each time I changed a Foreign Key or Many to Many relationship I ended up deleting the database (SQLite3) as I couldn’t figure how to migrate from the old schema to the new.

One reason for choosing Django was the idea that I wouldn’t have to write the backend code as the admin stuff would be right there for me. It took me a while to get the 1.0 admin to fire up but once it was running it did perform as advertised. One of the attractive things about the application was that the data model followed the conceptual language of the solution in a really powerful way. You could use the admin interface to have a Devotee perform Devotion to an Action. My geek excitement peaked anyway, YMMV.

So them’s the highlights of the experience. Overall Django delivered me a rapid web development process in an intuitive, powerful way and lived up to nearly all of the claims made on the tin. Deploying to Apache/mod_python was painful but most of the pain surrounded the infrastructure of my box (multiple versions of Python, Apache config files) and my lack of mad Apache admin skillz.

I would happily tackle another project in it again.

Perhaps of interest is how the Django development experience matches up against Rails or GAE  which would have been the other obvious choices. GAE would have been very similar but the deployment would have been better and I wouldn’t have had any automatic admin. In retrospect it may have been a better choice for a hack party type event. It certainly would have been my choice for personal projects for easy of deployment but now I have one Django app running perhaps that isn’t as relevant any more. Certainly the thing that has kept me from GAE before, the pain of data migration, doesn’t seem that better in Django (except that you control the datastore and its contents).

Compared to Rails?

• Admin is much more awesome than scaffolding.
• Django ORM is much less complex than Active Record, all the data required to create, deploy and use the object is in one place. Django doesn’t have Migrations but has its own brand of database versioning pain.
• RSpec is awesome (despite its monkey patching of Object) so you aren’t going to beat Rails for easy testing.
• Django templates are more powerful and easier to use than Erb but you have a lot of Ruby templating options so it’s hard to make a complete comparision. They probably both similar in the sense that you can find a template library that suits your preferences. Django is the purely solution out of the box.
• Routing and Controllers are much less involved in Django than Rails.
• Django is less opinionated about how you structure your application directories, which I like.
• Django doesn’t bake-in AJAX components but is “batteries included”, Rails probably generates better Web2.0 style apps for less effort.
• Finally Django uses only a few code generators because its basic structure is far less involved. It also generates far less “stuff” for each MVC element which I quite like as I don’t tend to use everything Rails generates.

Okay detailed analysis over, what’s the high-level view? Django and Rails are similar experiences but I think the major differences between them are almost what you could say about Python and Ruby. In Django you are going to get simplicity, clarity and a real choice of how you plug your infrastructure components together. In Rails you are going to get magic up front which is cool but also magic at the back end, which is not cool. Ultimately I think the answer is how opinionated you like your software. Well punk? How opinionated do you like it?

String Templates, or what I learned from Python and doing nothing

It’s an ill wind that blows no-one any good. The same is true of projects (although money generally helps more here; it’s an ill project that is making no-one any money).

I’m currently meant to be doing some work on Accessibilty for some new HTML pages. I thought it would be pretty easy but I was really wrong and it is changing the whole way I look at the View part of the (deceased) MVC web paradigm.

On my last project I was looking at things like Groovy’s Markup Builder and marvelling at how my collegues managed to put together a 30 line Freemarker template that did some pretty compex HTML assembly. In my spare time I have been looking at Haml as a way of escaping the verbosity and monotony of XHTML and to have the code guarantee the correctness of my page structure to avoid validation grind.

That’s because in those projects I was a developer/web designer. I wanted accurate, compliant HTML with minimum effort and which was easy to style without having awful CSS hacks.

On my current project I’m in the utterly baffling (for me anyway) world of .Net. There is no way that I can understand the huge variety of C#, XML and templating overrides that make up my current project. Having code generate HTML is a massive barrier to me being productive because, while I know a far bit of HTML having to root around an entire Visual Studio project to find the fragment that generates the problematic Div element you actually want to work with means I spend the whole day knowing nothing about .Net rather than applying the knowledge I do have.

Now some people are going to say that having a wacky Component model is different from having a nice templating language but look at something like Haml or Freemarker. The former is concise and fun and full of obscure rules; the latter is tremendously powerful, more firmly rooted in HTML and not much less obscure. For power users I agree, they are the bomb. They are a massive barrier to entry though, in a way that HTML just isn’t. People may do HTML badly but they rarely don’t do it at all.

This experience put combined with using Django/Jinja/Google App Engine is leading to me have a huge rethink about the way templating and views are put together. Passing a map of parameters to a template that is essentially exactly the way the output will look on the final device is obviously the way this problem should be tackled.

To try and get the HTML to generate in the current project I spent a day trying to get: SQLServer, BizTalk, Active Directory and Windows MQ to work together. This is utter madness and can only have been created by programmers who have no idea how to collaborate with non-programmers.

Why should I be trying to install BizTalk when what I want to do is actually generate some sample HTML so we can have a quick check of WAI standards? It should be possible to define some fixture data and then just generate HTML from the templates. It really shouldn’t be hard.

This experience is really changing the way I think about web frameworks. I am already determined to learn String Template, then I am going to look at whether my current favourite frameworks allow me to use it. I’m going to look at frameworks that ask you to put the HTML template next to the Java code. I want to know if I can put those templates in the same heirarchy that the actual website uses.

In short if I need to work with people outside the project team on a web project again, how can I get all the good things about templates and combine them with both simplicity and intuition as to how a website is organised?

The death of MVC

The MVC pattern is so embedded into the concept of modern web development I feel quite the heretic for declaring it over. Yet more and more I think we are moving away from it as a pattern. Views have been boiled down to a special case of templating and now Controllers are the next under the microscope. What does a Controller do? Well it marshalls the model and exposes it to the view.

However with the relentless march of REST how much controlling is the Controller really doing? The HTTP request tells you the format of the data, identifies the resource it is interested in. How much need is there for a controller for each request type? Surely the Uber Controller that responds to the HTTP request is all that’s required.

I have also been using Groovlets recently and when using them I feel like “why not mix your model lookup with the ‘view’”. In my Groovlets I essentially lookup the data for the view either directly via Groovy SQL or via the Service layer that is injected out of Guice. The view is then created using Markup Builder.

Since my scripts end up at around 50 lines of code I think that any benefit I might have in separating things is outweighed by the fact that the entire interaction is in one place and can be found, read and changed very easily.

MVC saved us from really painful web architectures but as we grow more sophisticated in the way we handle HTTP Requests and the more we understand the implications of HTTP and the less ceremonial our languages becomes the less benefit we get from it.

Towards Groovier Projects

My latest project has witnessed an influx of Groovy. The project buildfile is run by Gant, there are Groovlets providing lightweight pages but it is in the test folders that the Groovy has made its most insidious advances.

Markup builder makes fragments of HTML to test, Groovy’s built-in Sql is beginning to setup and check the results of data operations at a functional level. Soon I plan to implement a Groovy Struts 2 result and then it will begin to replace Freemarker.

The Pain of Rails

There are a lot of cool applications written in Rails but I often find that the configuration and setup is such a pain that you quickly lose the desire to run them. There’s only so much moaning about some library interface or missing gem or obscure data driver error you can take.

This kind of flakiness (particularly around the database area) has been driving me back to Java applications and alternatives such as Grails. It has also made me look at other frameworks than Rails. Since JRuby 1.1 was released JRuby has often been easier and faster to use than MRI Ruby.

Yesterday I had a kind of typical example of the kind of thing I find a pain. I had setup a new Ruby instance on Hardy Heron and was going to give Mephisto a go with SQLite3. I could do with an easy to use CMS and Ola Bini had already Tweeted that the setup had been easy under JRuby.

So having downloaded a few gems and the code, copied and set the database configuration I give the server a run and get an error about not being able to include the “initializer” file. Now this foxed me and I double-checked the documentation on installing Mephisto and tried to install an Initializer gem if there was one and I had a look at the boot.rb and saw that it was indeed trying to require initializer.

The problem is that when I had initially installed my gems I had forgot to install Rails itself. Nowhere was there a simply message telling me that though! Instead you have to kind of know that Initializer is the first script that Rails runs (and indeed that it is part of Rails).

There is a lot of Rails problems that I am sure are simple for people who work with it all the time but often I find myself coming with a task I want to do and coming away with a tussle with Rails instead.

Guicing Struts 2

Look how deceptively easy the Guice User Manual makes adding Guice to a Struts 2 app appear. Look, it says, just add one jar and all your dependency injection needs will be meet. Of course what it fails to tell you is that in addition to adding the plugin JAR, you also need the base jar, the servlet jar and if you want it to work you need to download a fixed plugin jar. At one point it was moaning about commons-logging as well!

With all that done though, it does indeed work as advertised.

Struts 2 and JSTL EL Quirk

This is quite an odd one. Struts 2 comes with it’s own tag library for accessing properties of actions but I thought it would be interesting to see if you can access it via JSTL EL instead. In the Hello World example I naively tried ${name} and ${requestScope.name}. Result: nada, as might be expected.

However if you reload the page (and repost the request) then suddenly the EL is populated! I presume that this is some artefact of re-executing the Action but it is very strange!

Update: this page set me right on how to mix JSTL EL and the Struts 2 Value Stack but I am still slightly mystified by what appears in which scope when the Result is returned.