There are a lot of reasons for choosing Jinja2 over Django for us. Better performance (atleast… it was a lot better with previous Django versions), way more options (named arguments, multiple arguments for filters, etc), macros and simply easier to extend. Writing custom tags is simply not needed anymore since you can just make any function callable from the templates.

But… during the conversion there are always moments when you need  a Django function in a Jinja template or vice versa. So… I created a few template tags to allow for Jinja code in Django templates (I’ve also created code to run Django code from Jinja, but I haven’t seen the need for it so I omitted it here).

A Jinja Include tag to include a template and let it be parsed by Jinja from a Django template:

from django import template
from coffin import shortcuts as jinja_shortcuts                                                                                                                                                      

register = template.Library()                                                                                                                                                                        

class JinjaInclude(template.Node):
    def __init__(self, filename):
        self.filename = filename                                                                                                                                                                     

    def render(self, context):
        return jinja_shortcuts.render_to_string(self.filename, context)                                                                                                                              

@register.tag
def jinja_include(parser, token):
    bits = token.contents.split()                                                                                                                                                                    

    '''Check if a filename was given'''
    if len(bits) != 2:
        raise template.TemplateSyntaxError('%r tag requires the name of the '
            'template to be included included ' % bits[0])
    filename = bits[1]                                                                                                                                                                               

    '''Remove quotes if used'''
    if filename[0] in ('"', "'") and filename[-1] == filename[0]:
        filename = bits[1:-1]                                                                                                                                                                        

    return JinjaInclude(filename)

Usage:

{% jinja_include "some_template.html" %}

A couple of noop nodes to make sure that when you convert your Jinja templates to be executed from Django, they won’t break because of the missing Django tag.

from django import template

class Empty(template.Node):
    def render(self, context):
        return ''                                                                                                                                                                                    

@register.tag
def django(parser, token):
    return Empty()                                                                                                                                                                                   

@register.tag
def end_django(parser, token):
    return Empty()

And the Jinja tag to allow Jinja blocks in Django templates.

from django import template
from coffin.template import Template

register = template.Library()

class Jinja(template.Node):
    def __init__(self, template):
        self.template = template                                                                                                                                                                     

    def render(self, context):
        return self.template.render(context)                                                                                                                                                         

@register.tag
def jinja(parser, token):
    '''Create a Jinja template block                                                                                                                                                                 

    Usage:
    {% jinja %}
    Although you're in a Django template, code here will be executed by Jinja
    {% end_jinja %}
    '''                                                                                                                                                                                              

    '''Generate the end tag from the currently used tag name'''
    end_tag = 'end_%s' % token.contents.split()[0]                                                                                                                                                   

    tokens = []
    '''Convert all tokens to the string representation of them
    That way we can keep Django template debugging with Jinja and feed the
    entire string to Jinja'''
    while parser.tokens:
        token = parser.next_token()
        if token.token_type == template.TOKEN_TEXT:
            tokens.append(token.contents)                                                                                                                                                            

        elif token.token_type == template.TOKEN_VAR:
            tokens.append(' '.join((
                template.VARIABLE_TAG_START,
                token.contents,
                template.VARIABLE_TAG_END,
            )))                                                                                                                                                                                      

        elif token.token_type == template.TOKEN_BLOCK:
            if token.contents == end_tag:
                break                                                                                                                                                                                

            tokens.append(' '.join((
                template.BLOCK_TAG_START,
                token.contents,
                template.BLOCK_TAG_END,
            )))                                                                                                                                                                                      

        elif token.token_type == template.TOKEN_COMMENT:
            pass                                                                                                                                                                                     

        else:
            raise template.TemplateSyntaxError('Unknown token type: "%s"' % token.token_type)                                                                                                        

    '''If our token has a `source` attribute than template_debugging is
    enabled. If it's enabled create a valid source attribute for the Django
    template debugger'''
    if hasattr(token, 'source'):
        source = token.source[0], (token.source[1][0], token.source[1][1])
    else:
        source = None                                                                                                                                                                                

    return Jinja(Template(''.join(tokens), source=source))

Do note that I have modified the “coffin.template.Template” to enable debugging completely. Just replace the “Template” class in “coffin/template/__init__.py” to make it work.

def _generate_django_exception(e, source=None):
    '''Generate a Django exception from a Jinja source'''
    from django.views.debug import linebreak_iter                                                                                                                                                    

    if source:
        exception = DjangoTemplateSyntaxError(e.message)
        exception_dict = e.__dict__
        del exception_dict['source']                                                                                                                                                                 

        '''Fetch the entire template in a string'''
        template_string = source[0].reload()                                                                                                                                                         

        '''Get the line number from the error message, if available'''
        match = re.match('.* at (\d+)$', e.message)                                                                                                                                                  

        start_index = 0
        stop_index = 0
        if match:
            '''Convert the position found in the stacktrace to a position
            the Django template debug system can use'''
            position = int(match.group(1)) + source[1][0] + 1                                                                                                                                        

            for index in linebreak_iter(template_string):
                if index >= position:
                    stop_index = min(index, position + 3)
                    start_index = min(index, position - 2)
                    break
                start_index = index                                                                                                                                                                  

        else:
            '''So there wasn't a matching error message, in that case we
            simply have to highlight the entire line instead of the specific
            words'''
            ignore_lines = 0
            for i, index in enumerate(linebreak_iter(template_string)):
                if source[1][0] > index:
                    ignore_lines += 1                                                                                                                                                                

                if i - ignore_lines == e.lineno:
                    stop_index = index
                    break                                                                                                                                                                            

                start_index = index                                                                                                                                                                  

        '''Convert the positions to a source that is compatible with the
        Django template debugger'''
        source = source[0], (
            start_index,
            stop_index,
        )
    else:
        '''No source available so we let Django fetch it for us'''
        lineno = e.lineno - 1
        template_string, source = django_loader.find_template_source(e.name)
        exception = DjangoTemplateSyntaxError(e.message)                                                                                                                                             

        '''Find the positions by the line number given in the exception'''
        start_index = 0
        for i in range(lineno):
            start_index = template_string.index('\n', start_index + 1)                                                                                                                               

        source = source, (
            start_index + 1,
            template_string.index('\n', start_index + 1) + 1,
        )                                                                                                                                                                                            

    exception.source = source
    return exception          

class Template(_Jinja2Template):
    """Fixes the incompabilites between Jinja2's template class and
    Django's.                                                                                                                                                                                        

    The end result should be a class that renders Jinja2 templates but
    is compatible with the interface specfied by Django.                                                                                                                                             

    This includes flattening a ``Context`` instance passed to render
    and making sure that this class will automatically use the global
    coffin environment.
    """                                                                                                                                                                                              

    def __new__(cls, template_string, origin=None, name=None, source=None):
        # We accept the "origin" and "name" arguments, but discard them
        # right away - Jinja's Template class (apparently) stores no
        # equivalent information.
        from coffin.common import env                                                                                                                                                                

        try:
            return env.from_string(template_string, template_class=cls)
        except JinjaTemplateSyntaxError, e:
            raise _generate_django_exception(e, source)                                                                                                                                              

    def __iter__(self):
        # TODO: Django allows iterating over the templates nodes. Should
        # be parse ourself and iterate over the AST?
        raise NotImplementedError()                                                                                                                                                                  

    def render(self, context=None):
        """Differs from Django's own render() slightly in that makes the
        ``context`` parameter optional. We try to strike a middle ground
        here between implementing Django's interface while still supporting
        Jinja's own call syntax as well.
        """
        if not context:
            context = {}
        else:
            context = dict_from_django_context(context)                                                                                                                                              

        try:
            return super(Template, self).render(context)
        except JinjaTemplateSyntaxError, e:
            raise _generate_django_exception(e)                                                                                                                                                      

def dict_from_django_context(context):
    """Flattens a Django :class:`django.template.context.Context` object.
    """
    if isinstance(context, DjangoContext):
        dict_ = {}
        # Newest dicts are up front, so update from oldest to newest.
        for subcontext in reversed(list(context)):
            dict_.update(dict_from_django_context(subcontext))
        return dict_
    else:
        return context

And you’re done, now you can just mix your Django and Jinja templates like this:

{% ifequal foo bar %}
Django style if...
{% endif %}

{% jinja %}
{% if foo == bar %}
Jinja style if...
{% endif %}
{% end_jinja %}

Usage is extremly straight forward:

from django_open_inviter.open_inviter import OpenInviter
o = OpenInviter()
contacts = o.contacts('example@example.com', 'test')

Get the code here.

Our button will be focusing on Fashiolista.com. Fashiolista is a social bookmarking site for fashion, which has seen rapid growth after launching at the next web. This tutorial explains the javascript (client side) aspects of the button. Feedback and improvements on the code would be greatly appreciated. You can find the full 450 lines of js on github.

This is what the end result looks like:

(If you are working on a shop in the fashion industry have a look at our installation instuctions.)

Step 1 – The markup

Its important to get the client side markup of the button right. Since other sites will be implementing this there is no way you can change it later on. The three major players each have their own way.

Facebook XFBML: Async script with XFBML or Iframe
Digg button: Async script with A elements
Tweetmeme: Normal script

<script type="text/javascript">
  //async script, fashiolista.com version
  (function() {
   var s = document.createElement('SCRIPT');
   var c = document.getElementsByTagName('script')[0];
   s.type = 'text/javascript';
   s.async = true;
   s.src = 'http://button.www.fashiolista.com/button/script/';
   c.parentNode.insertBefore(s, c);
  })();
</script>
<a class="fashiolista_button fashiolista_compact"
href="http://www.fashiolista.com/item_add_oe/">Love it!</a>

For Fashiolista we have chosen an async script approach with A elements. Normally loading a script element is a blocking operation for the browser. Loading the script async ensures faster page load times and a better experience if your site would ever go down. (Note that not all browsers support this option so it is still recommended to include the script tag at the bottom of the page). The function wrapped around the code ensures we don’t pollute the global scope. Furthermore the insertBefore in combination with a script tag technique is used by GA so should work in any scenario.

Step 2 – Creating the buttons, Iframe vs Script

The next step is to convert our A elements into actual buttons. We can choose to replace these A elements by our button’s html (digg, delicious approach) or load an iframe in their place (facebook, tweetmeme). The difference between these two approaches is actually pretty large. For Fashiolista you can see both an iframe and script approach. These are the most important differences I encountered.

Iframe vs Script

• + Popup communication possible
  The script approach cannot communicate with popups it creates due to the same origin restrictions. The iframe however can be of the same domain as the popup and freely communicate. This gives a better user experience when for instance logging in.
• + Easier to develop
  The iframe approach is easier to develop and requires less code.
• + Parallel download in IE
  IE doesn’t download the count scripts in parallel, but it does do so for the IFRAMEs. Making this approach somewhat faster.
• Independent CSS
  External sites don’t interfere with your button’s css if you use an iframe technique. The disadvantage is that it makes things likes hovers impossible to integrate with the other site. (For example Fashiolista’s compact button).
• Independent
  The iframe approach makes it very hard for other sites to game the users like/love action. With a script approach a foreign site can simply call your javascript to fake someone loving the product. This freedom can be abused but also allows for mashups.
• - Slower dom load
  Creating iframes takes a lot more time for the browser.
• - Slower perceived load
  The script approach allows you to format the buttons before the data is loaded. Vastly increasing the perceived load speed.
• - No shared functionality
  Buttons can’t share functionality. So when someone logs in for one button its is not possible to update the others.

The best choice differs for each project. For Fashiolista the more open script approach is currently the default.

Step 3 – Cross site scripting using JSONP

Essential to the bookmarking button is requesting the count for the given url. Cross site policies prevent us from using Ajax so we will do so by creating a script element.

_makeRequest: function (url) {
	//Simple create script element functionality
        var s = document.createElement('script');
        var b = document.body;

        s.setAttribute('type', 'text/javascript');
        s.setAttribute('async', 'true');
        s.setAttribute('src', url);

        b.appendChild(s);
}

The trouble with the script element is that you lack the nice APIs Ajax offers you. We work around this by using an url with a callback paramater, for example callback=button_loaded_3
The server side code then responds with something like this, executing the callback when the script is loaded.

button_loaded_3({"item_id": 26545, "url": "/item/26545/", "loves": 853})

This technique is often referred to as JSONP. We bind the response function to the global button_loaded_3 using the following code:

loadButtonInformation: function (buttonId) {
		//make a request to the script with the given callback
		var buttonInstance = this.buttonDict[buttonId];
		var buttonUrl = buttonInstance.lookupUrl;
		var path = '&url=' + encodeURIComponent(buttonUrl);
		var callbackFunctionName = 'button_loaded_' + buttonId;
		var scope = this;
		var callbackFunction = function(data) {
			//bind the scope and button id
			scope.buttonLoaded.call(scope, buttonId, data);
		};
		window[callbackFunctionName] = callbackFunction;
		this.makeRequest(this.countApi + path, callbackFunctionName, true);
}

Step 4 – Object oriented design

Since we are loading our code into someone else’s website we should be careful not to use similar variable names. We therefore hide as much code as possible in classes.

var fashiolistaClass = function(){ this.initialize.apply(this, arguments); };
fashiolistaClass.prototype = {
	//
	//Base class implementing the fashiolista button
	//
	initialize: function () {
		//load the buttons
		this.initializeCss();
		var fashiolistaButtons = this.findButtons();
		this.initializeButtons(fashiolistaButtons);
	}
}

Note that we are not simulating inheritance for these classes. Using them as simple namespaces is more than sufficient in this case.
The code is organized into 3 classes:

• fashiolistaClass
• fashiolistaUtilsClass
• fashiolistaButtonClass

The first one acts as a manager (finding the buttons, instantiating fashiolistaButtonClasses and retrieving counts). Fashiolista button contains the logic for individual buttons and fashiolista utils contains some string parsing and dom load functionality.

Step 5 – Caching requests in the google app engine

To prevent our servers from getting flooded we are routing all traffic through google servers using the google app engine. button.www.fashiolista.com is connected to a google app engine account which forwards and caches requests to fashiolista.com. This setup enables your button to withstand great amounts of traffic without killing your servers. Furthermore it immediately also acts as a cdn for our web requests, speeding up load times for our international visitors. Setting up caching in the google app engine would require another blog post though. Let us know in the comments if you would like to know more about it.

Conclusion

The full client side code can be found here. This blog post covered the most essential parts. Code review and questions are more than welcome. Be sure to let us know in the comments. Furthermore if you are running a webshop in the fashion industry consider implementing the button.

More information

• Server side standards – OExchange
• Installing the fashiolista button (for shops)
• Subscribe to this blog’s rss feed

Improvements/ Request for code review

• The domload technique is rather verbose, does anyone know a better method?
• The popup communication or lack thereof is not ideal for users, is there a better method?
• Script or Iframe what do you prefer?
• Suggestions to make it faster?

Fashiolista.com

Im releasing the source code for Django Facebook on github. Its a very early release, but it might help other developers trying to implement a facebook register/loging flow using the new open graph api. See Github for requirements and installation instructions.

Update:
Birthday formats are currently giving some troubles for some users.
Fixed and tests added to prevent future problems. (note Fashiolista.com may still give errors, will be resolved during our next release).

The conference itself should be very interesting. Definitely looking forward to Werner Vogels and Joe Stump. The later seems to be running Django for SimpleGeo.

Pretty please nominate youtellme.nl for the next web!
http://dsa.thenextweb.com/?lang=nl

Good articles coming soon to offset the bad karma for this shameless plug ;)

Update:

A couple of stats about the current codebase.

According to wc (including whitespace and comments) we currently have:

• Python: 3,363,188 characters, 87926 lines, 655 files
• Javascript: 940,536 characters, 24229 lines, 77 files

This solution chunks up the querying in bits of 1000 (by default). While this is somewhat heavier on your database (multiple queries) it seriously reduces the memory usage. Curious to hear how other django developers have worked around this problem.

import gc

def queryset_iterator(queryset, chunksize=1000):
    '''
    Iterate over a Django Queryset ordered by the primary key

    This method loads a maximum of chunksize (default: 1000) rows in it's
    memory at the same time while django normally would load all rows in it's
    memory. Using the iterator() method only causes it to not preload all the
    classes.

    Note that the implementation of the iterator does not support ordered query sets.
    '''
    pk = 0
    last_pk = queryset.order_by('-pk')[0].pk
    queryset = queryset.order_by('pk')
    while pk < last_pk:
        for row in queryset.filter(pk__gt=pk)[:chunksize]:
            pk = row.pk
            yield row
        gc.collect()

#Some Examples:
#old
MyItem.objects.all()

#better
MyItem.objects.all().iterator()

#even better
queryset_iterator(MyItem.objects.all())

Django snippet here.

Things are going well, looking very forward to international launch.
We’ve changed a lot since the first reviews!

Beter pictures coming after the event :P

PS. Thanks to Python and Django, for enabling us to beat the competition :)

PSS. Next2News, eduhub, come and join :)

charts

I was just browsing the code a bit. There is something peculiar there:

args, varargs, varkw, defaults = inspect.getargspec(func)

I don’t get the point of using the inspect functionality. Anyone care to explain?

Choosing Django:

Django has a few killer features which make it a better choice for many projects.

High Level Fields

As a starter there’s the usage of high level fields when describing your data model. This is best clarified by an example of a model definition in django:

class Author(models.Model):
   ip = models.IPAddressField()
   email = models.EmailField()
   company = models.ForeignKey(Company)
   picture = models.ImageField(upload_to='images/profile_pics', blank=True)
   homepage = models.URLField(verify_exists=True, blank=True)

Fields are specified by their purpose, such as Email, Url and Image. From this definition all subsequent logic such as form validation and file uploads are handled. The homepage field’s validation will even ping the url to see if it exists.

Read more: Creating Models (django project)

Form Handling

Starting with the knowledge that you have an email field you will often want a nice text input in your form with a regex to check if the email is valid. Django has all these standard use cases worked out for you. The following example clarifies this by using Django’s ModelForm. A model form is basically a normal Form class, with the fields pre-populated as one would expect it based on the given model.

#Form specification
class AuthorForm(ModelForm):
   class Meta:
      model = models.User
      fields = ('ip','email','picture','homepage', 'company')

#Using the form in the view (controller in Symfony terminology)
form = AuthorForm(request.POST, instance=author_instance)
if request.method == 'POST':
  if form.is_valid():
     form.save()
     return HttpResponseRedirect(request.path) 

#In the template (view in Symfony terminology)

	{{ uform.first_name.label_tag }}
: {{ uform.first_name }} {{ uform.first_name.errors }}

Django will display a file field for the image field, a text field for the url (with validation), a text input for the email field and a select box for the foreign key relation. Saving the result of the form to the database is as simple as calling save on the instance of the form. Writing custom widgets and field types is straightforward. Currently many localized fields such as a Dutch postal code Field and widgets are available. Symfony has been trying to emulate the Django newforms library. Unfortunately the syntax doesn’t seem very friendly. (Pity php doesn’t have metaclasses)

Read more: Forms (Django book)

Superb ORM

Probably the largest difference is caused by the ORM. In PHP both Propel and Doctrine are nice projects, but simply quite inadequate. The Django ORM is syntax heaven if you are coming from php. A small example:

#Find the first 5 authors which have a relation to a company
#with the name YouTellMe (exact match) and site url that contains youtellme.nl
Author.objects.filter(company__name = 'YouTellMe', company__site__icontains = 'youtellme.nl')[:5]

The possibilities of the standard Django ORM system are quite good. Your queries will be optimized into joins if you call select_related, many to many relations are supported and polymorphic keys are as well. The only part it fails at is query optimization in terms of column based lazy loading and support for complex relations. Fortunately you can fall back to using SQL alchemy, which is Python’s most prestigious ORM layer. SQL alchemy allows you exact query control for performance tuning and many more options you did not know you needed.
However it isn’t (yet, i hope) fully integrated into Django. It would really be great to see a tighter integration with SQL alchemy, but even the Django ORM strongly outperforms Doctrine and Propel.

Read more: DB api.

Python

You could see this both as an advantage or disadvantage. Discussing the differences between Python and PHP is probably best left for a later post. Suffice to say that I found my programming productivity to be substantially higher with Python compared to PHP. The disadvantage is a smaller number of available scripts, developers and hosts. If you are in a position in which you can choose, you really should give python a go. There must be a reason why YouTube and Google do ;)

Read more: Python in 10 Minutes

Speed

When arguing with my colleague about the choice of framework we conducted a speed test between Django and Symfony using Apache bench. At the time I was arguing in favor of Symfony. We compared a lightweight PHP framework, with Symfony and Django. Symfony was stripped down for performance and was only about 30% heavier compared to the lightweight framework. When comparing it to Django however, the results showed that Symfony was only able to handle half the load Django could. Using Python and Django seems to have a substantial effect on your server hardware requirements. (Note that these tests were only intended as an indication for internal usage. We didn’t test enough scenarios to be certain how the outcome would hold up on a live site. )

Symfony Still Rocks

When working with PHP Symfony is still an awesome framework. In many aspects it is even superior to Django. There are quite a few things Django could learn from Symfony:

Generic validation Classes and support for automatic js form validation

When using Symfony your javascript form validation is automatically generated. This is possible because of the usage of generic validation classes. In Django this would be hard to achieve since the validation system is not based on reusable classes.

A debug toolbar

One Symfony feature which I really miss in Django is the debug toolbar. Having an overview of your DB Queries, config settings, logging messages and caching information is very convenient. Especially for debugging a site with caching the caching indicators in Symfony are awesome. These caching indicators simply show which part of the page are taken from cache and which are freshly generated.

DRY templates

Symfony has a simple but very pleasant template tag called a component. A component tag calls a specified view and renders the corresponding template. The typical use case for this tag is sidebar with news items. You will want to show this sidebar on many pages, but you wouldn’t want to have to call that code inside each and every view which needs the sidebar. Doing so would clutter your view and hinder template caching. A nicer approach is to use a component template tag which calls the view responsible for retrieving the news from the database and rendering the sidebar template. This way of allowing the template to invoke code allows for DRY views and templates.

Clear Javascript and CSS management

In Django including a javascript or css file comes down to writing the respective tags in the template. In Symfony these things aren’t left to the template, but are set by the code. This allows for a few neat features. For instance the usage of an ajax utilizing template tag (helper in Symfony) will automatically ensure that prototype.js is loaded on that page. If you would set a textarea to rich, Symfony will automatically figure out you need TinyMCE to achieve the desired effect. Furthermore it allows for a general config file where you specify which assets should be loaded for a certain combination of application and view (in Django terminology). The main benefit of such an approach comes when you combine your css and javascript files and want to optimize the groupings. Here an example of the Symfony config:

// In the view.yml - comparable to settings.py
indexSuccess:
  stylesheets: [mystyle1, mystyle2]
  javascripts: [myscript]

[php]
// In the Action - Controller in Django terminology
$this->getResponse()->addStylesheet('mystyle1');
$this->getResponse()->addStylesheet('mystyle2');
$this->getResponse()->addJavascript('myscript');

// In the Template - The view in Django

Both are great, but Django more so

Having used both Django and Symfony I believe the two frameworks can learn a great deal from each other. Fortunately many people seem to experiment with a wide variety of frameworks (Including at least one delicious developer, version 3 of delicious maybe? ;)). Django in general has some excellent features, which make it a better choice for web development. If you are somehow bound to PHP, Symfony is still a good choice. The Django community is buzzing and active like no other and I look forward to posting on the various features.

If you didn’t try it yet:
django-project.com
djangobook.com

Note: Looking to hire Python and Javascript Coders

YouTellMe.nl is currently looking to hire Python and Javascript programmers in The Netherlands. Drop me an email at thierry [at] youtellme.com if you would like to know more or want to suggest someone for the job openings.