Archive for August, 2010

Mixing Django with Jinja2 without losing template debugging

At Fashiolista we’ve build nearly the entire site with Jinja instead of the Django template engine.

There are a lot of reasons for choosing Jinja2 over Django for us. Better performance (atleast… it was a lot better with previous Django versions), way more options (named arguments, multiple arguments for filters, etc), macros and simply easier to extend. Writing custom tags is simply not needed anymore since you can just make any function callable from the templates.

But… during the conversion there are always moments when you need  a Django function in a Jinja template or vice versa. So… I created a few template tags to allow for Jinja code in Django templates (I’ve also created code to run Django code from Jinja, but I haven’t seen the need for it so I omitted it here).

A Jinja Include tag to include a template and let it be parsed by Jinja from a Django template:

from django import template
from coffin import shortcuts as jinja_shortcuts                                                                                                                                                      

register = template.Library()                                                                                                                                                                        

class JinjaInclude(template.Node):
    def __init__(self, filename):
        self.filename = filename                                                                                                                                                                     

    def render(self, context):
        return jinja_shortcuts.render_to_string(self.filename, context)                                                                                                                              

@register.tag
def jinja_include(parser, token):
    bits = token.contents.split()                                                                                                                                                                    

    '''Check if a filename was given'''
    if len(bits) != 2:
        raise template.TemplateSyntaxError('%r tag requires the name of the '
            'template to be included included ' % bits[0])
    filename = bits[1]                                                                                                                                                                               

    '''Remove quotes if used'''
    if filename[0] in ('"', "'") and filename[-1] == filename[0]:
        filename = bits[1:-1]                                                                                                                                                                        

    return JinjaInclude(filename)

Usage:

{% jinja_include "some_template.html" %}

A couple of noop nodes to make sure that when you convert your Jinja templates to be executed from Django, they won’t break because of the missing Django tag.

from django import template

class Empty(template.Node):
    def render(self, context):
        return ''                                                                                                                                                                                    

@register.tag
def django(parser, token):
    return Empty()                                                                                                                                                                                   

@register.tag
def end_django(parser, token):
    return Empty()

And the Jinja tag to allow Jinja blocks in Django templates.

from django import template
from coffin.template import Template

register = template.Library()

class Jinja(template.Node):
    def __init__(self, template):
        self.template = template                                                                                                                                                                     

    def render(self, context):
        return self.template.render(context)                                                                                                                                                         

@register.tag
def jinja(parser, token):
    '''Create a Jinja template block                                                                                                                                                                 

    Usage:
    {% jinja %}
    Although you're in a Django template, code here will be executed by Jinja
    {% end_jinja %}
    '''                                                                                                                                                                                              

    '''Generate the end tag from the currently used tag name'''
    end_tag = 'end_%s' % token.contents.split()[0]                                                                                                                                                   

    tokens = []
    '''Convert all tokens to the string representation of them
    That way we can keep Django template debugging with Jinja and feed the
    entire string to Jinja'''
    while parser.tokens:
        token = parser.next_token()
        if token.token_type == template.TOKEN_TEXT:
            tokens.append(token.contents)                                                                                                                                                            

        elif token.token_type == template.TOKEN_VAR:
            tokens.append(' '.join((
                template.VARIABLE_TAG_START,
                token.contents,
                template.VARIABLE_TAG_END,
            )))                                                                                                                                                                                      

        elif token.token_type == template.TOKEN_BLOCK:
            if token.contents == end_tag:
                break                                                                                                                                                                                

            tokens.append(' '.join((
                template.BLOCK_TAG_START,
                token.contents,
                template.BLOCK_TAG_END,
            )))                                                                                                                                                                                      

        elif token.token_type == template.TOKEN_COMMENT:
            pass                                                                                                                                                                                     

        else:
            raise template.TemplateSyntaxError('Unknown token type: "%s"' % token.token_type)                                                                                                        

    '''If our token has a `source` attribute than template_debugging is
    enabled. If it's enabled create a valid source attribute for the Django
    template debugger'''
    if hasattr(token, 'source'):
        source = token.source[0], (token.source[1][0], token.source[1][1])
    else:
        source = None                                                                                                                                                                                

    return Jinja(Template(''.join(tokens), source=source))

Do note that I have modified the “coffin.template.Template” to enable debugging completely. Just replace the “Template” class in “coffin/template/__init__.py” to make it work.

def _generate_django_exception(e, source=None):
    '''Generate a Django exception from a Jinja source'''
    from django.views.debug import linebreak_iter                                                                                                                                                    

    if source:
        exception = DjangoTemplateSyntaxError(e.message)
        exception_dict = e.__dict__
        del exception_dict['source']                                                                                                                                                                 

        '''Fetch the entire template in a string'''
        template_string = source[0].reload()                                                                                                                                                         

        '''Get the line number from the error message, if available'''
        match = re.match('.* at (\d+)$', e.message)                                                                                                                                                  

        start_index = 0
        stop_index = 0
        if match:
            '''Convert the position found in the stacktrace to a position
            the Django template debug system can use'''
            position = int(match.group(1)) + source[1][0] + 1                                                                                                                                        

            for index in linebreak_iter(template_string):
                if index >= position:
                    stop_index = min(index, position + 3)
                    start_index = min(index, position - 2)
                    break
                start_index = index                                                                                                                                                                  

        else:
            '''So there wasn't a matching error message, in that case we
            simply have to highlight the entire line instead of the specific
            words'''
            ignore_lines = 0
            for i, index in enumerate(linebreak_iter(template_string)):
                if source[1][0] > index:
                    ignore_lines += 1                                                                                                                                                                

                if i - ignore_lines == e.lineno:
                    stop_index = index
                    break                                                                                                                                                                            

                start_index = index                                                                                                                                                                  

        '''Convert the positions to a source that is compatible with the
        Django template debugger'''
        source = source[0], (
            start_index,
            stop_index,
        )
    else:
        '''No source available so we let Django fetch it for us'''
        lineno = e.lineno - 1
        template_string, source = django_loader.find_template_source(e.name)
        exception = DjangoTemplateSyntaxError(e.message)                                                                                                                                             

        '''Find the positions by the line number given in the exception'''
        start_index = 0
        for i in range(lineno):
            start_index = template_string.index('\n', start_index + 1)                                                                                                                               

        source = source, (
            start_index + 1,
            template_string.index('\n', start_index + 1) + 1,
        )                                                                                                                                                                                            

    exception.source = source
    return exception          

class Template(_Jinja2Template):
    """Fixes the incompabilites between Jinja2's template class and
    Django's.                                                                                                                                                                                        

    The end result should be a class that renders Jinja2 templates but
    is compatible with the interface specfied by Django.                                                                                                                                             

    This includes flattening a ``Context`` instance passed to render
    and making sure that this class will automatically use the global
    coffin environment.
    """                                                                                                                                                                                              

    def __new__(cls, template_string, origin=None, name=None, source=None):
        # We accept the "origin" and "name" arguments, but discard them
        # right away - Jinja's Template class (apparently) stores no
        # equivalent information.
        from coffin.common import env                                                                                                                                                                

        try:
            return env.from_string(template_string, template_class=cls)
        except JinjaTemplateSyntaxError, e:
            raise _generate_django_exception(e, source)                                                                                                                                              

    def __iter__(self):
        # TODO: Django allows iterating over the templates nodes. Should
        # be parse ourself and iterate over the AST?
        raise NotImplementedError()                                                                                                                                                                  

    def render(self, context=None):
        """Differs from Django's own render() slightly in that makes the
        ``context`` parameter optional. We try to strike a middle ground
        here between implementing Django's interface while still supporting
        Jinja's own call syntax as well.
        """
        if not context:
            context = {}
        else:
            context = dict_from_django_context(context)                                                                                                                                              

        try:
            return super(Template, self).render(context)
        except JinjaTemplateSyntaxError, e:
            raise _generate_django_exception(e)                                                                                                                                                      

def dict_from_django_context(context):
    """Flattens a Django :class:`django.template.context.Context` object.
    """
    if isinstance(context, DjangoContext):
        dict_ = {}
        # Newest dicts are up front, so update from oldest to newest.
        for subcontext in reversed(list(context)):
            dict_.update(dict_from_django_context(subcontext))
        return dict_
    else:
        return context

And you’re done, now you can just mix your Django and Jinja templates like this:

{% ifequal foo bar %}
Django style if...
{% endif %}

{% jinja %}
{% if foo == bar %}
Jinja style if...
{% endif %}
{% end_jinja %}

Django & Jinja2 & Python & Web Development Rick van Hattem 24 Aug 2010 11 Comments

Twitter button to pull down the internet?

I was thrilled to see twitter releasing their own button. This is good news all around for us bloggers looking to promote our content. After looking at their code snippets a warning is in place though. The current twitter button implementation will severely break your site if their servers face load issues. Fortunately you can work around this issues by slightly modifying their implementation.

Technical explanation

For Fashiolista.com we have been writing a lot about the various techniques for implementing these type of buttons. Especially important here is the way the javascript is loaded. Twitter uses a simple blocking script approach, where as digg, facebook and fashiolista use the async dynamic script approach. There are two large differences:

  1. Blocking script loads make your site slower
  2. If twitter goes down, your site joins in

Example

In this example we are faking slow twitter servers. (By routing it through google’s app engine and delaying the response). You can see the difference for yourself (be patient and be sure to clear your browser cache using CTRL F5).

Default twitter version
(note that site content is not loading until the twitter button is loaded)

Async twitter script
(everything loads and then we wait for twitter)

Solution

Solving this is quite simple. Simply change the way the twitter javascript is loaded from the first example to the second version.

Twitter version


<script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>

Async twitter implementation

<script type="text/javascript">
  //async script, twitter button fashiolista.com style
  (function() {
   var s = document.createElement('SCRIPT');
   var c = document.getElementsByTagName('script')[0];
   s.type = 'text/javascript';
   s.async = true;
   s.src = 'http://platform.twitter.com/widgets.js';
   c.parentNode.insertBefore(s, c);
  })();
</script>

Hope the difference won’t matter and Twitter will stay up :)

Follow me at:
@tschellenbach

Thanks for spreading the word:
NextWeb

Business & Fashiolista & Google app engine & Javascript & Web Development & YouTellMe tschellenbach 13 Aug 2010 7 Comments

Django open inviter – contact importer – python

Django open inviter is a python port of the PHP api client for openinviter.com’s contact importer to work with Django. I build it for our fashion community, Fashiolista.com, where it is currently in production usage and fully functional. If you are a member of Fashiolista (which I highly doubt given the different audiences) you can test it by clicking find friends in your profile.

Usage is extremly straight forward:

from django_open_inviter.open_inviter import OpenInviter
o = OpenInviter()
contacts = o.contacts('example@example.com', 'test')

Get the code here.

Django & Fashiolista & PHP & Python & Web Development & YouTellMe tschellenbach 09 Aug 2010 3 Comments

Creating your own Digg/Facebook/Tweetmeme button

This quick walkthrough is going to bring you up to speed on how to create your own social bookmarking button. The three prime examples are the famous Digg button, Facebook’s like functionality and the tweetmeme button. For an implementation look slightly above this paragraph or check out mashable’s version on the left of their post.

Our button will be focusing on Fashiolista.com. Fashiolista is a social bookmarking site for fashion, which has seen rapid growth after launching at the next web. This tutorial explains the javascript (client side) aspects of the button. Feedback and improvements on the code would be greatly appreciated. You can find the full 450 lines of js on github.

This is what the end result looks like:

Compact Medium Large

Love it!

Love it!

Love it!

(If you are working on a shop in the fashion industry have a look at our installation instuctions.)

Step 1 – The markup

Its important to get the client side markup of the button right. Since other sites will be implementing this there is no way you can change it later on. The three major players each have their own way.

Facebook XFBML: Async script with XFBML or Iframe
Digg button: Async script with A elements
Tweetmeme: Normal script

<script type="text/javascript">
  //async script, fashiolista.com version
  (function() {
   var s = document.createElement('SCRIPT');
   var c = document.getElementsByTagName('script')[0];
   s.type = 'text/javascript';
   s.async = true;
   s.src = 'http://button.www.fashiolista.com/button/script/';
   c.parentNode.insertBefore(s, c);
  })();
</script>
<a class="fashiolista_button fashiolista_compact"
href="http://www.fashiolista.com/item_add_oe/">Love it!</a>

For Fashiolista we have chosen an async script approach with A elements. Normally loading a script element is a blocking operation for the browser. Loading the script async ensures faster page load times and a better experience if your site would ever go down. (Note that not all browsers support this option so it is still recommended to include the script tag at the bottom of the page). The function wrapped around the code ensures we don’t pollute the global scope. Furthermore the insertBefore in combination with a script tag technique is used by GA so should work in any scenario.

Step 2 – Creating the buttons, Iframe vs Script

The next step is to convert our A elements into actual buttons. We can choose to replace these A elements by our button’s html (digg, delicious approach) or load an iframe in their place (facebook, tweetmeme). The difference between these two approaches is actually pretty large. For Fashiolista you can see both an iframe and script approach. These are the most important differences I encountered.

Iframe vs Script

  • + Popup communication possible
    The script approach cannot communicate with popups it creates due to the same origin restrictions. The iframe however can be of the same domain as the popup and freely communicate. This gives a better user experience when for instance logging in.
  • + Easier to develop
    The iframe approach is easier to develop and requires less code.
  • + Parallel download in IE
    IE doesn’t download the count scripts in parallel, but it does do so for the IFRAMEs. Making this approach somewhat faster.
  • Independent CSS
    External sites don’t interfere with your button’s css if you use an iframe technique. The disadvantage is that it makes things likes hovers impossible to integrate with the other site. (For example Fashiolista’s compact button).
  • Independent
    The iframe approach makes it very hard for other sites to game the users like/love action. With a script approach a foreign site can simply call your javascript to fake someone loving the product. This freedom can be abused but also allows for mashups.
  • - Slower dom load
    Creating iframes takes a lot more time for the browser.
  • - Slower perceived load
    The script approach allows you to format the buttons before the data is loaded. Vastly increasing the perceived load speed.
  • - No shared functionality
    Buttons can’t share functionality. So when someone logs in for one button its is not possible to update the others.

The best choice differs for each project. For Fashiolista the more open script approach is currently the default.

Step 3 – Cross site scripting using JSONP

Essential to the bookmarking button is requesting the count for the given url. Cross site policies prevent us from using Ajax so we will do so by creating a script element.

_makeRequest: function (url) {
	//Simple create script element functionality
        var s = document.createElement('script');
        var b = document.body;

        s.setAttribute('type', 'text/javascript');
        s.setAttribute('async', 'true');
        s.setAttribute('src', url);

        b.appendChild(s);
}

The trouble with the script element is that you lack the nice APIs Ajax offers you. We work around this by using an url with a callback paramater, for example callback=button_loaded_3
The server side code then responds with something like this, executing the callback when the script is loaded.

button_loaded_3({"item_id": 26545, "url": "/item/26545/", "loves": 853})

This technique is often referred to as JSONP. We bind the response function to the global button_loaded_3 using the following code:

loadButtonInformation: function (buttonId) {
		//make a request to the script with the given callback
		var buttonInstance = this.buttonDict[buttonId];
		var buttonUrl = buttonInstance.lookupUrl;
		var path = '&url=' + encodeURIComponent(buttonUrl);
		var callbackFunctionName = 'button_loaded_' + buttonId;
		var scope = this;
		var callbackFunction = function(data) {
			//bind the scope and button id
			scope.buttonLoaded.call(scope, buttonId, data);
		};
		window[callbackFunctionName] = callbackFunction;
		this.makeRequest(this.countApi + path, callbackFunctionName, true);
}

Step 4 – Object oriented design

Since we are loading our code into someone else’s website we should be careful not to use similar variable names. We therefore hide as much code as possible in classes.

var fashiolistaClass = function(){ this.initialize.apply(this, arguments); };
fashiolistaClass.prototype = {
	//
	//Base class implementing the fashiolista button
	//
	initialize: function () {
		//load the buttons
		this.initializeCss();
		var fashiolistaButtons = this.findButtons();
		this.initializeButtons(fashiolistaButtons);
	}
}

Note that we are not simulating inheritance for these classes. Using them as simple namespaces is more than sufficient in this case.
The code is organized into 3 classes:

  • fashiolistaClass
  • fashiolistaUtilsClass
  • fashiolistaButtonClass

The first one acts as a manager (finding the buttons, instantiating fashiolistaButtonClasses and retrieving counts). Fashiolista button contains the logic for individual buttons and fashiolista utils contains some string parsing and dom load functionality.

Step 5 – Caching requests in the google app engine

appengineTo prevent our servers from getting flooded we are routing all traffic through google servers using the google app engine. button.www.fashiolista.com is connected to a google app engine account which forwards and caches requests to fashiolista.com. This setup enables your button to withstand great amounts of traffic without killing your servers. Furthermore it immediately also acts as a cdn for our web requests, speeding up load times for our international visitors. Setting up caching in the google app engine would require another blog post though. Let us know in the comments if you would like to know more about it.

Conclusion

The full client side code can be found here. This blog post covered the most essential parts. Code review and questions are more than welcome. Be sure to let us know in the comments. Furthermore if you are running a webshop in the fashion industry consider implementing the button.

More information

Improvements/ Request for code review

  • The domload technique is rather verbose, does anyone know a better method?
  • The popup communication or lack thereof is not ideal for users, is there a better method?
  • Script or Iframe what do you prefer?
  • Suggestions to make it faster?

Django & Fashiolista & Google app engine & JQuery & Javascript & Python & Web Development & YouTellMe tschellenbach 03 Aug 2010 4 Comments