Mellow-Morning

The 5.3 release makes it a lot easier to get started with Django Facebook.
Documentation has had a major overhaul, many small bugs have been fixed and setting validation will complain if you make mistakes during the install.

Have a look at the new docs. Especially the bit documenting how to use OpenFacebook will be interesting to many people:

Getting an OpenFacebook object
Making calls

The full docs are linked on github.

About a year ago we open sourced the feed component of Fashiolista called Feedly. At Fashiolista it powers the flat feed, aggregated feed and the notification system. In general it allows you to build newsfeed and notification systems using Cassandra and/or Redis. Examples of what you can build are applications like the Facebook newsfeed, your Twitter stream or your Pinterest following page.

The project has come a long way over the past year and we’re really happy with the 0.9 release. In this blogpost I’ll show you how to use Feedly, Django, Celery and Redis to build a scalable Pinterest like newsfeed. (Feedly isn’t tied to Django so feel free to try setting this up using another framework)

What we’re building

You can play around with the end result on Heroku. In case it goes down here is a screenshot:

Now most parts of this app aren’t very polished or scalable. The part we are focusing on is the flat feed. (The page showing all the pins by people you follow). The system works by storing a feed for every user. As soon as someone pins an item it gets stored in the feeds of all their followers. Using this approach it becomes really easy to shard a newsfeed system. If you want to know more about the reasoning behind this solution you’ll find the background articles very interesting.

Building

So let’s get started building this app. You can find the Feedly source code and the example app on Github.

First step, setting up the feeds

Now we need to store those pins somewhere, so let’s create two feeds. One for your own pins and one for the pins of all the people you follow. (example: pin_feed.py)

# the feed containing pins by people you follow
class PinFeed(RedisFeed):
    key_format = 'feed:normal:%(user_id)s'

# the feed containing only your pins
class UserPinFeed(PinFeed):
    key_format = 'feed:user:%(user_id)s'

Second step, connecting the followers

With the feeds setup we need a way to know who follows who. You can hook this up by creating a custom feedly class and implementing get_user_follower_ids as shown below. (example: pin_feedly.py)

class PinFeedly(Feedly):
    feed_classes = dict(
        normal=PinFeed,
    )
    user_feed_class = UserPinFeed

    def add_pin(self, pin):
        # see example core.models.py
        activity = pin.create_activity()
        # add user activity adds it to the user feed, and starts the fanout
        self.add_user_activity(pin.user_id, activity)

    def get_user_follower_ids(self, user_id):
        return Follow.objects.filter(target=user_id).values_list('user_id', flat=True)

feedly = PinFeedly()

Now when you call feedly.add_pin(pin) it will fetch the ids of all your followers and spawn many small tasks using Celery. Each of those tasks will push the pin to a few of your followers. This process is called the fanout.

Third step, the view

In the view you retrieve the feed instance by calling feedly.get_feeds(user_id)['normal']. The feed is slicable so doing feed[:25] gives you the top 25 activities in the feed.

# django example, this part would be different in another framework

@login_required
def feed(request):
    context = RequestContext(request)
    feed = feedly.get_feeds(request.user.id)['normal']
    activities = list(feed[:25])
    context['activities'] = activities
    response = render_to_response('core/feed.html', context)
    return response

That’s pretty much it. These 3 steps together with a bit of boilerplate for setting things up allow you to build the example app.

Concluding

The full docs for Feedly can be found on Github and Read the docs. In addition to the Pinterest newsfeed example you can use Feedly to build:

• A Twitter like newsfeed
• A notification system like Facebook’s or Fashiolista’s
• An activity stream like Wanelo’s stories or Etsy’s activity stream

We’re eager to see what applications you’ll build with this. Be sure to let us know on Github if you run into any issues.

Django Facebook now officially supports Django 1.5 and custom user models! Go try it out and upgrade to pip version 5.1.1. It’s backwards compatible and you can choose if you want to keep on using profiles, or migrate to the new custom user model. Installation instructions can be found on github.

Contributing

Thanks for all the contributions! My startup (Fashiolista) depends on a reliable Facebook integration and maintaining it would not be possible without all the pull requests from the community. Contributions are strongly appreciated. Seriously, give Github a try, fork and get started :)

About Django Facebook

Django Facebook enables your users to easily register using the Facebook API. It converts the Facebook user data and creates regular User and Profile objects. This makes it easy to integrate with your existing Django application.

I’ve built it for my startup Fashiolista.com and it’s currently used in production with thousands of signups per day. For a demo of the signup flow have a look at Fashiolista’s landing page (fashiolista.com)

After registration Django Facebook gives you access to user’s graph. Allowing for applications such as:

• Open graph/ Timeline functionality
• Seamless personalization
• Inviting friends
• Finding friends
• Posting to a users profile

Django Facebook helps you quickly develop Facebook applications using Django.
Let me know what features or issues you are encountering!

Last night Facebook changed the format they use for codes. (codes are an intermediate step in the process of requesting access tokens.)

This change broke the caching for the convert code step for Django Facebook, breaking login, connect and registration functionality. Fortunately this was quickly reported by developers in a country where there was no Queensday yesterday.

I encourage everybody to update to 5.0.13 to make sure your Facebook integration keeps on working.

As many apps will find out today, Facebook is removing it’s offline_access permission on October 3rd. For most apps this will mean that many of your open graph shares start failing.

Fashiolista has built up quite some experience with tweaking this and today we’ll be sharing some tips. The actual code we’re using has been included in the latest version of Django Facebook.

1.) Store and retry your shares

There are several advantages of storing your open graph shares in the database. You get:

• Stats
• Error tracking
• Ability to retry individual shares
• Ability to retry a user’s shares if you get an updated token
• Ability to delete shares (since you store the Facebook object id)

Especially the ability to retry shares was important to us. Django Facebook provides a convenient OpenGraphShare model to store all your shares in. Have a look at the example below:

class DjangoModel:
    def share_to_facebook(self, graph=None):
        from django_facebook.models import OpenGraphShare
        #this is where the magic happens
        share = OpenGraphShare.objects.create(
            user_id=self.user_id,
            action_domain='fashiolista:love',
            content_type=content_type,
            object_id=self.id,
        )
        share.set_share_dict(kwargs)
        share.save()
        result = share.send()

Note that we store all info required for sharing to Facebook in the OpenGraphShare model. The actual Facebook API request is sent when you call share.send().

Using this flow has the benefit that Django Facebook will retry your open graph shares if a user’s token updates. By doing this more of your open graph shares will actually reach Facebook.

2.) Extend your tokens

Facebook’s has deprecated offline access. By default you will now get a short lived access token (usually 1-2 hours). Often you will want longer lived tokens though. For example when your app keeps a user logged in for 30 days, storing a token once with a duration of 2 hours doesn’t get you very far. If you need longer lived tokens, you can now extend your token via a Facebook API call. These long lived tokens will keep working for 60 days.

The high level API for extending tokens is located in:

user.get_profile().extend_access_token()

In the new version of Django Facebook your token will be automatically extended upon connecting with Facebook. There is however a performance overhead to this extra API call. That’s where the following point comes in:

3.) Use Celery

Calling the Facebook API is something which takes quite a bit of time. If you’re unlucky a single API call can easily take up to 2000 ms. Celery is a tool which allows you to easily run tasks in the background, ensuring your pages stay fast for the users. At Fashiolista we use tasks for checking comment spam, adding loves, clearing cache, posting to Facebook and extending your Facebook access tokens. You can learn more about Celery in this guide.

Setting up Django Facebook to use Celery is trivial. Simply set the following settings to True.

#Use celery for storing friends or likes
FACEBOOK_CELERY_STORE = True
#Use celery for extending access tokens in the background
FACEBOOK_CELERY_TOKEN_EXTEND = True

Using Celery speeds things up enormously for the end user, so I recommend using it.

Concluding

Upgrading to the latest version of Django Facebook and implementing these tips will get you a very solid open graph implementation. Let me know in the comments what steps you’ve taken to get these numbers up.

This is a talk I gave at the Amsterdam Performance Meetup.

The presentation starts by introducing Fashiolista.
It still amazes me and 3 other guys started Fashiolista and grew it to the 2nd largest online Fashion community.
I guess Marc Andreesen had a point in his “Why Software Is Eating The World” article.

The talk focuses on how we use metrics to drive optimization at Fashiolista.
And narrows down on tools like:

• NewRelic
• Graphite
• PgFouine

If you’re looking up the links from the presentation, these are the most notable ones:

• Etsy on Statsd
• Instagram on PgFouine
• Fashiolista Jobs
• My Github
• Django Facebook

The Presentation

The Fashiolista team is in New York for the fashion week. I’m looking forward to meet some local tech startups. Drop me a mail or leave a comment if you want to grab a coffee.

Cheers,
Thierry Schellenbach
CTO/ Founder
Fashiolista