Here’s my first attempt at created my own public package.

It is a Django app, which add features to the admin. Originally published as django-admin-tools, I have decided to make a version of my own after intensively updating the original source code.

First, I had to make sure that my package could be used independently as a package, so any website project could use it and get the admin extended automatically. I did this by making sure they were no custom settings or custom templates in it, but rather only generic templates that would apply to any project. Custom templates being reserved for the project level.

Then, the code must be hosted publicly, so I have created a public repository on bitbucket here: https://bitbucket.org/guillaumepiot/cotidia-admin-tools, where I keep the latest version of the source code.

I have tagged a version of code in the repository, and get the download link for that version. For that you will need to download a tagged version, and upload it back to the download section of your repository.

(How to tag on GIT)

I created the following download link for the current version: https://bitbucket.org/guillaumepiot/cotidia-admin-tools/downloads/cotidia-admin-tools-0.4.1.tar.gz

Please note that although the download url works from Bitbucket, for some reason using a download url from Github will fail. For the code I have hosted on github, I had to upload manually a source code .tar.gz file to the upload section on pypi.python.org

The package source code must be organised in the following manner:

Setup.py

You will require at least a setup.py file, which includes instructions about your package:

AUTHORS

It is important to include who has worked on the code and more importantly any past contributors which may have originated the code.

In my example, I have taken over someone else’s work so I have kept their name referenced in that file.

README.md

The holy mardown file. Explaining how to install, configure and manage your package is very important. A non-documented code is just not good enough!

Testing the package

Before register our package with PyPI, it is important to test if the package actually works.

I first install it using the repository as a reference:

Note the ‘-e’ argument. This will keep the source code in a ‘../src’ folder, so it is easily accessible to make further changes if necessary during the development phase, or even for future improvements.

If it all works fine, then you are ready to register it publicly.

You will need an account on pypi.python.org to author a package, so you will need to register first.

Once your account has been confirmed, you will be able to publish your package using the following command:

You will be prompted will an authentication method to complete the registration, where I used my username and password as credentials.

Then a quick look up on PIP revealed my newly published package

I would definitely recommend to publish a useful package for the community first, but also to improve your workflow building new projects, and keeping your code easy to improve and update across multiple projects.

The long waited Textmate 2 being converted to an open source project still at an alpha stage hasn’t made it a tool of choice for me.

Luckily, my fellow developer @kin_hung has been using Sublime Text 2 for quite a while and has showed me a list of useful packages that could seriously increase my productivity. So the choice was made.

I have listed some cool packages I have installed so far, but I probably will add more later on as I discover them so it’ll be worth keeping an eye on this article.

First, let’s get a package installer

This will allow us to install (almost) any package from the Sublime command line which is a big time saver.

Follow the instructions on this page:

http://wbond.net/sublime_packages/package_control/installation

Or simply type the following in your Sublime console (Menu > View > Show Console):

Django templates, model fields & form fields

https://github.com/squ1b3r/Djaneiro

It really eases up writing Django code, I’m so glad I don’t need to write {% %} ever again!

Template examples

Model field examples

Form field examples

HTML & CSS high speed coding

http://code.google.com/p/zen-coding/
This is the most intuitive way I have seen to shortcut HTML. You can write a string like this:

div#page>div.logo+ul#navigation>li*5>a

… and it will generate this:

That’s at least 30s saved!

Jquery

A long list of jQuery shortcut that will save you plenty of time again.

https://github.com/mrmartineau/jQuery

Tutorials

If you wish to dig deeper into Sublime Text 2, then I recommend reading this tutorial:

https://tutsplus.com/course/improve-workflow-in-sublime-text-2/

Content quality is ensured on a registration process by approval, where each first fiction piece is reviewed by a moderator before according the ‘publisher’ account type to the registrant.

On the same lines a Dribbble.com, Burrst will maintain high quality content by selecting only the worthwhile. Looking back on the success on Dribbble, I’m sure this will have positive effects on the website.

Being a web developer and therefore very curious about what is powering this site, I have collected from Anthony a few things that I will share with you now.

So, what is Burrst.com built on?

Burrst is built on Tornado, a lightweight Python framework which has the unusual advantage of being an asynchronous web server as well (great for live notifications which we’ll discuss further down).

But why Tornado and not Django for example?

Well, Tornado is lightening fast (hence the name) and Ant like his applications to be fast (we all do really). So by being lightweight and well written, the result are staggering: under 300ms page load and under 500ms for a fully loaded home page, that’s just about the time to see the loading spin appearing in your browser window.

Speed is such as an important factor in application success, users tend to get frustrated after 1s an will probably leave at 4s, the whole user experience starts with speed, and more generally, responsiveness.

Template engine

Tornado is equipped with a lightweight template processor which works by inheritance (in the same way as django does) but don’t allow includes. You can run raw Python in the template but that may not be the best practice, keeping the logic away from the view lead to leaner code.

Form management

This is where things start to get tight. The built in form system is limited and is likely to be insufficient to build public apps, where a high level of validation, security and variety of fields are required. So Burrst has turned its eyes towards wtForms, a much more complete package that plugs in easily in Tornado, providing it with the necessary form tools, considerably diminishing the amount of development work.

Database handling

Tornado comes with a MySQL database handler but speed is not its forte. So it has been a wise decision to turn to Storm, a super fast database handler. Storm has the advantage to bundle up a series of queries before applying them, increasing processor efficiency, and also provide query result caching for accelerating those recurring query tasks.

One downside of Storm is that its developer haven’t completed the documentation, which that you will have to dig in the source to really get the best out of it.

Search

One thing that Tornado does not cover is a search engine. So Woosh have been implemented for its ease of use and fast search.

Just like creating a model in Django, Woosh allow the creation of search index objects where one can specify which attribute they want to feed in the index for later being queried against.

Index generation is not automatic so a recurring job must be in place to regularly update the index with new content. Alternatively, it could be updated when a user save new content but that may lead to an extra delay in page processing, unless you are using a Python queuing process so it won’t affect the immediate page load.

Overall it may also add on computer cycle which can be optimise by having a single global update during quiet times. I have recently integrated Mixpanel for application event tracking, and they strongly advise to use a python queuing processing to send tracking call without delaying the page load.

Cloud server

Burrst is running on the very famous Amazon EC2 platform, using a micro-instance (yes, micro!) to render an average of 35000 pages a month without a problem, thanks to the efficiency of the application. The instance offers 613MB of RAM and run on a 64bits core.
All media files are hosed on the S3 CDN, once again lightening the load on the micro-instance.

From idea to product

The process of delivering Burrst has been surprisingly fast, fitting is today’s modern laws of velocity where delivering small but fast and adapting along the way to the user needs is key to success.

The project has started on January 1st, 2012, from Anthony passion for writing fiction and not having an exclusive, content quality checked website to do so. It was time to offer a service that bring good prose to highlights rather being buried in a pile of mediocre work.

Only 6 months down the line, after a regular dedicated amount of hard work, Burrst was ready to hit the wild at the end of June 2012.

And it’s been an impressive start with already 131,283 words burrsted and nearly 200 members within the first 3 months.

The project has been a success so far, I believe first for its concept but undeniably propelled by the quality of the service delivered, in terms of content, usability and speed.

The project has been both a data processing and a user interface challenge. The amount of data that Amadeus deals with on a daily basis is enormous, and we have had the priviledge to work a sample of no less than:

• 4,000 airports
• 500 airlines
• 90,000 routes

We have worked hard at VIVA to condense all the data into an attractive and easy to use interface, so anyone can look for flights and find out about booking instructions. Even wondered who fly from where and to? Well, there’s a now an app for that!

So let’s get started

We first choose a region to start from before the website clusters the data for us.

Step 1: clustering the data

4,000 airports is a lot, especially when it’s down to plot them one by one on a Google map. The likeliness is that, your computer may be short on memory being old or simply busy on other tasks, making the plotting a tremendous effort.

So we had to come up with a solution, that is, clustering the airports into hot spots.

I have implemented a K-means algorithm, which basically plots a limited number of points on a surface area, then associate each airport to its nearest point (using Haversine formula for distance calculation on a sphere using geo-location points), and finally re-adjust each point at the center of its respective hot spot area. By recursing the algorithm about 3 times, we get a pretty accurate representation of the hot area, enabling us to draw a cluster.

The calculation is pretty intense, but with the right amount of Python optimisation (thanks to Anthony Blackshaw at Getme for his contribution), we managed to get the calculation done to less than a second!

Step 2: Making your journey

With a clever “type ahead” airport search, a user can quickly look for the airpot he would like to depart from, and then see all the routes that the airport offers.

After selecting a destination, the application will give the user a list of carriers that flight the selected route, where one must be picked in order to continue with the journey.

One after the other, the application will remember your path and draw your entire journey before your eyes.

A summary screen will then present you the list of flights, which you can remove at your convenience. The application will then re-draw your journey automatically.

Step 3: Export the booking instructions

The application will compile the journey summary and all the information related to each carrier into a neat and tidy PDF document.

This step might be too far for the usual curious, but it is a gold mine for travel agencies, who can instantly access a wide range of details about the airlines such as booking processes and ancillary services.

I strongly invite you to try the application for yourself and see if it works for you

Feel free to send us your feedback in the comment area below, good or bad, it will always be appreciated.

The project is about demonstrating the impact of football players performance (whether good or bad) on the pitch and how it affects their social media print off it.

Chris Jones & Gary Blowers have been doing a great job in writing consistently fresh content, pulling out and explaining some social facts we can see on the graph, along with interesting feedback on the players performances.

The design was created by Gavin Auty, and I have worked on a few refinement of the UI for optimal ease of use.

How does it work

The website is equiped with a cron job (a schedule task on the server), which fetches data every 30 mins for a list of players owning a Twitter and/or Facebook account. The data is then saved in a database (over 90,000 rows so far!) to be later processed by the charts.

In order to display a nice graph, we need to generate an array of values every 30 mins between a start date and end date. I had to program an extrapolation script which covers for missing data points. As good as Twitter’s API is, we do get the odd connection lost, which means we have holes in the data. Using the extrapolation, we just use the previous values if the current one is missing, avoiding holes in the graphs.

Datatainment

Once we had the data nicely processed, it was time to display it a beautiful charts. Lack of time forced me to use a third party chart code (amcharts.com) which in the end proved to be a great asset of the site.

We feed the data to the chart using a JSON data object that we get from an AJAX call. The data object is then processed in the chart into a beautiful line graph, which also allows you to refine the view by zooming and slicing.

Putting it all together

One of the biggest challenge has been to layout the home page of the site in a way that tell the visitor what is this all about in a single glance. Having using great imagery for the blog articles, we have featured a quick stat view on the right hand side which allow the visitor to quickly play around, but mainly give an incentive to use the great charts we produce.

Then comes the tournament results and fixtures for reference and how to connect with our social network.

We also decided to support The Great Ormond Street Hospital by giving a small donation for each unique visitors we get on the site.

Oh and also, it’s fully responsive.

Feel free to post any comment here about the site, any feedback is always welcome (good or bad).

First of all, we need to make sure that our Mac has the right library to generate translation files.

Installing gettext

If you run the following command in Terminal:

You will probably get the following error

/bin/sh: xgettext: command not found

The reason for this is that the gettext library is not installed.

Installation from source (#fail)

I have tried to download the gettext source (if not available, check the gettext site: www.gnu.org/software/gettext/)

Uncompress the package in finder, then in Terminal cd to it:

But then I got the following error:

stpncpy.c:34: error: expected declaration specifiers or ‘…’ before numeric constantstpncpy.c:34: error: expected ‘)’ before ‘!=’ token
stpncpy.c:34: error: expected ‘)’ before ‘?’ token
make[3]: *** [stpncpy.lo] Error 1
make[2]: *** [install] Error 2
make[1]: *** [install-recursive] Error 1
make: *** [install-recursive] Error 1

It seems like it won’t compile on Lion. So here’s the other way.

Installation with Homebrew

So instead I’m installing it using Homebrew (a package installer for Mac OS), follow those instructions to install it.

Once brew is installed, run the following command:

Then you MUST add that package to your path:

* Make sure that your path is correct as well!

Or you will get the following error:
/bin/sh: msguniq: command not found

We should be good to go to the next step now.

Generating language files

One thing the Django admin command won’t do, is creating the locale folders. So in your project directory (or app directory) add the following folder structure (one for each language):

Those folder will indicate in which language your project needs to be localised. Then run the following command to generate the language files

This command will compile all the languages at once (for more specific command check the Django docs).

So you will get something like that:

The PO file contains the original translation string followed byt the translated one (French example):

#: map/forms.py:9
msgid “Hello”
msgstr “Bonjour”

Once you (or the translator) have completed all the translations in that PO file, you can run the compile command:

It will generate the compiled MO files, which will be used by Django for translating all the texts.

You will have run the makemessages command everytime you had or modify transalted string in your app, then compile everytime you change the translations.

Translation helper

I’ve been using a great Django app to handle translation easily, called Rosetta. It will make form fields in your admin for each translation, which ease the process rather than going through a really long text file.

Happy translating!

Using the CSV library in Python, I managed to read and slice my csv accordingly, here is an example:

In the following example, I have already created the list of players, but some Polish characters in their name appear as “?”.

The problem is that the column in that database table (the one for the name) hasn’t got the right encoding charset, which should be UTF-8.

The best way to convert the column to UTF-8 is to run the following command in your mysql prompt, against the right database:

Once you’ve done this, re-run your Python import/update script and the name should be saved with the right characters.

So in the default Python settings, you may come up with an issue similar to this:

ascii' codec can't encode character u'\xe9' in position 3828: ordinal not in range(128)

The solution to this is to force Python to encode characters in UTF-8.

For that we need to find site.py on our Python path (print sys.path to find out), and add the following line right under import sys, os:

Now restart your site (if using wsgi) or else your web server, and you should be good to use all kind of utf-8 characters.

The reason I have implemented that change is because I had to match the URLs from my site to URL being proxied from another server. Since the proxied URL contains extra path (.eg example.com/welcome) and my site is running under the root (eg.mysite.com), I’m adding this prefix (in that case it would be /welcome) to all my site URL so it matches the ones from the proxy.

Ok so first, we must added this setting in settings.py:

Please we won’t use a slash in the end, since it’s already added by the URL resolver.

There are 2 areas where that prefix must be added:

In cms/models/page_model.py:

Modified line 122:

Modified line 125:

Modified line 128:

In menus/menu_pool.py:

Modified line 173:

Modified line 174:

Then I use pip to install PIL

(Make sure you have Xcode installed on your machine or you may run into C compiler errors)