========
Tutorial
========

Counting related objects
========================

Perhaps the most common use case for denormalization is to cache the number
of objects associated with an object instance through a ForeignKey.

So let's say we are building a gallery application with models like this::

    class Gallery(models.Model):
        name = models.TextField()

    class Picture(models.Model):
        image = models.ImageField(...)
        gallery = models.ForeignKey(Gallery)

To calculate the number of pictures in a gallery, we would normally have a Django view with code similar to::

    gallery = Gallery.objects.get(...)
    number_of_pictures = gallery.picture_set.count()

However, this code will result in a COUNT query on the database for every tiem the page is viewed.

To speed this up we can cache the number of pictures inside the gallery::

    from denorm import CountField

    class Gallery(models.Model):
        name = models.TextField()
        picture_count = CountField('picture_set')

    class Picture(models.Model):
        image = models.ImageField(...)
        gallery = models.ForeignKey(Gallery)

This will incrementally update the number when we add and delete related objects.
Note that ``CountField`` updates are not lazy (like the callbacks described below), their value always gets updated immediately.


Creating denormalized fields using callback functions
=====================================================

A denormalized field can be created from a python function by using the ``@denormalized`` decorator.
The decorator takes at least one argument: the database field type you want to use to store the computed
value. Any additional arguments will be passed into the constructor of the specified field when it is actually
created.

If you already use the ``@property`` decorator that comes with python to make your computed values accessible
like attributes, you can often just replace ``@property`` with ``@denormalized(..)`` and you  won't need to change any code outside your model.

Now whenever an instance of your model gets saved, the value stored in the field will get updated
with whatever value the decorated function returns.

Example::

    class SomeModel(models.Model):
        # the other fields
        
        @denormalized(models.CharField,max_length=100)
        def some_computation(self):
           # your code
           return some_value

in this example ``SomeModel`` will have a ``CharField`` named ``some_computation``.


Adding dependency information
-----------------------------

The above example will only work correctly if the return value of the
decorated function only depends on attributes of the same instance of the same
model it belongs to.

If the value somehow depends on information stored in other models, it will get
out of sync as those external information changes.

As this is a very undesirable effect, django-denorm provides a mechanism to
tell it what other model instances will effect the computed value. It provides
additional decorators to attach this dependency information to the function
before it gets turned into a field.

Depending on related models
^^^^^^^^^^^^^^^^^^^^^^^^^^^

In most cases your model probably contains a ForeignKey to some other model
(forward foreign key relationship), an other model has a ForeignKey to the
model containing the denormalized field (backward foreign key relationship)
or the two models are connected through a ManyToManyField,
and your function will somehow use the information in the related instance to
compute its return value.

This kind of dependency can be expressed like this::

    class SomeModel(models.Model):
        # the other fields
        other = models.ForeignKey('SomeOtherModel')

        @denormalized(models.CharField,max_length=100)
        @depend_on_related('SomeOtherModel')
        def some_computation(self):
           # your code
           return some_value

The ``@depend_on_related`` decorator takes the related model as an argument in
the same was ``ForeignKey`` does, so you can use the same conventions here.
``@depend_on_related`` will then detect what kind (forward/backward/m2m)
of relationship the two
models have and update the value whenever the related instance of the other
model changes.

In case of an ambiguous relationship (maybe there are multiple foreign keys
to the related model) an error will be raised, and you'll need to specify the
name of the ForeignKey to use like this::

    ...
        @depend_on_related('SomeOtherModel',foreign_key='other')
    ...

If this still is not enough information for django-denorm to pick the right
relation, there is probably a recursive dependency (on ``self``).
In that you also need to specify the direction of the relation::

    ...
        @depend_on_related('self',type='forward')
    ...
    
Denormalizing ForeignKeys
^^^^^^^^^^^^^^^^^^^^^^^^^

If you wish to denormalize a ForeignKey (for example to cache a relationship that
is through another model), then your computation should return the primary key of
the related model. For example::

    class SomeOtherModel(models.Model):
        third_model = models.ForeignKey('ThirdModel')

    class SomeModel(models.Model):
        # the other fields
        other = models.ForeignKey('SomeOtherModel')

       @denormalized(models.ForeignKey,to='ThirdModel',blank=True, null=True)
       @depend_on_related('SomeOtherModel')
       def third_model(self):
           return self.other.third_model.pk

Callbacks are lazy
------------------

Your fields won't get updated immediately after making changes to some data.
Instead potentially affected rows are marked as dirty in a special table and the
update will be done by the ``denorm.flush`` method.

Post-request flushing
^^^^^^^^^^^^^^^^^^^^^

The easiest way to call ``denorm.flush`` is to simply do it after every completed request.
This can be accomplished by adding ``DenormMiddleware`` to ``MIDDLEWARE_CLASSES`` in your ``settings.py``::

    MIDDLEWARE_CLASSES = (
    ...
        'django.middleware.transaction.TransactionMiddleware',
        'denorm.middleware.DenormMiddleware',
    ...
    )

As shown in the example, I recommend to place ``DenormMiddleware`` right after ``TransactionMiddleware``.

Using the daemon
^^^^^^^^^^^^^^^^

If the above solution causes problem like slowing the webserver down because
``denorm.flush`` takes to much time to complete, you can use a daemon to update the data.
The daemon will check for dirty rows in regular intervals and update them as necessary.
To run the daemon with an interval of 10 seconds run::

    ./manage.py denorm_daemon 10

The command will print the daemons pid and then detach itself from the terminal.

Final steps
===========

Now that the models contain all information needed for the denormalization to work,
we need to do some final steps to make the database use it. As django-denorm uses triggers,
those have to be created in the database with::

    ./manage.py denorm_init

This has to be redone after every time you make changes to denormalized fields.

Testing denormalized apps
=========================

When testing a denormalized app you will need to instal the triggers in the setUp method::

    from denorm import denorms

    class TestDenormalisation(TestCase):

        def setUp(self):
            denorms.install_triggers()