Moving things in shoebot – simple particles…

OK, part 3 - now for something fun - extending parts 1 + 2 into a simple particle system. Particles, generally means - a lot of things moving around (the particles) and a way to generate them, an "emitter" Here we're going to take the code from the previous two parts and add a couple of things to make a basic particle system. Note - shoebot, isn't the fastest; but we do get nice looking results. Here's a video of our arrows as particles (arrowsplosion!): (more…)

Moving things in shoebot, adding different behaviours..

In my last post we made an arrow move around the screen, in this post we'll look to extend things so it's easy to make many things move around the screen. This will make the code a little more complex, but as usual it makes things simpler later on. Note: This python code runs in shoebot, planar.py is used to handle coordinates https://github.com/shoebot planar.py At the end we'll have two arrows, a blue one controlled with the keyboard and a pink one that moves on it's own: (more…)

Natural movement using polar coordinates in shoebot

Here's a little shoebot bot to experiment with natural movement. This uses polar coordinates to decide the direction and velocity of an arrow on the screen. Polar coordinates mean we can give an object a sense of 'forward', 'back', 'left' and 'right' The code below works on the current version of shoebot https://github.com/shoebot/shoebot.git With planar.py to handle the directions and velocity https://pypi.python.org/pypi/planar (more…)

Adding files to the django auto refresh

Django runserver and the app django-autotest are both good as they'll restart when modifcations are made to files in the app they will restart, however not *all* files will trigger a restart. Wanting to get some other files noticed, I added some in the __init__.py of my app, so it looked like this:
try:
    from django.conf import settings

    if settings.DEBUG:
        import forms, util, tests
except:
    pass
This seemed to be working fine until I swapped to postgres and tried to do syncdb .. and reset, uh oh ! Postgres said (names changed to protect the innocent):
2012-05-10 09:47:57 BST STATEMENT:
                    SELECT c.relname
                    FROM pg_catalog.pg_class c
                    LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
                    WHERE c.relkind IN ('r', 'v', '')
                        AND n.nspname NOT IN ('pg_catalog', 'pg_toast')
                        AND pg_catalog.pg_table_is_visible(c.oid)
2012-05-10 09:47:57 BST LOG:  could not receive data from client: Connection reset by peer
2012-05-10 09:47:57 BST LOG:  unexpected EOF on client connection
2012-05-10 09:49:14 BST ERROR:  relation "sandwich_maker_filling" does not exist at character 72
2012-05-10 09:49:14 BST STATEMENT:  SELECT "sandwich_maker_filling"."id", "sandwich_maker_filling"."name" FROM "sandwich_maker_filling"
2012-05-10 09:49:14 BST ERROR:  current transaction is aborted, commands ignored until end of transaction block
And django wasn't too happy either:
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 60, in handle_noargs
    tables = connection.introspection.table_names()
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 896, in table_names
    return self.get_table_list(cursor)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/introspection.py", line 33, in get_table_list
    AND pg_catalog.pg_table_is_visible(c.oid)""")
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/db/backends/util.py", line 40, in execute
    return self.cursor.execute(sql, params)
  File "/home/stu/.virtualenvs/the_app/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 52, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: current transaction is aborted, commands ignored until end of transaction block
In the end the solution was simple, just check for the args where we need the refresh to happen automatically:
## Automatically reload when forms or tests changed
try:
    from django.conf import settings

    if settings.DEBUG:
        import sys
        if sys.argv == ['manage.py', 'autotest'] or sys.argv == ['manage.py', 'runserver']:
                # Need to check params as otherwise this can break syncdb, reset and
                # Friends !!
                import forms, util, tests
except:
    pass
Hopefully this will be helpful to somebody with a similar error or that wants autotest or runserver to see more files.   This article was particularly helpful in debugging this:  debugging django syncdb

Cairo with python Ctypes

Uploaded some experiments with python and ctypes to here: https://gitorious.org/pycairo-ctypes/pycairo-ctypes   This is a really rough proof of concept that the pycairo API can be implemented with ctypes + metclasses.     So far only ImageSurface is supported on the SVG backend, along with Contexts.   The nice thing about this is that you can use a pycairo like API on pypy, where things should be faster (in theory).   There's a test that can be run to show the outputs the same for pycairo and cairo ctypes.

Using PyCha with Django

A quick post on using Python Charts to generate nice SVG charts for your django website (I've had the code hanging around for ages - so should just post it). The code is based on the examples there, here I integrate it into Django. To install you'll need to do
pip install pycha
Heres the code for a simple view to output a chart directly to SVG:
# views.py

from StringIO import StringIO
from django.http import HttpResponse

import cairo

def colors(request):
    in_req = 1

    svg_buffer = StringIO()

    width, height = (500, 400)
    surface = cairo.SVGSurface(svg_buffer, width, height)

    dataSet = (
     ('dataSet 1', ((0, 1), (1, 3), (2, 2.5))),
     ('dataSet 2', ((0, 2), (1, 4), (2, 3))),
     ('dataSet 3', ((0, 5), (1, 1), (2, 0.5))),
    )

    options = {
       'legend': {'hide': True},
       'background': {'color': '#f0f0f0'},
    }

    #import pycha.bar
    #chart = pycha.bar.VerticalBarChart(surface, options)

    import pycha.line
    chart = pycha.line.LineChart(surface, options)
    chart.addDataset(dataSet)
    chart.render()

    del chart
    del surface

    response = ''
    response = HttpResponse(mimetype='image/svg+xml')
    svg_buffer.seek(0)
    response.write( svg_buffer.read() )
    return response
The basic idea is that - instead of the chart outputting to an svg file, the output goes to a buffer, this is triggered by calling the destructors of the chart and it's cairo surface.  Once the data is in the buffer, it is rewound and played back to the Http Response.  

Inline SVG

As is, this won't work as inline svg, because outputting the XML preamble in the middle of the page will cause problems.  Below is an example that let's you decide if you need the preamble (full SVG output), or not (SVG fragment for inclusion in a page or another SVG):
# Create your views here.

from StringIO import StringIO
from django.http import HttpResponse
from django.shortcuts import render_to_response

import cairo

XML_PREAMBLE = '<?xml version="1.0" encoding="UTF-8"?>'

def colors_chart(inline = False):
    """
    Generate colours chart

    Set inline to True to disable the XML preamble
    """
    in_req = 1

    svg_buffer = StringIO()

    width, height = (500, 400)
    surface = cairo.SVGSurface(svg_buffer, width, height)

    dataSet = (
     ('dataSet 1', ((0, 1), (1, 3), (2, 2.5))),
     ('dataSet 2', ((0, 2), (1, 4), (2, 3))),
     ('dataSet 3', ((0, 5), (1, 1), (2, 0.5))),
    )

    options = {
       'legend': {'hide': True},
       'background': {'color': '#f0f0f0'},
    }

    import pycha.bar
    chart = pycha.bar.VerticalBarChart(surface, options)

    #import pycha.line
    #chart = pycha.line.LineChart(surface, options)
    chart.addDataset(dataSet)
    chart.render()

    del chart
    del surface

    response = ''

    if inline:
        svg_buffer.seek(len(XML_PREAMBLE))
    else:
        svg_buffer.seek(0)
    return svg_buffer.read()

def colors_svg(request):
    """ render a pure SVG chart """
    response = HttpResponse(mimetype='image/svg+xml')
    response.write(colors_chart(inline = False))
    return response

def index(request):
    """ render a chart into the template """
    chart_svg = colors_chart(inline = True)

    return render_to_response(
        'shapes_index.html',
        { "chart" : chart_svg },
        mimetype='application/xhtml+xml')
An example project with the above code is available here: pycha_django   [EDIT 25/2/2011] Fixed PyCha URL

[EDIT: 2010/10/05] Oops, I've been using TCC/LE by JPSoft as my shell... the following doesn't work in the normal command prompt [/EDIT] I just had a look at SET and it lets you set user/system variables:
Display, create, modify, or delete environment variables.

SET [/A /D /E /P /R file... /S /U /V /X] [name[=][value ]]
        file:  One or more files containing variable definitions
        /A(rithmetic)           /S(ystem variables)
        /D(efault variables)    /U(ser variables)
        /E(nv vars)             /V(olatile variables)
        /R(ead from file)       /X override VariableExclude
        /P(ause)
So to set save a variables MYAPP_HOME as the current directory, just do
SET /U MYAPP_HOME=%CD
Wish I'd have looked that up a few years ago, it'll save a few seconds next time I need to setup some dev sdk :)

Simple python spectrograph with shoebot

Seeing"Realtime FFT Graph of Audio WAV File or Microphone Input with Python..."  on python.reddit.com reminded me of one I'd built in python with shoebot. While it works OK, I feel like I'm missing a higher level audio library (especially having seen Minim, for C++ and Java). To run it in shoebot:
sbot -w audiobot.bot

audiobot.bot

# Major library imports
import atexit
import pyaudio
from numpy import zeros, short, fromstring, array
from numpy.fft import fft

NUM_SAMPLES = 512
SAMPLING_RATE = 11025

def setup():
    size(350, 260)
    speed(SAMPLING_RATE / NUM_SAMPLES)

_stream = None

def read_fft():
    global _stream
    pa = None

    def cleanup_audio():
        if _stream:
            _stream.stop_stream()
            _stream.close()
        pa.terminate()

    if _stream is None:
        pa = pyaudio.PyAudio()
        _stream = pa.open(format=pyaudio.paInt16, channels=1,
                           rate=SAMPLING_RATE,
                           input=True, frames_per_buffer=NUM_SAMPLES)
        atexit.register(cleanup_audio)

    audio_data  = fromstring(_stream.read(NUM_SAMPLES), dtype=short)
    normalized_data = audio_data / 32768.0

    return fft(normalized_data)[1:1+NUM_SAMPLES/2]

def flatten_fft(scale = 1.0):
    """
    Produces a nicer graph, I'm not sure if this is correct
    """
    for i, v in enumerate(read_fft()):
        yield scale * (i * v) / NUM_SAMPLES

def triple(audio):
    '''return bass/mid/treble'''
    c = audio.copy()
    c.resize(3, 255 / 3)
    return c

def draw():
    '''Draw 3 different colour graphs'''
    global NUM_SAMPLES
    audio = array(list(flatten_fft(scale = 80)))
    freqs = len(audio)
    bass, mid, treble = triple(audio)

    colours = (0.5, 1.0, 0.5), (1, 1, 0), (1, 0.2, 0.5)

    fill(0, 0, 1)
    rect(0, 0, WIDTH, 400)
    translate(50, 200)

    for spectrum, col in zip((bass, mid, treble), colours):
        fill(col)
        for i, s in enumerate(spectrum):
            rect(i, 0, 1, -abs(s))
        else:
            translate(i, 0)

    audio = array(list(flatten_fft(scale = 80)))

Cairo: Surface for recording commands and playback

An update on my latest cairo adventures...

When cairo 1.10 comes out we'll get a RecordingSurface so you can record commands and play them back to another surface, but how to do something similar now ? Skip to the end if you just want the code, explanation of how I got there ahead: My first advice was to try using PDFSurface, and set the file object to None.
# Record a red rectangle onto a surface.
#
# Create another surface and draw a background on it
# Draw the first surface onto this surface
#
from cairo import PDFSurface, ImageSurface, FORMAT_ARGB32, Context

recording = PDFSurface(None, 200, 200)
target = ImageSurface(FORMAT_ARGB32, 200, 200)

# Record a red rectangle
cr = Context(recording)
cr.set_source_rgb(1.0, 0.0, 1.0)
cr.rectangle(20, 20, 10, 10)

cr.fill_preserve()
cr.set_source_rgb(0.0, 0.0, 1.0)
cr.stroke()

target_context = Context(target)

# Draw background
target_context.set_source_rgb(1.0, 1.0, 0)
target_context.paint()

# Replay recording to target
target_context.set_source_surface(recording)
target_context.paint()

target.write_to_png('output.png')
That seems to work, except when I tried in Windows, when it complained that the file object was wrong. OK, we can work round that:
def RecordingSurface(w, h):
    if os.name == 'nt':
        fobj = 'nul'
    else:
        fobj = None
    return PDFSurface(fobj, w, h)
This seems to be working, but my animation seemed slow... time for some benchmarking; I rendered 1000 frames and got a rough wall clock time: 1m48. Hmm... perhaps SVGSurface will be quicker: 1m28 This is much better, 20 seconds difference just by changing what kind of surface is returned!

Animation not smooth though

The animation still seemed jerky it occured to me that when the surfaces are disposed they will attempt to out their content to the file object ! Luckily, get_similar_surface() comes to the rescue; it returns a surface not associated with a file object. Using this the original surface can be kept around forever, and never output. Wallclock time: 50 seconds!

And here it is:

_svg_surface = None
def RecordingSurface(*size):
    '''
    We don't have RecordingSurfaces until cairo 1.10, so this kludge is used

    SVGSurfaces are created, but to stop them ever attempting to output, they
    are kept in a dict.

    When a surface is needed, create_similar is called to get a Surface from
    the SVGSurface of the same size
    '''
    global _svg_surface
    if os.name == 'nt':
        fobj = 'nul'
    else:
        fobj = None
    if _svg_surface is None:
        _svg_surface = SVGSurface(fobj, 0, 0)
    return _svg_surface.create_similar(cairo.CONTENT_COLOR_ALPHA, *size)
This is really useful, you can record commands and play them back to other surfaces.