Python WHL naming and platforms

Experimenting with Pypy, I wanted to try Panda3D - while its WHL builds OK,
attempting to install it generates an error about the platform -

$ pip install ./panda3d-1.10.0-cp35-cp35m-linux_x86_64.whl
panda3d-1.10.0-cp35-cp35m-linux_x86_64.whl is not a supported wheel on this platform.

Pypys CPython support keeps improving so I wanted to try it anyway - on the Panda3D forum I got a suggestion to try renaming the package.
A post on tensorflow[1] shows how to get pips supported python platforms, here's a snippet to list them:

$ python -c "import pip; print('\n'.join(['-'.join(platform) for platform in pip.pep425tags.get_supported()]))"

Here is some output for pypy3

$ python -c "import pip; print('\n'.join(['-'.join(platform) for platform in pip.pep425tags.get_supported()]))" | sort
pp3510-none-any
pp3510-none-linux_x86_64
pp3510-none-manylinux1_x86_64
pp3510-pypy3_510-linux_x86_64
pp3510-pypy3_510-manylinux1_x86_64
pp3-none-any
py350-none-any
py3510-none-any
py351-none-any
py352-none-any
py353-none-any
py354-none-any
py355-none-any
py356-none-any
py357-none-any
py358-none-any
py359-none-any
py3-none-any
py3-none-linux_x86_64
py3-none-manylinux1_x86_64

Unfortunately, the WHL doesn't work yet, even after renaming:

$ cp panda3d-1.10.0-cp35-cp35m-linux_x86_64.whl panda3d-1.10.0-pp3510-pypy3_510-linux_x86_64.whl
$ pip install ./panda3d-1.10.0-pp3510-pypy3_510-linux_x86_64.whl
Processing ./panda3d-1.10.0-pp3510-pypy3_510-linux_x86_64.whl
Installing collected packages: panda3d
Successfully installed panda3d-1.10.0
$ python -c "from panda3d.core import *"
Traceback (most recent call last):
File "", line 1, in 
ImportError: /home/stu/.virtualenvs/pypy-nightly-py3.5/site-packages/panda3d/libp3interrogatedb.so.1.10: undefined symbol: PyExc_ImportError

While this doesn't work, it's enough info to report a bug [2] - if that gets fixed then there are bound to be others, alternately it might just work.

[1] https://github.com/tensorflow/tensorflow/issues/7552
[2] https://bitbucket.org/pypy/pypy/issues/2664/panda3d-pyexc_importerror

Script to get Pypy nightly builds

Started a script to download Pypy nightly builds and create virtualenvs for them. https://gist.github.com/stuaxo/5d4c0363317a875617271a6d424abbba

Virtualenvs are at ~/.virtualenvs where virtualenvwrapper can find them and named by pypy branch - e.g.

pypy-nightly-trunk
pypy-nightly-py3.5
Downloads only happen when there are new builds, and will resume if they were interrupted.

Install favourite Windows apps with a Chocolatey Gist

I'm a big fan of the chocolatey package manager for Windows. While not perfect, it still saves a bunch of time.

Many people build meta-packages that install everything they need, but I feel this pollutes the package namespace. As an alternative, I've prototyped a gist that can be pasted and run to install packages I use.

https://gist.github.com/stuaxo/84480ee204a203002f59b68ffbfe82f9

Being a batch file, it's not as clean as.. well almost anything. Powershell would be a sane choice here, but I don't use Windows enough to justify learning it. There's also something quite fun about twisting batch files to do what you want. Hopefully this is useful for anyone that wants to use gist to store their own package list.

Progress: end-to-end music vis working

Initial versions of all the parts of my music vis / VJ tool are now in place: The midi controller connected to a python program, which sends OSC to the separate music vis (still the smilies). Now, I need to go back and rebuild/polish some of the more prototypey bits of the code so they can uploaded. Unfortunately the USB socket died on my physical midi controller, so controlling with a (prototype) emulator I built in this video. I've learned a lot more about the linux graphics stack than I ever wanted to, once these are polished it'll finally be time to make some different visuals.

Android NDK Notes

Some notes on NDK problems I've had to overcome in the last few days, there are bound to be corrections needed, I'll try and update these based on any comments.

Environment variables

Various scripts use different ones to discover the location of the NDK - so don't be surprised when you see ANDROID_NDK, ANDROID_NDK_ROOT, ANDROID_NDK_HOME, or anything else - they mean the same thing, right now I'm using ANDROID_NDK currently.

Cmake

There are lots of different versions of android_cmake, subtly incompatible, you probably will need to edit scripts. Before trying to use cmake with Android, make sure you can build some apps using the standard tools, or the scripts that come with them This is something I'll probably come back to, as I'm using gradle and Android.mk right now.

Android Studio

In transition

Android used to use Eclipse and ANT, now it's in transition to Android Studio and Gradle, hence some things don't work, but they can be tweaked - this situation will probably change.

Installing NDK

Has it's own copy of the NDK, you can install it from the preferences in the Android SDK section, if you already installed a copy it won't automatically find it.

Build works when launching from terminal, but not using desktop icon

Android Studio doesn't pick up variables set in '.bashrc' - this means scripts may work if you launch it from a terminal, but not from the desktop.

Taking ages opening project, message about gradle

Android studio downloads this the first time, if you want more information load it from the terminal and you will see some info as it downloads.

Error about "JCenter" when trying to build

This can be because you have a really old version of gradle installed (ie the one that comes with Ubuntu 15.04) - if on Ubuntu you can test this by building using the "gradlew" script in your project, if this works then upgrade gradle - on Ubuntu there is a gradle PPA.

Not picking up settings in local.properties

You can add this at the top of your app.gradle
Properties properties = new Properties()
properties.load(project.rootProject.file('local.properties').newDataInputStream())
def sdkDir = properties.getProperty('sdk.dir')
def ndkDir = properties.getProperty('ndk.dir')
Then add ndkDir to your ndkBuild section
// call regular ndk-build(.cmd) script from app directory
task ndkBuild(type: Exec) {
    if (Os.isFamily(Os.FAMILY_WINDOWS)) {
        commandLine ndkDir + '/ndk-build.cmd', '-C', file('src/main').absolutePath
    } else {
        commandLine ndkDir + '/ndk-build', '-C', file('src/main').absolutePath
    }
}

Android.mk

Including other Android.mk files

You can do this like this
include $(LOCAL_PATH)/../assimp/workspaces/Android-NDK/jni/Android.mk
Make sure to do these includes at the end of your own file  as they may overwrite variables like LOCAL_PATH, also $(call my-dir) returns the name of the latest included file. http://grokbase.com/t/gg/android-ndk/128zkazrb1/unable-to-generate-multiple-so-files-for-native-libraries-using-a-single-make-file   http://stackoverflow.com/a/4650113/62709

No rule to make target

The build couldn't find the mentioned file, if you use a variable like LOCAL_PATH, maybe it was overwritten - see the section on includes

Outputting values from Android.mk

$(info hi, LOCAL_PATH is $(LOCAL_PATH))
$(warning Uh oh...)
$(error this is the error message that will stop the build process)
http://stackoverflow.com/a/26204506/62709

Running on a real device

cannot locate symbol “srand”

Android 21 changed the location of some symbols - either set your minimum sdk to 21 and build with that, or have a lower one (e.g. 19) and build with that SDK http://discuss.cocos2d-x.org/t/android-debug-mode-works-but-release-dont-crashes-at-launch/20247/5

Everything else

Expect to do a lot of googling, expect to have to change makefiles. ... look at other projects using the libraries you use and see how they work, good luck !

Workflow for debugging issues on python on windows with a VM

Recently I had to look at an issue where the behaviour on windows was subtly different to that on linux... unfortunately most potential users are on Windows so it's not good enough to just write them off 🙂

This workflow was quite good for checking python internals on different versions of python and being able to revert things easily.

To get things going quickly I use the IE11 VM from microsoft, and download from xdissent

https://github.com/xdissent/ievms

That way you just need to enter a command to download it

curl -s https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | env IEVMS_VERSIONS="11" bash

Next I install all the tools I need before python (we will save the state of the VM before then).

chocolatey.org makes installing tools easy*

https://chocolatey.org/

Open a command Window as Administrator then install chocolatey:

@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

Once thats done you can install tools you need - I install everything *except* python:

choco install git conemu

While these are installing start the downloads for python and it's C compiler (remember not to install them yet):

Grab python from here:
https://www.python.org/downloads/

For python 2.7.x you will probably need the VC compiler
http://www.microsoft.com/en-gb/download/details.aspx?id=44266

My app was for PyGtk so I also downloaded that - download anything else you need at this stage.

Download your project...
Using git or whatever, grab your own project to a directory.

Save the VM state.
In virtualbox save the machine state, I called mine 'Tools Installed'.

From here you can install python + know that you can poke around in the internals, any time you want to check against a different version, just revert the VM to the previous state.

 

TL;DR - Install all tools except python, grab projects, SNAPSHOT, install python - find bugs.

*when it works.

What’s in a name?

...or why "my" is not a valid prefix for a variable

While I'm greatful for examples like this J2Me GameCanvas example I'm not so happy about this bit of it: /** Starts the canvas by firing up a thread */ public void start() { Thread myThread = new Thread(this); // Make sure we know we are running running = true; done = false; // Start myThread.start(); } What's wrong ? Quite simply 'myThread'.  You see the 'my' prefix used in code all over the web - it's extraneous... extra noise, you may as well add 'the' to the beginning of every variable. Variable naming is to a certain extent a matter of taste; a lot of people would say that it is worrying about nothing. The problem is, that most of the time spent with code is reading it, not writing it; as such we should try and be as succinct as possible 'my' adds absolutely nothing here. In the example above it would be better to either leave out the 'my' or call the thread canvasThread.

 Fuzzy

Where possible variable names should reflect what you want to use them for, if your naming is fuzzy then others will use them for things you did not intend and then your on the downward spiral... This is the other crime in variable naming, over generalising... quite often you see variables called 'data' when the name of the type would do fine... usually it is only in IO based applications dealing with arbitrary data that the name is appropriate. Of course, if you can't think what to name them, then don't pad them out as well, it's pointless.

My my my

A quick search on coders (excluding mysql) finds 301,040 hits for 'my*'. That's a lot of noise... stop the madness: ban 'my' now... if microsoft can remove it from Vista, how difficult can it be?