Python, a little too permisive?

Also known as, why static typing can rock.

So today I was tracking down a subtle bug. It is a long standing one that we have been having trouble with and things were finally quiet enough to really start digging in. Here is the type of problem it ended up being.

Say you have a bit of python that looks like this:

ids = {foo,bar,baz,moop}
for id in ids:
fxn(id)

with fuction doing something like:

def fxn(id)
"""
@param id: id of thing we are working with
@type id: str or unicode
"""
somedict[id] = do_stuff

Now, this will all work find untill you do something like:

names = {foo,bar,baz,moo}
for name in names:
fxn(id)

Now, what just happened here? At some point someone went and edited the loop and changed a variable name in one place, but not another. 'id' in python is a build in function (gives you the memory address of the parameter), so no error is raised for unknown variable.

However, once we get inside fxn, there is nothing making sure that 'id' is, in fact, a string. Instead it is happily using it as the index for the dictionary.

The result you get is, instead of:

foo: foostuff
bar: barstuff
baz: bazstuff
moop: moopstuff

you get

: moopstuff

and of course any further use of the dictionary will be missing 3 elements and the 4th one is misnamed so you can not access it anyway.

the two basic problems here:

(1) Permissive overloading by wildly different types: id the function gets replaced by id the string and no errors (or even warning) are raised.
(2) Permissive passing of arbitrary data to functions: nothing stops to warn the user that the function expecting a string just got a function instead.

Now, of course there are ways to protect against this,... for instance what I did was:

assert(isinstance(id,str) or isinstance(id,unicode))

Which essentially reimplementation strong typing.

So yeah.... python's permissiveness is a really powerful tool, but wow can it catch you as a project ages and is maintained. That tends to be the problem with loose and agile technologies/methodologies.. they are great for throwaway stuff, but they become increasingly difficult to maintain and keep 'correct' as they age. This is a class of bug that even C/C++/ObjC would choke on and not let buy. And Ada probably would break your fingers to boot.

C++ Concepts

I think after reading Bjarne Stroustrup's interview on concepts, I have reaffirmed by belief that C++ has become the wrong answer to pretty much any question.

The language just reeks of 'bored programmer syndrome'. More and more stuff added to it for the sake of expanding the language because using simple languages is 'boring'. C++ is crammed full of geek cred.. so many obscure features, strange interactions, special rules, etc. This is not a good thing for a language. A language should be quick and simple to learn with lots of useful APIs to use.

C++ on the other hand turns learning the language into a skill in and of itself. A bit like victorian erra protocol for dating. The complexity seems to exist purely so you can say you learned the complexity and thus show your aristocratic heritage. Meanwhile it did not actually change any of the actual abilities to date.

Social Networking Games.

Social network games are all the rage. For casual developers they are the next big thing. Or at least many seem to think so.

I am seeing a lot of casual games being shoe-horned to become 'social network' games. One of the ones I have been watching is Farm Town, which seems to be rather typical.

What these games seem to be focusing on is providing incentive to build up as large of social networks as possible. This is done though gatekeepers (i.e. certain functionality can not be turned on unless you have X friends in game) and gifting.

Both of these seem to be features for the sake of having features that can be called 'social'. The gatekeeper one is an especially shallow feature and does little more then either (a) keep people from features or (b) encourage the player to friend list a bunch of people for no reason.

The later, gifting, is a bit more clever and gives a real reward without actually preventing people from play. Still, the feature feels kinda hokey, something slapped on to make it 'social network compliant'.

This is a case where I am really not sure what a better solutions might be. It could be that the 'right' solution is none at all. Adding features for the sake of having them is not always a good thing.

An example of this would be Wii titles. For every title that has a good reason for motion control there are 5 that force motion control into the game just for the sake of having it. Generally when this has been done it has resulted in a title that does WORSE for it. For instance, DeBlob. Good game... if they had just dropped the motion control. It was not needed, added nothing, and detracted from the game.

Perhaps what social network games need is to stop trying to be social network games and start adding in social features as they truly extend playability.

I will be curious to see what CCP does with COSMOS. They are going the other direction, taking an existing game and adding a social layer to it from scratch. So building a social network site off a social game rather then building a social game off an existing site.

Time will tell.

CCP Case Study: On the problems of scale

Rambling a bit about someone else's upcoming difficulty.

EVE Online has a new feature in the works called Walking In Stations (WIS). It will actually be a backport of an engine they are developing for another game, specifically WhiteWolf's World of Darkness. The basic idea is that players will be able to get out of their ships and walk around as full bodied avatars in one of the stations in-game. It will not be an FPS, but instead something closer to second life.. you walk around, you talk, emote, and that is about it. They will probably expose some kind of store functionality, which really means taking the existing market controls and given them an agent front end. So no new gameplay. From the sounds of it, it will probably be very close in functionality to those virtual meetings packages that no one uses beyond the first two weeks, only with much prettier graphics.

Now, one of the problems they will probably encounter is one of the classic issues sci-fi games (and movies/tv) run into. What to do with scale.

Sci-Fi has a bad habit of making everything really really big. You see it in most movies/shows/games. There are even these wonderful image files out there that show how big things are in comparison to real world objects like famous building and air craft carriers. The usual result is that even small ships are bigger then the biggest thing we have built today because, well, big is really impressive and futury. But it poses a problem when you actually try to interact with it.

Agent or actor, at some point you have to create assets to present the viewer. If you are on a multi-mile long space dreadnaught that requires thousands of people and untold rooms and corridors, you somehow have to communicate this to the player. There are generally three ways to do this:

(1) Lots of repetitive auto generated content. Got 100km of corridors? Let the player walk around 100km of corridors. They all look the same and the only real cost is changing the {x,y,z} coordinate of the avatar. Add nearly identical shops every once in a while and backfill with NPCs, ideally with lots of canned dialog and motion to give the illusion of a bustling metropolis. This is generally the approach that sandbox games take (with varying degrees of custom content depending on your art budget).

(2) Limit the player's motion and include lots of huge background images and occasional giant rooms. Sure you only have two rooms for all the players, but look out the window at that immense city scape! This lets you concentrate your assets in a small area and make them fairly rich. It also allows you to concentrate your players in a small area so they interact with each other, minimizing the need for NPC clutter.

(3) Let players create content. This takes elements form (1), specifically lots of blank real-estate and (2) players concentrating assets into 'switched on' areas. Second Life does this to significant success. This lets the players participate, fills the space potentially with LOTS of content, but takes artistic control away from the company which CCP might not like.

Now, what could CCP do? (3) is unlikely. They will not even allow players to add aesthetic structures to POSes (Player Owned Starbase)(which NPCs do all the time, resulting in really cool looking places to visit or mission in), so if they will not allow that then i can not see them releasing artistic control on this level. (1) is unlikely due to, well, CCP built a huge universe including thousands of stations that are probably hundreds of cubic kilometers each.

My guess is that they will do (2). Probably some type of tram-system where you can go between multiple disconnected locations, each with a fairly rich (but identical between different stations) area that players can wander between. Both the tram and the rooms will likely have awe-inspiring vistas that can not be interacted with. They will probably consider connecting areas like hallways or malls but drop them as they run into scale issues. One of the problems is that stations can have either a handful of people (0-5) or hundreds at a time (Jita might have 1000+ people in it for instance) and thus any solution they put together will have to support either extreme without making these huge stations feel empty or cluttered.. since 0-1000 player avatars SHOULD be little more then a drop in the bucket.

How would I have approached it? I admit, I am a big proponent of (3), but I don't care about artistic control too much. I also would have made stations (and ships) fairly small things, with the idea that space is big and cold and lonely.

Tadaima

Ok, back to work.

Since consulting did not really pan out (I guess one needs to be more social then I am in order to make the connected needed to find work), yet I still have a craving for projects, I've decided I am going to take a long overdue new shot at some hobbyist work.

For a while I played around with writing some apps for EVE, specifically a python app that could read in both market dumps and the EVE database, plus a set of blueprints, then produce charts concerning total costs and total profits. It was fun, but rather limited.

I also wrote an app that you could scan through the database and find system that met certain criteria. For instance, find a system with stations, L4 agent, and no good refining.

But this was rather limited, so after playing some Farm Town on Facebook, I started thinking about Flash games. I've always wanted to try out this (ok, no longer 'emerging') technology but never got around to it.

So now I am exploring how this stuff works. The first thing I need to figure out is what tech to focus on. There are a whole bunch of techs used for doing stuff like this. Flex, Java, Silverlight, and AJAX.

Flex is well supported, and there are lots of 3rd party sites out there that host the files. Newgrounds, Albinoblacksheep, etc. It would also be something very new for me to learn.

Java I already know, but I am not sure how to handle hosting. It is also a pretty heavy technology and support seems to vary. While it would be the most 'comfortable' technology from a language perspective, I am not sure how comfortable I would be with it for actually writing something for other people to play.

Silverlight is still too Microsoft specific.

AJAX sounds like a cool idea, but it is a mix of techologies and I am not sure what the barrier to entry to would be.

There is also the question of server support. I would start with single player games, but I really like the idea of working on a multiplayer game (duh). I keep dreaming of making something in the veign of tradewars ^_^

Of course the scary part is, whatever I do, I will eventually need art. I need to learn computer art. Blark.

At the moment, I am leaning towards Flex due it having a low barrier to entry and cross platform support. This may change as I explore how it actually works.

Power Corrupts.

So yeah, I have not updated in quite some time. But now that things are calming down I am going to try to post a little more regularly. If nothing else I have a bunch of things to ramble about.

Time ran an article today that, while not directly related to, actually applies in a rather important way to software development.

Does Power Corrupt? Absolutely Not

The basic idea goes like this. Give someone power, and they care more about their job (since they feel more in control of it), and thus they are more efficient and make fewer mistakes. Take power away from people and they start messing up.

It was a blind study. Wide group of people split into 3 groups, each encouraged to think of themselves as dominant, submissive, and neutral. What the found was that comprehension and mistakes ended up being quite different between the groups.

It pointed to the idea that if you want people to work well, give them power over their own work. It also suggested that the roles people have ended up in are a cause rather then an effect.. that it isn't 'efficient people bubble to the top, inefficient sink' but instead the role you drop someone into actually effects how they preform.

So what does this have to do with software development? In writing software, building a game, etc, reducing mistakes and increasing emotional investment are absolutely vital to a product going well. You could easily say that the difference between a success and a failure in a design is directly related to how well people preform and how much investment they have in things going well. No amount of 'process' will overcome people who just don't care.

Looking back, I think this played a bigger role in why I left Merit then I had realized. I went from having significant power (small in the big scheme of course, but still it made me feel in control of things that I was involved in) to being shoved pretty far down the chain to the point I was, well, just programming. I think it REALLY effected my work. Looking at everything I did over the last year, I can't say I'm overly proud of any of it. I lost efficiency, I checked in more and more bad code, my designs got worse, it just stopped mattering. And that effected the whole product indirectly.

Bad Design - When 'it just works' Doesn't.

Bit of a rant, bit of an examination of what I consider a bad design. The Netflix Movie Player.

Now I can't speak to it's internal architecture of course, but I can talk about how it interacts with the user and the rest of the system a bit, and that is where it kinda fails.

So let us look at what is offered and how it is supposed to work. In theory you go to the netflix site, pick a movie, install an ActiveX control, and poof, you are up and running. No hassle, you have been walked through every step, you are up and going. If you have problems there are common 'faq' that you can access.

So where does this go horribly wrong?

For starters, it depends on some pretty specific bits of software being installed, Windows Media Player 11 and Internet Explorer 7. Not uncommon bits of software I will admit but not everyone has or run them. From a design perspective though it means that Netflix has tied their stack specifically into Windows full-chain DRM system which, lets face it, does little to stop pirates but does a lot to piss off the user. So part one is tieing to high-level APIs with known issues.

So lets say you get those apps onto your system (which eats about a gig or so) then you try to install the ActiveX control. Netflix is very helpful about how to install it assuming everything goes correctly... but if anything goes wrong, well, your options are pretty limited. The software has minimal introspection so it can't tell you why it failed, the FAQ really do not cover much, and, here is the important part... no support forum or support email address. You can call someone but that is pretty much your only option. That, I feel, is a big part of the 'bad design' of the entire process, poor options for customer support.

So lets say you get it installed. I managed to have it installed for a short while before it broke so I can still talk about this part. One of the first things it does is find out if you have enough drive space. The problem? It only checks C:. Many windows users are painfully aware of how the C drive tends to fill up rather unchecked. Figuring out what is eating up space in windows can be an exercise in futility (no equivalent of 'du -s') so many users will create 'clean' partitions or even second drives intended for large downloads and media. In my case my C: is about 8 Gig (7 of which is filled with just the OS and things that can not control where they install, such as IE and WMP), but I have 8GB of D: for temp files and 200GB of F: for things I actually install.

Unfortunately, the player doesn't have an option for controlling where it downloads things. In fact, it has no options what so ever. It doesn't even seem to have registry entries to control where things go. To be fair neither does IE, but a lot of people are unhappy with IE's design for just such reasons. So this is the other big design flaw.. lack of even basic user control of options, or more specifically, the design decision that the user should not have to have any options in the first place... 'it just works'.

Annoyingly enough, one reboot later Netflix couldn't detect their own player and attempts to reinstall it keep generating a "A network error occurred while attempting to read from the file: C:\ .... NF_Movie_Player_z11.msi" error, which gets back to the lack of introspection/debugging. When it fails, there is nothing you can do. So I never actually got to the point where I could actually PLAY a movie, and thus can not speak to how well it behaves from there.

In short, I do not think I will ever be using this downloadable service.