Hamlet Implementation

There are many existing frameworks and libraries in JavaScript that handle data-binding and application abstractions but none of them offer an integrated solution that works with higher level languages (CoffeeScript, Haml). You could come close with CoffeeScript + hamlc + Knockout or something similar but it would always be a bit of a kludge because they were never designed to work together.

There are three major issues that should be solved in clientside JavaScript applications:

  1. Improved Language (CoffeeScript, or Dart, TypeScript, etc.)
  2. HTML Domain Specific Language (Haml, Jade, Slim, others)
  3. Data-Binding (React, Knockout, others)

Hamlet is novel in that it provides a clean combined solution to these three issues. By building off of CoffeeScript in the compiler we get to have the same improved language inside and outside of our templates. haml-coffee provides a CoffeeScript aware HTML DSL, Knockout.js provides data-binding aware HTML, but no tool provided them together. What if we could truly have the best of both worlds?

%button(click=@say) Hello
%input(value=@value)
say: ->
  alert @value()
value: Observable "Hamlet"

This simple example demonstrates the power and simplicity of Hamlet. The value in the input field and the model stay in sync thanks to the Observable function. The template runtime is aware that some values may be observable, and when it finds one it sets up the bindings for you.

All of this fits within our < 4k runtime. The way we are able to achieve this is by having a compile step. Programmers accustomed to languages like Haml, Sass, and CoffeeScript (or insert your favorites here) are comfortable with a build step. Even plain JS/HTML developers use a build step for linters, testing, and minification. So granted that most web developers today are using a build step, why not make it do as much of the dirty work as we can?

The Hamlet compiler works together with the Hamlet runtime so that your data-bindings stay up to date automatically. By leveraging the power of the document object itself we can create elements and directly attach the events that are needed to observe changes in our data model. For input elements we can observe changes that would update the model. This is all possible because our compiler generates a simple linear list of instructions such as:

create a node
bind an id attribute to model.id
add a child node
...

As the runtime executes instructions it has the data that should be bound. Because the runtime is “Observable aware” it will automatically attach listeners as needed to keep the attribute or value in sync with the model.

Let’s follow the journey of our humble template.

             parser    compiler   browser+runtime
              |          |              |
haml template -> JSON IR -> JS function -> Interactive DOM Elements

The template starts out as a text string. This gets fed into the hamlet-cli which converts it into a JS function. When the JS function is invoked with a model, in the context of the browser and the Hamlet runtime, it produces a Node or DocumentFragment containing interactive data-bound HTML elements. This result may then be inserted into the DOM.

The parser is generated from jison. We use a simple lexer and a fairly readable DSL for the grammar.

There’s no strong reason to choose Haml over Slim or Jade, I just started with it because it was a syntax I knew well. The name Hamlet comes from “Little Haml” as it is a simplified subset of Haml. Adding support for a Jade or Slim style is as easy as creating a new lexer that with the appropriate subset of Jade or Slim.

Some of the simplifications to the language come from the power of the runtime to build DOM elements directly. We don’t need to worry about escaping because we’re building DOM elements and not strings. We can also avoid the DOCTYPE stuff and other server specific requirements that are not relevant to a clientside environment. Other reductions were chosen solely to make the language simpler, which has value in itself.

The core goal of Hamlet is to provide an HTML domain specific language that seamlessly inter-operates with CoffeeScript and provides bi-directional data binding. Each piece works together to provide an amazing overall experience. But you don’t have to take my word for it, try it out for yourself with our interactive demos.

Array#minimum and Array#maximum

Time for the next installment in 256 JS Game Extensions. It’s been a while hasn’t it? Well don’t worry because here are four new crazy cool additions to the Array class. This brings us up to 40!

Array::maxima = (valueFunction=Function.identity) ->
  @inject([-Infinity, []], (memo, item) ->
    value = valueFunction(item)
    [maxValue, maxItems] = memo
 
    if value > maxValue
      [value, [item]]
    else if value is maxValue
      [value, maxItems.concat(item)]
    else
      memo
  ).last()
 
Array::maximum = (valueFunction) ->
  @maxima(valueFunction).first()
 
Array::minima = (valueFunction=Function.identity) ->
  inverseFn = (x) ->
    -valueFunction(x)
 
  @maxima(inverseFn)
 
Array::minimum = (valueFunction) ->
  @minima(valueFunction).first()

Array#maxima is the core of this set, all the other methods are implemented based upon it. maxima returns a list of the elements that have the maximum value for a given value function. The default value function is the identity function which returns the item itself. This will work great for integers or strings: anything that correctly works with the > operator.

The value function can be overridden for example if you want to compute the maximum length word in a list you could pass in (word) -> word.length

The special case maximum delegates to maxima and returns only the first result. Similarly minima delegates to maxima but inverts the value function.

With these methods many problems that seem complex actually become quite a lot simpler by picking a value function and whether you want to maximize it or minimize it.

Red Ice Premortem – Hard Lessons in HTML5

Over the past two years I’ve been developing a multiplayer hockey game in CoffeeScript. What follows is a rough list of some things that I’ve learned.

And let’s be clear, these are all the mistakes I made before I got to the part where I built the game.

JavaScript
CoffeeScript is great as a programming language, but it’s still built on top of JavaScript. JavaScript has a few crippling problems when it comes to reusable libraries, like the lack of a require statement in the language itself. There are many tools that try to address this, falling mostly into two separate groups: compile/build tools and runtime tools.

Since I use CoffeeScript, a compile time tool fits well into my pipeline. For websites, generally I use Sprockets where you can explicitly require other files using special comments:

#= require player
#= require map
#= ...

This is good. Dependencies can be made explicit and you can require files from other files. The disadvantage is that this all happens during build time, so Sprockets itself doesn’t provide any way to require files at runtime. For rapid development some sort of guard script is needed to recompile all your files when any change. In practice this takes less than a second and changes are immediately available when refreshing the page.

Requirejs is an alternative that provides support for requiring additional files at runtime. This approach is great for development, but still requires a compilation step for optimizing distribution.

Either of these approaches work fine and the correct one to choose depends mostly on how well they integrate with the rest of your toolchain.

These problems would be mitigated greatly by a reliable and robust, game development-specific framework similar to Rails. There are many JS web development frameworks. There are even some that fit certain definitions of robust and reliable. I’ve even spent years trying to create one with its own crippling problems, but that’s a tale for another time.

As of yet I haven’t found any that I could recommend, and that’s coming from someone who spent three years building one. The features that I think are essential are:

  • Command line support
  • Packaging for web/download deployment
  • Rapid development environment
  • First-class testing environment
  • Extensions to core JavaScript libraries
  • Additional libraries specific to games
  • Dependency management as good as Bundler

Many people are working on these issues. For each one there are many attempts, but there isn’t yet an opinionated framework that brings them all together in a way that developers can get started with quickly and easily. The biggest problem is that even by choosing the most popular tool for each issue, you rapidly become the only person in the world who has used each of those tools together.

Distribution

HTML games provide a great multi-platform distribution mechanism: a website that players can go to and play the game directly in the browser. For selling a direct download, a zip file with a run.html file is easy to create – it’s pretty much the same as a zip of the website. Linux/Mac/Windows compatibility is practically free, though the major issue would be the player’s choice of browser. If it is extremely important that the game be played in a quality browser, Chrome Portable can be bundled with the game or provided as a separate install step.

Sharing with testers is also easy, just send a zip or link to the page where you’re developing your game. You could even host everything on Github Pages.

Gamepads

Gamepad support is now quite good in Chrome, but needs support from other major browsers. There are native plugins available that will enable all browsers on all platforms to provide equivalent gamepad support, but it’s too much trouble to assume players will install a plugin and too much additional work for developers to implement as well. Maybe if a plugin matched the API more exactly it would mitigate this from a developer perspective, but asking players to install native plugins defeats much of the gains from an HTML experience.

On the bright side, when playing Red Ice using Chrome, the gamepad support is better than most popular indie games. (Binding of Isaac, I’m looking at you!)

Multi Player

In an attempt to alienate all potential players, I originally designed Red Ice to be played 3v3 locally using XBox controllers. As you can see from this chart, number of friends drops rapidly when owning more than three controllers.

Right now Red Ice is 2v2, and that’s probably correct. I do want to do more experimental games with more than four players locally, but with six or more players, screen real estate starts to become an issue.

Capistrano cached copy takes a long time to copy during a deploy

If you’re using Capistrano to deploy your Rails projects it’s common knowledge to use a cached copy to prevent downloading your entire project from git or svn with every deploy.

What I didn’t know was that by default Capistrano copies over the .git directory when deploying. This was causing a 90 second delay in our deploys. Outrageous!

Fortunately the fix is simple, just cram this into your deploy.rb file.

set :copy_exclude, [ '.git' ]

DSLs In CoffeeScript

When programming if something is hard it means that you are not programming at the correct level of abstraction. Remember your ABCs (Always Be Coding-at-the-correct-level-of-abstraction).

What is the correct level of abstraction? The correct level of abstraction is the level that perfectly matches your problem domain. If you are creating an application about painting and geometry then you’ll probably be using points, lines, colors, areas, blending and more. Be sure to use those in your code as well, because doing cross products by hand is for mathletes, not painters.

I recently put this into practice when I was creating the menu screen for Red Ice:

gamestate "Tournament", MapState
gamestate "Versus", MatchSetupState
submenu "Mini-Games",
  minigame "Zamboni Defense"
  minigame "PushOut"
  minigame "Paint"
submenu "Options",
  item "Config", ->

The structure looks like a menu, with submenus being obvious by their indentation. There’s not a lot of extra syntax junk in the trunk. It’s clean an to the point.

I knew that I wanted the menu to be a simple list of strings and functions. I started out by just using object literals, but noticed much redundancy. In the natural process of drying up the code by removing redundancy and naming functions simply I ended up creating a DSL.

The starting point was creating a function called item.

  item = (text, fn) ->
    text: text.toUpperCase()
    action: fn

At that point the menu became a list of items, which was a big improvement, but several of the functions were similar, like opening a submenu or changing game states. So I decided to add more functions whenever anything was duplicated. These new methods were able to build on the item method I had created before, making them simpler as well.

  gamestate = (name, state) ->
    item name, ->
      engine.setState(state())
 
  minigame = (name) ->
    item name, ->
      engine.setState(Minigames[name]())

The submenu was a little more interesting. Because I made it a function, it was easy to automatically add a back button to each submenu just by sticking it in the function. Additionally implementing the back button was very simple by using the functions I had previously defined. One benefit of CoffeeScript is that I was able to use splats to allow any length of arguments to the submenu function, with each argument after the name being an item in the submenu.

  back = item "Back", popSubMenu
 
  submenu = (name, menuOptions...) ->
    item name, ->
      I.menus.push [back].concat(menuOptions)

Making nested submenus is also a breeze, not that you’d want to nest things deeply, but hey good DSLs make everything easy!

submenu "Mini-Games",
  submenu "Survival",
    minigame "Zamboni Defense"
    minigame "Mutant Attack"
    minigame "PushOut"
  submenu "Cooperative",
    minigame "Paint"

The biggest advantage of DSLs, correct abstractions, and higher-level languages, is that their power compounds. This would have been possible in pure JavaScript, but probably such a pain that it wouldn’t have been worth doing. Once you have a solid abstraction you can then build on it further and even combine different components in new and interesting ways.

If all you have is a hammer it’s going to take you a damn long time to build a house.

How to Create a Compass Extension as a RubyGem to Share Sass Mixins

I was searching for how to include sass mixins in Sprockets projects as a gem dependency. There may be some way to do it but I couldn’t figure it out. That’s why I ended up going for a Compass extension, since I use Compass anyway.

Create the compass extension:

compass create pixie_sass --using compass/extension

Remove the templates directory

rm -rf pixie_sass/templates

Create the gem via bundler:

bundle gem pixie_sass

Add compass as a gem dependency:

gem.add_dependency "compass"

Move the stylesheets folder to be within lib.

Register the plugin in lib/pixie_sass.rb


require "compass"

base_directory = File.join(File.dirname(__FILE__))
Compass::Frameworks.register('pixie_sass', :path => base_directory)

Add your sass files and mixins.

Push to github.

Require the gem in your other projects:

gem "pixie_sass", :git => "git://github.com/PixieEngine/pixie_sass"

See the example pixie_sass gem on github.

HTML5 JavaScript Pasting Image Data in Chrome

Pasting anything other than simple text into the browser has historically been stupendously impossible. Until now… It’s a miracle, image data pasted into the browser can be retrieved with JavaScript (at least since Chrome 13.0.782.220)! Just use this jQuery plugin and start receiving paste events with images all their sweet gooey image data. This picks up paste events initiated with Ctrl+V, it does not provide arbitrary clipboard access (which is probably sane from a security standpoint). What’s insane is that it used to be so hard to get pasted image data, but that is all behind us now.

A magical wizard

Copy and paste me in this webpage!

Give it a try now, right click this wizard and choose “Copy Image”, then jam Ctrl+V or Command+V. Be amazed (unless you’re not using Chrome, and in that case, get off my lawn). It also works in the pixel editor on PixieEngine. Use the wizard there too, the editor currently only handles small images. Generally there is no size restriction, you can even paste image data from your favorite image editing programs, boot one up and try it out on this page.

# Created by STRd6
# MIT License
# jquery.paste_image_reader.js.coffee
(($) ->
  $.event.fix = ((originalFix) ->
    (event) ->
      event = originalFix.apply(this, arguments)
 
      if event.type.indexOf('copy') == 0 || event.type.indexOf('paste') == 0
        event.clipboardData = event.originalEvent.clipboardData
 
      return event
 
  )($.event.fix)
 
  defaults =
    callback: $.noop
    matchType: /image.*/
 
  $.fn.pasteImageReader = (options) ->
    if typeof options == "function"
      options =
        callback: options
 
    options = $.extend({}, defaults, options)
 
    this.each ->
      element = this
      $this = $(this)
 
      $this.bind 'paste', (event) ->
        found = false
        clipboardData = event.clipboardData
 
        Array::forEach.call clipboardData.types, (type, i) ->
          return if found
 
          if type.match(options.matchType) or clipboardData.items[i].type.match(options.matchType)
            file = clipboardData.items[i].getAsFile()
 
            reader = new FileReader()
 
            reader.onload = (evt) ->
              options.callback.call element,
                dataURL: evt.target.result
                event: evt
                file: file
                name: file.name
 
            reader.readAsDataURL(file)
 
            found = true
 
)(jQuery)

Pretty simple plugin, eh? The first part is extending the copy and paste events in jQuery with the clipboardData object. Once the paste events have been extended with all the clipboard data that Chrome provides we can use that data to extract the image contents.

The meat of the plugin is binding a paste event to all the elements in the selector. When a paste event is triggered we loop through each MIME type until we hit one that claims to be an image. Once we find it we get the corresponding file data and load it as a dataURL. This can be used directly in CSS or passed on to the server and chopped up, base64 decoded, and stored as a regular png.

To use it you choose what element to listen to paste events on (html should get all of them). I haven’t messed around much with scoping it to other elements, but I don’t see why it wouldn’t work.

$("html").pasteImageReader (results) ->
  {filename, dataURL} = results
 
  $("body").css
    backgroundImage: "url(#{dataURL})"

Now when someone pastes a copied image to the page it sets the background to the pasted image. This is just scratching the surface, but the great thing is that you can now capture paste events containing images in pure JS/HTML.

What’s that? CoffeeScript is hot for you to handle? Well here’s the JS version:

// Created by STRd6
// MIT License
// jquery.paste_image_reader.js
(function($) {
  var defaults;
  $.event.fix = (function(originalFix) {
    return function(event) {
      event = originalFix.apply(this, arguments);
      if (event.type.indexOf('copy') === 0 || event.type.indexOf('paste') === 0) {
        event.clipboardData = event.originalEvent.clipboardData;
      }
      return event;
    };
  })($.event.fix);
  defaults = {
    callback: $.noop,
    matchType: /image.*/
  };
  return $.fn.pasteImageReader = function(options) {
    if (typeof options === "function") {
      options = {
        callback: options
      };
    }
    options = $.extend({}, defaults, options);
    return this.each(function() {
      var $this, element;
      element = this;
      $this = $(this);
      return $this.bind('paste', function(event) {
        var clipboardData, found;
        found = false;
        clipboardData = event.clipboardData;
        return Array.prototype.forEach.call(clipboardData.types, function(type, i) {
          var file, reader;
          if (found) {
            return;
          }
          if (type.match(options.matchType) || clipboardData.items[i].type.match(options.matchType)) {
            file = clipboardData.items[i].getAsFile();
            reader = new FileReader();
            reader.onload = function(evt) {
              return options.callback.call(element, {
                dataURL: evt.target.result,
                event: evt,
                file: file,
                name: file.name
              });
            };
            reader.readAsDataURL(file);
            return found = true;
          }
        });
      });
    });
  };
})(jQuery);

Boomstick – Hella Joysticks in JavaScript

Boomstick!

Hey! I know that you want to use joysticks and gamepads in your HTML/JavaScript games, so what are you waiting for? Check out Boomstick and give it a shot.

This cross-platform, multi-browser extension gives JavaScript access to all joysticks on all platforms. Pretty rad.

Right now I’m having great results with it in my game Red Ice. It can support up to 8 joysticks straight away, so don’t worry.

If you use any browser on Windows, then run the Windows installer.

If you’re using Chrome on any OS, bust out the Chrome Extension

Once you’ve got it plug in a joystick and check out the readme for how to access joystick data. Or you could just start developing on PixieEngine which has libraries for joystick handling built in.

But at any rate, use it, abuse it, and let me know what awesome games you make (also if you encounter any bugs or issues.)

Peace!

Optimizing Matrix and Point Classes in PixieEngine

I was able to go a long way without optimizing the Matrix and Point classes. Since most of my games and prototypes were relatively simple or made limited use of points and matrices it didn’t matter too much, that is until Red Ice, which was using 5 physics steps per frame and computing tons of vectors and collision responses.

Optimizing the Drawing

The problems became evident in the profiler. First was the classic of “draw less”. Because Red Ice is running at 1024×768 even clearing the entire canvas can get costly, not to mention copying over the ice and blood canvases to the main one. In order to break through this bottleneck I separated out the rink/background and blood canvases into layers that stay below the main canvas. Since the rink never redraws and since the blood only adds and erases strokes in a semi-permanent way, this saves copying a ton of image data to the main canvas, which in turn saves a ton of time. I still clear the entire main canvas every frame, but implementing selective clearing is a big project and would have limited performance gains (we still need to clear big chunks of the canvas most updates with all the stuff moving around).

Optimizing Joystick Input

Now that we are drawing less the next big bottleneck came from joystick input. This sounds pretty ridiculous for anyone coming from a non-web background, but considering that using 6 XBox 360 controllers as input into an HTML5 game was pretty much unheard of, it’s not inconceivable that the first time it has been done might not be the most optimal. The core of the issue was that every single piece of data that is transferred from the native extension into JavaScript slows things down. Even something as simple as an array of true/false values for buttons. Even an array of six integers for six axes. Even an object with two properties buttons and axes. Each single piece slows things down.

The first performance improvement I made was to use a single integer for all the buttons, holding each one in a bit. This gave an immediate 25% speedup, confirming that going from an array of 10 booleans to 1 integer (reducing the number of items passed across) speeds things up. Next, since I wasn’t using the additional 4 axes I decided to only pass 2 axes as a temporary fix. This also gave a decent speedup, especially at the 6 controller mark, because each additional controller was additional data.

At this point I was stumped for a while, but then I realized that if I could pass a single JSON string across that would only be a single item of data, no matter how many joysticks or axes were active. I assumed that browsers were pretty good at parsing JSON, since that is most of what the JavaScript interpreter does on all web pages, and after a day of struggling getting C++ to spit out JSON it was legit and joystick input was no longer an issue.

Optimizing the Points

This finally exposed the remaining problems of the Point class and garbage collection being the next bottle neck. Due to all the physics updates and point computations it was creating tons of new Point objects and each additional point operation created more because the operations were non-destructive. The good news is that I have a full test suite for the Point class, so that I can refactor and optimize without any fear. This was quite valuable when testing crazy new ideas to see if they would improve performance without breaking everything. I figured that since I was creating so many points that if I added optional destructive methods, and used them in the right places, that could reduce the new point creation and also GC load quite a bit.

I wish that JavaScript or CoffeeScript would allow for a shortcut to use ! as a method suffix like in Ruby, because it is a known convention for destructive methods. The closest I could come would be point["add!"](otherPoint), which was too brutal on the mind and eyes to make it into common usage. If CoffeeScript could auto-compile point.add!(otherPoint) into the index-operator notation like it does with a.class and a.new then it would be okay, but until then the ! suffix is out.

JavaScript does allow for the $ symbol in variable and method names so, by necessity, I have chosen to use $ as the glyph of destruction, which has its own poetry in a way. point.add$(otherPoint) not bad, but not my first choice.

Now, armed with new destructive methods, I set about looking for places in the physics and collisions where I could slip them in to prevent the creation of unwanted/unused points. Then a funny thing happened, the majority of the places I looked needed the operators to be non-destructive, and it was difficult to see exactly where a destructive method could be added without unwanted side-effects except in a few simple situations such as point = point.add$(delta) => point.add$(delta).

While I was cleaning up the Point code I was thinking about a conversation I had on GitHub about the performance benefits from using prototype inheritance rather than object augmentation. This sounded like a good idea in this case, as Points are primitive objects with their x and y properties on the outside and all of their methods using this everywhere. The one sticking point was that I could not abide having to stick new in front of the point constructor in ten thousand places in my existing and future code. If only there was some way to get the advantages of prototype performance, without pushing the syntactic hassles onto the people using the class.

The good news is that there is a way to set an objects prototype. All you need to do is to set the __proto__ property. So now my Point constructor looks like this:

  Point = (x, y) ->
    __proto__: Point::
    x: x || 0
    y: y || 0

You know you’ve been programming in the browser too long when __proto__: Point:: becomes a thing of beauty.

All the instance methods are defined below as follows:

  Point:: =
    copy: ->
      Point(this.x, this.y)
 
    add: (first, second) ->
      this.copy().add$(first, second)
 
    add$: (first, second) ->
      if second?
        this.x += first
        this.y += second
      else
        this.x += first.x
        this.y += first.y
 
      this
 
    ...

This gives all the performance benefits of using prototypes rather than making an anonymous function for every method as well as the additional side benefit that developers can extend Point.prototype with additional methods for use in their own projects if they want to. Another advantage is that the syntax remains unchanged, no need to use the new operator, and all the tests still pass.

The best news is that this provides a 90% reduction in time that the code spends constructing and garbage collecting points, and was simple enough to pull into the Matrix class with a two line change as well. For primitive objects like Points, Matrixes, Arrays, Numbers, and the like I wholeheartedly recommend this approach. For complex objects that require mixins, private variables, and instance variables I don’t think it will be possible because each object actually does need it’s own functions that are in the correct closure scope.

Another interesting thing is that this last optimization voided the assumption of my previous one about destructive operators. I assumed that because creating points was expensive it would be worthwhile to go to extra lengths to prevent their creation unnecessarily. Using the prototypesque construction the cost of point creation and garbage collection was reduced so much that it’s not worth it to try and squeeze out the now slight performance gains that would produce except in the hottest inner loops. I’ll still keep the destructive methods around for situations where points actually want to be updated in place, like p = p.norm(speed) => p.norm$(speed), but I won’t be quick to begin trying to “optimize” by defaulting to using destructive methods and then spend hours debugging issues that come up because two objects are actually sharing the same point reference.

Points are cheap now, use them freely!

Operator Overloading in PaperScript

I was looking into how paper.js did their operator overloading and what I found was pretty clever:

var operators = {
  '+': 'add',
  '-': 'subtract',
  '*': 'multiply',
  '/': 'divide',
  '%': 'modulo',
  '==': 'equals',
  '!=': 'equals'
};
 
function $eval(left, operator, right) {
  var handler = operators[operator];
  if (left && left[handler]) {
    var res = left[handler](right);
    return operator == '!=' ? !res : res;
  }
  switch (operator) {
  case '+': return left + right;
  case '-': return left - right;
  case '*': return left * right;
  case '/': return left / right;
  case '%': return left % right;
  case '==': return left == right;
  case '!=': return left != right;
  default:
    throw new Error('Implement Operator: ' + operator);
  }
};

Though technically this is PaperScript and not JavaScript, so they can get around the lack of native JS operator overloading. The PaperScript code is given a minimal compile pass which replaces calls to arithmetic operators with calls to $eval().

// PaperScript
var p = new Point(10, 20);
var r = p * 5;
 
// JavaScript
var p = new Point(10, 20);
var r = $eval(p, "*", 5);

Related PaperScript source

So as long as Point#multiply is defined then the operator is effectively overloaded.

I’m very interested in the implications for PixieEngine and CoffeeScript.