Two things I’ve learned from 12 years of programming

I’ve been writing code for almost as long as I can remember. I’ve written in BASIC, Pascal, C, C++, C#, JavaScript, Java, PHP. To name a few. I’ve written small applications; Games; Utility programs; Websites; Classroom projects. Applications with more than 750k lines of code. I’ve been paid to do this for over five years. I’ve done it on my own for fun for an additional 7.

Through all those projects, and all the experiences I’ve had during that time, there have been two things that have massively impacted how I read, write, and understand code on a day-to-day basis. These may be completely obvious to you! If so, I applaud your mental faculties. It’s taken me many years to finally boil down some of the finer points of these ideas and see how these ideas have played out in my own code over time.

Objects as Functions and Data

The first construct that has changed me has two parts to it. In the discipline of functional progrogramming there is an idea that states that the smallest reusable unit is not the object, it’s the function. Furthermore, objects are rather terrible, untestable, and unstable messes of state and data. Yes, I said it; and just incurred the wrath of almost every modern CS professor everywhere. Objects do have their place, they can be useful, and yes there are instances when an object can, and should represent the state of a real physical object and is a completely valid way to model small parts of your code. However, with that in mind, you also must realize that that writing an object such that it is nearly “unbreakable and uncrashable” is extremely difficult to do well. The tenants of functional programming haven’t caused me to dismiss object oriented programming entirely, but the way I think about objects has changed drastically.

In the world of functional programming everything is a function or data. Nothing else. Functions take inputs, and functions have outputs. Data is immutable. There is no state because functions are atomic. Something goes in, something comes out. If something is used by the function, it’s passed in. I’ve heard somewhere (Probably from Rich Hickey) that you could reduce 90% of the bugs of a junior programmer by making them declare all their methods as static. I’m not saying to go and do this, but imagine for a moment how a requirement like that would change your code. All of a sudden, you don’t have reference to “this” anymore. You have to pass your object in. You can’t just access a variable. You have to pass it in. With that one change, an instance object doesn’t do anything, because it can’t change itself. Your object is just a collection of data: simple values, pointers, and methods that act on a constrained set of data.

In reality this is how your code actually works under the covers in most languages. The runtime will never physically create multiple instances of you function no matter how many instances of your object you declare. Seriously. When you call “functionThatChangesX()” that changes variable x in your class, you don’t get multiple instances of “functionThatChangesX()”, it takes your nice little class function and turns it into “static void functionThatChangesX(classX this)” and changes “X” on “this” where “classX” is the name of your class.

See what just happened? Realize it or not, you already live in a world where your code is static and your data is not. There’s are rules in place that make the reality slightly more complicated (usually enforced by the compiler or runtime), but when it comes down to it, your instance methods are only a short hand parlor trick to make it look like you can “just access” things.

To take this even a bit further, you understand what a Map is right? A set of key value pairs, with unique keys. You can literally take an object’s fields, and translate it into a dictionary, use the property names as keys, and turn the methods into static functions that act on that dictionary. Have a list of other objects? They become a list of dictionaries. That list becomes a property of your first object. Your functions take a dictionary, or a list, or multiple dictionaries and spits out new ones. When you start thinking about objects as just lists and dictionaries, there are a lot of really cool things you could do with dictionaries that are just plain hard with objects. What if you want to select a subset of based on the value of a property? What if you wanted to select a subset of them based on the value of the sum of a list of one of the properties? Easy, if you could write a sum function that worked on arbitrary dictionaries given a property name, and a select function, and then just compose them together… I literally would not have to write loops anymore for 90% of the code that I write.

Now, there is a point to be made that having objects allow you to enforce compile time checking of properties. This is a completely valid concern. At the end of the day I’m not trying to tout functional programming over object oriented programming, as with all things, different problems will ultimately require different approaches. There will always be trade offs. I want to point out that functional programming has made me look at objects differently and use them as tools instead of a golden orb of truth. However; Functional programming has forced me to look at the inherent instability of objects themselves. Objects have three distinct things that cause complexity because they are not separate: state, identity and value. These three things can always changed, cannot be relied on, and force everything that touches it to deal with it. I cannot count the number of times I’ve tested properties on an object I’ve been passed to see if they are null, or if the object is in a correct state. I cannot count the number of times I’ve spent huge amounts of time trying to decide if instance A and instance B are the “the same”, and if they should be considered “the same”. Objects, and more generally, any kind of mutable (directly modifiable values), is prone to changing under your feet. Properties can be changed by other code, methods or events on the object can can change the state of the object without you knowing. Or even in the middle of a method if your application is threaded.

The point of this whole section is that functional programming has forced me to think through the instability of my code. In an object oriented world, it has opened my eyes to potential problems and allowed me to control them before I’ve actually run into those particular problems. It has allowed me to better understand and break apart chunks of code such that they can be reused.

Ultimately, all I can say is that the more I’ve applied the ideas of functional programming to my own code, the better my code has become, and the easier it’s been to create systems that work simply and effectively with minimal amounts of excess code. The problems of selection, joining, filtering, mutation, reducing, ziping and so on have been so well defined for so long. Once you can look at the problem and realize that in an objects the data and methods that operate on it are separate, even if they are written together, it becomes far easier to apply solutions to problems even if you have to write a custom join / select / filter / zip functions by hand.

Applications as Graphs

The second construct that I’ve just recently crystallized into a firm concept that I can apply to my code may be overly simple: It’s the idea that applications are graphs of objects (if we’re going with a functional mindset, then applications are litterally data graphs with static methods that modify and work with subsections of the graph). The running theory in my head is that for any reasonably complex application the code will always be a graph, not a tree. This may seem painfully obvious. After all, we talk about the object graphs all the time, there are graphing problems, and whole fields devoted to analyzing how graphs can be classified, sorted, traversed and so on. But until recently I personally have never stopped to realize what that means and how that impacts how my code is written.

For one, it means you will never be able to get away with passing every object you need in the constructor. Seriously, think about that. I will bet that most of you have heard that the right thing to do ™ when writing object oriented code in the “classic OOP manner” is to pass all your dependencies in through the constructor. They’re wrong. When you construct a data graph, it is always a two step process. You create all your objects and you link all the objects together in a separate pass. This means that in your application for at least some of your objects you must have some kind of initialize or set up method(s) or event handler subscriptions for at least some of your objects. You may try to hide it, make it simpler with dependency injection or to limit interaction to to just events, but the core idea will apply.

As an overly simple abstract example consider a circular dependency such that A depends on B depends on C that depends on A. There is no way to construct this graph if A takes a B in the constructor, B takes a C in the constructor and C takes an A in the constructor. It’s not possible to have 3 instances that all talk to one another unless you allow for the objects to be initialized or linked to in some way outside of the graph.

The more I’ve considered this, the more I’ve realized how often I try to cram and contort my design into an object tree. It’s absurd. Psychologically, I believe it’s inevitable that we do this to our designs, simply because we’re surrounded by hierarchy. Classes have one parent. Code is organized into directories and folders. File structures have a “Parent”. Organizations have a “Manager” a “Director” a “VP” and so on. You have a “Top” and a “Bottom” of your application. Your application is designed in “Layers”. We miss the power thinking about our application structure in terms of a graph because it’s not as simple as a top down structure of a tree. Trees are simpler to talk about and reason through because graphs can take so many forms. So, because thinking about graphs is hard, we try to structure our code into a tree, and we write constructs to talk to other parts of the tree “secretly” so that we don’t expose the dirty little fact that we actually do know about and depend on this other part of the tree. We usually call these events. Or message frameworks. Or whatever. The point is that your code will eventually reference things besides it’s “parent” or it’s “children”. Whatever those are.

To further explore this, lets take an almost real example: lets say you have a menu bar that needs to know the state of a dial inside a widget inside a group of widgets inside a larger component. In the “tree mindset” you would have to pass the value when it changed from the widget to it’s controller to its parent controller to the application controller to one of it’s child controller for the menu to the child controller for the menu item. That is a huge number of dependencies for your code to be a tree. It means that everybody above you and below you in the tree along the path from your dial widget to your menu item needs to know about this value and what to do with it! If you had a way to construct your app such that the menu had a direct knowledge of the widget controller (turning your tree into a graph) construction becomes harder (the graph problem that causes us to try to make a tree in the first place) but the overall complexity of your code goes way down

Thinking about how a menu will talk with a dial with a tree:

If you think about how it can talk as a graph:

It should be fairly obvious in these examples the amount of coupling that will be required in the first example. Every parent has to know about the combined behavior of every sub element and be able to handle changes in those sub components. Furthermore, it somehow has to route data and information up the tree such that it goes down the other branch of the tree and reaches the correct component. There is massive coupling. You may argue that there are interfaces in-between, or generic events, or you that use such-and-such messaging framework or data architecture with controller layers. The coupling and dependencies exist, like it or not, but the more you force communication of dependent components up and down the tree or through other messaging frameworks, the harder it will be to maintain and build on as the application gets larger. I’m talking about long term maintainability and growth, not small 1-3 month mini-projects.

In the second example, the object graph is going to be much harder to construct. You have the exact same problems as constructing an object graph of data, but the code required for communication between different components is much smaller. As in constructing normal graphs of data, there are two stages: Constructing the objects, and wiring the objects together. Is it easy? No. But it drastically reduces the amount of moving pieces needed. The simpler code there is, the less there is to change and maintain. The simpler code there is, the less there is that can go wrong.

Final Thoughts

I’ve covered two things: Objects as Functions and Data, and Applications as Graphs. I don’t want you to come away from this with a sudden desire to increase the amount of coupling in your code. That is the last thing I want. I want to get you to look beyond what’s easy to do, beyond what you are comfortable with and to write code that is simple. Simple is not easy. Simple is not “smaller”. Simple is not “more succinct.” Simple is directly correlated to the long term progress and maintainability of the codebase of a project.

When I say easy, or simple, I’m talking about something very specific, and I recommend watching Rich Hickey’s talk on the subject Simple Made Easy, but here is the quick defintion:

Simple: (vs Complex)

  • One role, one braid, one fold, one strand
  • Does NOT mean one of something, one operation
  • Simple can involve many things, but it is about not interleaving things
  • Simple is an objective notation

Easy: (vs Hard)

  • Near at hand (on our hard drive, tool set, IDE, gem install… etc…)
  • Near to our understanding (It’s familiar)
  • Near our capabilities
  • Easy is relative. For instance mountain climbing and german are easy for some, and hard for others.

As programmers, we have an infatuation with easy. We like things that can be “up and running in 2 minutes”. We argue about tools. About IDE’s. About programming languages. But easy is like a runner on a sprint. You can dash 100 meters really quick. It may start off really well, but it has no distance capability. As a programmer, at the end of the day, our job is to write simple, reliable, correct code. Code that can be reused, built on, and can be reasoned about in the long term. Functional programming has made me completely rethink how I write code to be simple. Looking at applications as graphs instead of trees and given me the mental mindset to recognize dependencies and simplify how the projects I’ve worked on have been built.

Where the Internet lives

It’s not often that I get to share something about the company I work for. Google, a company that tightly holds onto its secrets, doesn’t talk about its secret sauces. In spite of this, it remains one of the most open companies you will find on the inside as an employee. You walk through those doors and the world opens up to you. It’s a culture where information is free. A culture where the more you share and the more you ask the more you get. It’s a culture where people honestly want to help you succeed.

It is a rare occasion that Google will pull back the curtain and show off what makes Google, well, Google. Beyond the people, beyond the simple home page, beyond that small little search box lies a massive infrastructure of data centers that span the globe. A multibillion dollar investment of fiber and servers and code that every day, quite literally, download the internet.

The Wired article that published this morning puts it best:

This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds. This multibillion-dollar infrastructure allows the company to index 20 billion web pages a day. To handle more than 3 billion daily search queries. To conduct millions of ad auctions in real time. To offer free email storage to 425 million Gmail users. To zip millions of YouTube videos to users every day. To deliver search results before the user has finished typing the query.
Steven Levy, Wired Magazine

So, with that in mind I invite you, for the first time, to take a look at the world behind the curtain. The mechanical and electronic dream that makes this information age possible. Take a look. This, is where the internet lives.

It’s an amazing place to work, something I’m proud to be a part of. Heres to making the world a better place.

Welcome to Google

I work at Google.

Over the last 5 weeks, I’ve transitioned into a whole new world. I’ve always been on the outskirts, used the tools, apps, search and everything else. I still remember how I first heard about Google. He described to me as a “Website where you can type things in and it’ll find it for you.” (Brian, I’m looking at you) over 10 years ago. I still remember writing down “www.google.com” on a little scrap of paper, never knowing that someday I’d actually be working for that then little company.

It’s intimidating. Going from a great little company of around 20 people, to a global tech company whose name has become a verb in the english language because of how engrained into our culture this company has become. I’m looking forward the challenge. A whole new set of language, tools, and platforms. It’s so much to learn and take in. At times it feels like trying to drink from a firehose with the sheer quantity of information that is just thrown your way. It’s the kind of challenge I enjoy, and hopefully, will excel at.

As I sit on the GBus with WiFi writing this post and headed for the set of buildings collectively known as the Googleplex, I feel like I’m back in school. It’s incredible to think back on where I’ve been, the things that have happened to get me too this point, and on all the possibilities that the future holds.

Let the adventures continue.

Farewell InterKnowlogy, thanks for all the fish

As some of you may know, this last Friday (August 3rd) was my last day at InterKnowlogy. It has been a fantastic experience working with and being there for the past 3.33 years (1215 days to be exact). But, like all good things, there comes a time a better opportunity comes along that you just can’t resist. If you don’t know already, you’ll hear about it shortly.

I’ve been extremely fortunate to have started my career at InterKnowlogy. It has been incredible to work with people of that caliber, people who are not just technical, but people who genuinely care about the work they do and the people around them. I’ve been invested in and mentored during my time there and it’s to all of them that I owe the experience and skills that I have today. I know that these experiences will continue to shape me throughout my career. So to all of you who who are reading this that have helped me along the way: Thank you!

My best wishes and continued success to InterKnowlogy and all the people that work there,

You rock,

Paul

CSS 3D Clouds Retake

This post is going to cover my experience following through the tutorial on making CSS 3D clouds posted here: http://www.clicktorelease.com/tutorials/css3dclouds/. I didn’t make the original code, but I did run into several issues while I went through and I wanted to share my experience, work-arounds, and pieces of code that were missing from the original tutorial.

All the questions that came up and fixes changes here were done on the Chrome Beta (v20.0.1132.41 beta-m to be exact)

1. Creating the world and a camera

CSS 3D Clouds Step 1

In this step, you create two div’s in the body of your HTML 5 page, the outer <div> gets an id of viewport and the inner <div> gets an id of world. From there you setup some structural styling (viewport spans entire screen via absolute positioning, world is centered in the middle)

Initial Page Setup

Some initial styling needs to be done to make it look like the demo, here’s what I have:

* {
	box-sizing: border-box;
	margin: 0;
	padding: 0;
}
 
body {
	overflow: hidden;
}
 
#viewport {
	background-image: linear-gradient(bottom, rgb(69, 132, 180) 28%, rgb( 31, 71, 120 ) 64%);
	background-image: -o-linear-gradient(bottom, rgb(69, 132, 180) 28%, rgb( 31, 71, 120 ) 64%);
	background-image: -moz-linear-gradient(bottom, rgb(69, 132, 180) 28%, rgb( 31, 71, 120 ) 64%);
	background-image: -webkit-linear-gradient(bottom, rgb(69, 132, 180) 28%, rgb( 31, 71, 120 ) 64%);
	background-image: -ms-linear-gradient(bottom, rgb(69, 132, 180) 28%, rgb( 31, 71, 120 ) 64%);
}
 
#world {                    
	background-color: rgba(255, 0, 0, .2);
}

Vendor prefixes

I’m so used to Chrome being the “latest and greatest” that I honestly expected to be able to use non-prefixed CSS properties and have the code “just work”. That’s NOT the case. As of this writing you will need to use prefixed properties, so add the following prefixes into the appropriate rules:

#viewport {
	perspective: 400;
	-webkit-perspective: 400;
	-moz-perspective: 400;
	-o-perspective: 400;
}
 
#world {                    
	transform-style: preserve-3d;
	-webkit-transform-style: preserve-3d;
	-moz-transform-style: preserve-3d;
	-o-transform-style: preserve-3d;
}

Help, my javascript code doesn’t work!

You probably put your javascript in the <head> tag, which means that document.getElementById( 'world' ) will not work because the elements don’t exist yet. Put the script at the end right before the </body> tag and it should work if everything else is correct.

Besides, it’s just good practice to put your javascript last.

Help, my javascript code doesn’t work! (pt 2)

This just shows my ignorance of javascript, but if something still isn’t working, you might have this problem:

Javascript uses the \ character in your strings to tell the parser to treat the next line as if the string continued:

'translateZ( ' + d + 'px ) \
rotateX( ' + worldXAngle + 'deg) \
rotateY( ' + worldYAngle + 'deg)';

Is the same as:

'translateZ( ' + d + 'px ) rotateX( ' + worldXAngle + 'deg) rotateY( ' + worldYAngle + 'deg)';

Zooming javascript

The code samples in the original tutorial omit the code to zoom in and out with the mouse wheel. Here it is in all it’s javascripty wonderfulness:

window.addEventListener( 'mousewheel', onContainerMouseWheel );
window.addEventListener( 'DOMMouseScroll', onContainerMouseWheel );
 
function onContainerMouseWheel( event ) {
	event = event ? event : window.event;
	d = d - (event.detail ? event.detail * -5 : event.wheelDelta / 8);
	updateView();
}

2. Adding objects to our world

CSS 3D Clouds Step 2
E.g. .cloudBase.

Create cloud base code is incorrect

Instead of:

for( var j = 0; j <<; 5; j++ ) {

the correct line is:

for( var j = 0; j < 5; j++ ) {

Actual random numbers and prefixed transforms

The random numbers for createCloud():

var random_x = 256 - ( Math.random() * 512 );
var random_y = 256 - ( Math.random() * 512 );
var random_z = 256 - ( Math.random() * 512 );

The transform styles for createCloud()

div.style.transform = t;
div.style.webkitTransform = t;
div.style.MozTransform = t;
div.style.oTransform = t;

3. Adding layers to our objects

CSS 3D Clouds Step 3
There were a couple of things in this section that cause me to scratch my head and go whyyyyy?

Code for random generation and transforms.

Random variables:

var random_x = 256 - ( Math.random() * 512 );
var random_y = 256 - ( Math.random() * 512 );
var random_z = 100 - ( Math.random() * 200 );
var random_a = Math.random() * 360;
var random_s = .25 + Math.random();
random_x *= .2; random_y *= .2;

Vendor transforms:

cloud.style.transform = t;
cloud.style.webkitTransform = t;
cloud.style.MozTransform = t;
cloud.style.oTransform = t;

Why don’t I see the new squares?

You have to add in the style for .cloudLayer into your CSS:

.cloudLayer {
	position: absolute;
	left: 50%;
	top: 50%;
	width: 256px;
	height: 256px;
	margin-left: -128px;
	margin-top: -128px;
	background-color: rgba( 0, 255, 255, .1 );
	-webkit-transition: opacity .5s ease-out;
	-moz-transition: opacity .5s ease-out;
	-o-transition: opacity .5s ease-out;
}

I see the cloud layers, but why are they are all flat?

Yeah, this got me too, the parent div’s need to have preserve-3d, so add this into your CSS:

#world div {
	transform-style: preserve-3d;
	-webkit-transform-style: preserve-3d;
	-moz-transform-style: preserve-3d;    
	-o-transform-style: preserve-3d;    
}

4. Making the 3D effect work

CSS 3D Clouds Step 4

This section is essentially “make the layers point at the camera”. You still want them projected into the same locations, but you want them to always face the camera, giving you that sense of “volume” effect.

Vendor Transforms and Update()

First, here’s all the vendor transforms:

layer.style.transform = t;
layer.style.webkitTransform = t;
layer.style.MozTransform = t;
layer.style.oTransform = t;

Now, you also need to call this update manually once right before the end of your script. So right before the closing script tag, make sure you call this:

update();

Render Loop

Finally, even if you do this, you’ll notice that your layers still don’t point at the camera. You need to add in a function that loops and updates the layers in the #viewport at regular intervals. You could add a call to the update function inside your mouse move event, but we’ll need the loop to get the rotation to work in the next step, so it’s better if you just do this now.

(function() {
	var lastTime = 0;
	var vendors = ['ms', 'moz', 'webkit', 'o'];
	for(var x = 0; x < vendors.length && !window.requestAnimationFrame; ++x) {
		window.requestAnimationFrame = window[vendors[x]+'RequestAnimationFrame'];
		window.cancelRequestAnimationFrame = window[vendors[x]+
			'CancelRequestAnimationFrame'];
	}
 
	if (!window.requestAnimationFrame)
		window.requestAnimationFrame = function(callback, element) {
			var currTime = new Date().getTime();
			var timeToCall = Math.max(0, 16 - (currTime - lastTime));
			var id = window.setTimeout(function() { callback(currTime + timeToCall); }, 
				timeToCall);
			lastTime = currTime + timeToCall;
			return id;
		};
 
	if (!window.cancelAnimationFrame)
		window.cancelAnimationFrame = function(id) {
			clearTimeout(id);
		};
}())

Lets break this down. I discovered that this function is a polyfill for the browser animation spec here one Paul Irish’s awesome blog http://paulirish.com/2011/requestanimationframe-for-smart-animating/. Previously, when you would animate something, you would set a timer, and every few milliseconds go and update move something from point A to point B on the screen with a small increment. In order to get smoother animations, the browsers are starting to support this requestAnimationFrame function that allows several changes to be made, and update everything with a single reflow / redraw of the screen. This becomes especially handy when you are updating multiple elements on the screen and you want a clean responsive display. It also means that the browser can stop animating when you switch tabs, which means that you don’t eat up battery when someone isn’t looking at your page :)

All you really need to know is that this creates a function on the javascript window object that tells the browser to “please render an animation frame, and call back to this function when you are done and ready to render the next frame”.

5. Final words

CSS 3D Clouds Step 5

Subtle Rotating

Not mentioned, but if you want the clouds to slowly rotate like the demo, you need to add in the rotate z component into the transform update of your layers like so:

'rotateZ( ' + layer.data.a + 'deg ) /

And, you need to add in a speed property into your cloud layers when you create them:

speed: .2 * Math.random() - .1

Adding Cloud Images

Instead of this:

var cloud = document.createElement( 'div' );

Use this (or find your own cloud .png, it should be transparent to work properly)

var cloud = document.createElement( 'img' );
cloud.style.opacity = 0;
var src = 'http://www.clicktorelease.com/tutorials/css3dclouds/cloud.png';
( function( img ) { 
	img.addEventListener( 'load', function() {
		img.style.opacity = .8;
	} )
 } )( cloud );
cloud.setAttribute( 'src', src );

And finally to remove all the debug styles remove the following lines out of your CSS:

From #world remove:

background-color: rgba(255, 0, 0, .2);

From .cloudBase remove:

background-color: rgba( 255, 0, 255, .5 );

From .cloudLayer remove:

background-color: rgba( 0, 255, 255, .1 );

That should cover it! Now go make some happy clouds.

Credits

Everything here was completely taken from the Click To Release CSS 3D Clouds tutorial and expanded to include the missing parts.