9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog

Page 1 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
Welcome to the Wolfire Blog! This is where we keep everyone up to date on our progress on Overgrowth
and other new stuff. Be sure to subscribe to get the latest news! Also, be sure to check out the forums for
even more up to date news - or get on IRC for up to the second updates.
Starting Grass
Wolfire Answers Community Questions
Linear algebra for game developers ~ part 2
Add Comment! By David Rosen on July 3rd, 2009
Now that I've introduced vectors in Part 1, we need to look at some of the fundamental tools for working with
them. The most important tools to understand are length, normalization, distance, the dot product, and the
cross product. Once you wrap your mind around these concepts, and write functions to use them, you can
solve most vector problems you might encounter.
Length
If we have a ship with velocity vector V (4,3), we might also want to know how fast it is going, in order to
calculate how much the screen should shake or how much fuel it should use. To do that, we need to find the
length (or magnitude) of vector V. The length of a vector is often written using || for short, so the length of V
is |V|.
We can think of V as a right triangle with sides 4 and 3, and use the Pythagorean theorem to find the
hypotenuse: x
2
+ y
2
= h
2
. That is, the length of a vector H with components (x,y) is sqrt(x
2
+y
2
). So, to
calculate the speed of our ship, we just use:
|V| = sqrt(4
2
+3
2
) = sqrt(25) = 5
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 2 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
This works with 3D vectors as well -- the length of a vector with components (x,y,z) is sqrt(x
2
+y
2
+z
2
).
Distance
If the player P is at (3,3) and there is an explosion E at (1,2), we need to find the distance between them to
see how much damage the player takes. This is easy to find by combining two tools we have already gone
over: subtraction and length. We subtract P-E to get the vector between them, and then find the length of this
vector to get the distance between them. The order doesn't matter here, |E-P| will give us the same result.
Distance = |P-E| = |(3,3)-(1,2)| = |(2,1)| = sqrt(2
2
+1
2
) = sqrt(5) = 2.23
Normalization
When we are dealing with directions (as opposed to positions or velocities), it is important that they have unit
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 3 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
length (length of 1). This makes life a lot easier for us. For example, let's say there is a gun pointing in the
direction of (1,0) that shoots a bullet at 20 m/s. What is the velocity of the bullet? Since the direction has
length 1, we can just multiply the direction and the bullet speed to get the bullet velocity: (20,0). If the
direction vector had any other length, we couldn't do this -- the bullet would be too fast or too slow.
A vector with a length of 1 is called "normalized". So how do we normalize a vector (set its length to 1)?
Easy, we divide each component by the vector's length. If we want to normalize vector V with components
(3,4), we just divide each component by its length, 5, to get (
3
/
5
,
4
/
5
). Now we can use the pythagorean
theorem to prove that it has length 1:
(
3
/
5
)
2
+ (
4
/
5
)
2
=
9
/
25
+
16
/
25
=
25
/
25
= 1
Dot product
What is the dot product (written )? Let's hold off on that for a second, and look at how we calculate it. To get
the dot product of two vectors, we multiply the components, and then add them together.
(a
1
,a
2
)(b
1
,b
2
) = a
1
b
1
+ a
2
b
2
For example, (3,2)(1,4) = 3*1 + 2*4 = 11. This seems kind of useless at first, but lets look at a few
examples:
Here, we can see that when the vectors are pointing the same direction, the dot product is positive. When they
are perpendicular, the dot product is zero, and when they point in opposite directions, it is negative. Basically,
it is proportional to how much the vectors are pointing in the same direction. This is a small taste of the
power of the dot product, and it's already pretty useful!
Let's say we have a guard at position G (1,3) facing in the direction D (1,1), with a 180 field of view. We
have a hero sneaking by at position H (3,2). Is he in the guard's field of view? We can find out by checking
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 4 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
the sign of the dotproduct of D and V (the vector from the guard to the hero). This gives us:
V = H-G = (3,2)-(1,3) = (3-1,2-3) = (2,-1)
DV = (1,1)(2,-1) = 1*2+1*-1 = 2-1 = 1
Since 1 is positive, the hero is in the guard's field of view!
We know that the dot product is related to the extent to which the vectors are pointing in the same direction,
but what is the exact relation? It turns out that the exact equation for the dot product is:
AB = |A||B|cos!
Where ! (pronounced "theta") is the angle between A and B. This allows us to solve for ! if we want to find
out the angle:
! = acos([AB]/[|A||B|]).
As I mentioned before, normalizing vectors makes our life easier! If A and B are normalized, then the
equation is simply:
! = acos(AB)
Let's revisit the guard scenario above, except the guard's field of view is only 120. First, we get the
normalized vectors for the direction the guard is facing (D'), and the direction from the guard to the hero (V').
Then, we check the angle between them. If it is greater than 60 (half of the field of view), then the hero is not
seen.
D' = D/|D| = (1,1)/sqrt(1
2
+1
2
) = (1,1)/sqrt(2) = (0.71,0.71)
V' = V/|V| = (2,-1)/sqrt(2
2
+(-1)
2
) = (2,-1)/sqrt(5) = (0.89,-0.45)
! = acos(D'V') = acos(0.71*0.89 + 0.71*(-0.45)) = acos(0.31) = 72
The angle between the center of the guard's vision and the hero is 72, so the guard does not see him!
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 5 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
I know this looks like a lot of work, and it is, because I'm doing it by hand. However, in a program, this is
pretty simple. Here is what this would like in Overgrowth using the C++ vector libraries I wrote (inspired by
GLSL syntax).
//Initialize starting vectors
vec2 guard_pos = vec2(1,3);
vec2 guard_facing = vec2(1,1);
vec2 hero_pos = vec2(3,2);
//Prepare normalized vectors
vec2 guard_facing_n = normalize(guard_facing);
vec2 guard_to_hero = normalize(hero_pos - guard_pos);
//Check angle
float angle = acos(dot(guard_facing_n, guard_to_hero));
Cross Product
Let's say you have a boat that has cannons that fire to the left and right. Given that the boat is facing along the
direction vector (2,1), in which directions do the cannons fire? This is easy in 2D: to rotate 90 degrees
clockwise, just flip the two vector components, and then switch the sign of the second component. (a,b)
becomes (b,-a). So, if the boat is facing along (2,1), the right-facing cannons fire towards (1,-2). The left-
facing cannons fire in the opposite direction, so we flip both signs to get: (-1,2).
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 6 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
So, what if we want to do this in 3D? Let's revisit our sailing ship. We have a vector for the direction of the
mast M, going straight up (0,1,0), and the direction of the north-north-east wind W (1,0,2), and we want to
find the direction the sail S should stick out in order to best catch the wind. The sail has to be perpendicular
to the mast, and also perpendicular to the wind. To solve this, we can use the cross product: S = M x W.
The cross product of A(a
1
,a
2
,a
3
)) and B(b
1
,b
2
,b
3
)) is:
(a
2
b
3
-a
3
b
2
, a
3
b
1
-a
1
b
3
, a
1
b
2
-a
2
b
1
)
So now we can plug in our numbers and solve our problem:
S = MxW = (0,1,0)x(1,0,2) = ([1*2-0*0], [0*1-0*2], [0*0-1*1]) = (2,0,-1)
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 7 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
This is pretty ugly to do by hand. For most graphics and game work I would recommend just encapsulating it
in a function like the one below, and never thinking about the details again.
vec3 cross(vec3 a, vec3 b) {
vec3 result;
result[0] = a[1] * b[2] - a[2] * b[1];
result[1] = a[2] * b[0] - a[0] * b[2];
result[2] = a[0] * b[1] - a[1] * b[0];
return result;
}
Another common use for the cross product in games is to find surface normals -- the direction that a surface
is facing. For example, let's take a triangle with vertex vectors A, B and C. How do we find the direction that
the triangle is facing? It seems tricky, but we have the tools to do it now. We can use subtraction to get the
direction from A to C (C-A) 'Edge 1' and A to B (B-A) 'Edge 2', and then use the cross product to find a new
vector N perpendicular to both of them... the surface normal.
Here is what that would like in code:
vec3 GetTriangleNormal(vec3 a, vec3 b, vec3 c) {
vec3 edge1 = b-a;
vec3 edge2 = c-a;
vec3 normal = cross(edge1,edge2);
return normal;
}
Fun fact: the basic expression for lighting in games is just NL, where N is the surface normal, and L is the
normalized light direction. This is what makes surfaces bright when they face towards the light, and dark
when they don't.
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 8 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
Next time
We covered what vectors are in Part 1, and now we have a solid toolbox for working with them. Next, I
would like to talk about vector spaces and matrix transformations.
This post was a lot more complicated than the last one -- is there anything confusing here that you have
questions about?
Click here for part 3!
Tweet Tweet 8
Starting Grass
Wolfire Answers Community Questions
51 Comments Wolfire Blog Login
!
Sort by Best Share
"
Join the discussion
! Reply !
jph ! 5 years ago
Great stuff,. nicely explained. For those of us that where not figuring math had much use to us,
at the time we should have been learning it,. . then went on to try to build a game. Math teatures
in highschool really should teach simple game dev. to get kids interested.
6
% &

! Reply !
M.Carabas ! 5 years ago # jph
that's so weird...
I'm a private tutor and when tutoring Grade 11s and 12s for Algebra I usually ask them if
they would like to write a simple computer game like pong (or tetris for the more
adventurous)... usually the response is very enthusiastic...
I've found this is the best way to get kids to take Algebra seriously... it's so much easier
to explain vectors in terms of bullets ricocheting off walls instead of canoes being
paddled against currents.
3
% &

! Reply !
Phil (Sumguy720) ! 5 years ago # jph
You guys should release better math teachers along with the alphas.
2
% &

TheBigCheese ! 5 years ago # jph
Favorite
$
Share
Share
Share
61 Share Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 9 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
! Reply !
TheBigCheese ! 5 years ago # jph
Yeah, honestly I don't know why they don't take that approach.
It really brings a lot of abstract down to earth.

% &

! Reply !
August Toman-Yih ! 5 years ago
Your explanation of the dot and cross products were really good; I learned more about them
from your article than a semester of calculus.
4
% &

! Reply !
jarav ! 5 years ago
Here is an applet illustrating the cross-product ( requires Flash 10 ):
http://www.physics101online.co...
2
% &

! Reply !
Mathieu ! 5 years ago
Excellent information here!
Thank you for taking the time to write all this, I would lova having all this explained when I
started game development 8 years ago, but it's still definitely useful even now.
2
% &

! Reply !
Fedhaghafoor ! 2 years ago
Product of two Vectors
Product of vectors, cross product, scalar product, dot product, vector product, properties of
vector product.
http://www.infoaw.com/article....
1
% &

! Reply !
CraigStern ! 5 years ago
Hi, this is very helpful! You lost me during the second paragraph of the section on normalizing a
vector, though. I feel like there was supposed to be a paragraph in the middle explaining how
the latter paragraph relates to the first, or defining the term "normalizing a vector," for that
matter. :S
1
% &

Adonis_paradise ! 3 years ago # CraigStern
Yes, I got lost on the normalize section too:
"For example, let's say there is a gun pointing in the direction of (1,0)
that shoots a bullet at 20 m/s. What is the velocity of the bullet?"
... 20 m/s ?? What have I missed?
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 10 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
! Reply !
... 20 m/s ?? What have I missed?

% &

! Reply !
Korban3 ! 2 years ago # Adonis_paradise
Kind of an old one to reply to, but hell why not?
20 m/s is the speed. Speed is just a unit and a time frame. Velocity is a unit, a
timeframe of travel for that unit and a direction, expressed as a normalized
(Length of one/Unit Length) vector who's magnitude is the speed.
That portion was about taking the direction of the vector, (1,0), and then making
the magnitude one. If the vector were (10, 0), then multiplying it by 20 m/s (Read
"Applying the speed to the direction to create a velocity) would give us a bullet
traveling 200m/s rather than 20 m/s.
It is a bit confusing first time through, but I love how much better he does
compared to how other things handle it. Like algebra class.

% &

! Reply !
Xeirxes ! 5 years ago
I was not aware of how useful the dot and cross products were :) I always thought that they
were the harder to use functions. I guess now I can imagine how useful they would be in
physics applications as well. A falling object bouncing off of a slanted surface, for instance,
could be easily calculated with the cross product.
Thanks for the interesting read! I'm looking forward to seeing more from you guys in the future.
1
% &

! Reply !
abreu20011 ! a year ago
Thaaaanks you!!! :D
It's very, very useful your post :)

% &

! Reply !
VlOlz Elxenoanizd ! a year ago
There's a little mistake you made in the Cross Product section. C-A is edge_2 not edge_1. Love
what you're doing. Wish school teachers would be like you, everybody then would love math! :-)

% &

! Reply !
Tom Sterkenburg ! 2 years ago
not only do i understand the math now, but i dont have to write my own functions lol

% &

Abdul ! 3 years ago
What will you recommend for further reading, I found this wonderful and amazing, I am very
keen to learn more on game programming and linear algebra. please recommend some books
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 11 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
! Reply !
keen to learn more on game programming and linear algebra. please recommend some books
for us.
thanks

% &

! Reply !
Abdul Mateen ! 3 years ago
What do you recommend for further reading, I found this astonishing and wonderful, I am very
curious to learn more, can you recommend any book ?

% &

! Reply !
Romoku ! 3 years ago
The cross product is given by the determinant of
i j k
|x1 y1 z1| = i|y2 z2| -j|x2 z2| +k|x2 y2|
|x2 y2 z2| |y3 z3| |x3 z3| |x3 y3|
|x3 y3 z3|
Simplify:
= (y2*z3-z2*y3)i - (x2*z3-x3*z2)j + (x2*y3-x3*y2)k

% &

! Reply !
dubdng ! 3 years ago
Direction vectors are normalized which seems logical, but then why do you use (2,1) in the Boat
example as the Direction Vector ?

% &

! Reply !
Ugkwan ! 3 years ago
Thanks^^

% &

! Reply !
daniellla ! 4 years ago
thanks a lot :-)

% &

! Reply !
Mohamed idris ! 5 years ago
Hi everyone plz I'm new here i mean it's my first year in the university so i want to understand
this subject cause it's very important for me really cause i didn't understand anything in the
university so if anyone can explain it to me from begging i will be thanks full for him or her this is
my email:maed_89@hotmail.com

% &

Share
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 12 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
! Reply !
ukuk ! 5 years ago
Fantastically well explained. Thumbs up! Everything suddenly became clear and simple...
Looking forward to reading the next part.

% &

! Reply !
Pete ! 5 years ago
AWESOME post. This is the best I have seen covering vector math with good examples. More
more more plz

% &

! Reply !
Oliver ! 5 years ago
Really keen for Vector Spaces and Matrix Transformations.
You have a great way of explaining things to people.
Is there somewhere I can learn the basics of Matrices? Recommendations anyone?

% &

! Reply !
M ! 5 years ago # Oliver
The library is a good place to start, this article is a rare find in its quality. Books are
generally better than most internet resources.

% &

! Reply !
Sophie Houlden ! 5 years ago
Just as motivation to continue with these in case you need it, I was just in need of a way to get
diagonal movement to move at the right speed, and then I remembered this post and it made it
extra easy for me :D
keep it up david, my input scripts will forever be a paragraph shorter :D

% &

! Reply !
konsnos ! 5 years ago
I must say, I couldn't understand it at first, and reread it many times (got confused by the
absence of coordinations in images), but now I thing I can catch on. Keep it up.

% &

! Reply !
David ! 5 years ago Mod # konsnos
I added some labels to the grid lines -- please let me know if that helps!

% &

! Reply !
konsnos ! 5 years ago # David
Yes, very helpful. Thank you.

% &

Share
Share
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 13 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
! Reply !
David ! 5 years ago Mod # konsnos
Sorry about that, I kept moving the origin around to try and keep the images from
wasting too much space. Maybe I should label the grid lines next time!

% &

! Reply !
gstamp ! 5 years ago
Excellent. More of this thanks. Most of the articles on this stuff have overly technical
explanations. This wasn't hard to follow at all.

% &

! Reply !
Rory ! 5 years ago
A future topic I suggest is optimisations! For example always compare distance squared values
instead of distance. Using distance squared saves doing a computationally expensive square
root which is quite a saving when you are doing hundreds of thousands a second.

% &

! Reply !
David ! 5 years ago Mod # Rory
That's a good idea: I always use distance squared for comparisons, as well as only
normalizing vectors when necessary, and generally avoiding sqrt()s.

% &

! Reply !
Mashandar ! 5 years ago
Thank you so much for that! I always wondered how the normal fit into things and how the
cross product worked, and you've explained it fabulously!
I'm looking forward to future posts

% &

! Reply !
Whiterabbit ! 5 years ago
I would just like to say thank you David.
I'd like to think my self as a game developer (although only really got my toes in the pool so far)
And I'm not the greatest with math (shot in the foot right ?)
this whole look at vectors in games is amazingly understandable for me.
They totally need to teach math in terms of game design as a staple :D

% &

AJS ! 5 years ago
Great article! I was feeling a bit rusty on these topics until this little gem; can't wait to read the
next part!
Btw, small typo in the post-- I think the code should read:
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 14 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
! Reply !
Btw, small typo in the post-- I think the code should read:
//Check angle
float angle = acos(dot(guard_facing_normalized, guard_to_hero));

% &

! Reply !
jamemcm03 ! 5 years ago
I know most of this from A-level mathematics but you really are great to make posts like this. It
encourages people to learn mathematics and shows them the reality of programming basic
games.
If only they did this in schools.

% &

! Reply !
timidteacher ! 5 years ago # jamemcm03
They usually do. You just forgot.

% &

! Reply !
Ahruman ! 5 years ago
" = acos(D!B) should read " = acos(D!V).
Well, acksherly, it should read " = acos(D' V') or " = acos(D# $ V#). Why yes, I am a typography
geek.
Also, your sailing example is utterly wrong. Sails dont work like that. ;-) Keep it up, though!

% &

! Reply !
John ! 5 years ago Mod # Ahruman
David was referring to the angle that would maximize the surface area of the sail
presented to the wind. He never claimed to be calculating the optimum sail trim angle for
maximizing boat speed. Though catching as much wind as possible as David suggested
is definitely optimal if you are running dead down wind.
2
% &

! Reply !
David ! 5 years ago Mod # Ahruman
You got me, I don't know anything about sails or math typography :) Fixed the error you
noticed, good catch!
1
% &

! Reply !
Net ! 5 years ago
Thanks a lot for this. Looking forward to the next part, it's what I need to learn most.

% &

Share
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 15 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
Load more comments
! Reply !
RazZziel ! 5 years ago
Should't S be WxM instead of MxW?
According to the Right Hand Rule (http://en.wikipedia.org/wiki/C... S'=MxW would face the
opposite direction.

% &

! Reply !
G. Mudskipper ! 5 years ago # RazZziel
I think the reason the cross products are backwards is because these examples use a
left-handed coordinate system (X east, Y up, Z north).
If Y was north and Z was up, the Right Hand Rule would apply.

% &

! Reply !
Kip Robinson ! 4 years ago # G. Mudskipper
Is a left-handed coordinate system common in the game industry? (I know I'm a
year late, so maybe I won't get an answer, but oh well.)

% &

! Reply !
Shayne Butts ! 2 years ago # Kip Robinson
I'm a year late, too :D
Left hand coordinate system is common for world coordinates. Screen
coordinates are a bit different. The Y axis is pointed downward and the Z
axis points into your face (I think, could be the other way).

% &

! Reply !
Ragdollsoft ! 5 years ago
Be sure to check that distance is not 0 when you normalize, or you'll get some hard to find bugs
that happen every once in a while.

% &

! Reply !
David ! 5 years ago Mod # Ragdollsoft
That's a good point, it can be frustrating to track down why your vectors are becoming
INF or NaN!

% &

! Reply !
matto1990 ! 5 years ago
A nice post. I like the addition od code in this post :)

% &

Share
Share
Share
Share
Share
Share
Share
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 16 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
Subscribe
'
Add Disqus to your site
(
Subscribe
Email Updates
Sign Up
Search
Search
Categories
Alpha
Art
Business
Design Tour
Game Design
Game Tech
Other
Overgrowth
Pre-Overgrowth
Press
Videos
Blog Roll
2D Boy
Cryptic Sea
DanLabGames
Edmund McMillen
Fun Motion
icculus.org
IndieGames Blog
Infinite Ammo
Nimblebit
Positech Blog
TIGSource
9/19/14 8:10 AM Linear algebra for game developers ~ part 2 - Wolfire Games Blog
Page 17 of 17 http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/
Join Us
Facebook
ModDB
Steam
Twitter
YouTube
2013 Wolfire Games