Code Reading 101

TWO THINGS TO
KNOW ABOUT ME

I wrote the TextMate book

My name is Jim Weirich

JAMES EDWARD
GRAY II
I wrote two books for the Pragmatic Programmers: Best of
Ruby Quiz and TextMate: Power Editing for the Mac

I’ve contributed documentation and patches for some

standard libraries, which I now help to maintain

I built FasterCSV (now CSV), HighLine (with Greg), Elif, and

a few other libraries people don’t use

I created the Ruby Quiz and ran it for the first three years
HI. I’M JAMES AND
I READ CODE.
 HOW MUCH SHOULD
YOU READ?
0% 25% 50% 75%
100%
Novice
Advanced Beginner
Competent
Proficient
Expert

My opinion based on the Dreyfus Model of Skill Acquisition.

WHY IS CODE
READING IMPORTANT?
It can show you common idioms

It’s good to practice breaking down possibly challenging

code, because you will always have to work with other’s code

Understanding how something works gives you insight into

any limitations it has

Seeing bad code helps you write better code

Knowledge workers always need more ideas

INTRODUCING
RESTCLIENT
WHAT IS RESTCLIENT?

Sinatra’s sister library (sometimes called “reverse Sinatra”)

It provides a very clean interface to RESTful web services

Simple well-written code (around 500 lines of clear code for

the core functionality)

Plus a couple of exciting features

require "rubygems"
require "rest_client"
require "json"

twitter = RestClient::Resource.new( "http://twitter.com/statuses",

:user => "JEG2",
:password => "secret" )

json = twitter["friends_timeline.json"].get
tweets = JSON.parse(json)
tweets.each do |tweet|
# ...
end

BASIC GET
Reading tweets with Twitter’s API
require "rubygems"
require "rest_client"
require "json"

twitter = RestClient::Resource.new( "http://twitter.com/statuses",

:user => "JEG2",
:password => "secret" )

json = twitter["update.json"].post(:status => "Hello from #mwrc!")

tweet = JSON.parse(json)
# ...

BASIC POST
Posting a tweet with Twitter’s API
NETWORKING CODE
DONE RIGHT
 def process_result(res)
if res.code =~ /\A2\d{2}\z/
decode res['content-encoding'], res.body if res.body
elsif %w(301 302 303).include? res.code
url = res.header['Location']
def transmit(uri, req, payload)
setup_credentials(req) if url !~ /^http/
uri = URI.parse(@url)
net = net_http_class.new(uri.host, uri.port) uri.path = "/#{url}".squeeze('/')
net.use_ssl = uri.is_a?(URI::HTTPS) url = uri.to_s
net.verify_mode = OpenSSL::SSL::VERIFY_NONE end
net.read_timeout = @timeout if @timeout
net.open_timeout = @open_timeout if @open_timeout raise Redirect, url
elsif res.code == "304"
display_log request_log raise NotModified, res
elsif res.code == "401"
net.start do |http| raise Unauthorized, res
res = http.request(req, payload) elsif res.code == "404"
display_log response_log(res) raise ResourceNotFound, res
string = process_result(res) else
raise RequestFailed, res
if string or @method == :head end
Response.new(string, res) end
else
nil
end
end
rescue EOFError
raise RestClient::ServerBrokeConnection def decode(content_encoding, body)
rescue Timeout::Error if content_encoding == 'gzip' and not body.empty?
raise RestClient::RequestTimeout Zlib::GzipReader.new(StringIO.new(body)).read
end elsif content_encoding == 'deflate'
Zlib::Inflate.new.inflate(body)
else
body
end
end
A RESTFUL SHELL
(IN 90 LOC)
$ restclient \
> get http://twitter.com/statuses/friends_timeline.json?count=1 \
> JEG2 secret
[{"text":"Sent out first round of Twitter client betas…",
"user":{"name":"Jeremy McAnally","screen_name":"jeremymcanally",…},
…}]

CURL-ISH REQUESTS
Fetching the latest tweet !om Twitter’s API
$ restclient http://twitter.com/statuses JEG2 secret
>> post "update.json", :status => "The RestClient shell is fun."
=> "{\"text\":\"The RestClient shell is fun.\",…}"
>> get "friends_timeline.json?count=1"
=> "[{\"text\":\"The RestClient shell is fun.\",…}]"

RESTFUL IRB
Interacting with the Twitter API
LOGGING IN RUBY
 $ RESTCLIENT_LOG=twitter_fun.rb restclient …
>> post "update.json", :status => "The RestClient shell is fun."
=> …
>> get "friends_timeline.json?count=1"
=> …

# twitter_fun.rb
RestClient.post "http://twitter.com/statuses/update.json",
"status=The%20RestClient%20shell%20is%20fun.",
:content_type=>"application/x-www-form-urlencoded"
# => 200 OK | application/json 379 bytes
RestClient.get "http://twitter.com/statuses/friends_timeline.json?count=1"
# => 200 OK | application/json 381 bytes

GENERATING RUBY
Interactively building a RESTful Ruby script
FASTERCSV IS
THE NEW CSV
THE LESS BORING
PARTS OF CSV
Tricky data structures are needed

Rows need ordered access for their columns

Users also like to work with them by header name

Column names often repeat

Now that we have m17n, CSV parses in the encoding of your

data (no transcoding is done on your data)
THE ARRAY-HASH-
WITH-DUPLICATES
DATA THING
require "csv" # using Ruby 1.9

data = <<END_DATA
Console,Units Sold 2007,Percent,Units Sold 2008,Percent
Wii,"719,141",49.4%,"1,184,651",49.6%
XBox 360,"333,084",22.9%,"743,976",31.1%
PlayStation 3,"404,900",27.8%,"459,777",19.3%
END_DATA
ps3 = CSV.parse(data, :headers => true, :header_converters => :symbol)[-1]

ps3[0] # => "PlayStation 3"

ps3[:percent] # => "27.8%"
ps3[:percent, 3] # => "19.3%"
ps3[:percent, ps3.index(:units_sold_2008)] # => "19.3%"

CSV::ROW
The various ways to refer to data
M17N IN ACTION
@io = if data.is_a? String then StringIO.new(data) else data end
@encoding = if @io.respond_to? :internal_encoding
@io.internal_encoding || @io.external_encoding
elsif @io.is_a? StringIO
@io.string.encoding
end
@encoding ||= Encoding.default_internal || Encoding.default_external

def encode_re(*chunks)
Regexp.new(encode_str(*chunks))
end

def encode_str(*chunks)
chunks.map { |chunk| chunk.encode(@encoding.name) }.join
end

sample = read_to_char(1024)
sample += read_to_char(1) if sample[-1..-1] == encode_str("\r") and
not @io.eof?

if sample =~ encode_re("\r\n?|\n")
@row_sep = $&
break
end
OTHER POINTS
OF INTEREST
The parser

Ruby 1.9’s CSV library uses primarily one ugly regular

expression from Master Regular Expressions

The old FasterCSV has switched to a non-regex parser to

dodge some regex engine weaknesses

FasterCSV::Table is another interesting data structure that

can work in columns or rows
 BJ, SLAVE,
AND TERMINATOR
WHY THESE
LIBRARIES?
They teach how to build multiprocessing Unix software

Covers Thread and fork(), apart and together

Using pipes

Signal handling

And much more

Robust code written by an expert

HOW TO ASK YOUR
CHILD TO KILL YOU
def terminate options = {}, &block
options = { :seconds => Float(options).to_i } unless Hash === options

seconds = getopt :seconds, options

trap = getopt :trap, options,
lambda{ eval("raise(::Terminator::Error, '#{ seconds }s')", block) }

handler = Signal.trap(signal, &trap)

plot_to_kill pid, :in => seconds, :with => signal

begin
block.call
ensure
Signal.trap(signal, handler)
end
end
def plot_to_kill pid, options = {}
seconds = getopt :in, options
signal = getopt :with, options
process.puts [pid, seconds, signal].join(' ')
process.flush
end

fattr :process do
process = IO.popen "#{ ruby } #{ program.inspect }", 'w+'
at_exit do
begin
Process.kill -9, process.pid
rescue Object
end
end
process.sync = true
process
end
fattr :program do
code = <<-code
while(( line = STDIN.gets ))
pid, seconds, signal, *ignored = line.strip.split

pid = Float(pid).to_i
seconds = Float(seconds)
signal = Float(signal).to_i rescue String(signal)

sleep seconds

begin
Process.kill signal, pid
rescue Object
end
end
code
tmp = Tempfile.new "#{ ppid }-#{ pid }-#{ rand }"
tmp.write code
tmp.close
tmp.path
end
OTHER POINTS
OF INTEREST
bj – A robust background priority queue for Rails

Noticing changes from the outside world via signals

Managing and monitoring an external job

slave – Trivial multiprocessing with built-in IPC

How to set up a “heartbeat” between processes

How to run DRb over Unix domain sockets

 THE ART OF
CODE READING
PROCESS TIPS

Take a deep breath and relax

Not all code sucks

Don’t start with Rails

There’s a ton of great stuff in there

But it’s a big and complex beast

Have a goal
GETTING THE CODE

gem unpack GEM_NAME

Use anonymous VCS access to pull a local copy of the code

Open it in your standard environment as you would if you

were going to edit it
FINDING THINGS
Try the conventions first

Executables are probably in the bin/ directory

Look for MyModule::MyClass in lib/my_module/

my_class.rb

Look for methods in super classes and mixed in modules

But remember Ruby is quite dynamic

Hunt for some “core extensions”

UNDERSTANDING
THE CODE

Start with the tests/specs if there are any

Check for an “examples/” directory

Try to load and play with certain classes in isolation

irb -r a_class_to_play_with

Remember Ruby’s reflection methods, like methods()

 QUESTIONS?
About code or other important topics…