Skip to content

Instantly share code, notes, and snippets.

@steelThread

steelThread/1-Purpose.md

Created July 2, 2011 20:52
Show Gist options
  • Save steelThread/1061636 to your computer and use it in GitHub Desktop.
Save steelThread/1061636 to your computer and use it in GitHub Desktop.
Download ZIP
Heroku Bench Raw Http runs
Raw
1-Purpose.md

Purpose

Take a look at a couple of very simple application stacks deployed to Heroku's cedar PaaS to understand just the raw http processing capabilities of a support ruby frameworks and node.

The Contenders

  • Rails
  • Sinatra
  • Golaith
  • NodeJS

The Setup

Simple static hello world controller.

Why?

To understand the basic performance characteristics with a focus on just the frameworks request processing throughput.

Should also

Will also perform a simple test accessing a single doc in mongodb. This can help us understand and compare non blocking IO frameworks with respect to request through put.

Raw
2-rubes.rb
require 'json'
class HelloController < ApplicationController
def index
render :text => {hello: 'Newman'}.to_json
end
end
#####
require 'sinatra'
require 'json'
get '/' do
content_type :json
{:hello => :newman}.to_json
end
#####
require 'goliath'
require 'json'
class Hello < Goliath::API
def response(env)
[200, {}, {hello: 'Newman'}.to_json]
end
end
Raw
3-node.js
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
res.end(JSON.stringify({hello: 'Newman'}));
}).listen(process.env.PORT || 3000);
Raw
5-single-dnyo
$> ab -n 2000 -c 100 http://burma-rails.herokuapp.com/
Server Software: thin
Server Hostname: burma-rails.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 4.856 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 622000 bytes
HTML transferred: 36000 bytes
Requests per second: 411.82 [#/sec] (mean)
Time per request: 242.823 [ms] (mean)
Time per request: 2.428 [ms] (mean, across all concurrent requests)
Transfer rate: 125.07 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 3 4 3.2 3 61
Processing: 14 234 31.2 239 321
Waiting: 14 234 31.2 239 321
Total: 17 238 31.0 242 324
Percentage of the requests served within a certain time (ms)
50% 242
66% 248
75% 251
80% 255
90% 263
95% 275
98% 289
99% 294
100% 324 (longest request)
$> ab -n 5000 -c 100 http://burma-sinatra.herokuapp.com/
Server Software: thin
Server Hostname: burma-sinatra.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 1.352 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 306612 bytes
HTML transferred: 36072 bytes
Requests per second: 1478.82 [#/sec] (mean)
Time per request: 67.621 [ms] (mean)
Time per request: 0.676 [ms] (mean, across all concurrent requests)
Transfer rate: 221.40 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 24 134.3 5 981
Processing: 12 42 17.4 40 242
Waiting: 11 41 16.2 39 242
Total: 15 65 136.2 45 1050
Percentage of the requests served within a certain time (ms)
50% 45
66% 53
75% 57
80% 60
90% 68
95% 74
98% 249
99% 1024
100% 1050 (longest request)
$> ab -n 2000 -c 100 http://burma-goliath.herokuapp.com/
Server Software: Goliath
Server Hostname: burma-goliath.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 0.979 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 260260 bytes
HTML transferred: 36036 bytes
Requests per second: 2043.35 [#/sec] (mean)
Time per request: 48.939 [ms] (mean)
Time per request: 0.489 [ms] (mean, across all concurrent requests)
Transfer rate: 259.67 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 4 1.8 3 10
Processing: 12 34 11.0 34 249
Waiting: 12 34 8.8 34 230
Total: 18 38 11.2 38 259
Percentage of the requests served within a certain time (ms)
50% 38
66% 41
75% 42
80% 43
90% 46
95% 49
98% 57
99% 63
100% 259 (longest request)
$> ab -n 2000 -c 100 http://burma-node.herokuapp.com/
Server Software:
Server Hostname: burma-node.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 0.704 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 215960 bytes
HTML transferred: 36000 bytes
Requests per second: 2841.04 [#/sec] (mean)
Time per request: 35.198 [ms] (mean)
Time per request: 0.352 [ms] (mean, across all concurrent requests)
Transfer rate: 299.59 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 3 0.6 3 6
Processing: 15 31 8.6 29 75
Waiting: 15 31 8.6 29 75
Total: 19 34 8.7 33 78
Percentage of the requests served within a certain time (ms)
50% 33
66% 36
75% 38
80% 40
90% 47
95% 52
98% 59
99% 62
100% 78 (longest request)
Raw
6-five-dynos.txt
$> ab -n 2000 -c 100 http://burma-rails.herokuapp.com/
Server Software: thin
Server Hostname: burma-rails.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 1.286 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 623555 bytes
HTML transferred: 36090 bytes
Requests per second: 1555.10 [#/sec] (mean)
Time per request: 64.305 [ms] (mean)
Time per request: 0.643 [ms] (mean, across all concurrent requests)
Transfer rate: 473.48 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 3 0.5 2 5
Processing: 8 60 30.4 56 192
Waiting: 8 60 30.4 56 192
Total: 10 62 30.5 58 197
ERROR: The median and mean for the initial connection time are more than twice the standard
deviation apart. These results are NOT reliable.
Percentage of the requests served within a certain time (ms)
50% 58
66% 70
75% 79
80% 84
90% 99
95% 117
98% 144
99% 174
100% 197 (longest request)
$> ab -n 2000 -c 100 http://burma-sinatra.herokuapp.com/
Server Software: thin
Server Hostname: burma-sinatra.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 0.867 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 306306 bytes
HTML transferred: 36036 bytes
Requests per second: 2307.52 [#/sec] (mean)
Time per request: 43.337 [ms] (mean)
Time per request: 0.433 [ms] (mean, across all concurrent requests)
Transfer rate: 345.12 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 3 0.6 3 5
Processing: 13 39 9.0 39 72
Waiting: 13 39 9.0 39 72
Total: 16 42 9.1 42 75
Percentage of the requests served within a certain time (ms)
50% 42
66% 45
75% 47
80% 49
90% 54
95% 59
98% 65
99% 69
100% 75 (longest request)
$> ab -n 2000 -c 100 http://burma-goliath.herokuapp.com/
Server Software: Goliath
Server Hostname: burma-goliath.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 0.724 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 260000 bytes
HTML transferred: 36000 bytes
Requests per second: 2760.63 [#/sec] (mean)
Time per request: 36.224 [ms] (mean)
Time per request: 0.362 [ms] (mean, across all concurrent requests)
Transfer rate: 350.47 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 3 0.6 3 6
Processing: 7 32 13.0 30 82
Waiting: 7 32 13.0 30 81
Total: 9 35 13.1 33 85
Percentage of the requests served within a certain time (ms)
50% 33
66% 38
75% 42
80% 45
90% 53
95% 60
98% 69
99% 73
100% 85 (longest request)
$> ab -n 2000 -c 100 http://burma-node.herokuapp.com/
Server Software:
Server Hostname: burma-node.herokuapp.com
Server Port: 80
Document Path: /
Document Length: 18 bytes
Concurrency Level: 100
Time taken for tests: 0.634 seconds
Complete requests: 2000
Failed requests: 0
Write errors: 0
Total transferred: 216000 bytes
HTML transferred: 36000 bytes
Requests per second: 3156.44 [#/sec] (mean)
Time per request: 31.681 [ms] (mean)
Time per request: 0.317 [ms] (mean, across all concurrent requests)
Transfer rate: 332.91 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 4 5.2 3 35
Processing: 7 25 26.9 20 260
Waiting: 7 24 23.1 20 259
Total: 9 29 27.2 23 262
Percentage of the requests served within a certain time (ms)
50% 23
66% 26
75% 28
80% 30
90% 50
95% 56
98% 61
99% 254
100% 262 (longest request)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment