Live stream a database dump (or any other STDOUT) using Rails 4. Why would you want this? If you have a large database dump and want to avoid storing it in memory as Rails streams it. This allows pipe the dump directly into the http response instead of storing it as a file, sending it, and then deleting it. Let me know what you think! I've teste…

live_database_dump.rb

class DatabaseController < ApplicationController
  def database_dump
    database = Rails.configuration.database_configuration[Rails.env]["database"]
    send_file_headers!(:type => 'application/octet-stream', :filename => "#{database}_#{Time.now.to_s(:human)}.backup")

    pipe = IO.popen("pg_dump '#{database}' -F c")
    stream = response.stream
    while (line = pipe.gets)
      stream.write line
      sleep 0.0001 # HACK: Prevent server instance from sleeping forever if client disconnects during download
    end
  rescue IOError
    # Client Disconnected
  ensure
    pipe.close
    response.stream.close
  end
  
  # Code that allows us to only mix in the live methods if we're accessing the desired action

  def dispatch(name, *args)
    extend ActionController::Live if name.to_s == 'database_dump'
    super
  end
end

I booted it up again in Ruby 2.1, Rails 4.0.2, and using Puma 2.8.2 and Passenger 4.0.42.
I trigger the streaming download, and then after a few seconds, cancel it in the browser download window.

Puma

No sleep 0.0001

Downloads are streaming but cancelling one causes the server to stop responding.

Using sleep 0.0001

Downloads are streaming but queued, one will not start before the previous one is cancelled.
Random number of downloads (usually between 2 and 8) can be cancelled before the server stops responding.

ctrl + c the server consistently causes the next-response-in-line to return the following error.

Puma caught this error: Attempt to unlock a mutex which is not locked (ThreadError)
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/rack-1.5.2/lib/rack/lock.rb:22:in `unlock'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/rack-1.5.2/lib/rack/lock.rb:22:in `ensure in call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/rack-1.5.2/lib/rack/lock.rb:23:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/actionpack-4.0.2/lib/action_dispatch/middleware/static.rb:64:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/airbrake-3.1.15/lib/airbrake/user_informer.rb:16:in `_call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/airbrake-3.1.15/lib/airbrake/user_informer.rb:12:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/railties-4.0.2/lib/rails/engine.rb:511:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/railties-4.0.2/lib/rails/application.rb:97:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/railties-4.0.2/lib/rails/railtie/configurable.rb:30:in `method_missing'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/rack_patch.rb:13:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/configuration.rb:71:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/server.rb:490:in `handle_request'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/server.rb:361:in `process_client'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/server.rb:254:in `block in run'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `call'
/Users/nicholas/.rvm/gems/ruby-2.1.2/gems/puma-2.8.2/lib/puma/thread_pool.rb:92:in `block in spawn_thread'

Thread.pass

Downloads are streaming but queued, one will not start before the previous one is cancelled.
Doesn't seem to lock up the server at all.
All requests reach response.stream.close.

Puma worker threads puma -w

No effect on above behaviour

Passenger

No sleep 0.0001

Downloads are streaming and download in parallel.
Random number of downloads (usually between 2 and 8) can be cancelled before the server stops responding.
Cancelled requests don't seem to reach response.stream.close.

Using sleep 0.0001

Downloads are streaming and download in parallel.
Doesn't seem to lock up the server at all.

Thread.pass

Downloads are streaming and download in parallel.
Doesn't seem to lock up the server at all.
All requests reach response.stream.close.

@njakobsen btw, you probably don't want to use gets here: I imagine read(n) will be much more efficient. Most of the "lines" you see are going to be pretty tiny.