Skip to content

Instantly share code, notes, and snippets.

@nofxx
Last active December 24, 2015 13:49
Show Gist options
  • Select an option

  • Save nofxx/6807671 to your computer and use it in GitHub Desktop.

Select an option

Save nofxx/6807671 to your computer and use it in GitHub Desktop.
Centralized Ruby/Rails/MongoDB/Redis/Sidekiq/Elasticsearch/Haproxy/PG/OS -> syslog-ng -> logstash -> elasticsearch -> kibana =D
#
# Logstash conf!
#
input {
tcp {
host => "0.0.0.0"
port => 514
type => syslog
}
udp {
host => "0.0.0.0"
port => 514
type => syslog
}
}
filter {
grok {
match => { "message" => "(?:<%{INT:priority}>)?%{SYSLOGBASE2} (?:\s?%{LOGLEVEL:log_level} )?(?:\s?%{WORD:log_format}: )?%{GREEDYDATA:syslog_message}" }
}
syslog_pri { }
if ("_grokparsefailure" in [tags]) {
mutate {
replace => ["@message", "TOPARSE: %{message}"]
}
} else if [log_format] == "json" {
mutate {
gsub => ["syslog_message", "@timestamp", "syslog_timestamp"]
}
json {
source => "syslog_message"
}
mutate {
replace => ["@message", "%{message}"]
}
} else {
mutate {
replace => [ "@message", "%{syslog_message}" ]
}
}
mutate {
remove_field => [
"syslog_hostname", "syslog_message", "syslog_timestamp",
"syslog_severity_code", "syslog_facility_code",
"message" #facility_label", "severity_label"
]
}
}
output {
stdout {
debug => true
}
elasticsearch {
# bind_host => "foo"
host => "localhost"
embedded => false
}
}
require 'log4r'
require 'log4r/yamlconfigurator'
require 'log4r/outputter/syslogoutputter'
module App
class Application < Rails::Application
log4r_config= YAML.load_file(File.join(File.dirname(__FILE__),"log4r.yml"))
Log4r::YamlConfigurator.decode_yaml( log4r_config['log4r_config'] )
config.logger = Log4r::Logger['rails']
config.log_level = Log4r::DEBUG
Mongoid.logger = Log4r::Logger['mongoid']
Moped.logger = Log4r::Logger['mongoid']
...
# Lograge
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Logstash.new
destination logstash { udp("logs.xxx.com"); };
log { source(src); destination(logstash); };
@nofxx

nofxx commented Oct 3, 2013

Copy link
Copy Markdown
Author

Syslog parameters correctly parsed: LOGLEVEL (INFO|WARN...), LOGTITLE (logtitle: some log...)

@anlek

anlek commented Aug 16, 2014

Copy link
Copy Markdown

How does your log4r.yml look like?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment