lihonosov

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

lihonosov

case class RemoteIteratorWrapper[T](underlying: org.apache.hadoop.fs.RemoteIterator[T]) extends scala.collection.AbstractIterator[T] with scala.collection.Iterator[T] {
  def hasNext = underlying.hasNext
  def next() = underlying.next()
}

object Conversions {
  implicit def remoteIterator2ScalaIterator[T](underlying: org.apache.hadoop.fs.RemoteIterator[T]) : scala.collection.Iterator[T] = RemoteIteratorWrapper[T](underlying)
}

lihonosov

Apache Hadoop - add native libraries

If native libraries are not available the following message is displayed with every hadoop command: hadoop checknative

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

• Clone hadoop source code

lihonosov

Studying for a Tech Interview Sucks, so Here's a Cheat Sheet to Help

This list is meant to be a both a quick guide and reference for further research into these topics. It's basically a summary of that comp sci course you never took or forgot about, so there's no way it can cover everything in depth. It also will be available as a gist on Github for everyone to edit and add to.

Data Structure Basics

###Array ####Definition:

• Stores data elements based on an sequential, most commonly 0 based, index.
• Based on tuples from set theory.

lihonosov

Latency Comparison Numbers
--------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

lihonosov

#!/usr/bin/env ruby -w

# simple script to count row changes in a svn repo

def status dir=''
    files = []
    svn = `svn status #{dir}`
    svn.split("\n").each do |line|
        next if line =~ /^[^A-Z]/
        line =~ /^[A-Z]\s+.*\s+(.+)$/

lihonosov

realm=Sonatype Nexus Repository Manager
host=nexus.company.com
user=admin
password=admin123

lihonosov

ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:DHE-RSA-CAMELLIA256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:DHE-RSA-CAMELLIA128-SHA:DES-CBC3-SHA';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:50m;
ssl_dhparam /etc/nginx/ssl/dh4096.pem;
ssl_stapling on;
ssl_stapling_verify on;
ssl_ecdh_curve secp384r1;

lihonosov

theme crunch

lihonosov

export ANT_HOME=/opt/ant
export M2_HOME=/opt/maven
export JAVA_HOME=/opt/java
export GRADLE_HOME=/opt/gradle
export PLAY_HOME=/opt/play
export SBT_HOME=/opt/sbt
export SCALA_HOME=/opt/scala

export PATH=$PATH:$ANT_HOME/bin:$M2_HOME/bin:$JAVA_HOME/bin:$GRADLE_HOME/bin:$PLAY_HOME:$SBT_HOME/bin:$SCALA_HOME/bin