Skip to content

Instantly share code, notes, and snippets.

@laser
Last active July 26, 2018 22:40

Revisions

  1. laser revised this gist Jul 26, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -174,6 +174,7 @@ Ping received pong
    Ping received pong
    Ping received pong
    ping finished
    ```

    ## Discussion

  2. laser revised this gist Jul 26, 2018. 1 changed file with 13 additions and 0 deletions.
    13 changes: 13 additions & 0 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -162,6 +162,19 @@ start_ping(Pong_Node) ->
    spawn(tut17, ping, [3, Pong_Node]).
    ```

    ```erlang
    (pong@gollum)1> tut17:start_pong().
    true
    ```

    ```erlang
    (ping@kosken)1> tut17:start_ping(pong@gollum).
    <0.37.0>
    Ping received pong
    Ping received pong
    Ping received pong
    ping finished

    ## Discussion

    ### Topic A
  3. laser revised this gist Jul 26, 2018. 1 changed file with 2 additions and 3 deletions.
    5 changes: 2 additions & 3 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -3,9 +3,8 @@
    General rules for making a distributed book club like this not-awful:

    - keep your mic muted unless you're speaking
    - serializing communication is *very important*
    - raise your hand if you have something to say and wait for someone to call on you
    - if you're raising your hand and nobody calls on you, type a message into the chat
    - raise your hand if you have something to say and wait for someone to call on you
    - if you're raising your hand and nobody calls on you, type a message into the chat

    ## Chapter 1

  4. laser revised this gist Jul 26, 2018. 1 changed file with 3 additions and 1 deletion.
    4 changes: 3 additions & 1 deletion notes.md
    Original file line number Diff line number Diff line change
    @@ -3,7 +3,9 @@
    General rules for making a distributed book club like this not-awful:

    - keep your mic muted unless you're speaking
    - raise your hand if you have something to say and wait for someone to call on you
    - serializing communication is *very important*
    - raise your hand if you have something to say and wait for someone to call on you
    - if you're raising your hand and nobody calls on you, type a message into the chat

    ## Chapter 1

  5. laser revised this gist Jul 26, 2018. 1 changed file with 48 additions and 3 deletions.
    51 changes: 48 additions & 3 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,10 @@
    ## Housekeeping

    General rules for making a distributed book club like this not-awful:

    - keep your mic muted unless you're speaking
    - raise your hand if you have something to say and wait for someone to call on you

    ## Chapter 1

    ### Why Erlang?
    @@ -120,7 +127,39 @@ normal() ->
    - nodes in cluster can be distributed on same host or on different ones
    - message passing model is the same regardless of clustered/not

    > Out of the box, Erlang distribution is not designed to support systems operating across potentially hostile environments such as the Internet or shared cloud instances.
    ```erlang
    -module(tut17).

    -export([start_ping/1, start_pong/0, ping/2, pong/0]).

    ping(0, Pong_Node) ->
    {pong, Pong_Node} ! finished,
    io:format("ping finished~n", []);

    ping(N, Pong_Node) ->
    {pong, Pong_Node} ! {ping, self()},
    receive
    pong ->
    io:format("Ping received pong~n", [])
    end,
    ping(N - 1, Pong_Node).

    pong() ->
    receive
    finished ->
    io:format("Pong finished~n", []);
    {ping, Ping_PID} ->
    io:format("Pong received ping~n", []),
    Ping_PID ! pong,
    pong()
    end.

    start_pong() ->
    register(pong, spawn(tut17, pong, [])).

    start_ping(Pong_Node) ->
    spawn(tut17, ping, [3, Pong_Node]).
    ```

    ## Discussion

    @@ -132,9 +171,9 @@ normal() ->

    ### Topic B

    > Each process is allowed to execute a predefined number of reductions before being preempted, allowing the process at the head of the run queue to execute. The number of reductions each process is allowed to execute before being suspended and the reduction count of each instruction *are purposely not documented to discourage premature optimization*, because the reduction count and the total number of reductions the scheduler allows a process to execute may change from one release and hardware architecture to another.
    > Out of the box, Erlang distribution is not designed to support systems operating across potentially hostile environments such as the Internet or shared cloud instances.
    1. Is this good?
    Does anyone know why this is the case?

    ### Topic C

    @@ -151,6 +190,12 @@ normal() ->

    ### Topic E

    > Each process is allowed to execute a predefined number of reductions before being preempted, allowing the process at the head of the run queue to execute. The number of reductions each process is allowed to execute before being suspended and the reduction count of each instruction *are purposely not documented to discourage premature optimization*, because the reduction count and the total number of reductions the scheduler allows a process to execute may change from one release and hardware architecture to another.
    1. Is this good?

    ### Topic F

    > You need to think hard about your requirements and properties, making certain you pick the right libraries and design patterns that ensure the final system behaves the way you want it to and does what you originally intended. In your quest, you will have to make tradeoffs that are mutually dependent — tradeoffs on time, resources, and features and tradeoffs on availability, scalability, and reliability. No ready-made library can help you if you do not know what you want to get out of your system.
    1. What are some of the tradeoff decisions on availability, scalability, and reliability that you've had to make on client projects?
  6. laser revised this gist Jul 26, 2018. 1 changed file with 12 additions and 2 deletions.
    14 changes: 12 additions & 2 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -110,15 +110,25 @@ normal() ->

    ### Deploying

    > At any one time, two versions of a module may exist in the virtual machine: the old and current versions.
    - can deploy new software w/out taking the node offline
    - can upgrade a module by compiling it in the shell or explicitly loading it
    - at any one time, two versions of a module may exist in the VM: old and current

    ### Distributed Erlang

    - processes can transparently spawn processes on other nodes, by name of node
    - nodes in cluster can be distributed on same host or on different ones
    - message passing model is the same regardless of clustered/not

    > Out of the box, Erlang distribution is not designed to support systems operating across potentially hostile environments such as the Internet or shared cloud instances.
    ## Discussion

    ### Topic A

    > At any one time, two versions of a module may exist in the virtual machine: the old and current versions.
    1. What are some considerations for deploying code in this manner?
    1. What are some of the considerations of running two modules of different versions in a cluster at the same time?

    ### Topic B

  7. laser revised this gist Jul 26, 2018. 1 changed file with 35 additions and 8 deletions.
    43 changes: 35 additions & 8 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -52,6 +52,7 @@ Language features:
    ### Processes and Message Passing

    - processes spawn other processes and send messages to them
    - messages are stored in a process's mailbox
    - processes do not share memory (messages are copied from stack of sending process to heap of receiver)

    ### Selective Receive
    @@ -91,29 +92,55 @@ normal() ->
    > This lets you only care about the messages that are useful. Ignoring some messages to handle them later in the manner described above is the essence of selective receives. While they're useful, the problem with them is that if your process has a lot of messages you never care about, reading useful messages will actually take longer and longer (and the processes will grow in size too).
    ### Scheduler

    - for ever core, BEAM starts a thread which runs a scheduler
    - each scheduler is responsible for a group of processes
    - at any one time, a process from each scheduler executes in parallel on each core
    - schedulers try to balance CPU time across all processes ("What the BEAM virtual machine tries to do is avoid cases where processes in a run queue with 10 processes get twice as much CPU time as those in a run queue with 20 processes.")
    - processes are migrated between run queues (cores)
    - schedulers can preempt processes based on workload they've executed (*reduction count* - kinda works like Ethereum "gas")

    ### Supervision

    - links (bi-directional)
    - when process X dies, process Y will also die (if linked)
    - monitors (one direction)
    - used for resource management, e.g. distributed lock

    ### Deploying

    > At any one time, two versions of a module may exist in the virtual machine: the old and current versions.
    ## Discussion

    ### Topic A

    > At any one time, two versions of a module may exist in the virtual machine: the old and current versions.
    1. What are some considerations for deploying code in this manner?

    ### Topic B

    > Each process is allowed to execute a predefined number of reductions before being preempted, allowing the process at the head of the run queue to execute. The number of reductions each process is allowed to execute before being suspended and the reduction count of each instruction *are purposely not documented to discourage premature optimization*, because the reduction count and the total number of reductions the scheduler allows a process to execute may change from one release and hardware architecture to another.
    1. Is this good?

    ### Topic C

    > For decades, the computing industry has explored how programming languages can support distribution. Designing general-purpose languages is difficult enough; designing them to support distribution significantly adds to that difficulty. Because of this, a common approach is to add distribution support to nondistributed programming languages through optional libraries. This approach has the benefit of allowing distribution support to evolve separately from the language itself, but it often suffers from an impedance mismatch with the language, feeling to developers as if it were “bolted on.”
    1. What are some examples of tools in other languages (addressing distributed systems) which feel "bolted on?"

    ### Topic B
    ### Topic D

    > [...] consider that Erlang clusters do not require master or leader nodes, which means that using them for peer-to-peer systems of replicas works well
    1. What are some of the benefits of a leaderless cluster?
    1. What are some of the benefits of a hierarchical design?

    ### Topic C
    ### Topic E

    > You need to think hard about your requirements and properties, making certain you pick the right libraries and design patterns that ensure the final system behaves the way you want it to and does what you originally intended. In your quest, you will have to make tradeoffs that are mutually dependent — tradeoffs on time, resources, and features and tradeoffs on availability, scalability, and reliability. No ready-made library can help you if you do not know what you want to get out of your system.
    1. What are some of the tradeoff decisions on availability, scalability, and reliability that you've had to make on client projects?

    ### Topic D

    > Erlang lets you run all your code on top of a virtual machine highly optimized for concurrency, with a per-process garbage collector, yielding predictable and simple system behavior.
    1. Why is per-process garbage collection good?
  8. laser revised this gist Jul 26, 2018. 1 changed file with 15 additions and 6 deletions.
    21 changes: 15 additions & 6 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -43,11 +43,20 @@ What do these terms mean?
    Language features:

    - anonymous functions
    - TCO
    - structural pattern matching w/out exhaustiveness checking
    - tail-call optimization
    - structural pattern matching
    - tuples, maps, lists
    - list comprehensions
    - macros

    #### Selective Receive
    ### Processes and Message Passing

    - processes spawn other processes and send messages to them
    - processes do not share memory (messages are copied from stack of sending process to heap of receiver)

    ### Selective Receive

    > This function will build a list of all messages with those with a priority above 10 coming first:
    ```erlang
    important() ->
    @@ -76,11 +85,11 @@ normal() ->
    [high,high,low,low]
    ```

    > When messages are sent to a process, they're stored in the mailbox until the process reads them and they match a pattern there. As said in the previous chapter, the messages are stored in the order they were received.
    ### Processes and Message Passing
    > When there is no way to match a given message, it is put in a save queue and the next message is tried. If the second message matches, the first message is put back on top of the mailbox to be retried later.
    - processes spawn other processes and send messages to them
    - processes do not share memory (messages are copied from stack of sending process to heap of receiver)
    > This lets you only care about the messages that are useful. Ignoring some messages to handle them later in the manner described above is the essence of selective receives. While they're useful, the problem with them is that if your process has a lot of messages you never care about, reading useful messages will actually take longer and longer (and the processes will grow in size too).
    ## Discussion

  9. laser revised this gist Jul 26, 2018. 1 changed file with 53 additions and 4 deletions.
    57 changes: 53 additions & 4 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -18,20 +18,69 @@ What do these terms mean?
    ### What's OTP?

    > OTP is short for Open Telecom Platform, a name coined by Bjarne Däcker
    > OTP is a domain-independent set of frameworks, principles, and patterns that guide and support the structure, design, implementation, and deployment of Erlang systems
    > OTP is short for Open Telecom Platform
    ### OPT Components

    1. Erlang: Includes the semantics of the language and its underlying Virtual Machine
    1. Tools and libs: applications (erts, kernel, stdlib, dialyzer, debugger, etc.), FFI (ei, erl_interface, jinterface), snmp, mnesia (anybody use this?), eunit
    1. Design principles: behaviors (e.g. worker and supervisor processes) which are seen as a formalizaton of concurrent design patterns

    ### Distribution, Infrastructure, Multicore

    > In Erlang, processes communicate via asynchronous message passing. This works even if a process is on a remote node because the Erlang virtual machine supports passing messages from one node to another.
    > When one node joins another, it also becomes aware of any nodes already known to the other. In this manner, all the nodes in a cluster form a mesh, enabling any process to send a message to another process on any other node in the cluster.
    > Each node in the cluster also automatically tracks liveness of other nodes in order to become aware of nonresponsive nodes.
    *Implication: Programming for a clustered environment should feel identical to programming for a non-clustered environment.*

    ## Chapter 2

    ### Erlang

    Language features:

    - anonymous functions
    - TCO
    - structural pattern matching w/out exhaustiveness checking
    - list comprehensions

    #### Selective Receive

    ```erlang
    important() ->
    receive
    {Priority, Message} when Priority > 10 ->
    [Message | important()]
    after 0 ->
    normal()
    end.

    normal() ->
    receive
    {_, Message} ->
    [Message | normal()]
    after 0 ->
    []
    end.
    ```

    ```erlang
    1> c(multiproc).
    {ok,multiproc}
    2> self() ! {15, high}, self() ! {7, low}, self() ! {1, low}, self() ! {17, high}.
    {17,high}
    3> multiproc:important().
    [high,high,low,low]
    ```

    ### Erlang Nodes

    ### Processes and Message Passing

    - processes spawn other processes and send messages to them
    - processes do not share memory (messages are copied from stack of sending process to heap of receiver)

    ## Discussion

  10. laser revised this gist Jul 26, 2018. 1 changed file with 18 additions and 2 deletions.
    20 changes: 18 additions & 2 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -29,17 +29,33 @@ What do these terms mean?
    1. Design principles: behaviors (e.g. worker and supervisor processes) which are seen as a formalizaton of concurrent design patterns


    ### Erlang Nodes



    ## Discussion

    > You need to think hard about your requirements and properties, making certain you pick the right libraries and design patterns that ensure the final system behaves the way you want it to and does what you originally intended. In your quest, you will have to make tradeoffs that are mutually dependent — tradeoffs on time, resources, and features and tradeoffs on availability, scalability, and reliability. No ready-made library can help you if you do not know what you want to get out of your system.
    ### Topic A

    1. What are some of the tradeoff decisions on availability, scalability, and reliability that you've had to make on client projects?
    > For decades, the computing industry has explored how programming languages can support distribution. Designing general-purpose languages is difficult enough; designing them to support distribution significantly adds to that difficulty. Because of this, a common approach is to add distribution support to nondistributed programming languages through optional libraries. This approach has the benefit of allowing distribution support to evolve separately from the language itself, but it often suffers from an impedance mismatch with the language, feeling to developers as if it were “bolted on.”
    1. What are some examples of tools in other languages (addressing distributed systems) which feel "bolted on?"

    ### Topic B

    > [...] consider that Erlang clusters do not require master or leader nodes, which means that using them for peer-to-peer systems of replicas works well
    1. What are some of the benefits of a leaderless cluster?
    1. What are some of the benefits of a hierarchical design?

    ### Topic C

    > You need to think hard about your requirements and properties, making certain you pick the right libraries and design patterns that ensure the final system behaves the way you want it to and does what you originally intended. In your quest, you will have to make tradeoffs that are mutually dependent — tradeoffs on time, resources, and features and tradeoffs on availability, scalability, and reliability. No ready-made library can help you if you do not know what you want to get out of your system.
    1. What are some of the tradeoff decisions on availability, scalability, and reliability that you've had to make on client projects?

    ### Topic D

    > Erlang lets you run all your code on top of a virtual machine highly optimized for concurrency, with a per-process garbage collector, yielding predictable and simple system behavior.
    1. Why is per-process garbage collection good?
  11. laser revised this gist Jul 26, 2018. 1 changed file with 35 additions and 5 deletions.
    40 changes: 35 additions & 5 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -1,15 +1,45 @@
    ## Chapter 1

    > Erlang/OTP is unique among programming languages and frameworks in the breadth, depth, and consistency of the features it provides for scalable, fault-tolerant systems with requirements for high availability
    ### Why Erlang?

    > Erlang/OTP is unique among programming languages and frameworks in the breadth, depth, and consistency of the features it provides for scalable, fault-tolerant systems with requirements for high availability.
    What do these terms mean?

    > Scalable refers to how well a computing system can adapt to changes in load or available resources
    > Scalable refers to how well a computing system can adapt to changes in load or available resources.
    > Distributed refers to how systems are clustered together and interact with each other
    > Distributed refers to how systems are clustered together and interact with each other.
    > Systems that are fault tolerant continue to operate predictably when things in their environment are failing.
    > By soft real-time, we mean the predictability of response and latency, handling a constant throughput, and guaranteeing a response within an acceptable time frame
    > By soft real-time, we mean the predictability of response and latency, handling a constant throughput, and guaranteeing a response within an acceptable time frame.
    > High availability minimizes or completely eliminates downtime as a result of bugs, outages, upgrades, or other operational activities.
    ### What's OTP?

    > OTP is short for Open Telecom Platform, a name coined by Bjarne Däcker
    > OTP is a domain-independent set of frameworks, principles, and patterns that guide and support the structure, design, implementation, and deployment of Erlang systems
    ### OPT Components

    1. Erlang: Includes the semantics of the language and its underlying Virtual Machine
    1. Tools and libs: applications (erts, kernel, stdlib, dialyzer, debugger, etc.), FFI (ei, erl_interface, jinterface), snmp, mnesia (anybody use this?), eunit
    1. Design principles: behaviors (e.g. worker and supervisor processes) which are seen as a formalizaton of concurrent design patterns


    ## Discussion

    > You need to think hard about your requirements and properties, making certain you pick the right libraries and design patterns that ensure the final system behaves the way you want it to and does what you originally intended. In your quest, you will have to make tradeoffs that are mutually dependent — tradeoffs on time, resources, and features and tradeoffs on availability, scalability, and reliability. No ready-made library can help you if you do not know what you want to get out of your system.
    1. What are some of the tradeoff decisions on availability, scalability, and reliability that you've had to make on client projects?

    > [...] consider that Erlang clusters do not require master or leader nodes, which means that using them for peer-to-peer systems of replicas works well
    1. What are some of the benefits of a leaderless cluster?
    1. What are some of the benefits of a hierarchical design?

    > Erlang lets you run all your code on top of a virtual machine highly optimized for concurrency, with a per-process garbage collector, yielding predictable and simple system behavior.
    > High availability minimizes or completely eliminates downtime as a result of bugs, outages, upgrades, or other operational activities.
    1. Why is per-process garbage collection good?
  12. laser created this gist Jul 26, 2018.
    15 changes: 15 additions & 0 deletions notes.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,15 @@
    ## Chapter 1

    > Erlang/OTP is unique among programming languages and frameworks in the breadth, depth, and consistency of the features it provides for scalable, fault-tolerant systems with requirements for high availability
    What do these terms mean?

    > Scalable refers to how well a computing system can adapt to changes in load or available resources
    > Distributed refers to how systems are clustered together and interact with each other
    > Systems that are fault tolerant continue to operate predictably when things in their environment are failing.
    > By soft real-time, we mean the predictability of response and latency, handling a constant throughput, and guaranteeing a response within an acceptable time frame
    > High availability minimizes or completely eliminates downtime as a result of bugs, outages, upgrades, or other operational activities.