Skip to content

Instantly share code, notes, and snippets.

# define stage 1: transform the column feature_2 to numeric
stage_1 = StringIndexer(inputCol= 'feature_2', outputCol= 'feature_2_index')
# define stage 2: transform the column feature_3 to numeric
stage_2 = StringIndexer(inputCol= 'feature_3', outputCol= 'feature_3_index')
# define stage 3: one hot encode the numeric versions of feature 2 and 3 generated from stage 1 and stage 2
stage_3 = OneHotEncoderEstimator(inputCols=[stage_1.getOutputCol(), stage_2.getOutputCol()],
outputCols= ['feature_2_encoded', 'feature_3_encoded'])
# define stage 4: create a vector of all the features required to train the logistic regression model
stage_4 = VectorAssembler(inputCols=['feature_1', 'feature_2_encoded', 'feature_3_encoded', 'feature_4'],
outputCol='features')
Questions are not from any actual exam!!!
Q: Create a job that calculates pi to 2000 decimal points using the container with the image named perl
and the following commands issued to the container: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
Once the job has completed, check the logs to and export the result to pi-result.txt.
Solution:
@abstractart
abstractart / books.md
Last active February 28, 2025 11:33
Free Programming Ebooks - O'Reilly Media. Codeship free ebooks here - https://bit.ly/2oQ0knQ
@augbog
augbog / Free O'Reilly Books.md
Last active March 20, 2025 07:40
Free O'Reilly Books

Free O'Reilly books and convenient script to just download them.

Thanks /u/FallenAege/ and /u/ShPavel/ from this Reddit post

How to use:

  1. Take the download.sh file and put it into a directory where you want the files to be saved.
  2. cd into the directory and make sure that it has executable permissions (chmod +x download.sh should do it)
  3. Run ./download.sh and wee there it goes. Also if you do not want all the files, just simply comment the ones you do not want.
@vasanthk
vasanthk / System Design.md
Last active May 11, 2025 15:30
System Design Cheatsheet

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

  1. Clarify and agree on the scope of the system
  • User cases (description of sequences of events that, taken together, lead to a system doing something useful)
    • Who is going to use it?
    • How are they going to use it?