thanhleviet

--[[

SLURM job submit filter for QOS

Some code and ideas pulled from https://github.com/edf-hpc/slurm-llnl-misc-plugins/blob/master/job_submit.lua

--]]

--########################################################################--
--

thanhleviet

#!/usr/bin/env python3
"""
Running this script is (intended to be) equivalent to running the following Snakefile:

include: "pipeline.conf"  # Should be an empty file

shell.prefix("set -euo pipefail;")

rule all:
    input:

thanhleviet

libvirt + prometheus + grafana

KVM を libvirt でホストしていて、prometheus と grafana で観測したい場合。 exporter の実装は 3 種類みつかる。流儀が異なり、メトリクスが異なるので注意。

• libvirt_exporter (Go)
  • grafana json + prometheus recording rule
  • prometheus の設定ファイルで recording rule を有効化します。
  • job 名は libvirt_exporter

thanhleviet

# List of cheatsheet for linux find.
# Taken from here http://alvinalexander.com/unix/edu/examples/find.shtml

# basic 'find file' commands
# --------------------------
find / -name foo.txt -type f -print             # full command
find / -name foo.txt -type f                    # -print isn't necessary
find / -name foo.txt                            # don't have to specify "type==file"
find . -name foo.txt                            # search under the current dir
find . -name "foo.*"                            # wildcard

thanhleviet

SAM and BAM filtering one-liners

@author: David Fredman, [email protected] (sans poly-A tail)
@dependencies: http://sourceforge.net/projects/bamtools/ and http://samtools.sourceforge.net/

Please extend with additional/faster/better solutions via a pull request!

BWA mapping (using piping for minimal disk I/O)

thanhleviet

#reference https://unix.stackexchange.com/questions/136371/how-to-download-a-folder-from-google-drive-using-terminal
#get cookie and code
wget --save-cookies cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/Code: \1\n/p'

#download the file 
wget --load-cookies cookies.txt 'https://docs.google.com/uc?export=download&confirm=CODE_FROM_ABOVE&id=FILEID'

thanhleviet

# SAM to consensus FASTA code golf, inspired by http://lab.loman.net/2015/07/28/calling-haploid-consensus-sequence/

# Starting with a SAM:
samtools view -bS seqs.sam | samtools sort - seqs # Generate and sort BAM
samtools index seqs.bam # Index BAM

# Starting with an indexed BAM:
samtools mpileup -ud 1000 -f seqs_ref.fasta seqs.bam | bcftools call -c | vcfutils.pl vcf2fq | seqtk seq -a - > seqs.consensus.fa # Generate pileup, call variants, convert to fq, convert to fa

# Who can do better? The bar is set low...

thanhleviet

library(dplyr)
library(tidyr)
library(ggplot2)
library(animation)
#Data from https://crudata.uea.ac.uk/cru/data/temperature/
#As well as data read in script
source("read_cru_hemi.R")
temp_dat <- read_cru_hemi("./HadCRUT4-gl.dat")

#remove cover

thanhleviet

#!/bin/python

import os
from flask import Flask, Response, request, abort, render_template_string, send_from_directory
import Image
import StringIO

app = Flask(__name__)

WIDTH = 1000

thanhleviet

#!/usr/bin/python

import sys

'''
Just a simple tool that adds the line number at
the end of each line
'''

with open(sys.argv[1]) as f_in, open(sys.argv[2], 'w') as f_out: