Skip to content

Instantly share code, notes, and snippets.

View dutchiechris's full-sized avatar

Chris Madden dutchiechris

9 followers · 0 following
  • Amsterdam, The Netherlands
View GitHub Profile
@dutchiechris
dutchiechris / main.go
Last active May 13, 2025 18:28
GCS multipart uploads using the golang AWS S3 SDK
/*
# Make a HdX disk with 5GiB/s+ perf, or use local SSD, and create test file
sudo mkfs.ext4 /dev/disk/by-id/google-adhoc1
sudo mount /dev/disk/by-id/google-adhoc1 /mnt/tmpfs
dd if=/dev/zero of=/mnt/tmpfs/1g.dat bs=1M count=50000
# Set keys with access to your bucket
export AWS_ACCESS_KEY_ID="<YOUR GCS HMAC ACCESS KEY>"
export AWS_SECRET_ACCESS_KEY="<YOUR GCS HMAC SECRET KEY>"
@dutchiechris
dutchiechris / gcp-speech-api.md
Last active January 23, 2018 11:11
Google Cloud Speech API test

Test steps for using the Google Cloud Platform Speech API on an existing recording

  1. Take an existing media file and ensure it's encoded in a supported format; check here
  • If you have an unsupported encoding you can use vlc to convert it (check the file menu) to be a WAV mono which is supported.

  1. Create a Cloud Storage bucket and put your audio file in there

  2. Go to Speech API longrunningrecognize docs and put a snippet like this where the gs:// stuff is your file in cloud storage:

{
@dutchiechris
dutchiechris / big-query-join-example.sql
Created January 2, 2018 14:42
Simple example of BigQuery (bq) sql join on airline public data set
SELECT
w.airline_name,
COUNT(departure_delay) AS delay
FROM
`bigquery-samples.airline_ontime_data.flights` AS f
JOIN (
SELECT
REGEXP_EXTRACT( airline ,r'.*: (.*)') as airline_code,
REGEXP_EXTRACT( airline ,r'(.*):.*') as airline_name
FROM
@dutchiechris
dutchiechris / git-workflow.md
Last active December 28, 2017 10:31
Common flow using git

Clone the desired repo:

git clone https://github.com/dutchiechris/fio-conf.git

Create a branch for your changes:

git checkout -b feature123
@dutchiechris
dutchiechris / bigquery-copy-between-regions.sh
Created December 20, 2017 15:55
Google Cloud Platform Big Query script to copy data from one region to another
#!/bin/sh
#
# Google Cloud Platform Big Query script to copy data from one region to another
#
# Disclaimer: This is not an officially supported Google product.
# Written code can be used as a baseline but is not meant for production usage.
# Project that owns source and destination datasets
PROJECT=my-project
@dutchiechris
dutchiechris / docker-registry-with-s3-notes.txt
Created September 1, 2017 18:57
docker-registry-with-s3-notes.txt
This doc explains how the registry API works:
https://github.com/docker/distribution/blob/master/docs/spec/api.md
Here’s an example using curl.
Check details of an image that was pushed to the registry:
[root@3-kvm1 ~]# curl -v -X GET 3-docker10.nltestlab.hq.netapp.com:5000/v2/busybox/manifests/latest
* About to connect() to 3-docker10.nltestlab.hq.netapp.com port 5000 (#0)
* Trying 10.64.28.160...
@dutchiechris
dutchiechris / s3cmd_setup_sgws
Created June 20, 2017 12:44
setup s3cmd with netapp storageGRID Webscale (SGWS)
1) Install s3cmd
# pip install s3cmd
2) setup s3 (file on windows is C:\Users\<USER>\AppData\Roaming\s3cmd.ini)
access_key = <KEY_HERE>
secret_key = <SECRET_HERE>
check_ssl_certificate = False
check_ssl_hostname = False
host_base = <host>:8082
host_bucket = <host>:8082/%(bucket)
3) Run commands!
@dutchiechris
dutchiechris / ifcfg-vlan-bridge-example
Last active November 11, 2020 11:12
CentOS 7 ifcfg syntax for VLAN tagged interface + bridge for KVM (referenced from http://www.beginswithdata.com/2016/12/30/centos7-kvm-coreos/)
[root@server02 network-scripts]# cat ifcfg-enp14s0
DEVICE=enp14s0
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
NM_CONTROLLED=no
[root@server02 network-scripts]# cat ifcfg-enp14s0.35
DEVICE=enp14s0.35
TYPE=Ethernet
@dutchiechris
dutchiechris / manage.py
Last active June 18, 2018 02:42
Script to create/start/stop/destroy many CoreOS VMs running on a KVM host (referenced from http://www.beginswithdata.com/2016/12/30/centos7-kvm-coreos/)
#! /usr/bin/python
################################################################################
# Manage (create/start/stop/undefine) CoreOS VMs on CentOS 7 using #
# QEMU/KVM/libvirt. Makes it easy to create/undefine container hosts #
# for lab purposes. #
# #
# For more info see this blog post: #
# http://beginswithdata.com/2016/12/30/centos7-kvm-coreos/ #
# #
# Copyright (c) Chris Madden. All rights reserved. #
@dutchiechris
dutchiechris / user_data
Last active December 21, 2017 03:32
Example CoreOS cloud-config file (referenced from http://www.beginswithdata.com/2016/12/30/centos7-kvm-coreos/)
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEAt4qwYNjFYPnPJObaNC+VoMYBFcEuUzCCBw6YzcU5f27bm0AwKvZKbVbwCo7IJ7gpNP0tIKvDK9z/qRQmcjWlW9p/cqpCa2a1fb0+8O2BhFXaD0BxuZWeuX8SF+mfJ2NzhgpTnR= kvm
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCh1LOHY/929X7qVXqpFQZRJt4uz3rkLVaCHdVD57Y+CEXjEQpoDBBdD6AEkPCvJ+vzFm9LC6KKUFVS7VXxZRUvXvKSWTKUrdsYV9vhNRLFGXYwzyUJov05w+/fxlels/L5Zc== coreos
hostname: @@VMNAME@@
users:
- name: "core"