Priyesh Narayanan pnarayanan

pnarayanan / MsgFmt

Last active October 29, 2017 09:45

Gist of the change within Message Format to support encryption at rest


	To support encryption, Message Format associated with messages now needs to store the encryption key.
	Since we need to encrypt/decrypt both the User Metadata and the Blob payload, we have two choices:
	a) Duplicate the key within both the User Metadata subrecord and Blob subrecord. This is the least
	invasive change and has some merits, but has the burden of duplication of keys.
	b) Create a new Blob Encryption Key record and store the key just once.

	We decided to go with the second approach that avoids duplication and lends itself better to potential
	use cases such as key replacements.

pnarayanan / partition_layout_2partitions.json

Created November 11, 2016 18:42

pnarayanan / hardware_layout_3node.json

Last active November 11, 2016 18:41

	{
	"clusterName": "Perf",
	"datacenters": [
	{
	"dataNodes": [
	{
	"hardwareState": "AVAILABLE",
	"hostname": "lva1-app2038.stg.linkedin.com",
	"port": 15088,
	"sslport": 15288,

pnarayanan / ambry_participant_spectator.java

Last active November 12, 2016 01:12

	// Helix agents will run on datanodes and frontends (routers)

	// DataNode will instantiate a datanode agent at startup:
	class AmbryDataNodeHelixAgent {
	private AmbryClusterParticipant ambryClusterParticipant;
	private AmbryClusterSpectator ambryClusterSpectator;

	AmbryDataNodeHelixAgent(...)
	{
	ambryClusterParticipant = new AmbryClusterParticipant(...);

pnarayanan / helix_bootstrap_upgrade_tool.java

Last active November 12, 2016 01:12

	// This will be run on an admin node.
	//
	// This will be done as part of bootstrapping to map existing clustermap to Helix.
	//
	// On an ongoing basis, this can be used when we expand - to populate new partitions.
	//
	// The script will therefore ignore nodes and partitions that are already added to Helix, and
	// only add those partitions that are new.

	class StaticClusterMapToHelixMapper {

pnarayanan / 1_non_blocking_router_correctness_test_observations

Last active June 6, 2016 23:29

Observations from basic correctness tests done for the non blocking router


	We tested out the new NonBlockingRouter for correctness with small to sufficiently large blobs.
	Blobs were put and got and diffed and verified for content equality.

	Preparation
	===========
	- New MessageFormat was enabled locally and built.
	- frontend-nb Restserver config was modified: s/CoordinatorBackedRouterFactory/NonBlockingRouterFactory

	Setup