Skip to content

Instantly share code, notes, and snippets.

@tzaffi

tzaffi/Justfile

Last active August 2, 2023 19:23
Show Gist options
  • Save tzaffi/0b456532a7d6511ede7fe1bdae241e75 to your computer and use it in GitHub Desktop.
Save tzaffi/0b456532a7d6511ede7fe1bdae241e75 to your computer and use it in GitHub Desktop.
Download ZIP
Conduit Scripts
Raw
__how-to-install.md

Installing just

Taken from github - casey/just

curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to DEST

In my case DEST=~/bin

Raw
_CONDUIT.md 
❯ just nuke
deleting /Users/zeph/networks/niftynetwork

❯ export DATA_NODE=Follower

❯ just pub-create conduit testnet
Ready for public network with node datadir: /Users/zeph/networks/niftynetwork/Follower
mkdir -p $ALGORAND_DATA
algocfg profile set conduit -d $ALGORAND_DATA
cp ${GO_ALGORAND}/installer/genesis/testnet/genesis.json $ALGORAND_DATA

❯ just tree
/Users/zeph/networks/niftynetwork
└── Follower
    ├── config.json
    └── genesis.json

1 directory, 2 files

❯ just start
Algorand node successfully started!

❯ just status
Last committed block: 64
Time since last block: 2.4s
Sync Time: 0.0s
Last consensus protocol: https://github.com/algorand/spec/tree/a26ed78ed8f834e2b9ccb6eb7d3ee9f629a6e622
Next consensus protocol: https://github.com/algorand/spec/tree/a26ed78ed8f834e2b9ccb6eb7d3ee9f629a6e622
Round for next consensus protocol: 65
Next consensus protocol supported: true
Last Catchpoint: 
Consensus upgrade state: Voting
Yes votes: 64
No votes: 0
Votes remaining: 9936
Yes votes required: 9000
Vote window close round: 10001
Genesis ID: testnet-v1.0
Genesis hash: SGO1GKSzyE7IEPItTxCByw9x8FmnrCDexi9/cOUJOiI=
RUNNING

❯ just algod | jq '.["last-round"]'
...
64

❯ just algod /v2/ledger/sync
...
{"round":1}

❯ just algod /v2/deltas/1
... looks good ...

❯ just stop
❯ just start
❯ just stop
❯ just start

❯ just algod /v2/ledger/sync
...
{"round":1}

❯ just algod /v2/deltas/1
... looks good ...

❯ just algod /v2/ledger/sync/1337 POST
...

❯ just algod /v2/ledger/sync
...
{"round":1337}

❯ just algod | jq '.["last-round"]'
...
1400

❯ just algod /v2/deltas/1
...
{"message":"failed retrieving State Delta"}

❯ just algod /v2/deltas/1337
... looks good ...
Raw
_justify.md 
❯ python make_just.py --just-out-file Justfile3 --name blue-whale

❯ grep blue-whale Justfile3
NAME := "blue-whale"
Raw
conduit_yq.sh
#!/usr/bin/env bash
# --- Courtesy of Will Winder --- #
unset ALGORAND_DATA
export ALGORAND_DATA=~/algorand/conduit/foo/algod_data
unset CONDUIT_DATA
export CONDUIT_DATA=~/algorand/conduit/foo/conduit_data
# Prep
if [[ -d $ALGORAND_DATA ]]; then
goal node stop -d "$ALGORAND_DATA"
rm -rf $ALGORAND_DATA
fi
if [[ -d $CONDUIT_DATA ]]; then
rm -rf $CONDUIT_DATA
fi
docker stop some-postgres || true
docker rm some-postgres || true
#########
# algod #
#########
mkdir -p $ALGORAND_DATA
algocfg profile set conduit -d "$ALGORAND_DATA"
cp ~/algorand/go-algorand/installer/genesis/mainnet/genesis.json "$ALGORAND_DATA"/genesis.json
goal node start -d "$ALGORAND_DATA"
######################
# Postgres container #
######################
pg_port=5555
pg_pass=pgpass
pg_user=algorand
pg_db=conduitdb
docker run -d --name some-postgres -p $pg_port:5432 -e POSTGRES_PASSWORD=$pg_pass -e POSTGRES_USER=$pg_user -e POSTGRES_DB=$pg_db postgres
###########
# conduit #
###########
# Initialize and start conduit
mkdir -p $CONDUIT_DATA
conduit init -i algod -e postgresql > "$CONDUIT_DATA"/conduit.yml
# update config with yq
algod_addr=$(cat "$ALGORAND_DATA"/algod.net)
yq -i ".importer.config.netaddr = \"http://$algod_addr\"" conduit_data/conduit.yml
algod_token=$(cat "$ALGORAND_DATA"/algod.token)
yq -i ".importer.config.token = \"$algod_token\"" conduit_data/conduit.yml
algod_admin=$(cat "$ALGORAND_DATA"/algod.admin.token)
yq -i ".importer.config.catchup-config.admin-token = \"$algod_admin\"" conduit_data/conduit.yml
# postgres connection string
yq -i ".exporter.config.connection-string = \"host=localhost port=$pg_port user=$pg_user password=$pg_pass dbname=$pg_db\"" conduit_data/conduit.yml
echo "Run 'conduit -d conduit_data'"
Raw
Justfile
set export
set shell := ["zsh", "-cu"]
NETWORKS := `echo $HOME` + "/networks"
NAME := "conduitnetwork"
CURR_NETWORK := NETWORKS + "/" + NAME
GO_ALGORAND := "/Users/zeph/github/tzaffi/go-algorand"
NODE_TEMPLATE := GO_ALGORAND + "/test/testdata/nettemplates/TwoNodesFollower100Second.json"
PRIVATE_DATA_NODE := "Primary"
DATA_NODE := env_var_or_default("DATA_NODE", PRIVATE_DATA_NODE)
IS_PUBLIC_TRUTHY := env_var_or_default("DATA_NODE", "")
ALGORAND_DATA := CURR_NETWORK + "/" + DATA_NODE
ALGORAND_TOKEN_PATH := ALGORAND_DATA + "/algod.token"
ALGORAND_ADTOKEN_PATH := ALGORAND_DATA + "/algod.admin.token"
ALGORAND_ALGOD_PATH := ALGORAND_DATA + "/algod.net"
ALGORAND_PID_PATH := ALGORAND_DATA + "/algod.pid"
ENDPOINT_PORT := "56765"
CONDUIT := "./conduit"
CONDUIT_DATA := CURR_NETWORK + "/conduit"
CONDUIT_CONFIG := CONDUIT_DATA + "/conduit.yml"
CONDUIT_LOG := CONDUIT_DATA + "/conduit.log"
CONDUIT_DOCKER := "conduit-postgres"
LOG_LEVEL := "INFO" # "TRACE"
# These fake pre-set tokens make it easier to test against a local network
PRESET_ALGOD_TOKEN := "16b29a0a2bbcc535f1e9e40f0c0888013f3789bf2bd34e7907c8fb1ae9d16024"
PRESET_ALGOD_ADMIN := "20064faacad1e590e757ac9492506c2d948633d7c458651b16a3991d26997695"
# Older:
BOXES_TEAL := "boxes.teal"
# --- SUMMARY --- #
# list all available commands
default:
just --list
# echo all variables
@echo:
echo NETWORKS: $NETWORKS
echo NAME: $NAME
echo CURR_NETWORK: $CURR_NETWORK
echo GO_ALGORAND: $GO_ALGORAND
echo NODE_TEMPLATE: $NODE_TEMPLATE
echo PRIVATE_DATA_NODE: $PRIVATE_DATA_NODE
echo DATA_NODE: $DATA_NODE
echo IS_PUBLIC_TRUTHY: $IS_PUBLIC_TRUTHY
echo ALGORAND_DATA: $ALGORAND_DATA
echo ALGORAND_TOKEN_PATH: $ALGORAND_TOKEN_PATH
echo ALGORAND_ALGOD_PATH: $ALGORAND_ALGOD_PATH
echo ALGORAND_PID_PATH: $ALGORAND_PID_PATH
echo ENDPOINT_PORT: $ENDPOINT_PORT
echo CONDUIT: $CONDUIT
echo CONDUIT_DATA: $CONDUIT_DATA
echo CONDUIT_CONFIG: $CONDUIT_CONFIG
echo CONDUIT_LOG: $CONDUIT_LOG
echo CONDUIT_DOCKER: $CONDUIT_DOCKER
echo BOXES_TEAL: $BOXES_TEAL
# --- algod curl --- #
algod ENDPOINT="/v2/status" VERB="GET":
#! /usr/bin/env bash
# set -euxo pipefail
set -euo pipefail
ALGORAND_TOKEN=$(cat ${ALGORAND_TOKEN_PATH})
ALGORAND_ALGOD=$(cat ${ALGORAND_ALGOD_PATH})
ALGORAND_PID=$(cat ${ALGORAND_PID_PATH})
curl -X ${VERB} "http://${ALGORAND_ALGOD}${ENDPOINT}" -H "Authorization: Bearer ${ALGORAND_TOKEN}"
# pass thru goal command but with the $ALGORAND_DATA set
goal *ARGS:
goal {{ARGS}}
# --- GENERATOR SCRIPT COMMANDS --- #
# generate an arbitrary number of app and box scenarios, each with up to BOXES_PER_APP boxes
gen-mult-app-boxes NUM_APPS="10" BOXES_PER_APP="2048":
#!/usr/bin/env python3
import subprocess
from subprocess import CalledProcessError
num_apps = int({{NUM_APPS}})
print(f"{num_apps=}")
for i in range(num_apps):
print("\n", "\n", "\n", f"gen-app-and-box-scenarios #{i+1}" )
subprocess.run(["just", "gen-app-and-box-scenarios", "{{BOXES_PER_APP}}"]).check_returncode()
# create an app and add up to BOXES_PER_APP random boxes to it in a multi-threaded fashion
gen-app-and-box-scenarios BOXES_PER_APP="10":
#!/usr/bin/env python3
from concurrent.futures import ThreadPoolExecutor
import json
import logging
import random
import string
import subprocess
from subprocess import CalledProcessError
import time
CHARS = string.digits + string.ascii_letters
VAL_SIZE = 24
BOXES_PER_APP = int({{BOXES_PER_APP}})
NLS = "\n" * 3
subprocess.run(["just", "app-create_fund"]).check_returncode()
def worker(thread_number):
logging.info(f"HELLO from {thread_number}!")
create_cpe = set_cpe = test_cpe = del_cpe = None
rand_key_size = random.randint(4, 64)
rand_key = "".join(random.choice(CHARS) for _ in range(rand_key_size))
print(f"{NLS}{thread_number}: {rand_key=}")
try:
subprocess.run(["just", "box-create", rand_key]).check_returncode()
except CalledProcessError as cpe:
create_cpe = str(cpe)
rand_val = "".join(random.choice(CHARS) for _ in range(VAL_SIZE))
print(f"{NLS}{thread_number}: {rand_val=}")
try:
subprocess.run(["just", "box-set", rand_key, rand_val]).check_returncode()
except CalledProcessError as cpe:
set_cpe = str(cpe)
print(f"{NLS}{thread_number}: checking {rand_val=}")
try:
subprocess.run(["just", "box-test", rand_key, rand_val]).check_returncode()
except CalledProcessError as cpe:
test_cpe = str(cpe)
delete = random.choice([True, False])
if delete:
print(f"{NLS}{thread_number}: deleting")
try:
subprocess.run(["just", "box-delete", rand_key]).check_returncode()
except CalledProcessError as cpe:
del_cpe = str(cpe)
return {
"thread_number": thread_number,
"key_size": rand_key_size,
"key": rand_key,
"val": rand_val,
"deleted": delete,
"called_process_errors": {
"create_cpe": create_cpe,
"set_cpe": set_cpe,
"test_cpe": test_cpe,
"del_cpe": del_cpe,
},
}
format = "%(asctime)s: %(message)s"
logging.basicConfig(format=format, level=logging.INFO,
datefmt="%H:%M:%S")
results = []
with ThreadPoolExecutor() as executor:
for r in executor.map(worker, range(BOXES_PER_APP)):
results.append(r)
print(json.dumps(results, indent=2))
for result in results:
for err in result["called_process_errors"].values():
if err:
raise err
# --- HIGHER LEVEL --- #
# create and then start (error if already created)
@create_and_start: create start status
sleep 5
just status
# create an app and then fund it
@app-create_fund: app-create last-app-fund
# --- BOX PUT: HIGHER LEVEL --- #
# create box[BOX] for last app with provided key variable BOX
@box-create $BOX:
just app-call-last '\"create\", \"{{BOX}}\"'
# set box[BOX]=VAL for last app with key BOX and val VAL
@box-set $BOX $VAL:
just app-call-last '\"set\", \"{{BOX}}\", \"{{VAL}}\"'
# set box[BOX]=VAL for last app with key BOX and val VAL
@box-test $BOX $VAL:
just app-call-last '\"check\", \"{{BOX}}\", \"{{VAL}}\"'
# delete box[BOX] for last app
@box-delete $BOX:
just app-call-last '\"delete\", \"{{BOX}}\"'
# stop and tear down the node network. WARNING: YOU WILL LOSE ALL YOUR NODE DATA FROM THE FILE SYSTEM.
@stop_and_nuke: stop nuke
# --- PRE-REQUISITES --- #
# calculate an app's address using the python SDK
app-address *ARGS:
#!/usr/bin/env python3
from algosdk import logic
print(logic.get_application_address({{ ARGS }}))
# --- NETWORKS / NODES --- #
# Private vs. Public Networks. Typical workflow:
# 1. Create a directory for your network (CURR_NETWORK = NETWORKS/NAME)
# 2. Populate the node information under CURR_NETWORK. Branch on Public vs. Private.
# In either case the the data directory is ALGORAND_DATA == CURR_NETWORK/DATA_NODE == NETWORKS/NAME/DATA_NODE.
# a. Public: Use algocfg to configure a single node: see `just pub-create`
# b. Private: Use `goal` to configure a network nodes under CURR_NETWORK. See `just create`
# 3. Start the network: `just start`
#
# NOTE: To run a public network commands, you need to supply the env var `DATA_NODE`. EG:
# DATA_NODE=Follower just pub-validate-datadir
# Or, export and run:
# export DATA_NODE=Follower
# just pub-validate-datadir
# create a private network with one node (error if already created)
create:
mkdir -p $NETWORKS
goal network create -n $NAME -r $CURR_NETWORK -t $NODE_TEMPLATE
# print out the current network's data directory tree
@tree:
tree $CURR_NETWORK
# start a the network (error if already running or not created)
@start:
goal node start
# PUBLIC NETWORKS BEGIN
# check that is ready to connect to public network
pub-validate-datadir:
#! /usr/bin/env bash
set -euxo pipefail
[ -z "$IS_PUBLIC_TRUTHY" ] && { echo "Error: DATA_NODE env var required for public network" ; exit 1; }
echo "Ready for public network with node datadir: $ALGORAND_DATA"
# list available profiles for configuring a network
@pub-cfg-list:
algocfg profile list
# show the current network configuration
@pub-cfg-show:
echo "cat ${ALGORAND_DATA}/config.json"
cat ${ALGORAND_DATA}/config.json
# prepare for connecting to public network
pub-prepare: pub-validate-datadir
mkdir -p $ALGORAND_DATA
# configure a Conduit's network using `algocfg`
pub-create NODE_PROFILE="conduit" NETWORK="testnet" ENDPOINT="127.0.0.1:${ENDPOINT_PORT}" PT="1" PAT="1": pub-prepare
algocfg profile set {{NODE_PROFILE}} -d $ALGORAND_DATA
[ -n {{ENDPOINT}} ] && echo "setting ENDPOINT={{ENDPOINT}}" && algocfg set -p EndpointAddress -v {{ENDPOINT}}
[ {{PT}} = "1" ] && echo "setting ALGOD TOKEN=${PRESET_ALGOD_TOKEN}" && echo ${PRESET_ALGOD_TOKEN} > ${ALGORAND_TOKEN_PATH}
[ {{PAT}} = "1" ] && echo "setting ALGOD ADMIN TOKEN=${PRESET_ALGOD_ADMIN}" && echo ${PRESET_ALGOD_ADMIN} > ${ALGORAND_ADTOKEN_PATH}
cp ${GO_ALGORAND}/installer/genesis/{{NETWORK}}/genesis.json $ALGORAND_DATA
# status of network node
@status:
goal node status && echo "RUNNING" || echo "NOT RUNNING"
# stop the running node (error if not running)
@stop:
goal node stop
# remove the node's data from the file system
@nuke:
echo "deleting $CURR_NETWORK"
rm -rf $CURR_NETWORK
echo "killing endpoint on port ${ENDPOINT_PORT}"
pid=$(lsof -ti :${ENDPOINT_PORT}) && [ ! -z "$pid" ] && kill -9 $pid || true
# --- ACCOUNTS --- #
# list all associated accounts
@list:
goal account list
# create a new account without renaming it to a human friendly local alias
@raw-new-account:
goal account new | awk '{print $NF}'
# echo an account's alias
@account-alias $ACCOUNT:
just list | grep {{ACCOUNT}} | awk '{print $2}'
# create a new locally aliased account
@new-account $ALIAS $ACCOUNT=`just raw-new-account`:
goal account rename `just account-alias {{ACCOUNT}}` {{ALIAS}}
# create a new multisig account with threshold 1 using provided accounts (cannot handle aliases)
@raw-msig-account *ACCOUNTS:
goal account multisig new -T 1 {{ACCOUNTS}} | awk '{print $NF}'
# create a new multisig account with given ALIAS and threshold 1 using provided accounts (cannot handle aliases)
@new-msig-account $ALIAS *ACCOUNTS:
goal account rename `just account-alias $(just raw-msig-account {{ACCOUNTS}})` {{ALIAS}}
# funding account's address
@funder:
just list | awk '{print $2}'
# provide information about a given account
@info $ACCOUNT=`just funder`:
goal account info --address {{ACCOUNT}}
# provide an account's balance
@balance $ACCOUNT=`just funder`:
goal account balance --address {{ACCOUNT}}
# funder's most recently created app-id
@last-app-id:
just info | grep ID | tail -n 1 | cut -d "," -f1 | awk '{print $2}'
# the account address of the funders most recently created app-id
@last-app-address:
just app-address `just last-app-id`
# --- ASSETS --- #
# create a dummy asset for the provided FUNDER. Copy pasta from: https://dappradar.com/blog/algorand-dapp-development-2-standard-asset-management
@asset-create $FUNDER=`just funder`:
goal asset create --creator {{FUNDER}} --total 1000000 --unitname bUSD --name "Balgorand USD" --asseturl "https://b-usd.com" --decimals 9
# --- APPLICATIONS --- #
# information about an application of given id
@app-info $APP_ID=`just last-app-id`:
goal app info --app-id {{APP_ID}}
# print out the boxes teal program
@boxes_teal:
cat $BOXES_TEAL
# shortcut for the approval and clear program `goal app create` params
@programs:
echo "--approval-prog $BOXES_TEAL --clear-prog clear.teal"
# shortcut for the storage params of `goal app create`
@app-vars $GBS="0" $GI="0" $LBS="0" $LI="0":
echo "--global-byteslices $GBS --global-ints $GI --local-byteslices $LBS --local-ints $LI"
# shortcut for creating the arguments of an app call
box-app-args *ARGS:
#!/usr/bin/env python3
args = []
box_arg = ""
i = 0
for arg in {{ARGS}}:
try:
int(arg)
arg = f"int:{arg}"
except Exception:
arg = f"str:{arg}"
args.extend(["--app-arg", arg])
if i == 1:
box_arg = f"--box {arg}"
i += 1
if box_arg:
args = [box_arg] + args
print(*args)
# create an app funded by funder account
@app-create $GBS="0" $GI="0" $LBS="0" $LI="0":
echo "goal app create --creator `just funder` `just programs` `just app-vars $GBS $GI $LBS $LI`"
goal app create --creator `just funder` `just programs` `just app-vars $GBS $GI $LBS $LI`
# call the last app from the funder address using ARGS
@app-call-last *ARGS='\"create\", \"mybox\"':
(set -x; goal app call --app-id `just last-app-id` --from `just funder` `just box-app-args {{ARGS}}`)
# --- BOX INFO --- #
# get all the boxes associated a given app-id
app-box-list $APP_ID=`just last-app-id`:
goal app box list --app-id {{APP_ID}} --max 0
# get box information for a given app-id and box name
app-box-info $APP_ID=`just last-app-id` $BOX="str:mybox":
goal app box info --app-id {{APP_ID}} --name {{BOX}}
# --- CLERK --- #
# send from one account to another a given amount
@send $FROM $TO $AMOUNT:
goal clerk send --from {{FROM}} --to {{TO}} --amount {{AMOUNT}}
# fund the most recently created app
@last-app-fund $AMOUNT=`echo 133713371337`:
just send `just funder` `just last-app-address` {{AMOUNT}}
# --- CONDUIT --- #
# start a postgres docker container for Conduit
conduit-pg-up $pg_port="5555" $pg_db="conduitdb" $pg_user="algorand" $pg_pass="pgpass":
# docker run -d --name ${CONDUIT_DOCKER} -e POSTGRES_USER=$pg_user -e POSTGRES_PASSWORD=$pg_pass -p $pg_port:5432 -e POSTGRES_DB=$pg_db postgres
# docker run -d --name {args.pg_container} -e POSTGRES_USER=algorand -e POSTGRES_PASSWORD=algorand -p {args.pg_port}:5432 postgres
docker run -d --name ${CONDUIT_DOCKER} -e POSTGRES_USER=$pg_user -e POSTGRES_PASSWORD=$pg_pass -p $pg_port:5432 postgres
echo "snoozing 3 secs..."
sleep 3
docker exec -t ${CONDUIT_DOCKER} psql -U${pg_user} -c "create database ${pg_db};"
# stop and remove the postgres docker container for Conduit
conduit-pg-clean:
docker stop ${CONDUIT_DOCKER} || true
docker rm ${CONDUIT_DOCKER} || true
# enter the postgres docker container for Conduit
conduit-pg-enter $pg_user="algorand" $pg_db="conduitdb":
docker exec -it ${CONDUIT_DOCKER} psql -U ${pg_user} -d ${pg_db}
# query the Conduit database
conduit-pg-query $sql="select COUNT(*) from block_header;" $pg_user="algorand" $pg_db="conduitdb":
docker exec -t ${CONDUIT_DOCKER} psql -U{{pg_user}} -d {{pg_db}} -c "{{sql}}"
# connect to the postgress database for Conduit without entering the container
@conduit-pg-connect $pg_port="5555" $pg_user="algorand" $pg_pass="pgpass": # $pg_db="conduitdb"
echo "connecting to: postgresql://${pg_user}@localhost:${pg_port}" # /${pg_db}
psql postgresql://${pg_user}:${pg_pass}@localhost:${pg_port} # /${pg_db}
# stand up a Conduit postgres container and initialize the Conduit config
conduit-init $pg_port="5555" $pg_user="algorand" $pg_pass="pgpass" $pg_db="conduitdb": conduit-pg-clean conduit-pg-up
#! /usr/bin/env bash
set -euo pipefail
TOKEN=$(cat ${ALGORAND_TOKEN_PATH})
ADTOKEN=$(cat ${ALGORAND_ADTOKEN_PATH})
ALGOD=$(cat ${ALGORAND_ALGOD_PATH})
echo "pg_port: ${pg_port}"
echo "pg_db: ${pg_db}"
echo "pg_user: ${pg_user}"
echo "pg_pass: ${pg_pass}"
echo "algod.token: ${TOKEN}"
echo "algod.admin.token: ${ADTOKEN}"
echo "algod.net: ${ALGOD}"
echo "CONDUIT: ${CONDUIT}"
echo "CONDUIT_CONFIG: ${CONDUIT_CONFIG}"
rm -rf ${CONDUIT_DATA}
mkdir -p ${CONDUIT_DATA}
${CONDUIT} init -i algod -e postgresql > ${CONDUIT_CONFIG}
yq -i ".log-level = \"${LOG_LEVEL}\"" ${CONDUIT_CONFIG}
yq -i ".log-file = \"${CONDUIT_LOG}\"" ${CONDUIT_CONFIG}
yq -i ".importer.config.netaddr = \"http://${ALGOD}\"" ${CONDUIT_CONFIG}
yq -i ".importer.config.token = \"${TOKEN}\"" ${CONDUIT_CONFIG}
yq -i ".importer.config.catchup-config.admin-token = \"${ADTOKEN}\"" ${CONDUIT_CONFIG}
yq -i ".exporter.config.connection-string = \"host=localhost port=$pg_port user=$pg_user password=$pg_pass dbname=$pg_db\"" ${CONDUIT_CONFIG}
# start the network, run Conduit for a prescribed duration, and report the number of rounds that were processed
conduit-bootstrap-and-go $duration="60" $pg_user="algorand" $pg_db="conduitdb": nuke pub-create start conduit-init
echo "launching Conduit for ${duration} seconds..."
timeout ${duration} ${CONDUIT} -d ${CONDUIT_DATA} || true
tail ${CONDUIT_LOG}
tail -n 100 ${CONDUIT_LOG} | grep -i "Pipeline round:"
just conduit-pg-query "select MAX(round) from block_header;" {{pg_user}} {{pg_db}}
wc -l ${CONDUIT_LOG}
# --- CONSENSUS PARAMS --- #
# list all consensus param declarations
@consensus-params-list:
cat ${GO_ALGORAND}/config/consensus.go | egrep " (bool|byte|int$|uint|map\[|Duration|PaysetCommitType)" | egrep -v "type|=|func|,|//"
# print out the value history of a consensus param
@consensus-param $CP="MaximumMinimumBalance":
cat ${GO_ALGORAND}/config/consensus.go | grep {{CP}} || echo "{{CP}} not found in consensus.go"
# consensus params for program size
consensus-prog-size:
just consensus-param LogicSigMaxSize
just consensus-param MaxAppProgramLen
# consensus param for LogicSigMaxCost
consensus-prog-cost:
just consensus-param MaxCost
just consensus-param MaxAppProgramCost
# consensus params for program pages
@consensus-prog-pages:
just consensus-param MaxExtraAppProgramPages
# consensus params for foreign refs:
consensus-foreign-refs:
just consensus-param MaxAppTxnAccounts
just consensus-param MaxAppTxnForeignApps
just consensus-param MaxAppTxnForeignAssets
just consensus-param MaxAppTotalTxnReferences
just consensus-param MaxAppBoxReferences
# consensus params for local/global storage:
consensus-storage:
just consensus-param MaxAppKeyLen
just consensus-param MaxAppBytesValueLen
just consensus-param MaxAppSumKeyValueLens
just consensus-param MaxLocalSchemaEntries
just consensus-param MaxGlobalSchemaEntries
# consensus params for min-balance calc:
consensus-minbal:
just consensus-param SchemaMinBalancePerEntry
just consensus-param SchemaUintMinBalance
just consensus-param SchemaBytesMinBalance
just consensus-param BoxFlatMinBalance
just consensus-param BoxByteMinBalance
# consensus params for boxes:
consensus-boxes:
just consensus-param MaxAppKeyLen
just consensus-param MaxBoxSize
just consensus-param BoxFlatMinBalance
just consensus-param BoxByteMinBalance
just consensus-param MaxAppBoxReferences
just consensus-param BytesPerBoxReference
# --- MISCELLANEOUS --- #
# print out the network's algod & kmd token and network/process info
client-info:
#! /usr/bin/env bash
set -euo pipefail
ALGORAND_TOKEN=$(cat ${ALGORAND_TOKEN_PATH})
ALGORAND_ADTOKEN=$(cat ${ALGORAND_ADTOKEN_PATH})
ALGORAND_ALGOD=$(cat ${ALGORAND_ALGOD_PATH})
ALGORAND_PID=$(cat ${ALGORAND_PID_PATH})
echo "algod.token: ${ALGORAND_TOKEN}"
echo "algod.admin.token: ${ALGORAND_ADTOKEN}"
echo "algod.net: ${ALGORAND_ALGOD}"
echo "algod.pid: ${ALGORAND_PID}"
if [[ -f "${ALGORAND_DATA}/kmd-v0.5/kmd.token" ]]; then
KMD=$(cat "${ALGORAND_DATA}/kmd-v0.5/kmd.token")
else
KMD=""
fi
echo "kmd.token: ${KMD}"
if [[ -n $KMD ]]; then
echo "kmd.net---->"
cat ${ALGORAND_DATA}/kmd-v0.5/kmd.log | grep 127.0.0.1 | head -n 1 | cut -d '"' -f4
fi
# print out broadcastQueueBulk's channel size ... the default is 100 which is too small for the example
@broadcast-queue-size:
echo "default is 100. What is it actually in network/wsNetwork.go ?"
cat {{GO_ALGORAND}}/network/wsNetwork.go | grep "wn.broadcastQueueBulk = make(chan broadcastRequest" | cut -d "," -f2 | cut -d ")" -f1 | awk '{print $1}'
Raw
Justfile.tmpl
set export
set shell := ["zsh", "-cu"]
NETWORKS := `echo $HOME` + "/networks"
NAME := "conduitnetwork"
CURR_NETWORK := NETWORKS + "/" + NAME
GO_ALGORAND := "/Users/zeph/github/tzaffi/go-algorand"
NODE_TEMPLATE := GO_ALGORAND + "/test/testdata/nettemplates/TwoNodesFollower100Second.json"
PRIVATE_DATA_NODE := "Primary"
DATA_NODE := env_var_or_default("DATA_NODE", PRIVATE_DATA_NODE)
IS_PUBLIC_TRUTHY := env_var_or_default("DATA_NODE", "")
ALGORAND_DATA := CURR_NETWORK + "/" + DATA_NODE
ALGORAND_TOKEN_PATH := ALGORAND_DATA + "/algod.token"
ALGORAND_ADTOKEN_PATH := ALGORAND_DATA + "/algod.admin.token"
ALGORAND_ALGOD_PATH := ALGORAND_DATA + "/algod.net"
ALGORAND_PID_PATH := ALGORAND_DATA + "/algod.pid"
ENDPOINT_PORT := "56765"
CONDUIT := "./conduit"
CONDUIT_DATA := CURR_NETWORK + "/conduit"
CONDUIT_CONFIG := CONDUIT_DATA + "/conduit.yml"
CONDUIT_LOG := CONDUIT_DATA + "/conduit.log"
CONDUIT_DOCKER := "conduit-postgres"
LOG_LEVEL := "INFO" # "INFO" # "TRACE"
# These fake pre-set tokens make it easier to test against a local network
PRESET_ALGOD_TOKEN := "16b29a0a2bbcc535f1e9e40f0c0888013f3789bf2bd34e7907c8fb1ae9d16024"
PRESET_ALGOD_ADMIN := "20064faacad1e590e757ac9492506c2d948633d7c458651b16a3991d26997695"
# Older:
BOXES_TEAL := "boxes.teal"
# --- SUMMARY --- #
# list all available commands
default:
just --list
# echo all variables
@echo:
echo NETWORKS: $NETWORKS
echo NAME: $NAME
echo CURR_NETWORK: $CURR_NETWORK
echo GO_ALGORAND: $GO_ALGORAND
echo NODE_TEMPLATE: $NODE_TEMPLATE
echo PRIVATE_DATA_NODE: $PRIVATE_DATA_NODE
echo DATA_NODE: $DATA_NODE
echo IS_PUBLIC_TRUTHY: $IS_PUBLIC_TRUTHY
echo ALGORAND_DATA: $ALGORAND_DATA
echo ALGORAND_TOKEN_PATH: $ALGORAND_TOKEN_PATH
echo ALGORAND_ALGOD_PATH: $ALGORAND_ALGOD_PATH
echo ALGORAND_PID_PATH: $ALGORAND_PID_PATH
echo ENDPOINT_PORT: $ENDPOINT_PORT
echo CONDUIT: $CONDUIT
echo CONDUIT_DATA: $CONDUIT_DATA
echo CONDUIT_CONFIG: $CONDUIT_CONFIG
echo CONDUIT_LOG: $CONDUIT_LOG
echo CONDUIT_DOCKER: $CONDUIT_DOCKER
echo BOXES_TEAL: $BOXES_TEAL
# --- algod curl --- #
algod ENDPOINT="/v2/status" VERB="GET":
#! /usr/bin/env bash
# set -euxo pipefail
set -euo pipefail
ALGORAND_TOKEN=$(cat ${ALGORAND_TOKEN_PATH})
ALGORAND_ALGOD=$(cat ${ALGORAND_ALGOD_PATH})
ALGORAND_PID=$(cat ${ALGORAND_PID_PATH})
curl -X ${VERB} "http://${ALGORAND_ALGOD}${ENDPOINT}" -H "Authorization: Bearer ${ALGORAND_TOKEN}"
# pass thru goal command but with the $ALGORAND_DATA set
goal *ARGS:
goal {{ARGS}}
# --- GENERATOR SCRIPT COMMANDS --- #
# generate an arbitrary number of app and box scenarios, each with up to BOXES_PER_APP boxes
gen-mult-app-boxes NUM_APPS="10" BOXES_PER_APP="2048":
#!/usr/bin/env python3
import subprocess
from subprocess import CalledProcessError
num_apps = int({{NUM_APPS}})
print(f"{num_apps=}")
for i in range(num_apps):
print("\n", "\n", "\n", f"gen-app-and-box-scenarios #{i+1}" )
subprocess.run(["just", "gen-app-and-box-scenarios", "{{BOXES_PER_APP}}"]).check_returncode()
# create an app and add up to BOXES_PER_APP random boxes to it in a multi-threaded fashion
gen-app-and-box-scenarios BOXES_PER_APP="10":
#!/usr/bin/env python3
from concurrent.futures import ThreadPoolExecutor
import json
import logging
import random
import string
import subprocess
from subprocess import CalledProcessError
import time
CHARS = string.digits + string.ascii_letters
VAL_SIZE = 24
BOXES_PER_APP = int({{BOXES_PER_APP}})
NLS = "\n" * 3
subprocess.run(["just", "app-create_fund"]).check_returncode()
def worker(thread_number):
logging.info(f"HELLO from {thread_number}!")
create_cpe = set_cpe = test_cpe = del_cpe = None
rand_key_size = random.randint(4, 64)
rand_key = "".join(random.choice(CHARS) for _ in range(rand_key_size))
print(f"{NLS}{thread_number}: {rand_key=}")
try:
subprocess.run(["just", "box-create", rand_key]).check_returncode()
except CalledProcessError as cpe:
create_cpe = str(cpe)
rand_val = "".join(random.choice(CHARS) for _ in range(VAL_SIZE))
print(f"{NLS}{thread_number}: {rand_val=}")
try:
subprocess.run(["just", "box-set", rand_key, rand_val]).check_returncode()
except CalledProcessError as cpe:
set_cpe = str(cpe)
print(f"{NLS}{thread_number}: checking {rand_val=}")
try:
subprocess.run(["just", "box-test", rand_key, rand_val]).check_returncode()
except CalledProcessError as cpe:
test_cpe = str(cpe)
delete = random.choice([True, False])
if delete:
print(f"{NLS}{thread_number}: deleting")
try:
subprocess.run(["just", "box-delete", rand_key]).check_returncode()
except CalledProcessError as cpe:
del_cpe = str(cpe)
return {
"thread_number": thread_number,
"key_size": rand_key_size,
"key": rand_key,
"val": rand_val,
"deleted": delete,
"called_process_errors": {
"create_cpe": create_cpe,
"set_cpe": set_cpe,
"test_cpe": test_cpe,
"del_cpe": del_cpe,
},
}
format = "%(asctime)s: %(message)s"
logging.basicConfig(format=format, level=logging.INFO,
datefmt="%H:%M:%S")
results = []
with ThreadPoolExecutor() as executor:
for r in executor.map(worker, range(BOXES_PER_APP)):
results.append(r)
print(json.dumps(results, indent=2))
for result in results:
for err in result["called_process_errors"].values():
if err:
raise err
# --- HIGHER LEVEL --- #
# create and then start (error if already created)
@create_and_start: create start status
sleep 5
just status
# create an app and then fund it
@app-create_fund: app-create last-app-fund
# --- BOX PUT: HIGHER LEVEL --- #
# create box[BOX] for last app with provided key variable BOX
@box-create $BOX:
just app-call-last '\"create\", \"{{BOX}}\"'
# set box[BOX]=VAL for last app with key BOX and val VAL
@box-set $BOX $VAL:
just app-call-last '\"set\", \"{{BOX}}\", \"{{VAL}}\"'
# set box[BOX]=VAL for last app with key BOX and val VAL
@box-test $BOX $VAL:
just app-call-last '\"check\", \"{{BOX}}\", \"{{VAL}}\"'
# delete box[BOX] for last app
@box-delete $BOX:
just app-call-last '\"delete\", \"{{BOX}}\"'
# stop and tear down the node network. WARNING: YOU WILL LOSE ALL YOUR NODE DATA FROM THE FILE SYSTEM.
@stop_and_nuke: stop nuke
# --- PRE-REQUISITES --- #
# calculate an app's address using the python SDK
app-address *ARGS:
#!/usr/bin/env python3
from algosdk import logic
print(logic.get_application_address({{ ARGS }}))
# --- NETWORKS / NODES --- #
# Private vs. Public Networks. Typical workflow:
# 1. Create a directory for your network (CURR_NETWORK = NETWORKS/NAME)
# 2. Populate the node information under CURR_NETWORK. Branch on Public vs. Private.
# In either case the the data directory is ALGORAND_DATA == CURR_NETWORK/DATA_NODE == NETWORKS/NAME/DATA_NODE.
# a. Public: Use algocfg to configure a single node: see `just pub-create`
# b. Private: Use `goal` to configure a network nodes under CURR_NETWORK. See `just create`
# 3. Start the network: `just start`
#
# NOTE: To run a public network commands, you need to supply the env var `DATA_NODE`. EG:
# DATA_NODE=Follower just pub-validate-datadir
# Or, export and run:
# export DATA_NODE=Follower
# just pub-validate-datadir
# create a private network with one node (error if already created)
create:
mkdir -p $NETWORKS
goal network create -n $NAME -r $CURR_NETWORK -t $NODE_TEMPLATE
# print out the current network's data directory tree
@tree:
tree $CURR_NETWORK
# start a the network (error if already running or not created)
@start:
goal node start
# PUBLIC NETWORKS BEGIN
# check that is ready to connect to public network
pub-validate-datadir:
#! /usr/bin/env bash
set -euxo pipefail
[ -z "$IS_PUBLIC_TRUTHY" ] && { echo "Error: DATA_NODE env var required for public network" ; exit 1; }
echo "Ready for public network with node datadir: $ALGORAND_DATA"
# list available profiles for configuring a network
@pub-cfg-list:
algocfg profile list
# show the current network configuration
@pub-cfg-show:
echo "cat ${ALGORAND_DATA}/config.json"
cat ${ALGORAND_DATA}/config.json
# prepare for connecting to public network
pub-prepare: pub-validate-datadir
mkdir -p $ALGORAND_DATA
# configure a Conduit's network using `algocfg`
pub-create NODE_PROFILE="conduit" NETWORK="testnet" ENDPOINT="127.0.0.1:${ENDPOINT_PORT}" PT="1" PAT="1": pub-prepare
algocfg profile set {{NODE_PROFILE}} -d $ALGORAND_DATA
[ -n {{ENDPOINT}} ] && echo "setting ENDPOINT={{ENDPOINT}}" && algocfg set -p EndpointAddress -v {{ENDPOINT}}
[ {{PT}} = "1" ] && echo "setting ALGOD TOKEN=${PRESET_ALGOD_TOKEN}" && echo ${PRESET_ALGOD_TOKEN} > ${ALGORAND_TOKEN_PATH}
[ {{PAT}} = "1" ] && echo "setting ALGOD ADMIN TOKEN=${PRESET_ALGOD_ADMIN}" && echo ${PRESET_ALGOD_ADMIN} > ${ALGORAND_ADTOKEN_PATH}
cp ${GO_ALGORAND}/installer/genesis/{{NETWORK}}/genesis.json $ALGORAND_DATA
# status of network node
@status:
goal node status && echo "RUNNING" || echo "NOT RUNNING"
# stop the running node (error if not running)
@stop:
goal node stop
# remove the node's data from the file system
@nuke:
echo "deleting $CURR_NETWORK"
rm -rf $CURR_NETWORK
echo "killing endpoint on port ${ENDPOINT_PORT}"
pid=$(lsof -ti :${ENDPOINT_PORT}) && [ ! -z "$pid" ] && kill -9 $pid || true
# --- ACCOUNTS --- #
# list all associated accounts
@list:
goal account list
# create a new account without renaming it to a human friendly local alias
@raw-new-account:
goal account new | awk '{print $NF}'
# echo an account's alias
@account-alias $ACCOUNT:
just list | grep {{ACCOUNT}} | awk '{print $2}'
# create a new locally aliased account
@new-account $ALIAS $ACCOUNT=`just raw-new-account`:
goal account rename `just account-alias {{ACCOUNT}}` {{ALIAS}}
# create a new multisig account with threshold 1 using provided accounts (cannot handle aliases)
@raw-msig-account *ACCOUNTS:
goal account multisig new -T 1 {{ACCOUNTS}} | awk '{print $NF}'
# create a new multisig account with given ALIAS and threshold 1 using provided accounts (cannot handle aliases)
@new-msig-account $ALIAS *ACCOUNTS:
goal account rename `just account-alias $(just raw-msig-account {{ACCOUNTS}})` {{ALIAS}}
# funding account's address
@funder:
just list | awk '{print $2}'
# provide information about a given account
@info $ACCOUNT=`just funder`:
goal account info --address {{ACCOUNT}}
# provide an account's balance
@balance $ACCOUNT=`just funder`:
goal account balance --address {{ACCOUNT}}
# funder's most recently created app-id
@last-app-id:
just info | grep ID | tail -n 1 | cut -d "," -f1 | awk '{print $2}'
# the account address of the funders most recently created app-id
@last-app-address:
just app-address `just last-app-id`
# --- ASSETS --- #
# create a dummy asset for the provided FUNDER. Copy pasta from: https://dappradar.com/blog/algorand-dapp-development-2-standard-asset-management
@asset-create $FUNDER=`just funder`:
goal asset create --creator {{FUNDER}} --total 1000000 --unitname bUSD --name "Balgorand USD" --asseturl "https://b-usd.com" --decimals 9
# --- APPLICATIONS --- #
# information about an application of given id
@app-info $APP_ID=`just last-app-id`:
goal app info --app-id {{APP_ID}}
# print out the boxes teal program
@boxes_teal:
cat $BOXES_TEAL
# shortcut for the approval and clear program `goal app create` params
@programs:
echo "--approval-prog $BOXES_TEAL --clear-prog clear.teal"
# shortcut for the storage params of `goal app create`
@app-vars $GBS="0" $GI="0" $LBS="0" $LI="0":
echo "--global-byteslices $GBS --global-ints $GI --local-byteslices $LBS --local-ints $LI"
# shortcut for creating the arguments of an app call
box-app-args *ARGS:
#!/usr/bin/env python3
args = []
box_arg = ""
i = 0
for arg in {{ARGS}}:
try:
int(arg)
arg = f"int:{arg}"
except Exception:
arg = f"str:{arg}"
args.extend(["--app-arg", arg])
if i == 1:
box_arg = f"--box {arg}"
i += 1
if box_arg:
args = [box_arg] + args
print(*args)
# create an app funded by funder account
@app-create $GBS="0" $GI="0" $LBS="0" $LI="0":
echo "goal app create --creator `just funder` `just programs` `just app-vars $GBS $GI $LBS $LI`"
goal app create --creator `just funder` `just programs` `just app-vars $GBS $GI $LBS $LI`
# call the last app from the funder address using ARGS
@app-call-last *ARGS='\"create\", \"mybox\"':
(set -x; goal app call --app-id `just last-app-id` --from `just funder` `just box-app-args {{ARGS}}`)
# --- BOX INFO --- #
# get all the boxes associated a given app-id
app-box-list $APP_ID=`just last-app-id`:
goal app box list --app-id {{APP_ID}} --max 0
# get box information for a given app-id and box name
app-box-info $APP_ID=`just last-app-id` $BOX="str:mybox":
goal app box info --app-id {{APP_ID}} --name {{BOX}}
# --- CLERK --- #
# send from one account to another a given amount
@send $FROM $TO $AMOUNT:
goal clerk send --from {{FROM}} --to {{TO}} --amount {{AMOUNT}}
# fund the most recently created app
@last-app-fund $AMOUNT=`echo 133713371337`:
just send `just funder` `just last-app-address` {{AMOUNT}}
# --- CONDUIT --- #
# start a postgres docker container for Conduit
conduit-pg-up $pg_port="5555" $pg_db="conduitdb" $pg_user="algorand" $pg_pass="pgpass":
# docker run -d --name ${CONDUIT_DOCKER} -e POSTGRES_USER=$pg_user -e POSTGRES_PASSWORD=$pg_pass -p $pg_port:5432 -e POSTGRES_DB=$pg_db postgres
# docker run -d --name {args.pg_container} -e POSTGRES_USER=algorand -e POSTGRES_PASSWORD=algorand -p {args.pg_port}:5432 postgres
docker run -d --name ${CONDUIT_DOCKER} -e POSTGRES_USER=$pg_user -e POSTGRES_PASSWORD=$pg_pass -p $pg_port:5432 postgres
echo "snoozing 3 secs..."
sleep 3
docker exec -t ${CONDUIT_DOCKER} psql -U${pg_user} -c "create database ${pg_db};"
# stop and remove the postgres docker container for Conduit
conduit-pg-clean:
docker stop ${CONDUIT_DOCKER} || true
docker rm ${CONDUIT_DOCKER} || true
# enter the postgres docker container for Conduit
conduit-pg-enter $pg_user="algorand" $pg_db="conduitdb":
docker exec -it ${CONDUIT_DOCKER} psql -U ${pg_user} -d ${pg_db}
# query the Conduit database
conduit-pg-query $sql="select COUNT(*) from block_header;" $pg_user="algorand" $pg_db="conduitdb":
docker exec -t ${CONDUIT_DOCKER} psql -U{{pg_user}} -d {{pg_db}} -c "{{sql}}"
# connect to the postgress database for Conduit without entering the container
@conduit-pg-connect $pg_port="5555" $pg_user="algorand" $pg_pass="pgpass": # $pg_db="conduitdb"
echo "connecting to: postgresql://${pg_user}@localhost:${pg_port}" # /${pg_db}
psql postgresql://${pg_user}:${pg_pass}@localhost:${pg_port} # /${pg_db}
# stand up a Conduit postgres container and initialize the Conduit config
conduit-init $pg_port="5555" $pg_user="algorand" $pg_pass="pgpass" $pg_db="conduitdb": conduit-pg-clean conduit-pg-up
#! /usr/bin/env bash
set -euo pipefail
TOKEN=$(cat ${ALGORAND_TOKEN_PATH})
ADTOKEN=$(cat ${ALGORAND_ADTOKEN_PATH})
ALGOD=$(cat ${ALGORAND_ALGOD_PATH})
echo "pg_port: ${pg_port}"
echo "pg_db: ${pg_db}"
echo "pg_user: ${pg_user}"
echo "pg_pass: ${pg_pass}"
echo "algod.token: ${TOKEN}"
echo "algod.admin.token: ${ADTOKEN}"
echo "algod.net: ${ALGOD}"
echo "CONDUIT: ${CONDUIT}"
echo "CONDUIT_CONFIG: ${CONDUIT_CONFIG}"
rm -rf ${CONDUIT_DATA}
mkdir -p ${CONDUIT_DATA}
${CONDUIT} init -i algod -e postgresql > ${CONDUIT_CONFIG}
yq -i ".log-level = \"${LOG_LEVEL}\"" ${CONDUIT_CONFIG}
yq -i ".log-file = \"${CONDUIT_LOG}\"" ${CONDUIT_CONFIG}
yq -i ".importer.config.netaddr = \"http://${ALGOD}\"" ${CONDUIT_CONFIG}
yq -i ".importer.config.token = \"${TOKEN}\"" ${CONDUIT_CONFIG}
yq -i ".importer.config.catchup-config.admin-token = \"${ADTOKEN}\"" ${CONDUIT_CONFIG}
yq -i ".exporter.config.connection-string = \"host=localhost port=$pg_port user=$pg_user password=$pg_pass dbname=$pg_db\"" ${CONDUIT_CONFIG}
# start the network, run Conduit for a prescribed duration, and report the number of rounds that were processed
conduit-experiment $duration="60" $pg_user="algorand" $pg_db="conduitdb": nuke pub-create start conduit-init
echo "launching Conduit for ${duration} seconds..."
timeout ${duration} ${CONDUIT} -d ${CONDUIT_DATA} || true
tail ${CONDUIT_LOG}
tail -n 100 ${CONDUIT_LOG} | grep -i "FINISHED Pipeline round"
just conduit-pg-query "select MAX(round) from block_header;" {{pg_user}} {{pg_db}}
wc -l ${CONDUIT_LOG}
# --- CONSENSUS PARAMS --- #
# list all consensus param declarations
@consensus-params-list:
cat ${GO_ALGORAND}/config/consensus.go | egrep " (bool|byte|int$|uint|map\[|Duration|PaysetCommitType)" | egrep -v "type|=|func|,|//"
# print out the value history of a consensus param
@consensus-param $CP="MaximumMinimumBalance":
cat ${GO_ALGORAND}/config/consensus.go | grep {{CP}} || echo "{{CP}} not found in consensus.go"
# consensus params for program size
consensus-prog-size:
just consensus-param LogicSigMaxSize
just consensus-param MaxAppProgramLen
# consensus param for LogicSigMaxCost
consensus-prog-cost:
just consensus-param MaxCost
just consensus-param MaxAppProgramCost
# consensus params for program pages
@consensus-prog-pages:
just consensus-param MaxExtraAppProgramPages
# consensus params for foreign refs:
consensus-foreign-refs:
just consensus-param MaxAppTxnAccounts
just consensus-param MaxAppTxnForeignApps
just consensus-param MaxAppTxnForeignAssets
just consensus-param MaxAppTotalTxnReferences
just consensus-param MaxAppBoxReferences
# consensus params for local/global storage:
consensus-storage:
just consensus-param MaxAppKeyLen
just consensus-param MaxAppBytesValueLen
just consensus-param MaxAppSumKeyValueLens
just consensus-param MaxLocalSchemaEntries
just consensus-param MaxGlobalSchemaEntries
# consensus params for min-balance calc:
consensus-minbal:
just consensus-param SchemaMinBalancePerEntry
just consensus-param SchemaUintMinBalance
just consensus-param SchemaBytesMinBalance
just consensus-param BoxFlatMinBalance
just consensus-param BoxByteMinBalance
# consensus params for boxes:
consensus-boxes:
just consensus-param MaxAppKeyLen
just consensus-param MaxBoxSize
just consensus-param BoxFlatMinBalance
just consensus-param BoxByteMinBalance
just consensus-param MaxAppBoxReferences
just consensus-param BytesPerBoxReference
# --- MISCELLANEOUS --- #
# print out the network's algod & kmd token and network/process info
client-info:
#! /usr/bin/env bash
set -euo pipefail
ALGORAND_TOKEN=$(cat ${ALGORAND_TOKEN_PATH})
ALGORAND_ADTOKEN=$(cat ${ALGORAND_ADTOKEN_PATH})
ALGORAND_ALGOD=$(cat ${ALGORAND_ALGOD_PATH})
ALGORAND_PID=$(cat ${ALGORAND_PID_PATH})
echo "algod.token: ${ALGORAND_TOKEN}"
echo "algod.admin.token: ${ALGORAND_ADTOKEN}"
echo "algod.net: ${ALGORAND_ALGOD}"
echo "algod.pid: ${ALGORAND_PID}"
if [[ -f "${ALGORAND_DATA}/kmd-v0.5/kmd.token" ]]; then
KMD=$(cat "${ALGORAND_DATA}/kmd-v0.5/kmd.token")
else
KMD=""
fi
echo "kmd.token: ${KMD}"
if [[ -n $KMD ]]; then
echo "kmd.net---->"
cat ${ALGORAND_DATA}/kmd-v0.5/kmd.log | grep 127.0.0.1 | head -n 1 | cut -d '"' -f4
fi
# print out broadcastQueueBulk's channel size ... the default is 100 which is too small for the example
@broadcast-queue-size:
echo "default is 100. What is it actually in network/wsNetwork.go ?"
cat {{GO_ALGORAND}}/network/wsNetwork.go | grep "wn.broadcastQueueBulk = make(chan broadcastRequest" | cut -d "," -f2 | cut -d ")" -f1 | awk '{print $1}'
Raw
make_just.py
import argparse
from string import Template
# EG:
#
# ❯ python make_just.py --name blue-whale
class DblDollars(Template):
delimiter = '$$' # Use $$ as the delimiter
# Define the substitutions
substitutions = {
"NAME": "niftynetwork",
"NETWORKS": '`echo $HOME` + "/networks"',
"GO_ALGORAND": "/Users/zeph/github/algorand/go-algorand",
"NODE_TEMPLATE": "OneNodeFuture.json",
"PRIVATE_DATA_NODE": "Primary",
}
# Parse command-line arguments
parser = argparse.ArgumentParser()
parser.add_argument("--just-out-file", default="Justfile.tmp")
parser.add_argument("--name")
parser.add_argument("--networks")
parser.add_argument("--go-algorand")
parser.add_argument("--node-template")
parser.add_argument("--private-data-node")
args = parser.parse_args()
templ_args = vars(args)
file_out = templ_args.pop("just_out_file")
# Update the substitutions with the arguments
substitutions.update({
key: templ_args[k] for key in substitutions if templ_args[(k := key.lower())] is not None
})
# Open the template and substitute the placeholders with actual values
with open('Justfile.tmpl', 'r') as f:
template = DblDollars(f.read())
result = template.substitute(substitutions)
# Write the result to the output file
with open(file_out, 'w') as f:
f.write(result)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment