Elliott Arnold si3mshady

📝 RL Journal: Day 3 – Stopping My Agent From Getting Lost in Space (Mjollnir Intercept V12)

This Gist contains the latest working script for my personal Reinforcement Learning (RL) project, where I am building an intelligent drone agent, Thor’s Hammer, using the PPO algorithm in a custom Gymnasium environment (MjollnirEnv).

This code represents the V12 iteration of the environment, primarily focused on fixing inefficient training caused by the agent wasting time drifting off-screen.

🎯 V12 Focus: Environment Shaping and Termination Logic

Gymnasium Custom Environment: Annotated GridWorld (gridworld_env.py)

This file provides a fully commented, self-contained implementation of a simple 2D GridWorld environment, built on the standard Gymnasium API (the successor to OpenAI Gym). This environment is designed to serve as a foundational, educational example for anyone learning how to create custom environments for Reinforcement Learning (RL) agents.

Key Features Demonstrated

Environment Initialization (init): Defines the size of the world and sets up rendering.

Observation & Action Spaces: Clearly defines the Dict observation space (agent and target coordinates) and the Discrete(4) action space (Up, Down, Left, Right).

IoT Water-Leak Detection (Audio) — AWS (Kinesis → Lambda → SageMaker → SNS/SQS)

This project deploys a lean, secure, and production-minded pipeline for detecting water leaks (e.g., a continuously running toilet) using short audio snippets analyzed by a pre-deployed SageMaker inference endpoint.

How it works

Edge device (Pi/PC/Phone) records a 1–2s audio snippet when a trigger fires (timer, sound level, manual), base64-encodes it, and sends a JSON payload to Kinesis Data Streams.
Lambda consumer reads Kinesis records, calls your SageMaker endpoint with the audio, and parses the model’s response.
If the model predicts your alert label (default: toilet) with confidence ≥ threshold (default: 0.5), Lambda publishes a JSON alert to SNS.
SQS subscribes to the topic so a Streamlit dashboard (or any worker) can read alerts reliably.

audioprep

audioprep is a simple command-line tool for preparing audio datasets for machine learning. It resamples, pads/trims, normalizes, and augments audio files, then exports WAVs and optional spectrogram / mel features. It also prints step-by-step explanations so you can see what’s happening.

	# Check existing PyTorch
	!python -c "import torch, torchvision; print(f'PyTorch: {torch.__version__}, TorchVision: {torchvision.__version__}')"

	# System packages
	!apt-get update -qq
	!apt-get install -y -qq git wget curl build-essential

	# Clone MapAnything
	!git clone https://github.com/facebookresearch/map-anything.git
	%cd map-anything

	#!/bin/bash

	# === STEP 0: Update System ===
	echo "[*] Updating system..."
	sudo apt-get update && sudo apt-get upgrade -y

	# === STEP 1: Install XFCE Desktop Environment ===
	echo "[*] Installing XFCE4 desktop environment..."
	sudo DEBIAN_FRONTEND=noninteractive apt-get install -y xfce4 xfce4-session

	import os
	import base64
	from dotenv import load_dotenv
	from crewai import Agent, Task, Crew
	from crewai_tools import BaseTool
	from google.auth.transport.requests import Request
	from google.oauth2.credentials import Credentials
	from google_auth_oauthlib.flow import InstalledAppFlow
	from googleapiclient.discovery import build
	from email.mime.text import MIMEText

	apiVersion: v1
	kind: List
	items:
	# Deployment for meal-app
	- apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: meal-app
	spec:
	replicas: 1

	#!/bin/bash

	DOCKER_USERNAME="si3mshady"
	API_KEY="your_openai_api_key_here" # Replace with your actual OpenAI API key

	# Create the project directory structure
	mkdir -p prometheus grafana
	mkdir -p meal_app/public drink_app/public workout_app/public nutrition_app/public

	# Create Prometheus configuration file