Aayush Garg garg-aayush

Self-Hosted Gemma 4 Chat with Web UI

These are the steps to set up a self-hosted Gemma 4 chat with a web UI that you can use from your phone and laptop, keeping all your data and models private. It is just llama.cpp built-in web UI served over Tailscale.

The setup gives you:

A chat interface accessible from any device on your Tailscale network
Web search via MCP so the model can look things up (important since models have a knowledge cutoff)
Streaming responses, conversation history and the same UI everywhere

My Setup:

Setting Up Your Personal GPU Server: A Step-by-Step Guide

I've been using a GPU workstation with an RTX 4090 for almost a year now, and it's been one of the best decisions I've made. With a personal GPU server, you no longer need to rely on cloud-based GPU instances from services like RunPod or Vast.ai every time you want to run a job or try new models. The best part? No stress about recurring GPU instance costs! :-)

However, I rarely work directly on my workstation. Instead, I prefer the flexibility of accessing the GPU remotely using my MacBook, whether I'm working from different locations within my home, from a co-working space, or a cozy cafe in another part of town.

In this blog, I will walk you through the steps to configure a personal GPU Ubuntu server.

For this guide, I assume you already have a workstation running Ubuntu with a GPU and it is connected to your local network

Steps to manage multiple CUDA environments

Latest Update: May 19th, 2024

This gist contains all the steps required to:

Install multiple CUDA versions (e.g., CUDA 11.8 and CUDA 12.1
Manage multiple CUDA environments on Ubuntu using the utility called environment modules.
Use this approach to avoid CUDA environment conflicts.

Environment Modules is a package that provides for the dynamic modification of a user's environment via modulefiles. You can find more on it at https://modules.readthedocs.io/en/latest/

	▐▛███▜▌ Claude Code v2.1.123
	▝▜█████▛▘ Opus 4.7 · Claude Max
	▘▘ ▝▝ ~/repos/garg-aayush.github.io

	❯ Now, we have a working dashbard for the weather. You also understand how, I
	am pulling the date for live feed that is updated every 15-30 mins and the
	data that is fetched for 24hr/7d/24d feed. We need to search online and
	decide on the data sources we are using. Currently, we are using data sources
	from two different APIs.
	Given our usecase and the number of calls we are making per minute/hr. We

	▐▛███▜▌ Claude Code v2.1.123
	▝▜█████▛▘ Opus 4.7 · Claude Max
	▘▘ ▝▝ ~/repos/garg-aayush.github.io

	✻ Conversation compacted (ctrl+o for history)


	❯ /compact
	⎿ Compacted (ctrl+o to see full summary)
	⎿ Read india-weather.qmd (86 lines)

	▐▛███▜▌ Claude Code v2.1.123
	▝▜█████▛▘ Opus 4.7 · Claude Max
	▘▘ ▝▝ ~/repos/garg-aayush.github.io

	❯ I have this current new tab "India Weather" that shows the AQI, humidity and
	Temperature for the top 8 metropolitan cities in India. Now, I am thinking we
	can improve the overall experience by showing the plots of these parameter
	changes over the last 24 hours. We can have the graph plots below the map
	that shows the these parameter changes over the last 24 hours, user can
	select the city and it shows the changes. We have to make sure that we keep

	▐▛███▜▌ Claude Code v2.1.123
	▝▜█████▛▘ Sonnet 4.6 · Claude Max
	▘▘ ▝▝ ~/repos/garg-aayush.github.io

	❯ Currently, this is how the values look over the 7d and 30d period, I dont
	this that is the best way to view the values over the 7d or 30d period? What
	is the standard way to show these values? [Image #1]
	⎿ [Image #1]

	⏺ The standard approach for 7d/30d weather charts is daily aggregation with

	▐▛███▜▌ Claude Code v2.1.121
	▝▜█████▛▘ Opus 4.7 · Claude Max
	▘▘ ▝▝ ~/repos/garg-aayush.github.io

	❯ /model
	⎿ Set model to Opus 4.7 (1M context) (default)

	❯ /model
	⎿ Set model to Opus 4.7 (1M context) (default) with high effort

	{%- macro format_parameters(properties, required) -%}
	{%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
	{%- set ns = namespace(found_first=false) -%}
	{%- for key, value in properties \| dictsort -%}
	{%- set add_comma = false -%}
	{%- if key not in standard_keys -%}
	{%- if ns.found_first %},{% endif -%}
	{%- set ns.found_first = true -%}
	{{ key }}:{
	{%- if value['description'] -%}

	{%- set image_count = namespace(value=0) %}
	{%- set video_count = namespace(value=0) %}
	{%- macro render_content(content, do_vision_count, is_system_content=false) %}
	{%- if content is string %}
	{{- content }}
	{%- elif content is iterable and content is not mapping %}
	{%- for item in content %}
	{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
	{%- if is_system_content %}
	{{- raise_exception('System message cannot contain images.') }}