Setting Up A Working Galaxy Environment

This gist has migrated to GitLab

https://gitlab.com/mtekman/galaxy-snippets

(The copy here is archived. Please see the above link for the latest developments)

TL;DR: Clone tools-iuc, use Ubuntu 18.04 LTS, and follow the official planemo instructions to get a working Galaxy installation. Otherwise continue below.

Getting Galaxy

In Galaxy there are two types of developers: Galaxy Developers, and Tool Developers.

Galaxy Developers: They speak strange and ancient tongues, casting dark shadows in caves, and are not meant to be understood by the likes of us mere mortals. Their realm is galaxyproject/galaxy, very close to the heart of the beast.
Tool Developers: These brave warriors come from another land, hauling tools and ideas that they naively believe can be applied in this strange new place - but alas, they are wrong, and everything they bring with them is just extra baggage. They dwell in the galaxyproject/tools-iuc realm, where they play with the entrails of the beast like lost feral children.

So anyway, this guide assumes that you are a Tool Developer. All you need to do at this stage is to clone the tools-iuc repository somewhere. I usually go with ~/repos/tools-iuc, and that’s what this guide follows.

Why Ubuntu?

Ubuntu is your best bet when it comes to a good development operating system, since it is widely-used, has a good history of fixing bugs, and overall keeps a pretty stable list of packages that are neither outdated nor bleeding-edge.

Galaxy tool development relies on Conda, a package management system that was initially written for Python tools, but was later expanded to include all kinds of tools written in many different languages. Since OS’s usually come bundled with their own package manager, Conda and the OS package manager usually have some kind of conflict due to the way each handles libraries.

Conda has a good history Ubuntu. It does not have a good history with Arch Linux, and especially not with NixOS, despite both of these being widely used highly customizable OS’s (hint: the customizability is part of the problem).

Which Ubuntu Version?

Ubuntu 20.04 LTS: This is the latest LTS release, but it comes with many problems that wont allow you to install conda or planemo properly.
Ubuntu 18.04 LTS: This is the one before and is a bit more stable. We’ll be using this.

Download Ubuntu 18.04 LTS from here.

Installation of OS

If you want an easy life, use Ubuntu 18.04 as your main bare metal OS.

Otherwise if you prefer to use another OS and still want to develop for Galaxy then I can happily recommend that you stay far far away from Arch or NixOS, the former being mess of bleeding-edge libraries that do not make for a stable development environment, and the latter simply not being compatible with Conda due to a lack of a fixed /lib position.

If, despite these warnings, you want to proceed with another OS and still want to develop a tool in Galaxy, then using a Virtual Machine is the way forward.

I can recommend VirtualBox.

Option 1: Bare Metal Ubuntu

Follow the Ubuntu installation guide. Usually this entails extracting the ISO to a USB via dd, and booting, hitting Enter at each prompt, and then rebooting. If asked to enable openssh, tick yes.

Option 2: VirtualBox method

Initialise

Install virtualbox and the guest addons. This will likely patch your kernel with virtualbox drivers and might take some time.
Create a new image, and set the following:
- RAM to around 2GB
- Disk space to around 10GB (Conda and Galaxy can be expensive)
- If offered to create a dynamically-expanding volume, go for it.
Boot the OS and, when prompted, select the Ubuntu ISO you downloaded
Follow the Bare Metal Ubuntu method above to install

At this point you should be able to boot into Ubuntu with your username of choice. From now on I will assume your username is mewtwo after our lord and savior. I will also assume that you have tools-iuc cloned somewhere in the directory ~/repos as outlined in Getting Galaxy.

Sharing your Data from Host to Guest

Here we will setup a shared directory of ~/repos which we can edit outside the VM and read/write/execute from within the VM.

Method derived from here.

Install pre-requisites from “cdrom”

Log into your VM and do the following

# Use the following command to mount the CD
sudo mkdir /media/cdrom
sudo mount -t iso9660 /dev/cdrom /media/cdrom

# Install dependencies for VirtualBox guest additions
sudo apt-get update
sudo apt-get install -y build-essential linux-headers-`uname -r`

# Run the installation script for the guest additions.
# Wait until the installation completes.
sudo /media/cdrom/./VBoxLinuxAdditions.run

Set mount points

Open up /etc/fstab (with vi or nano) and put the following in:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda2 during curtin installation
/dev/disk/by-uuid/a51f860f-5638-4088-96e1-678edf76fd84 / ext4 defaults 0 0
/swap.img       none    swap    sw      0       0
repos /home/mewtwo/repos vboxsf defaults,uid=1000,gid=1000 0 0

Then do shutdown -P now, to close the VM.

Setup Network and Shared Folder Settings

Under Settings:

Network
- Attached to : Bridged Adapter (to give it internet access which you will need)
Shared Folders
- Add:
  - Folder Path: /path/to/repos
  - Folder Name: “repos”
  - Do not tick Read Only or Auto-mount
  - Mount Point: /home/<your-vm-username>/repos
  - Tick Make Permanent.

Start the VM again. If lucky, it should automount your “repos” definition above to /home/mewtwo/repos in the VM and you should be able to read and write from it.

Setting up SSH

Perform ssh-keygen as a user in the VM, and hit enter at every prompt.
Check your ip address with ip a
Outside of the VM see if you can ssh into it using the username and ip address ssh mewtwo@<ip a output>

Installing Galaxy

The instructions in this section should be applied to the development environment, which is either your native Ubuntu installation or the VM.

Galaxy is installed easily through planemo which handles the Conda installation too, so we will setup planemo first.

Pre-requisites

Planemo

The installation of planemo is reasonably straight-forward, and the instructions are taken from here.

sudo apt install virualenv python-pip

virtualenv .venv
. .venv/bin/activate
pip install "pip>=7" # Upgrade pip if needed.
pip install planemo

Conda

planemo conda_install

If planemo is not found, check that you have activated the virtual environment as in the previous section.

Next, create a file ~/.condarc and place the following in it:

channels:
  - bioconda
  - r
  - conda-forge
  - defaults

Running Galaxy

Git clone tools-iuc somewhere in ~/repos if it does not already exist.

Navigate to the directory and pick an easy tool to test. I will choose allegro.

cd ~/repos/tools-iuc/tools/allegro
  
# Test planemo lints
planemo l allegro.xml

Now let’s download a clean galaxy and test that we can serve tools. We use 0.0.0.0 as a host because otherwise we can’t see it outside of the VM. If you are not using a VM you can omit the –host flag.

planemo s allegro.xml --host 0.0.0.0

This will download the main galaxy repo and place it in ~/.planemo/gx-repo, it might take a few minutes.

Closing Remarks

Voila, you have finished setting up a Galaxy development environment. Here you can edit files in the ~/repos directory and then serve them to your Galaxy environment.

mtekman/setting-up-galaxy.org