Skip to content

Instantly share code, notes, and snippets.

@timcardenuto

timcardenuto/redhawk-raspberry-pi.md

Last active September 10, 2022 14:56
Show Gist options
  • Save timcardenuto/8b497030795b07cdc681eb8074fd2003 to your computer and use it in GitHub Desktop.
Save timcardenuto/8b497030795b07cdc681eb8074fd2003 to your computer and use it in GitHub Desktop.
Download ZIP
How to build Redhawk for Raspberry Pi, Raspbian Stretch, using QEMU
Raw
redhawk-raspberry-pi.md

Redhawk on Raspberry Pi

One option for compiling Redhawk for the Raspberry Pi is using QEMU on a host to compile it into the image before loading on the rpi hardware - this takes advantage of the superior hardware on your host to decrease compile time. These instructions verified for host system Ubuntu 16.04, Raspberry Pi image Raspbian Stretch 2017-09-07, and Redhawk 2.0.7.

It is important to note that Redhawk is only officially supported on CentOS 6 and 7 and therefore any time you try to compile it for other OS's it's possible (likely on systems like Ubuntu/Debian) that you'll have issues with newer versions of dependencies, the most likely offenders in the past have been:

Library CentOS 6.9 CentOS 7.4 Ubuntu 16.04 Debian 9.2 Raspbian Stretch Fedora 28
gcc 4.4.7 4.8.5 5.4.0 6.3.0 6.3.0 8.1.1
libstdc++ 4.4.7 4.8.5 5.4.0 6.3.0 6.3.0 8.1.1
boost 1.41 1.53 1.58 1.62 1.62 1.66
omniorb 4.1.6 4.2.0 4.1.6 4.1.6 4.1.6 4.2.2
xsd 3.3.0 4.0.0 4.0.0 4.0.0 4.0.0 4.0.0
libxml2 2.7.6 2.9.1 2.9.3 2.9.4 2.9.4 2.9.7
expat 2.0.1 2.1.0 2.1.0 2.2.0 2.2.0 2.2.5

If you run into build errors, and you're sure the environment is set up properly, it's likely a problem where the Redhawk developers built/tested their code with an older version of one of those libraries, and the new version of the library you have has a breaking change. In these cases you can:

  1. Try installing the older/compatible version of the library on your system.
  2. More correctly, make a patch to the Redhawk source code that allows it to build with the current version of the library you have - preferably also without breaking backwards compatibilty.

The biggest issue encountered when building Redhawk for Raspian Stretch has been due to the default usage of C++14 standard in GCC 6.3. Since Redhawk is really only built/tested with GCC 4.x, the default is still C++98. This means C++11, C++14, etc will have some problems with the syntax in Redhawk. I have patched each one of the errors encountered and described below, but these may not be the optimal fixes.

Let's get started! On your host system (Ubuntu 16.04 for me):

# install dependecies
apt-get install qemu qemu-user-static binfmt-support

# download raspbian image
wget https://downloads.raspberrypi.org/raspbian_latest

# extract raspbian image
unzip raspbian_latest

# extend raspbian image by 3GB
dd if=/dev/zero bs=1M count=3072 >> <raspbian_image_name>.img

# set up image as loop device
sudo losetup -f -P --show <raspbian_image_name>.img

You now have to resize the image, which can be a bit tricky. First check the end size of the disk in MB by using parted. The terminal lines for that program look like (parted).

$ sudo parted /dev/loop0
GNU Parted 3.2
Using /dev/loop0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Model: Loopback device (loopback)
Disk /dev/loop0: 4351MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      4194kB  62.9MB  58.7MB  primary  fat16        lba
 2      62.9MB  3277MB  3214MB  primary  ext4

The important number to pull out here is 4351MB from the line Disk /dev/loop0: 4351MB and the Start size of the 2nd Disk Flag which is 62.9MB in this case. The numbers will vary depending on your image and the amount of space you added. With these numbers do the following to resize:

(parted) rm 2                                                             
(parted) mkpart primary 62.9 4351                                         
(parted) print                                                            
Model: Loopback device (loopback)
Disk /dev/loop0: 4351MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      4194kB  62.9MB  58.7MB  primary  fat16        lba
 2      62.9MB  4351MB  4288MB  primary               lba

You should see the End size for the 2nd Disk Flag now matches the total Disk size. Now check and resize:

$ e2fsck -f /dev/loop0p2
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop2: 86233/261632 files (0.1% non-contiguous), 634636/1046784 blocks

$ resize2fs /dev/loop0p2
resize2fs 1.42.12 (29-Aug-2014)
Resizing the filesystem on /dev/loop2 to 1046784 (4k) blocks.
The filesystem on /dev/loop2 is now 1046784 (4k) blocks long.

If you were ready to load the image at this point you could close the loopback device loop0 but we're not done. We need to mount the image and chroot into it to build Redhawk. If you did close the loopback device loop0 or are starting from some other point where you didn't open it yet, run this before continuing sudo losetup -f -P --show <raspbian_image_name>.img.

mkdir ~/rpi_mnt

# mount partition
sudo mount -o rw /dev/loop0p2  ~/rpi_mnt
sudo mount -o rw /dev/loop0p1 ~/rpi_mnt/boot

# mount binds
sudo mount --bind /dev ~/rpi_mnt/dev/
sudo mount --bind /sys ~/rpi_mnt/sys/
sudo mount --bind /proc ~/rpi_mnt/proc/
sudo mount --bind /dev/pts ~/rpi_mnt/dev/pts

# ld.so.preload fix
sudo sed -i 's/^/#/g' ~/rpi_mnt/etc/ld.so.preload

# copy qemu binary
sudo cp /usr/bin/qemu-arm-static ~/rpi_mnt/usr/bin/

# chroot to raspbian
cd ~/rpi_mnt/
sudo chroot . bin/bash

Now you're in the Raspbian image as root and should be able to do stuff! There's one thing required to fix the environment locale settings first:

# fix environment locale problem
nano /etc/locale.gen

# uncomment the following line
en_US.UTF-8 UTF-8

# run generate
locale-gen

# exit chroot
exit

Build Redhawk

Next time you log in, the settings should be fixed. Let's install Redhawk! Note: I do not attempt to install Java so can't confirm that there aren't problems with that.

# chroot to raspbian
sudo chroot ~/rpi_mnt ~/rpi_mnt/bin/bash

# set environment variables
export LANG=C
export LC_ALL=en_US.UTF-8
export OSSIEHOME=/usr/local/redhawk/core
export SDRROOT=/var/redhawk/sdr
export PYTHONPATH=/usr/local/redhawk/core/lib/python
export PATH=$PATH:/usr/local/redhawk/core/bin
export LD_LIBRARY_PATH=/usr/local/redhawk/core/lib/

# update the image
apt update
apt upgrade

# install Redhawk dependencies
apt install uuid-runtime uuid-dev libuuid1 libboost1.62-all-dev libomnievents-dev libomniorb4-dev libomniorb4-1 omnievents omniidl omniidl-python omniorb omniorb-idl omniorb-nameserver python-omniorb xsdcxx autoconf automake autotools-dev libtool libexpat1-dev expat libcppunit-dev python-dev python-numpy python-matplotlib python-lxml pyqt4-dev-tools junit4 liblog4cxx-dev libgstreamer1.0 libcos4-dev

# download latest Redhawk
mkdir /home/pi/redhawk
cd /home/pi/redhawk
wget https://github.com/RedhawkSDR/redhawk/releases/download/2.0.7/redhawk-src-2.0.7-201709281322.tar.gz
tar -zxf redhawk-src-2.0.7-201709281322.tar.gz

Make sure your boost libraries are all version 1.62 (1.61 might work as well) - the 1.58 version caused some linking issues with the Redhawk GPP relate to the use of GCC 6.3. A full upgrade to Boost 1.62 fixed this.

Patches needed to Redhawk 2.0.7

If you're following these instructions on newer versions of Redhawk you should try a build first, since they may have been fixed. Otherwise you may run into some of the following:

  1. There is a problem with running the build from the top level build script b/c it can't find boost libraries. If you go down a few levels you can add a flag and build each submodule individually. Comment out line in configure.ac that checks for old version of XSD. This is not correct since Redhawk will build against XSD 4.x

     cd redhawk-src-2.0.7/src/redhawk
 nano configure.ac
     #AC_MSG_FAILURE([XSD version 3.3.0 required])

  2. Error in control/framework/nodebooter.cpp logging message at line 144:

     nodebooter.cpp:144:59: error: no match for ‘operator<<’ (operand types are ‘std::basic_ostream<char>’ and ‘std::ifstream {aka std::basic_ifstream<char>}’)
  LOG_ERROR(nodebooter, "Failed to parse PRF file " << prfStream<< ". " << parser_error_line << "The XML parser returned the following error: " << ex.what());

    Probable mistake in using prfStream where prfFile is needed.

  3. Error in the Domain Manager code control/sdr/dommgr/applicationSupport.cpp. It produces the following error message:

     applicationSupport.cpp: In member function 'bool ossie::ComponentInfo::isAssignedToDevice() const':
 applicationSupport.cpp:882:12: error: cannot convert 'const boost::shared_ptr<ossie::DeviceNode>' to 'bool' in return
      return assignedDevice;
      ^~~~~~~~~~~~~~~~~~~~~

    This can be fixed by following advice here, to replace the line return assignedDevice; with the line return static_cast<bool> (assignedDevice);. Can't confirm this is the best way to do this but it passes build.

  4. Error in testing/sdr/dev/devices/CppTestDevice/cpp/CppTestDevice.h:

     CppTestDevice.h:31:33: error: 'constexpr' needed for in-class initialization of static data member 'const float CppTestDevice_i::MAX_LOAD' of non-integral type [-fpermissive]
     static const float MAX_LOAD = 4.0;
                                   ^~~

    This can be fixed by following advice here, and replacing the line static const float MAX_LOAD = 4.0; with the line static constexpr float MAX_LOAD = 4.0;. Can't confirm this is the best way to do this but it passes build.

  5. Error in testing/sdr/dom/components/msg_through_cpp/cpp/Makefile.am, missing the $(OMNICOS_LIBS) linker flag.

  6. Error in GPP/cpp/GPP.cpp:

    GPP.cpp:583:24: error: cannot bind 'std::basic_ostream' lvalue to 'std::basic_ostream&&' errstr << "Unable to read "<<stat_filename<<". The process is no longer there"; ~~~~~~~^~~~~~~~~~~~~~~~~~~~

    Change stat_filename to stat_filename.str().

  7. Error in bulkioInterfaces/libsrc/cpp/bulkio_out_port.cpp:

     cpp/bulkio_out_port.cpp:500:94: error: no matching function for call to 'make_pair(BULKIO::StreamSRI&, bool)'
     ret[cSri->first] = std::make_pair< BULKIO::StreamSRI, bool >( cSri->second.sri, false );

    Possible fix to change line 500 to ret[cSri->first] = std::make_pair( cSri->second.sri, false ); which matches other use of make_pair and follows advice to remove explicit template here.

  8. Error in bulkioInterfaces/libsrc/cpp/bulkio_attachable_port:

     cpp/bulkio_attachable_port.cpp:1737:25:   required from ‘void bulkio::OutAttachablePort<StreamDefinition, PT, StreamSequence>::pushSRI(const BULKIO::StreamSRI&, const BULKIO::PrecisionUTCTime&) [with StreamDefinition = BULKIO::VITA49StreamDefinition; PT = BULKIO::dataVITA49; StreamSequence = BULKIO::VITA49StreamSequence]’
 cpp/bulkio_attachable_port.cpp:2142:18:   required from here
 /usr/include/c++/6/bits/stl_tree.h:1889:28: error: no matching function for call to ‘std::_Rb_tree<std::__cxx11::basic_string<char>, std::pair<const std::__cxx11::basic_string<char>, bulkio::AttachableSriMapStruct>, std::_Select1st<std::pair<const std::__cxx11::basic_string<char>, bulkio::AttachableSriMapStruct> >, std::less<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, bulkio::AttachableSriMapStruct> > >::_M_get_insert_unique_pos(std::pair<_CORBA_String_member, bulkio::AttachableSriMapStruct>::first_type&)’
   = _M_get_insert_unique_pos(_KeyOfValue()(__v));

    Seems fixable by changing line 1737 to currentSRIs.insert(std::make_pair(ossie::corba::returnString(H.streamID), sri_ctx));

Warnings for future Redhawk builds

  1. Possible mistake in logging message in control/framework/nodebooter.cpp line 539, references dcdFile instead of dmdFile. Should be:

     LOG_ERROR(nodebooter, "Failed to parse DMD file " << dmdFile << ". " << parser_error_line << "The XML parser returned the following error: " << ex.what());

  2. There are warnings for std::auto_ptr being deprecated, probable fix describe here. These occur often in the libossieparser build.

     ../include/ossie/ossieparser.h:113:18: warning: 'template<class> class std::auto_ptr' is deprecated [-Wdeprecated-declarations]
     std::auto_ptr<T> _p;
          ^~~~~~~~
  In file included from /usr/include/c++/6/memory:81:0,
                   from /usr/include/boost/config/no_tr1/memory.hpp:21,
                   from /usr/include/boost/smart_ptr/shared_ptr.hpp:23,
                   from /usr/include/boost/shared_ptr.hpp:17,
                   from ../include/ossie/SoftPkg.h:29,
                   from SoftPkg.cpp:22:
  /usr/include/c++/6/bits/unique_ptr.h:49:28: note: declared here
     template<typename> class auto_ptr;
                              ^~~~~~~~

  3. When running make install for burstioInterfaces I get the warning libtool: warning: relinking 'libburstio.la'... seems disconcerting.

  4. Many warnings when running ./reconf for the interface modules like this:

     root@debian9:/home/pi/redhawk/redhawk-src-2.0.7/frontendInterfaces# ./reconf
 configure.ac:24: installing './compile'
 configure.ac:23: installing './missing'
 Makefile.am.ide:14: warning: source file 'src/cpp/$(LIBRARY_NAME)/$(IDL_MODULE)/FrontendSK.cpp' is in a subdirectory,
 Makefile.am.ide:14: but option 'subdir-objects' is disabled
 Makefile.am:37:   'Makefile.am.ide' included from here
 automake: warning: possible forward-incompatibility.
 automake: At least a source file is in a subdirectory, but the 'subdir-objects'
 automake: automake option hasn't been enabled.  For now, the corresponding output
 automake: object file(s) will be placed in the top-level directory.  However,
 automake: this behaviour will change in future Automake versions: they will
 automake: unconditionally cause object files to be placed in the same subdirectory
 automake: of the corresponding sources.
 automake: You are advised to start using 'subdir-objects' option throughout your
 automake: project, to avoid future incompatibilities.

Continue Redhawk Build

When you're ready to try to build the redhawk submodule:

# build redhawk module
cd redhawk/src/
./reconf
./configure --disable-java --with-boost-libdir=/usr/lib/arm-linux-gnueabihf
make -j4
make install
cd ../../

# build bulkio library
cd bulkioInterfaces/libsrc/cpp
./reconf
./configure --disable-java --with-boost-libdir=/usr/lib/arm-linux-gnueabihf
make -j4
make install
cd ../../../

# build burstio library
cd burstioInterfaces/
./reconf
./configure --disable-java --with-boost-libdir=/usr/lib/arm-linux-gnueabihf
make -j4
make install
cd ../

# build frontend library
cd frontendnterfaces/
./reconf
./configure --disable-java --with-boost-libdir=/usr/lib/arm-linux-gnueabihf
make -j4
make install
cd ../

# install throughput module
cd throughput/
./reconf
./configure --disable-java --with-boost-libdir=/usr/lib/arm-linux-gnueabihf
make -j4
make install
cd ../

# build GPP
cd GPP/cpp
./reconf
./configure --disable-java --with-boost-libdir=/usr/lib/arm-linux-gnueabihf
make -j4
make install
cd ../../

# install redhawk-codegen
cd redhawk-codegen
python setup.py install
cd ../

# setup omniorb and start the services
cp /etc/omniORB.cfg /etc/omniORB.cfg.orig
echo -e "InitRef = NameService=corbaname::127.0.0.1:2809\nsupportBootstrapAgent = 1" > /etc/omniORB.cfg
mkdir /var/lib/omniEvents
rm /var/lib/omniorb/*
/usr/bin/omniNames -start &
/usr/sbin/omniEvents

# edit the Domain Manager config file to use the armv7l architecture (that's a lower case L not a one)
# you need to remove the references to x86_64 and x86 and add one with arm7l
nano $SDRROOT/dom/mgr/DomainManager.spd.xml
    <processor name="armv7l"/>

# edit the Device Manager config as well to look like the following:
cp $SDRROOT/dev/mgr/DeviceManager.Linux.armv7l.prf.xml $SDRROOT/dev/mgr/DeviceManager.prf.xml
nano $SDRROOT/dev/mgr/DeviceManager.spd.xml
    <!DOCTYPE softpkg PUBLIC "-//JTRS//DTD SCA V2.2.2 SPD//EN" "softpkg.dtd">
    <softpkg name="DeviceManager" id="DCE:82f6515a-de05-47f0-8e7a-1c9f621c00ee">
        <author>
            <name></name>
        </author>
        <propertyfile>
            <localfile name="DeviceManager.prf.xml"/>
        </propertyfile>
        <descriptor>
            <localfile name="DeviceManager.scd.xml"/>
        </descriptor>
        <implementation id="DCE:5fbef908-52fa-4fd8-ad88-2a72a6b8b5d4">
            <description>armv7l Implementation of a Device Manager</description>
            <propertyfile>
                <localfile name="DeviceManager.Linux.prf.xml"/>
            </propertyfile>
            <code type="Executable">
                <localfile name="DeviceManager"/>
                <entrypoint>DeviceManager</entrypoint>
            </code>
            <os name="Linux" />
            <processor name="armv7l"/>
        </implementation>
    </softpkg>

# edit the GPP config as well
nano $SDRROOT/dev/devices/GPP/GPP.spd.xml
    <processor name="armv7l"/>
    
# generate basic node
$SDRROOT/dev/devices/GPP/cpp/gpp_setup

Check/test install

# make sure your environment is setup properly
chmod +x /usr/local/redhawk/core/etc/profile.d/*
/usr/local/redhawk/core/etc/profile.d/redhawk.sh
/usr/local/redhawk/core/etc/profile.d/redhawk-sdrroot.sh

# start Domain Manager
nodeBooter -D &

# start Device Manager
nodeBooter -d $SDRROOT/dev/nodes/DevMgr_debian9/DeviceManager.dcd.xml &

# inspect domain
nameclt list REDHAWK_DEV

# not sure what codegenTesting is for or how to use it. Seems like there are exectutables and stuff missing.

Package stuff as DEB for others

You have to go back and edit the install directory to a folder you'll use for the package, make it look like root, run configure with these paths and re-run make install like this:

mkdir -p /home/pi/redhawk/redhawk-2.0.7-armv7l/var/redhawk/sdr
mkdir -p /home/pi/redhawk/redhawk-2.0.7-armv7l/usr/local/redhawk/core
mkdir -p /home/pi/redhawk/redhawk-2.0.7-armv7l/etc

cp -r $SDRROOT/* /home/pi/redhawk/redhawk-2.0.7-armv7l/var/redhawk/sdr
cp -r $OSSIEHOME/* /home/pi/redhawk/redhawk-2.0.7-armv7l/usr/local/redhawk/core
cp -r $OSSIEHOME/etc/* /home/pi/redhawk/redhawk-2.0.7-armv7l/etc

# TODO post install script to setup environment??

# add DEB descriptor
mkdir redhawk-2.0.7-armv7l/DEBIAN
nano redhawk-2.0.7-armv7l/DEBIAN/control
    Package: redhawk
    Version: 2.0.7
    Section: base
    Priority: optional
    Architecture: armhf
    Depends: libstdc++6 (>= 6.3.0), liblog4cxx10v5 (>= 0.10.0), libaprutil1 (>= 1.5.4), libapr1 (>= 1.5.2), uuid (>= 1.6.2), libboost-serialization1.62.0 (>= 1.62.0), libboost-filesystem1.62.0 (>= 1.62.0), libboost-system1.62.0 (>= 1.62.0), libboost-regex1.62.0 (>= 1.62.0), libboost-thread1.62.0 (>= 1.62.0), omniorb (>= 4.1.6), omniidl (>= 4.1.6), omniidl-python (>= 3.6), omniorb-idl (>= 4.1.6), omniorb-nameserver (>= 4.1.6), python-omniorb (>= 3.6), libomniorb4-1 (>= 4.1.6), libcos4-1 (>= 4.1.6), omnievents (>= 2.6.2), xsdcxx (>= 4.0.0), expat (>= 2.2.0), python-numpy (>= 1.12.1), python-lxml (>= 3.7.1), python-matplotlib (>= 2.0.0)  
    Maintainer: Tim Cardenuto <[email protected]>
    Description: Redhawk SDR
     Redhawk is a Software Defined Radio (SDR) framework for signal processing applications.
     This package is built for Raspberry Pi architecture (armv7l) on Raspbian Stretch.

# build .deb
dpkg-deb --build redhawk-2.0.7-armv7l/

# to install .deb, though you don't need to here if you did build instructions above
dpkg -i redhawk-2.0.7-armv7l.deb
# if you didn't have all the dependencies and it complains run this after to download/install deps
apt --fix-broken install

Cleanup

When you're all done exit the chroot and clean up back in the host:

# exit chroot
exit

# revert ld.so.preload fix
sudo sed -i 's/^#//g' ~/rpi_mnt/etc/ld.so.preload

# unmount everything
sudo umount ~/rpi_mnt/{dev/pts,dev,sys,proc,boot,}

# unmount loop device
sudo losetup -d /dev/loop0

Post Install Setup From DEB

There are a few things you need to do after installing the deb to get things working in a new system

# environment variables (should be done by .deb already but if not....)
chmod +x /usr/local/redhawk/core/etc/profile.d/*
sudo cp /usr/local/redhawk/core/etc/* /etc/
sudo /etc/profile.d/redhawk.sh
sudo /etc/profile.d/redhawk-sdrroot.sh

# change owner/group for $OSSIEHOME and $SDRROOT (probably could do redhawk group but didn't work...)
sudo chown -R <user>:<user> $OSSIEHOME
sudo chown -R <user>:<user> $SDRROOT

# might actually need to edit architecture settings of Domain/Device managers - Raspbian has 2 kernels for older (armv6l) and newer (armv7l) architectures - code works on both but when using QEMU it loaded the armv7l and therefore XML files include that

# might need to edit /etc/omniORB.cfg to only have the following
sudo vi /etc/omniORB.cfg
    InitRef = NameService=corbaname::127.0.0.1:2809
    InitRef = EventService=corbaloc::127.0.0.1:11169/omniEvents
    endPoint = giop:tcp:127.0.0.1:
             = giop:unix:
    supportBootstrapAgent = 1

# might need to kill omni services, clear omniORB logs, and restart
# don't know how to auto-kill services without looking up the PID
sudo rm /var/lib/omniEvents/*
sudo rm /var/lib/omniorb/*
sudo /usr/bin/omniNames -start &
sudo /usr/sbin/omniEvents

# start Domain Manager
nodeBooter -D > /dev/null &
# start Device Manager
nodeBooter -d $SDRROOT/dev/nodes/DevMgr_debian9/DeviceManager.dcd.xml > /dev/null &

# check Domain/Device
nameclt list REDHAWK_DEV

Rip image from SD card

You might want to copy an image off the SD card, once you've used it and want to make backups or send copies to other people. You can do this with dd:

# check for correct device name when you plug in SD card
df -h
    Filesystem           Size  Used Avail Use% Mounted on
    <devicename>
sudo dd if=/dev/<devicename> of=redhawk-2.0.7-raspian-stretch-20171017.img bs=4M status=progress

However you'll notice that the image that comes back will be the size of the entire SD card. To shrink it follow the instructions here

TODO

  • strip binaries for smaller .deb release
  • post install script for .deb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment