Skip to content

Instantly share code, notes, and snippets.

@cpcloud
Created September 19, 2024 16:38
Show Gist options
  • Save cpcloud/e53448fcad4df388798f905fb02fa218 to your computer and use it in GitHub Desktop.
Save cpcloud/e53448fcad4df388798f905fb02fa218 to your computer and use it in GitHub Desktop.
script to build pyarrow pyodide wheel
#!/usr/bin/env bash
set -euo pipefail
export TZ=America/New_York DEBIAN_FRONTEND=noninteractive
apt-get update -y -qq
apt-get install -y build-essential software-properties-common git -qq
apt-get update -y -qq
apt-get autoremove python3 -y -qq
apt-get install -y -qq python3.12 xz-utils python3.12-venv ninja-build jq neovim curl
pushd / || exit
curl -LsSO 'https://www.apache.org/dyn/closer.lua/arrow/arrow-17.0.0/apache-arrow-17.0.0.tar.gz?action=download'
tar xzf apache-arrow-17.0.0.tar.gz
mv apache-arrow-17.0.0 arrow
popd || exit
git clone https://github.com/emscripten-core/emsdk.git /emsdk
python3.12 -m venv /pyodide-pyarrow
source /pyodide-pyarrow/bin/activate
pushd /emsdk || exit
./emsdk install 3.1.58
./emsdk activate 3.1.58
source /emsdk/emsdk_env.sh
popd || exit
pip install pyodide-build==0.26.2
pushd /arrow/cpp || exit
jq \
'(.configurePresets[] | select(.name == "features-emscripten").cacheVariables) += {ARROW_FILESYSTEM: "ON", ARROW_ORC: "OFF", ARROW_SUBSTRAIT: "OFF", ARROW_ACERO: "OFF"}' \
< CMakePresets.json > /tmp/out.json
mv /tmp/out.json CMakePresets.json
emcmake cmake --preset "ninja-release-emscripten"
ninja install
pushd ../python || exit
pyodide build
popd || exit
popd || exit
@cpcloud
Copy link
Author

cpcloud commented Sep 19, 2024

Run using some version of

docker run -it -v /data/pyodide:/scripts:ro -w /scripts ubuntu:24.10

and then

./build.sh once inside the container

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment