Install `pycurl` on MacOS Big Sur 11.0.1

pycurl.md

Introduction

Installing pycurl on MacOS with Python 3.6+ and newer proved to be tricky, especially since a lot of the available resources seem to be outdated (1, 2, 3).

This gist was last updated at 2020-12-31.

Steps:

1. Remove previously installed Homebrew versions of curl:

    brew uninstall curl
    brew uninstall openssl
    brew uninstall curl-openssl

2. Install openssl and a curl version that uses openssl rather than SecureTransport, which MacOS curl uses by default. Note that Homebrew no longer allows options while installing, so it's not possible to do brew install curl --with-openssl. This was replaced with a different brew formula, curl-openssl, and again replaced on 2020-01-20 with the current version of just curl.:

    brew install openssl
    brew install curl  # this is the latest version with openssl

3. Now we need to use the new curl when installing pycurl. We can also export the path to the new curl to make it used by default:

    echo 'export PATH="/usr/local/opt/[email protected]/bin:$PATH"' >> ~/.zshrc  # or ~/.bash_profile
    echo 'export PATH="/usr/local/opt/curl/bin:$PATH"' >> ~/.zshrc
    source ~/.zshrc
    source ~/.virtualenvs/foo_venv/bin/activate  # or whichever venv you're using
    export PYCURL_SSL_LIBRARY=openssl
    export LDFLAGS="-L/usr/local/opt/curl/lib"
    export CPPFLAGS="-I/usr/local/opt/curl/include"
    pip install --no-cache-dir --compile --ignore-installed --install-option="--with-openssl" --install-option="--openssl-dir=/usr/local/opt/[email protected]" pycurl

4. Test with:

    python
    >>> import pycurl

There should now be two different versions of curl:

1. The original OS X one in /usr/bin/curl:

    $ /usr/bin/curl --version
curl 7.64.1 (x86_64-apple-darwin20.0) libcurl/7.64.1 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.41.0
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets

2. The Homebrew one in /usr/local/opt/curl/bin/curl:

    $ /usr/local/opt/curl/bin/curl --version
curl 7.74.0 (x86_64-apple-darwin20.2.0) libcurl/7.74.0 (SecureTransport) OpenSSL/1.1.1i zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.42.0 librtmp/2.3
Release-Date: 2020-12-09
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink MultiSSL NTLM NTLM_WB SPNEGO SSL TLS-SRP UnixSockets zstd

And our pycurl should use the second one, both at linker level and at compile level.

None of the above solutions worked for me on Mac 12.1 and I was getting error ImportError: pycurl: libcurl link-time version (7.77.0) is older than compile-time version (7.81.0)

I managed to work around it building the exact curl binary I needed (7.77.0) from the libcurl repo and wiring that into pycurl using install_name_tool. Locations of these will depend on your system but these were my steps. I already had xcode cli tools installed.

Idea is:
install_name_tool -change old.dylib new.dylib pycurl.so

For example:
install_name_tool -change /usr/lib/libcurl.4.dylib /usr/local/opt/curl/lib/libcurl.4.dylib /Users/user/Library/Python/2.7/lib/python/site-packages/pycurl.so

Another example on a different mac with python 3.9 and pyenv I ran this:
install_name_tool -change /usr/local/Cellar/curl/7.81.0/lib/libcurl.4.dylib /opt/vagrant/embedded/lib/libcurl.4.dylib /Users/username/.local/share/virtualenvs/website-TeaFLnO_/lib/python3.9/site-packages/pycurl.cpython-39-darwin.so

1. brew uninstall curl (might not be necessary)
2. Use install_name_tool to link the dynamic library you need. Use find \ -name if to locate the dylib's if needed.
3. Build the exact version of curl you need. Run autoreconf -fi, ./configure --prefix=/usr/local/opt/curl --without-ssl, make
4. pip uninstall pycurl && pip install --no-cache-dir --compile --install-option="--with-openssl" pycurl

Sanity is restored and everything just works now. Hope this helps someone!

Faced the same problem with Mac 11.6.5 and python 3.10

~ > pip install pycurl
~ > python -c 'import pycurl'
ImportError: pycurl: libcurl link-time version (7.64.1) is older than compile-time version (7.77.0)

I didn't manage to change link-time version (7.64.1), so I download exact version of linked curl-7.64.1 and use it for pycurl compilation.

# Remove installed version
pip uninstall -y pycurl
brew uninstall curl

# Install openssl for pycurl compiling
brew install openssl || brew update openssl

# Downloading sources
wget https://curl.haxx.se/download/curl-7.64.1.tar.bz2
tar -xf curl-7.64.1.tar.bz2

# Setting up compiler flags and PATH
export PATH="$(pwd)/.cache/curl-${CURL_VERSION}/bin:$PATH"
export LDFLAGS="-L$(pwd)/curl-7.64.1/lib -L/usr/local/opt/openssl@3/lib"
export CPPFLAGS="-I$(pwd)/curl-7.64.1/include -I/usr/local/opt/openssl@3/include"

# Installing
pip install --no-cache-dir --compile --install-option="--with-openssl" pycurl

# Checking up 
python -c "import pycurl" && echo "Success!"

@eugen1j ,
This results to me in a:
ImportError: pycurl: libcurl link-time ssl backend (none/other) is different from compile-time ssl backend (openssl)

Any suggestion?

@eugen1j , This results to me in a: ImportError: pycurl: libcurl link-time ssl backend (none/other) is different from compile-time ssl backend (openssl)

Any suggestion?

@nffdiogosilva Seems like you have no ssl backend in link-time libcurl. Try to set export PYCURL_SSL_LIBRARY=openssl before running python -c "import pycurl" && echo "Success!"

Also, you can try to compile pycurl without --install-option="--with-openssl" flag

Thanks @eugen1j ! Based on your solution I was able to implement it for Mojave and Python2:

MacBook-Pro:v1.0 zollak$ source .env2.7/bin/activate
(.env2.7) MacBook-Pro:v1.0 zollak$ pip freeze
pycurl==7.43.0.5
(.env2.7) MacBook-Pro:v1.0 zollak$ python -c 'import pycurl'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: pycurl: libcurl link-time version (7.54.0) is older than compile-time version (7.64.1)

I didn't manage to change link-time version (7.54.0), so I download exact version of linked curl-7.54.0 and use it for pycurl compilation.

# Remove installed version
(.env2.7) MacBook-Pro:v1.0 zollak$ pip uninstall -y pycurl

# Downloading sources
(.env2.7) MacBook-Pro:v1.0 zollak$ mkdir curl
(.env2.7) MacBook-Pro:v1.0 zollak$ cd curl/
(.env2.7) MacBook-Pro:curl zollak$ wget https://curl.haxx.se/download/curl-7.54.0.tar.bz2
(.env2.7) MacBook-Pro:curl zollak$ tar -xf curl-7.54.0.tar.bz2

# Setting up compiler flags
(.env2.7) MacBook-Pro:curl zollak$ export LDFLAGS="-L$(pwd)/curl-7.54.0/lib -L/usr/local/opt/[email protected]/lib"
(.env2.7) MacBook-Pro:curl zollak$ export CPPFLAGS="-I$(pwd)/curl-7.54.0/include -I/usr/local/opt/[email protected]/include"

# Installing
(.env2.7) MacBook-Pro:curl zollak$ pip install --no-cache-dir --compile --install-option="--with-openssl" pycurl
Successfully installed pycurl-7.43.0.5

# Checking up 
(.env2.7) MacBook-Pro:curl zollak$ python -c 'import pycurl' && echo "Success"
Success

I fell into this deep rabbit hole too many times, and finally this is probably the ultimate way of fixing this. Thanks @zollak. This works too for

• Python 3
• macOS Monterey 12.6.
• Python installed using Pyenv

--install-option is deprecated in the latest version of pip (e.g. pip 23.1.2).

Use --config-setting instead, e.g.

pip install --no-cache-dir --compile --ignore-installed  --config-setting="--build-option=--with-openssl" pycurl

Edit: Thanks to @jivens for pointing out, removed extra hyphen

The above command is absolutely wonderful except that it contains an extra hyphen, so use this:

pip install --no-cache-dir --compile --ignore-installed --config-setting="--build-option=--with-openssl" pycurl

Below command worked form me
MacOS: Apple M1 Pro (Ventura v13.4)

    pip uninstall pycurl
    PYCURL_SSL_LIBRARY=openssl LDFLAGS="-L/opt/homebrew/opt/openssl/lib" CPPFLAGS="-I/opt/homebrew/opt/openssl/include" pip install --no-cache-dir pycurl

Thanks @keshari-nandan , perfect job for me

@keshari-nandan Thanks