How to make pypy work on macOS Big Sur

How to make pypy work on macOS Big Sur

Recently, I ran into a couple of situations where I knew that having a faster python interpreter (say, pypy) could speed up the execution for the python programs drastically.

So, I fired up the terminal and typed in brew install pypy3.

To my surprise, brew took over 10 minutes(!) and then failed to install pypy3 with the following cryptic error:

==> Summary
🍺  /usr/local/akash/Homebrew/Cellar/tcl-tk/8.6.11: 3,041 files, 34.4MB, built in 10 minutes 34 seconds
==> Installing pypy3 dependency: pypy
==> /private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/bootstrap/bin/pypy /private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/rpython/bin/rpython -Oji
Last 15 lines from /Users/avacher/Library/Logs/Homebrew/pypy/01.pypy:
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/rpython/translator/goal/translate.py", line 327, in main
    debug(True)
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/rpython/translator/goal/translate.py", line 280, in debug
    pdb_plus_show.start(tb)
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/rpython/translator/tool/pdbplus.py", line 442, in start
    fn(*args)
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/rpython/translator/tool/pdbplus.py", line 25, in post_mortem
    self.interaction(t.tb_frame, t)
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/bootstrap/lib-python/2.7/pdb.py", line 210, in interaction
    self.cmdloop()
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/bootstrap/lib-python/2.7/cmd.py", line 109, in cmdloop
    self.preloop()
  File "/private/tmp/pypy-20210117-11515-1ie0xew/pypy2.7-v7.3.3-src/rpython/translator/tool/pdbplus.py", line 29, in preloop
    raise NoTTY("Cannot start the debugger when stdout is captured.")
NoTTY: Cannot start the debugger when stdout is captured.

Do not report this issue to Homebrew/brew or Homebrew/core!

Great. That doesn't help me at all.

After a bit of futile Googling, and searching the pypy bug tracker without getting anything directly related to my issue, it suddenly hit me:

I don't necessarily need to install pypy, I just want the ability to run my python code using the pypy interpreter!

Now, if you are puzzled and asking how can I execute pypy without ever installing pypy in the first place, let me tell you a little bit about Docker.

🐳 Docker 101

A simple overview of Docker from the official docs :

Docker provides the ability to package and run an application in a loosely isolated environment called a container.

Containers are lightweight because they don’t need the extra load of a hypervisor, but run directly within the host machine’s kernel.

For our purposes, if we can get a pypy interpreter running in a docker container, it should theoretically work irrespective of the Mac OS version and/or underlying CPU architecture (I'm looking at you Apple M1 ).

In a nutshell - docker can be used to run applications reliably on multiple different operating systems without the need to actually install anything on the underlying operating system!

The only prerequisite to running docker containers is to first install Docker.

Running 🐍 on 🐳

At this point, I have 2 choices: I either build a docker image from scratch and include pypy3 inside it, or I just go and search for an already built readily available Docker image.

Thankfully, I didn't have to go very far as a quick web search later I reached the official Docker image page for pypy which had the exact command that I was looking for 😄

$ docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp pypy:3 pypy3 your-daemon-or-script.py

Note that the first time you run this command, it'll be a bit slow as it will download the necessary Docker image layer files before it can start the docker container. Subsequent runs will not need to repeat this download and will only spend a minuscule amount of time in starting the docker container.

I ran the command above and timed the execution to see if pypy made a difference in the execution time or not:

akash in ~/code/programs
$ time python3 slow.py 
All Done!
python3 slow.py  84.38s user 1.83s system 93% cpu 1:32.07 total

akash in ~/code/programs
$ time docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp pypy:3 pypy3 slow.py 
All Done!
docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp pypy:3 pypy3   0.03s user 0.07s system 0% cpu 25.782 total

Result: 1m 32s execution time down to ~26s

Looks like we achieved what we set out to do.

Additional notes

The command to run the pypy container is long and definitely not easy to remember, so I added an alias for it by appending this line to my .zshrc file:

alias pypy="docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp pypy:3 pypy3"

I just went ahead and got pypy3 as my python programs were written for python3. If for some reason you wished to get pypy for python2 instead, your docker run command would look like this instead:

$ docker run -it --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp pypy:2 pypy your-daemon-or-script.py

Conclusion

Docker is awesome.

The best part about running a command-line application via docker is that the user experience is almost identical to that of a natively installed CLI as the docker container start time (which is spent at every execution) is negligible for most use cases.

Hence the fact that docker is working under the hood becomes transparent.