sandbox

Abstractions for creating a Modal Sandbox, and interacting with it via SSH.

flowchart TD
    A[Create the app] --> B[Define the image]
    B --> C[Launch the sandbox]
    C --> D[Set up SSH]

Modal is a library that allows you to straightfowardly use cloud GPU compute programmatically in Python. No extra scripts, YAML files or whatnot needed.

Modal allows you to run your code on a GPU in multiple ways.

Use a decorator like @app.function(gpu=...) for stateless tasks. When you run a function wrapped like this, Modal spins up a container, runs your function, and then recycles it. Good for one-off compute like inference or batch processing.
Use @app.cls(gpu=...) for stateful classes. Akin to loading a model once at startup and repeatedly calling its methods. Containers stay alive between calls, and hooks like @modal.enter() let you set things up when the container starts.
Or, run a persistent interactive shell via modal.Sandbox. This gives you a full Linux environment that runs continuously. You can SSH in, install packages, and run code interactively.

The third option is what solveit-modal will be using.

mk_app


def mk_app(
    name:str, # App name to look up or create
)->App:

Look up a Modal App by name, creating it if missing.

An app is like a folder for our projects, or sandboxes in this case. When we create a sandbox, we’ll register it under this app. An app can contain multiple sandboxes.

app = mk_app('solveit-sandbox'); app

<modal.app.App>

mk_img


def mk_img(
    pips:list, # pip packages to have installed
    apts:list, # apt packages to have installed
)->Image:

Create a Modal Image with system + Python packages.

modal.Image.debian_slim?

def blocking_debian_slim(
    python_version:Optional=None, force_build:bool=False
)->_Image:

Default image, based on the official `python` Docker images.

File: ~/.local/lib/python3.12/site-packages/modal/image.py

Type: function

The image describes our Linux environment setup. By default, a build of Debian is used, and we can choose which apt packages, pip packages, or other packages we can have pre-installed.

img = mk_img(pips=['amogus'], apts=['openssh-server']); img

Image(<function _Image.pip_install.<locals>.build_dockerfile>)

mk_vol


def mk_vol(
    name:str, # Volume name to look up or create
)->dict:

Look up a Modal Volume by name, creating it if missing.

vol = mk_vol('solveit-volume'); vol

{'/data': modal.Volume.from_name('solveit-volume')}

We can also create a volume in which we can persistently store data every time the sandbox is loaded.

log.info('haha')

INFO - haha | 2026-06-17 06:18:55,225

mk_sandbox


def mk_sandbox(
    app:App, # Modal App to register the Sandbox with
    img:Image, # Image for the Sandbox environment
    vol:dict, # Volume for the Sandbox environment
    timeout:int, # auto-terminate after this many seconds
    gpu:str, # GPU type (e.g. "T4", "A100") or None
    secrets:dict, # Sandbox secrets
)->Sandbox:

Create a Modal Sandbox with optional GPU, SSH, and Volumes.

sleep infinity keeps the sandbox alive until it times out or is stopped.
Port 22 is exposed as a raw TCP tunnel so we can SSH into it.

It’s now time to construct our sandbox with our defined app, image, and volume on Modal.

sb = mk_sandbox(app, img, vol, timeout=600, gpu='T4', secrets={}); sb

INFO - ∞ creating sandbox; this may take 5-10 minutes if you are creating this sandbox for the first time... | 2026-06-17 06:19:52,242
INFO - ✔ sandbox ready | 2026-06-17 06:19:58,555

Sandbox()

Our sandbox is now running. To connect, we need to fetch the tunnel opened on port 22.

get_tunnel


def get_tunnel(
    sb:Sandbox, # Sandbox with an exposed TCP tunnel
)->tuple:

Get unencrypted host and port for a Sandbox’s TCP tunnel.

host,port = get_tunnel(sb); host,port

INFO - ✔ gotten tunnel: r438.modal.host:42091 | 2026-06-17 06:20:22,784

('r438.modal.host', 42091)

get_pubkey


def get_pubkey(
    
)->str:

SSH public key from environment.

We fetch SolveIt’s public key from our environment and inject it into the sandbox’s authorized keys to grant SSH access.

get_pubkey()

'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID5fw9cXzFDJDm8mCuWUOesThxZZfdlPEsKmMSYrDmmH solveit@8321abd9ea3b'

inject_pubkey


def inject_pubkey(
    sb:Sandbox, # Sandbox to inject the key into
    pubkey:str, # SSH public key string
):

Inject an SSH public key into the Sandbox’s authorized_keys.

inject_pubkey(sb, get_pubkey())

INFO - ✔ public key injected | 2026-06-17 06:21:00,117

We can now start our SSH service.

start_ssh


def start_ssh(
    sb:Sandbox, # Sandbox with SSH server installed
):

Start SSH service in the Sandbox, waiting up to ~6 seconds for it to come online.

start_ssh(sb)

INFO - ✔ started ssh service | 2026-06-17 06:21:29,500

?run

def run(
    cmd, rest:VAR_POSITIONAL, same_in_win:bool=False, ignore_ex:bool=False, as_bytes:bool=False, stderr:bool=True
):

Pass `cmd` (splitting with `shlex` if string) to `subprocess.run`; return `stdout`; raise `IOError` if fails

File: /usr/local/lib/python3.12/site-packages/fastcore/xtras.py

Type: function

Crafting a function to allow us to easily run bash commands in the sandbox.

mk_ssh


def mk_ssh(
    host:str, # Tunnel host
    port:int, # Tunnel port
)->Callable:

Return an SSH function for the given Modal tunnel.

ssh = mk_ssh(host,port); ssh

<function __main__.mk_ssh.<locals>.<lambda>(*cmd)>

verify_ssh


def verify_ssh(
    ssh:Callable, # SSH function returned by mk_ssh
):

Verify SSH connection to Sandbox and print system info.

verify_ssh(ssh)

System: Linux
Hostname: modal
User: root
Kernel: 4.4.0
Architecture: x86_64
OS Type: GNU/Linux
GPU: Tesla T4

And SolveIt has a connection to Modal! But we’re not quite done yet. What’s left is spinning up an IPython kernel on Modal, and then swapping out SolveIt’s IPython kernel with the one started on Modal. See the ipyfernel module for that.

sb.terminate()

Additional helper functions

Additional functions to help you customize your Modal sandbox instance.

get_apts


def get_apts(
    
)->list:

List installed system package names.

get_apts()[:10]

['adduser',
 'alsa-topology-conf',
 'alsa-ucm-conf',
 'apt',
 'aria2',
 'autoconf',
 'automake',
 'autotools-dev',
 'base-files',
 'base-passwd']

get_pips


def get_pips(
    
)->list:

List installed Python packages (name==version).

get_pips()[:10]

['accelerate==1.13.0',
 'anki==25.9.2',
 'beartype==0.22.5',
 'blis==1.3.0',
 'catalogue==2.0.10',
 'cbor2==5.8.0',
 'cloudpathlib==0.23.0',
 'cloudpickle==3.1.2',
 'confection==0.1.5',
 'coolname==2.2.0']

in_modal


def in_modal(
    
)->str:

Checks whether current environment is Modal or not.

in_modal()

False

in_solveit


def in_solveit(
    
):

Checks whether current environment is SolveIt or not.

in_solveit()

True