core

Add/remove a GPU to/from your SolveIt instance in one function.

flowchart TD
    A[Provision Modal sandbox]
    B[Setup SSH connection]
    C[Swap IPython kernel]

    A --> B --> C

enable_mermaid()

We first have to provision a sandbox.

provision_sandbox


def provision_sandbox(
    app_name:str, # App name to look up or create
    pips:list, # pip packages to have installed
    apts:list, # apt packages to have installed
    vol_name:str, # Volume name to look up or create
    timeout:int, # Auto-terminate sandbox after this many seconds
    gpu:str, # GPU type (e.g, 'T4', 'A100')
    secrets:dict, # Sandbox secrets
)->tuple: # (sandbox, host, port)

Call self as a function.

get_secrets


def get_secrets(
    
)->dict:

Secrets from solveit_settings.json.

To make things as seamless as possible, I’ll be adding the secrets in this SolveIt instance to the Modal sandbox.

sb, host, port = provision_sandbox(app_name='solveit-sandbox', pips=default_pips, apts=['openssh-server'], vol_name='solveit-sandbox', timeout=600, gpu='T4', secrets=get_secrets()); sb, host, port

INFO - ∞ creating sandbox; this may take 5-10 minutes if you are creating this sandbox for the first time... | 2026-06-17 06:58:11,132
INFO - ✔ sandbox ready | 2026-06-17 06:58:11,651
INFO - ✔ gotten tunnel: r448.modal.host:40691 | 2026-06-17 06:58:25,188

(Sandbox(), 'r448.modal.host', 40691)

With the sandbox provisioned, it’s now time to set up the SSH connection.

setup_ssh


def setup_ssh(
    sb:Sandbox, # Modal Sandbox object
    host:str, # Tunnel hostname
    port:int, # Tunnel port
)->Callable: # SSH connection

Inject public key, start SSH service, and verify connectivity.

ssh = setup_ssh(sb, host, port); ssh

INFO - ✔ public key injected | 2026-06-17 06:58:35,425
INFO - ✔ started ssh service | 2026-06-17 06:58:36,090

System: Linux
Hostname: modal
User: root
Kernel: 4.4.0
Architecture: x86_64
OS Type: GNU/Linux
GPU: Tesla T4

<function solveit_modal.sandbox.mk_ssh.<locals>.<lambda>(*cmd)>

ssh('hostname')

'modal'

The last step is to swap the IPython kernel.

I’d do _all_ = ipyfernel.core.__all__, but nbdev would through an error.

link_remote_kernel


def link_remote_kernel(
    ssh:object, # Paramiko SSH connection
    host:str, # Tunnel hostname
    port:int, # Tunnel port
    sticky:bool=False, # Set `True` to execute code remotely by default
):

Launch remote IPython kernel on sandbox and connect ipyfernel to it.

link_remote_kernel(ssh, host, port)

INFO - ∞ starting kernel | 2026-06-17 06:59:28,756
INFO - ✔ remote kernel ready: /root/.local/share/jupyter/runtime/kernel-ipyf.json | 2026-06-17 06:59:33,555

ipyf_remote_kernel is already a registered kernel
/app/data/.ssh/config file updated.
Successfully created connection file and forwarded ports!

INFO - ✔ connected to remote kernel | 2026-06-17 06:59:35,452
WARNING - ! remote kernel environment has a different set of libraries installed | 2026-06-17 06:59:35,453

Success: connected to remote kernel via r448.modal.host:40691

And we’re done!

#%%remote <-- cell magic (remove # when actually using)
!hostname

modal

stop_remote()

Code cells will now run locally.
Shutting down remote kernel

sb.terminate()

I’ll wrap everything up into a single function so we can attach a GPU in one line.

gpu_on


def gpu_on(
    app_name:str='solveit-modal', # App name to look up or create
    pips:list=['ipykernel', 'fastai', 'transformers', 'diffusers', 'accelerate', 'datasets', 'huggingface_hub', 'peft', 'safetensors', 'sentence-transformers', 'xformers', 'bitsandbytes', 'ninja', 'einops', 'wandb', 'gradio', 'python-fasthtml', 'plotly', 'ipywidgets', 'fsspec', 's3fs', 'gcsfs', 'librosa', 'imageio', 'imageio-ffmpeg'], # pip packages to have installed
    apts:list=['openssh-server'], # apt packages to have installed
    vol_name:str='solveit-volume', # Volume name to look up or create
    timeout:int=1800, # Auto terminate sandbox after this many seconds
    gpu:str='T4', # GPU type (e.g., 'T4', 'A100')
    secrets:dict | None=None, # Sandbox secrets
    sticky:bool=False, # Set `True` to execute code remotely by default
)->tuple: # (sandbox, ssh)

Provision a GPU sandbox, wire up SSH, and hijack cells onto a remote kernel.

sb, _ = gpu_on()

INFO - ∞ creating sandbox; this may take 5-10 minutes if you are creating this sandbox for the first time... | 2026-06-17 07:00:27,183
INFO - ✔ sandbox ready | 2026-06-17 07:00:27,637
INFO - ✔ gotten tunnel: r450.modal.host:44339 | 2026-06-17 07:00:43,028
INFO - ✔ public key injected | 2026-06-17 07:00:43,430
INFO - ✔ started ssh service | 2026-06-17 07:00:44,169
INFO - ∞ starting kernel | 2026-06-17 07:00:46,276

System: Linux
Hostname: modal
User: root
Kernel: 4.4.0
Architecture: x86_64
OS Type: GNU/Linux
GPU: Tesla T4

INFO - ✔ remote kernel ready: /root/.local/share/jupyter/runtime/kernel-ipyf.json | 2026-06-17 07:00:50,969

ipyf_remote_kernel is already a registered kernel
/app/data/.ssh/config file updated.
Successfully created connection file and forwarded ports!

INFO - ✔ connected to remote kernel | 2026-06-17 07:00:53,514
WARNING - ! remote kernel environment has a different set of libraries installed | 2026-06-17 07:00:53,515

Success: connected to remote kernel via r450.modal.host:44339

Wed Jun 17 07:00:57 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Tesla T4                       On  |   00000000:F4:00.0 Off |                    0 |
| N/A   27C    P8             10W /   70W |       0MiB /  15360MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

I’ll similarly create another function so the sandbox can be turned off in one line as well.

gpu_off


def gpu_off(
    sb:Sandbox, # Sandbox object from gpu_on
):

Restore local kernel, disconnect remote, and teardown sandbox. Must be called from a %%local cell.

gpu_off(sb)

INFO - ✔ unlinked from remote kernel | 2026-06-17 07:01:26,152
INFO - ✔ terminated sandbox | 2026-06-17 07:01:26,214

Code cells will now run locally.
Code cells will now run locally.
Shutting down remote kernel