Using Enroot on AMD Accelerator Cloud

This page explains how to import Docker images into Enroot, manage containers, and integrate Enroot with Slurm through the Pyxis plugin. It is intended for researchers, developers, and system administrators working with AAC compute nodes.

Importing Docker Images into Enroot

Public Images

To import a public docker image into enroot:

enroot import docker://<full_image_path>
# Example:
enroot import docker://rocm/pytorch-training:v25.5

Private Images (Requires Authentication)

Interactive Entry (Manual Password)

Run the command with your username; Enroot will prompt for the password/token.

enroot import docker://<username>@<private_image_path>

Stored Credentials (Automated)

Store your login details in the credentials file to avoid manual input. Location: $HOME/.config/enroot/.credentials

machine auth.docker.io login <docker_username> password <docker_password_or_pat>
machine index.docker.io login <docker_username>

Basic Container Lifecycle

Create container from image

enroot create <image_name.sqsh>

List containers

enroot list -f

Start container interactively

enroot start <container_name>
Tip: Keep .sqsh images in a shared read-only directory for compute nodes.

Remove container

enroot remove <container_name>

Using Enroot with Slurm (Pyxis)

# Example srun command to run a containerized job:
srun --container-image=/shareddata/amd_int/aac/venkatesh/megatron-lm+v25.5_py310.sqsh \
--container-writable \
--container-workdir="/workspace/Megatron-LM" \
--container-env="TOKENIZER_MODEL,MOCK_DATA,HF_TOKEN" \
bash -c "TEE_OUTPUT=1 MBS=2 BS=128 TP=1 TE_FP8=1 SEQ_LENGTH=8192 MODEL_SIZE=8 /workspace/Megatron-LM/examples/llama/train_llama3.sh"
Note: `srun --help | grep container` will list all the available options which can be used when using enroot with slurm