Install SkyPilot using pip:
$ # SkyPilot requires python >= 3.7. For Apple Silicon, use >= 3.8. $ # Recommended: use a new conda env to avoid package conflicts. $ conda create -y -n sky python=3.8 $ conda activate sky $ # Choose an extra (default: [aws]) $ pip install skypilot $ # pip install "skypilot[gcp]" $ # pip install "skypilot[azure]" $ # pip install "skypilot[lambda]" $ # pip install "skypilot[ibm]" $ # pip install "skypilot[scp]" $ # pip install "skypilot[oci]" $ # pip install "skypilot[all]"
SkyPilot currently supports these cloud providers: AWS, GCP, Azure, IBM, OCI, SCP, Lambda Cloud, and Cloudflare (for R2 object store).
If you only have access to certain clouds, use any combination of
the pip extras above (e.g.,
"[aws,gcp]") to reduce the
You may also install SkyPilot from source.
$ git clone https://github.com/skypilot-org/skypilot.git $ cd skypilot $ pip install ".[all]"
For Macs, macOS >= 10.15 is required to install SkyPilot. Apple Silicon-based devices (e.g. Apple M1) must run
pip uninstall grpcio; conda install -c conda-forge grpcio=1.43.0 prior to installing SkyPilot.
As an alternative to installing SkyPilot on your laptop, we also provide a Docker image as a quick way to try out SkyPilot. See instructions below on running SkyPilot in a container.
Cloud account setup¶
If you already have cloud access set up on your local machine, run
sky check to verify that SkyPilot can properly access your enabled clouds.
Otherwise, configure access to at least one cloud, then run
Amazon Web Services (AWS)¶
To get the AWS access key required by
aws configure, please go to the AWS IAM Management Console and click on the “Access keys” dropdown (detailed instructions here). The Default region name [None]: and Default output format [None]: fields are optional and can be left blank to choose defaults.
$ # Install boto $ pip install boto3 $ # Configure your AWS credentials $ aws configure
To use AWS IAM Identity Center (AWS SSO), see here for instructions.
Google Cloud Platform (GCP)¶
$ pip install google-api-python-client $ conda install -c conda-forge google-cloud-sdk $ gcloud init $ # Run this if you don't have a credentials file. $ # This will generate ~/.config/gcloud/application_default_credentials.json. $ gcloud auth application-default login
conda install -c conda-forge google-cloud-sdk produces the error “RemoveError: ‘requests’ is a dependency of conda and cannot be removed from conda’s operating environment”, try
conda update --force conda first and rerun the command.
Note: if you encounter Authorization Error (Error 400: invalid_request) with the url generated by
gcloud auth login, try installing the latest version of the Google Cloud SDK (e.g., with
conda install -c conda-forge google-cloud-sdk) on your local machine (which opened the browser) and rerun the command.
If you are using multiple GCP projects, list all the configs by
gcloud config configurations list and activate one by
gcloud config configurations activate <CONFIG_NAME> (See GCP docs).
$ # Login $ az login $ # Set the subscription to use $ az account set -s <subscription_id>
az account subscription list to get a list of subscription IDs under your account.
Lambda Cloud is a cloud provider offering low-cost GPUs. To configure Lambda Cloud access, go to the API Keys page on your Lambda console to generate a key and then add it to
$ mkdir -p ~/.lambda_cloud $ echo "api_key = <your_api_key_here>" > ~/.lambda_cloud/lambda_keys
To access IBM’s services, store the following fields in
iam_api_key: <user_personal_api_key> resource_group_id: <resource_group_user_is_a_member_of>
Create a new API key by following this guide.
Obtain a resource group’s ID from the web console.
Stock images aren’t currently providing ML tools out of the box. Create private images with the necessary tools (e.g. CUDA), by following the IBM segment in this documentation.
Oracle Cloud Infrastructure (OCI)¶
To access Oracle Cloud Infrastructure (OCI), setup the credentials by following this guide. After completing the steps in the guide, the
~/.oci folder should contain the following files:
~/.oci/config file should contain the following fields:
[DEFAULT] user=ocid1.user.oc1..aaaaaaaa fingerprint=aa:bb:cc:dd:ee:ff:gg:hh:ii:jj:kk:ll:mm:nn:oo:pp tenancy=ocid1.tenancy.oc1..aaaaaaaa region=us-sanjose-1 key_file=~/.oci/oci_api_key.pem
Cloudflare offers R2, an S3-compatible object storage without any egress charges. SkyPilot can download/upload data to R2 buckets and mount them as local filesystem on clusters launched by SkyPilot. To set up R2 support, run:
$ # Install boto $ pip install boto3 $ # Configure your R2 credentials $ AWS_SHARED_CREDENTIALS_FILE=~/.cloudflare/r2.credentials aws configure --profile r2
In the prompt, enter your R2 Access Key ID and Secret Access Key (see instructions to generate R2 credentials). Select
auto for the default region and
json for the default output format.
AWS Access Key ID [None]: <access_key_id> AWS Secret Access Key [None]: <access_key_secret> Default region name [None]: auto Default output format [None]: json
Next, get your Account ID from your R2 dashboard and store it in
$ mkdir -p ~/.cloudflare $ echo <YOUR_ACCOUNT_ID_HERE> > ~/.cloudflare/accountid
Samsung Cloud Platform (SCP)¶
Samsung Cloud Platform(SCP) provides cloud services optimized for enterprise customers. You can learn more about SCP here.
To configure SCP access, you need access keys and the ID of the project your tasks will run. Go to the Access Key Management page on your SCP console to generate the access keys, and the Project Overview page for the project ID. Then, add them to
~/.scp/scp_credential by running:
$ # Create directory if required $ mkdir -p ~/.scp $ # Add the lines for "access_key", "secret_key", and "project_id" to scp_credential file $ echo "access_key = <your_access_key>" >> ~/.scp/scp_credential $ echo "secret_key = <your_secret_key>" >> ~/.scp/scp_credential $ echo "project_id = <your_project_id>" >> ~/.scp/scp_credential
Multi-node clusters are currently not supported on SCP.
Verifying cloud access¶
After configuring the desired clouds, you can optionally run
sky check to verify that credentials are correctly set up:
$ sky check
This will produce a summary like:
Checking credentials to enable clouds for SkyPilot. AWS: enabled GCP: enabled Azure: enabled Lambda: enabled IBM: enabled SCP: enabled OCI: enabled Cloudflare (for R2 object store): enabled SkyPilot will use only the enabled clouds to run tasks. To change this, configure cloud credentials, and run sky check.
Requesting quotas for first time users¶
If your cloud account has not been used to launch instances before, the respective quotas are likely set to zero or a low limit. This is especially true for GPU instances.
Please follow Requesting Quota Increase to check quotas and request quota increases before proceeding.
Quick alternative: trying in Docker¶
As a quick alternative to installing SkyPilot on your laptop, we also provide a Docker image with SkyPilot and its dependencies installed for users to quickly try out SkyPilot. You can simply run:
$ docker run -td --name sky --rm -v "$HOME/.sky:/root/.sky:rw" -v "$HOME/.aws:/root/.aws:rw" -v "$HOME/.config/gcloud:/root/.config/gcloud:rw" -v "$HOME/.azure:/root/.azure:rw" public.ecr.aws/a9w6z7w5/sky:latest $ docker exec -it sky /bin/bash
If your cloud CLIs are already setup, your credentials will be mounted to the container and you can proceed to Quickstart. Else you can follow the instructions in Cloud account setup inside the container to setup your cloud accounts.
If you exit from the shell in the container, the container will keep running
in the background. You can start a new shell with
docker exec -it sky /bin/bash.
Once you are done with experimenting with sky, remember to delete any clusters and storage resources you may have created using the following commands:
# Run inside the container $ sky down -a -y $ sky storage delete -a -y
Finally, you can stop the container with:
$ docker stop sky
Enabling shell completion¶
SkyPilot supports shell completion for Bash (Version 4.4 and up), Zsh and Fish. This is only available for
click versions 8.0 and up (use
pip install click==8.0.4 to install).
To enable shell completion after installing SkyPilot, you will need to modify your shell configuration.
SkyPilot automates this process using the
--install-shell-completion option, which you should call using the appropriate shell name or
$ sky --install-shell-completion auto $ # sky --install-shell-completion zsh $ # sky --install-shell-completion bash $ # sky --install-shell-completion fish
Shell completion may perform poorly on certain shells and machines.
If you experience any issues after installation, you can use the
--uninstall-shell-completion option to uninstall it, which you should similarly call using the appropriate shell name or
$ sky --uninstall-shell-completion auto $ # sky --uninstall-shell-completion zsh $ # sky --uninstall-shell-completion bash $ # sky --uninstall-shell-completion fish