Contributing to SkyPilot#
Thank you for your interest in contributing to SkyPilot! We welcome and value all contributions to the project, including but not limited to:
Pull requests for bug fixes and new features
Test cases to make the codebase more robust
Tutorials, blog posts and talks on SkyPilot
We use GitHub to track issues and features. For new contributors, we recommend looking at issues labeled “good first issue”.
Installing SkyPilot for development#
# SkyPilot requires python >= 3.7.
# You can just install the dependencies for
# certain clouds, e.g., ".[aws,azure,gcp,lambda]"
pip install -e ".[all]"
pip install -r requirements-dev.txt
To run smoke tests (NOTE: Running all smoke tests launches ~20 clusters):
# Run all tests except for AWS and Lambda Cloud
# Terminate a test's cluster even if the test failed (default is to keep it around for debugging)
pytest tests/test_smoke.py --terminate-on-failure
# Re-run last failed tests
# Run one of the smoke tests
# Only run managed spot tests
pytest tests/test_smoke.py --managed-spot
# Only run test for AWS + generic tests
pytest tests/test_smoke.py --aws
# Change cloud for generic tests to aws
pytest tests/test_smoke.py --generic-cloud aws
For profiling code, use:
pip install tuna # Tuna is used for visualization of profiling data.
python3 -m cProfile -o sky.prof -m sky.cli status # Or some other command
Testing in a container#
It is often useful to test your changes in a clean environment set up from scratch. Using a container is a good way to do this.
We have a dev container image
berkeleyskypilot/skypilot-debug which we use for debugging skypilot inside a container. Use this image by running:
docker run -it --rm --name skypilot-debug berkeleyskypilot/skypilot-debug /bin/bash
# On Apple silicon Macs:
docker run --platform linux/amd64 -it --rm --name skypilot-debug berkeleyskypilot/skypilot-debug /bin/bash
It has some convenience features which you might find helpful (see Dockerfile):
Common dependencies and some utilities (rsync, screen, vim, nano etc) are pre-installed
requirements-dev.txt is pre-installed
Environment variables for dev/debug are set correctly
Automatically clones the latest master to
/sky_repo/skypilotwhen the container is launched.
Note that you still have to manually run
pip install -e ".[all]"to install skypilot, it is not pre-installed.
If your branch is on the SkyPilot repo, you can run
git checkout <your_branch>to switch to your branch.
Submitting pull requests#
Fork the SkyPilot repository and create a new branch for your changes.
If relevant, add tests for your changes. For changes that touch the core system, run the smoke tests and ensure they pass.
Follow the Google style guide.
Ensure code is properly formatted by running
Push your changes to your fork and open a pull request in the SkyPilot repository.
In the PR description, write a
Tested:section to describe relevant tests performed.
Some general engineering practice suggestions#
These are suggestions, not strict rules to follow. When in doubt, follow the style guide.
FIXME(author_name)instead of blank
TODO/FIXME. This is critical for tracking down issues. You can write TODOs with your name and assign it to others (on github) if it is someone else’s issue.
Delete your branch after merging it. This keeps the repo clean and faster to sync.
Use an exception if this is an error. Only use
assertfor debugging or proof-checking purposes. This is because exception messages usually contain more information.
Use modern python features and styles that increases code quality.
Use f-string instead of
.format()for short expressions to increase readability.
class MyClass:instead of
class MyClass(object):. The later one was a workaround for python2.x.
abcmodule for abstract classes to ensure all abstract methods are implemented.
Use python typing. But you should not import external objects just for typing. Instead, import typing-only external objects under
Environment variables for developers#
export SKYPILOT_DISABLE_USAGE_COLLECTION=1to disable usage logging.
export SKYPILOT_DEBUG=1to show debugging logs (use logging.DEBUG level).
export SKYPILOT_MINIMIZE_LOGGING=1to minimize logging. Useful when trying to avoid multiple lines of output, such as for demos.