# Deploy from Git Kilby can build a service from source. It reads the repository, works out how to build and run it (the **plan**, which you can edit before anything is deployed), builds an image outside your cluster, and deploys it like any other service. The first deploy and every later one show their stages (clone, detect, build, push, release, health check, live) with a log, and a failed build comes with a diagnosis in plain English. ## Two ways to point Kilby at code | | Public repository | Private repository | |---|---|---| | Where | A repository on github.com, gitlab.com and bitbucket.org that anyone can read | A repository on GitHub that your workspace shares with the Kilby GitHub App | | What you give Kilby | The repository's address | A connection to your GitHub account or organisation | | How Kilby gets the code | Downloads the host's public archive, with no credential | Downloads it with a short-lived token of the App | | New commits | You ask for them, or call a [build hook](#build-hooks), or Kilby [checks](#check-for-new-commits) | A push that changes the service's folder builds it, through a webhook | | Available here | Yes | Yes | ## Deploy a public repository In the dashboard, choose **New service**, then the **Public Git repository** tab. 1. Paste the repository's address, for example `https://github.com/owner/repo`. Kilby accepts github.com, gitlab.com and bitbucket.org, over https, without a port, a user name or a password. A branch page such as `https://github.com/owner/repo/tree/main/apps/api` works too: it fills in the branch and the folder. 2. As you type, Kilby checks that the repository exists and shows which commit the branch is at. The check is **one archive request** to the host, cut after the first kilobytes, and its answer is kept for 60 seconds. 3. Leave **Branch or tag** empty for the repository's default branch, or type a branch, a tag or a full commit id. 4. **Continue** downloads the archive and shows the same plan screen as for GitHub: the runtime and framework Kilby found, the install, build and start commands, the port, the variables the code reads, and warnings. Change anything, choose an instance type, and deploy. With the [CLI](/docs/cli.md): `kilby deploy --git https://github.com/owner/repo --ref main`. Run it again later and it builds only when the commit changed. With the [REST API](/docs/api.md), send a `source` of kind `public_git` to `POST /api/v1/services`: ```json { "name": "my-api", "type": "web", "source": { "kind": "public_git", "url": "https://github.com/acme/express-api", "ref": "main" } } ``` `GET /api/v1/git/public/check` and `GET /api/v1/git/public/plan` do the same reading as the dashboard, so a script can look before it creates. ### What Kilby does and does not send Kilby asks the host for the repository's **archive** and nothing else. The request has no credential of any kind: no token, no cookie, no header that says who you are. It never calls the host's REST API: an address shared by every Kilby customer would use up an API limit that is counted for the address, not for you. - Only the hosts of the list are asked, over https, and a redirect is followed only inside the host's own family (`github.com` hands the download to `codeload.github.com`; anything else is refused). - The address a connection really goes to is checked when it is made: an address that is not on the public internet is refused, whatever name led to it. - The archive is checked while it downloads, with the same rules as any source: at most 200 MB, a limited number of files, no link that points outside the repository, no path that leaves it, no device files. An archive that breaks a rule is refused, and no build starts. - The archive goes into Kilby's build store and the build downloads it from a short-lived address, exactly as for a repository on GitHub. Your code runs only in the build service, never inside Kilby. ## Which commit was built The archive says which commit it is of: `git archive` writes the commit id into the archive's header, and GitHub, GitLab and Bitbucket all make their archives that way. Kilby reads it from there and shows it on the deploy and in the service's header. If the header has none, Kilby looks at the archive's top folder, which some hosts name after the full commit id. If neither gives a full commit id, the build says **unknown commit**. Kilby does not make one up. A public archive carries no commit message and no author, so a deploy has a number and a commit id but no title. An unknown commit has a cost: Kilby cannot tell whether the repository changed, so **Deploy latest commit** always builds, and the check for new commits cannot work for that repository (it says so). ## Getting new commits A public repository has no push webhook: nobody can tell Kilby when it changes. There are three ways to get a new commit onto the service. ### Deploy latest commit The button on the deploys page and in the service's settings. Kilby asks the host for the latest commit of the branch. If it is the commit of the newest build, nothing new is built and the page says so, with a way to build it again anyway. Otherwise it builds that exact commit. ### Build hooks A **build hook** is a secret URL that your CI calls after a push. Create it in the service's settings; the URL is shown once, so keep it in your CI's secrets. ```bash # Build the latest commit if it changed since the last build curl -fsS -X POST "$KILBY_BUILD_HOOK" # Build it even if it did not change curl -fsS -X POST "$KILBY_BUILD_HOOK?force=1" ``` The answer is `202` with the build, or `200` with `"status": "up_to_date"` when there is nothing new. The hook can start a build of this one service and nothing else: no parameter names a repository, a ref or a commit. It allows 4 calls a minute per hook and 12 forced builds an hour. It works while the person who created it is a member of the workspace and may deploy; revoking it makes the URL stop working at once. It is a different URL from the [deploy hook](/docs/deploys.md#what-starts-a-deploy) that deploys an image. ### Check for new commits Off by default. When you turn it on for a service, Kilby asks the host for the latest commit of its branch every 15 minutes and builds it when it changed. Services that follow the same repository and branch share one request. A workspace can turn it on for up to 5 services. Because every customer reaches the host from the same address, the checks of all services together have a **budget** of 120 requests an hour, and none is sent while a host has asked Kilby to slow down. When the budget is used up, the services that waited longest are checked first, and the time between two checks of a service grows; the service's settings show when it was last checked and why a check could not tell. A build that failed is not started again for the same commit. ## Limits, and when a host says slow down A workspace runs up to 2 builds at once; on Hobby, 1 at a time. A build that cannot start yet waits in the queue, and its page says what it is waiting for. Kilby makes at most 30 archive requests a minute to one host from each of its servers, for everyone together, and keeps a short list of repositories it has just asked about. Per person, the dashboard and the API allow 40 checks and 10 new plans a minute. When a host answers that it is limiting downloads (HTTP status 429, or 403), Kilby stops asking that host until the wait it named is over (two minutes when it named none, fifteen at most) and says **Try again in a few minutes** with the time. A rate limit is never shown as "not found". It is nobody's fault, and nothing you did. ## Private repositories A private repository and one that does not exist look the same to a client with no credentials, so Kilby cannot tell them apart. When the host answers that it has no such repository, Kilby says: **We could not find that public repository. Check the address. If it is private, connect GitHub to deploy private repositories.** Private repositories come from GitHub, through the Kilby GitHub App. You connect your GitHub account or organisation to a workspace once, and choose there which repositories Kilby may read. ### Connect GitHub 1. Open **New service** and the **GitHub** tab (or **Workspace settings**, then **GitHub**) and choose **Connect GitHub**. 2. GitHub asks where to install the Kilby app. Choose your account or an organisation, and whether the app sees **all repositories** or **only the ones you select**. 3. GitHub brings you back to Kilby. The **GitHub** tab lists the repositories the app can see, the most recently updated first, with **Private** or **Public** and when each was updated. Search by name, or type `owner/name`. If you connected more than one account, choose the account above the list. 4. Choose a repository. Kilby reads it and shows the plan; the **Branch** menu there has the repository's default branch selected, and a different branch reads the repository again. Choose an instance type and deploy. The tab also has **Add account**, to connect another account or organisation, and **Configure GitHub access**, which opens the installation on GitHub: that is where you change which repositories Kilby sees. When an account shares no repository, the tab says so and takes you there. ### Who can connect an account A connection gives the workspace every repository the app can see on that account, so Kilby checks twice. In Kilby, only the owner and the admins of the workspace connect accounts. On GitHub, only an **owner** of an organisation can connect it, and a personal account can only be connected by its own owner. Kilby asks GitHub about you with your own login, once, while you connect, and does not keep it. Somebody who can read one repository of an organisation cannot connect the organisation. If you are not an owner of the organisation, GitHub asks an owner to approve the installation, and Kilby tells you it is waiting for them. Nothing is connected yet. When an owner has approved it, an owner of the organisation connects it to the workspace: in Kilby they choose **Connect GitHub**, then **Already installed it? Choose from my GitHub accounts**, and **Connect** the organisation. ### What the app can do The app reads the contents of the repositories you allow, to build them, and writes commit statuses and checks, so GitHub shows whether a deploy worked. It can also read and comment on pull requests. It reads the members of an organisation only to check who may connect it. It cannot push code or change settings, and it cannot see repositories you did not allow. To build, Kilby uses a token that GitHub gives it for one installation and that lasts for one hour. ### Which pushes build With **Deploy automatically** on, a push to the branch a service follows builds and deploys it when the push changed a file in the folder the service builds: its **root directory**, or the whole repository when it has none. A file that was added, removed, renamed or modified in the folder counts. The folder is compared whole: a push that changes `web/landing/index.html` counts for the folder `web/landing`, and one that changes only `web/landing-old/index.html` does not. In a repository with several services this means that a push that changes only `docs/`, or only the folder of another service, builds none of the others. Kilby does not list a push it did not build among the service's deploys. Kilby takes the list of changed files from GitHub's push event, and builds whenever it cannot be sure the list is whole. It builds the service although no file of its folder is listed when: - the push has 20 commits or more, because GitHub cuts the list of commits there; - a commit of the push lists no files at all, as a merge commit or an empty commit does; - the push was forced, or it created the branch; - one of its commits was already in the repository on another branch; - the event does not list its commits, or does not end at the commit it names; - the branch has moved on since the event was sent. Kilby always builds the latest commit of the branch, and the files of an older push do not describe it. **Deploy latest commit** and **Retry deploy** always build, whatever changed. ### Disconnect **Unlink** in **Workspace settings**, then **GitHub**, removes the connection from the workspace. Services already built from it keep running; new deploys from its repositories stop. The app stays installed on GitHub until you uninstall it there. If you uninstall it on GitHub, the connection is removed from Kilby too. ## Coming from Render or Heroku Some files those platforms use are read too, for Python apps: - **`build.sh`**: when the folder has one (Render's guides call the build script so), the plan runs it as the build command, unless it touches the database. Kilby builds have no database access, so the steps that need it (`manage.py migrate`, `createsuperuser`, `loaddata`, `flush`, the app's own management commands, `flask db upgrade`, `alembic upgrade`, `prisma migrate` and the like) are moved to the **pre-deploy command**, which runs before each deploy with the service's variables, and the build command runs the rest of the script, in its order and with bash. Kilby splits a script only when each such step is on a line of its own, outside any `if` or loop, and the script does not change folder before it; otherwise the plan says so and leaves the two commands for you to write. - **`Aptfile`**: the Debian packages it names, one per line, are installed in the image, both where it is built and where it runs (Tesseract, ffmpeg, poppler-utils...), before your dependencies and before the build command, so a build script finds them. Lines that are not package names, such as `:repo:` lines and links to `.deb` files, are left out, and the plan says which. - **Django static files**: the plan only suggests adding WhiteNoise when it is not in your dependencies, and `collectstatic` when `build.sh` does not run it. `collectstatic` runs during the build, where only the variables marked **Build** exist. - **Python version**: without Render compatibility, the plan takes it from `.python-version`, then `runtime.txt`, then `requires-python`, and warns when that version no longer gets security fixes. Render does not read `runtime.txt`, so in a repository with a `render.yaml` the plan says the app may run another Python there; with Render compatibility on, `PYTHON_VERSION` sets it (see [Render compatibility](/docs/environment.md#render-compatibility)). A build never changes a running service's Python by itself. - **PostgreSQL drivers**: `psycopg` (version 3) without `[binary]` loads PostgreSQL's client library, libpq, when it starts, so the image gets `libpq5`; `psycopg2` and `psycopg[c]` are compiled, so the build gets `libpq-dev` too. `psycopg[binary]`, `psycopg2-binary`, `asyncpg` and `pg8000` need nothing more. - **Render's variables**: see [Render compatibility](/docs/environment.md#render-compatibility). To bring a whole `render.yaml` across, [import it as a blueprint](/docs/blueprints.md). ## When something goes wrong | What you see | What it means | What to do | |---|---|---| | We could not find that public repository | The host has no public repository at that address, branch, tag or commit | Check the address and the ref. For a private repository, connect GitHub | | The host is limiting downloads from Kilby | A rate limit, of the host or of Kilby's own budget | Wait the time shown, then try again | | The repository is larger than the build limit | The archive is over 200 MB | Deploy a smaller repository, or remove large files from it | | The repository holds a link that points outside it | The archive breaks the safety rules of a source | Remove the link from the repository | | Unknown commit | The host's archive does not say which commit it is of | The build works as usual; Deploy latest commit always builds | | That commit is no longer available | A retry asked for a commit the host no longer has, for example after a force push | Deploy the latest commit | ## Related - [Services](/docs/services.md): the kinds of service and their settings. - [Deploys, rollbacks and downtime](/docs/deploys.md): what starts a deploy, and why a deploy fails. - [Environment variables and secrets](/docs/environment.md): the variables your code reads, and build variables. - [CLI](/docs/cli.md): `kilby deploy` and `kilby builds`.