Skip to the page
KilbyDocs
Build and run View as Markdown
On this page

Logs, metrics and the shell

Logs#

The Logs tab shows what your service prints to its output (standard output and standard error), live. Print to the console and it shows up here within seconds.

  • Search the lines and filter by level (errors, warnings, info, debug).
  • Live shows the newest lines straight from the instances (200 to 2,000 of them). A time range (the last 15 minutes, hour or 24 hours, the last 7 days, or a custom range) reads what was logged before, including by instances that crashed or were replaced by a deploy.
  • Follow keeps the newest lines in view. Copy saves what is on screen. In Live, Download saves the same; for a time range it saves the stored lines of the whole range, and a file of a very long range stops at a limit and says so in its last line.
  • With several instances, a menu picks one, or shows all of them together.
  • Times are in UTC.

Kilby keeps the logs of every service for 7 days, then deletes them. To keep them longer, or to search them with your own tools, send them to a log stream (see Log streams) or to your own logging service from your program.

Every member of the workspace who can see the service can read its logs, a Viewer included. Do not print secrets.

From the command line or an agent:

sh
kilby logs my-api --tail 100
curl -sS -H "Authorization: Bearer $KILBY_API_KEY" "https://onkilby.com/api/v1/services/<id>/logs?tail=100"

The API answers with plain text, oldest line first. It is a snapshot, not a stream: call it again for newer lines. tail is 1 to 2000 and defaults to 200. With since (an RFC 3339 time, or a duration back from now such as 1h or 7d), until, q, level, limit (1 to 1000) or format (json or text), it searches the stored logs of the last 7 days instead: see the API reference. The output of a cron job's runs is in its Runs tab. The API lists the runs and their results (GET /api/v1/services/<id>/runs) but not their output.

Metrics#

The Metrics tab charts the CPU and memory a service used against the limit of its instance type, for the last 1 hour, 6 hours, 24 hours or 7 days, in UTC. Kilby measures every service once a minute while it runs. A service with several instances also shows how many instances ran. A cron job has no live charts: its tab shows the run time per day, and the Usage page has the month's totals.

For a web service the tab also shows Requests: requests per second, responses by status class (1xx to 5xx), the share of 5xx responses and how long the service took to answer (p50, p95 and p99), for the last 1 hour, 6 hours, 24 hours, 7 days or 30 days. Kilby counts them at the proxy that serves your service, every 30 seconds, so only requests that reach it through Kilby count; paths, hosts, addresses and headers are never recorded. kilby metrics requests my-api and GET /api/v1/services/<id>/metrics/requests?range=24h give the same numbers.

A database has the same charts for the last 24 hours on its page.

GET /api/v1/services/<id>/metrics?range=24h returns the same points as JSON (1h, 6h, 24h or 7d), and GET /api/v1/usage returns what the workspace used this month.

Above each chart, markers show the events Kilby recorded in the same range (deploys, restarts of a container, scaling, suspend and resume, changes of settings, scheduled maintenance), and a list beside the charts gives each one with its time and who did it: hover a marker to read it, click it to find it in the list, and filter the list by kind. Kilby does not record the steps of the autoscaler, so they are not marked; GET /api/v1/services/<id>/events?range=24h returns the same events as JSON. What a restart means is in Troubleshooting.

A service that uses more memory than its instance type has is stopped. If memory reaches the limit line, choose a larger instance type: see Services.

The shell#

The Shell tab opens a terminal inside a running container of the service: bash if the image has it, otherwise /bin/sh. The image must contain a shell. Use it to look around, run a one-off command or check a connection from inside the workspace.

  • Developers, Admins and the Owner can open a shell. Viewers cannot.
  • Opening a shell is recorded in Activity with how long the session lasted, and by default Kilby emails the workspace's owners and admins each time (never the person who opened it).
  • A person can have up to 3 shells open at once. A shell closes after 20 minutes without input, and after 2 hours at most.
  • It is an instance, not a copy: changes to files are lost on the next deploy, when the container is replaced. A service with a disk keeps what is on the disk.
  • It needs a running service. A suspended service has no shell.

The shell is for people. The REST API has no endpoint for it, and an API key cannot open one.