Heartbeat

Heartbeat is a service that helps keep track of your digital heartbeat. It works by keeping track of pings from your devices (or, clients) and keeping track of how long it has been since you last used a device.

Well-behaved clients would normally ping the server only while they are still active (not being used or idle, the specifics depend on the device), usually once every minute.

This book aims to be a one-stop resource to get your own instance of the heartbeat server and clients up and running. You can contribute to this book on GitHub.

Getting Started

To get started with Heartbeat, you will need to install Heartbeat, set up a database, and configure the server.

Installation

Installing Heartbeat

The easiest way to install heartbeat is by obtaining the latest release archive from the releases. These release binaries are (mostly) statically linked and require no runtime dependencies. You may simply download the archive corresponding to your platform and extract it.

Docker

First-party tagged and annotated Docker images are built on every commit to the main branch and on every tag and can be pulled from the GitHub Container Registry.

$ docker pull ghcr.io/lmaotrigine/heartbeat:latest

Valid tags are:

  • latest: Always points to the tip of main. Breaking changes may occur unannounced.
  • The first 7 characters of a commit on main (corresponding to the output of git rev-parse --short).
  • Any tag corresponding to a tagged release (e.g. 0.1.0)

Customizing the Image

You may optionally pass these additional build arguments while building the Heartbeat Docker image yourself:

  • RUST_VERSION: defaults to alpine, which is the latest stable. Please make sure to use an Alpine base since it allows the C runtime to be linked statically. glibc-based distros like Debian will make the final image a lot bigger. However, Heartbeat’s licence is GPL-compatible, should you wish to statically link against glibc. (Please don’t, though).
  • FEATURES: the features to enable in the resultant binary. By default, only the default feature set is enabled. The only relevant setting for overriding this is sqlx-tls, which allows you to connect to your PostgreSQL database over TLS.

Build and Install Heartbeat from Source

Alternatively, you can build heartbeat from source.

Requirements

Heartbeat requires the following tools and packages to build:

  • cargo and rustc
  • A C compiler for your platform
  • git (to clone this repository and also to attach some build metadata to the resulting library)

Compiling

First, you’ll want to check out this repository:

$ git clone https://github.com/lmaotrigine/heartbeat.git
$ cd heartbeat

With cargo installed, you can simply run:

$ cargo build --release

Features

Some functionality is gated behind feature flags. All features are enabled in the pre-built binaries.

  • badges: Enables support for the /badge/* routes. This enables generation of SVG badges in the style of shields.io, without having to write long URLs for the dynamic badges that shields.io provides. Enabled by default.
  • webhook: Enables logging selected events to a Discord webhook. Enabled by default.
  • migrate: Required to run the embedded database migrations. You will need to run this if the database schema is changed at some point. Such changes will be considered breaking and backwards incompatible. The migrations will help you to upgrade from previous versions of the schema.
  • sqlx-tls: Allows you to connect to a PostgreSQL server over TLS. Such a setup is not necessary if your database instance is running on a local network and is not exposed publicly.

Setting up the Database

Heartbeat uses a PostgreSQL database to store statistics, device information and beat history. This was chosen due to it being battle-tested and robust and scalable. There are no immediate plans to make the storage backend configurable.

A first-party docker-compose.yml file is provided to run Heartbeat alongside Postgres on the same docker network on the same host. Any other setup requires that you bring your own database. Please consult the PostgreSQL documentation for the same.

Running

Heartbeat requires some configuration information at runtime to communicate with the database, set up routing, and finally to listen and serve. Sensible defaults for the various options are picked if not provided. Detailed information on configuring the server are provided in a separate section.

A sample configuration file is distributed with the source, and in release archives. You can view the latest version online on GitHub. This file has comments explaining the various fields.

After configuring the server, it can be simply started in the background.

The quickest way to get started by using Docker is to clone the repository and run

$ docker compose up

And visit http://127.0.0.1:6060 in a browser to check if everything went well.

Registering your first device

Assuming that you set a value for the secret_key parameter – there are several ways to generate one, one of which is to run heartbeat gen-key, which is distributed in release archives and can also be built from source – you can now hit the /api/devices endpoint to register your first device.

First, you’ll need a name for your device, let’s call it Laptop (but you can get creative!)

Using the command line, with curl installed (it should be by default on Windows and any sane Linux distro, and on macOS will prompt you to install Xcode command line tools), you can just run

$ curl -XPOST -H 'Authorization: <my_secret_key>' -H 'Content-Type: application/json' -d '{"name": "Laptop"}' http://127.0.0.1:6060/api/devices

That’s a long one! You will probably want to make a convenience wrapper for this. We don’t provide one out of the box because various users might have multiple very creative ways to store their secrets, and we leave it to them to tailor it to their needs.

Once you’ve registered a device, you will get a response back with a token field in it. Note this down because you cannot retrieve it from the server at any other time. You can then use this token as the Authorization header when you send your first beat.

Use this token wherever appropriate when you set up the client for your device, and test it out by sending your first beat. If all goes well, the homepage should refresh with new statistics.

Configuration

This document explains how Heartbeat’s configuration system works, as well as available keys or configuration.

Hierarchical structure

Heartbeat allows configuration to be specified through a file, through environment variables, or in the command line during invocation. If a key is specified in multiple locations, the value in the location with the highest priority will take precedence. The locations in decreasing order of priority are:

  • Command line
  • Environment variables
  • Config file fields

Specifying the configuration file

By default, Heartbeat looks for a configuration file called config.toml in the heartbeat home folder. This is usually ~/.heartbeat on Unix platforms and %USERPROFILE%\.heartbeat on Windows. This can be overridden by the HEARTBEAT_HOME environment variable. If the file doesn’t exist, it silently moves on. This can be overridden by the environment variable HEARTBEAT_CONFIG_FILE, or the command line flag -c/--config-file. If a regular file is not found at the location pointed to by the value of this parameter, an error will be printed and Heartbeat will exit.

Configuration format

Configuration files are written in the TOML format, with simple key-value pairs and sections (tables). Values can be specified based on profiles using the debug or release tables, both of which are optional. An example config file with all fields filled out is given below:

# the address to bind to
# this will be parsed as a `std::net::SocketAddr`
bind = "0.0.0.0:6060"

# this is used for <title> tags
# and some headings
server_name = "Some person's heartbeat"

# the full url to the server
live_url = "http://127.0.0.1:6060"

# a random URL-safe string.
# if left blank, addition of devices is disabled.
# this may be generated using `openssl rand -base64 45`
secret_key = ""

# don't change this unless you're using a fork
repo = "https://github.com/lmaotrigine/heartbeat"

[database]
# this is the default if you're using the docker-compose file
# otherwise, the format is postgresql://username:password@host/database
dsn = "postgresql://heartbeat@db/heartbeat"

[webhook]
# leave this blank to disable webhooks
url = ""

# one of:
# - "all"             log each beat + the below
# - "new_devices"     log new devices + the below
# - "long_absences"   log absences longer than 1 hour
# - "none"            don't log anything
level = "none"

# override some values for debug builds for easier testing.

[debug]
bind = "127.0.0.1:6061"
live_url = "http://localhost:6061"

# nested keys work as expected
[debug.database]
dsn = "postgresql://heartbeat@localhost/heartbeat"

Environment variables

Heartbeat can be configured through environment variables in addition to the TOML configuration file. Environment variable names take the form of HEARTBEAT_<KEY> where <KEY> is the full path to the key in the TOML configuration, in uppercase, with dots (.) replaced with underscores (_). For example, the database.dsn config value can be specified through the environment variable HEARTBEAT_DATABASE_DSN. Environment variables will override the values in the TOML configuration file, regardless of profile. Environment variable names must not contain the profile key.

Command line arguments

Heartbeat can be configured by passing configuration values as command line parameters. Command line flags are formatted as per the POSIX standard, with two leading dashes (--) and dots (.) and underscores (_) replaced with dashes (-). For example, the database.dsn config value can be passed via the command line argument --database-dsn=. The equals sign is optional. For a full list of available configuration values, their usage, default values, and available shorthands, run heartbeat --help.

If any required configuration parameter is not specified (all parameters with no defaults are considered required), Heartbeat will print an error message and exit.

Configuration keys

This section documents all configuration keys.

[database]

the [database] table contains configuration related to the PostgreSQL database

database.dsn

  • Type: string
  • Default: postgresql://heartbeat@db/heartbeat if running within Docker, postgresql://postgres@localhost/postgres otherwise.
  • Environment: HEARTBEAT_DATABASE_DSN
  • Command line: -d/--database-dsn

The PostgreSQL connection string for the database that Heartbeat should use. The database must exist, and the user must have at least CREATE SCHEMA privileges on it.

[webhook]

The [webhook] table deals with configuration related to logging events to Discord webhooks. This is only relevant if the webhooks feature is enabled, which is the default.

webhook.url

  • Type: string
  • Default: empty
  • Environment: HEARTBEAT_WEBHOOK_URL
  • Command line: --webhook-url

The URL to the Discord webhook to log events to. If empty, logging is disabled.

webhook.level

  • Type: string, one of all, new_devices, long_absences, none
  • Default: none
  • Environment: HEARTBEAT_WEBHOOK_LEVEL
  • Command line: --webhook-level

The maximum level of events to log to the webhook. The possible values in descending order of level are:

  • all: logs beats, along with everything below
  • new_devices: logs when a new device is added, along with everything below
  • long_absences: logs when an absence longer than 1 hour has ended.
  • none: No events are logs.

secret_key

  • Type: string
  • Default: none
  • Environment: HEARTBEAT_SECRET_KEY
  • Command line: -s/--secret-key

A random, header value safe string (≤256 bytes) that will be the master authentication token for administrative actions like adding devices or regenerating their tokens. If this value is empty, administrative actions are disabled.

repo

  • Type: string
  • Default: https://github.com/lmaotrigine/heartbeat
  • Environment: HEARTBEAT_REPO
  • Command line: -r/--repo

The URL to the repository of the Heartbeat source. This should be left unchanged unless you are running a fork.

server_name

  • Type: string
  • Default: Some person's heartbeat
  • Environment: HEARTBEAT_SERVER_NAME
  • Command line: --server-name

A human-readable name for the server. Used in <title> tags and other metadata.

live_url

  • Type: string
  • Default http://127.0.0.1:6060
  • Environment: HEARTBEAT_LIVE_URL
  • Command line: -u/--live-url

The publicly accessibly base URL for the Heartbeat server. This is used to format URLs in webhook logs so that they resolve to the right routes. When running in production, this must be overridden.

bind

  • Type: string, must be a valid socket address (either <host>:<port>, or a UDS).
  • Default: 127.0.0.1:6060
  • Environment: HEARTBEAT_BIND
  • Command line: -b/--bind

The socket address for the server to bind to and listen on.

config_file

  • Type: string, must be a path to a valid file
  • Default: $HEARTBEAT_HOME/config.toml (see above for more information)
  • Environment: HEARTBEAT_CONFIG_FILE
  • Command line: -c/--config-file

This value obviously cannot be specified in the TOML configuration file. It can be set in the environment or provided in the command line to override the default config file location.

Clients

First-party clients are available for a variety of platforms. The future of these implementations and the possibility of new ones being added depend entirely on the bandwidth of the maintainers and the possibility of testing them, including access to relevant hardware and software. Currently, supported client platforms are:

  • Linux: via heartbeat-unix, only supports X.org. Support for Wayland is planned, but not immediately in the works, and help in this area would be appreciated.
  • macOS: via heartbeat-unix.
  • Android: A Tasker project bundle is available on TaskerNet. An Android app is currently under development, but there is no ETA.
  • Windows: via heartbeat-windows, tested on the latest stable build of Windows 10 and latest insiders build of Windows 11.

Implementing your own client to support other platforms is a straightforward process. You must implement the API, specifically for the /api/beat endpoint, and hit it every so often while the device is actively being used. This can be determined by various factors such as the last time an input device was used, last time the screen was unlocked, the last time the device was awakened from an idle state, etc. At the very least you will need to make network requests, so devices without this capability cannot be supported.

API Reference

This document is a hand-crafted reference of the entire public API exposed by the server. Care must be taken to keep this document up-to-date whenever changes are made to the request or response types, or new routes are added or existing routes removed.

Devices

Actions relating to devices. These routes are non-existent if the secret_key configuration parameter is left empty.

POST /api/devices

Register a new device.

  • Authentication: Authorization header with the same value as the secret_key configuration parameter of the server.
  • Request body:
    • Content Type: application/json
    • Schema: {name: string}
    • Example: {"name": "Laptop"}
  • Response:
    • Content Type: application/json
    • Schema: 
      {
  id: number,
  name: string,
  token: string
}
    • Example: 
      {
  "id": 0,
  "name": "Laptop",
  "token": "barfooed"
}
  • Errors
    • 400: Invalid request body
    • 401: Invalid or missing Authorization header
    • 405: Not a POST request

POST /api/devices/:id/token/generate

(Re)generate the token for a registered device.

  • Authentication: Authorization header with the same value as the secret_key configuration parameter of the server.
  • Path parameters:
    • id: The ID of the device to regenerate the token for
  • Response:
    • Content Type: application/json
    • Schema: 
      {
  id: number,
  name: string,
  token: string
}
    • Example: 
      {
  "id": 0,
  "name": "Laptop",
  "token": "barfooed"
}
  • Errors:
    • 401: Invalid or missing Authorization header
    • 404: Device with the provided ID does not exist
    • 405: Not a POST request

Beats

Actions that a client will have to implement.

POST /api/beat

  • Authentication: Authorization header with the device token which was obtained during registration. If regenerated, old values of the token are no longer considered valid.
  • Response:
    • Content Type: text/plain
    • Schema: A Unix timestamp corresponding to the time the beat was acknowledged.
    • Example: 1698915036
  • Errors:
    • 401: Invalid or missing Authorization header
    • 405: Not a POST request

Statistics

Operations to retrieve statistics about the server.

GET /api/stats

  • Authentication: none
  • Response:
    • Content Type: application/json
    • Schema: 
      {
  last_seen: number, // Unix timestamp of last beat
  last_seen_relative: number, // number of whole seconds since last beat
  longest_absence: number, // duration of longest absence ever (in seconds)
  num_visits: number, // number of visits to the site (not including API calls)
  total_beats: number, // number of beats since the server started operating
  devices: Device[], // array of devices, see type definition below.
  uptime: number, // number of whole seconds since the server was first set up
}
/////
type Device = {
  id: number,
  name: string,
  num_beats: number,  // number of beats by this device since the server started operating
}
    • Example: 
      {
  "last_seen": 1698915320,
  "last_seen_relative": 41,
  "longest_absence": 84898,
  "num_visits": 1628,
  "total_beats": 120062,
  "devices": [
    {
      "id": 0,
      "name": "Laptop",
      "last_beat": 1698825626,
      "num_beats": 36308
    },
    {
      "id": 1,
      "name": "Phone",
      "last_beat": 1698825626,
      "num_beats": 639
    },
    {
      "id": 2,
      "name": "Workstation",
      "last_beat": 1698915320,
      "num_beats": 83115
    }
  ],
  "uptime": 8082297
}

GET /api/stats/ws

A WebSocket endpoint to stream statistics. Responses are JSON strings in the same schema as above, and are streamed at the rate of 1/second.

Troubleshooting

The first step to debugging issues with your installation is to look at the logs. To make it less of a privacy law hassle, all logs exist ephemerally only in the console I/O. i.e. they are streamed to stdout/stderr depending on their origin and severity. Fortunately, inspecting them is easy enough:

  • If you are simply running the server in the background using screen/tmux you can switch to the appropriate windows to see them.
  • With Docker, you can view them using docker logs, or docker compose logs if you are using docker-compose.
  • If you are using a process manager like systemd or pm2, you can check your journal, or whatever logging facility it provides. The pertinent error messages will most likely be helpful enough to find out and fix your problem.

By default, the maximum log level is INFO for release builds and DEBBUG for debug builds. You should be using release builds in production. To control the log level, set the RUST_LOG environment variable to an acceptable value. Consult this documentation for more information.

Some common pitfalls are documented here, with suggested debugging solutions. If your issue is not listed here, and you cannot figure it out, we’re happy to help. Open an issue or discussion on GitHub, provide us with the relevant logs and reproduction steps, and we will do our best to help you out

Frequent Problems

Can’t connect using Docker

The default port is 6060. You should map this port to a host port (preferably the same one). If you still cannot access it from localhost, check docker logs or docker compose logs to verify that the server is listening where you expect it to, and that the configuration is loaded correctly.

Can’t connect while not using Docker

Without Docker’s networking shenanigans, you should be able to access the server on localhost always. Ensure that the server hasn’t prematurely exited due to an error or signal. If not, please check that the bind address is the same one that you are trying to access.

401 errors on /api/beat

When you believe you have the right token, but the server is rejecting it, it is likely that the token was regenerated at some point. Regenerate it once more to ensure that you note it down this time.

401 errors on /api/device routes

If the secret key isn’t being recognized as valid by the server, try temporarily overriding it using environment variables or command line flags, and see if the issue persists. This will likely help identify the cause of the issue if it isn’t just a typo.

Config values from TOML file not being read on Docker

Make sure your config.toml file is mounted at /.heartbeat/config.toml. This is where the executable looks for the config file by default. If you overrode this, make sure the mount location matches with the override.

error communicating with database: failed to lookup address information: Name or service not known

Heartbeat could not establish a connection to the PostgreSQL server. Make sure that the server is online and reachable from the network Heartbeat is running on. If there is another message indicating secure connection failure, ensure that the sqlx-tls feature is enabled in your binary. In release archives, this is always the case, but in Docker, it requires passing it to the FEATURES build argument.

pool timed out while waiting for an open connection

Your PostgreSQL server is possibly at capacity for maximum connections. Check the server logs or try again after some time. Failing that, enable debug logging