Skip to contents

Installation

Source: vignettes/installation.Rmd
installation.Rmd

This vignette covers CS9 installation options and infrastructure requirements.

Start here: SQLite, with no server

CS9 reads its database settings from CS9_DBCONFIG_* environment variables. Set the driver to SQLite and every access level becomes a file on disk. There is no server to install, no Docker container to start, no schema to create, and no database user to authenticate as.

  1. Install CS9:

  2. Put these six settings in your .Renviron. Give CS9_PATH a real directory: an empty value counts as missing and check_environment_setup() reports it as an error.

    # CS9 Core Configuration
CS9_AUTO=0
CS9_PATH=/home/myuser/cs9

# One SQLite file per access level
CS9_DBCONFIG_ACCESS=config/anon
CS9_DBCONFIG_DRIVER=SQLite
CS9_DBCONFIG_DB_CONFIG=/home/myuser/cs9/config.sqlite
CS9_DBCONFIG_DB_ANON=/home/myuser/cs9/anon.sqlite

  3. Restart R, then check the configuration:

    library(cs9)
cs9::check_environment_setup(verbose = FALSE)$status
#> [1] "ok"

  4. Open the database. Step 3 only validates the variables. It reads no file and creates no table, because CS9 builds its table objects lazily. Connecting is what proves SQLite works:

    cs9::config$tables$config_log$connect()
#> Creating table config_log

con <- DBI::dbConnect(RSQLite::SQLite(), Sys.getenv("CS9_DBCONFIG_DB_CONFIG"))
DBI::dbListTables(con)
#> [1] "config_log"
DBI::dbDisconnect(con)

    DBI and RSQLite arrive with csdb, which CS9 depends on, so neither needs installing separately.

CS9_DBCONFIG_ACCESS is a /-separated list of access levels, and it must include config: CS9 builds its four configuration tables from that access level. Every access level in the list needs its own CS9_DBCONFIG_DB_<ACCESS> variable holding the path to its file, so the list above needs CS9_DBCONFIG_DB_CONFIG and CS9_DBCONFIG_DB_ANON. CS9 creates each file, and any missing parent directory, on the first connection.

The driver string is matched case-insensitively, so SQLite, sqlite and SQLITE all select it.

What SQLite does not need

check_environment_setup() does not ask for CS9_DBCONFIG_SERVER, CS9_DBCONFIG_PORT, CS9_DBCONFIG_USER, CS9_DBCONFIG_PASSWORD or any CS9_DBCONFIG_SCHEMA_* variable when the driver is SQLite. SQLite is a file, and it has no schemas.

That is about validation, not about reading. CS9 reads every CS9_DBCONFIG_* variable into its configuration list whatever the driver is, because that list has one shape for every backend. The SQLite connection then uses none of them. Setting CS9_DBCONFIG_SERVER under SQLite is recorded and ignored, not rejected.

vignette("backends", package = "cs9") puts the SQLite and PostgreSQL environments side by side, and lists what each backend does with every variable.

cs9example needs no .Renviron at all

cs9example is a complete surveillance system you can run on a bare machine. It sets its own SQLite configuration under tempdir() whenever CS9_DBCONFIG_ACCESS is empty, so it needs no .Renviron, no server and no configuration of any kind:

pak::pak("niphr/cs9example")
library(cs9example)
names(global$ss$tables)

A real .Renviron still wins. The defaults apply only when CS9_DBCONFIG_ACCESS is unset.

Which backend to use

Use SQLite for a local install, for examples, for tests, and for a single-writer pipeline on one machine. Use PostgreSQL for a production surveillance system, where several processes write at once and the data outgrows one machine.

If you chose PostgreSQL, continue below.

Installation options

CS9 can be used in two configurations.

CRAN installation (limited functionality)

Install CS9 from CRAN for documentation, examples, and development tools:

What works:

  • Package documentation and help files
  • Function examples and vignettes
  • Environment diagnostics (cs9::check_environment_setup())
  • Code templates and development utilities
  • Basic configuration management

What does not work:

  • Database connectivity
  • Running surveillance tasks

Full installation (database-driven surveillance)

For complete surveillance functionality, you need a database backend and proper environment configuration. For production deployments, Docker is the recommended approach.

Database setup

CS9 uses PostgreSQL as its production database backend. The rest of this section sets one up. If you want the SQLite backend instead, everything you need is at the top of this vignette and none of the steps below apply.

PostgreSQL installation 

# Run PostgreSQL in Docker
docker run --name cs9-postgres -e POSTGRES_PASSWORD=yourStrongPassword100 \
  -e POSTGRES_DB=cs9_surveillance -p 5432:5432 -d postgres:13

# Connect to create additional databases/schemas as needed
docker exec -it cs9-postgres psql -U postgres -d cs9_surveillance

Option 2: System installation 

# Ubuntu/Debian
sudo apt-get install postgresql postgresql-contrib

# macOS with Homebrew
brew install postgresql

# Start PostgreSQL service
sudo systemctl start postgresql  # Linux
brew services start postgresql   # macOS

Database schema setup

Once PostgreSQL is running, create your surveillance database:

-- Connect as postgres user
CREATE DATABASE cs9_surveillance;
CREATE USER cs9_user WITH PASSWORD 'yourStrongPassword100';
GRANT ALL PRIVILEGES ON DATABASE cs9_surveillance TO cs9_user;

-- Create schemas for different access levels
\c cs9_surveillance
CREATE SCHEMA config;
CREATE SCHEMA anon;
GRANT ALL ON SCHEMA config TO cs9_user;
GRANT ALL ON SCHEMA anon TO cs9_user;

Environment variable configuration

CS9 reads its database connection settings from environment variables, which are typically set in your .Renviron file.

Setting up .Renviron

  1. Find or create your .Renviron file:

    # Check current .Renviron location
Sys.getenv("R_ENVIRON_USER")

# Open .Renviron file for editing
file.edit("~/.Renviron")

  2. Add CS9 configuration variables:

# CS9 Core Configuration
CS9_AUTO=0
CS9_PATH=/home/myuser/cs9

# Database Connection Settings
CS9_DBCONFIG_ACCESS=config/anon
CS9_DBCONFIG_DRIVER=PostgreSQL Unicode
CS9_DBCONFIG_PORT=5432
CS9_DBCONFIG_SERVER=localhost
CS9_DBCONFIG_USER=cs9_user
CS9_DBCONFIG_PASSWORD=yourStrongPassword100
CS9_DBCONFIG_SSLMODE=prefer
CS9_DBCONFIG_ROLE_CREATE_TABLE=x

# Database Schema Configuration
CS9_DBCONFIG_SCHEMA_CONFIG=config
CS9_DBCONFIG_DB_CONFIG=cs9_surveillance
CS9_DBCONFIG_SCHEMA_ANON=anon
CS9_DBCONFIG_DB_ANON=cs9_surveillance

CS9_PATH must be a real directory. An empty value counts as missing, and check_environment_setup() reports it as an error.

Set CS9_DBCONFIG_ROLE_CREATE_TABLE even when you want no role, and use x for that. Leaving the variable out is not the same as leaving it unset in R: Sys.getenv() returns "", csdb substitutes the x sentinel only for a NULL, and its PostgreSQL create_table then emits SET ROLE "" in front of the CREATE TABLE.

  1. Restart your R session after editing .Renviron.

Variable descriptions

Variable Example Value Description
CS9_AUTO 0 Set to 0 for interactive mode, 1 for automated mode
CS9_PATH /home/myuser/cs9 Base path for cs9::path function. Must be a real directory: an empty value counts as missing and fails validation
CS9_DBCONFIG_ACCESS config/anon Database access levels (slash-separated)
CS9_DBCONFIG_DRIVER PostgreSQL Unicode Database driver name
CS9_DBCONFIG_SERVER localhost Database server hostname or IP
CS9_DBCONFIG_PORT 5432 Database server port
CS9_DBCONFIG_USER cs9_user Database username
CS9_DBCONFIG_PASSWORD yourStrongPassword100 Database password
CS9_DBCONFIG_SSLMODE prefer SSL connection preference
CS9_DBCONFIG_ROLE_CREATE_TABLE x Role to take when creating tables. x means take no role
CS9_DBCONFIG_SCHEMA_CONFIG config Schema for CS9 configuration tables
CS9_DBCONFIG_DB_CONFIG cs9_surveillance Database for configuration
CS9_DBCONFIG_SCHEMA_ANON anon Schema for anonymous data tables
CS9_DBCONFIG_DB_ANON cs9_surveillance Database for surveillance data

Package behavior without database configuration

When the database is not configured, CS9 loads with limited functionality and prints messages like:

Environment setup failed: Missing required environment variables: CS9_DBCONFIG_ACCESS, CS9_DBCONFIG_DRIVER, CS9_DBCONFIG_PORT, CS9_DBCONFIG_SERVER
CS9 database configuration not available. Package loaded with limited functionality.
Use cs9::check_environment_setup() to diagnose configuration issues.

Run cs9::check_environment_setup() to see which configuration items are missing.

Docker

An example docker-compose file is available here.

To get started quickly, clone the cs9example repository, which provides a complete template for a CS9 surveillance system.

Apache Airflow Integration

For production surveillance systems, CS9 tasks are typically orchestrated by Apache Airflow. NorSySS uses Airflow to schedule overnight processing pipelines that sequence data retrieval, cleaning, analysis, and output generation — with each step verified before the next begins.

Airflow scheduling pattern

Each CS9 task maps to an Airflow task within a DAG (Directed Acyclic Graph). A typical surveillance DAG runs overnight:

# Example Airflow DAG for CS9 surveillance pipeline
# Schedule: daily at 02:00
dag = DAG(
    'norsyss_surveillance',
    schedule_interval='0 2 * * *',
    default_args={'retries': 2}
)

import_data >> clean_data >> [estimate_trends, nowcasting] >> produce_figures

Benefits of Airflow + CS9

  • Dependency management: Airflow ensures cleaning completes before analysis begins
  • Retry logic: Failed tasks retry automatically with configurable policies
  • Monitoring: Web UI shows pipeline status, execution times, and failure history
  • Scheduling: Cron-based scheduling for daily, weekly, or custom intervals
  • Alerting: Notifications on pipeline failures via email, Slack, or other channels

Docker Compose with Airflow

For a complete Airflow + CS9 deployment, see the docker-examples-csids repository. A typical production setup includes:

  • Airflow webserver + scheduler: Pipeline orchestration (port 8080)
  • PostgreSQL: Database backend for both Airflow metadata and surveillance data
  • CS9 worker: R environment with cs9 and surveillance packages installed
  • Posit Workbench (optional): Interactive development environment (port 8786)

Moving from CRAN to full setup

If you installed CS9 from CRAN and want to enable full surveillance functionality:

  1. Check your current status:

  2. Install PostgreSQL, create the surveillance database, and note the connection parameters (host, port, username, password).

  3. Add the CS9_DBCONFIG_* variables to your .Renviron file, restart R, and verify with cs9::check_environment_setup().

  4. Test that the package now loads fully:

    ss <- cs9::SurveillanceSystem_v9$new(name = "test_system")

Verification and troubleshooting

Verify your setup 

# Load CS9
library(cs9)

# Check environment configuration
cs9::check_environment_setup()

# Test surveillance system creation
ss <- cs9::SurveillanceSystem_v9$new(name = "test_surveillance")
print(ss$name)  # Should print "test_surveillance"

Common issues

“Missing required environment variables”: Check that all CS9_DBCONFIG_* variables are in your .Renviron file and restart R.

“Could not connect to database”: - Verify PostgreSQL is running: sudo systemctl status postgresql (Linux) or brew services list | grep postgres (macOS) - Confirm that the credentials in .Renviron match the database user you created - Test manually: docker exec -it cs9-postgres psql -U cs9_user -d cs9_surveillance

“Package loaded with limited functionality”: Expected when no database is configured. Follow the database setup steps above.

Permission denied errors: Grant the required privileges:

GRANT ALL PRIVILEGES ON DATABASE cs9_surveillance TO cs9_user;
GRANT ALL ON SCHEMA config TO cs9_user;
GRANT ALL ON SCHEMA anon TO cs9_user;

Next steps