A SyntheticTable holds the contract of one table and its fixed synthetic
datasets. Every data.table that a SyntheticTable returns passed
$validate().
A generator is a function(profile, n, seed, world) that returns a
data.table. generator = NULL uses simulate_from_profile() on profile,
or on profile_schema() of field_types and keys when profile is
NULL. n = NULL means the generator's own size. A generator MAY then
choose its row count, for example a complete grid of series and weeks.
simulate_from_profile() gives 1000 rows for n = NULL.
$validate(d, world) checks d in this order, and stops at the first
check that fails. The error names the check.
columns:dis a data.table, and its column names and order equalnames(field_types).types: each column has the R type of its field type. These areTEXTcharacter,INTEGERinteger,DOUBLEdouble,BOOLEANlogical,DATEDate andDATETIMEPOSIXct.keys: no two rows have the same values ofkeys.validator_field_contents:validator_field_contents(d)isTRUE.invariant '<name>': each invariant returnsTRUE. A character return is the error message.
An invariant is a function(d) or a function(d, world). The class calls
an invariant with two or more formal arguments as f(d, world), and any
other invariant as f(d). $random(), $dataset() and $add_dataset()
give the invariants their world.
The directory dir holds the datasets of the table:
datasets.yaml: one entry for each recipe, withname,kind,n,seed,description,overrides,world_md5,rowsandmd5.nis empty forn = NULL.world_md5is absent forworld = NULL.rowsis the row count of the data.handmade/<name>.csv: one handmade dataset. Each column is read as text and converted to its field type. An empty cell isNA.profile.yaml: optional.synthetic_table_from_dir()reads it.$new()does not read it.
A dataset name is lower snake case. The name default is the generator
output at seed = 1 and n = 1000 or n = NULL.
The md5 of a recipe comes from tools::md5sum() of the data that
data.table::fwrite() writes with eol = "\n". The columns are in
field_types order, the rows are sorted by every column, dates are
ISO 8601 and NA is empty.
The md5 of a world comes from tools::md5sum() of the text that
yaml::as.yaml() gives for the world, written with LF line ends. The
names are sorted in C-locale order. Each value is text: a date or a
datetime in ISO 8601, any other value by as.character(). The order of
the values within a name counts. Every element of the world counts, also
an element that names no column of the table.
$dataset(name, world) regenerates a recipe with world. When the md5 of
world equals world_md5, it checks the data against the recipe. It
stops with an error that names the dataset when the row count differs
from rows, or the md5 differs from md5. For any other world, the
recipe is validated, its md5 is not checked, and a message says so. Two
NULL worlds are equal. A recipe without rows has no row check.
Active bindings
nameThe name of the table.
field_typesThe field types of the table.
keysThe key columns of the table.
dirThe directory of the datasets, or
NULL.
Methods
SyntheticTable$new()
Create a new SyntheticTable.
Usage
SyntheticTable$new(
name,
field_types,
keys = NULL,
validator_field_contents = validator_field_contents_blank,
invariants = list(),
profile = NULL,
generator = NULL,
dir = NULL
)Arguments
nameThe name of the table.
field_typesA named character vector of csdb field types.
keysA character vector of key column names, or
NULL.validator_field_contentsA function of a data.table that returns
TRUEfor valid contents.invariantsA named list of
function(d)orfunction(d, world). Each returnsTRUE, or a message that says whydfails.profileA
csdb_profile, orNULL.generatorA
function(profile, n, seed, world)that returns a data.table, orNULLforsimulate_from_profile().dirThe directory of the datasets, or
NULLfor no datasets.
SyntheticTable$random()
Generate and validate new data.
SyntheticTable$dataset()
Get a fixed dataset by name.
SyntheticTable$add_dataset()
Add a recipe, its row count and its md5 to datasets.yaml.
Usage
SyntheticTable$add_dataset(
name,
n,
seed,
description,
overrides = list(),
world = NULL
)Arguments
nameThe name of the dataset, lower snake case and new.
nThe number of rows, or
NULLfor the generator's own size.seedThe seed.
descriptionOne string that says what the dataset is for.
overridesA named list that maps a column to one fixed value. The value replaces the column after generation.
worldNULL, or a list that the generator receives. The recipe stores its md5 asworld_md5.
SyntheticTable$check_hashes()
Regenerate each recipe that world made, and compare its row count
and its md5.
