Skip to main content

Subset a dump

caution

Only PostgreSQL supports Subsetting at the moment. Feel free to contribute to accelerate the support of MySQL and MongoDB

Subsetting is a powerful feature to only import a smaller consistent part from your production database.

How Subsetting works

Check out how subsetting works under the hood here.

Configuration

Using Subsetting feature is as simple as adding new parameters in your conf.yaml

add database_subset object
source:
connection_uri: postgres://user:password@host:port/db
transformers:
- database: public
table: customers
columns:
- name: first_name
transformer_name: first-name
- name: last_name
transformer_name: random
- name: contact_phone
transformer_name: phone-number
- name: contact_email
transformer_name: email
database_subset:
database: public
table: customers
strategy_name: random
strategy_options:
percent: 10
passthrough_tables:
- product_catalog

By applying this configuration, Replibyte will:

  • Keep around 10% of the full database
  • Go down the whole tables linked to public.customers
  • Keep the whole rows from product_catalog

Subset Strategy

TODO

Considerations

This feature is still under active improvement. Feel free to open an issue if you face any trouble.