Security enforcement

[akademic]

Introduced features are great!

But database structure evolves and you can't just setup filters once and live in peace.

One day someone will add new table with secrets and theese secrets will go to staging without obfuscation.

It will be great to have option to skip all tables not added to config.
Also this option will obfuscate all undescribed columns.

This change will force to update nxs-data-anonymizer config every time we have changed database structure (added column, created table).
This will provide "secure by default" mode.

[borisershov]

You mean something like this (example):

global:
  filters:
    columns:
      api:
        value: "{{- randAlphaNum 50 | nospace | lower -}}"
filters:
  groups:
    columns:
      title:
        value: "{{- randAlphaNum 50 | nospace | lower -}}"

Globally a filters will be applied for column api for every table within the database. And for column title only for database groups.

?

[akademic]

Something like:

global:
   skip_tables_without_filters: true
   randomize_columns_without_filters: true

filters:
  groups:
    columns:
      id:
        passthrough: true
      api:
        value: "{{- randAlphaNum 50 | nospace | lower -}}"
      // other columns if exist will be randomized with respect to types: int -> random int, string ->random string

[akademic]

In product you referred to they have syntax:

filters
  only:

https://github.com/datanymizer/datanymizer#tables-filter

But I have not found option for default hiding undescribed in filters columns

[borisershov]

Ok, I see. Let me think about this a couple of days )

[escalate]

This feature would be very helpful.

[borisershov]

Hi everyone! I've summarized your suggestions. Please check the config options described below (draft). Does it solves the issue?

Global settings

Option	Type	Required	Default value	Description
security	Security	No	-	Security enforcement for anonymizer

Security settings

Option	Type	Required	Default value	Description
policy	Policy	No	-	Security policy for entities
exceptions	Exceptions	No	-	Exceptions for entities
defaults	Defaults	No	-	Default filters for entities

Policy settings

Option	Type	Required	Default value	Description
tables	String	No	-	Default security policy for tables. If value skip is used all undescribed tables in config will be skipped while anonymization
columns	String	No	-	Default security policy for columns. If value randomize is used all undescribed columns in config will be randomized (with respect to types) while anonymization

Exceptions settings

Option	Type	Required	Default value	Description
tables	Slice of strings	No	-	Table names without filters which are not be skipped while anonymization if option security.policy.tables set to skip
columns	Slice of strings	No	-	Column names (in any table) without filters which are not be randomized while anonymization if option security.policy.columns set to randomize

Defaults settings

Option	Type	Required	Default value	Description
columns	Map of Filters	No	-	Default filter for columns (in any table). That filters will be applied for columns with this names without described filters

Example of config:

security:
  policy:
    tables: skip
    columns: randomize
  exceptions:
    tables:
    - goods
    columns:
    - id
  defaults:
    columns:
      password:
        value: "{{- randAlphaNum 50 | nospace | lower -}}"

filters:
  ...

[escalate]

Hello @borisershov ,

I don't understand that point.

"Default security policy for tables. If value skip is used all undescribed tables in config will be skipped while anonymization"

Does "skipped" mean that the table is preserved or does "skipped" mean that the table is excluded and is not part of the output?

Best regards
Felix

[borisershov]

Hi, @escalate!

Does "skipped" mean that the table is preserved or does "skipped" mean that the table is excluded and is not part of the output?

The second, i.e. the tables will be dropped from a resulting anonymized dump.

A logic of a new security section is following:

• For tables that have any rules in a filters section:
  • For columns that have any rules in filters section: will be anonymized in appropriate with specified rules
  • For columns without any rules in filters section:
    • If column name specified in security.exceptions.columns: passed without any changes
    • If not:
      • If column name specified in security.defaults.columns: will be anonymized in appropriate with specified rules
      • For all other columns: randomize its values with respect to types
• For tables without any rules in filters section:
  • If table name specified in security.exceptions.tables: passed without any changes
  • If not: dropped from a resulting anonymized dump (table is excluded and is not part of the output)

[borisershov]

Does it that you are expect from anonymizer security enforcement?

[tsyganov-ivan]

Hi!
We also interested in this feature.
Latest logic looks brilliant for our case.

[borisershov]

Hi everyone! Features described above has released in v1.5.0. At the time new abilities available only for MySQL, Percona, and MariaDB (for PgSQL will be added in next release)

[borisershov]

Hi all! Requested abilities released in v1.6.0