Commands

The three tools within this toolkit (lookml, dbt-test and dbml) each have a sub-command. These commands must be predicated by the droughty command.

lookml

Generates lookml based of the analytics layer of your warehouse, run:

droughty lookml

in your terminal.

This will output a file within the root of your active git repo as follows:

/lookml/base...

This file will contain explores and measures based on the multiple layers in your warehouse, defined by the targets within your profile.yaml file. It assigns dimension datatypes, descriptions (if available). It also assigns pk’s, hides pk(s) and fk’s by default.

dbt

run:

droughty dbt

Generates standard schema tests, currently coverage is:

unicity of pk's
at least one value in all columns
valid_from <= valid_to

It will output a schema.yml file in the follow dir at the root of your active repo:

/models/schema.yml

The target for your test schemas is defined in your profiles.yaml file within the /.droughty dir. Find an example below:

test_schemas:

  - example_dev_staging

  - example_dev_integration

  - example_analytics_dev

dbml

Generates dbml based on the analytics layer within your warehouse, run:

droughty dbml

It will output the dbml model into the follow dir:

/db_docs/....

As with lookml, this assigns data types and also identifies pk, fk references and assigns them

cube

Generates cube based on the analytics layer within your warehouse, run:

droughty cube

It will output the cube models into the follow dir:

/schema/....

As with lookml, this assigns data types and also identifies pk, fk references and assigns them

docs

Generates inferred warehouse descriptions using openai, run:

droughty docs

By default, it will output the inferred metadata into:

/warehouse_docs/openai_field_descriptions.md

Droughty docs will read from your field_descriptions.md file and only create descriptions that are absent for entities in the warehouse.

Openai isn’t prefect so make sure you sanity check the output

resolve

Uses pandas_dudupe which is derived from depupe.io to resolve warehouse entities:

droughty resolve

Once you have added the required parameters available within the configuration documentation, you can use the cli to resolve warehouse entities and output them to either big query or snowflake

arguments

  • –profiles-dir - This allows you to specify the profile directory if you don’t want to use the default

  • –project-dir - This allows you to specify the project directory if you don’t want to use the default

  • –env-vars (enabled) - This allows you to enable the use of environment variables