Skip to main content

Rebuilding Aggregates Using the REST API

You can use the aggregates-batch endpoint of the AtScale engine REST API to trigger full or incremental builds for all aggregates of a deployed model. When running an incremental build, you can optionally specify grace period overrides for specific datasets in the model.

One use case for this feature is the need to rebuild aggregates after ETL (Extract, Transform, Load) jobs. Many companies have ETL jobs that they routinely run to cleanse raw data and load it into the data warehouse. You can use AtScale's REST API to programmatically rebuild aggregates for a model as a part of your ETL processing.

To facilitate running builds via API, AtScale provides the full cURL command for each of your deployed models. The command is prepopulated with the catalog and model IDs; you simply provide your API token and any grace period overrides you want to set.

The following sections describe how to use the aggregates-batch endpoint to run different types of aggregate builds.

Trigger a full aggregate build​

You can use aggregates-batch to trigger a full aggregate build for a specific model.

Prerequisites​

Before triggering an aggregate build, ensure that:

  • You have the aggregates_manage role assigned in the Identity Broker. For more information, see Identity Broker Default Roles.
  • The model you want to run builds for has been deployed.

Procedure​

  1. On the Deployed Catalogs panel, open the catalog you want to run a build for, then select the Build tab.

  2. Hover over the model and click Copy cURL.

    image

  3. Paste the command into your command prompt and edit the following lines:

    • curl -L 'https://<host>/v1/aggregates-batch/catalogs/<catalog_ID>/models/<model_ID>?isFullBuild=false' \: Change isFullBuild=false to isFullBuild=true to run a full build.
    • -H 'Authorization: Bearer <TOKEN>' \: Replace <TOKEN> with your API token. This can be copied from the Account panel in Design Center.

  4. Run the command.

Trigger an incremental aggregate build​

You can also use the API to trigger incremental aggregate builds for specific datasets within a model. This enables you to run ad-hoc incremental builds without having to modify the datasets’ configured grace periods.

Prerequisites​

Before triggering an incremental aggregate build via API, ensure that:

  • You have the aggregates_manage role assigned in the Identity Broker. For more information, see Identity Broker Default Roles.

  • Grace period overrides are enabled at the global level, as well as on the model you want to run aggregate builds for:

    • Set the aggregate.batch.gracePeriodOverrides.enabled global setting to true. This setting is disabled by default. You do not need to restart the engine after changing it. For more information, see Aggregate Settings.
    • Set the aggregate.batch.cube.gracePeriodOverrides.enabled setting to true on the models you want to run incremental builds for. This setting is disabled by default. For more information, see Aggregate Settings.

  • Incremental builds are enabled on the datasets you want to run aggregate builds for. For more information, see Setting Properties to Allow Incremental Rebuilds of Aggregates.

  • The models you want to run builds for have been deployed.

Important

Enabling this functionality can potentially cause your system to become strained by expensive aggregate rebuilds.

Procedure​

  1. On the Deployed Catalogs panel, open the catalog you want to run a build for, then select the Build tab.

  2. Hover over the model and click Copy cURL.

    image

  3. Paste the command into your command prompt and edit the following lines:

    • -H 'Authorization: Bearer <TOKEN>' \: Replace <TOKEN> with your API token. This can be copied from the Account panel in Design Center.

    • "gracePeriodOverrides": {}: Optionally, specify any datasets you want to provide grace period overrides for. For example:

      "gracePeriodOverrides": {
      "factinternetsales": 30}

  4. Run the command.

Additional information​