Filter Data

Filter Data #

You may want to filter the Extracted data to fit your needs. You can do this using the Filter Transformation. Additionally, database sources allow you to Extract data using a SQL query to filter data. Furthermore, the Fusion Registry 10 source allows to filter data on Extraction using Dataflow filters and Start/End time period.

In this example, we will Extract data from an Excel file, filter the data and transfer it to a CSV file. Before beginning the work, make sure that there are Predefined sources and targets present in ETL data_snake.

  1. Select a name for your Workflow and select the Step it will belong to.

Optionally, you can add a description to your Workflow by clicking the button and selecting Edit description.

In a new window, simply add the description in the field and click the Save button.

  1. Drag the Source Component to the Workspace and select the desired source from the list. Remember to configure the Source Node.

  1. Because all columns in data Extracted from the Excel Source will be of String type, first you need to Convert some column data to correct types. Therefore, drag the Convert Transformation to the Workspace and connect it with the Source Node.

  1. Configure the Convert Node and cast columns to desired values.

  1. Drag the Filter Transformation to the Workspace and connect it with the Convert Node.

In this example, the Excel file has data for each month of the year 2017 and we only want data from the first quarter.
  1. In the Configuration window of the Filter Transformation click the Add filter button.

  1. Select the desired column.

  1. Select the condition for the filter.

  1. Fill out the desired value.

  1. Drag the Target Component to the Workspace and select the desired target from the list. Remember to configure the Target Node.

  1. After configuring the Target Component, connect the Filter Node with it.

  1. Validate the finished Workflow by clicking the Validate button.

  1. If there are no validation errors, click the Save button (marked red) and then close the Modeler, by clicking the Exit button (circled red) icon.