Targets

Targets #

The Targets Components are the last building blocks of a Modifier. While creating a Modifier, you need to provide a target to where the data will be loaded. Without a working Target, the Modifier will not be valid.

Predefined Targets can only be defined by the system administrator.

Oracle Database Target #

Component for loading data to an Oracle Database. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. DB Host - the address where the database is located (Predefined),
  3. Port - the port of the address (Predefined),
  4. Username - the username with access to the database (Predefined),
  5. Password - the password needed to access the database (Predefined),
  6. Service name - the alias of the database (use either Service name or SID) (Predefined),
  7. SID - the unique name of the database (use either Service name or SID) (Predefined),
  8. Table - the table to which we want to load data,
  9. Save mode - how to load data into the database:
    • Merge - updates existing data and inserts new data to the table,
    • Full replace - deletes table data and inserts only new data to the table,

Postgres Database Target #

Component for loading data to a Postgres Database. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. DB Host - the address where the database is located (Predefined),
  3. Port - the port of the address (Predefined),
  4. DB Name - name of the database (Predefined),
  5. Username - the username with access to the database (Predefined),
  6. Password - the password needed to access the database (Predefined),
  7. Table - the table to which we want to load data,
  8. Save mode - how to load data into the database:
    • Merge - updates existing data and inserts new data to the table,
    • Full replace - deletes table data and inserts only new data to the table,

MySQL Database Target #

Component for loading data to a MySQL Database. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. DB Host - the address where the database is located (Predefined),
  3. Port - the port of the address (Predefined),
  4. DB Name - name of the database (Predefined),
  5. Username - the username with access to the database (Predefined),
  6. Password - the password needed to access the database (Predefined),
  7. Table - the table to which we want to load data,
  8. Save mode - how to load data into the database:
    • Merge - updates existing data and inserts new data to the table,
    • Full replace - deletes table data and inserts only new data to the table,

Microsoft SQL Server Database Target #

Component for loading data to a Microsoft SQL Server Database. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. DB Host - the address where the database is located (Predefined),
  3. Port - the port of the address (Predefined),
  4. DB Name - name of the database (Predefined),
  5. Username - the username with access to the database (Predefined),
  6. Password - the password needed to access the database (Predefined),
  7. Table - the table to which we want to load data,
  8. Save mode - how to load data into the database:
    • Merge - updates existing data and inserts new data to the table,
    • Full replace - deletes table data and inserts only new data to the table,

FusionRegistry10 Target #

Component for loading data to a Fusion Registry Dataflow. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. Fusion URL - the URL address of the Fusion Registry instance (Predefined),
  3. Username - the username with access to Fusion Registry dataflows (Predefined),
  4. Password - the password needed to access Fusion Registry dataflows (Predefined),
  5. Mode - how to load data into the Dataflow:
    • Append - only insert new data to the Dataflow,
    • Replace - update existing data and insert new data to the Dataflow,
    • Delete - delete matching data from the Dataflow; the work of this mode depends on data in the Dataflow.

Script Target #

This Target can only be modified by the system administrator!

Component for loading data to any target. It is for advanced users to create a Python script that does not need to return any data. The input data tables are available from the INPUT_DATA list variable. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. Script code - the code to use for extracting data (Predefined),

Available Packages #

Here is a list of available packages to use in your Script:

  • all packages in the Python Standard Library,
  • the pandas manipulation tool,
  • the dateutil module for date and time manipulations,
  • the NumPy fundamental package for scientific computing with Python,
  • the OpenPyXL library to read/write Excel 2010 xlsx/xlsm/xltx/xltm files,
  • the pandaSDMX library for SDMX-ML data manipulation,
  • the xlrd library to read old xls Excel files,
  • the odfpy library to read and write OpenDocument files (i.e. ods),
  • the SQLAlchemy Python SQL toolkit and Object Relational Mapper,
  • the Requests library for sending HTTP requests,
  • the Beautiful Soup library for pulling data out of HTML and XML files,
  • the pyodbc module for accessing ODBC databases,
  • the wbgapi module for accessing the World Bank’s economic data,
  • the pandas-datareader library for fetching financial data into pandas DataFrames.
  • the html5lib pure-python library for parsing HTML.

File based targets #

The Targets below are uploaded to the Resources of ETL data_snake. If you wish to transfer the created files, you need to download them through the Resources section.

Excel Target #

Component for loading data to an Excel file (xlsx only). This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. Path - the filepath in the Resources folder (Predefined),
  3. Sheet - the name of the sheet for data to be loaded to,
  4. Start from - the column from which to start loading data to the sheet,
  5. Export columns names - whether to load column headers to the file,
  6. Protected file - whether to rewrite the file if it exists; enabling this will cause that no new data will be written to the existing file, even if to a different sheet.

CSV Target #

Component for loading data to a CSV file. This Target needs the following fields to be provided:

  1. Name - a name of the component on the list of Targets (Predefined),
  2. Path - the filepath in the Resources folder (Predefined),
  3. Delimiter - symbol used to divide the values in the CSV file (Predefined),
  4. Protected file - whether to rewrite the file if it exists.