2.3.Matillion Environments setup steps
Step 1: Environment Setup
1. Log in to your Matillion instance.
2. Navigate to ‘Project’ → ‘Manage Environments’.
3. Click ‘Add Environment’
4. Enter a name for your environment.
5. Select your cloud data warehouse (e.g., Amazon Redshift, Snowflake, BigQuery).
6. Provide connection details such as host, port, database name, username, and password..
7. Click ‘Test’ to verify the connection
8. Click ‘OK’ to save the environment
Step 2: Create an Orchestration Job
1. Go to the ‘Project’ menu and select ‘New Job’.
2. Name your job (e.g., ‘LoadAirportData’) and choose ‘Orchestration’ as the job type.
3. Drag and drop components from the Components panel to the canvas:
— ‘S3 Load’ to load data from Amazon S3
— Configure the S3 Load component with bucket name, file path, and target table.
4. Link the components using arrows to define the workflow.
– ‘SQL Script’ to run SQL commands if needed.
5.Save the job and click ‘Run’ to execute it.
Step 3: Create a Transformation Job
1. Create a new job and select ‘Transformation’ as the job type.
2. Drag and drop components such as:
– ‘Table Input’ to read data from a table.
– ‘Filter’ to apply conditions.
— ‘Join’ to combine data from multiple sources.
— ‘Calculator’ to create new columns.
— ‘Table Output’ to write the transformed data to a new table.
3. Configure each component with appropriate settings.
4. Link the components to define the data flow.
5. Save and run the job to perform the transformation.
Step 4: Document and Link Jobs
1. Use the ‘Comment’ component to add notes and documentation.
2. Use the ‘Run Transformation’ component in orchestration jobs to trigger transformation jobs.
3. Organize jobs into folders for better management.
4.Use version control features to track changes.