5.Loading Data into cloud warehouse
Loading data into a cloud warehouse is the final step of the ELT process. This typically involves a two-step process: first, uploading the data to a cloud object storage service, and then using a tool or command within the warehouse to load it.
Step 1: Upload Data to Cloud Storage
Before loading data into a warehouse, it is typically staged in a cloud object storage service like Oracle Object Storage or Azure Data Lake Storage .
🡆Using a GUI: You can use the cloud provider’s web console to manually drag and drop files from your local computer into a storage “bucket” .
🡆Using Command-Line Tools: For automated or large-scale uploads, command-line tools can be used to transfer files to the object storage bucket .
Step 2: Load Data from Object Storage into the Warehouse
Once the data is in object storage, you use the data warehouse’s native capabilities to load it into a table .
🡆Using a Loading Tool: Many cloud data warehouses have a specific data loading tool or package. For example, Oracle Autonomous Data Warehouse uses the DBMS_CLOUD PL/SQL package to load data from object storage .
🡆From an External Table: In some systems like Azure Synapse, you can create an external table that points to the files in your object storage. You can then use a SQL command to copy the data from the external table into a native table within the warehouse .
🡆Using Automation Tasks: Data integration tools can be configured to automatically pull data from your object store on a schedule, as specified in an Oracle tutorial .