Extract-Transform-Load (ETL)
Extract-Transform-Load (ETL)

Extract-Transform-Load (ETL)

Blockchain sounds like the technology to get IT sceptics who have trust issues to finally get onboard with the adoption of technologies.  This is crucial for the metaverse as we can’t physically see or interact with those that we meet there.  We need to have a way to seamlessly prove that those animated and unanimated objects that we engage with in the metaverse are legitimate.  Blockchain seems to be the answer to solve this major problem.  However, this is not entirely true. 

 

Blockchain only guarantees the integrity of the data once it enters the chain.  Before this stage, more traditional processes and technologies are used for data entry.  This means that we still need to rely on 3rd parties to ensure that data integrity is upheld off the chain.  What are some of these key aspects for data integrity?

Regulations – We still need to define what are illegal activities when it comes to the application of blockchains and penalize those that commits fraud.

Business Processes – These processes governs how and when actors (human or computers) interact with the blockchain. It embodies the purpose of using the chain and the value that the chain brings.   

Data Validation – Before data enters the chain, they need to be validated against business and technical rules to ensure that the right kind and type of data are submitted.

Data Harmonization – Different systems that inputs data into the same blockchain might have different formats for representing the same type of data. For instance, a number can be in decimal (float) or integer formats.  The blockchain has the final say of what the data type and format should be for each data field. 

In general, the data cleansing processes of ETL apply to blockchain.  Without this, we basically will have GIGO (garbage in, garbage out). 

 

And now I chime in… 😊  My specialization in IT is business intelligence (BI) which deals a lot more with information management than say infrastructure maintenance and support.  One of the key processes of BI is ETL.  One of my experiences was to manage a Data Warehouse for General Motors Call Center division.  I was working at their world Headquarters in Detroit, Michigan, USA where we loaded data from 3 call center sites around the US into the data warehouse.  I had to create and maintain nightly batch jobs containing these ETL packages using Microsoft SQL Server. 

 

This experience is quite valuable for an educational institution that needs to understand how to transform all different types of data from all different types of medium (physical and digital) to IT systems for process automations and/or decision support.  For vendors, my experiences can quickly build close connections with all key stakeholders that needs to be covered in the IT division: from executives all the way down to the database administrators and application developers.  If you want me to talk code, I love to do it!  Also, if you wanna know more, Let’s Talk! 

Leave a Reply

Your email address will not be published. Required fields are marked *