Over the course of 4 months, I worked with the Data and Insights team to build data pipelines and scripts to streamline data access for Capital Market teams across London, Singapore, New York, and Toronto offices. In addition, Leveraged ETL and Object-Oriented principles to ensure good coding standards were adhered to when building a documention automation
Position
Data Engineer Intern
Team
Data Insights and Innovation
Tools Used
Python (Pandas), Alteryx
Employment Type
Contract Full-Time
Location
Toronto, CA
Develop a data pipeline for the Capital Market's team to use when conducting Securities analysis.
I had four-months to deliver a fully function data pipeline that would be realised to production for the Capital Market's team to leverage on for analysis of securities.
Over the course 4-months I would have the opportuity to solidfy my understanding of software languages such as Python, Big Data transformation platforms such as Alteryx and Python scripting. My goal was to Develop a data pipeline that extracts data from mulitple databases and applies custom formulas to select columns and generates a file for the trade team.
During this time I had to implement various solutions using the three technologys listed below. I had soc much fun gaining a deeper practical understanding of each the technology's mentioned below
Industry standard programming languge in perfomring data analytics, ETL and data visualizations
Data Transformation software that leverages a low-code platform to aid in ETL of data from different databases
Publishing custome RPA's for the wider Data and Insights to use and leverage in their workflow documentation process
I was able to successfully develope a data pipeline to extract up to 40,000 records of currency rate data and transform it into 30 columns with accurate formula application using Python-Pandas. In addition I was able to create a script that saw a time savings on documenting workflow automation. I leveraged on Object-Oriented programming principlesto ensure the script applied to different team templates.
Overall this was a great learning experience I learned how to develop a pipeline, how to contribute during design and analysis meetings. development and how to deliver technical presentations to my team. Also, it was great to be part of such a large coorporation and see how my teams work contributed to the grander vision.
Looking Forward
With this experience, I'm looking forward to applying what I learned in future jobs and personal projects!
Back Home