Wipro Ltd
|Senior Software Engineer
Hyderabad, Telangana, India
Summary
Currently leading the development and optimization of critical data pipelines and business intelligence solutions within the Admin Review System (ARS) for a major financial client, enhancing operational oversight and regulatory compliance.
Highlights
Orchestrated the complete SSIS lifecycle, developing, deploying, and executing packages across all environments to streamline data integration for the Admin Review System.
Monitored pipeline execution and managed run history to track status and performance, ensuring reliable data flow for critical administrative processes.
Developed Spark applications using PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats, uncovering key customer usage patterns.
Transformed data through Azure Data Bricks notebooks and integrated them as activities for robust data factory workflows, improving data processing efficiency.
Developed PySpark scripts, created clusters, and managed jobs in Azure Databricks to facilitate large-scale dataset processing and manipulation.
Utilized Azure Data Factory for efficient data migration from On-Premise systems to Azure SQL Data Warehouse, ensuring seamless data availability.