Venugopal Reddy

Skills

Azure Data Factory, Azure Databricks, PySpark, Snowflake, SQL, ETL DataStage

About

• Senior Data Engineer with 11+ years of extensive experience in architecting, designing, and developing complex data solutions
• Experience in developing ETL solutions using Azure Data Factory (ADF), Azure Databricks, PySpark, Snowflake and ETL DataStage
• Having hands on experience in developing ETL process to extract, transform and load data from various sources.
• Hands on experience to implement Data Warehousing, Integration and Enhancement projects.
• Experience in creating pipelines, data flows and triggers in ADF.
• Experience in ADF to debug, validate, run and monitor ADF pipelines.
• Experience in Azure Databricks for data extraction, transformation and loading.
• Extensive working knowledge on technologies such as UNIX, Oracle, DB2 and Snowflake.
• Efficient in incorporation of various data sources such as Flat files, XML and Tables.
• Managed software project life cycle (Requirement analysis, Functional/Design Specifications, HLD, LLD & Test plans designing)
• Extensive working knowledge on tools like PVCS, Confluence, Jira and Control-M.
• Extensive working experience on ETL DataStage designer and director.
• Have knowledge on data warehouse concepts and data models like star schema, snowflake schema
• Experience on CICD process through DevOps, GitHub, Jenkins, Nexus and G3-RTC
• Team Worker with good interpersonal, communication and fast learning skills
• Have working experience on AWS services EMR, EC2, Lambda, Athena, S3, CloudWatch, SNS, SQS and PySpark in Jupiter notebooks.