Senior Data Engineer-Big Data

Details of the offer

What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment! This is creating many opportunities for people like YOU — people with an entrepreneurial spirit who want to make a difference in this world. At Cognizant, together with your colleagues from all around the world, you will collaborate on creating solutions for the world's leading companies and help them become more flexible, more innovative, and successful. Moreover, this is your chance to be part of the success story. Position Summary: Design End-to-end architecture and design solutions using one or more platforms for large-scale high-throughput applications. Design and build data ingestion jobs for ingesting data from multiple heterogeneous sources, APIs and processing using Pyspark, Amazon Document DB and R. Analyze the data patterns, apply data cleansing and modernization using Amazon Bedrock, code whisperer, AWS Lambda, Amazon Athena and build a lake warehouse, cloud storages. Using Flume agent as target for API and store data to Kafka topic and process using Spark Streaming. Use Google Api for API management, Build Data pipeline for Data ingestion from API real time and batch sources. Build enterprise data lake to store raw and processed data for analysis and decision making. Use Azure data factory, Azure synapse, Azure Databricks for data pipeline and process and store data on Cloud. Use Kafka Event hub to process real time data. Use HBase for NoSQL data storage, Use Hive for Meta data management and building data lake in Hadoop. Use Oozie, Airflow for Orchestration of data pipelines and Spark Shell jobs. Experience of creating dev-ops CICD pipelines with Bamboo and Jenkins. End to End project execution, Requirement gathering, transforming legacy design to Bigdata eco system. Development, Testing, UAT Support and GO Live support. Understanding and developing PowerBI and Tableau reports would be an additional advantage. Mandatory Skills: Must have minimum 6+ years of experience in Big Data/AWS/Azure. Strong knowledge and expertise in digital transformation implementations, Data warehousing, ETL and Big Data. Data Lake building, Spark, Amazon Document DB, NoSQL database, Airflow, Oozie, Kafka, Scala, Pyspark, Azure Cloud, Azure EventHub, Azure Data Factory, Azure function, Azure Synapse, Azure functions, Azure HDInsight, Billing & Revenue Management Devices. AWS Bedrock (image), Amazon Code Whisperer, AWS Open Search, Amazon Comprehend Medical, Amazon Polly, AWS Lambda, Amazon Comprehend, Node js, Amazon Athena, AWS Bedrock (text), Amazon Neptune, Amazon's Alexa and Lex, AWS Cloud Compute, AWS RDS, Amazon Sage maker, AWS Kendra. Azure DevOps, JIRA, GIT, Bitbucket, Bamboo, Jenkins, Confluence. Duties and Responsibilities: Responsible for architecting end to end data processing for the Billing and revenue management platform. Design and implement technical governance process as appropriate across the application development lifecycle coding guidelines, quality standards, best practices, code acceptance criteria, automation. Create Implementation strategy documents, which is an IT strategy approach for creation of customized software solutions requiring the application of advanced and complex IT theories and methods. Build end to end Big Data solutions using Cognizant's Big data accelerators to cover all possible business scenarios. Research, evaluate and deploy new tools frameworks and patterns to build sustainable Big Data platform. Identify gaps and opportunities for improvement of existing solutions. Responsible for migrating and transforming data from on-prem to on-cloud. Building CICD pipelines for automation. Participate in business requirements and functional requirements review meetings. Define and develop APIs for integration of various data sources. Coordinate business and functional acceptance testing which involves ensuring technical specifications meet business requirements. Responsible for daily and weekly status reports to update business teams and project managers. Salary Range: $100,000 Date of posting: 24/10/2024 Next Steps: If you feel this opportunity suits you, or Cognizant is the type of organization you would like to join, we want to have a conversation with you! Please apply directly with us. For a complete list of open opportunities with Cognizant, visit http://www.cognizant.com/careers. Cognizant is committed to providing Equal Employment Opportunities. Successful candidates will be required to undergo a background check. About Cognizant: Cognizant (Nasdaq-100: CTSH) is one of the world's leading professional services companies, transforming clients' business, operating and technology models for the digital era. Our unique industry-based, consultative approach helps clients envision, build and run more innovative and efficient businesses. Headquartered in the U.S. Learn how Cognizant helps clients lead with digital at www.cognizant.com or follow us @Cognizant. #LI-CTSAPAC
#J-18808-Ljbffr


Nominal Salary: To be agreed

Source: Whatjobs_Ppc

Requirements

Tprm Governance & Oversight Senior Specialist

About Us At ANZ, we're applying new ways technology and data can be harnessed as we work towards a common goal: to improve the financial wellbeing and sustai...


Anz - Victoria

Published 6 days ago

Senior Salesforce Developer Melbourne, Aus 5 - 10 Years Of Experience 15-03-2024

Melbourne, Australia 5 – 10 years  of experience 2024-03-15 CRMIT Solutions is a pioneer CRM Solutions Provider focusing on Salesforce digital transformatio...


CRMIT Solutions - Victoria

Published 6 days ago

Lead Developer-Java

What makes Cognizant a unique place to work? The combination of rapid growth and an international and innovative environment! This is creating many opportuni...


Cognizant - Victoria

Published 6 days ago

Data Engineer

Take the next step. Be part of something bigger at Suncorp. We're never just satisfied with how things are – because we know how things could be. And it's ou...


Suncorp - Victoria

Published 6 days ago

Built at: 2024-11-22T21:44:26.929Z