Data Engineer+Python+SQL // Remote // USC-GC

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Apton Inc, is seeking the following. Apply via Dice today! USC/H1(I140 Approved) Need strong candidate with Data Migration, Data Lake ,Data Bricks ,Python and SQL coding Job Description 12+ years of proven experience in Data Engineering, worked on designing and developing software with Big Data, Data Lake/ Lake House ecosystem, Data Analytics, backend microservices architecture, and heterogeneous data types at scale Proven in-depth experience in creating ELT/integrations pipelines using Databricks, Spark, Python, SQL, Scala, Kafka, Presto, Parquet, Streaming, events, bots, AWS/cloud ecosystem. Proficient in developing Micro Services and using AWS frameworks such as SQS, Stream, Kubernetes, EC2, S3, Lambda etc. Experience with data pipelines/analysis/visualization tooling such as Elastic stack, Logstash, Kibana, Kafka, Grafana, Splunk, Pandas, Message brokers, Data modeling. Expertise in Data Lakehouse architecture and end-to-end Databricks techniques. Have worked on connecting Data Lake to SAP Ecosystem, aware on technological ecosystem on how to onboard data from SAP ECC application layer and ingest back to SAP via application layer Have designed and built PB sized scalable data lake and structured/unstructured data query interfaces and microservices to ingest, index, mine, transform, and compose large datasets. Worked on end-to-end data lifecycle from Data Ingestion, Data Transformation, and Data Consumption layer. Strong experience building Data Lake-led APIs and its usability for consuming large-scale data and ingesting real-time ecosystems. Expert in Spark, Parquet, steaming, events, Kafka, telemetry, MapReduce, Hadoop, Hive, Presto, Spark, data query approaches, and dashboarding. The one who have implemented Enterprise use cases like CMDB, Governance, time series classification, telemetry anomaly detection, logs, and real-time data ingestion through APIs. Experience with structured data such as Avro, Parquet, Protobuf, Thrift, and concepts like schema evolution. Enable end-to-end monitoring pipeline for associated applications and ingest to Datalike, build metrics log monitoring. End to end data pipeline monitoring for Data Platform and Data Monitoring for data integrity and meet the business agreed SLA Work on multi-die project & associated deliverables for Observability enabling lifecycle of applications, leveraging these use cases enable scalable self-service data pipeline. Build self-service data ingestion, self-service consumption through API, enable data streaming to Data Lake, services for DB replication from other DBs to Data Lake, Streaming Pipeline using cloud events while ensuring Data Parsing metadata extraction, e.g., for Stream, File etc. Building analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. Apply tot his job Apply tot his job