1. Difference between HDFS and S3. 2. how to optimise Spark job. 3. Simple problem on array swapping. 4. Simple SQL Queries. 5. Very Very generic assignment. 6. Spark Structured Streaming (Stream join)
Sr Data Engineer Interview Questions
2,564 sr data engineer interview questions shared by candidates
Specific questions revolved around Spark's internal optimisations for SparkSQL and others.
General behavioral, resume, some technical concepts
- List some kinds of attacks on web applications
Domain Knowledge, Technical skills, Management related
- questions on terminal operations in spark - questions on message comat. in Kafka
System design a pipeline that ingests data about orders and matches to menu's for recommendations to customers.
How would you diagnose a query that takes too long to run?
It was a senior role. Most of discussion went around system design, practical Spark problems.
Discussion around current project and challenges around it. How would you scale the current infrastructure? System design related questions. Write a program to find x number of max elements from m number of sorted arrays. General discussion around your project. Write spark code to find the unique routes from a given flight routes table where there could be individual entry for MumBai --> Kolkata/Kolkata--> Mumbai Write a spark scala code to get the employee who earns highest among employess whose salary is more than avg salary and age is less than the avg age. What is master in Spark? What does appName do in Spark? How the local scheduler work in Apache Airflow? How it differs from Kubernetes? Questions around airflow architecture. Given two programs for word count tell the difference between reduceBykey and groupbyKey?
Viewing 671 - 680 interview questions
See Interview Questions for Similar Jobs
Data EngineeringSenior Data EngineerBig Data EngineerSenior Big Data EngineerData EngineerSenior Data ConsultantLead Data EngineerData ScientistsData Scientist IiiPrincipal Data EngineerData Mining EngineerBigdata EngineerSr Data AnalystEtl Data Stage DeveloperSr Database EngineerDatabase Administrator IiSenior Data Stage DeveloperEtl Informatica Developer