Discussion around current project and challenges around it. How would you scale the current infrastructure? System design related questions. Write a program to find x number of max elements from m number of sorted arrays. General discussion around your project. Write spark code to find the unique routes from a given flight routes table where there could be individual entry for MumBai --> Kolkata/Kolkata--> Mumbai Write a spark scala code to get the employee who earns highest among employess whose salary is more than avg salary and age is less than the avg age. What is master in Spark? What does appName do in Spark? How the local scheduler work in Apache Airflow? How it differs from Kubernetes? Questions around airflow architecture. Given two programs for word count tell the difference between reduceBykey and groupbyKey?
Senior Data Engineer Interview Questions
2,565 senior data engineer interview questions shared by candidates
- questions on terminal operations in spark - questions on message comat. in Kafka
1. Explain the difference between SCD Type1, SCD Type2, SCD Type3
what's you expected salary?
Third pat - payroll, Azure services, Databricks, optimizations
Explain Hbase minor and major compaction
They asked me real-time scenario questions related to my work experience and how I handle production issues.
SQL Query Writing: **Table A** Columns: col1 col2 **Table B** Columns: col3 col4 There is no common column to implement the JOIN condition and the Required Output is: col1, col2, col3, col4
SQL Query Writing: **Table EMP** Input Columns: EMPID, DEPID, SAL, DOJ Output Columns: EMPID, DEPID, SAL, DOJ, NEXT_SAL NEXT_SAL can be derived as the salary of immediate next employee in the given records.
Which AWS services you have used (Hands-On)? Tell me one use case and one limitation of AWS Lambda function?
Viewing 701 - 710 interview questions