Initial Interview: Cultural and general tech questions Tech interview: General Python technical questions CTO interview: Motivation, experience and professional interests questions
Senior Data Engineer Interview Questions
2,565 senior data engineer interview questions shared by candidates
A developer on the team was running the following line in a function for logging purposes, and the job crashed with an "out of memory" exception. The developer says that the cluster has many workers with a lot of memory and disk and still the job crashes. Can you help explain how come to this line makes the job crash with OOM even though the cluster is huge? def someFunc(): for row in df.collect(): print(f'Customerr{row["customer"]} => Paid {row["price"]}')
What feedback did your manager give you
Describe a project you are particularly proud of
How did you design data lake what are the three layers. How did you design using what services. why this services over the other Difference between schema in data warehouse Whats the purpose of this service. How you would store some secret keys and how you would secure the data. what are the methods and how you did it. whats the typical project activities dev,test,prod the cycle.
persist vs cache? what does init do? args vs kargs? json vs parquet? columnar storage? distributed systems? spark dataframe vs pandas dataframe
How do you manage conflict? How do you break down technical issues to non-technical audience? Have you implemented any Data Governance tools?
Spark and Scala, detecting bank fraud
What technologies have I worked with, how were my previous experiences, technical questions related to the stack used
I don't remember any specifically difficult questions or unexpected questions. I thought the interview was fair and the SQL test was fair based on the practice test provided.
Viewing 1851 - 1860 interview questions