Department: Data Engineering
Project Location(s): Bangalore, Pune, Chennai
Education: Any Degree
-
- 4+ years of experience designing and developing big data processing systems using distributed computing.
- Fluency in Python, Experience in Scala/Java is a huge plus
- Expert knowledge in optimizing complex SQL queries An affinity for automation
- Experience working with cloud platforms such as AWS ; GCP Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs
- Experience with at least one object oriented language (ex: Java)
- Strong OO design, data structure, and algorithm design skills
- Strong interest in emerging technologies: Spark, Hadoop, Hive, ElasticSearch, NoSQL
- Experience with data science / modelling (TensorFlow and/or MXNet)
- Familiarity with API development and design.
- Amazon Web Services (S3, EMR, Athena, Glue, Kinesis, Lambda)
- Google Cloud Platform (GCS, Dataflow, BigQuery)
- Python, R, Spark (PySpark)
- Airflow,Tableau.
- Design, implement and maintain high performance big data infrastructure/systems, big data processing pipelines scaling to billions of structured and unstructured events daily.
- Design, implement and maintain deep integration with up-stream systems, Monitor performance of the data platform and optimize as needed
- Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
- Support product with the overall roadmap and ensure updates to senior leadership are 100% technically correct.
- Data analysis, understanding of business requirements and translation into logical pipelines & processes
- Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)
- Evaluate and prototype new technologies in the area of data processing
- Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
- High energy level, strong team player and good work ethic