Data engineer is expected to lead data analysis project including understanding business requirement, engineering analysis features, building models, evaluating results, deploying models, developing relative algorithms for specific purpose and communicating with relative data stakeholders.
This position will have part of work relative to data engineering and will be responsible for maintaining/enlarging distributed systems, constructing reliable and high quality data pipelines, using web framework to properly serve data and collaborate with other data scientists to move project into production. Last but not the least, investigating new technologies and study new knowledge that would strengthen company business is essential.
1. Work with Hadoop, Hive, Impala and HDFS access.
2. Batch data processing and real-time data processing with Python, or java script or Spark.
3. Implement web service and back end database access.
4. Data visualization and web/data crawler development and maintenance.
5. Apply machine learning algorithm in a practical way.