Data Engineer 大数据开发工程师

该职位来源于猎聘

• Expert experience with mainstream RDBMS like MySQL(mandatory), PostgreSQL, Oracle and NoSQL such as Redis, HBase;
• Familiar with the distributed database architecture, ability to design the OLTP with high availability, scalability and performance; solid hands-on experience with distributed DB products in mainstream Cloud environment like AliCloud DRDS or GCP spanner;
• Extensive experience in big data acquisition, pre-processing, storage, and cleansing technologies including Hadoop (HDFS, MapReduce, Yarn,..), Kafka, Sqoop, Spark, Flink and etc;
• Experience with programming languages: Java, Python or Shell;
• Knowledge of Graph Database like Neo4j is a plus Responsibility:
• Responsible for designing, building, testing and maintaining data systems, making sure the data structure, ingestion and processing systems, supporting digital product teams’ requirements and user needs;
• Responsible for implementing methods to improve data reliability, efficiency and quality;
• Act as technical expert in regard to integration and connectivity methods, protocols,
• security and practices for data management systems;
• Responsible for designing and creating the optimal data pipeline;
• Work closely with data architects/scientists and other stakeholders on future data handling technologies, including input on risks, costs, benefits