Purpose

Key Responsibility

• Perform IT system architecture design, new technology research, and provide recommendation.
• Design and implement optimal data pipeline architecture (considered high data volume, data governance, etc.).
• Work with PRODUCT/BIZ teams to assist with new data platform re-engineering or data-related technical issues.
• Build and implement the features and services of an intelligent device control platform on Kubernetes (K8S) environment
• Build multiple data connectivity capability
• Perform cloud native microservices design and build service library for workflow pipeline integration.
• Design solution architecture for non-stop services upgrade and operation
• Implement service message exchange with new technology
• Build and implement the features and services of Big Data analytic platform on high availability NoSQL DB (e.g.Cassandra, HDFS, S3/MinIO, MariaDB, MongoDB etc.) on K8s environment.
• Build small data lake initially then enlarge for company data lake
• Build small image/video lake initially then enlarge

Qualification

• B. E in Computer Science, Electrical Engineering, Mechanical Engineering or related fields (Information Engineering, Information Management, Data engineering, Applied Mathematics, Robot/Automation.

Experience

• Experience in container and cloud-native solution (K8S/Docker, Istio, ArgoCD, MongoDB, Redis, …etc)
• Experience in microservice implementation, conductor workflow implementation, Kafka queue implementation on K8S environment
• Experience in tool connection to IT systems (IoT, Image/Voice, SECS/GEM, PLC/DCS, add-on sensors... etc)
• Experience in big data ecosystem (Hadoop, Spark, Cassendra...etc) and SQL NoSQL database (MariaDB, MongoDB, MinIO(S3)...etc).
• Experience in data orchestration among various data sources, data parallel processing, metadata data management, data quality assurance, data security and governance, and new data technology evaluation and enablement.
• Knowledge of object stores for image and semi/unstructured data.
• Experience in big data solutions development, data pipelines for batch and streaming applications.
• Experience in monitoring and analysis tools, such as Nagios, ELK, Splunk, and Grafana.
• Ability to design solutions independently based on high-level architecture.
• Cloud native deployment practices, specifically, containers and container orchestration platforms.

Additional Technical Requirements

• Experience in Hadoop ecosystem and SQL NoSQL databases.
• Experience in TypeScript/JavaScript, Kotlin, Java, F#, C/C++/C#, Rust, SQL or other relevant programming languages
• Big Data Solutions development, data pipelines for batch and streaming applications.

Competencies