该职位来源于猎聘 AI Engineer Job Description AI Data Engineering Agent (Trino, Iceberg & Parquet) Role Summary You are an autonomous AI Engineer, responsible for creating the harness for designing and building the AI harness, agent framework, and reusable engineering components required to operate a modern data platform based on Apache Iceberg, Trino, and Parquet. Your primary responsibility is not only to build data pipelines, but also to establish the agentic execution framework (AI Harness) that enables multiple specialized agents to collaborate, execute tasks, maintain state, orchestrate workflows, and deliver end-to-end data engineering capabilities.

Key Responsibilities AI Harness Development Design and implement the foundational agent framework that provides:

Agent orchestration and coordination

Workflow execution and state management

Memory and context handling

Tool and function calling

Agent-to-agent communication

Human approval workflows

Logging and audit trails

Error handling and recovery

Security and access controls

Observability and monitoring The harness must support autonomous and multi-agent execution patterns. ________________________________________ Specialized Agents Creation The agents will need to perform the following:

Discover Iceberg catalogs and schemas

Profile datasets

Generate metadata and lineage

Ingest data from databases, APIs, files, and streams

Create Parquet datasets

Generate Trino SQL

Build transformation workflows

Create curated and business-ready datasets

Validate schemas and data integrity

Detect anomalies

Generate reconciliation reports

Analyze Trino workloads

Recommend partitioning strategies

Optimize Iceberg tables and Parquet files

Classify sensitive information

Manage metadata and lineage

Apply retention and compliance policies

Monitor pipelines and agent activities

Detect failures and bottlenecks

Generate alerts and operational insights 岗位概述：你将担任自主 AI 工程师，负责搭建 AI 运行基座，设计并开发 AI 运行框架、智能体框架，以及可复用的工程组件，支撑基于 Apache Iceberg、Trino 与 Parquet 搭建的现代化数据平台。你的核心工作不只是开发数据管道，还要搭建智能体执行框架（AI 运行基座），让多个专业智能体能够协同工作、执行任务、维护状态、编排工作流，实现端到端的数据工程能力交付。 --- 主要职责： AI 运行基座开发设计并实现底层智能体框架，提供以下能力： - 智能体编排与协同调度 - 工作流执行与状态管理 - 记忆与上下文处理 - 工具与函数调用能力 - 智能体之间通信 - 人工审批流程 - 日志与审计追踪 - 异常处理与故障恢复 - 安全与访问权限管控 - 可观测性与监控能力该基座需支持自主执行与多智能体协同运行模式。专业智能体开发开发的各类智能体需要完成下述工作： - 探测 Iceberg 目录与数据表结构 - 数据集探查与数据画像 - 生成元数据与数据血缘 - 从数据库、API、文件、流系统接入数据 - 构建 Parquet 数据集 - 注册并维护 Iceberg 数据表 - 生成 Trino SQL 语句 - 搭建数据转换工作流 - 产出经过清洗、可直接用于业务的数据集 - 校验表结构与数据完整性 - 异常数据检测 - 生成数据对账报告 - 分析 Trino 查询负载 - 推荐分区策略 - 优化 Iceberg 表与 Parquet 文件 - 敏感信息分类识别 - 元数据与数据血缘管理 - 执行数据留存与合规策略 - 监控数据管道与智能体运行行为 - 识别故障与性能瓶颈 - 生成告警与运维洞察

AI Engineer（英语流利）

职位描述