Five services.
One layer.
Migration, re-platforming, the modern data stack, data mesh and governed AI: five services on one open layer, each with the proof already built. Then the situations teams arrive in, what every decision-maker gets back, what to bring, and an honest comparison by category.
What we deliver, and what is already built
Five services on the same open layer. Every claim below points at something that runs; nothing is a roadmap item.
AI-powered ETL and scheduling migration
Export in, assessment out: what converts as-is, what needs a hand, what to retire. Deterministic conversion first, an agent for the long tail, a human review on every diff. Parity against the old estate is the acceptance test, not a promise.
Control-M → Airflow converter: 30 job types, tested against a real 499-folder estate export, 127 automated tests.
The service in detail →Re-platforming across engines
Add an engine, change an engine or run two side by side without a second copy of the data. One Apache Iceberg copy in your storage, each engine reading it, one column-level lineage that crosses the boundary. Leaving an engine becomes a per-workload decision.
Four engines reading one Iceberg copy: Snowflake, Databricks, Athena and ClickHouse.
The service in detail →Modern data stack, in your account
The stack a ten-person data team would take a year to assemble, deployed into your cloud account with agents doing the pipeline work and your people approving through a governance gate. Nothing hosted by us; nothing to migrate off later.
Our own estate runs this way and is demonstrated on platform.datap.ai: pipelines, dbt, lineage, BI and Text-to-SQL in one console.
The service in detail →Data mesh, implemented in code
Each domain owns its dbt project, its models, tests and owners. The platform holds the standards: one catalog, one lineage, one set of controls that every domain inherits. A data mesh built in dbt and Iceberg rather than in slides, so the guardrails are in the build, where drift is caught.
Our own estate runs as independent producer and consumer dbt repositories on a shared Iceberg lake, with lineage stitched centrally and the cross-engine edges counted on every build.
The service in detail →Governed AI on your data
Text-to-SQL, assistants and agents over the same governed tables, allowed to see only what the column classification permits, with the model chosen by data-residency policy and the evidence of every call kept in your own tables.
Text-to-SQL on Databricks and Snowflake, gated by classification, with every call logged.
The service in detail →Where you are, and what changes
Three situations we meet most. Each links to what we have actually built for it.
We run Snowflake and Databricks — or are moving between them
Two engines, two catalogs, two lineages, and an audit trail that stops at the boundary. Every migration is a re-platform.
One copy on Iceberg in your storage, both engines reading it, one column-level lineage across them, held in your own tables. Leaving either engine becomes a per-workload decision, with the evidence to show a risk committee.
The build, with numbers →We are a regulated enterprise on Azure
Most of what is written about open lakehouses assumes AWS. Your estate is Azure Data Lake Storage, Azure Databricks, Snowflake on Azure, Azure OpenAI.
The same design, layer for layer: ADLS for the open copy, Unity Catalog or Snowflake Open Catalog as the catalog of record, the same governance layer, models in your tenancy. What is built and what is planned is stated plainly.
The Azure mapping →We want AI on our data without a compliance incident
Agents and copilots are arriving faster than the controls. Nobody can say which data an answer used, whether it held personal data, or why the model was allowed to see it.
Governed tables first: lineage and personal-data flags produced with the pipelines. Then models you choose, running where you choose, gated by that classification, with the evidence of every call in your own tables.
AI Sovereignty →Concerns We Address
The specific data and AI risks regulated buyers raise — and how the layer handles each.
| Concern | How we address it | Relevant framework |
|---|---|---|
| Where did this number come from? | Column-level lineage computed from the pipelines, stored per build, one click from any number to its raw source columns | APRA CPS 230 · ASIC reporting · BCBS 239 principles |
| Locked into one warehouse's table format | Every table the layer builds is Apache Iceberg on your object storage; Snowflake, Databricks, Athena, ClickHouse read the same files | Open table format · portability by architecture |
| Going open means losing governance | Lineage, quality tests, personal-data map and masking policy travel with the data — on every engine, in your own tables, readable by any catalog | APRA CPS 234 · Privacy Act APPs |
| Personal data hiding in derived columns | Classification propagates through renames, joins and hashes; masking policy per sensitivity level, permissive by default | Privacy Act APP 6 · APP 11 · OAIC guidance |
| Pipelines take quarters and arrive untested | Plan → generate → build & repair → prove; row-count, range, set and null tests written with the pipeline; parity checks on migrations | APRA CPS 230 operational resilience |
| LLM leaking credentials or PII | Secret-request pattern blocks before any LLM call; PII write patterns forced through redaction | OAIC APP.11 · OWASP ASI02.6 |
| AI agent making unauthorised trades / actions | Multi-agent metadata triggers escalation + human-in-the-loop; per-action authorisation re-verified | ASIC AGENTIC_AI · APRA CPS230.ACCOUNTABLE_PERSON |
| AI in transaction monitoring / KYC / SMR drafting (AML) | AI-flagged matters require human suspicion-formation; tipping-off gate on customer-facing GenAI; 7-year decision provenance retained | AUSTRAC AML/CTF Act s41 · s107 · s123 (tipping-off) · AUSTRAC AI/ML Industry Guidance 2024-25 |
| Sanctions / PEP screening AI false-positive bias | Calibrated thresholds with documented FAR/FRR; annual recalibration evidence; manual override trail; bias profile across customer cohorts | AUSTRAC Sanctions Compliance Guidance · AML/CTF Rules Pt 4.4 |
| AI-assisted personal advice — best-interests duty + RG 255 | Advice model registry; BID step records with adviser-AI review notes; record-of-advice carries AI-source attribution; advice quality reviewed on cycle | ASIC AFSL · Corporations Act s912A / s961B · RG 255 (digital advice) · RG 271 (IDR) |
| AI-driven product recommendation pushing outside Target Market Determination | AI recommendation outputs constrained to TMD-eligible customers; distributor reporting; complaints linked back to TMD-breach root cause | ASIC AFSL · Corporations Act Pt 7.8A · RG 274 (DDO) |
| AI-generated marketing or PDS content with misleading claims | AI-content review log; claim-substantiation files; marketing approval workflow; customer-impact monitoring for adverse outcomes | ASIC Act s12DA · Corporations Act s1041H · ASIC AFSL s912A |
| Prompt injection / goal hijacking | Gate validates user intent vs agent intent at run time; prompts treated as untrusted input | OWASP ASI01 · ASIC Market Integrity |
| Hallucinated financial / clinical advice | Advice classification forces confirmation + professional-adviser disclaimer; post-call validator catches overconfidence | ASIC s912A · FINRA Communications · AU 6 Principle 6 |
| Untraceable AI decisions at audit time | Every decision written once with cited controls; SCD2 catalogue allows point-in-time reconstruction | ISO 42001 · APRA CPS 230 record-keeping |
| Data leaving sovereign boundaries | Container runs inside your VPC; data is designed to remain within your cloud account by architecture; air-gapped option available | APRA CPS 234 · OAIC APP 8 |
| Regulator publishes new guidance you miss | Web-extraction agents watch the regulators you subscribe to and propose catalogue updates as reviewable, versioned changes | Aligned to whichever regulators you subscribe to |
| Supply chain / MCP / third-party agent risk | Third-party agent registry; tool-descriptor validation; signed inter-agent messages | OWASP ASI04 (Agentic Supply Chain) |
What You Get
One layer, a different answer for every decision-maker.
Nothing to replace
A layer on the platform you already run. Same engines, same scheduler, same BI. Open storage means no second lock-in.
Every column accounted for
Column-level lineage, quality tests and a personal-data map produced on every build — from your own pipelines, in your own tables.
Reviewed, not hand-written
The layer plans, writes, runs and repairs the dbt models. Your team reviews the plan and the diff, not the boilerplate.
No data leaves your walls
Deploys inside your VPC, in your region. Your keys, your KMS, your IAM. Models you choose, running where you choose.
Auditor-ready, every day
Source-to-report lineage, classification and test results kept per run. AI decisions cite the regulator inline.
Production-safe from day one
Text-to-SQL, assistants and agents over the same governed data, gated by the classification the pipelines produced.
How It Integrates
Plugs into the stack you already run. No rip-and-replace.
Open storage
- •Apache Iceberg
- •AWS S3
- •Azure Data Lake Storage
- •Google Cloud Storage
Query engines
- •Snowflake
- •Databricks
- •BigQuery
- •Redshift
- •Athena · Glue
- •ClickHouse
- •Trino · Spark
Transformation & orchestration
- •dbt (or we generate one)
- •Airflow
- •BMC Control-M
- •AWS Glue
- •Your existing scheduler
Catalogs & lineage
- •dbt docs (two-way links)
- •AWS Glue Data Catalog
- •Snowflake Open Catalog
- •OpenLineage / DataZone (roadmap)
Cloud & region
- •AWS
- •Azure
- •GCP
- •On-premise
- •Sydney · Singapore · Tokyo · Hong Kong
AI models
- •Anthropic (Claude)
- •OpenAI
- •Google (Gemini)
- •AWS Bedrock
- •Llama · Qwen (local)
Agent frameworks
- •AutoGen
- •LangGraph
- •LangChain
- •Custom (via SDK)
BI & identity
- •Tableau
- •Power BI
- •Looker
- •Lightdash
- •Microsoft Entra · Okta · Cognito
What You Provide
Grounded expectations — exactly what your team needs to bring to a walkthrough.
Cloud account + VPC
AWS / Azure / GCP / on-premise. Your IAM, your networking, your logging.
Object store
S3, ADLS or GCS bucket for the Iceberg tables and the audit archive. Your bucket, your KMS keys.
Query engine(s)
Snowflake, Databricks, BigQuery, Redshift, Athena or ClickHouse — whichever you already use, one or several.
A source schema or existing pipelines
Read-only access to a few tables, or your Glue / Dataflow / SQL / dbt project. dbt is optional — you get one.
IdP for SSO
Microsoft Entra, Okta, Cognito or equivalent.
Scheduler (optional)
Airflow, BMC Control-M or your existing orchestrator. Jobs run as standard tasks.
How We Compare
Honest positioning against the categories buyers benchmark us against.
| Dimension | DATAP.AI | Warehouse-native governance | Catalog / GRC / AI-governance SaaS | Single-cloud native services |
|---|---|---|---|---|
| Storage format | Apache Iceberg in your object store; every engine reads the same files | Proprietary tables inside the warehouse | Reads metadata only; storage untouched | Native to that cloud only |
| Column-level lineage | Across engines, computed from the pipelines, stored in your tables, linked to dbt docs | Inside the vendor's catalog only | Table-level, or manual mapping | Within that cloud's services |
| Pipeline creation | Planned, generated, self-repaired, with tests — from a source schema or legacy jobs | Hand-written by your team | Not in scope | Visual builders; hand-written code |
| Personal data | Classified once, propagated through lineage; masking policy in a config table | Scanner on stored data; derived columns missed | Scanner on stored data | Per-service scanners |
| AI governance | Gate on every LLM call, citing regulators; gated by the same column classification | Content-safety filters | Policy documents; limited runtime enforcement | Single-LLM content-safety focus |
| Deployment & region | Your VPC, your region — Sydney, Singapore, Tokyo, Hong Kong or any you run | Vendor-hosted | Vendor SaaS typical | That cloud's regions |
| Commercial model | Effort-based engagements; the layer reduces the effort — no per-seat, no fixed-price promises | Consumption plus per-seat | Per-seat plus per-framework add-ons | Consumption-based |
See it on your own data.
Give us read-only access to a few tables. We come back with column-level lineage, a personal-data map and a first generated pipeline — on your engine, in your account. No slideware. No data leaving your walls.