How OptimaFlo is built.
See the AI team, the pipeline canvas, the open table format and the compute engines under the platform.
Seven AI data agents.
One orchestrator.
Each agent has a job. The Orchestrator plans the work and hands each task to the right one.
Helps plan work and coordinate tasks.
You set the goal. Your team approves the work.
How a change
gets checked.
The agents write the work. A person on your team approves it before it runs. This is where you step in.
- 1
You approve the plan.
The Orchestrator shows a build plan with Approve and Reject buttons before it builds anything.
- 2
The SQL is checked first.
Before you see generated SQL, it gets a security check, a check that its tables and columns exist, and a dry run.
Before you see the SQL
- Security check
- Tables and columns exist
- Dry run
- 3
You approve the SQL.
For the Clean and Ready layers, you approve it, edit it, or reject it before the pipeline runs. Your edits must be a read-only SELECT.
- 4
The data is checked after.
After the SQL runs, the Quality Agent's checks run on the result. A failed check on the Ready layer stops the run. On Raw and Clean it can show as a warning.
Every run leaves a record. Any table can roll back to an earlier snapshot.
See your whole pipeline.
Edit any step.
The visual canvas draws every step of a pipeline as a graph. Open a step to preview its data or edit its SQL.
Build on a canvas
Drag and drop steps, and see how each one connects.
Preview the data
Check the data at a step before you run anything.
Edit the SQL
AI drafts the SQL. Open any step to read it or change it.
Raw. Clean. Ready.
Then your dashboards.
Data moves through three layers, each with one job, and you can look inside any of them. Your dashboards read from the last one.
Source records exactly as they arrived, with the full history kept.
From raw records to the dashboard your team reads.
Open tables.
Your data keeps its history.
OptimaFlo stores data as Apache Iceberg tables, an open table format. You get time travel, snapshots, schema changes and safe writes.
Time travel
Read a table as it was at an earlier point in time.
Snapshots
Every change makes a snapshot. To roll back, pick an earlier one, review what changes, and confirm with a reason.
Schema changes
Add or change columns without rewriting the table.
Safe writes
A write commits all together or not at all, so no one reads half a load.
Open standards. Apache Iceberg tables. Apache Polaris catalogs.
SQL engines.
DuckDB or your own warehouse.
OptimaFlo runs supported workloads on DuckDB, Google BigQuery or Amazon Athena.
- Tried first, always free
DuckDB
A fast, in-process SQL engine for analytics.
- Big, one-time jobs
Your warehouse
Google BigQuery on Google Cloud, or Amazon Athena on AWS, for workloads in your own account.
- Big, recurring jobs
Sail
PlannedPlanned support for Spark compatible data processing.
The context layer.
One set of rules for AI.
Five parts make up the context layer. Supported AI workflows can use them, so AI works with your rules.
Certified Metrics
Define, review, and approve key business metrics. Track formula versions and lineage. Approved formulas can be applied when generating supported Ready-layer SQL.
Dimensions & Relationships
Define how data is grouped and how tables join together. Give AI a clearer map of your data.
Business Glossary
Create shared definitions for important business terms. Supported AI workflows can use these terms.
Ontology & Impact
See a map of your data objects and their links. Check what other parts of the system may be affected by a change.
AI Instructions
Set rules for a workspace or a pipeline. Supported AI tools use these instructions when helping with work.
Trust your data
before anyone sees it.
The Quality Agent helps find data problems and set the rules that catch them.
Find the problems
Check for missing values, duplicates, invalid data, inconsistent records, and outdated information.
Stop bad data
Set checks that can pause a workflow when a quality rule fails.
Stay informed
Review quality scores and receive supported alerts.
Set up for you.
Under your control.
OptimaFlo helps set up supported resources in your own AWS or Google Cloud account, then schedules the work.
Automated setup
Automated provisioning helps set up supported storage, compute, orchestration, and catalog resources.
Scheduled for you
Apache Airflow schedules pipelines, retries failed work, and runs past data again when needed.
A record of changes
Each run keeps a record of its status, the layer it reached, any errors, and its times. Clean SQL keeps its earlier versions. Review pipeline activity, table history, and supported data lineage.
Everything inside OptimaFlo.
Pick a group for the details.
Move from raw data to ready-to-use information.
- Data connections
- Connect supported databases, APIs, apps, and cloud storage.
- Visual pipeline builder
- Build workflows on a drag-and-drop canvas. Preview data, edit SQL, and see how each step connects.
- AI-assisted SQL
- Use AI to write and improve SQL for pipelines and data models.
- Raw, Clean, Ready
- Keep raw source records, prepare clean data, and create tables for business reporting.
- Data models
- Build tables and transformations that help turn raw information into useful business data.
- Integrations and exports
- Connect supported sources and export results to supported cloud storage and warehouses. Use webhooks to send workflow updates.
AI help for each part of your data work.
OptimaFlo has seven specialized AI agents:
- Orchestrator
- Helps plan work and coordinate tasks.
- Connect Agent
- Helps connect and bring in data.
- Pipeline Agent
- Helps build and update pipelines.
- Transform Agent
- Helps create data models and SQL.
- Insights Agent
- Helps query data and explore results.
- Dashboard Agent
- Helps build dashboards and reports.
- Quality Agent
- Helps find and check data issues.
- Use your preferred AI provider
- Connect supported models from OpenAI, Anthropic, and Google using your own API credentials.
- Review AI-generated work
- Inspect and edit proposed SQL, pipelines, and reports before approval.
Help AI use your business rules, and check its work.
- Certified Metrics
- Define, review, and approve key business metrics. Track formula versions and lineage. Approved formulas can be applied when generating supported Ready-layer SQL.
- Dimensions & Relationships
- Define how data is grouped and how tables join together. Give AI a clearer map of your data.
- Business Glossary
- Create shared definitions for important business terms. Supported AI workflows can use these terms.
- Ontology & Impact
- See a map of your data objects and their links. Check what other parts of the system may be affected by a change.
- AI Instructions
- Set rules for a workspace or a pipeline. Supported AI tools use these instructions when helping with work.
- Data Quality Checks
- Check for missing values, duplicates, invalid data, inconsistent records, and outdated information.
- Quality Gates & Alerts
- Set checks that can pause workflows when quality rules fail. Review scores and receive supported alerts.
- Data Lineage
- See where data comes from and how it moves through your workflows.
Run data workflows and use the results.
- Dashboards
- Use AI assistance to create charts, filters, and dashboards.
- Ask Questions with AI
- Ask questions in plain English. OptimaFlo can help create SQL queries and show the results.
- Scheduled Reports
- Set up BI summaries and supported updates through email, Slack, or webhooks.
- Workflow Scheduling
- Use Apache Airflow to schedule pipelines, retry failed work, and run past data again when needed.
- Compute Engines
- Run supported workloads using DuckDB, Google BigQuery, or Amazon Athena.
- Apache Iceberg
- Store data using open tables that support snapshots, table history, and schema changes.
- Apache Polaris
- Manage Iceberg table catalogs using an open catalog system.
- Coming Soon: Sail
- Planned support for Spark-compatible data processing.
Keep your team in charge of your data platform.
- Bring Your Own Cloud
- Deploy supported data workloads in your own AWS or Google Cloud account.
- Human Approval
- Review AI-generated changes before they are deployed.
- Roles & Permissions
- Control who can view, build, and approve work in your workspace.
- Track Changes
- Review pipeline activity, table history, and supported data lineage.
- Cloud Setup
- Use automated provisioning to help set up supported storage, compute, orchestration, and catalog resources.
- Your AI Model Keys
- Use your own supported provider credentials. AI usage and cloud costs are billed separately.
- How Data Is Handled
- Your data tables are stored in your cloud under supported BYOC deployments. OptimaFlo also uses a hosted control plane, and some AI features may send relevant context or samples to your selected model provider.
Put your data to work with AI.
Spend less time preparing data and more time using it. With OptimaFlo, your team can build data pipelines, check for errors, and create reports, all with AI-powered help.
- Rows land as they are
- Full history kept
- Open Apache Iceberg tables
- Runs in your own AWS or GCP account
- SQL your team approves
- Snapshots you can roll back