Databricks Migration: A Practical Guide to Seamless BI Cutover

A practical guide to migrating Oracle, Teradata and SSIS workloads to Databricks, covering the migration framework, automation limits, and how to keep dashboards live through cutover.

A successful Databricks migration is best approached as a staged modernization programme rather than a code-conversion exercise. Code conversion can accelerate migration, but scheduling, dependency mapping, control-flow redesign, testing and reconciliation can remain substantial parts of the work, particularly in complex enterprise estates. Automated migration tooling can convert substantial portions of compatible SQL, reducing repetitive rewriting. Complex procedural logic, unsupported constructs and source-specific dependencies still require engineering review, testing and remediation.

Migration timelines vary significantly with the size of the estate, number of workloads, source-system dependencies, data volume, testing requirements and target architecture. A focused domain pilot can be completed much faster than a full-enterprise migration.

Why Enterprises Are Moving Off Oracle, Teradata and SSIS

Oracle and Teradata remain capable enterprise data platforms, but organizations may choose to modernize them when they need a more unified architecture for analytics, data engineering, machine learning, streaming and semi-structured data. Organizations can face additional architectural and integration complexity when they try to combine traditional warehouse workloads with streaming, semi-structured data, data science and AI workloads. Legacy estates can also accumulate duplicated data, overlapping pipelines and conflicting metrics across teams over time, which can become a driver for modernization.

None of this means these platforms were poorly built. It means organizations may now have broader requirements than their existing architecture was designed to support efficiently. The lakehouse architecture combines data-lake storage with warehouse-style data management and analytics capabilities, allowing organizations to support analytics, data engineering and AI workloads on a shared data platform. Databricks describes this approach as a way to unify data warehousing with broader data and AI workloads rather than treating the migration as simply eliminating data warehousing.

What Actually Has to Move

Ask most planning teams what a Databricks migration from legacy data warehouse platforms involves, and the conversation stays on code: stored procedures, SQL scripts, SSIS packages. That is the visible slice, and it is genuinely one of the areas where migration tooling can provide significant acceleration. Automation coverage depends heavily on workload complexity. Straightforward SQL transformations are generally easier to convert than proprietary SQL, procedural code, cursor-based processing, complex SSIS control flows and workloads with external dependencies. The most reliable way to estimate automation potential is to run a representative pilot against the actual estate.

Code is only one part of the migration. The programme also has to address data, orchestration, dependencies, governance, security, lineage and downstream BI.

• Code logic. Stored procedures, SQL transformations and SSIS data flow tasks, often the only record of business rules nobody wrote down elsewhere.
• Job schedules and orchestration. The Control-M, AutoSys, TWS or Tidal chains, or SSIS job schedules, that decide when thousands of jobs run and in what order. Lakeflow Jobs provides Databricks-native workflow orchestration for scheduling and coordinating tasks, including ETL and machine learning workflows. It also supports dependencies, branching, looping and retries, although complex legacy orchestration may still need to be redesigned rather than reproduced one-to-one.
• Control flow. Loops, branches, error handling and cursor-based logic that may have been implemented efficiently enough in a legacy environment but need to be reconsidered for distributed execution, and often require redesign rather than a straight translation.
• Lineage and reconciliation. Evidence that the new workload produces the expected results, from aggregate checks and record counts through to targeted record-level comparisons where required.

The non-code portion of a migration can be substantial, but there is no universal percentage that applies across enterprise estates. The balance depends on the complexity of the source workloads, the amount of procedural logic, the number of dependencies, the target architecture and the degree of modernization planned. Treating code conversion as the entire migration is what creates the biggest gap between an initial estimate and the work required to reach production.

The Six-Stage Migration Framework

A Teradata to Databricks or Oracle to Databricks migration holds together better as a repeatable pipeline than as a series of separate conversion sprints. Six stages provide a practical framework for organizing the migration, although discovery, validation, governance and remediation typically overlap across migration waves.

1. Discover and assess. Parse the legacy ETL artefacts, stored procedures, SSIS packages and scheduler configurations, then build a complexity inventory across the estate. Dead-code detection matters here because legacy environments can contain obsolete, duplicated or rarely used workloads. Identifying them early can reduce the scope of the migration and prevent unnecessary conversion work.

2. Model. Convert extracted logic into an intent-based model rather than translating line by line. This stage is also where anti-patterns surface: row-by-row loops, cursor-based processing, vendor-specific SQL constructs and other procedural patterns that may need a redesign for a distributed execution model rather than a direct port.

3. Generate. Produce native target artefacts: Spark notebooks, SQL scripts and pipeline definitions, then integrate them with parameters, scheduling, dependencies, version control and deployment automation. Databricks' current agentic code converter can convert supported Oracle and Teradata SQL into Databricks SQL or notebook-based output, while validating and iteratively correcting generated code.

4. Validate. Run data quality checks, schema validation and source-to-target reconciliation, including row counts, aggregates, hashes or checksums where appropriate, and targeted record-level diffs for high-risk data. Keep evidence artefacts with a pass, warn or fail status for each check. This stage is what turns "we believe it matches" into a documented answer someone can audit.

5. Govern and secure. Configure Unity Catalog for access control, lineage, auditing and, where required, row-level filtering and column masking. Gate promotions through CI/CD and run workloads under least-privilege identities. Governance designed into the migration reduces the risk of having to retrofit access controls, lineage and compliance processes after go-live. Unity Catalog currently provides centralized governance capabilities including fine-grained access control, lineage and auditing.

6. Publish and cut over. Publish semantic models to the BI layer, run source and target systems in parallel where the workload requires it, maintain a rollback path, and support the cutover window with live dashboards and go/no-go checkpoints at each gate.

Published migration guidance for legacy platforms tends to follow a similar shape: choosing a target architecture, migrating data into the target storage format, refactoring pipelines where required, validating and reconciling outputs, integrating downstream analytics and only then decommissioning the legacy system. The framework above simply names each of those phases as a discrete, auditable stage rather than a loose sequence of tasks.

Landing Architecture: Lakeflow Pipelines, PySpark and Unity Catalog

Three Databricks capabilities carry much of the weight once code has landed on the new platform.

Lakeflow Pipelines provides a declarative approach to building data pipelines. Previously known as Delta Live Tables (DLT), the technology now sits within the Lakeflow platform. Instead of hand-coding every dependency between transformations, engineers define pipeline datasets and their relationships, while the framework manages execution and supports data-quality expectations. Expectations can report violations, drop invalid records or fail processing depending on configuration. For a team coming from imperative SSIS control flows, this is a significant mental shift: control moves from an explicit sequence of implementation steps toward a set of declared transformations and outcomes.

PySpark is one of the target options for converted transformation logic. Oracle PL/SQL procedures and SSIS data flow transformations can resolve into PySpark, Spark SQL or other Databricks-native patterns depending on the original workload. A simplified illustration of the shift:

-- Oracle PL/SQL (imperative, row-by-row)
FOR rec IN (SELECT * FROM orders WHERE status = 'PENDING') LOOP
  UPDATE orders SET status = 'PROCESSED' WHERE order_id = rec.order_id;
END LOOP;

# PySpark (set-based transformation)
orders_df = orders_df.withColumn(
    "status",
    when(col("status") == "PENDING", lit("PROCESSED"))
    .otherwise(col("status"))
)

The Oracle version processes one record at a time. The PySpark version expresses the transformation across the dataset rather than issuing one update operation per row. The resulting DataFrame would then be written back to the target table using the appropriate write or merge strategy. That difference is exactly why row-by-row and cursor-based logic gets flagged for redesign in the modelling stage rather than translated as-is.

Unity Catalog provides centralized governance across the Databricks environment: access controls, data and AI lineage, auditing and fine-grained controls such as row filters and column masks. Unity Catalog automatically captures lineage for Databricks queries and can trace data flows down to the column level.

Landing the estate on a Medallion architecture, Bronze, Silver and Gold layers, can turn a migration into a broader modernization effort rather than simply reproducing the legacy schema unchanged:

Tier	What it does
Bronze (raw)	Raw or minimally transformed source data, persisted so downstream layers can be rebuilt when needed.
Silver (cleaned)	Cleansed, validated, deduplicated and enriched data, including type normalization and business-rule transformations.
Gold (BI-ready)	Business-ready data products, dimensional models, aggregates and metrics optimized for analytics and BI.

Databricks describes Bronze as the raw layer, Silver as the refined layer and Gold as the business-ready layer, with data quality increasing as data moves through the architecture.

Keeping BI Alive Through the Cutover

The question every stakeholder eventually asks is some version of: what happens to my dashboards while this is running? A migrate SSIS to Databricks or Teradata cutover that touches dozens of downstream reports cannot simply go dark for the duration of a migration.

The answer is a controlled dual-run cutover with a rollback path where the workload requires it. Source and target pipelines can run in parallel for a defined window while source-to-target validation confirms that the new workload meets agreed accuracy and quality thresholds. Reports can then be moved in controlled waves as their underlying datasets and KPIs pass validation, rather than waiting for every report in the estate to be migrated at once. Rollback remains available through hypercare in case an issue surfaces after cutover. Gold-layer semantic models, the certified KPIs and business definitions reports depend on, can be established as part of the target architecture from the start rather than treated as a separate post-migration project.

Common migration risks in a Teradata cutover include differences in data types, precision, scale, timestamp semantics, implicit casting and expression behavior. Even when both platforms support equivalent numeric or timestamp types, differences in conversion rules or query behavior can affect results. Reconciliation is not a formality here; it is what provides evidence that the migrated workload produces the expected business results before and after cutover.

A Short Post-Mortem: What a Slipped Migration Actually Costs

Migration complexity often increases when dependencies across systems are overlooked or when unnecessary workloads are included in scope. Our articles on multi-platform data migration and selective data migration to reduce cost and complexity explore these challenges in greater detail, including how better discovery and scope decisions can reduce rework and migration risk.

Where This Is Heading

Reconciliation, orchestration rebuilding and anti-pattern redesign remain important migration challenges, but AI-assisted tooling is changing how much of the repetitive work can be automated. AI-assisted migration tools can generate first-pass conversions, identify errors and iterate on generated code. Databricks' current agentic code converter, for example, validates converted SQL and iteratively addresses conversion errors.

The realistic near-term direction is not full automation of a legacy warehouse migration; it is a shrinking manual core, concentrated on genuinely ambiguous logic, redesign decisions, exception handling, testing and governance sign-off. Discovery, code generation and validation will continue to become more automated, but the percentage that can be automated will remain dependent on the characteristics of the source estate rather than a single universal benchmark.

Where DataVolve Fits

DataVolve is Tarento's end-to-end data migration platform for modernizing complex legacy data estates, not just converting code. It brings discovery, dependency analysis, AI-assisted code conversion, orchestration, validation, reconciliation and governance into a unified migration framework.

Unlike tools focused primarily on code conversion, DataVolve addresses the wider migration effort: understanding dependencies, converting legacy SQL and ETL logic, rebuilding orchestration, and validating source-to-target parity. This enables a controlled path from discovery and assessment through engineering, validation and cutover.

For heterogeneous enterprise estates spanning databases, ETL platforms, schedulers and BI workloads, DataVolve brings these moving parts into one framework—reducing rework and the need to re-scope the migration as new dependencies emerge.

The goal is not to replace engineering judgement, but to automate repeatable migration work so teams can focus on business logic, architectural decisions, exceptions, validation and governance.

---

Frequently Asked Questions

What is a Databricks migration from a legacy data warehouse?

It is the structured move of data, code and reporting workloads off a platform such as Oracle, Teradata or a SQL Server and SSIS estate, onto the Databricks lakehouse. It covers more than moving tables: schemas and historical data can be landed in formats such as Delta Lake, stored procedures and ETL logic are rewritten or redesigned for the target execution model, job schedulers can be rebuilt using Lakeflow Jobs, and downstream reports are validated against their existing outputs before they are moved. Done properly, it is a re-platforming and modernization exercise rather than a simple lift-and-shift. Migration timelines vary widely according to workload volume, dependencies, data movement, testing requirements and the scope of modernization.

How do you migrate Oracle to Databricks?

The work typically runs in four phases. First, inventory Oracle objects: tables, views, packages, procedures, triggers and scheduled jobs, along with their dependencies. Second, move the data itself, using an appropriate database connection, replication technology or bulk-transfer process, and land it in the target architecture. Third, convert and redesign the transformation logic: Oracle SQL can be assisted by Databricks' migration tooling, while PL/SQL constructs such as cursors, loops, packages and other Oracle-specific features may require additional refactoring or redesign. Databricks' current agentic code converter supports Oracle SQL conversion into Databricks SQL or notebook-based output, but complex procedural logic still requires engineering review. Fourth, validate: row counts, aggregates, data types and query outputs are compared between Oracle and Databricks before workloads are cut over, with parallel operation used where appropriate until confidence is high enough to retire the source.

How do you migrate Teradata to Databricks?

Teradata migrations can range from minimal-change migration to broader refactoring and modernization, depending on business priorities and workload complexity. Data typically moves using an appropriate bulk-transfer, replication or ingestion approach, alongside migration tooling that can assist with compatible SQL conversion. Databricks' agentic code converter supports Teradata SQL and can generate Databricks SQL or notebook-based output, with validation and remediation still required. Common migration risks include differences in data types, precision, scale, timestamp semantics, implicit casting and expression behavior. Reconciliation should combine row counts, aggregates, checksums or hashes and targeted record-level comparisons rather than relying on a single test. Job orchestration can then be implemented using Lakeflow Jobs or retained in an external orchestrator where that better fits the target architecture.

Can SSIS packages be migrated to Databricks?

Yes, with caveats. SSIS packages contain control-flow logic, data-flow transformations, parameters, connections and execution dependencies that need to be analyzed and reimplemented in the target architecture. Some transformations have straightforward equivalents, while others require redesign or custom implementation. SSIS migration can be accelerated with conversion utilities and automated code-generation approaches, but package-level control flow, dependencies and business logic still require validation and, in some cases, manual remediation. Job scheduling and sequencing across packages may need to be rebuilt using Lakeflow Jobs or an external orchestrator. Row-by-row processing also needs to be assessed for redesign because a literal implementation may not provide the scalability or performance expected from distributed Spark-based processing.

How much of a Databricks migration can be automated?

The answer depends on the workload. Straightforward SQL conversion is generally more automatable than proprietary SQL, procedural logic, complex SSIS control flows, external dependencies and business rules embedded in legacy jobs.

Databricks' current agentic code converter supports Oracle and Teradata SQL and can generate Databricks SQL or notebooks, while validating and iteratively correcting generated code. The converter is designed to accelerate supported SQL migration, but its output still requires engineering review and testing.

Rather than relying on a generic automation percentage, run a representative pilot and measure conversion success, remediation effort, validation effort and runtime performance. Those results provide a more defensible estimate for the wider estate.

---

Plan Your Migration With Confidence

Moving off Oracle, Teradata or SSIS is a significant decision, and the projects that stay on schedule are the ones that scope the full effort, not just the code, from day one. If your team is evaluating a Databricks migration from a legacy data warehouse, a short landscape assessment can map your estate's complexity, flag the workloads carrying the most risk, and give you a realistic timeline and automation estimate before you commit to a full programme.

Talk to our migration team to scope your Oracle, Teradata or SSIS migration and get a clear, evidence-based plan for moving to Databricks without disrupting BI.