Moving ten tables and converting their SQL does not mean the reporting workload that uses them has moved. The nightly load may still run in the old warehouse.
A finance dashboard may still query its old semantic model. An export consumed by another system may never have been identified.
An enterprise data warehouse migration is complete only when the business capability works from the target, its consumers and controls have been accounted for, and the legacy implementation has a deliberate disposition.
This playbook starts after the organization has chosen Databricks and agreed on the migration scope. For the earlier work of inventorying the estate and approving that scope, see Databricks Discovery Phase.
Treat the production workload as the unit of disposition and acceptance: a connected set of source inputs, stored data, transformations, triggers, access rules, outputs, consumers, service levels and operational controls that delivers a recognizable business result.
One such workload might produce the daily revenue dashboard and month-end finance export. Moving the SALES schema describes only part of that job.
Give every workload an explicit disposition
Start with the approved inventory and decide what outcome each workload needs. The labels below describe the dominant treatment, not a command to apply the same treatment to every object inside it.
| Disposition | Use when | Evidence required before closing the decision |
|---|---|---|
| Retire | No justified consumer remains, or a duplicate output has an agreed replacement | Usage evidence, consumer/owner confirmation, retention obligations and an approved shutdown |
| Retain or coexist | A constraint or dependent consumer keeps the workload on the source for now | Documented system of record, sync boundary, owner and reason to revisit |
| Replatform or translate | Existing behavior can be preserved with limited implementation change | Dialect review, results comparison and target service-level test |
| Refactor | The business behavior remains but the logic or execution pattern needs substantial changes | Agreed behavioral contract and targeted regression evidence |
| Redesign | Source behavior relies on a pattern unsuitable for the target, or the business has approved a different behavior | Explicitly approved target contract; distinguish intended changes from defects |
| Consolidate | Several redundant implementations can feed one governed output | Consumer mapping, reconciled definitions and an agreed owner of the shared result |
Retirement is a valid migration outcome. So is temporary coexistence. Neither should be a silent consequence of an unconverted script.
A workload classified as refactor might include translated tables, a redesigned procedure, a retired report and a downstream consumer that temporarily stays on the source. Record those component decisions under the workload.
The choice should follow observed consumption, dependencies, criticality, dialect and data compatibility, service levels, policy requirements and the expected target behavior.
Migration alone is no reason to rebuild sound business logic, but object-by-object copying is no reason to preserve obsolete logic.
Use a workload migration card
One card should travel with each production workload from conversion through sign-off. It can be a ticket, spreadsheet row with linked evidence, or a document; the fields matter more than the format.
| Field | Record the answer |
|---|---|
| Business outcome and owner | What output or capability is being moved, and who can accept it? |
| Boundary and components | Which source feeds, tables, views, procedures, jobs, reports and interfaces belong to it? |
| Upstream and downstream dependencies | What must run before it, and which users or systems consume its outputs? |
| Source baseline | What definitions, results, service levels and failure behavior must be preserved? |
| Disposition and component exceptions | Retire, coexist, translate, refactor, redesign or consolidate; which components differ? |
| Target pattern and conversion method | What runs where, and what is translated, manually changed or rebuilt? |
| Data and logic acceptance | Which comparisons prove required data and business behavior, and what exceptions are approved? |
| Service and control acceptance | Which performance, scheduling, monitoring, permissions and audit requirements must be met? |
| Consumer switch and fallback | Who changes each connection or export, what is the cutover condition, and what happens on failure? |
| Retirement gate | Which residual queries, jobs, licences, archives and approval steps must close before disabling the source? |
| Evidence and decision | Links to results, exceptions, sign-off, date and accountable owner |
The card is deliberately narrower than a platform implementation checklist. It says what must be true for this workload to leave the old warehouse. The implementation checklist covers the broader Databricks environment.
Move a dependency-complete unit, not a convenient collection of objects
Suppose a report depends on an ERP extract, a nightly load, dimensional tables, a stored procedure, a semantic model, a Power BI dashboard and a scheduled CSV export.
These objects can be built in separate tickets, but they must be accepted as one connected outcome. If the dashboard points to Databricks while the export still reads a legacy table, the workload is in coexistence, not finished.
Group the objects that must run and be tested together. Shared input tables may serve several workloads and can move earlier; their presence on Databricks does not automatically complete every downstream workload.
Record cross-workload contracts so one group does not change a common metric or refresh time without another group noticing.
The execution group must include everything needed to prove and switch a coherent business outcome. For sequencing and duration, see the separate migration timeline guide.
Decide how each workload family moves
Stored data, schemas and views
Map the data the workload actually uses, including historical ranges and late-arriving corrections. Choose whether each target asset is copied, rebuilt from source, queried in place temporarily or no longer needed.
Check data types, decimal precision, timestamp interpretation, null behavior, keys and source constraints rather than treating a successful load as proof of equivalent data.
Databricks notes that primary and foreign keys are informational and that warehouse systems differ in transaction semantics and types.
A source process that depends on enforced relational constraints or a transaction spanning several statements needs an explicit target behavior and test, not a one-to-one DDL substitution.
The Databricks EDW migration guidance describes these differences. If the architecture itself is still under debate, Warehouse vs Data Lake vs Data Lakehouse owns that comparison.
Ingestion and CDC
Keep the contract in view: required source coverage, initial backfill, update/delete semantics, ordering, duplicates, freshness and restart behavior. A nightly batch feed may simply need a new destination and equivalent controls.
A CDC feed requires a credible way to establish the initial state, catch up changes and avoid gaps or double application during cutover.
Lakeflow Connect offers managed and customizable ingestion options, with connector support depending on source and deployment. A connector’s availability is a starting point, not acceptance evidence.
Dateonic’s CDC guide covers the implementation of change processing in more detail.
ETL and ELT transformations
Separate code translation from behavioral preservation. Standard SQL transformations or supported dbt models may need limited edits. Procedural chains, vendor extensions, external ETL and deeply coupled intermediate tables often need refactoring or a new target pattern.
The Databricks ETL migration documentation describes available approaches, but the amount of refactoring is source- and workload-dependent.
Compare the business result at meaningful boundaries: a curated order fact, an adjustment total, a published mart. Check incremental runs and corrections as well as an initial full load.
A converted query that runs successfully can still round differently, filter a null differently or apply the wrong effective-date rule.
Procedures, functions and proprietary logic
A stored procedure may be doing far more than returning rows. Trace temporary objects, dynamic SQL, branching, side effects, transactions, error handling and business rules.
Then decide whether its behavior belongs in Databricks SQL, a pipeline, a job, a notebook, an external service or a deliberately retained source process. This is a workload decision; the procedure name alone does not determine the target.
For example, a procedure that calculates revenue adjustments and emails a file has at least two contracts: the financial calculation and outbound delivery.
Converting its arithmetic while dropping the delivery side effect fails the workload. Source-specific translation, such as detailed Snowflake object mapping, belongs in the Snowflake migration guide.
Orchestration and scheduling
Preserve the dependency graph and failure semantics, not merely the cron expression. Record triggers, predecessor success conditions, retry limits, idempotency, recovery from partial runs and what happens when a source arrives late.
The workload may use Lakeflow Jobs or retain an external orchestrator; the acceptance question is whether the production outcome arrives correctly and can be operated after a failure.
BI, interactive SQL and outbound interfaces
Inventory direct queries, semantic models, extracts, dashboards, analyst workflows, APIs, file drops, downstream databases and application reads.
A target table can be correct while a dashboard still applies an old calculation, an extract uses a stale credential or a downstream app expects a different schema. Test the consumer’s actual connection, results, refresh schedule and relevant concurrency or latency baseline.
Databricks SQL can serve BI workloads through SQL warehouses and supported integrations, but a product capability does not reconnect a consumer for you. Document every switch and identify any consumer that must deliberately remain on the source.
The migration boundary extends to the output that people and systems use.
Access, policy and operations attached to the workload
Record who owns the output, which people and service identities may read or modify it, and any row or column restrictions, retention and audit needs. Reproduce the intended effect or approve a change; never infer equivalence from the presence of a target permission grant.
Unity Catalog provides row filters and column masks, while Dateonic’s governance guide covers the larger implementation design.
Assign an operational owner who can respond when freshness or output checks fail.
Validate the workload at the points where it can fail
Before switching a production consumer, the owner needs evidence that the target delivers the agreed business outcome. What counts as sufficient depends on the workload:
| Layer | Evidence to capture for the workload |
|---|---|
| Structure | Expected assets, columns, types, history range and source-to-target mappings |
| Data reconciliation | Comparable source and target data and business totals |
| Semantics | Business rules and intended exceptions produce the agreed result |
| Performance and freshness | Runtime, query response, concurrency and arrival time against the workload’s baseline |
| Operations | Dependencies, retries and failure handling work as required |
| Security | Intended users and services have the required access, without unintended access |
| Consumers | Dashboard results, exports, API contracts and application behavior accepted by their owners |
Compare equivalent business periods and distinguish approved changes in behavior from defects. The business definition and service requirement determine acceptable differences; there is no universal variance threshold or mandatory length of parallel operation.
Lakebridge Reconcile can compare supported sources and Databricks at schema, row and data-value levels. Its report is one source of evidence. The workload owner still needs to accept the business result, consumer behavior and operational and access requirements.
Use automation where it reduces mechanical work
Lakebridge separates live-workload profiling, code analysis, conversion and data reconciliation. Its support matrix differs by source and activity. Analyzer output can help identify interdependent files; it does not reveal every business-owned spreadsheet or external application.
Lakebridge is a Databricks Labs project supplied without a formal support SLA.
Databricks also documents an agentic code converter in Beta. It organizes SQL files into migration projects and supports selected source dialects, including T-SQL, Snowflake, Redshift, Oracle, BigQuery and Teradata.
The documented limits include 300 files per batch and 1,000 lines per script; Teradata BTEQ is unsupported. Check current availability and limits in the intended workspace before planning around it.
Conversion status is not workload acceptance. Run the converted code and confirm the outputs, consumers, controls and service levels. Databricks recommends comparing source and target results after conversion.
Cut over consumers, then apply a retirement gate
The target must first produce accepted outputs. Parallel comparison can provide evidence for critical or cyclical workloads, but its length and form should follow the actual business cycle and risk.
At cutover, identify the system of record, stop ambiguous dual writes, switch each named consumer and keep a tested fallback while it remains viable. If a source continues to serve another workload, document that coexistence instead of calling the entire estate retired.
The retirement gate is more demanding than “the job ran on Databricks.”
Confirm that required consumers now use the accepted target output; residual source queries and schedules are known; access, retention and archive duties are satisfied; operational ownership is in place; and the accountable owner authorizes disabling the old path.
If a dependency requires coexistence, give it an owner and a revisit condition. For formal tracking of cutover exposures, see Dateonic’s migration risk register.
Example: order and revenue reporting
Consider an illustrative enterprise reporting workload. An ERP extract feeds a nightly warehouse load and dimensional tables. A stored procedure applies revenue adjustments.
A finance semantic model serves Power BI dashboards, and a monthly CSV goes to a separate planning application.
The migration card names the finance owner, dashboard users and export recipient. It records source totals for ordinary days, late corrections and month-end, together with dashboard refresh time, export format, permissions and the current run’s failure and retry behavior.
The team replatforms the required order and dimension data, refactors the transformations that depend on source dialect behavior, and redesigns the adjustment procedure only where its procedural steps cannot preserve the agreed rule in the target pattern.
A duplicate, unused report is retired after its consumer is checked. The planning application’s CSV remains in scope even though it is outside the warehouse.
Target pipelines load and transform the data. The finance owner compares revenue and adjustment totals for the same business periods and investigates every material difference against agreed rules.
Engineers test late changes, restart behavior, access for finance and service identities, dashboard calculations, refresh performance and the exact CSV consumed by the planning application. A technically correct table does not compensate for a missing month-end export.
Once finance accepts the evidence, the dashboard and export switch to target outputs under a documented fallback. The team watches the next relevant reporting cycle, checks residual source activity and then disables the old jobs and connections with the owner’s approval.
These are example decisions, not reported results from a Dateonic client.
Each completed card records what was moved, what remains and who accepted the result. The separate guides address migration timing and implementation cost. For help executing the work across an EDW estate, see Dateonic’s migration services.