Migrating in · the usage analyser

Start from what you run today.

Run one query in your own account and send us the file it writes. The analyser shows what that work costs today, beside a range for the work it can price on RunningLake, and says what each side leaves out. It is an estimate from a usage export, not a quote.

BUILDING Parts of this page describe what is not built yet, and each says so.

What the analyser asks for

You run a query we publish, in your own account, and send us the file it writes. There is one for a Databricks account and one for a Snowflake account. Each selects no statement text, no person and no table name, and no name of anything you called.

The analyser holds no Databricks or Snowflake login and takes none: you run the query yourself and send the file.

The queries were written from the vendor documentation and have not been run against a real account.

The queries themselves, their columns and their limits are on the docs page.

What it returns

Today’s side is what the export states, averaged to a month over the period it covers: dollars at the vendor’s list price, before any price you negotiated; or units at a rate you type, which the analyser never fills in for you; or units alone, with no dollars.

The RunningLake side is a range from low to high, with the rate card it was priced with and the region it assumed. The floor and the management fee are lines of their own. The fee is the management fee of the Team tier, and its rate is on the pricing page. A Business or Enterprise platform fee is not in the range. The RunningLake side does not price storage, or the services a plane keeps running whether or not any work runs.

One engine, Trino, has had its scan speed measured on a RunningLake plane, on one machine type. A Snowflake export that states the bytes scanned gets a range for its SQL warehouse work, at low confidence. The compute part of the range is the bytes in the export divided by that speed and priced at that machine's on-demand list price in us-east-2, an AWS region, which is the region the analysis states among its assumptions, with its low end at the typical speed measured and its high end at a slow one, and a range appears only when the control plane has a rate card loaded that is not out of date. The bytes in the export may not be counted the same way as the engine counts them, and the export does not say whether they are before or after compression, so the range could be too high or too low. The measurement was made on one sample table, read repeatedly from one file, on a node that looked idle, with warm caches, and on Trino 483 alone. It is not shown to hold for wide, nested, small-file or many-partition tables. Real tables may cost more than the range says, and the range does not bound work dominated by skewed joins. The high end of the range is the slowest of the cases the range was built from, at its second-slowest of ten runs. A skewed join was measured, was slower still, and is left out. That high end is not a worst case for your tables, and the low end is not a floor. A narrow range does not mean a small uncertainty: the gap between its two ends exists only because two speeds measured on the engine are apart, and nothing in the range allows for the bytes in the export not being comparable to the engine's. Cold start, memory growth and a busy shared node were not measured.

A Databricks export states no bytes scanned, so its work is not priced. Scheduled work, which the analyser would run on DuckDB, is not priced, because DuckDB has not been measured. When part of the work is not priced, the RunningLake side shows no total.

Work the analyser cannot place is listed with its reason and kept out of both sides: either the work is of a kind RunningLake does not run yet, or the export does not say what kind of work it is.

What it never claims

It shows two totals side by side, each with what it leaves out. It does not subtract one from the other, turn the two into a ratio, or promise that a move will cost what the range says. It is an estimate from a usage export, not a quote.

Read an inventory of your tables and roles

The analyser says what your work costs. The inventory reader starts a migration plan: you run one more query in your own Snowflake account, send us the file it writes, and say which project and environment the tables would land in. The console then shows a plan with one item for each table and one for each role you created. Each item says what it would become here, or why it needs a person or is refused.

The query selects the names, types, row counts and sizes of your tables and their columns, your roles, and the privileges granted to each role directly. It selects no person, no view text, no policy name or body, and no row of any table. It also sends the name of each view and of what depends on what, and which tables and columns carry a policy, never the policy's name or body. The control plane keeps the number of views and no name of a view or a dependency. Only names, types, counts, sizes and a few yes-or-no flags leave your account.

It leaves out privileges on objects that do not exist yet, and which role holds which role. It leaves out the SNOWFLAKE and SNOWFLAKE_SAMPLE_DATA databases, databases shared with you, the personal databases whose names begin USER$, and every INFORMATION_SCHEMA schema; the file counts the databases it left out.

The inventory query was written from the vendor documentation and has not been run against a real account. The type names the mapping recognises come from the same documentation; a type reported under another name is refused.

What is built today and what is not

Reading a Snowflake inventory, and mapping each table and role in it to what it would become here, is built. So is reading a usage export, and so is planning a mapped table as a change set that a person reviews and approves.

• Copying the data of a table is not built yet.
• Translating the SQL of views is not built yet.
• Moving jobs and models in is not built yet.
• Reading a Databricks inventory is not built yet.
• Reading a dbt manifest is not built yet.

What happens to your file

Of a usage export (the analyser keeps no names):

The export is read in memory and not kept. What is kept is the monthly total for each workload that has dollars, from the export or from a rate you typed, the units the export counts in aggregate, the kind of work each workload is, the id the export gives it, the period it covers and how many rows it had, who sent it and when, and the figures the analyser worked out from them, with the rate card and assumptions it used. The audit log keeps a keyed fingerprint of the file. An export is at most 16 MiB and at most 5,000 workloads.

Of an inventory (names are kept):

What you move has names, so an inventory is kept on its own terms. The file is read in memory and not kept. What is kept is the name of each table, column and role it lists, the database and schema each table is in, the type of each column, the privileges each role holds on tables, what each would become here and why an item needs a person, the row count and size of each table, who sent it and when, with the target you chose. No statement text is read, and no user or email field is selected, but a table, column or role may still be named after a person. The audit entries a migration writes keep a keyed fingerprint of the file and counts, the kind of a principal a person chooses, and no name. Everyone who may read the project can read the names, except those under a layer they may not see, and an item with no layer proposed is shown to everyone who may read the project. A person can choose who the grants of a role are for; the principal they type is kept with the role, may be an email address, is read by everyone who can read the names, and is erased when the migration is deleted. A person can also say that an item was done some other way, with rl migrate handled-outside or the API, and must write how; that note is your own words and may name anyone, it is kept with the item, is read by everyone who can read the names and is erased when the migration is deleted, and an audit entry says only that the statement was made or taken back, and whether the item is a table or a role. RunningLake does not do the item or check the note. Deleting a migration erases its plan and its items, and the names they hold, from the database the control plane keeps. A migration with a change set still draft, proposed or open cannot be deleted until that change set is withdrawn or closed. A change set already made from its items is not changed. Its documents, in the control plane's database and, once approved, in your configuration repository, keep the table, column and team names and the principal the grants of a role go to, which may be an email address, and deleting the migration does not reach them. A backup of that database made before the delete still holds the names and notes until that backup is removed. The audit entry of the delete keeps that it was deleted and how many items it had, and no name. An inventory is at most 16 MiB, and an organisation keeps at most 10 migration plans.

You can delete an analysis, or a migration with the names it holds, in the console, with rl migrate delete, or with the API. Deleting an analysis erases its totals and the figures worked out from them. The audit log keeps that it was deleted, and what it already kept when the analysis was made (the source, the period, how many rows and workloads, and a keyed fingerprint of the file), and no figure. An organisation keeps at most 200 analyses, and one that has that many is refused another. Deleting an analysis frees a place under the limit of 200 analyses, and deleting a migration frees a place under the limit of 10 migration plans. Closing an account does not erase them by itself yet: delete a migration first if you want its own copy of the names and notes gone. The change sets made from it keep theirs.

Analyse an export in the console

The analyser is a page of the console: open Migrate and choose Analyse a usage export. To read an inventory, open Migrate and start a migration. The console is at swim.runninglake.com ↗ and sign-in works — for addresses on our list. The Google app behind it is still in testing, so anyone we have not added is refused at the account picker and never sent back, and we do not see the attempt. Email first and we will add you.

How to ask is on the access page.