BUILDING Parts of this page describe what is not built yet, and each says so.

Governance

Governance is a group in the console’s sidebar for the data a project holds and what is said about it. It sits between the project’s own screens and Organisation, and it has five entries. All of them work today, Coverage, Data assets, Glossary, Lineage and Stewardship.

The menu

The entries come in this order. Each has a scope: the project and environment named at the foot of the sidebar, or the whole organisation, whichever project is selected. The table says which each entry reads, and each screen says it again in its head.

AI metadata and Data access are planned and are not in the menu yet.

What works today

Data assets lists every layer, schema, table, view and volume that the selected environment declares and that you are allowed to see. For each one it shows:

• its kind and its layer;
• for a table, its tier, the classification your data plane last reported for it (public, internal, confidential or restricted, or pending until someone classifies it), or “not reported yet” if it has not reported one;
• for a table, whether your data plane says it is described: “described”, “not described”, or “not reported yet” if it has not reported the table. If git declares a comment for the table, the cell shows that text instead, marked Declared in git. It is never a dash, because a dash would say empty;
• for a layer, schema, view or volume, the description it declares in git, marked Declared in git, or “None declared in git” if it declares none;
• its owner, its steward and its domain, and for a table or a view its certification. The control plane works each of them out for you. One it did not say reads “not reported yet”, and one nobody holds reads “No owner”, “No steward” or “No domain”.

A layer can be marked limited. A limited layer, and everything in it, is left out of this list for anyone the project has not granted access to it. That hides the listing and nothing more: who can read the layer’s data is decided by grants, as before.

You can filter the list by kind, layer and tier. The filters are kept in the page address, and the list is read a page at a time.

Under the Analyst Agent’s bar, Search names finds assets by words. Type some and press Enter or Search; nothing is searched while you type. Words are split at spaces and dots, so silver.sales.orders is three words, and an asset matches when every word is found in its name, its display name or the description git declares for it, in any case. Each result says where its words were found, for example “Found in its name and its description”. The search keeps the filters you set, is kept in the page address, and finds only assets you may see: when none of those matches, it says so, and it does not say whether anything else would. A search takes at most 8 words, each at most 64 characters, and 200 characters in all.

Search reads names and the descriptions git declares; descriptions held in your data plane are not searched yet.

The list also has a count of the tables your data plane holds that git does not declare. It reads “not reported yet”, never 0, because nothing counts them yet.

Above the filters, Data assets has a bar for asking the Analyst Agent about your data: it is meant to find the assets a question is about and explain them. The bar is drawn and does nothing yet: its field is switched off, and nothing is sent anywhere.

An asset’s name opens its record page. It names the file that declares the asset. For a table it also lists the columns git declares, with their types, and shows what your data plane last reported: rows, files, the last commit, the snapshot, and the tier with who chose it and when. A table your data plane has not reported says so and shows no figure, never a 0. A column your data plane holds and git does not declare is never named.

A table’s record has an Edit… button for an owner, an admin or an engineer, and for the table’s steward or owner by assignment. So does each column of the table. A schema’s record has none, because a schema’s description, like a layer’s, a view’s and a volume’s, is declared in git. Edit… sends the text to your data plane when you press Save, and your plane writes it onto the table: a table’s description is its comment, and a column’s is that field’s doc. It is not a change set: there is no diff, nothing to plan and nothing to submit. The text travels down to your plane. Until your plane confirms the write, or fifteen minutes pass, it waits in the command that carries it; after that the control plane keeps none of it. The audit log records that you edited this table or column, with a keyed digest of the text and its length, never the text. The dialog says “Sent to your plane”, because the table changes when your plane has written it.

Saving needs a data plane whose release carries it. If yours does not, the dialog says so and nothing is written. A description git declares is not edited this way: it is marked Declared in git and changes by a change set.

On the record, a table’s description reads as one of four things, and never as its text: the text git declares for it, marked Declared in git; “Written by a person” or “Written outside RunningLake”, from what your data plane counted; “not described”; or “not reported yet”, when your plane has not reported the table. The Columns tab says how many of the columns your plane holds are described, by who wrote them, and does not say which of them. A table’s page in the Data lake links to its record from a read-only Governance section (see Where descriptions will live).

Coverage counts what the selected environment declares, over what you are allowed to see. For layers, schemas, views and volumes it counts how many have a description in git. For tables it counts how many your data plane has reported with a tier someone chose, how many are pending, and how many it has not reported at all. Under the cards it lists what waits on people: today, the tables waiting for a classification, with a link to the Data lake.

Every figure is the control plane’s count, shown as given, except how many tables your data plane has not reported at all: that is the total less the other two. There is no score, and nothing says how much is out of sight.

Coverage also counts how many tables and columns your data plane reports as described, split by who wrote them: declared in git, written by people, or written outside RunningLake. Where some tables have not reported, the card says how many. A plane that has reported nothing is never counted as 0: the card reads “not reported yet”.

Coverage also draws the figures it will count later: owners, stewards, certification and glossary terms, and the tables your data plane holds that git does not declare. Until they are counted, each reads “not reported yet”, never 0, because a 0 would say someone counted and found none.

Stewardship reads the whole organisation, whichever project is selected. It has three tabs. Domains lists the organisation’s domains, each with its owner, its steward and how many tables and views it reaches that you may see. People lists each member with what they own or steward. Your queue lists what waits on you: the terms waiting for your approval, and the certifications that have fallen due by age. An owner or an admin can create a domain, change it, archive it and name an owner or a steward for a domain, a layer, a schema or a table.

An asset’s record shows its owner, its steward, its domain and its certification, and each owner or steward says where it came from: set on the asset itself, or taken from its schema, its layer or, for a steward, its domain. An owner, an admin, or the asset’s steward or owner by assignment, can mark a table or a view certified or deprecated. Only an owner or an admin can remove the mark. A certified mark falls due after a year, and then waits in the queue of the asset’s steward or owner. It also falls due when the table is no longer the one that was certified, and then it reads Certification due on the record and waits in the same queue, dated by the last change to the table. Being a steward grants no access to data. Stewardship has its own page, Stewardship, which says who may do what.

A certification also holds a table’s columns and its name. A change that removes, renames or retypes a column a certified table declares in its own file, deletes that file, or changes the namespace or the name the file declares, waits for an owner or an admin to approve it, whichever environment it is planned in, and the change set’s page lists the assets downstream of the table that you may see, in the environment the file is for. Adding a column to the table’s own file does not wait, and a column added inside a nested type reads as a retype and does. An environment’s own patch of the table waits when it changes the table’s schema, namespace or name in any way and the table is certified in that environment. If the control plane cannot tell whether a table is certified, a change that adds, removes, renames or retypes a column, deletes the file or changes its namespace or name waits, and so does a file that cannot be read and an environment’s patch that changes the table’s schema, namespace or name. A commit pushed to your repository is not a change set, so no approval reason holds it: an environment that does not review projects it without an approval, and one that reviews holds what the branch brings until a person approves that transition. Certification on the Stewardship page and the Lineage page say what counts and what does not.

The Permissions page lists certification.set for owners and admins, which is what a role gives. It cannot show the other way in: a steward or owner by assignment may certify what they are assigned, in a signed-in session. An API token is judged by its own role alone.

The Glossary reads the whole organisation too. It lists the organisation’s terms, each with its definition, its domain and its status, and you can filter them by a word in a name, a synonym or a definition, by domain and by status. Any member proposes a term, except a viewer, and a term is a draft until an owner, an admin or the steward of its domain approves it. Nobody approves what they proposed themselves, unless the organisation has only one member, and a steward’s authority by assignment counts in a signed-in session and not for an API token. Every change to a term is a new version, and the version in force stays in force until a later one is approved. A term can be rejected, with a reason, and deprecated in favour of another term. Export YAML downloads the terms in force and the domains as one file. A term links to no column or table and nothing suggests one. The Glossary has its own page, Glossary, which says who may do what.

Lineage reads the project and environment too. It shows where an asset’s data comes from and where it goes, at table level, for the edges your project declares (what a view declares it reads, and the order a pipeline gives its own assets) and for the tables a view’s SQL and a pipeline asset’s inline SQL read, which are marked parsed. Every edge says how it is known, and the answer says which kinds of edge the control plane records for the environment, and whether the SQL that writes an asset was read. You choose an asset, a direction and a depth of up to ten, read the graph or a list of its edges, and ask for the impact of an asset: every asset downstream of it, at any depth, with its owner, steward and certification. An asset you may not see is not in it, and the asset next to a cut says “Some lineage is not visible to you.” A record’s Lineage tab and the Data lake show one hop and link to the explorer. Lineage has its own page, Lineage, which says what is declared, what is parsed and what is not.

Opening Data assets reads nothing from your data plane: the list comes from what your project declares in git, the tier your plane has already reported for each table, the counts of described tables and columns it has reported, and the owners, stewards, domains and certification marks the control plane holds. The record reads the same way. Coverage counts what your project declares and what your plane has reported.

Where descriptions will live

A description that a person writes is kept in your data plane, where your data is: on the table or the column it describes. Beside the text, the same table keeps a small marker that says only whether a person or a model wrote it, and never the text, a name or a time. The metadata tables beside your data, and the descriptions a model writes, are not built; when they are, they will be kept in your plane too. Whether that plane runs in your own cloud account depends on your tier, and the security page says which.

Descriptions will be read from your data plane straight to your browser, the way query results are, and not through RunningLake’s control plane. Only counts and ids travel up to RunningLake: how many columns have a description and who wrote them, never what a description says, and never a sample value from your data. The text you write travels down to your data plane when you save it. Until your plane confirms the write, or fifteen minutes pass, it waits in the command that carries it; after that the control plane keeps none of it. The audit log records that you edited this table or column, with a keyed digest of the text and its length, never the text.

What RunningLake does hold is what your project declares in git, which is part of its configuration and is what Data assets shows today; the terms and definitions your people type into the glossary; what your people record in Stewardship: each domain’s name and description, who owns and stewards what, and each certification mark with the note typed beside it; and a record of each description saved from the console: which asset, who saved it and how long the text was. The audit log keeps a digest of the text that your people type, never the text itself.

No screen in Governance shows a sample value, and none will. The most any screen will show about what a column holds is its type, its null percentage and its distinct percentage.

AI-written descriptions

AI-written descriptions will be optional, and off until an owner, administrator or engineer turns them on. Once they are on, a model will write descriptions from the bottom up: columns first, then each table from its columns, then each schema from its tables.

Each description a model writes will read “AI-generated, not reviewed” until a person reviews it. The person who started the run will not be allowed to review its descriptions, unless the organisation has only one member. A description that a person wrote or reviewed will never be overwritten by a model.

Two ways to change things

There are two ways to change a description or anything your project declares, and they do not mix.

• A description is edited directly. A person who may edit it opens the table or the column, changes the text and saves it, and the text goes to your data plane. It is not a change set: there is no diff and nothing to submit. A description that git declares is not edited this way: it is marked Declared in git, and its change goes through a change set.
• Anything your project declares in git, and every grant, is changed by a change set. It is planned, reviewed and approved on Changes, the same way as any other change to your project. Where approval is needed, its own author cannot give it, unless nobody else can (no other member who is not banned may approve there).

What Stewardship and the Glossary hold is changed in neither of these ways. Domains, who owns and stewards what, certification marks and the glossary’s terms are set directly in the control plane. A term is proposed by one person and approved by another. They are not change sets, they are not in git and they do not go to your data plane. An owner or an admin creates a domain and names an owner or a steward; the asset’s steward or owner by assignment may also certify it. Each change is in the audit log.

Not built yet

None of the following is available yet. Where the console has a screen for it, the screen says so.

• Descriptions. Nobody can review a description or see its history yet. Descriptions are not read from your data plane yet, so the only ones Governance shows are the ones your project declares in git.
• Stewardship. A steward cannot review a description yet.
• Certified tables. When a change that removes, renames or retypes a column of a certified table merges, its certification does not fall due yet. A change to a top-level column’s required flag or defaults, to a column’s or a table’s doc, to a table’s labels, annotations, identifier fields, schema id, column order, partitions, properties, write order, routing format, routing engine or routing consumers, the adding of column ids to a file that had none or the dropping of them from one that had them, and the creation of a table, are not escalated for review yet. The reason a change set gives counts the columns a change removes, renames or retypes and does not name them yet.
• The glossary. Nobody can link a term to a column or a table yet, and no link is suggested by name. A model does not draft a definition yet.
• Lineage. A statement kept in a file is not read yet, so what it reads is not shown. Observed lineage is not recorded yet.
• Access requests and reviews. Nobody can request access to an asset, or review who has it, yet.
• AI-written descriptions. No description is written by a model yet.
• Asking about your data. The Analyst Agent cannot answer a question about your assets yet.

Request early access