A code-to-column change-impact knowledge graph for AI coding agents.
The SQL lane is the half of the engine that reads your database side. It turns a
DDL file (or a fetched catalog snapshot) into tables and columns, MyBatis XML
into statements, and each statement into the columns it reads and writes. That
gives you mapper → statement → table → column.
It needs Python and one library. It never opens a database connection while analyzing, so you can run it on a laptop with no access to the real server.
For the other half, endpoint → controller → service → mapper, see
java-lane.md. The Java lane is a separate, JDK-only,
parse-only lane, and you do not need it to get value out of this one.
| What | Pin | Why |
|---|---|---|
| Python | 3.12.x | the reference stack, and what sqlglot is tested against here |
| an interpreter with sqlglot | CASCADE_PYTHON, else .venv/ at the repo root, else <cascade home>/venv |
a run tries those three in that order, and cascade doctor prints which one it found |
| sqlglot | sqlglot==30.17.0 |
the SQL parser, dialect by dialect, for Oracle, MySQL and PostgreSQL |
node bin/cascade.mjs setup # builds the interpreter and installs the pinned sqlglot
node bin/cascade.mjs doctor # tells you what is still missing, and the fix
setup builds <engine>/.venv inside a checkout of this repository and
<cascade home>/venv anywhere else, because an installed copy lives in a
directory the next upgrade replaces. --force rebuilds one; --home puts it in
the tool home even inside a checkout. setup repairs a managed environment
whose parser is missing, broken or at a different version; --force also
reinstalls a matching environment. It verifies a replacement before discarding
the backup, restores the previous environment on failure, and preserves data
outside the venv. See installation and recovery.
Doing it by hand still works:
python3 -m venv .venv
.venv/bin/pip install -r adapters/sql/requirements.txt
On Windows the interpreter lands in .venv\Scripts\python.exe, and that is
where a run looks for it there; the same two lines read
py -m venv .venv and .venv\Scripts\python.exe -m pip install -r adapters\sql\requirements.txt.
CASCADE_PYTHON names another interpreter on any platform. Both setup and
doctor require its exact pinned sqlglot version. setup treats this external
interpreter as read-only, including with --force: install the reported pin
yourself using that interpreter’s -m pip, or unset CASCADE_PYTHON and let
setup manage an environment.
The version is pinned in adapters/sql/requirements.txt, not chosen at install
time. sqlglot ships often, and a different version parses some statements
differently, which would move the pack digest. The digest has to be
reproducible, so the pin comes first and the upgrade is its own change: bump the
file, run the suite, and read the diff.
Live-catalog drivers (PyMySQL, psycopg, oracledb) are listed in the same file
but commented out. Install one only if you actually run
cascade catalog fetch.
profile.sqlDialects.main names your database. Two separate things follow from
it: which parser sqlglot uses, and the rule that decides when two
spellings mean one object. The second is a property of the database rather
than of the parser, so we declare it here instead of inheriting whatever the
parser happens to do.
sqlDialects.main |
sqlglot parser | identifier case | where the rule comes from |
|---|---|---|---|
mysql, mariadb |
mysql |
fold-lower |
MySQL always compares column, index, trigger and event names case-insensitively. Table and database names follow the server’s lower_case_table_names (0 stores and compares as written, 1 lower-cases, 2 compares insensitively). A DDL file does not carry that setting, so the lane takes the portable value the MySQL manual recommends for cross-platform schemas. |
postgres, postgresql |
postgres |
fold-lower |
PostgreSQL folds unquoted identifiers to lower case (manual §4.1.1). |
oracle, oracle-11g, oracle-19c |
oracle |
fold-upper |
Oracle stores and compares unquoted identifiers in upper case. |
hsqldb |
sqlglot’s default (ANSI) parser | fold-upper |
HSQLDB follows the SQL standard, where unquoted identifiers fold to upper case. sqlglot 30.17.0 ships no HSQLDB parser, so we use the standard-SQL one and say so. The case rule still comes from HSQLDB. |
h2 |
sqlglot’s default (ANSI) parser | fold-upper |
H2’s classic default (DATABASE_TO_UPPER=TRUE) folds unquoted identifiers to upper case. Same parser note as hsqldb. |
tibero, altibase, goldilocks |
oracle |
exact |
three databases the Korean market runs and sqlglot has never heard of. Each was built to be Oracle-compatible, so the Oracle grammar is the honest reading of their SQL. The identifier rule is not inherited with it: this engine has no citation for how any of the three folds an unquoted name, so it fails closed on exact and says so. Declare sqlIdentifierCase if you know your deployment’s rule. |
cubrid |
mysql |
exact |
the same, for the database that was built MySQL-compatible. |
| anything else | refused. A dialect we cannot route is an error rather than a quiet fall back to MySQL: a wrong dialect mis-parses every statement, and the result would still look fine. |
Two rules apply under every row.
"Item", `Item`) is exact everywhere. We match
it against the spelling the catalog declares and never fold it.fold-upper still answers
pms_product.profile.sqlIdentifierCase overrides the table with fold-lower, fold-upper
or exact. The default, null, takes the dialect’s own rule. Use exact for a
case-sensitive MySQL deployment (lower_case_table_names=0) whose schema really
does tell Item from ITEM. Changing the key changes what matches, so it
invalidates the cached lineage of every statement. Every key a profile can
hold, with an example of each, is listed in
concepts.md.
If two catalog names fold onto one key, the run prints a
folded_identifier_collision warning naming both, the first declaration keeps
the key, and both tables stay in the pack. Nothing is merged in silence.
ALTER leaves unsaid is the database’sThe DDL reader (catalog_ddl.py) parses the files with the same grammar the
statement lane uses, the standard grammar for H2 and HSQLDB included
(--dialect); for H2 and HSQLDB a CREATE TABLE that grammar cannot read is
read by one of their compatibility modes’ grammars (see below). It is told which
database the files are for (--database, the database sqlDialects.main
names, when that is not the grammar’s own name), and, when sqlDialects.main
is not declared, that the database is only assumed (--database-assumed). Each
of these is part of the cached catalog’s key.
The second matters because a migration’s ALTER depends on rules the file does
not write, and those rules belong to the database, not to the grammar it was
parsed with. DROP CONSTRAINT x drops the primary key only if x is the key’s
name, and the CREATE often never gave it one.
| Rule | PostgreSQL | MySQL, MariaDB | Oracle |
|---|---|---|---|
| a key’s name when the DDL gives none | <table>_pkey, the server’s convention, so a drop matched through it says so; a name past 63 bytes is not known |
always PRIMARY, whatever the DDL wrote |
not known: the server numbers it (SYS_C...) |
| dropping one column of a key | drops the whole key | the column leaves the key, the rest stays | drops the whole key |
ALTER INDEX a RENAME TO b |
renames the key a is the index of |
||
MODIFY c ... |
restates the whole column, so what it leaves out (NOT NULL, a comment) is gone |
changes only what it says |
With no database declared, the MySQL column applies, as an assumption: each
conclusion that rests on one of its rules (a key’s name is PRIMARY, one column
dropped from a key of several, what a MODIFY leaves out) is said as
alter_rule_assumed, whose sentence ends by asking to declare
sqlDialects.main if the files are for another database. The overview counts
them in its catalog-rules-assumed gap, the catalog axis is degraded with the
cause rules-assumed, and the fix it names is to declare sqlDialects.
A blank cell is a rule this reader does not hold for that database. A database
with no rules of its own here (H2, HSQLDB, CUBRID, Tibero, Altibase,
Goldilocks) borrows none from the grammar it is parsed with: speaking MySQL’s or
Oracle’s SQL does not show that a database follows MySQL’s or Oracle’s other
rules. Wherever a rule is missing, a key the DDL did not name has no known
name, and a statement whose effect hangs on the missing rule keeps what the
catalog had and says so: alter_primary_key_unknown (a drop or a rename that
may hit the key, one column dropped from a key of several) and
alter_modify_unsaid_unknown (a MODIFY that leaves out the type, the
nullability or the comment). Before this, H2 and HSQLDB files were read as
MySQL, rules included, so dropping an H2 key by the name its CREATE gave it
left the key in place, said as changing nothing.
The reader also applies what it used to skip: PostgreSQL’s RENAME c TO d
without COLUMN (a column, not the table), DROP c without COLUMN, Oracle’s
DROP (c, d), the USING INDEX ... ENABLE an Oracle export writes after a key
and MODIFY c NOT NULL ENABLE, H2’s ALTER COLUMN c RENAME TO d, MariaDB’s
MODIFY and CHANGE COLUMN IF EXISTS, and clauses sqlglot keeps as one piece
of text (DROP c, DROP d). It also applies MySQL’s ALTER TABLE ... COMMENT
(the table comment), H2’s DROP COLUMN (c, d) (each column), and H2’s and
HSQLDB’s ALTER COLUMN c <definition>, read as MODIFY is: what it leaves out
is kept and said (alter_modify_unsaid_unknown). An Oracle constraint state in
an ALTER clause (NOT DEFERRABLE ... VALIDATE) is read without it. An option
the grammar splits off, such as SET STATISTICS, or a MySQL table option
(ENGINE, CHARSET), is named as written in alter_clause_unsupported; a
clause that cannot be read is quoted as written. IF EXISTS on a column the
table does not have changes nothing, as in the database, and is said as
alter_if_exists_absent. A RENAME it cannot read is alter_unreadable.
CREATE TABLE the grammar cannot readAn Oracle export (NOT NULL ENABLE, storage clauses), an HSQLDB .script, an
H2 file written for MODE=MySQL: a CREATE TABLE the grammar cannot read as
written used to be lost without a word. Now it is read, or named.
ENABLE, VALIDATE,
NOVALIDATE, RELY, DEFERRABLE, INITIALLY ..., USING INDEX ...,
EXCEPTIONS INTO) and HSQLDB’s CACHED, MEMORY and TEXT; then what
follows the column list, only when every word of it is a table option the
reader knows (PCTFREE, STORAGE, TABLESPACE, SEGMENT CREATION,
ENGINE and the like; a MySQL table COMMENT is kept); then the table
constraints the catalog holds nothing of, each set aside only when its whole
shape is one (KEY, INDEX, UNIQUE, FULLTEXT or SPATIAL with a list of
columns, CHECK (...), FOREIGN KEY (...) REFERENCES, EXCLUDE), never the
primary key. What was set aside is said (create_clause_not_held).key or index (PostgreSQL, HSQLDB, H2, Oracle allow it) is a
column, and the primary key it carries stays. MySQL’s KEY idx (c), which
H2’s own grammar reads as a column named KEY, is not a column.DISABLE is not enforced, so it is not read: a disabled
NOT NULL leaves the column nullable, a disabled key is no key
(create_constraint_disabled).INHERITS is no table option. A PostgreSQL child table holds its parent’s
columns first, NOT NULL with them, the key and comments not inherited; a
parent no earlier file declared is said (create_parent_unknown)._DATABASE_RULES): H2’s MySQL, PostgreSQL, Oracle and MSSQLServer,
HSQLDB’s sql.syntax_mys, _pgs, _ora and _mss. A CREATE TABLE the
database’s own grammar cannot read, even with parts set aside, is read by a
mode’s grammar when every mode that reads it agrees on its columns and key,
and said (create_read_in_mode). The mode a project declares is not read,
and a mode lends none of that database’s ALTER rules..script that starts with SET) are read one by one: text that runs on is
cut where a statement word starts a line.CREATE TABLE that still cannot be read is named once
(create_table_unreadable), in every dialect, and one folded into another
statement is named with its line (create_table_unread). A file the grammar
library itself fails on no longer ends the run: the reader reads that file
one statement at a time, and keeps the statement it fails on as text, which it
reads again or names like any other. A parse that stopped only in statements
that declare and change no table (H2’s DROP TABLE t IF EXISTS, a GO
script’s DROP) says the catalog may have lost nothing by it
(parse_error_not_held); one that stopped where an ALTER or a RENAME is,
or where the reader cannot tell, stays parse_error.What the reader said is in the pack. The reader’s diagnostics used to go
to the terminal only. Its header now carries them, so a catalog read back from
the cache says them too. The pack groups them by code
(meta.laneStats.catalog: each code’s count, the first ten tables it named
and its first five sentences) with one diagnostic per kind in
meta.diagnostics (CATALOG_CREATE_TABLE_UNREADABLE, CATALOG_RULE_ASSUMED,
CATALOG_FILE_PART_NOT_HELD, CATALOG_IF_EXISTS_ABSENT and the rest), so a
schema that gives three hundred of one is one line with its count. The overview
counts them into three gaps: catalog-tables-unread, catalog-rules-assumed
and catalog-read-in-part; a clause the catalog does not hold and the reader’s
notes count into none. Tables lost or a database assumed make the catalog axis
degraded, with the cause (tables-unread, rules-assumed); the fix named is
sqlDialects for an assumed database and none for lost tables.
ruoyi-vue-pro’s SQL Server schema read with MySQL’s grammar gave 0 of its 60
tables and the axis said shipped; it now says degraded, cause tables-unread.
dolphinscheduler’s H2 schema went from 57 to 64 tables and from 583 to 622
columns.
The catalog worker is catalog-ddl/13, so a cached catalog is read again once.
A repository that has to run on seven databases ships its schema seven times:
DATABASE/
altibase/all_ddl_altibase.sql
cubrid/all_ddl_cubrid.sql
mysql/all_ddl_mysql.sql
oracle/all_ddl_oracle.sql
postgres/all_ddl_postgres.sql
tibero/all_ddl_tibero.sql
goldilocks/all_ddl_goldilocks.sql
That is every eGovFrame project, and reading all seven is not a fuller catalog:
it is the same tables declared seven times. Measured on
eGovFramework/egovframe-common-components before this rule existed: 182
tables declared eight times over, 13,505 duplicate-declaration warnings, and
four vendors’ files failing to parse under a grammar they were not written for.
cascade init groups the DDL by the vendor each file is FOR (a vendor-named
directory or a vendor name in the file name), and when the tree names one
vendor, that vendor’s files become catalog.ddl and the rest are recorded as
catalog.ddlAlternatives:
"catalog": {
"source": "file",
"ddl": ["../DATABASE/mysql/all_ddl_mysql.sql"],
"ddlAlternatives": {
"oracle": ["../DATABASE/oracle/all_ddl_oracle.sql"],
"tibero": ["../DATABASE/tibero/all_ddl_tibero.sql"]
}
}
Swapping to Oracle is then an edit of catalog.ddl, not a walk of the tree.
Which vendor, and where the answer comes from, in this order:
sqlDialects.main, when somebody already wrote it down. Their word stands.db-type,
database-type, db-dialect or dbms, whose value is a vendor this engine
routes. This is eGovFrame’s Globals.DbType = mysql, and it is the one that
is true, because the jdbc url beside it is commented out six times and live
once.No answer means no answer: catalog.source stays "none", the diagnostic names
the vendors it found, and you pick.
A tree that ships ONE vendor’s schema has nothing to choose between, but it
still has a dialect, and init writes it to sqlDialects.main in this order: the
dialect a profile already declares, then a MySQL marker in the DDL itself
(backquotes, ENGINE=), then the vendor every
jdbc url agrees on, then the dialect the schema file’s own text is written in
(SERIAL, a ::type cast or OWNER TO for PostgreSQL; VARCHAR2( or NUMBER(
for Oracle). With none of them the key stays empty, and the run reads the SQL as MySQL and
says so: PROFILE_DEFAULT_ASSUMED when sqlDialects is empty, and
alter_rule_assumed wherever an ALTER is decided by one of MySQL’s rules. Measured on a PostgreSQL MES
whose schema is one pg_dump: before this rule its catalog was parsed as MySQL
and not one of its tables was read.
The schema is not the only thing a project like that ships seven times. The SQL is too:
mapper/let/sym/prm/
EgovProgrmManage_SQL_altibase.xml
EgovProgrmManage_SQL_cubrid.xml
EgovProgrmManage_SQL_hsql.xml
EgovProgrmManage_SQL_mysql.xml
EgovProgrmManage_SQL_oracle.xml
EgovProgrmManage_SQL_postgres.xml
EgovProgrmManage_SQL_tibero.xml
Seven files, one <mapper namespace="progrmManageDAO"> between them. Read
together they are not seven mappers: they are one mapper whose statements
overwrite each other, so the SQL a statement ends up with is whichever copy the
walk read last — and that copy is then parsed under the ONE dialect the run
chose. Measured on eGovFramework/egovframe-enterprise-business-template before
this rule existed: 189 mapper files, 1,423 statements collapsing onto 205 ids,
47 of them failing to parse because they were written for a database this
project does not run.
cascade init groups the mappers the same way the DDL is grouped. Two things
have to be true for a set, and neither alone is enough: the files declare the
same namespace, and their paths are the same path apart from a vendor’s
name. EgovProgrmManage_SQL_mysql.xml and EgovProgrmManageDtl_SQL_mysql.xml
share a namespace and stay two different mappers, because cutting the vendor
word out leaves two different paths.
The copy for the vendor catalog.ddl chose is read; the rest go into
mappers.alternatives, and the statement lane reads past them by name:
"mappers": {
"alternatives": {
"oracle": ["../src/main/resources/mapper/let/sym/prm/EgovProgrmManage_SQL_oracle.xml"],
"tibero": ["../src/main/resources/mapper/let/sym/prm/EgovProgrmManage_SQL_tibero.xml"]
}
}
The run says both numbers:
mapper XML: 27 file(s), 162 left out as other vendors' copies (profile mappers.alternatives)
A set with no copy for the chosen vendor keeps the first copy by sorted path
and says so (MAPPER_VENDOR_UNMATCHED): dropping the set would lose the
statements, and choosing by reading the SQL would be this engine deciding which
database a project runs on. Move a vendor’s files out of mappers.alternatives
to read that vendor instead.
The statement lane reads two elements, not one. <mapper namespace="…"> is
MyBatis 3; <sqlMap namespace="…"> is iBATIS 2, which every eGovFrame project
written before 3.x runs on, and a tree can hold both at once. One worker reads
both, and one framework pack (mybatis-xml) declares the lane for either.
What is different about the older element, and what this lane does with it:
| iBATIS 2 | read as |
|---|---|
<select> <insert> <update> <delete> <procedure> <statement> |
all six are SQL the runtime sends |
#name# |
a bind parameter, the same as MyBatis’ #{name} |
$name$ |
a raw substitution, the same as ${name}, and the statement is flagged |
$dbMain$.tb_user |
a schema qualifier when schema.propertyNames declares it |
<dynamic prepend="WHERE"> |
the prepend is folded in, and the first AND/OR under it is dropped |
<dynamic> with no prepend |
the body is kept as it stands, conjunctions and all |
<isNotNull> <isEqual> <isNotEmpty> … |
the affixes plus the body; the test is not modelled, exactly as <if test> is not |
<iterate open close conjunction> |
the body ONCE, with open/close folded in |
<include refid> / <sql id> |
resolved, in the same index the MyBatis fragments use |
What a MyBatis statement is called is its namespace plus its id, except that
an id already starting with its namespace is not prefixed again, the rule MyBatis
applies itself: <mapper namespace="loginDAO"> with
<select id="loginDAO.actionLogin"> is loginDAO.actionLogin.
What an iBATIS statement is called depends on one setting.
<sqlMapConfig><settings useStatementNamespaces="true"/> makes the runtime key
namespace.id; without it — and iBATIS’ own default is without it — the key is
the bare id, global across every sqlMap file in the deployment. The worker
reads that setting out of any <sqlMapConfig> among the lane’s inputs
(discovery hands it the directory the file sits in), and two configurations that
disagree are reported with the default left standing. Two files declaring one id
is a duplicate_statement_id warning naming both.
The Java side of the same shape is in docs/setup/java-lane.md: which receivers
run an iBATIS statement, and how a bare id binds.
A mapper that writes {call p_close_order(?, ?)} names a procedure, and the
tables are inside the procedure. Korean SI systems keep a large share of their
business logic that way, in Oracle packages and PostgreSQL functions, and a
lineage that stops at the call stops before the tables. So the catalog reads the
routines the DDL declares, and a statement that calls one is read through it.
What the catalog reads, as text, from the same DDL files:
| written | read as |
|---|---|
CREATE [OR REPLACE] FUNCTION\|PROCEDURE name(...) ... AS $$ body $$ |
a PostgreSQL routine; any $tag$ quote works, and LANGUAGE is recorded |
CREATE [OR REPLACE] PROCEDURE\|FUNCTION name ... IS\|AS ... END; then / |
an Oracle, Tibero or Altibase routine |
CREATE [OR REPLACE] PACKAGE BODY pkg IS PROCEDURE p ... FUNCTION f ... |
one routine per member, named pkg.p and pkg.f |
A comment or a string literal that says CREATE FUNCTION is not a routine. Each
one is a routine record in the catalog stream, beside the tables.
What a routine runs is every SELECT, INSERT, UPDATE, DELETE, MERGE
and WITH its body holds, taken out of the control flow around it (IF … THEN,
FOR r IN … LOOP, cursor declarations). PL/SQL’s own SELECT a INTO v FROM t
loses its INTO v, so the rest is SQL. A branch is not modelled: every statement
a body holds is one it CAN run, which is the reading a MyBatis <if> already gets.
How a statement reaches one. A statement names a routine by
{call x}, {? = call x}, CALL x, EXEC x, PERFORM x, a BEGIN x(...); END;
block, or x(...) inside its SQL. The name is looked up case-insensitively, whole
(app.p_close_order) or bare (p_close_order). A bare name two routines share is
not looked up by its bare spelling: which one an unqualified call reaches is the
database’s search path, and it is not guessed. A routine that calls a routine is
followed four deep, and a cycle stops at the names already read.
What the routines reach joins the statement’s own facts marked
"via": "routine", and the statement’s record lists what was read:
"routines": [
{ "name": "app.p_close_order", "depth": 1, "statements": 3, "parsed": 3 },
{ "name": "app.f_log", "depth": 2, "statements": 2, "parsed": 2 }
]
On the graph those are EXECUTES, READS and WRITES edges graded
SOUND_SET, with the routine names in the evidence. The statement’s own SQL
stays EXACT. When the catalog holds any routine, a statement that is nothing but a
call is not handed to the parser at all, so it records no parse failure. A statement that calls no routine is
byte-identical to what it was.
What it does not do. A routine the DDL does not declare is not read, and
routines are read from DDL files only, not from a live database. SQL a body builds
as a string and runs (EXECUTE IMMEDIATE, EXECUTE format(...)) is not read. A
trigger is not followed: a statement that writes a table does not reach what a
trigger on that table runs.
Column answers carry business meaning only when the catalog carries comments. There are two ways to get them, and you pick one per project.
A file in the repo. If the tree ships a schema dump with CREATE TABLE and
COMMENT, point catalog.source: "file" and catalog.connectionFrom at it.
This needs nothing but Python and sqlglot, and it is where to start.
A snapshot you fetch once. For a schema that lives only in a running
database, cascade catalog fetch opens one read-only connection, reads the
tables, columns and comments, and writes a snapshot into .cascade/. It then
writes catalog.source: "jdbc" into the profile itself, so nothing is left for
you to hand-edit. Analysis reads the snapshot, never the server. Read
db-catalog.md for the connection and credential rules. A
password is never committed, and never written into .cascade/, the pack or a
log: the one place it may be kept is ~/.cascade/credentials, at mode 0600.
With no catalog at all, the lane still works: a table or column exists in the
pack wherever a statement named it. The pack then declares the catalog axis
not-shipped and says why, so you can see what you are missing rather than
guess.
macrozheng/mall is a good first target: Apache-2.0, Spring Boot with MyBatis XML mappers, a MySQL dump in the repo, and a matching Vue admin. About 30 tables, which is big enough to show one column reaching many screens and small enough to iterate on.
Keep target checkouts outside this repository, in a sibling folder. The name
target-examples/ says what it is: disposable, safe to delete later without
anyone having to investigate first.
git clone --depth 1 https://github.com/macrozheng/mall ../target-examples/mall
node bin/cascade.mjs init --root ../target-examples/mall --project mall
node bin/cascade.mjs analyze --root ../target-examples/mall
node bin/cascade.mjs view --project mall
init writes .cascade/manifest.json (the checkout path and its pinned commit)
and .cascade/profile.json (the build tool, the framework packs, the dialect,
and where the catalog comes from). analyze takes no lane flags: it reads those
two files plus what discovery found. If you want to know what a run would ship
before you run it, cascade estimate says so axis by axis.
The venv (.venv/) and __pycache__/ are gitignored engine runtime, and are
not committed.
.venv/bin/python -m unittest discover -s adapters/sql -q # the Python side
npm run test:quick # the Node side
scripts/generality-gate.mjs runs the whole engine over a pinned corpus of real
repositories that nobody wrote it for, Apache DolphinScheduler and JeecgBoot
among them. That corpus exists to catch a rule that only works on the project it
was written against; it is not a fallback target, and it is not a demo. It
clones outside this repository and is skipped unless you ask for it.