Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
Changelog
=========

0.22.0
------

* Feat: Respect naming conventions for autogenerated spatial indexes @adrien-berchet (#628)

0.21.0
------

Expand Down
54 changes: 54 additions & 0 deletions doc/alembic.rst
Original file line number Diff line number Diff line change
Expand Up @@ -261,3 +261,57 @@ thus the ``env.py`` file should look like the following:
render_item=alembic_helpers.render_item,
)
# ...


.. _naming_conventions_for_spatial_indexes:

Naming Conventions for Spatial Indexes
--------------------------------------

By default, GeoAlchemy 2 generates spatial index names using a fixed template (such as
``idx_<tablename>_<columnname>``). When using SQLAlchemy metadata naming conventions (e.g.
``MetaData(naming_convention={"ix": "ix_%(table_name)s_%(column_0_N_name)s"})``), you can opt in to
having autogenerated spatial indexes follow this naming convention by setting
``follow_naming_convention=True`` on spatial types (:class:`~geoalchemy2.types.Geometry`,
:class:`~geoalchemy2.types.Geography`, or :class:`~geoalchemy2.types.Raster`):

.. code-block:: python

metadata = MetaData(naming_convention={"ix": "ix_%(table_name)s_%(column_0_N_name)s"})
Base = declarative_base(metadata=metadata)

class Lake(Base):
__tablename__ = "lake"
id = Column(Integer, primary_key=True)
geom = Column(
Geometry(
geometry_type="LINESTRING",
srid=4326,
spatial_index=True,
follow_naming_convention=True,
)
)

When ``follow_naming_convention=True`` is enabled:

1. **Naming Resolution and Fallback**: The index name is generated according to the ``"ix"`` key of the
naming convention (for example, ``ix_%(table_name)s_%(column_0_N_name)s`` produces ``ix_lake_geom``,
or ``ix_gis_lake_geom`` if a schema token is used). If the naming convention does not define an
``"ix"`` key, the index name safely falls back to the default ``idx_<tablename>_<columnname>``.
2. **Dialect DDL Behavior**: On PostgreSQL and MSSQL, the convention name is used directly. On MySQL,
where autogenerated spatial indexes are created unnamed by default, enabling the flag creates the
spatial index with its explicit convention name (``ALTER TABLE <table> ADD SPATIAL INDEX <idx_name>(<col>)``).
3. **Alembic Writer Requirement**: You must register the :obj:`~geoalchemy2.alembic_helpers.writer`
in ``env.py`` via ``process_revision_directives=alembic_helpers.writer``. The writer handles
spatial index operations and rewrites SQLite/GeoPackage table-modification directives to suppress
false diffs.
4. **SQLite / SpatiaLite and GeoPackage**: SpatiaLite implements spatial indexing via R-tree virtual
tables whose internal trigger and table names are fixed by SpatiaLite (``idx_<tablename>_<columnname>``).
Because of this, naming conventions do not rename the underlying R-tree virtual tables on SpatiaLite.
While Alembic revision autogeneration through ``process_revision_directives=alembic_helpers.writer``
filters out spurious index differences, direct calls to ``autogen.compare_metadata()`` without the
writer will detect the difference between the convention-named metadata index and the database R-tree.
5. **Switching on Existing Databases**: When switching ``follow_naming_convention=True`` on an existing
database (where the spatial index was previously created with the default ``idx_<tablename>_<columnname>``
name), Alembic autogenerate will propose dropping the old index and creating the new convention-named
spatial index. Ensure you test this migration before deploying.
8 changes: 8 additions & 0 deletions doc/core_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,14 @@ In that case every ``Table`` that's referenced to by ``metadata`` would be
created in the database. The ``metadata`` object includes one ``Table`` here,
our now well-known ``lake_table`` object.

.. note::

**Spatial Indexes**

By default, GeoAlchemy 2 automatically creates a spatial index on spatial columns
(named ``idx_<tablename>_<columnname>``). To configure naming conventions, custom
indexes, or dialect options, see the :ref:`spatial_indexes` guide.

Reflecting tables
-----------------

Expand Down
1 change: 1 addition & 0 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ system. If you're new to GeoAlchemy 2 start with the :ref:`orm_tutorial` page. I

orm_tutorial
core_tutorial
spatial_indexes
dialect_specific_features
gallery/index
alembic
Expand Down
8 changes: 8 additions & 0 deletions doc/orm_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,14 @@ If we wanted to drop the table we'd use::

>>> Lake.__table__.drop(engine)

.. note::

**Spatial Indexes**

By default, GeoAlchemy 2 automatically creates a spatial index on spatial columns
(named ``idx_<tablename>_<columnname>``). To configure naming conventions, custom
indexes, or dialect options, see the :ref:`spatial_indexes` guide.


Create an Instance of the Mapped Class
--------------------------------------
Expand Down
203 changes: 203 additions & 0 deletions doc/spatial_indexes.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
.. _spatial_indexes:

Spatial Indexes
===============

Spatial databases use spatial indexes to accelerate queries involving spatial operators and functions
(such as bounding-box intersections, distance queries, and geometric containment).

GeoAlchemy 2 simplifies spatial index management by automatically creating and managing spatial
indexes for :class:`~geoalchemy2.types.Geometry`, :class:`~geoalchemy2.types.Geography`, and
:class:`~geoalchemy2.types.Raster` columns.


Default Behavior
----------------

By default, any spatial column defined with ``spatial_index=True`` (the default) automatically
registers a spatial index in the table's metadata::

from sqlalchemy import Column, Integer, MetaData, Table
from geoalchemy2 import Geometry

metadata = MetaData()
lake_table = Table(
"lake",
metadata,
Column("id", Integer, primary_key=True),
Column("geom", Geometry("POLYGON")),
)

The autogenerated index is assigned a default name using the template ``idx_<tablename>_<columnname>``::

>>> [idx.name for idx in lake_table.indexes]
['idx_lake_geom']

When creating the table with ``lake_table.create(engine)`` or ``metadata.create_all(engine)``,
GeoAlchemy 2 automatically creates the corresponding spatial index using the native indexing mechanism
of the target database dialect.


Naming Conventions
------------------

SQLAlchemy allows you to define automated `constraint and index naming conventions
<https://docs.sqlalchemy.org/en/latest/core/constraints.html#constraint-naming-conventions>`_ on
:class:`~sqlalchemy.schema.MetaData`.

By default, GeoAlchemy 2 uses its hardcoded ``idx_<tablename>_<columnname>`` naming scheme for
backward compatibility. To instruct GeoAlchemy 2 to follow the naming convention configured on the
metadata, set ``follow_naming_convention=True`` on the spatial type.

SQLAlchemy Core
^^^^^^^^^^^^^^^

In SQLAlchemy Core, pass ``follow_naming_convention=True`` when defining the column::

from sqlalchemy import Column, Integer, MetaData, Table
from geoalchemy2 import Geometry

metadata = MetaData(naming_convention={"ix": "ix_%(table_name)s_%(column_0_N_name)s"})
lake_table = Table(
"lake",
metadata,
Column("id", Integer, primary_key=True),
Column("geom", Geometry("POLYGON", follow_naming_convention=True)),
)

>>> [idx.name for idx in lake_table.indexes]
['ix_lake_geom']

SQLAlchemy ORM
^^^^^^^^^^^^^^

In the SQLAlchemy ORM, associate the naming convention with the declarative base's metadata::

from sqlalchemy import Column, Integer, MetaData
from sqlalchemy.orm import declarative_base
from geoalchemy2 import Geometry

metadata = MetaData(naming_convention={"ix": "ix_%(table_name)s_%(column_0_N_name)s"})
Base = declarative_base(metadata=metadata)

class Lake(Base):
__tablename__ = "lake"
id = Column(Integer, primary_key=True)
geom = Column(Geometry("POLYGON", follow_naming_convention=True))

>>> [idx.name for idx in Lake.__table__.indexes]
['ix_lake_geom']

Or using the SQLAlchemy 2.0 ``DeclarativeBase`` class::

from sqlalchemy import MetaData
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
from geoalchemy2 import Geometry, WKBElement

metadata = MetaData(naming_convention={"ix": "ix_%(table_name)s_%(column_0_N_name)s"})

class Base(DeclarativeBase):
metadata = metadata

class Lake(Base):
__tablename__ = "lake"
id: Mapped[int] = mapped_column(primary_key=True)
geom: Mapped[WKBElement] = mapped_column(Geometry("POLYGON", follow_naming_convention=True))

Resolution and Fallback Rules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When ``follow_naming_convention=True`` is enabled:

1. **Resolution**: The spatial index name is resolved using the ``"ix"`` key (or ``Index`` class key)
defined in the metadata's naming convention dictionary.
2. **Fallback**: If the naming convention does not define an ``"ix"`` key (or if no naming convention
is configured), the index name safely falls back to ``idx_<tablename>_<columnname>``.


Custom User-Defined Spatial Indexes
-----------------------------------

If you need full control over the spatial index (for example, to specify index storage parameters,
create a partial index with a ``WHERE`` clause, or index multiple columns), you should disable
automatic index generation by setting ``spatial_index=False`` and define an explicit
:class:`~sqlalchemy.schema.Index`::

from sqlalchemy import Column, Index, Integer, MetaData, Table
from geoalchemy2 import Geometry

metadata = MetaData()
lake_table = Table(
"lake",
metadata,
Column("id", Integer, primary_key=True),
Column("geom", Geometry("POLYGON", spatial_index=False)),
)

# Define a partial GiST spatial index on PostgreSQL
Index(
"idx_lake_active_geom",
lake_table.c.geom,
postgresql_using="gist",
postgresql_where=lake_table.c.id > 100,
)


Dialect-Specific Details
------------------------

PostgreSQL / PostGIS
^^^^^^^^^^^^^^^^^^^^

- PostGIS spatial indexes are created as GiST (Generalized Search Tree) indexes using
``USING gist``.
- **N-Dimensional Indexes**: By default, a 2D bounding-box index is created. For 3D or 4D geometries,
you can set ``use_N_D_index=True`` on the spatial type to generate an N-D index using the
``gist_geometry_ops_nd`` operator class.
- **Managed Columns (`use_typmod=False`)**: When PostGIS type modifiers are disabled, columns are
added via ``AddGeometryColumn()`` after table creation, and the spatial index is created in an
``after_create`` event using the resolved convention name.

MySQL and MariaDB
^^^^^^^^^^^^^^^^^

- Spatial indexes are created as ``SPATIAL INDEX``.
- **Default naming**: In MySQL, autogenerated spatial indexes are created unnamed by default
(``ALTER TABLE <table> ADD SPATIAL INDEX (<column>)``), which causes MySQL to assign the column's
name to the index.
- **Naming Conventions**: When ``follow_naming_convention=True`` is enabled, GeoAlchemy 2 explicitly
names the index in the DDL statement (``ALTER TABLE <table> ADD SPATIAL INDEX <index_name> (<column>)``).

SQLite / SpatiaLite and GeoPackage
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- SpatiaLite implements spatial indexing via R-tree virtual tables managed by C-extension functions
(``CreateSpatialIndex`` / ``DisableSpatialIndex``).
- The names of these virtual tables and their internal triggers are fixed by SpatiaLite
(``idx_<tablename>_<columnname>``). As a result, naming conventions cannot rename SpatiaLite R-tree
virtual tables.
- GeoPackage likewise uses fixed R-tree virtual table names (``rtree_<tablename>_<columnname>``)
managed via ``gpkgAddSpatialIndex``.

MSSQL
^^^^^

- SQL Server requires spatial indexes to be created with a bounding box and tessellation grid scheme
(``GEOMETRY_GRID`` or ``GEOGRAPHY_GRID``).
- GeoAlchemy 2 automatically constructs and executes the required ``CREATE SPATIAL INDEX`` statement
after the table is created, applying the resolved naming convention.


Using with Alembic Migrations
-----------------------------

When using Alembic for database migrations, autogenerated spatial indexes interact with Alembic's
autogenerate comparisons:

- Ensure you register :obj:`~geoalchemy2.alembic_helpers.writer` in your Alembic ``env.py`` to handle
spatial index operations and suppress false diffs on SQLite/GeoPackage.
- When switching an existing database from default naming to ``follow_naming_convention=True``, Alembic
will generate a migration to drop the old ``idx_<tablename>_<columnname>`` index and create the new
convention-named index.

For detailed migration setup and examples, see :ref:`alembic_use`.
19 changes: 13 additions & 6 deletions geoalchemy2/admin/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ def after_parent_attach(column, table):
kwargs = {
"postgresql_using": "gist",
"_column_flag": True,
"info": {"geoalchemy2_spatial_index": True},
}
col = column
if _check_spatial_type(column.type, (Geometry, Geography)):
Expand All @@ -91,13 +92,19 @@ def after_parent_attach(column, table):
elif _check_spatial_type(column.type, Raster):
col = func.ST_ConvexHull(column)

table.append_constraint(
Index(
_spatial_idx_name(table.name, column.name),
col,
**kwargs,
)
name = (
None
if getattr(column.type, "follow_naming_convention", False)
else _spatial_idx_name(table.name, column.name)
)
idx = Index(
name,
col,
**kwargs,
)
table.append_constraint(idx)
if idx.name is None:
idx.name = _spatial_idx_name(table.name, column.name)

@event.listens_for(Table, "column_reflect")
def column_reflect(inspector, table, column_info):
Expand Down
5 changes: 5 additions & 0 deletions geoalchemy2/admin/dialects/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ def _spatial_idx_name(table_name, column_name):
return f"idx_{table_name}_{column_name}"


def _is_generated_spatial_index(idx):
"""Check whether an index is an autogenerated GeoAlchemy2 spatial index."""
return bool(idx.info.get("geoalchemy2_spatial_index", False))


def _format_select_args(*args):
if _SQLALCHEMY_VERSION_BEFORE_14:
return [args]
Expand Down
7 changes: 5 additions & 2 deletions geoalchemy2/admin/dialects/geopackage.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from geoalchemy2 import functions
from geoalchemy2.admin.dialects.common import _check_spatial_type
from geoalchemy2.admin.dialects.common import _format_select_args
from geoalchemy2.admin.dialects.common import _spatial_idx_name
from geoalchemy2.admin.dialects.common import _is_generated_spatial_index
from geoalchemy2.admin.dialects.common import setup_create_drop
from geoalchemy2.admin.dialects.sqlite import _SQLITE_FUNCTIONS
from geoalchemy2.admin.dialects.sqlite import _compile_GeomFromWKB_SQLite
Expand Down Expand Up @@ -195,15 +195,18 @@ def before_create(table, bind, **kw):
# created during the table.create() step since the associated columns do not exist
# at this time.
table.info["_after_create_indexes"] = []
generated = table.info.setdefault("_generated_spatial_indexes", {})
current_indexes = set(table.indexes)
for idx in current_indexes:
for col in table.info["_saved_columns"]:
if _check_spatial_type(col.type, Geometry, dialect) and col in idx.columns.values():
table.indexes.remove(idx)
if idx.name != _spatial_idx_name(table.name, col.name) or not getattr(
if not _is_generated_spatial_index(idx) or not getattr(
col.type, "spatial_index", False
):
table.info["_after_create_indexes"].append(idx)
else:
generated[col.name] = idx

if len(gis_cols) > 1:
raise ValueError("Only one geometry column is allowed for a table stored in a GeoPackage.")
Expand Down
Loading