convert svn to hg
This commit is contained in:
iElectric
commit
9cedba8b8b
3
MANIFEST.in
Normal file
3
MANIFEST.in
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
include CHANGELOG
|
||||||
|
include docs/*.rst
|
||||||
|
include docs/conf.py
|
||||||
35
README
Normal file
35
README
Normal file
@ -0,0 +1,35 @@
|
|||||||
|
Inspired by Ruby on Rails' migrations, Migrate provides a way to deal with database schema changes in `SQLAlchemy <http://sqlalchemy.org>`_ projects.
|
||||||
|
|
||||||
|
Migrate extends SQLAlchemy to have database changeset handling. It provides a database change repository mechanism which can be used from the command line as well as from inside python code.
|
||||||
|
|
||||||
|
Help
|
||||||
|
----
|
||||||
|
|
||||||
|
Sphinx documentation is available at the project page `packages.python.org <http://packages.python.org/sqlalchemy-migrate/>`_.
|
||||||
|
|
||||||
|
Users and developers can be found at #sqlalchemy-migrate on Freenode IRC network
|
||||||
|
and at the public users mailing list `migrate-users <http://groups.google.com/group/migrate-users>`_.
|
||||||
|
|
||||||
|
New releases and major changes are announced at the public announce
|
||||||
|
mailing list `migrate-announce <http://groups.google.com/group/migrate-announce>`_
|
||||||
|
and at the Python package index `sqlalchemy-migrate <http://pypi.python.org/pypi/sqlalchemy-migrate>`_.
|
||||||
|
|
||||||
|
Homepage is located at `code.google.com <http://code.google.com/p/sqlalchemy-migrate/>`_
|
||||||
|
|
||||||
|
You can also download `development version <http://sqlalchemy-migrate.googlecode.com/svn/trunk/#egg=sqlalchemy-migrate-dev>` from SVN trunk.
|
||||||
|
|
||||||
|
|
||||||
|
Tests and Bugs
|
||||||
|
--------------
|
||||||
|
|
||||||
|
To run automated tests:
|
||||||
|
|
||||||
|
- Copy test_db.cfg.tmpl to test_db.cfg
|
||||||
|
- Edit test_db.cfg with database connection strings suitable for running tests. (Use empty databases.)
|
||||||
|
- python setup.py test
|
||||||
|
|
||||||
|
Note that `nose <http://somethingaboutorange.com/mrl/projects/nose/>`_ is required to run migrate's tests. It should be
|
||||||
|
installed automatically; if not, try "easy_install nose".
|
||||||
|
|
||||||
|
Please report any issues with sqlalchemy-migrate to the issue tracker
|
||||||
|
at `code.google.com issues <http://code.google.com/p/sqlalchemy-migrate/issues/list>`_
|
||||||
8
TODO
Normal file
8
TODO
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
- better SQL scripts support (testing, source viewing)
|
||||||
|
|
||||||
|
make_update_script_for_model:
|
||||||
|
- calculated differences between models are actually differences between metas
|
||||||
|
- columns are not compared?
|
||||||
|
- even if two "models" are equal, it doesn't yield so
|
||||||
|
|
||||||
|
- controlledschema.drop() drops whole migrate table, maybe there are some other repositories bound to it!
|
||||||
75
docs/Makefile
Normal file
75
docs/Makefile
Normal file
@ -0,0 +1,75 @@
|
|||||||
|
# Makefile for Sphinx documentation
|
||||||
|
#
|
||||||
|
|
||||||
|
# You can set these variables from the command line.
|
||||||
|
SPHINXOPTS =
|
||||||
|
SPHINXBUILD = sphinx-build
|
||||||
|
PAPER =
|
||||||
|
|
||||||
|
# Internal variables.
|
||||||
|
PAPEROPT_a4 = -D latex_paper_size=a4
|
||||||
|
PAPEROPT_letter = -D latex_paper_size=letter
|
||||||
|
ALLSPHINXOPTS = -d _build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
||||||
|
|
||||||
|
.PHONY: help clean html web pickle htmlhelp latex changes linkcheck
|
||||||
|
|
||||||
|
help:
|
||||||
|
@echo "Please use \`make <target>' where <target> is one of"
|
||||||
|
@echo " html to make standalone HTML files"
|
||||||
|
@echo " pickle to make pickle files"
|
||||||
|
@echo " json to make JSON files"
|
||||||
|
@echo " htmlhelp to make HTML files and a HTML help project"
|
||||||
|
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
|
||||||
|
@echo " changes to make an overview over all changed/added/deprecated items"
|
||||||
|
@echo " linkcheck to check all external links for integrity"
|
||||||
|
|
||||||
|
clean:
|
||||||
|
-rm -rf _build/*
|
||||||
|
|
||||||
|
html:
|
||||||
|
mkdir -p _build/html _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) _build/html
|
||||||
|
@echo
|
||||||
|
@echo "Build finished. The HTML pages are in _build/html."
|
||||||
|
|
||||||
|
pickle:
|
||||||
|
mkdir -p _build/pickle _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) _build/pickle
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can process the pickle files."
|
||||||
|
|
||||||
|
web: pickle
|
||||||
|
|
||||||
|
json:
|
||||||
|
mkdir -p _build/json _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) _build/json
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can process the JSON files."
|
||||||
|
|
||||||
|
htmlhelp:
|
||||||
|
mkdir -p _build/htmlhelp _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) _build/htmlhelp
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can run HTML Help Workshop with the" \
|
||||||
|
".hhp project file in _build/htmlhelp."
|
||||||
|
|
||||||
|
latex:
|
||||||
|
mkdir -p _build/latex _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) _build/latex
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; the LaTeX files are in _build/latex."
|
||||||
|
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
|
||||||
|
"run these through (pdf)latex."
|
||||||
|
|
||||||
|
changes:
|
||||||
|
mkdir -p _build/changes _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) _build/changes
|
||||||
|
@echo
|
||||||
|
@echo "The overview file is in _build/changes."
|
||||||
|
|
||||||
|
linkcheck:
|
||||||
|
mkdir -p _build/linkcheck _build/doctrees
|
||||||
|
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) _build/linkcheck
|
||||||
|
@echo
|
||||||
|
@echo "Link check complete; look for any errors in the above output " \
|
||||||
|
"or in _build/linkcheck/output.txt."
|
||||||
199
docs/api.rst
Normal file
199
docs/api.rst
Normal file
@ -0,0 +1,199 @@
|
|||||||
|
Module :mod:`migrate.changeset`
|
||||||
|
===============================
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset
|
||||||
|
:members:
|
||||||
|
:synopsis: Database changeset management
|
||||||
|
|
||||||
|
Module :mod:`ansisql <migrate.changeset.ansisql>`
|
||||||
|
-------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.ansisql
|
||||||
|
:members:
|
||||||
|
:member-order: groupwise
|
||||||
|
:synopsis: Standard SQL implementation for altering database schemas
|
||||||
|
|
||||||
|
Module :mod:`constraint <migrate.changeset.constraint>`
|
||||||
|
-------------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.constraint
|
||||||
|
:members:
|
||||||
|
:show-inheritance:
|
||||||
|
:member-order: groupwise
|
||||||
|
:synopsis: Standalone schema constraint objects
|
||||||
|
|
||||||
|
Module :mod:`databases <migrate.changeset.databases>`
|
||||||
|
-----------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases
|
||||||
|
:members:
|
||||||
|
:synopsis: Database specific changeset implementations
|
||||||
|
|
||||||
|
.. _mysql-d:
|
||||||
|
|
||||||
|
Module :mod:`mysql <migrate.changeset.databases.mysql>`
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases.mysql
|
||||||
|
:members:
|
||||||
|
:synopsis: MySQL database specific changeset implementations
|
||||||
|
|
||||||
|
.. _firebird-d:
|
||||||
|
|
||||||
|
Module :mod:`firebird <migrate.changeset.databases.firebird>`
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases.firebird
|
||||||
|
:members:
|
||||||
|
:synopsis: Firebird database specific changeset implementations
|
||||||
|
|
||||||
|
.. _oracle-d:
|
||||||
|
|
||||||
|
Module :mod:`oracle <migrate.changeset.databases.oracle>`
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases.oracle
|
||||||
|
:members:
|
||||||
|
:synopsis: Oracle database specific changeset implementations
|
||||||
|
|
||||||
|
.. _postgres-d:
|
||||||
|
|
||||||
|
Module :mod:`postgres <migrate.changeset.databases.postgres>`
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases.postgres
|
||||||
|
:members:
|
||||||
|
:synopsis: PostgreSQL database specific changeset implementations
|
||||||
|
|
||||||
|
.. _sqlite-d:
|
||||||
|
|
||||||
|
Module :mod:`sqlite <migrate.changeset.databases.sqlite>`
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases.sqlite
|
||||||
|
:members:
|
||||||
|
:synopsis: SQLite database specific changeset implementations
|
||||||
|
|
||||||
|
Module :mod:`visitor <migrate.changeset.databases.visitor>`
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.databases.visitor
|
||||||
|
:members:
|
||||||
|
|
||||||
|
Module :mod:`exceptions <migrate.changeset.exceptions>`
|
||||||
|
-------------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.exceptions
|
||||||
|
:members:
|
||||||
|
:synopsis: Changeset exception classes
|
||||||
|
|
||||||
|
Module :mod:`schema <migrate.changeset.schema>`
|
||||||
|
-----------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.changeset.schema
|
||||||
|
:members:
|
||||||
|
:synopsis: Schema changeset handling functions
|
||||||
|
|
||||||
|
|
||||||
|
Module :mod:`migrate.versioning`
|
||||||
|
================================
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning
|
||||||
|
:members:
|
||||||
|
:synopsis: Database version and repository management
|
||||||
|
|
||||||
|
Module :mod:`api <migrate.versioning.api>`
|
||||||
|
------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.api
|
||||||
|
:members:
|
||||||
|
:synopsis: External API for :mod:`migrate.versioning`
|
||||||
|
|
||||||
|
Module :mod:`exceptions <migrate.versioning.exceptions>`
|
||||||
|
--------------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.exceptions
|
||||||
|
:members:
|
||||||
|
:synopsis: Exception classes for :mod:`migrate.versioning`
|
||||||
|
|
||||||
|
Module :mod:`genmodel <migrate.versioning.genmodel>`
|
||||||
|
----------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.genmodel
|
||||||
|
:members:
|
||||||
|
:synopsis: Python database model generator and differencer
|
||||||
|
|
||||||
|
Module :mod:`pathed <migrate.versioning.pathed>`
|
||||||
|
------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.pathed
|
||||||
|
:members:
|
||||||
|
:synopsis: File/Directory handling class
|
||||||
|
|
||||||
|
Module :mod:`repository <migrate.versioning.repository>`
|
||||||
|
--------------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.repository
|
||||||
|
:members:
|
||||||
|
:synopsis: SQLAlchemy migrate repository management
|
||||||
|
:member-order: groupwise
|
||||||
|
|
||||||
|
Module :mod:`schema <migrate.versioning.schema>`
|
||||||
|
------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.schema
|
||||||
|
:members:
|
||||||
|
:member-order: groupwise
|
||||||
|
:synopsis: Database schema management
|
||||||
|
|
||||||
|
Module :mod:`schemadiff <migrate.versioning.schemadiff>`
|
||||||
|
--------------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.schemadiff
|
||||||
|
:members:
|
||||||
|
:synopsis: Database schema and model differencing
|
||||||
|
|
||||||
|
Module :mod:`script <migrate.versioning.script>`
|
||||||
|
------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.script.base
|
||||||
|
:synopsis: Script utilities
|
||||||
|
:member-order: groupwise
|
||||||
|
:members:
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.script.py
|
||||||
|
:members:
|
||||||
|
:member-order: groupwise
|
||||||
|
:inherited-members:
|
||||||
|
:show-inheritance:
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.script.sql
|
||||||
|
:members:
|
||||||
|
:member-order: groupwise
|
||||||
|
:show-inheritance:
|
||||||
|
:inherited-members:
|
||||||
|
|
||||||
|
Module :mod:`shell <migrate.versioning.shell>`
|
||||||
|
----------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.shell
|
||||||
|
:members:
|
||||||
|
:synopsis: Shell commands
|
||||||
|
|
||||||
|
Module :mod:`util <migrate.versioning.util>`
|
||||||
|
--------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.util
|
||||||
|
:members:
|
||||||
|
:synopsis: Utility functions
|
||||||
|
|
||||||
|
Module :mod:`version <migrate.versioning.version>`
|
||||||
|
--------------------------------------------------
|
||||||
|
|
||||||
|
.. automodule:: migrate.versioning.version
|
||||||
|
:members:
|
||||||
|
:member-order: groupwise
|
||||||
|
:synopsis: Version management
|
||||||
152
docs/changelog.rst
Normal file
152
docs/changelog.rst
Normal file
@ -0,0 +1,152 @@
|
|||||||
|
0.5.5
|
||||||
|
-----
|
||||||
|
|
||||||
|
- url parameter can also be an Engine instance (this usage is discouraged though sometimes necessary)
|
||||||
|
- added support for SQLAlchemy 0.6 (missing oracle and firebird) by Michael Bayer
|
||||||
|
- alter, create, drop column / rename table / rename index constructs now accept `alter_metadata` parameter. If True, it will modify Column/Table objects according to changes. Otherwise, everything will be untouched.
|
||||||
|
- complete refactoring of :class:`~migrate.changeset.schema.ColumnDelta` (fixes issue 23)
|
||||||
|
- added support for :ref:`firebird <firebird-d>`
|
||||||
|
- fixed bug when :meth:`Column.alter <migrate.changeset.schema.ChangesetColumn.alter>`\(server_default='string') was not properly set
|
||||||
|
- `server_defaults` passed to :meth:`Column.create <migrate.changeset.schema.ChangesetColumn.create>` are now issued correctly
|
||||||
|
- constraints passed to :meth:`Column.create <migrate.changeset.schema.ChangesetColumn.create>` are correctly interpreted (``ALTER TABLE ADD CONSTRAINT`` is issued after ``ATLER TABLE ADD COLUMN``)
|
||||||
|
- :meth:`Column.create <migrate.changeset.schema.ChangesetColumn.create>` accepts `primary_key_name`, `unique_name` and `index_name` as string value which is used as contraint name when adding a column
|
||||||
|
- Constraint classes have `cascade=True` keyword argument to issue ``DROP CASCADE`` where supported
|
||||||
|
- added :class:`~migrate.changeset.constraint.UniqueConstraint`/:class:`~migrate.changeset.constraint.CheckConstraint` and corresponding create/drop methods
|
||||||
|
- use SQLAlchemy quoting system to avoid name conflicts (for issue 32)
|
||||||
|
- code coverage is up to 99% with more than 100 tests
|
||||||
|
- partial refactoring of :mod:`changeset` package
|
||||||
|
- majoy update to documentation
|
||||||
|
- :ref:`dialect support <dialect-support>` table was added to documentation
|
||||||
|
|
||||||
|
.. _backwards-055:
|
||||||
|
|
||||||
|
**Backward incompatible changes**:
|
||||||
|
|
||||||
|
- python upgrade/downgrade scripts do not import `migrate_engine` magically, but recieve engine as the only parameter to function (eg. ``def upgrade(migrate_engine):``)
|
||||||
|
- :meth:`Column.alter <migrate.changeset.schema.ChangesetColumn.alter>` does not accept `current_name` anymore, it extracts name from the old column.
|
||||||
|
|
||||||
|
0.5.4
|
||||||
|
-----
|
||||||
|
|
||||||
|
- fixed preview_sql parameter for downgrade/upgrade. Now it prints SQL if the step is SQL script and runs step with mocked engine to only print SQL statements if ORM is used. [Domen Kozar]
|
||||||
|
- use entrypoints terminology to specify dotted model names (module.model:User) [Domen Kozar]
|
||||||
|
- added engine_dict and engine_arg_* parameters to all api functions (deprecated echo) [Domen Kozar]
|
||||||
|
- make --echo parameter a bit more forgivable (better Python API support) [Domen Kozar]
|
||||||
|
- apply patch to refactor cmd line parsing for Issue 54 by Domen Kozar
|
||||||
|
|
||||||
|
0.5.3
|
||||||
|
-----
|
||||||
|
|
||||||
|
- apply patch for Issue 29 by Jonathan Ellis
|
||||||
|
- fix Issue 52 by removing needless parameters from object.__new__ calls
|
||||||
|
|
||||||
|
0.5.2
|
||||||
|
-----
|
||||||
|
|
||||||
|
- move sphinx and nose dependencies to extras_require and tests_require
|
||||||
|
- integrate patch for Issue 36 by Kumar McMillan
|
||||||
|
- fix unit tests
|
||||||
|
- mark ALTER TABLE ADD COLUMN with FOREIGN KEY as not supported by SQLite
|
||||||
|
|
||||||
|
0.5.1.2
|
||||||
|
-------
|
||||||
|
|
||||||
|
- corrected build
|
||||||
|
|
||||||
|
0.5.1.1
|
||||||
|
-------
|
||||||
|
|
||||||
|
- add documentation in tarball
|
||||||
|
- add a MANIFEST.in
|
||||||
|
|
||||||
|
0.5.1
|
||||||
|
-----
|
||||||
|
|
||||||
|
- SA 0.5.x support. SQLAlchemy < 0.5.1 not supported anymore.
|
||||||
|
- use nose instead of py.test for testing
|
||||||
|
- Added --echo=True option for all commands, which will make the sqlalchemy connection echo SQL statements.
|
||||||
|
- Better PostgreSQL support, especially for schemas.
|
||||||
|
- modification to the downgrade command to simplify the calling (old way still works just fine)
|
||||||
|
- improved support for SQLite
|
||||||
|
- add support for check constraints (EXPERIMENTAL)
|
||||||
|
- print statements removed from APIs
|
||||||
|
- improved sphinx based documentation
|
||||||
|
- removal of old commented code
|
||||||
|
- PEP-8 clean code
|
||||||
|
|
||||||
|
0.4.5
|
||||||
|
-----
|
||||||
|
|
||||||
|
- work by Christian Simms to compare metadata against databases
|
||||||
|
- new repository format
|
||||||
|
- a repository format migration tool is in migrate/versioning/migrate_repository.py
|
||||||
|
- support for default SQL scripts
|
||||||
|
- EXPERIMENTAL support for dumping database to model
|
||||||
|
|
||||||
|
0.4.4
|
||||||
|
-----
|
||||||
|
|
||||||
|
- patch by pwannygoodness for Issue #15
|
||||||
|
- fixed unit tests to work with py.test 0.9.1
|
||||||
|
- fix for a SQLAlchemy deprecation warning
|
||||||
|
|
||||||
|
0.4.3
|
||||||
|
-----
|
||||||
|
|
||||||
|
- patch by Kevin Dangoor to handle database versions as packages and ignore their __init__.py files in version.py
|
||||||
|
- fixed unit tests and Oracle changeset support by Christian Simms
|
||||||
|
|
||||||
|
0.4.2
|
||||||
|
-----
|
||||||
|
|
||||||
|
- package name is sqlalchemy-migrate again to make pypi work
|
||||||
|
- make import of sqlalchemy's SchemaGenerator work regardless of previous imports
|
||||||
|
|
||||||
|
0.4.1
|
||||||
|
-----
|
||||||
|
|
||||||
|
- setuptools patch by Kevin Dangoor
|
||||||
|
- re-rename module to migrate
|
||||||
|
|
||||||
|
0.4.0
|
||||||
|
-----
|
||||||
|
|
||||||
|
- SA 0.4.0 compatibility thanks to Christian Simms
|
||||||
|
- all unit tests are working now (with sqlalchemy >= 0.3.10)
|
||||||
|
|
||||||
|
0.3
|
||||||
|
---
|
||||||
|
|
||||||
|
- SA 0.3.10 compatibility
|
||||||
|
|
||||||
|
0.2.3
|
||||||
|
-----
|
||||||
|
|
||||||
|
- Removed lots of SA monkeypatching in Migrate's internals
|
||||||
|
- SA 0.3.3 compatibility
|
||||||
|
- Removed logsql (#75)
|
||||||
|
- Updated py.test version from 0.8 to 0.9; added a download link to setup.py
|
||||||
|
- Fixed incorrect "function not defined" error (#88)
|
||||||
|
- Fixed SQLite and .sql scripts (#87)
|
||||||
|
|
||||||
|
0.2.2
|
||||||
|
-----
|
||||||
|
|
||||||
|
- Deprecated driver(engine) in favor of engine.name (#80)
|
||||||
|
- Deprecated logsql (#75)
|
||||||
|
- Comments in .sql scripts don't make things fail silently now (#74)
|
||||||
|
- Errors while downgrading (and probably other places) are shown on their own line
|
||||||
|
- Created mailing list and announcements list, updated documentation accordingly
|
||||||
|
- Automated tests now require py.test (#66)
|
||||||
|
- Documentation fix to .sql script commits (#72)
|
||||||
|
- Fixed a pretty major bug involving logengine, dealing with commits/tests (#64)
|
||||||
|
- Fixes to the online docs - default DB versioning table name (#68)
|
||||||
|
- Fixed the engine name in the scripts created by the command 'migrate script' (#69)
|
||||||
|
- Added Evan's email to the online docs
|
||||||
|
|
||||||
|
0.2.1
|
||||||
|
-----
|
||||||
|
|
||||||
|
- Created this changelog
|
||||||
|
- Now requires (and is now compatible with) SA 0.3
|
||||||
|
- Commits across filesystems now allowed (shutil.move instead of os.rename) (#62)
|
||||||
195
docs/changeset.rst
Normal file
195
docs/changeset.rst
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
.. _changeset-system:
|
||||||
|
.. highlight:: python
|
||||||
|
|
||||||
|
******************
|
||||||
|
Database changeset
|
||||||
|
******************
|
||||||
|
|
||||||
|
.. currentmodule:: migrate.changeset.schema
|
||||||
|
|
||||||
|
Importing :mod:`migrate.changeset` adds some new methods to existing
|
||||||
|
SA objects, as well as creating functions of its own. Most operations
|
||||||
|
can be done either by a method or a function. Methods match
|
||||||
|
SQLAlchemy's existing API and are more intuitive when the object is
|
||||||
|
available; functions allow one to make changes when only the name of
|
||||||
|
an object is available (for example, adding a column to a table in the
|
||||||
|
database without having to load that table into Python).
|
||||||
|
|
||||||
|
Changeset operations can be used independently of SQLAlchemy Migrate's
|
||||||
|
:ref:`versioning <versioning-system>`.
|
||||||
|
|
||||||
|
For more information, see the generated documentation for
|
||||||
|
:mod:`migrate.changeset`.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
alter_metadata keyword defaults to True.
|
||||||
|
|
||||||
|
Column
|
||||||
|
======
|
||||||
|
|
||||||
|
Given a standard SQLAlchemy table::
|
||||||
|
|
||||||
|
table = Table('mytable', meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
)
|
||||||
|
table.create()
|
||||||
|
|
||||||
|
.. _column-create:
|
||||||
|
|
||||||
|
:meth:`Create a column <ChangesetColumn.create>`::
|
||||||
|
|
||||||
|
col = Column('col1', String)
|
||||||
|
col.create(table)
|
||||||
|
|
||||||
|
# Column is added to table based on its name
|
||||||
|
assert col is table.c.col1
|
||||||
|
|
||||||
|
.. _column-drop:
|
||||||
|
|
||||||
|
:meth:`Drop a column <ChangesetColumn.drop>`::
|
||||||
|
|
||||||
|
col.drop()
|
||||||
|
|
||||||
|
|
||||||
|
.. _column-alter:
|
||||||
|
|
||||||
|
:meth:`Alter a column <ChangesetColumn.alter>`::
|
||||||
|
|
||||||
|
col.alter(name='col2')
|
||||||
|
|
||||||
|
# Renaming a column affects how it's accessed by the table object
|
||||||
|
assert col is table.c.col2
|
||||||
|
|
||||||
|
# Other properties can be modified as well
|
||||||
|
col.alter(type=String(42), default="life, the universe, and everything", nullable=False)
|
||||||
|
|
||||||
|
# Given another column object, col1.alter(col2), col1 will be changed to match col2
|
||||||
|
col.alter(Column('col3', String(77), nullable=True))
|
||||||
|
assert col.nullable
|
||||||
|
assert table.c.col3 is col
|
||||||
|
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Since version ``0.5.5`` you can pass primary_key_name, index_name and unique_name to column.create method to issue ALTER TABLE ADD CONSTRAINT after changing the column. Note for multi columns constraints and other advanced configuration, check :ref:`constraint tutorial <constraint-tutorial>`.
|
||||||
|
|
||||||
|
.. _table-rename:
|
||||||
|
|
||||||
|
Table
|
||||||
|
=====
|
||||||
|
|
||||||
|
SQLAlchemy supports `table create/drop`_.
|
||||||
|
|
||||||
|
:meth:`Rename a table <ChangesetTable.rename>`::
|
||||||
|
|
||||||
|
table.rename('newtablename')
|
||||||
|
|
||||||
|
.. _`table create/drop`: http://www.sqlalchemy.org/docs/05/metadata.html#creating-and-dropping-database-tables
|
||||||
|
.. currentmodule:: migrate.changeset.constraint
|
||||||
|
|
||||||
|
|
||||||
|
.. _index-rename:
|
||||||
|
|
||||||
|
Index
|
||||||
|
=====
|
||||||
|
|
||||||
|
SQLAlchemy supports `index create/drop`_.
|
||||||
|
|
||||||
|
:meth:`Rename an index <migrate.changeset.schema.ChangesetIndex.rename>`, given an SQLAlchemy ``Index`` object::
|
||||||
|
|
||||||
|
index.rename('newindexname')
|
||||||
|
|
||||||
|
.. _`index create/drop`: http://www.sqlalchemy.org/docs/05/metadata.html#indexes
|
||||||
|
|
||||||
|
|
||||||
|
.. _constraint-tutorial:
|
||||||
|
|
||||||
|
Constraint
|
||||||
|
==========
|
||||||
|
|
||||||
|
SQLAlchemy supports creating/dropping constraints at the same time a table is created/dropped. SQLAlchemy Migrate adds support for creating/dropping :class:`PrimaryKeyConstraint`/:class:`ForeignKeyConstraint`/:class:`CheckConstraint`/:class:`UniqueConstraint` constraints independently. (as ALTER TABLE statements).
|
||||||
|
|
||||||
|
The following rundowns are true for all constraints classes:
|
||||||
|
|
||||||
|
1. Make sure you do ``from migrate.changeset import *`` after SQLAlchemy imports since `migrate` does not patch SA's Constraints.
|
||||||
|
|
||||||
|
2. You can also use Constraints as in SQLAlchemy. In this case passing table argument explicitly is required::
|
||||||
|
|
||||||
|
cons = PrimaryKeyConstraint('id', 'num', table=self.table)
|
||||||
|
|
||||||
|
# Create the constraint
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
# Drop the constraint
|
||||||
|
cons.drop()
|
||||||
|
|
||||||
|
or you can pass in column objects (and table argument can be left out)::
|
||||||
|
|
||||||
|
cons = PrimaryKeyConstraint(col1, col2)
|
||||||
|
|
||||||
|
3. Some dialects support CASCADE option when dropping constraints::
|
||||||
|
|
||||||
|
cons = PrimaryKeyConstraint(col1, col2)
|
||||||
|
|
||||||
|
# Create the constraint
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
# Drop the constraint
|
||||||
|
cons.drop(cascade=True)
|
||||||
|
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
SQLAlchemy Migrate will try to guess the name of the constraints for databases, but if it's something other than the default, you'll need to give its name. Best practice is to always name your constraints. Note that Oracle requires that you state the name of the constraint to be created/dropped.
|
||||||
|
|
||||||
|
|
||||||
|
Examples
|
||||||
|
---------
|
||||||
|
|
||||||
|
Primary key constraints::
|
||||||
|
|
||||||
|
from migrate.changeset import *
|
||||||
|
|
||||||
|
cons = PrimaryKeyConstraint(col1, col2)
|
||||||
|
|
||||||
|
# Create the constraint
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
# Drop the constraint
|
||||||
|
cons.drop()
|
||||||
|
|
||||||
|
Foreign key constraints::
|
||||||
|
|
||||||
|
from migrate.changeset import *
|
||||||
|
|
||||||
|
cons = ForeignKeyConstraint([table.c.fkey], [othertable.c.id])
|
||||||
|
|
||||||
|
# Create the constraint
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
# Drop the constraint
|
||||||
|
cons.drop()
|
||||||
|
|
||||||
|
Check constraints::
|
||||||
|
|
||||||
|
from migrate.changeset import *
|
||||||
|
|
||||||
|
cons = CheckConstraint('id > 3', columns=[table.c.id])
|
||||||
|
|
||||||
|
# Create the constraint
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
# Drop the constraint
|
||||||
|
cons.drop()
|
||||||
|
|
||||||
|
Unique constraints::
|
||||||
|
|
||||||
|
from migrate.changeset import *
|
||||||
|
|
||||||
|
cons = UniqueConstraint('id', 'age', table=self.table)
|
||||||
|
|
||||||
|
# Create the constraint
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
# Drop the constraint
|
||||||
|
cons.drop()
|
||||||
195
docs/conf.py
Normal file
195
docs/conf.py
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
#
|
||||||
|
# SQLAlchemy Migrate documentation build configuration file, created by
|
||||||
|
# sphinx-quickstart on Fri Feb 13 12:58:57 2009.
|
||||||
|
#
|
||||||
|
# This file is execfile()d with the current directory set to its containing dir.
|
||||||
|
#
|
||||||
|
# The contents of this file are pickled, so don't put values in the namespace
|
||||||
|
# that aren't pickleable (module imports are okay, they're removed automatically).
|
||||||
|
#
|
||||||
|
# Note that not all possible configuration values are present in this
|
||||||
|
# autogenerated file.
|
||||||
|
#
|
||||||
|
# All configuration values have a default; values that are commented out
|
||||||
|
# serve to show the default.
|
||||||
|
|
||||||
|
import sys, os
|
||||||
|
|
||||||
|
# If your extensions are in another directory, add it here. If the directory
|
||||||
|
# is relative to the documentation root, use os.path.abspath to make it
|
||||||
|
# absolute, like shown here.
|
||||||
|
#sys.path.append(os.path.abspath('.'))
|
||||||
|
# Allow module docs to build without having sqlalchemy-migrate installed:
|
||||||
|
sys.path.append(os.path.dirname(os.path.abspath('.')))
|
||||||
|
|
||||||
|
# General configuration
|
||||||
|
# ---------------------
|
||||||
|
|
||||||
|
# Add any Sphinx extension module names here, as strings. They can be extensions
|
||||||
|
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
||||||
|
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx']
|
||||||
|
|
||||||
|
# link to sqlalchemy docs
|
||||||
|
intersphinx_mapping = {'http://www.sqlalchemy.org/docs/05/': None}
|
||||||
|
|
||||||
|
# Add any paths that contain templates here, relative to this directory.
|
||||||
|
templates_path = ['_templates']
|
||||||
|
|
||||||
|
# The suffix of source filenames.
|
||||||
|
source_suffix = '.rst'
|
||||||
|
|
||||||
|
# The encoding of source files.
|
||||||
|
#source_encoding = 'utf-8'
|
||||||
|
|
||||||
|
# The master toctree document.
|
||||||
|
master_doc = 'index'
|
||||||
|
|
||||||
|
# General information about the project.
|
||||||
|
project = u'SQLAlchemy Migrate'
|
||||||
|
copyright = u'2009, Evan Rosson, Jan Dittberner'
|
||||||
|
|
||||||
|
# The version info for the project you're documenting, acts as replacement for
|
||||||
|
# |version| and |release|, also used in various other places throughout the
|
||||||
|
# built documents.
|
||||||
|
#
|
||||||
|
# The short X.Y version.
|
||||||
|
version = '0.5.4'
|
||||||
|
# The full version, including alpha/beta/rc tags.
|
||||||
|
release = '0.5.4'
|
||||||
|
|
||||||
|
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||||
|
# for a list of supported languages.
|
||||||
|
#language = None
|
||||||
|
|
||||||
|
# There are two options for replacing |today|: either, you set today to some
|
||||||
|
# non-false value, then it is used:
|
||||||
|
#today = ''
|
||||||
|
# Else, today_fmt is used as the format for a strftime call.
|
||||||
|
#today_fmt = '%B %d, %Y'
|
||||||
|
|
||||||
|
# List of documents that shouldn't be included in the build.
|
||||||
|
#unused_docs = []
|
||||||
|
|
||||||
|
# List of directories, relative to source directory, that shouldn't be searched
|
||||||
|
# for source files.
|
||||||
|
exclude_trees = ['_build']
|
||||||
|
|
||||||
|
# The reST default role (used for this markup: `text`) to use for all documents.
|
||||||
|
#default_role = None
|
||||||
|
|
||||||
|
# If true, '()' will be appended to :func: etc. cross-reference text.
|
||||||
|
#add_function_parentheses = True
|
||||||
|
|
||||||
|
# If true, the current module name will be prepended to all description
|
||||||
|
# unit titles (such as .. function::).
|
||||||
|
#add_module_names = True
|
||||||
|
|
||||||
|
# If true, sectionauthor and moduleauthor directives will be shown in the
|
||||||
|
# output. They are ignored by default.
|
||||||
|
#show_authors = False
|
||||||
|
|
||||||
|
# The name of the Pygments (syntax highlighting) style to use.
|
||||||
|
pygments_style = 'sphinx'
|
||||||
|
|
||||||
|
|
||||||
|
# Options for HTML output
|
||||||
|
# -----------------------
|
||||||
|
|
||||||
|
# The style sheet to use for HTML and HTML Help pages. A file of that name
|
||||||
|
# must exist either in Sphinx' static/ path, or in one of the custom paths
|
||||||
|
# given in html_static_path.
|
||||||
|
html_style = 'default.css'
|
||||||
|
|
||||||
|
# The name for this set of Sphinx documents. If None, it defaults to
|
||||||
|
# "<project> v<release> documentation".
|
||||||
|
#html_title = None
|
||||||
|
|
||||||
|
# A shorter title for the navigation bar. Default is the same as html_title.
|
||||||
|
#html_short_title = None
|
||||||
|
|
||||||
|
# The name of an image file (relative to this directory) to place at the top
|
||||||
|
# of the sidebar.
|
||||||
|
#html_logo = None
|
||||||
|
|
||||||
|
# The name of an image file (within the static path) to use as favicon of the
|
||||||
|
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
|
||||||
|
# pixels large.
|
||||||
|
#html_favicon = None
|
||||||
|
|
||||||
|
# Add any paths that contain custom static files (such as style sheets) here,
|
||||||
|
# relative to this directory. They are copied after the builtin static files,
|
||||||
|
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||||
|
html_static_path = ['_static']
|
||||||
|
|
||||||
|
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
|
||||||
|
# using the given strftime format.
|
||||||
|
#html_last_updated_fmt = '%b %d, %Y'
|
||||||
|
|
||||||
|
# If true, SmartyPants will be used to convert quotes and dashes to
|
||||||
|
# typographically correct entities.
|
||||||
|
#html_use_smartypants = True
|
||||||
|
|
||||||
|
# Custom sidebar templates, maps document names to template names.
|
||||||
|
#html_sidebars = {}
|
||||||
|
|
||||||
|
# Additional templates that should be rendered to pages, maps page names to
|
||||||
|
# template names.
|
||||||
|
#html_additional_pages = {}
|
||||||
|
|
||||||
|
# If false, no module index is generated.
|
||||||
|
#html_use_modindex = True
|
||||||
|
|
||||||
|
# If false, no index is generated.
|
||||||
|
#html_use_index = True
|
||||||
|
|
||||||
|
# If true, the index is split into individual pages for each letter.
|
||||||
|
#html_split_index = False
|
||||||
|
|
||||||
|
# If true, the reST sources are included in the HTML build as _sources/<name>.
|
||||||
|
#html_copy_source = True
|
||||||
|
|
||||||
|
# If true, an OpenSearch description file will be output, and all pages will
|
||||||
|
# contain a <link> tag referring to it. The value of this option must be the
|
||||||
|
# base URL from which the finished HTML is served.
|
||||||
|
#html_use_opensearch = ''
|
||||||
|
|
||||||
|
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
|
||||||
|
#html_file_suffix = ''
|
||||||
|
|
||||||
|
# Output file base name for HTML help builder.
|
||||||
|
htmlhelp_basename = 'SQLAlchemyMigratedoc'
|
||||||
|
|
||||||
|
|
||||||
|
# Options for LaTeX output
|
||||||
|
# ------------------------
|
||||||
|
|
||||||
|
# The paper size ('letter' or 'a4').
|
||||||
|
#latex_paper_size = 'letter'
|
||||||
|
|
||||||
|
# The font size ('10pt', '11pt' or '12pt').
|
||||||
|
#latex_font_size = '10pt'
|
||||||
|
|
||||||
|
# Grouping the document tree into LaTeX files. List of tuples
|
||||||
|
# (source start file, target name, title, author, document class [howto/manual]).
|
||||||
|
latex_documents = [
|
||||||
|
('index', 'SQLAlchemyMigrate.tex', ur'SQLAlchemy Migrate Documentation',
|
||||||
|
ur'Evan Rosson, Jan Dittberner', 'manual'),
|
||||||
|
]
|
||||||
|
|
||||||
|
# The name of an image file (relative to this directory) to place at the top of
|
||||||
|
# the title page.
|
||||||
|
#latex_logo = None
|
||||||
|
|
||||||
|
# For "manual" documents, if this is true, then toplevel headings are parts,
|
||||||
|
# not chapters.
|
||||||
|
#latex_use_parts = False
|
||||||
|
|
||||||
|
# Additional stuff for the LaTeX preamble.
|
||||||
|
#latex_preamble = ''
|
||||||
|
|
||||||
|
# Documents to append as an appendix to all manuals.
|
||||||
|
#latex_appendices = []
|
||||||
|
|
||||||
|
# If false, no module index is generated.
|
||||||
|
#latex_use_modindex = True
|
||||||
42
docs/download.rst
Normal file
42
docs/download.rst
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
Download
|
||||||
|
--------
|
||||||
|
|
||||||
|
You can get the latest version of SQLAlchemy Migrate from the
|
||||||
|
`project's download page`_, the `cheese shop`_, or via easy_install_::
|
||||||
|
|
||||||
|
easy_install sqlalchemy-migrate
|
||||||
|
|
||||||
|
You should now be able to use the *migrate* command from the command
|
||||||
|
line::
|
||||||
|
|
||||||
|
migrate
|
||||||
|
|
||||||
|
This should list all available commands. ``migrate help COMMAND`` will
|
||||||
|
display more information about each command.
|
||||||
|
|
||||||
|
If you'd like to be notified when new versions of SQLAlchemy Migrate
|
||||||
|
are released, subscribe to `migrate-announce`_.
|
||||||
|
|
||||||
|
.. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install
|
||||||
|
.. _sqlalchemy: http://www.sqlalchemy.org/download.html
|
||||||
|
.. _`project's download page`: http://code.google.com/p/sqlalchemy-migrate/downloads/list
|
||||||
|
.. _`cheese shop`: http://pypi.python.org/pypi/sqlalchemy-migrate
|
||||||
|
.. _`migrate-announce`: http://groups.google.com/group/migrate-announce
|
||||||
|
|
||||||
|
Development
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Migrate's Subversion_ repository is at
|
||||||
|
http://sqlalchemy-migrate.googlecode.com/svn/
|
||||||
|
|
||||||
|
To get the latest trunk::
|
||||||
|
|
||||||
|
svn co http://sqlalchemy-migrate.googlecode.com/svn/trunk
|
||||||
|
|
||||||
|
Patches should be submitted to the `issue tracker`_.
|
||||||
|
|
||||||
|
We use `buildbot`_ to help us run tests on all databases that migrate supports.
|
||||||
|
|
||||||
|
.. _subversion: http://subversion.tigris.org/
|
||||||
|
.. _issue tracker: http://code.google.com/p/sqlalchemy-migrate/issues/list
|
||||||
|
.. _buildbot: http://buildbot.fubar.si
|
||||||
26
docs/historical/ProjectDesignDecisionsAutomation.trac
Normal file
26
docs/historical/ProjectDesignDecisionsAutomation.trac
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
There are many migrations that don't require a lot of thought - for example, if we add a column to a table definition, we probably want to have an "ALTER TABLE...ADD COLUMN" statement show up in our migration.
|
||||||
|
|
||||||
|
The difficulty lies in the automation of changes where the requirements aren't obvious. What happens when you add a unique constraint to a column whose data is not already unique? What happens when we split an existing table in two? Completely automating database migrations is not possible.
|
||||||
|
|
||||||
|
That said - we shouldn't have to hunt down and handwrite the ALTER TABLE statements for every new column; this is often just tedious. Many other common migration tasks require little serious thought; such tasks are ripe for automation. Any automation attempted, however, should not interfere with our ability to write scripts by hand if we so choose; our tool should ''not'' be centered around automation.
|
||||||
|
|
||||||
|
|
||||||
|
Automatically generating the code for this sort of task seems like a good solution:
|
||||||
|
* It does not obstruct us from writing changes by hand; if we don't like the autogenerated code, delete it or don't generate it to begin with
|
||||||
|
* We can easily add other migration tasks to the autogenerated code
|
||||||
|
* We can see right away if the code is what we're expecting, or if it's wrong
|
||||||
|
* If the generated code is wrong, it is easily modified; we can use parts of the generated code, rather than being required to use either 100% or 0%
|
||||||
|
* Maintence, usually a problem with auto-generated code, is not an issue: old database migration scripts are not the subject of maintenance; the correct solution is usually a new migration script.
|
||||||
|
|
||||||
|
|
||||||
|
Implementation is a problem: finding the 'diff' of two databases to determine what columns to add is not trivial. Fortunately, there exist tools that claim to do this for us: [http://sqlfairy.sourceforge.net/ SQL::Translator] and [http://xml2ddl.berlios.de/ XML to DDL] both claim to have this capability.
|
||||||
|
|
||||||
|
...
|
||||||
|
|
||||||
|
All that said, this is ''not'' something I'm going to attempt during the Summer of Code.
|
||||||
|
* I'd have to rely tremendously on a tool I'm not at all familiar with
|
||||||
|
* Creates a risk of the project itself relying too much on the automation, a Bad Thing
|
||||||
|
* The project has a deadline and I have plenty else to do already
|
||||||
|
* Lots of people with more experience than me say this would take more time than it's worth
|
||||||
|
|
||||||
|
It's something that might be considered for future work if this project is successful, though.
|
||||||
147
docs/historical/ProjectDesignDecisionsScriptFormat.trac
Normal file
147
docs/historical/ProjectDesignDecisionsScriptFormat.trac
Normal file
@ -0,0 +1,147 @@
|
|||||||
|
Important to our system is the API used for making database changes.
|
||||||
|
|
||||||
|
=== Raw SQL; .sql script ===
|
||||||
|
Require users to write raw SQL. Migration scripts are .sql scripts (with database version information in a header comment).
|
||||||
|
|
||||||
|
+ Familiar interface for experienced DBAs.
|
||||||
|
|
||||||
|
+ No new API to learn[[br]]
|
||||||
|
SQL is used elsewhere; many people know SQL already. Those who are still learning SQL will gain expertise not in the API of a specific tool, but in a language which will help them elsewhere. (On the other hand, those who are familiar with Python with no desire to learn SQL might find a Python API more intuitive.)
|
||||||
|
|
||||||
|
- Difficult to extend when necessary[[br]]
|
||||||
|
.sql scripts mean that we can't write new functions specific to our migration system when necessary. (We can't always assume that the DBMS supports functions/procedures.)
|
||||||
|
|
||||||
|
- Lose the power of Python[[br]]
|
||||||
|
Some things are possible in Python that aren't in SQL - for example, suppose we want to use some functions from our application in a migration script. (The user might also simply prefer Python.)
|
||||||
|
|
||||||
|
- Loss of database independence.[[br]]
|
||||||
|
There isn't much we can do to specify different actions for a particular DBMS besides copying the .sql file, which is obviously bad form.
|
||||||
|
|
||||||
|
=== Raw SQL; Python script ===
|
||||||
|
Require users to write raw SQL. Migration scripts are python scripts whose API does little beyond specifying what DBMS(es) a particular statement should apply to.
|
||||||
|
|
||||||
|
For example,
|
||||||
|
{{{
|
||||||
|
run("CREATE TABLE test[...]") # runs for all databases
|
||||||
|
run("ALTER TABLE test ADD COLUMN varchar2[...]",oracle) # runs for Oracle only
|
||||||
|
run("ALTER TABLE test ADD COLUMN varchar[...]",postgres|mysql) # runs for Postgres or MySQL only
|
||||||
|
}}}
|
||||||
|
|
||||||
|
We could also allow parts of a single statement to apply to a specific DBMS:
|
||||||
|
{{{
|
||||||
|
run("ALTER TABLE test ADD COLUMN"+sql("varchar",postgres|mysql)+sql("varchar2",oracle))
|
||||||
|
}}}
|
||||||
|
or, the same thing:
|
||||||
|
{{{
|
||||||
|
run("ALTER TABLE test ADD COLUMN"+sql("varchar",postgres|mysql,"varchar2",oracle))
|
||||||
|
}}}
|
||||||
|
|
||||||
|
+ Allows the user to write migration scripts for multiple DBMSes.
|
||||||
|
|
||||||
|
- The user must manage the conflicts between different databases themselves. [[br]]
|
||||||
|
The user can write scripts to deal with conflicts between databases, but they're not really database-independent: the user has to deal with conflicts between databases; our system doesn't help them.
|
||||||
|
|
||||||
|
+ Minimal new API to learn. [[br]]
|
||||||
|
There is a new API to learn, but it is extremely small, depending mostly on SQL DDL. This has the advantages of "no new API" in our first solution.
|
||||||
|
|
||||||
|
- More verbose than .sql scripts.
|
||||||
|
|
||||||
|
=== Raw SQL; automatic translation between each dialect ===
|
||||||
|
Same as the above suggestion, but allow the user to specify a 'default' dialect of SQL that we'll interpret and whose quirks we'll deal with.
|
||||||
|
That is, write everything in SQL and try to automatically resolve the conflicts of different DBMSes.
|
||||||
|
|
||||||
|
For example, take the following script:
|
||||||
|
{{{
|
||||||
|
engine=postgres
|
||||||
|
|
||||||
|
run("""
|
||||||
|
CREATE TABLE test (
|
||||||
|
id serial
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
}}}
|
||||||
|
Running this on a Postgres database, surprisingly enough, would generate exactly what we typed:
|
||||||
|
{{{
|
||||||
|
CREATE TABLE test (
|
||||||
|
id serial
|
||||||
|
)
|
||||||
|
}}}
|
||||||
|
|
||||||
|
Running it on a MySQL database, however, would generate something like
|
||||||
|
{{{
|
||||||
|
CREATE TABLE test (
|
||||||
|
id integer auto_increment
|
||||||
|
)
|
||||||
|
}}}
|
||||||
|
|
||||||
|
+ Database-independence issues of the above SQL solutions are resolved.[[br]]
|
||||||
|
Ideally, this solution would be as database-independent as a Python API for database changes (discussed next), but with all the advantages of writing SQL (no new API).
|
||||||
|
|
||||||
|
- Difficult implementation[[br]]
|
||||||
|
Obviously, this is not easy to implement - there is a great deal of parsing logic and a great many things that need to be accounted for. In addition, this is a complex operation; any implementation will likely have errors somewhere.
|
||||||
|
|
||||||
|
It seems tools for this already exist; an effective tool would trivialize this implementation. I experimented a bit with [http://sqlfairy.sourceforge.net/ SQL::Translator] and [http://xml2ddl.berlios.de/ XML to DDL]; however, I had difficulties with both.
|
||||||
|
|
||||||
|
- Database-specific features ensure that this cannot possibly be "complete". [[br]]
|
||||||
|
For example, Postgres has an 'interval' type to represent times and (AFAIK) MySQL does not.
|
||||||
|
|
||||||
|
=== Database-independent Python API ===
|
||||||
|
Create a Python API through which we may manage database changes. Scripts would be based on the existing SQLAlchemy API when possible.
|
||||||
|
|
||||||
|
Scripts would look something like
|
||||||
|
{{{
|
||||||
|
# Create a table
|
||||||
|
test_table = table('test'
|
||||||
|
,Column('id',Integer,notNull=True)
|
||||||
|
)
|
||||||
|
table.create()
|
||||||
|
# Add a column to an existing table
|
||||||
|
test_table.add_column('id',Integer,notNull=True)
|
||||||
|
# Or, use a column object instead of its parameters
|
||||||
|
test_table.add_column(Column('id',Integer,notNull=True))
|
||||||
|
# Or, don't use a table object at all
|
||||||
|
add_column('test','id',Integer,notNull=True)
|
||||||
|
}}}
|
||||||
|
This would use engines, similar to SQLAlchemy's, to deal with database-independence issues.
|
||||||
|
|
||||||
|
We would, of course, allow users to write raw SQL if they wish. This would be done in the manner outlined in the second solution above; this allows us to write our entire script in SQL and ignore the Python API if we wish, or write parts of our solution in SQL to deal with specific databases.
|
||||||
|
|
||||||
|
+ Deals with database-independence thoroughly and with minimal user effort.[[br]]
|
||||||
|
SQLAlchemy-style engines would be used for this; issues of different DBMS syntax are resolved with minimal user effort. (Database-specific features would still need handwritten SQL.)
|
||||||
|
|
||||||
|
+ Familiar interface for SQLAlchemy users.[[br]]
|
||||||
|
In addition, we can often cut-and-paste column definitions from SQLAlchemy tables, easing one particular task.
|
||||||
|
|
||||||
|
- Requires that the user learn a new API. [[br]]
|
||||||
|
SQL already exists; people know it. SQL newbies might be more comfortable with a Python interface, but folks who already know SQL must learn a whole new API. (On the other hand, the user *can* write things in SQL if they wish, learning only the most minimal of APIs, if they are willing to resolve issues of database-independence themself.)
|
||||||
|
|
||||||
|
- More difficult to implement than pure SQL solutions. [[br]]
|
||||||
|
SQL already exists/has been tested. A new Python API does not/has not, and much of the work seems to consist of little more than reinventing the wheel.
|
||||||
|
|
||||||
|
- Script behavior might change under different versions of the project.[[br]]
|
||||||
|
...where .sql scripts behave the same regardless of the project's version.
|
||||||
|
|
||||||
|
=== Generate .sql scripts from a Python API ===
|
||||||
|
Attempts to take the best of the first and last solutions. An API similar to the previous solution would be used, but rather than immediately being applied to the database, .sql scripts are generated for each type of database we're interested in. These .sql scripts are what's actually applied to the database.
|
||||||
|
|
||||||
|
This would essentially allow users to skip the Python script step entirely if they wished, and write migration scripts in SQL instead, as in solution 1.
|
||||||
|
|
||||||
|
+ Database-independence is an option, when needed.
|
||||||
|
|
||||||
|
+ A familiar interface/an interface that can interact with other tools is an option, when needed.
|
||||||
|
|
||||||
|
+ Easy to inspect the SQL generated by a script, to ensure it's what we're expecting.
|
||||||
|
|
||||||
|
+ Migration scripts won't change behavior across different versions of the project. [[br]]
|
||||||
|
Once a Python script is translated to a .sql script, its behavior is consistent across different versions of the project, unlike a pure Python solution.
|
||||||
|
|
||||||
|
- Multiple ways to do a single task: not Pythonic.[[br]]
|
||||||
|
I never really liked that word - "Pythonic" - but it does apply here. Multiple ways to do a single task has the potential to cause confusion, especially in a large project if many people do the same task different ways. We have to support both ways of doing things, as well.
|
||||||
|
|
||||||
|
----
|
||||||
|
|
||||||
|
'''Conclusion''': The last solution, generating .sql scripts from a Python API, seems to be best.
|
||||||
|
|
||||||
|
The first solution (.sql scripts) suffers from a lack of database-independence, but is familiar to experienced database developers, useful with other tools, and shows exactly what will be done to the database. The Python API solution has no trouble with database-independence, but suffers from other problems that the .sql solution doesn't. The last solution resolves both reasonably well. Multiple ways to do a single task might be called "not Pythonic", but IMO, the trade-off is worth this cost.
|
||||||
|
|
||||||
|
Automatic translation between different dialects of SQL might have potential for use in a solution, but existing tools for this aren't reliable enough, as far as I can tell.
|
||||||
56
docs/historical/ProjectDesignDecisionsVersioning.trac
Normal file
56
docs/historical/ProjectDesignDecisionsVersioning.trac
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
An important aspect of this project is database versioning. For migration scripts to be most useful, we need to know what version the database is: that is, has a particular migration script already been run?
|
||||||
|
|
||||||
|
An option not discussed below is "no versioning"; that is, simply apply any script we're given, and rely on the user to ensure it's valid. This is entirely too error-prone to seriously consider, and takes a lot of the usefulness out of the proposed tool.
|
||||||
|
|
||||||
|
|
||||||
|
=== Database-wide version numbers ===
|
||||||
|
A single integer version number would specify the version of each database. This is stored in the database in a table, let's call it "schema"; each migration script is associated with a certain database version number.
|
||||||
|
|
||||||
|
+ Simple implementation[[br]]
|
||||||
|
Of the 3 solutions presented here, this one is by far the simplest.
|
||||||
|
|
||||||
|
+ Past success[[br]]
|
||||||
|
Used in [http://www.rubyonrails.org/ Ruby on Rails' migrations].
|
||||||
|
|
||||||
|
~ Can detect corrupt schemas, but requires some extra work and a *complete* set of migrations.[[br]]
|
||||||
|
If we have a set of database migration scripts that build the database from the ground up, we can apply them in sequence to a 'dummy' database, dump a diff of the real and dummy schemas, and expect a valid schema to match the dummy schema.
|
||||||
|
|
||||||
|
- Requires changes to the database schema.[[br]]
|
||||||
|
Not a tremendous change - a single table with a single column and a single row - but a change nonetheless.
|
||||||
|
|
||||||
|
=== Table/object-specific version numbers ===
|
||||||
|
Each database "object" - usually tables, though we might also deal with other database objects, such as stored procedures or Postgres' sequences - would have a version associated with it, initially 1. These versions are stored in a table, let's call it "schema". This table has two columns: the name of the database object and its current version number.
|
||||||
|
|
||||||
|
+ Allows us to write migration scripts for a subset of the database.[[br]]
|
||||||
|
If we have multiple people working on a very large database, we may want to write migration scripts for a section of the database without stepping on another person's work. This allows unrelated to
|
||||||
|
|
||||||
|
- Requires changes to the database schema.
|
||||||
|
Similar to the database-wide version number; the contents of the new table are more complex, but still shouldn't conflict with anything.
|
||||||
|
|
||||||
|
- More difficult to implement than a database-wide version number.
|
||||||
|
|
||||||
|
- Determining the version of database-specific objects (ie. stored procedures, functions) is difficult.
|
||||||
|
|
||||||
|
- Ultimately gains nothing over the previous solution.[[br]]
|
||||||
|
The intent here was to allow multiple people to write scripts for a single database, but if database-wide version numbers aren't assigned until the script is placed in the repository, we could already do this.
|
||||||
|
|
||||||
|
=== Version determined via introspection ===
|
||||||
|
Each script has a schema associated with it, rather than a version number. The database schema is loaded, analyzed, and compared to the schema expected by the script.
|
||||||
|
|
||||||
|
+ No modifications to the database are necessary for this versioning system.[[br]]
|
||||||
|
The primary advantage here is that no changes to the database are required.
|
||||||
|
|
||||||
|
- Most difficult solution to implement, by far.[[br]]
|
||||||
|
Comparing the state of every schema object in the database is much more complex than simply comparing a version number, especially since we need to do it in a database-independent way (ie. we can't just diff the dump of each schema). SQLAlchemy's reflection would certainly be very helpful, but this remains the most complex solution.
|
||||||
|
|
||||||
|
+ "Automatically" detects corrupt schemas.[[br]]
|
||||||
|
A corrupt schema won't match any migration script.
|
||||||
|
|
||||||
|
- Difficult to deal with corrupt schemas.[[br]]
|
||||||
|
When version numbers are stored in the database, you have some idea of where an error occurred. Without this, we have no idea what version the database was in before corruption.
|
||||||
|
|
||||||
|
- Potential ambiguity: what if two database migration scripts expect the same schema?
|
||||||
|
|
||||||
|
----
|
||||||
|
|
||||||
|
'''Conclusion''': database-wide version numbers are the best way to go.
|
||||||
29
docs/historical/ProjectDetailedDesign.trac
Normal file
29
docs/historical/ProjectDetailedDesign.trac
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
This is very much a draft/brainstorm right now. It should be made prettier and thought about in more detail later, but it at least gives some idea of the direction we're headed right now.
|
||||||
|
----
|
||||||
|
* Two distinct tools; should not be coupled (can work independently):
|
||||||
|
* Versioning tool
|
||||||
|
* Command line tool; let's call it "samigrate"
|
||||||
|
* Organizes old migration scripts into repositories
|
||||||
|
* Runs groups of migration scripts on a database, updating it to a specified version/latest version
|
||||||
|
* Helps run various tests
|
||||||
|
* usage
|
||||||
|
* "samigrate create PATH": Create project migration-script repository
|
||||||
|
* We shouldn't have to enter the path for every other command. Use a hidden file
|
||||||
|
* (This means we can't move the repository after it's created. Oh well)
|
||||||
|
* "samigrate add SCRIPT [VERSION]": Add script to this project's repository; latest version
|
||||||
|
* If a .sql script: how to determine engine, operation (up/down)? Options:
|
||||||
|
* specify at the command line: "samigrate add SCRIPT UP_OR_DOWN ENGINE"
|
||||||
|
* naming convention: SCRIPT is named something like NAME.postgres.up.sql
|
||||||
|
* "samigrate upgrade CONNECTION_STRING [VERSION] [SCRIPT...]": connect to the specified database and upgrade (or downgrade) it to the specified version (default latest)
|
||||||
|
* If SCRIPT... specified: act like these scripts are in the repository (useful for testing?)
|
||||||
|
* "samigrate dump CONNECTION_STRING [VERSION] [SCRIPT...]": like update, but sends all sql to stdout instead of the db
|
||||||
|
* (Later: some more commands, to be used for script testing tools)
|
||||||
|
* Alchemy API extensions for altering schema
|
||||||
|
* Operations here are DB-independent
|
||||||
|
* Each database modification is a script that may use this API
|
||||||
|
* Can handwrite SQL for all databases or a single database
|
||||||
|
* upgrade()/downgrade() functions: need only one file for both operations
|
||||||
|
* sql scripts reqire either (2 files, *.up.sql;*.down.sql) or (don't use downgrade)
|
||||||
|
* usage
|
||||||
|
* "python NAME.py ENGINE up": upgrade sql > stdout
|
||||||
|
* "python NAME.py ENGINE down": downgrade sql > stdout
|
||||||
50
docs/historical/ProjectGoals.trac
Normal file
50
docs/historical/ProjectGoals.trac
Normal file
@ -0,0 +1,50 @@
|
|||||||
|
== Goals ==
|
||||||
|
|
||||||
|
=== DBMS-independent schema changes ===
|
||||||
|
Many projects need to run on more than one DBMS. Similar changes need to be applied to both types of databases upon a schema change. The usual solution to database changes - .sql scripts with ALTER statements - runs into problems since different DBMSes have different dialects of SQL; we end up having to create a different script for each DBMS. This project will simplify this by providing an API, similar to the table definition API that already exists in SQLAlchemy, to alter a table independent of the DBMS being used, where possible.
|
||||||
|
|
||||||
|
This project will support all DBMSes currently supported by SQLAlchemy: SQLite, Postgres, MySQL, Oracle, and MS SQL. Adding support for more should be as possible as it is in SQLAlchemy.
|
||||||
|
|
||||||
|
Many are already used to writing .sql scripts for database changes, aren't interested in learning a new API, and have projects where DBMS-independence isn't an issue. Writing SQL statements as part of a (Python) change script must be an option, of course. Writing change scripts as .sql scripts, eliminating Python scripts from the picture entirely, would be nice too, although this is a lower-priority goal.
|
||||||
|
|
||||||
|
=== Database versioning and change script organization ===
|
||||||
|
Once we've accumulated a set of change scripts, it's important to know which ones have been applied/need to be applied to a particular database: suppose we need to upgrade a database that's extremenly out-of-date; figuring out the scripts to run by hand is tedious. Applying changes in the wrong order, or applying changes when they shouldn't be applied, is bad; attempting to manage all of this by hand inevitably leads to an accident. This project will be able to detect the version of a particular database and apply the scripts required to bring it up to the latest version, or up to any specified version number (given all change scripts required to reach that version number).
|
||||||
|
|
||||||
|
Sometimes we need to be able to revert a schema to an older version. There's no automatic way to do this without rebuilding the database from scratch, so our project will allow one to write scripts to downgrade the database as well as upgrade it. If such scripts have been written, we should be able to apply them in the correct order, just like upgrading.
|
||||||
|
|
||||||
|
Large projects inevitably accumulate a large number of database change scripts; it's important that we have a place to keep them. Once a script has been written, this project will deal with organizing it among existing change scripts, and the user will never have to look at it again.
|
||||||
|
|
||||||
|
=== Change testing ===
|
||||||
|
It's important to test one's database changes before applying them to a production database (unless you happen to like disasters). Much testing is up to the user and can't be automated, but there's a few places we can help ensure at least a minimal level of schema integrity. A few examples are below; we could add more later.
|
||||||
|
|
||||||
|
Given an obsolete schema, a database change script, and an up-to-date schema known to be correct, this project will be able to ensure that applying the
|
||||||
|
change script to the obsolete schema will result in an up-to-date schema - all without actually changing the obsolete database. Folks who have SQLAlchemy create their database using table.create() might find this useful; this is also useful for ensuring database downgrade scripts are correct.
|
||||||
|
|
||||||
|
Given a schema of a known version and a complete set of change scripts up to that version, this project will be able to detect if the schema matches its version. If a schema has gone through changes not present in migration scripts, this test will fail; if applying all scripts in sequence up to the specified version creates an identical schema, this test will succeed. Identifying that a schema is corrupt is sufficient; it would be nice if we could give a clue as to what's wrong, but this is lower priority. (Implementation: we'll probably show a diff of two schema dumps; this should be enough to tell the user what's gone wrong.)
|
||||||
|
|
||||||
|
== Non-Goals ==
|
||||||
|
ie. things we will '''not''' try to do (at least, during the Summer of Code)
|
||||||
|
|
||||||
|
=== Automatic generation of schema changes ===
|
||||||
|
For example, one might define a table:
|
||||||
|
{{{
|
||||||
|
CREATE TABLE person (
|
||||||
|
id integer,
|
||||||
|
name varchar(80)
|
||||||
|
);
|
||||||
|
}}}
|
||||||
|
Later, we might add additional columns to the definition:
|
||||||
|
{{{
|
||||||
|
CREATE TABLE person (
|
||||||
|
id integer,
|
||||||
|
name varchar(80),
|
||||||
|
profile text
|
||||||
|
);
|
||||||
|
}}}
|
||||||
|
It might be nice if a tool could look at both table definitions and spit out a change script; something like
|
||||||
|
{{{
|
||||||
|
ALTER TABLE person ADD COLUMN profile text;
|
||||||
|
}}}
|
||||||
|
This is a difficult problem for a number of reasons. I have no intention of tackling this problem as part of the Summer of Code. This project aims to give you a better way to write that ALTER statement and make sure it's applied correctly, not to write it for you.
|
||||||
|
|
||||||
|
(Using an [http://sqlfairy.sourceforge.net/ existing] [http://xml2ddl.berlios.de/ tool] to add this sort of thing later might be worth looking into, but it will not be done during the Summer of Code. Among other reasons, methinks it's best to start with a system that isn't dependent on this sort of automation.)
|
||||||
73
docs/historical/ProjectProposal.txt
Normal file
73
docs/historical/ProjectProposal.txt
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
Evan Rosson
|
||||||
|
|
||||||
|
Project
|
||||||
|
---
|
||||||
|
SQLAlchemy Schema Migration
|
||||||
|
|
||||||
|
|
||||||
|
Synopsis
|
||||||
|
---
|
||||||
|
SQLAlchemy is an excellent object-relational database mapper for Python projects. Currently, it does a fine job of creating a database from scratch, but provides no tool to assist the user in modifying an existing database. This project aims to provide such a tool.
|
||||||
|
|
||||||
|
|
||||||
|
Benefits
|
||||||
|
---
|
||||||
|
Application requirements change; a database schema must be able to change with them. It's possible to write SQL scripts that make the proper modifications without any special tools, but this setup quickly becomes difficult to manage - when we need to apply multiple updates to a database, organize old migration scripts, or have a single application support more than one DBMS, a tool to support database changes becomes necessary. This tool will aid the creation of organizing migration scripts, applying multiple updates or removing updates to revert to an old version, and creating DBMS-independent migration scripts.
|
||||||
|
|
||||||
|
Writing one's schema migration scripts by hand often results in problems when dealing with multiple obsolete database instances - we must figure out what scripts are necessary to bring the database up-to-date. Database versioning tools are helpful for this task; this project will track the version of a particular database to determine what scripts are necessary to update an old schema.
|
||||||
|
|
||||||
|
|
||||||
|
Description
|
||||||
|
---
|
||||||
|
The migration system used by Ruby on Rails has had much success, and for good reason - the system is easy to understand, generally database-independent, as powerful as the application itself, and capable of dealing nicely with a schema with multiple instances of different versions. A migration system similar to that of Rails is a fine place to begin this project.
|
||||||
|
|
||||||
|
Each instance of the schema will have a version associated with it; this version is tracked using a single table with a single row and a single integer column. A set of changes to the database schema will increment the schema's version number; each migration script will be associated with a schema version.
|
||||||
|
|
||||||
|
A migration script will be written by the user, and consist of two functions:
|
||||||
|
- upgrade(): brings an old database up-to-date, from version n-1 to version n
|
||||||
|
- downgrade(): reverts an up-to-date database to the previous schema; an 'undo' for upgrade()
|
||||||
|
|
||||||
|
When applying multiple updates to an old schema instance, migration scripts are applied in sequence: when updating a schema to version n from version n-2, two migration scripts are run; n-2 => n-1 => n.
|
||||||
|
|
||||||
|
A command-line tool will create empty migration scripts (empty upgrade()/downgrade() functions), display the SQL that will be generated by a migration script for a particular DBMS, and apply migration scripts to a specified database.
|
||||||
|
|
||||||
|
This project will implement the command-line tool that manages the above functionality. This project will also extend SQLAlchemy with the functions necessary to construct DBMS-independent migration scripts: in particular, column creation/deletion/alteration and the ability to rename existing tables/indexes/columns will be implemented. We'll also need a way to write raw SQL for a specific DBMS/set of DBMSes for situations where our abstraction doesn't fit a script's requirements. The creation/deletion of existing tables and indexes are operations already provided by SQLAlchemy.
|
||||||
|
|
||||||
|
|
||||||
|
On DBMS support - I intend to support MySQL, Postgres, SQLite, Oracle, and MS-SQL by the end of the project. (Update: I previously omitted support for Oracle and MS-SQL because I don't have access to the full version of each; I wasn't aware Oracle Lite and MS-SQL Express were available for free.) The system will be abstracted in such a way that adding support for other databases will not be any more difficult than adding support for them in SQLAlchemy.
|
||||||
|
|
||||||
|
|
||||||
|
Schedule
|
||||||
|
---
|
||||||
|
This project will be my primary activity this summer. Unfortunately, I am in school when things begin, until June 9, but I can still begin the project during that period. I have no other commitments this summer - I can easily make up any lost time.
|
||||||
|
I will be spending my spare time this summer further developing my online game (discussed below), but this has no deadline and will not interfere with the project proposed here.
|
||||||
|
|
||||||
|
|
||||||
|
I'll begin by familiarizing myself with the internals of SQLAlchemy and creating a detailed plan for the project. This plan will be reviewed by the current SQLAlchemy developers and other potential users, and will be modified based on their feedback. This will be completed no later than May 30, one week after SoC begins.
|
||||||
|
|
||||||
|
Development will follow, in this order:
|
||||||
|
- The database versioning system. This will manage the creation and application of (initially empty) migration scripts. Complete by June 16.
|
||||||
|
- Access the database; read/update the schema's version number
|
||||||
|
- Apply a single (empty) script to the database
|
||||||
|
- Apply a set of (empty) scripts to upgrade/downgrade the database to a specified version; examine all migration scripts and apply all to update the database to the latest version available
|
||||||
|
- An API for table/column alterations, to make the above system useful. Complete by August 11.
|
||||||
|
- Implement an empty API - does nothing at this point, but written in such a way that syntax for each supported DBMS may be added as a module. Completed June 26-30, the mid-project review deadline.
|
||||||
|
- Implement/test the above API for a single DBMS (probably Postgres, as I'm familiar with it). Users should be able to test the 'complete' application with this DBMS.
|
||||||
|
- Implement the database modification API for other supported databases
|
||||||
|
|
||||||
|
All development will have unit tests written where appropriate. Unit testing the SQL generated for each DBMS will be particularly important.
|
||||||
|
|
||||||
|
The project will finish with various wrap-up activities, documentation, and some final tests, to be completed by the project deadline.
|
||||||
|
|
||||||
|
|
||||||
|
About me
|
||||||
|
---
|
||||||
|
I am a 3rd year BS Computer Science student; Cal Poly, San Luis Obispo, California, USA; currently applying for a Master's degree in CS from the same school. I've taken several classes dealing with databases, though much of what I know on the subject is self-taught. Outside of class, I've developed a browser-based online game, Zeal, at http://zealgame.com ; it has been running for well over a year and gone through many changes. It has taught me firsthand the importance of using appropriate tools and designing one's application well early on (largely through the pain that follows when you don't); I've learned a great many other things from the experience as well.
|
||||||
|
|
||||||
|
One recurring problem I've had with this project is dealing with changes to the database schema. I've thought much about how I'd like to see this solved, but hadn't done much to implement it.
|
||||||
|
|
||||||
|
I'm now working on another project that will be making use of SQLAlchemy: it fits many of my project's requirements, but lacks a migration tool that will be much needed. This presents an opportunity for me to make my first contribution to open source - I've long been interested in open source software and use it regularly, but haven't contributed to any until now. I'm particularly interested in the application of this tool with the TurboGears framework, as this project was inspired by a suggestion the TurboGears mailing list and I'm working on a project using TurboGears - but there is no reason to couple an SQLAlchemy enhancement with TurboGears; this project may be used by anyone who uses SQLAlchemy.
|
||||||
|
|
||||||
|
|
||||||
|
Further information:
|
||||||
|
http://evan.zealgame.com/soc
|
||||||
56
docs/historical/RepositoryFormat.trac
Normal file
56
docs/historical/RepositoryFormat.trac
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
This plan has several problems and has been modified; new plan is discussed in wiki:RepositoryFormat2
|
||||||
|
|
||||||
|
----
|
||||||
|
|
||||||
|
One problem with [http://www.rubyonrails.org/ Ruby on Rails'] (very good) schema migration system is the behavior of scripts that depend on outside sources; ie. the application. If those change, there's no guarantee that such scripts will behave as they did before, and you'll get strange results.
|
||||||
|
|
||||||
|
For example, suppose one defines a SQLAlchemy table:
|
||||||
|
{{{
|
||||||
|
users = Table('users', metadata,
|
||||||
|
Column('user_id', Integer, primary_key = True),
|
||||||
|
Column('user_name', String(16), nullable = False),
|
||||||
|
Column('password', String(20), nullable = False)
|
||||||
|
)
|
||||||
|
}}}
|
||||||
|
and creates it in a change script:
|
||||||
|
{{{
|
||||||
|
from project import table
|
||||||
|
|
||||||
|
def upgrade():
|
||||||
|
table.users.create()
|
||||||
|
}}}
|
||||||
|
|
||||||
|
Suppose we later add a column to this table. We write an appropriate change script:
|
||||||
|
{{{
|
||||||
|
from project import table
|
||||||
|
|
||||||
|
def upgrade():
|
||||||
|
# This syntax isn't set in stone yet
|
||||||
|
table.users.add_column('email_address', String(60), key='email')
|
||||||
|
}}}
|
||||||
|
...and change our application's table definition:
|
||||||
|
{{{
|
||||||
|
users = Table('users', metadata,
|
||||||
|
Column('user_id', Integer, primary_key = True),
|
||||||
|
Column('user_name', String(16), nullable = False),
|
||||||
|
Column('password', String(20), nullable = False),
|
||||||
|
Column('email_address', String(60), key='email') #new column
|
||||||
|
)
|
||||||
|
}}}
|
||||||
|
|
||||||
|
Modifying the table definition changes how our first script behaves - it will create the table with the new column. This might work if we only apply change scripts to a few database which are always kept up to date (or very close), but we'll run into errors eventually if our migration scripts' behavior isn't consistent.
|
||||||
|
|
||||||
|
----
|
||||||
|
|
||||||
|
One solution is to generate .sql files from a Python change script at the time it's added to a repository. The sql generated by the script for each database is set in stone at this point; changes to outside files won't affect it.
|
||||||
|
|
||||||
|
This limits what change scripts are capable of - we can't write dynamic SQL; ie., we can't do something like this:
|
||||||
|
{{{
|
||||||
|
for row in db.execute("select id from table1"):
|
||||||
|
db.execute("insert into table2 (table1_id, value) values (:id,42)",**row)
|
||||||
|
}}}
|
||||||
|
But SQL is usually powerful enough to where the above is rarely necessary in a migration script:
|
||||||
|
{{{
|
||||||
|
db.execute("insert into table2 select id,42 from table1")
|
||||||
|
}}}
|
||||||
|
This is a reasonable solution. The limitations aren't serious (everything possible in a traditional .sql script is still possible), and change scripts are much less prone to error.
|
||||||
28
docs/historical/RepositoryFormat2.trac
Normal file
28
docs/historical/RepositoryFormat2.trac
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
My original plan for Migrate's RepositoryFormat had several problems:
|
||||||
|
|
||||||
|
* Bind parameters: We needed to bind parameters into statements to get something suitable for an .sql file. For some types of parameters, there's no clean way to do this without writing an entire parser - too great a cost for this project. There's a reason why SQLAlchemy's logs display the statement and its parameters separately: the binding is done at a lower level than we have access to.
|
||||||
|
* Failure: Discussed in #17, the old format had no easy way to find the Python statements associated with an SQL error. This makes it difficult to debug scripts.
|
||||||
|
|
||||||
|
A new format will be used to solve this problem instead.
|
||||||
|
Similar to our previous solution, where one .sql file was created per version/operation/DBMS (version_1.upgrade.postgres.sql, for example), one file will be created per version/operation/DBMS here.
|
||||||
|
These files will contain the following information:
|
||||||
|
|
||||||
|
* The dialect used to perform the logging. Particularly,
|
||||||
|
* The paramstyle expected by the dbapi
|
||||||
|
* The DBMS this log applies to
|
||||||
|
* Information on each logged SQL statement, each of which contains:
|
||||||
|
* The text of the statement
|
||||||
|
* Parameters to be bound to the statement
|
||||||
|
* A Python stack trace at the point the statement was logged - this allows us to tell what Python statements are associated with an SQL statement when there's an error
|
||||||
|
|
||||||
|
These files will be created by pickling a Python object with the above information.
|
||||||
|
|
||||||
|
Such files may be executed by loading the log and having SQLAlchemy execute them as it might have before.
|
||||||
|
|
||||||
|
Good:
|
||||||
|
* Since the statements and bind parameters are stored separately and executed as SQLAlchemy would normally execute them, one problem discussed above is eliminated.
|
||||||
|
* Storing the stack trace at the point each statement was logged allows us to identify what Python statements are responsible for an SQL error. This makes it much easier for users to debug their scripts.
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
* It's less trivial to commit .sql scripts to our repository, since they're no longer used internally. This isn't a huge loss, and .sql commits can still be implemented later if need be.
|
||||||
|
* There's some danger of script behavior changing if changes are made to the dbapi the script is associated with. The primary place where problems would occur is during parameter binding, but the chance of this changing significantly isn't large. The danger of changes in behavior due to changes in the user's application is not affected.
|
||||||
120
docs/index.rst
Normal file
120
docs/index.rst
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
:mod:`migrate` - SQLAlchemy Migrate (schema change management)
|
||||||
|
==============================================================
|
||||||
|
|
||||||
|
.. module:: migrate
|
||||||
|
.. moduleauthor:: Evan Rosson
|
||||||
|
|
||||||
|
:Author: Evan Rosson
|
||||||
|
:Maintainer: Domen Kozar <domenNO@SPAMdev.si>
|
||||||
|
:Source code: http://code.google.com/p/sqlalchemy-migrate/issues/list
|
||||||
|
:Issues: http://code.google.com/p/sqlalchemy-migrate/
|
||||||
|
:Version: |release|
|
||||||
|
|
||||||
|
|
||||||
|
.. topic:: Overview
|
||||||
|
|
||||||
|
Inspired by Ruby on Rails' migrations, SQLAlchemy Migrate provides a
|
||||||
|
way to deal with database schema changes in SQLAlchemy_ projects.
|
||||||
|
|
||||||
|
Migrate was started as part of `Google's Summer of Code`_ by Evan
|
||||||
|
Rosson, mentored by Jonathan LaCour.
|
||||||
|
|
||||||
|
The project was taken over by a small group of volunteers when Evan
|
||||||
|
had no free time for the project. It is now hosted as a `Google Code
|
||||||
|
project`_. During the hosting change the project was renamed to
|
||||||
|
SQLAlchemy Migrate.
|
||||||
|
|
||||||
|
Currently, sqlalchemy-migrate supports Python versions from 2.4 to 2.6.
|
||||||
|
SQLAlchemy >=0.5 is supported only.
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
Version **0.5.5** breaks backward compatability, please read :ref:`changelog <backwards-055>` for more info.
|
||||||
|
|
||||||
|
|
||||||
|
Download and Development
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
|
||||||
|
download
|
||||||
|
|
||||||
|
.. _dialect-support:
|
||||||
|
|
||||||
|
Dialect support
|
||||||
|
---------------
|
||||||
|
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| Operation / Dialect | :ref:`sqlite <sqlite-d>` | :ref:`postgres <postgres-d>` | :ref:`mysql <mysql-d>` | :ref:`oracle <oracle-d>` | :ref:`firebird <firebird-d>` | mssql |
|
||||||
|
| | | | | | | |
|
||||||
|
+=========================================================+==========================+==============================+========================+===========================+===============================+=======+
|
||||||
|
| :ref:`ALTER TABLE RENAME TABLE <table-rename>` | yes | yes | yes | yes | no | |
|
||||||
|
| | | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`ALTER TABLE RENAME COLUMN <column-alter>` | yes | yes | yes | yes | yes | |
|
||||||
|
| | (workaround) [#1]_ | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`ALTER TABLE ADD COLUMN <column-create>` | yes | yes | yes | yes | yes | |
|
||||||
|
| | (with limitations) [#2]_ | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`ALTER TABLE DROP COLUMN <column-drop>` | yes | yes | yes | yes | yes | |
|
||||||
|
| | (workaround) [#1]_ | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`ALTER TABLE ALTER COLUMN <column-alter>` | yes | yes | yes | yes | yes [#4]_ | |
|
||||||
|
| | (workaround) [#1]_ | | | (with limitations) [#3]_ | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`ALTER TABLE ADD CONSTRAINT <constraint-tutorial>` | no | yes | yes | yes | yes | |
|
||||||
|
| | | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`ALTER TABLE DROP CONSTRAINT <constraint-tutorial>`| no | yes | yes | yes | yes | |
|
||||||
|
| | | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
| :ref:`RENAME INDEX <index-rename>` | no | yes | no | yes | yes | |
|
||||||
|
| | | | | | | |
|
||||||
|
+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+
|
||||||
|
|
||||||
|
|
||||||
|
.. [#1] Table is renamed to temporary table, new table is created followed by INSERT statements.
|
||||||
|
.. [#2] Visit http://www.sqlite.org/lang_altertable.html for more information.
|
||||||
|
.. [#3] You can not change datatype or rename column if table has NOT NULL data, see http://blogs.x2line.com/al/archive/2005/08/30/1231.aspx for more information.
|
||||||
|
.. [#4] Changing nullable is not supported
|
||||||
|
|
||||||
|
Documentation
|
||||||
|
-------------
|
||||||
|
|
||||||
|
SQLAlchemy is split into two parts, database schema versioning and
|
||||||
|
database changeset management. This is represented by two python
|
||||||
|
packages :mod:`migrate.versioning` and :mod:`migrate.changeset`. The
|
||||||
|
versioning API is available as the :ref:`migrate <command-line-usage>` command.
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
|
||||||
|
versioning
|
||||||
|
changeset
|
||||||
|
tools
|
||||||
|
|
||||||
|
.. _`google's summer of code`: http://code.google.com/soc
|
||||||
|
.. _`Google Code project`: http://code.google.com/p/sqlalchemy-migrate
|
||||||
|
.. _sqlalchemy: http://www.sqlalchemy.org
|
||||||
|
|
||||||
|
API Documentation
|
||||||
|
------------------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
|
||||||
|
api
|
||||||
|
|
||||||
|
Changelog
|
||||||
|
---------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
|
||||||
|
changelog
|
||||||
|
|
||||||
|
|
||||||
|
Indices and tables
|
||||||
|
==================
|
||||||
|
|
||||||
|
* :ref:`genindex`
|
||||||
|
* :ref:`modindex`
|
||||||
|
* :ref:`search`
|
||||||
288
docs/theme/almodovar.css
vendored
Normal file
288
docs/theme/almodovar.css
vendored
Normal file
@ -0,0 +1,288 @@
|
|||||||
|
/*
|
||||||
|
* Original theme modified by Evan Rosson
|
||||||
|
* http://erosson.com/migrate
|
||||||
|
* ---
|
||||||
|
*
|
||||||
|
* Theme Name: Almodovar
|
||||||
|
* Theme URI: http://blog.ratterobert.com/archiv/2005/03/09/almodovar/
|
||||||
|
* Description: Das Theme basiert im Ursprung auf Michael Heilemanns <a href="http://binarybonsai.com/kubrick/">Kubrick</a>-Template und ist von dem einen oder anderen Gimmick anderer sehr guter Templates inspiriert worden.
|
||||||
|
* Version: 0.7
|
||||||
|
* Author: ratte / robert
|
||||||
|
* Author URI: http://blog.ratterobert.com/
|
||||||
|
* */
|
||||||
|
|
||||||
|
/* Begin Typography & Colors */
|
||||||
|
body {
|
||||||
|
font-size: 75%;
|
||||||
|
font-family: 'Lucida Grande', 'Trebuchet MS', 'Bitstream Vera Sans', Sans-Serif;
|
||||||
|
background-color: #CCF;
|
||||||
|
color: #333;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
#page {
|
||||||
|
background-color: #fff;
|
||||||
|
border: 1px solid #88f;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
#content {
|
||||||
|
font-size: 1.2em;
|
||||||
|
margin: 1em;
|
||||||
|
}
|
||||||
|
|
||||||
|
#content p,
|
||||||
|
#content ul,
|
||||||
|
#content blockquote {
|
||||||
|
line-height: 1.6em;
|
||||||
|
}
|
||||||
|
|
||||||
|
#footer {
|
||||||
|
border-top: 1px solid #006;
|
||||||
|
margin-top: 2em;
|
||||||
|
}
|
||||||
|
|
||||||
|
small {
|
||||||
|
font-family: 'Trebuchet MS', Arial, Helvetica, Sans-Serif;
|
||||||
|
font-size: 0.9em;
|
||||||
|
line-height: 1.5em;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1, h2, h3 {
|
||||||
|
font-family: 'Trebuchet MS', 'Lucida Grande', Verdana, Arial, Sans-Serif;
|
||||||
|
font-weight: bold;
|
||||||
|
margin-top: .7em;
|
||||||
|
margin-bottom: .7em;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1 {
|
||||||
|
font-size: 2.5em;
|
||||||
|
}
|
||||||
|
h2 {
|
||||||
|
font-size: 2em;
|
||||||
|
}
|
||||||
|
h3 {
|
||||||
|
font-size: 1.5em;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1, h2, h3 {
|
||||||
|
color: #33a;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1 a, h2 a, h3 a {
|
||||||
|
color: #33a;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1, h1 a, h1 a:hover, h1 a:visited,
|
||||||
|
h2, h2 a, h2 a:hover, h2 a:visited,
|
||||||
|
h3, h3 a, h3 a:hover, h3 a:visited,
|
||||||
|
cite {
|
||||||
|
text-decoration: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
#content p a:visited {
|
||||||
|
color: #004099;
|
||||||
|
/*font-weight: normal;*/
|
||||||
|
}
|
||||||
|
|
||||||
|
small, blockquote, strike {
|
||||||
|
color: #33a;
|
||||||
|
}
|
||||||
|
|
||||||
|
#links ul ul li, #links li {
|
||||||
|
list-style: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
code {
|
||||||
|
font: 1.1em 'Courier', 'Courier New', Fixed;
|
||||||
|
}
|
||||||
|
|
||||||
|
acronym, abbr, span.caps {
|
||||||
|
font-size: 0.9em;
|
||||||
|
letter-spacing: .07em;
|
||||||
|
}
|
||||||
|
|
||||||
|
a {
|
||||||
|
color: #0050FF;
|
||||||
|
/*text-decoration: none;*/
|
||||||
|
text-decoration:underline;
|
||||||
|
/*font-weight: bold;*/
|
||||||
|
}
|
||||||
|
a:hover {
|
||||||
|
color: #0080FF;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Special case doc-title */
|
||||||
|
h1.doc-title {
|
||||||
|
text-transform: lowercase;
|
||||||
|
font-size: 4em;
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
h1.doc-title a {
|
||||||
|
display: block;
|
||||||
|
padding-left: 0.8em;
|
||||||
|
padding-bottom: .5em;
|
||||||
|
padding-top: .5em;
|
||||||
|
margin: 0;
|
||||||
|
border-bottom: 1px #fff solid;
|
||||||
|
}
|
||||||
|
h1.doc-title,
|
||||||
|
h1.doc-title a,
|
||||||
|
h1.doc-title a:visited,
|
||||||
|
h1.doc-title a:hover {
|
||||||
|
text-decoration: none;
|
||||||
|
color: #0050FF;
|
||||||
|
}
|
||||||
|
/* End Typography & Colors */
|
||||||
|
|
||||||
|
|
||||||
|
/* Begin Structure */
|
||||||
|
body {
|
||||||
|
margin: 0;
|
||||||
|
padding: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
#page {
|
||||||
|
background-color: white;
|
||||||
|
margin: 0 auto 0 9em;
|
||||||
|
padding: 0;
|
||||||
|
max-width: 60em;
|
||||||
|
border: 1px solid #555596;
|
||||||
|
}
|
||||||
|
* html #page {
|
||||||
|
* width: 60em;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* #content {
|
||||||
|
* margin: 0 1em 0 3em;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* #content h1 {
|
||||||
|
* margin-left: 0;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* #footer {
|
||||||
|
* padding: 0 0 0 1px;
|
||||||
|
* margin: 0;
|
||||||
|
* margin-top: 1.5em;
|
||||||
|
* clear: both;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* #footer p {
|
||||||
|
* margin: 1em;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
*/* End Structure */
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/* Begin Headers */
|
||||||
|
.description {
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* End Headers */
|
||||||
|
|
||||||
|
|
||||||
|
/* Begin Form Elements */
|
||||||
|
#searchform {
|
||||||
|
margin: 1em auto;
|
||||||
|
text-align: right;
|
||||||
|
}
|
||||||
|
|
||||||
|
#searchform #s {
|
||||||
|
width: 100px;
|
||||||
|
padding: 2px;
|
||||||
|
}
|
||||||
|
|
||||||
|
#searchsubmit {
|
||||||
|
padding: 1px;
|
||||||
|
}
|
||||||
|
/* End Form Elements */
|
||||||
|
|
||||||
|
|
||||||
|
/* Begin Various Tags & Classes */
|
||||||
|
acronym, abbr, span.caps {
|
||||||
|
cursor: help;
|
||||||
|
}
|
||||||
|
|
||||||
|
acronym, abbr {
|
||||||
|
border-bottom: 1px dashed #999;
|
||||||
|
}
|
||||||
|
|
||||||
|
blockquote {
|
||||||
|
margin: 15px 30px 0 10px;
|
||||||
|
padding-left: 20px;
|
||||||
|
border-left: 5px solid #CCC;
|
||||||
|
}
|
||||||
|
|
||||||
|
blockquote cite {
|
||||||
|
margin: 5px 0 0;
|
||||||
|
display: block;
|
||||||
|
}
|
||||||
|
|
||||||
|
hr {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
a img {
|
||||||
|
border: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.navigation {
|
||||||
|
display: block;
|
||||||
|
text-align: center;
|
||||||
|
margin-top: 10px;
|
||||||
|
margin-bottom: 60px;
|
||||||
|
}
|
||||||
|
/* End Various Tags & Classes*/
|
||||||
|
|
||||||
|
span a { color: #CCC; }
|
||||||
|
|
||||||
|
span a:hover { color: #0050FF; }
|
||||||
|
|
||||||
|
#navcontainer {
|
||||||
|
margin-top: 0px;
|
||||||
|
padding-top: 0px;
|
||||||
|
width: 100%;
|
||||||
|
background-color: #AAF;
|
||||||
|
text-align: right;
|
||||||
|
}
|
||||||
|
|
||||||
|
#navlist ul {
|
||||||
|
margin-left: 0;
|
||||||
|
margin-right: 5px;
|
||||||
|
padding-left: 0;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
#navlist li {
|
||||||
|
display: inline;
|
||||||
|
list-style-type: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
#navlist a {
|
||||||
|
padding: 3px 10px;
|
||||||
|
color: #fff;
|
||||||
|
background-color: #339;
|
||||||
|
text-decoration: none;
|
||||||
|
border: 1px solid #44F;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
#navlist a:hover {
|
||||||
|
color: #000;
|
||||||
|
background-color: #FFF;
|
||||||
|
text-decoration: none;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
|
#navlist a:active, #navlist a.selected {
|
||||||
|
padding: 3px 10px;
|
||||||
|
color: #000;
|
||||||
|
background-color: #EEF;
|
||||||
|
text-decoration: none;
|
||||||
|
border: 1px solid #CCF;
|
||||||
|
font-weight: normal;
|
||||||
|
}
|
||||||
|
|
||||||
123
docs/theme/layout.css
vendored
Normal file
123
docs/theme/layout.css
vendored
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
@import url("pudge.css");
|
||||||
|
@import url("almodovar.css");
|
||||||
|
|
||||||
|
/* Basic Style
|
||||||
|
----------------------------------- */
|
||||||
|
|
||||||
|
h1.pudge-member-page-heading {
|
||||||
|
font-size: 300%;
|
||||||
|
}
|
||||||
|
h4.pudge-member-page-subheading {
|
||||||
|
font-size: 130%;
|
||||||
|
font-style: italic;
|
||||||
|
margin-top: -2.0em;
|
||||||
|
margin-left: 2em;
|
||||||
|
margin-bottom: .3em;
|
||||||
|
color: #0050CC;
|
||||||
|
}
|
||||||
|
p.pudge-member-blurb {
|
||||||
|
font-style: italic;
|
||||||
|
font-weight: bold;
|
||||||
|
font-size: 120%;
|
||||||
|
margin-top: 0.2em;
|
||||||
|
color: #999;
|
||||||
|
}
|
||||||
|
p.pudge-member-parent-link {
|
||||||
|
margin-top: 0;
|
||||||
|
}
|
||||||
|
/*div.pudge-module-doc {
|
||||||
|
max-width: 45em;
|
||||||
|
}*/
|
||||||
|
div.pudge-section {
|
||||||
|
margin-left: 2em;
|
||||||
|
max-width: 45em;
|
||||||
|
}
|
||||||
|
/* Section Navigation
|
||||||
|
----------------------------------- */
|
||||||
|
|
||||||
|
div#pudge-section-nav
|
||||||
|
{
|
||||||
|
margin: 1em 0 1.5em 0;
|
||||||
|
padding: 0;
|
||||||
|
height: 20px;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#pudge-section-nav ul {
|
||||||
|
border: 0;
|
||||||
|
margin: 0;
|
||||||
|
padding: 0;
|
||||||
|
list-style-type: none;
|
||||||
|
text-align: center;
|
||||||
|
border-right: 1px solid #aaa;
|
||||||
|
}
|
||||||
|
div#pudge-section-nav ul li
|
||||||
|
{
|
||||||
|
display: block;
|
||||||
|
float: left;
|
||||||
|
text-align: center;
|
||||||
|
padding: 0;
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#pudge-section-nav ul li .pudge-section-link,
|
||||||
|
div#pudge-section-nav ul li .pudge-missing-section-link
|
||||||
|
{
|
||||||
|
background: #aaa;
|
||||||
|
width: 9em;
|
||||||
|
height: 1.8em;
|
||||||
|
border: 1px solid #bbb;
|
||||||
|
padding: 0;
|
||||||
|
margin: 0 0 10px 0;
|
||||||
|
color: #ddd;
|
||||||
|
text-decoration: none;
|
||||||
|
display: block;
|
||||||
|
text-align: center;
|
||||||
|
font: 11px/20px "Verdana", "Lucida Grande";
|
||||||
|
cursor: hand;
|
||||||
|
text-transform: lowercase;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#pudge-section-nav ul li a:hover {
|
||||||
|
color: #000;
|
||||||
|
background: #fff;
|
||||||
|
}
|
||||||
|
|
||||||
|
div#pudge-section-nav ul li .pudge-section-link {
|
||||||
|
background: #888;
|
||||||
|
color: #eee;
|
||||||
|
border: 1px solid #bbb;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Module Lists
|
||||||
|
----------------------------------- */
|
||||||
|
dl.pudge-module-list dt {
|
||||||
|
font-style: normal;
|
||||||
|
font-size: 110%;
|
||||||
|
}
|
||||||
|
dl.pudge-module-list dd {
|
||||||
|
color: #555;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Misc Overrides */
|
||||||
|
.rst-doc p.topic-title a {
|
||||||
|
color: #777;
|
||||||
|
}
|
||||||
|
.rst-doc ul.auto-toc a,
|
||||||
|
.rst-doc div.contents a {
|
||||||
|
color: #333;
|
||||||
|
}
|
||||||
|
pre { background: #eee; }
|
||||||
|
|
||||||
|
.rst-doc dl dt {
|
||||||
|
color: #444;
|
||||||
|
margin-top: 1em;
|
||||||
|
font-weight: bold;
|
||||||
|
}
|
||||||
|
.rst-doc dl dd {
|
||||||
|
margin-top: .2em;
|
||||||
|
}
|
||||||
|
.rst-doc hr {
|
||||||
|
display: block;
|
||||||
|
margin: 2em 0;
|
||||||
|
}
|
||||||
|
|
||||||
90
docs/theme/layout.html
vendored
Normal file
90
docs/theme/layout.html
vendored
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
<?xml version="1.0"?>
|
||||||
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
||||||
|
|
||||||
|
<?python
|
||||||
|
import pudge
|
||||||
|
|
||||||
|
def initialize(t):
|
||||||
|
g = t.generator
|
||||||
|
if not hasattr(t, 'title'):
|
||||||
|
t.title = 'Untitled'
|
||||||
|
t.doc_title = g.index_document['title']
|
||||||
|
t.home_url = g.organization_url or g.blog_url or g.trac_url
|
||||||
|
t.home_title = g.organization
|
||||||
|
?>
|
||||||
|
|
||||||
|
<html xmlns="http://www.w3.org/1999/xhtml"
|
||||||
|
xmlns:py="http://purl.org/kid/ns#"
|
||||||
|
py:def="layout">
|
||||||
|
|
||||||
|
<head>
|
||||||
|
<title>${title}</title>
|
||||||
|
<link rel="stylesheet" type="text/css" href="layout.css"/>
|
||||||
|
<link py:if="generator.syndication_url"
|
||||||
|
rel="alternate"
|
||||||
|
type="application/rss+xml"
|
||||||
|
title="RSS 2.0" href="${generator.syndication_url}"/>
|
||||||
|
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="page">
|
||||||
|
<h1 class="doc-title"><a href="${home_url}">${home_title}</a></h1>
|
||||||
|
<div id="navcontainer">
|
||||||
|
<ul id="navlist">
|
||||||
|
<li class="pagenav">
|
||||||
|
<ul>
|
||||||
|
<li class="page_item">
|
||||||
|
<a href="index.html"
|
||||||
|
class="${'index.html'== destfile and 'selected' or ''}"
|
||||||
|
title="Project Home / Index">${doc_title}</a>
|
||||||
|
</li>
|
||||||
|
<li class="page_item">
|
||||||
|
<a href="module-index.html"
|
||||||
|
class="${'module-index.html'== destfile and 'selected' or ''}"
|
||||||
|
title="${doc_title.lower()} package and module reference">Modules</a>
|
||||||
|
</li>
|
||||||
|
<?python
|
||||||
|
trac_url = generator.trac_url
|
||||||
|
mailing_list_url = generator.mailing_list_url
|
||||||
|
?>
|
||||||
|
<li py:if="trac_url">
|
||||||
|
<a href="${trac_url}"
|
||||||
|
title="Wiki / Subversion / Roadmap / Bug Tracker"
|
||||||
|
>Trac</a>
|
||||||
|
</li>
|
||||||
|
<li py:if="generator.blog_url">
|
||||||
|
<a href="${generator.blog_url}">Blog</a>
|
||||||
|
</li>
|
||||||
|
<li py:if="mailing_list_url">
|
||||||
|
<a href="${mailing_list_url}"
|
||||||
|
class="${mailing_list_url == destfile and 'selected' or ''}"
|
||||||
|
title="Mailing List">Discuss</a>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<hr />
|
||||||
|
|
||||||
|
<div id="content" py:content="content()"/>
|
||||||
|
|
||||||
|
<div id="footer">
|
||||||
|
<?python license = generator.get_document('doc-license') ?>
|
||||||
|
|
||||||
|
<p style="float: left;">
|
||||||
|
built with
|
||||||
|
<a href="http://lesscode.org/projects/pudge/"
|
||||||
|
>pudge/${pudge.__version__}</a><br />
|
||||||
|
original design by
|
||||||
|
<a href="http://blog.ratterobert.com/"
|
||||||
|
>ratter / robert</a><br />
|
||||||
|
</p>
|
||||||
|
<p style="float:right;">
|
||||||
|
evan.rosson (at) gmail.com
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</body>
|
||||||
|
|
||||||
|
</html>
|
||||||
17
docs/tools.rst
Normal file
17
docs/tools.rst
Normal file
@ -0,0 +1,17 @@
|
|||||||
|
SQLAlchemy migrate tools
|
||||||
|
========================
|
||||||
|
|
||||||
|
The most commonly used tool is the :ref:`migrate <command-line-usage>`.
|
||||||
|
|
||||||
|
.. index:: repository migration
|
||||||
|
|
||||||
|
There is a second tool :command:`migrate_repository.py` that may be
|
||||||
|
used to migrate your repository from a version before 0.4.5 of
|
||||||
|
SQLAlchemy migrate to the current version.
|
||||||
|
|
||||||
|
.. module:: migrate.versioning.migrate_repository
|
||||||
|
:synopsis: Tool for migrating pre 0.4.5 repositories to current layout
|
||||||
|
|
||||||
|
Running :command:`migrate_repository.py` is as easy as:
|
||||||
|
|
||||||
|
:samp:`migrate_repository.py {repository_directory}`
|
||||||
495
docs/versioning.rst
Normal file
495
docs/versioning.rst
Normal file
@ -0,0 +1,495 @@
|
|||||||
|
.. _versioning-system:
|
||||||
|
.. currentmodule:: migrate.versioning
|
||||||
|
.. highlight:: bash
|
||||||
|
|
||||||
|
***********************************
|
||||||
|
Database schema versioning workflow
|
||||||
|
***********************************
|
||||||
|
|
||||||
|
SQLAlchemy migrate provides the :mod:`migrate.versioning` API that is
|
||||||
|
also available as the :ref:`migrate <command-line-usage>` command.
|
||||||
|
|
||||||
|
Purpose of this package is frontend for migrations. It provides commands
|
||||||
|
to manage migrate repository and database selection aswell as script versioning.
|
||||||
|
|
||||||
|
|
||||||
|
Project Setup
|
||||||
|
=============
|
||||||
|
|
||||||
|
.. _create_change_repository:
|
||||||
|
|
||||||
|
Create a change repository
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
To begin, we'll need to create a *repository* for our
|
||||||
|
project. Repositories are associated with a single database schema,
|
||||||
|
and store collections of change scripts to manage that schema. The
|
||||||
|
scripts in a repository may be applied to any number of databases.
|
||||||
|
|
||||||
|
Repositories each have a name. This name is used to identify the
|
||||||
|
repository we're working with.
|
||||||
|
|
||||||
|
All work with repositories is done using the migrate command. Let's
|
||||||
|
create our project's repository::
|
||||||
|
|
||||||
|
$ migrate create my_repository "Example project"
|
||||||
|
|
||||||
|
This creates an initially empty repository relative to current directory at
|
||||||
|
my_repository/ named `Example project`. The repository directory
|
||||||
|
contains a sub directory versions that will store the schema versions,
|
||||||
|
a configuration file :file:`migrate.cfg` that contains
|
||||||
|
:ref:`repository configuration <repository_configuration>`, a
|
||||||
|
:file:`README` file containing information that the directory is an
|
||||||
|
sqlalchemy-migrate repository and a script :ref:`manage.py <project_management_script>`
|
||||||
|
that has the same functionality as the :ref:`migrate <command-line-usage>` command but is
|
||||||
|
preconfigured with the repository.
|
||||||
|
|
||||||
|
Version-control a database
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
Next, we need to create a database and declare it to be under version
|
||||||
|
control. Information on a database's version is stored in the database
|
||||||
|
itself; declaring a database to be under version control creates a
|
||||||
|
table, named 'migrate_version' by default, and associates it with your
|
||||||
|
repository.
|
||||||
|
|
||||||
|
The database is specified as a `SQLAlchemy database url`_.
|
||||||
|
|
||||||
|
.. _`sqlalchemy database url`:
|
||||||
|
http://www.sqlalchemy.org/docs/05/dbengine.html#create-engine-url-arguments
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
$ python my_repository/manage.py version_control sqlite:///project.db
|
||||||
|
|
||||||
|
We can have any number of databases under this repository's version
|
||||||
|
control.
|
||||||
|
|
||||||
|
Each schema has a version that SQLAlchemy Migrate manages. Each change
|
||||||
|
script applied to the database increments this version number. You can
|
||||||
|
see a database's current version::
|
||||||
|
|
||||||
|
$ python my_repository/manage.py db_version sqlite:///project.db
|
||||||
|
0
|
||||||
|
|
||||||
|
A freshly versioned database begins at version 0 by default. This
|
||||||
|
assumes the database is empty. (If this is a bad assumption, you can
|
||||||
|
specify the version at the time the database is declared under version
|
||||||
|
control, with the "version_control" command.) We'll see that creating
|
||||||
|
and applying change scripts changes the database's version number.
|
||||||
|
|
||||||
|
Similarly, we can also see the latest version available in a
|
||||||
|
repository with the command::
|
||||||
|
|
||||||
|
$ python my_repository/manage.py version
|
||||||
|
0
|
||||||
|
|
||||||
|
We've entered no changes so far, so our repository cannot upgrade a
|
||||||
|
database past version 0.
|
||||||
|
|
||||||
|
Project management script
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
.. _project_management_script:
|
||||||
|
|
||||||
|
Many commands need to know our project's database url and repository
|
||||||
|
path - typing them each time is tedious. We can create a script for
|
||||||
|
our project that remembers the database and repository we're using,
|
||||||
|
and use it to perform commands::
|
||||||
|
|
||||||
|
$ migrate manage manage.py --repository=my_repository --url=sqlite:///project.db
|
||||||
|
$ python manage.py db_version
|
||||||
|
0
|
||||||
|
|
||||||
|
The script manage.py was created. All commands we perform with it are
|
||||||
|
the same as those performed with the 'migrate' tool, using the
|
||||||
|
repository and database connection entered above. The difference
|
||||||
|
between the script :file:`manage.py` in the current directory and the
|
||||||
|
script inside the repository is, that the one in the current directory
|
||||||
|
has the database URL preconfigured.
|
||||||
|
|
||||||
|
|
||||||
|
Making schema changes
|
||||||
|
=====================
|
||||||
|
|
||||||
|
All changes to a database schema under version control should be done
|
||||||
|
via change scripts - you should avoid schema modifications (creating
|
||||||
|
tables, etc.) outside of change scripts. This allows you to determine
|
||||||
|
what the schema looks like based on the version number alone, and
|
||||||
|
helps ensure multiple databases you're working with are consistent.
|
||||||
|
|
||||||
|
Create a change script
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
Our first change script will create a simple table
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
account = Table('account', meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
Column('login', String(40)),
|
||||||
|
Column('passwd', String(40)),
|
||||||
|
)
|
||||||
|
|
||||||
|
This table should be created in a change script. Let's create one::
|
||||||
|
|
||||||
|
$ python manage.py script "Add account table"
|
||||||
|
|
||||||
|
This creates an empty change script at
|
||||||
|
:file:`my_repository/versions/001_Add_account_table.py`. Next, we'll
|
||||||
|
edit this script to create our table.
|
||||||
|
|
||||||
|
Edit the change script
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
Our change script defines two functions, currently empty:
|
||||||
|
:func:`upgrade`` and :func:`downgrade`. We'll fill those in
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
|
||||||
|
meta = MetaData()
|
||||||
|
account = Table('account', meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
Column('login', String(40)),
|
||||||
|
Column('passwd', String(40)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def upgrade(migrate_engine):
|
||||||
|
meta.bind(migrate_engine)
|
||||||
|
account.create()
|
||||||
|
|
||||||
|
def downgrade(migrate_engine):
|
||||||
|
meta.bind(migrate_engine)
|
||||||
|
account.drop()
|
||||||
|
|
||||||
|
As you might have guessed, :func:`upgrade` upgrades the database to the next
|
||||||
|
version. This function should contain the changes we want to perform;
|
||||||
|
here, we're creating a table. :func:`downgrade` should reverse changes made
|
||||||
|
by :func:`upgrade`. You'll need to write both functions for every change
|
||||||
|
script. (Well, you don't *have* to write downgrade, but you won't be
|
||||||
|
able to revert to an older version of the database or test your
|
||||||
|
scripts without it.)
|
||||||
|
|
||||||
|
As you can see, **migrate_engine** is passed to both functions.
|
||||||
|
You should use this in your change scripts, rather
|
||||||
|
than creating your own engine.
|
||||||
|
|
||||||
|
You should be very careful about importing files from the rest of your
|
||||||
|
application, as your change scripts might break when your application
|
||||||
|
changes. More about `writing scripts with consistent behavior`_.
|
||||||
|
|
||||||
|
Test the change script
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
Change scripts should be tested before they are committed. Testing a
|
||||||
|
script will run its :func:`upgrade` and :func:`downgrade` functions on a specified
|
||||||
|
database; you can ensure the script runs without error. You should be
|
||||||
|
testing on a test database - if something goes wrong here, you'll need
|
||||||
|
to correct it by hand. If the test is successful, the database should
|
||||||
|
appear unchanged after upgrade() and downgrade() run.
|
||||||
|
|
||||||
|
To test the script::
|
||||||
|
|
||||||
|
$ python manage.py test
|
||||||
|
Upgrading... done
|
||||||
|
Downgrading... done
|
||||||
|
Success
|
||||||
|
|
||||||
|
Our script runs on our database (``sqlite:///project.db``, as
|
||||||
|
specified in manage.py) without any errors.
|
||||||
|
|
||||||
|
Our repository's version now is::
|
||||||
|
|
||||||
|
$ python manage.py version
|
||||||
|
1
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
test command exectues actual script, be sure you are NOT doing this on production database.
|
||||||
|
|
||||||
|
|
||||||
|
Upgrade the database
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
Now, we can apply this change script to our database::
|
||||||
|
|
||||||
|
$ python manage.py upgrade
|
||||||
|
0 -> 1... done
|
||||||
|
|
||||||
|
This upgrades the database (``sqlite:///project.db``, as specified
|
||||||
|
when we created manage.py above) to the latest available version. (We
|
||||||
|
could also specify a version number if we wished, using the ``--version``
|
||||||
|
option.) We can see the database's version number has changed, and our
|
||||||
|
table has been created::
|
||||||
|
|
||||||
|
$ python manage.py db_version
|
||||||
|
1
|
||||||
|
$ sqlite3 project.db
|
||||||
|
sqlite> .tables
|
||||||
|
account migrate_version
|
||||||
|
|
||||||
|
Our account table was created - success! As our application evolves,
|
||||||
|
we can create more change scripts using a similar process.
|
||||||
|
|
||||||
|
Writing change scripts
|
||||||
|
======================
|
||||||
|
|
||||||
|
By default, change scripts may do anything any other SQLAlchemy
|
||||||
|
program can do.
|
||||||
|
|
||||||
|
SQLAlchemy Migrate extends SQLAlchemy with several operations used to
|
||||||
|
change existing schemas - ie. ``ALTER TABLE`` stuff. See
|
||||||
|
:ref:`changeset <changeset-system>` documentation for details.
|
||||||
|
|
||||||
|
|
||||||
|
Writing scripts with consistent behavior
|
||||||
|
----------------------------------------
|
||||||
|
|
||||||
|
Normally, it's important to write change scripts in a way that's
|
||||||
|
independent of your application - the same SQL should be generated
|
||||||
|
every time, despite any changes to your app's source code. You don't
|
||||||
|
want your change scripts' behavior changing when your source code
|
||||||
|
does.
|
||||||
|
|
||||||
|
**Consider the following example of what can go wrong (i.e. what NOT to
|
||||||
|
do)**:
|
||||||
|
|
||||||
|
Your application defines a table in the model.py file:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
|
||||||
|
meta = MetaData()
|
||||||
|
table = Table('mytable', meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
... and uses this file to create a table in a change script:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
import model
|
||||||
|
|
||||||
|
def upgrade(migrate_engine):
|
||||||
|
model.meta.bind(migrate_engine)
|
||||||
|
|
||||||
|
def downgrade(migrate_engine):
|
||||||
|
model.meta.bind(migrate_engine)
|
||||||
|
model.table.drop()
|
||||||
|
|
||||||
|
This runs successfully the first time. But what happens if we change
|
||||||
|
the table definition?
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
table = Table('mytable', meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
Column('data', String(42)),
|
||||||
|
)
|
||||||
|
|
||||||
|
We'll create a new column with a matching change script
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
import model
|
||||||
|
|
||||||
|
def upgrade(migrate_engine):
|
||||||
|
model.meta.bind(migrate_engine)
|
||||||
|
model.table.data.create()
|
||||||
|
def downgrade(migrate_engine):
|
||||||
|
model.meta.bind(migrate_engine)
|
||||||
|
model.table.data.drop()
|
||||||
|
|
||||||
|
This appears to run fine when upgrading an existing database - but the
|
||||||
|
first script's behavior changed! Running all our change scripts on a
|
||||||
|
new database will result in an error - the first script creates the
|
||||||
|
table based on the new definition, with both columns; the second
|
||||||
|
cannot add the column because it already exists.
|
||||||
|
|
||||||
|
To avoid the above problem, you should copy-paste your table
|
||||||
|
definition into each change script rather than importing parts of your
|
||||||
|
application.
|
||||||
|
|
||||||
|
Writing for a specific database
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
Sometimes you need to write code for a specific database. Migrate
|
||||||
|
scripts can run under any database, however - the engine you're given
|
||||||
|
might belong to any database. Use engine.name to get the name of the
|
||||||
|
database you're working with
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
>>> from sqlalchemy import *
|
||||||
|
>>> from migrate import *
|
||||||
|
>>>
|
||||||
|
>>> engine = create_engine('sqlite:///:memory:')
|
||||||
|
>>> engine.name
|
||||||
|
'sqlite'
|
||||||
|
|
||||||
|
Writings .sql scripts
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
You might prefer to write your change scripts in SQL, as .sql files,
|
||||||
|
rather than as Python scripts. SQLAlchemy-migrate can work with that
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
$ python manage.py version
|
||||||
|
1
|
||||||
|
$ python manage.py script_sql postgres
|
||||||
|
|
||||||
|
This creates two scripts
|
||||||
|
:file:`my_repository/versions/002_postgresql_upgrade.sql` and
|
||||||
|
:file:`my_repository/versions/002_postgresql_downgrade.sql`, one for
|
||||||
|
each *operation*, or function defined in a Python change script -
|
||||||
|
upgrade and downgrade. Both are specified to run with Postgres
|
||||||
|
databases - we can add more for different databases if we like. Any
|
||||||
|
database defined by SQLAlchemy may be used here - ex. sqlite,
|
||||||
|
postgres, oracle, mysql...
|
||||||
|
|
||||||
|
|
||||||
|
.. _command-line-usage:
|
||||||
|
|
||||||
|
Command line usage
|
||||||
|
==================
|
||||||
|
|
||||||
|
.. currentmodule:: migrate.versioning.shell
|
||||||
|
|
||||||
|
:command:`migrate` command is used for API interface. For list of commands and help use::
|
||||||
|
|
||||||
|
$ migrate --help
|
||||||
|
|
||||||
|
:program:`migrate` command exectues :func:`main` function.
|
||||||
|
For ease of usage, generate your own :ref:`project management script <project_management_script>`,
|
||||||
|
which calls :func:`main` function with keywords arguments.
|
||||||
|
You may want to specify `url` and `repository` arguments which almost all API functions require.
|
||||||
|
|
||||||
|
If api command looks like::
|
||||||
|
|
||||||
|
$ migrate downgrade URL REPOSITORY VERSION [--preview_sql|--preview_py]
|
||||||
|
|
||||||
|
and you have a project management script that looks like
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from migrate.versioning.shell import main
|
||||||
|
|
||||||
|
main(url='sqlite://', repository='./project/migrations/')
|
||||||
|
|
||||||
|
you have first two slots filed, and command line usage would look like::
|
||||||
|
|
||||||
|
# preview Python script
|
||||||
|
migrate downgrade 2 --preview_py
|
||||||
|
|
||||||
|
# downgrade to version 2
|
||||||
|
migrate downgrade 2
|
||||||
|
|
||||||
|
.. versionchanged:: 0.5.4
|
||||||
|
Command line parsing refactored: positional parameters usage
|
||||||
|
|
||||||
|
Whole command line parsing was rewriten from scratch with use of OptionParser.
|
||||||
|
Options passed as kwargs to :func:`~migrate.versioning.shell.main` are now parsed correctly.
|
||||||
|
Options are passed to commands in the following priority (starting from highest):
|
||||||
|
|
||||||
|
- optional (given by ``--some_option`` in commandline)
|
||||||
|
- positional arguments
|
||||||
|
- kwargs passed to migrate.versioning.shell.main
|
||||||
|
|
||||||
|
|
||||||
|
Python API
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. currentmodule:: migrate.versioning.api
|
||||||
|
|
||||||
|
All commands available from the command line are also available for
|
||||||
|
your Python scripts by importing :mod:`migrate.versioning.api`. See the
|
||||||
|
:mod:`migrate.versioning.api` documentation for a list of functions;
|
||||||
|
function names match equivalent shell commands. You can use this to
|
||||||
|
help integrate SQLAlchemy Migrate with your existing update process.
|
||||||
|
|
||||||
|
For example, the following commands are similar:
|
||||||
|
|
||||||
|
*From the command line*::
|
||||||
|
|
||||||
|
$ migrate help help
|
||||||
|
/usr/bin/migrate help COMMAND
|
||||||
|
|
||||||
|
Displays help on a given command.
|
||||||
|
|
||||||
|
*From Python*
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
import migrate.versioning.api
|
||||||
|
migrate.versioning.api.help('help')
|
||||||
|
# Output:
|
||||||
|
# %prog help COMMAND
|
||||||
|
#
|
||||||
|
# Displays help on a given command.
|
||||||
|
|
||||||
|
|
||||||
|
.. _migrate.versioning.api: module-migrate.versioning.api.html
|
||||||
|
|
||||||
|
.. _repository_configuration:
|
||||||
|
|
||||||
|
Experimental commands
|
||||||
|
=====================
|
||||||
|
|
||||||
|
Some interesting new features to create SQLAlchemy db models from
|
||||||
|
existing databases and vice versa were developed by Christian Simms
|
||||||
|
during the development of SQLAlchemy-migrate 0.4.5. These features are
|
||||||
|
roughly documented in a `thread in migrate-users`_.
|
||||||
|
|
||||||
|
.. _`thread in migrate-users`:
|
||||||
|
http://groups.google.com/group/migrate-users/browse_thread/thread/a5605184e08abf33#msg_85c803b71b29993f
|
||||||
|
|
||||||
|
Here are the commands' descriptions as given by ``migrate help <command>``:
|
||||||
|
|
||||||
|
- ``compare_model_to_db``: Compare the current model (assumed to be a
|
||||||
|
module level variable of type sqlalchemy.MetaData) against the
|
||||||
|
current database.
|
||||||
|
- ``create_model``: Dump the current database as a Python model to
|
||||||
|
stdout.
|
||||||
|
- ``make_update_script_for_model``: Create a script changing the old
|
||||||
|
Python model to the new (current) Python model, sending to stdout.
|
||||||
|
- ``upgrade_db_from_model``: Modify the database to match the
|
||||||
|
structure of the current Python model. This also sets the db_version
|
||||||
|
number to the latest in the repository.
|
||||||
|
|
||||||
|
As this sections headline says: These features are EXPERIMENTAL. Take
|
||||||
|
the necessary arguments to the commands from the output of ``migrate
|
||||||
|
help <command>``.
|
||||||
|
|
||||||
|
Repository configuration
|
||||||
|
========================
|
||||||
|
|
||||||
|
SQLAlchemy-migrate repositories can be configured in their migrate.cfg
|
||||||
|
files. The initial configuration is performed by the `migrate create`
|
||||||
|
call explained in :ref:`Create a change repository
|
||||||
|
<create_change_repository>`. The following options are available
|
||||||
|
currently:
|
||||||
|
|
||||||
|
- `repository_id` Used to identify which repository this database is
|
||||||
|
versioned under. You can use the name of your project.
|
||||||
|
- `version_table` The name of the database table used to track the
|
||||||
|
schema version. This name shouldn't already be used by your
|
||||||
|
project. If this is changed once a database is under version
|
||||||
|
control, you'll need to change the table name in each database too.
|
||||||
|
- `required_dbs` When committing a change script, SQLAlchemy-migrate
|
||||||
|
will attempt to generate the sql for all supported databases;
|
||||||
|
normally, if one of them fails - probably because you don't have
|
||||||
|
that database installed - it is ignored and the commit continues,
|
||||||
|
perhaps ending successfully. Databases in this list MUST compile
|
||||||
|
successfully during a commit, or the entire commit will fail. List
|
||||||
|
the databases your application will actually be using to ensure your
|
||||||
|
updates to that database work properly. This must be a list;
|
||||||
|
example: `['postgres', 'sqlite']`
|
||||||
6
migrate/__init__.py
Normal file
6
migrate/__init__.py
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
"""
|
||||||
|
SQLAlchemy migrate provides two APIs :mod:`migrate.versioning` for
|
||||||
|
database schema version and repository management and
|
||||||
|
:mod:`migrate.changeset` that allows to define database schema changes
|
||||||
|
using Python.
|
||||||
|
"""
|
||||||
BIN
migrate/__init__.pyc
Normal file
BIN
migrate/__init__.pyc
Normal file
Binary file not shown.
28
migrate/changeset/__init__.py
Normal file
28
migrate/changeset/__init__.py
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
"""
|
||||||
|
This module extends SQLAlchemy and provides additional DDL [#]_
|
||||||
|
support.
|
||||||
|
|
||||||
|
.. [#] SQL Data Definition Language
|
||||||
|
"""
|
||||||
|
import sqlalchemy
|
||||||
|
|
||||||
|
from sqlalchemy import __version__ as _sa_version
|
||||||
|
import re
|
||||||
|
|
||||||
|
_sa_version = tuple(int(re.match("\d+", x).group(0)) for x in _sa_version.split("."))
|
||||||
|
SQLA_06 = _sa_version >= (0, 6)
|
||||||
|
|
||||||
|
del re
|
||||||
|
del _sa_version
|
||||||
|
|
||||||
|
from migrate.changeset.schema import *
|
||||||
|
from migrate.changeset.constraint import *
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
sqlalchemy.schema.Table.__bases__ += (ChangesetTable, )
|
||||||
|
sqlalchemy.schema.Column.__bases__ += (ChangesetColumn, )
|
||||||
|
sqlalchemy.schema.Index.__bases__ += (ChangesetIndex, )
|
||||||
|
|
||||||
|
sqlalchemy.schema.DefaultClause.__bases__ += (ChangesetDefaultClause, )
|
||||||
340
migrate/changeset/ansisql.py
Normal file
340
migrate/changeset/ansisql.py
Normal file
@ -0,0 +1,340 @@
|
|||||||
|
"""
|
||||||
|
Extensions to SQLAlchemy for altering existing tables.
|
||||||
|
|
||||||
|
At the moment, this isn't so much based off of ANSI as much as
|
||||||
|
things that just happen to work with multiple databases.
|
||||||
|
"""
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from sqlalchemy.schema import SchemaVisitor
|
||||||
|
from sqlalchemy.engine.default import DefaultDialect
|
||||||
|
from sqlalchemy.schema import (ForeignKeyConstraint,
|
||||||
|
PrimaryKeyConstraint,
|
||||||
|
CheckConstraint,
|
||||||
|
UniqueConstraint,
|
||||||
|
Index)
|
||||||
|
|
||||||
|
from migrate.changeset import exceptions, constraint, SQLA_06
|
||||||
|
import StringIO
|
||||||
|
|
||||||
|
if not SQLA_06:
|
||||||
|
from sqlalchemy.sql.compiler import SchemaGenerator, SchemaDropper
|
||||||
|
else:
|
||||||
|
from sqlalchemy.schema import AddConstraint, DropConstraint
|
||||||
|
from sqlalchemy.sql.compiler import DDLCompiler
|
||||||
|
SchemaGenerator = SchemaDropper = DDLCompiler
|
||||||
|
|
||||||
|
class AlterTableVisitor(SchemaVisitor):
|
||||||
|
"""Common operations for ``ALTER TABLE`` statements."""
|
||||||
|
|
||||||
|
def append(self, s):
|
||||||
|
"""Append content to the SchemaIterator's query buffer."""
|
||||||
|
|
||||||
|
self.buffer.write(s)
|
||||||
|
|
||||||
|
def execute(self):
|
||||||
|
"""Execute the contents of the SchemaIterator's buffer."""
|
||||||
|
|
||||||
|
try:
|
||||||
|
return self.connection.execute(self.buffer.getvalue())
|
||||||
|
finally:
|
||||||
|
self.buffer.truncate(0)
|
||||||
|
|
||||||
|
def __init__(self, dialect, connection, **kw):
|
||||||
|
self.connection = connection
|
||||||
|
self.buffer = StringIO.StringIO()
|
||||||
|
self.preparer = dialect.identifier_preparer
|
||||||
|
self.dialect = dialect
|
||||||
|
|
||||||
|
def traverse_single(self, elem):
|
||||||
|
ret = super(AlterTableVisitor, self).traverse_single(elem)
|
||||||
|
if ret:
|
||||||
|
# adapt to 0.6 which uses a string-returning
|
||||||
|
# object
|
||||||
|
self.append(ret)
|
||||||
|
|
||||||
|
def _to_table(self, param):
|
||||||
|
"""Returns the table object for the given param object."""
|
||||||
|
if isinstance(param, (sa.Column, sa.Index, sa.schema.Constraint)):
|
||||||
|
ret = param.table
|
||||||
|
else:
|
||||||
|
ret = param
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def start_alter_table(self, param):
|
||||||
|
"""Returns the start of an ``ALTER TABLE`` SQL-Statement.
|
||||||
|
|
||||||
|
Use the param object to determine the table name and use it
|
||||||
|
for building the SQL statement.
|
||||||
|
|
||||||
|
:param param: object to determine the table from
|
||||||
|
:type param: :class:`sqlalchemy.Column`, :class:`sqlalchemy.Index`,
|
||||||
|
:class:`sqlalchemy.schema.Constraint`, :class:`sqlalchemy.Table`,
|
||||||
|
or string (table name)
|
||||||
|
"""
|
||||||
|
table = self._to_table(param)
|
||||||
|
self.append('\nALTER TABLE %s ' % self.preparer.format_table(table))
|
||||||
|
return table
|
||||||
|
|
||||||
|
|
||||||
|
class ANSIColumnGenerator(AlterTableVisitor, SchemaGenerator):
|
||||||
|
"""Extends ansisql generator for column creation (alter table add col)"""
|
||||||
|
|
||||||
|
def visit_column(self, column):
|
||||||
|
"""Create a column (table already exists).
|
||||||
|
|
||||||
|
:param column: column object
|
||||||
|
:type column: :class:`sqlalchemy.Column` instance
|
||||||
|
"""
|
||||||
|
if column.default is not None:
|
||||||
|
self.traverse_single(column.default)
|
||||||
|
|
||||||
|
table = self.start_alter_table(column)
|
||||||
|
self.append("ADD ")
|
||||||
|
self.append(self.get_column_specification(column))
|
||||||
|
|
||||||
|
for cons in column.constraints:
|
||||||
|
self.traverse_single(cons)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
# ALTER TABLE STATEMENTS
|
||||||
|
|
||||||
|
# add indexes and unique constraints
|
||||||
|
if column.index_name:
|
||||||
|
ix = Index(column.index_name,
|
||||||
|
column,
|
||||||
|
unique=bool(column.index_name or column.index))
|
||||||
|
ix.create()
|
||||||
|
elif column.unique_name:
|
||||||
|
constraint.UniqueConstraint(column,
|
||||||
|
name=column.unique_name).create()
|
||||||
|
|
||||||
|
# SA bounds FK constraints to table, add manually
|
||||||
|
for fk in column.foreign_keys:
|
||||||
|
self.add_foreignkey(fk.constraint)
|
||||||
|
|
||||||
|
# add primary key constraint if needed
|
||||||
|
if column.primary_key_name:
|
||||||
|
cons = constraint.PrimaryKeyConstraint(column,
|
||||||
|
name=column.primary_key_name)
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
if SQLA_06:
|
||||||
|
def add_foreignkey(self, fk):
|
||||||
|
self.connection.execute(AddConstraint(fk))
|
||||||
|
|
||||||
|
class ANSIColumnDropper(AlterTableVisitor, SchemaDropper):
|
||||||
|
"""Extends ANSI SQL dropper for column dropping (``ALTER TABLE
|
||||||
|
DROP COLUMN``).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def visit_column(self, column):
|
||||||
|
"""Drop a column from its table.
|
||||||
|
|
||||||
|
:param column: the column object
|
||||||
|
:type column: :class:`sqlalchemy.Column`
|
||||||
|
"""
|
||||||
|
table = self.start_alter_table(column)
|
||||||
|
self.append('DROP COLUMN %s' % self.preparer.format_column(column))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
|
||||||
|
class ANSISchemaChanger(AlterTableVisitor, SchemaGenerator):
|
||||||
|
"""Manages changes to existing schema elements.
|
||||||
|
|
||||||
|
Note that columns are schema elements; ``ALTER TABLE ADD COLUMN``
|
||||||
|
is in SchemaGenerator.
|
||||||
|
|
||||||
|
All items may be renamed. Columns can also have many of their properties -
|
||||||
|
type, for example - changed.
|
||||||
|
|
||||||
|
Each function is passed a tuple, containing (object, name); where
|
||||||
|
object is a type of object you'd expect for that function
|
||||||
|
(ie. table for visit_table) and name is the object's new
|
||||||
|
name. NONE means the name is unchanged.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def visit_table(self, table):
|
||||||
|
"""Rename a table. Other ops aren't supported."""
|
||||||
|
self.start_alter_table(table)
|
||||||
|
self.append("RENAME TO %s" % self.preparer.quote(table.new_name,
|
||||||
|
table.quote))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_index(self, index):
|
||||||
|
"""Rename an index"""
|
||||||
|
self.append("ALTER INDEX %s RENAME TO %s" %
|
||||||
|
(self.preparer.quote(self._validate_identifier(index.name,
|
||||||
|
True), index.quote),
|
||||||
|
self.preparer.quote(self._validate_identifier(index.new_name,
|
||||||
|
True), index.quote)))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_column(self, delta):
|
||||||
|
"""Rename/change a column."""
|
||||||
|
# ALTER COLUMN is implemented as several ALTER statements
|
||||||
|
keys = delta.keys()
|
||||||
|
if 'type' in keys:
|
||||||
|
self._run_subvisit(delta, self._visit_column_type)
|
||||||
|
if 'nullable' in keys:
|
||||||
|
self._run_subvisit(delta, self._visit_column_nullable)
|
||||||
|
if 'server_default' in keys:
|
||||||
|
# Skip 'default': only handle server-side defaults, others
|
||||||
|
# are managed by the app, not the db.
|
||||||
|
self._run_subvisit(delta, self._visit_column_default)
|
||||||
|
if 'name' in keys:
|
||||||
|
self._run_subvisit(delta, self._visit_column_name, start_alter=False)
|
||||||
|
|
||||||
|
def _run_subvisit(self, delta, func, start_alter=True):
|
||||||
|
"""Runs visit method based on what needs to be changed on column"""
|
||||||
|
table = self._to_table(delta.table)
|
||||||
|
col_name = delta.current_name
|
||||||
|
if start_alter:
|
||||||
|
self.start_alter_column(table, col_name)
|
||||||
|
ret = func(table, delta.result_column, delta)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def start_alter_column(self, table, col_name):
|
||||||
|
"""Starts ALTER COLUMN"""
|
||||||
|
self.start_alter_table(table)
|
||||||
|
self.append("ALTER COLUMN %s " % self.preparer.quote(col_name, table.quote))
|
||||||
|
|
||||||
|
def _visit_column_nullable(self, table, column, delta):
|
||||||
|
nullable = delta['nullable']
|
||||||
|
if nullable:
|
||||||
|
self.append("DROP NOT NULL")
|
||||||
|
else:
|
||||||
|
self.append("SET NOT NULL")
|
||||||
|
|
||||||
|
def _visit_column_default(self, table, column, delta):
|
||||||
|
default_text = self.get_column_default_string(column)
|
||||||
|
if default_text is not None:
|
||||||
|
self.append("SET DEFAULT %s" % default_text)
|
||||||
|
else:
|
||||||
|
self.append("DROP DEFAULT")
|
||||||
|
|
||||||
|
def _visit_column_type(self, table, column, delta):
|
||||||
|
type_ = delta['type']
|
||||||
|
if SQLA_06:
|
||||||
|
type_text = str(type_.compile(dialect=self.dialect))
|
||||||
|
else:
|
||||||
|
type_text = type_.dialect_impl(self.dialect).get_col_spec()
|
||||||
|
self.append("TYPE %s" % type_text)
|
||||||
|
|
||||||
|
def _visit_column_name(self, table, column, delta):
|
||||||
|
self.start_alter_table(table)
|
||||||
|
col_name = self.preparer.quote(delta.current_name, table.quote)
|
||||||
|
new_name = self.preparer.format_column(delta.result_column)
|
||||||
|
self.append('RENAME COLUMN %s TO %s' % (col_name, new_name))
|
||||||
|
|
||||||
|
|
||||||
|
class ANSIConstraintCommon(AlterTableVisitor):
|
||||||
|
"""
|
||||||
|
Migrate's constraints require a separate creation function from
|
||||||
|
SA's: Migrate's constraints are created independently of a table;
|
||||||
|
SA's are created at the same time as the table.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def get_constraint_name(self, cons):
|
||||||
|
"""Gets a name for the given constraint.
|
||||||
|
|
||||||
|
If the name is already set it will be used otherwise the
|
||||||
|
constraint's :meth:`autoname <migrate.changeset.constraint.ConstraintChangeset.autoname>`
|
||||||
|
method is used.
|
||||||
|
|
||||||
|
:param cons: constraint object
|
||||||
|
"""
|
||||||
|
if cons.name is not None:
|
||||||
|
ret = cons.name
|
||||||
|
else:
|
||||||
|
ret = cons.name = cons.autoname()
|
||||||
|
return self.preparer.quote(ret, cons.quote)
|
||||||
|
|
||||||
|
def visit_migrate_primary_key_constraint(self, *p, **k):
|
||||||
|
self._visit_constraint(*p, **k)
|
||||||
|
|
||||||
|
def visit_migrate_foreign_key_constraint(self, *p, **k):
|
||||||
|
self._visit_constraint(*p, **k)
|
||||||
|
|
||||||
|
def visit_migrate_check_constraint(self, *p, **k):
|
||||||
|
self._visit_constraint(*p, **k)
|
||||||
|
|
||||||
|
def visit_migrate_unique_constraint(self, *p, **k):
|
||||||
|
self._visit_constraint(*p, **k)
|
||||||
|
|
||||||
|
if SQLA_06:
|
||||||
|
class ANSIConstraintGenerator(ANSIConstraintCommon, SchemaGenerator):
|
||||||
|
def _visit_constraint(self, constraint):
|
||||||
|
constraint.name = self.get_constraint_name(constraint)
|
||||||
|
self.append(self.process(AddConstraint(constraint)))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
class ANSIConstraintDropper(ANSIConstraintCommon, SchemaDropper):
|
||||||
|
def _visit_constraint(self, constraint):
|
||||||
|
constraint.name = self.get_constraint_name(constraint)
|
||||||
|
self.append(self.process(DropConstraint(constraint, cascade=constraint.cascade)))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
else:
|
||||||
|
class ANSIConstraintGenerator(ANSIConstraintCommon, SchemaGenerator):
|
||||||
|
|
||||||
|
def get_constraint_specification(self, cons, **kwargs):
|
||||||
|
"""Constaint SQL generators.
|
||||||
|
|
||||||
|
We cannot use SA visitors because they append comma.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if isinstance(cons, PrimaryKeyConstraint):
|
||||||
|
if cons.name is not None:
|
||||||
|
self.append("CONSTRAINT %s " % self.preparer.format_constraint(cons))
|
||||||
|
self.append("PRIMARY KEY ")
|
||||||
|
self.append("(%s)" % ', '.join(self.preparer.quote(c.name, c.quote)
|
||||||
|
for c in cons))
|
||||||
|
self.define_constraint_deferrability(cons)
|
||||||
|
elif isinstance(cons, ForeignKeyConstraint):
|
||||||
|
self.define_foreign_key(cons)
|
||||||
|
elif isinstance(cons, CheckConstraint):
|
||||||
|
if cons.name is not None:
|
||||||
|
self.append("CONSTRAINT %s " %
|
||||||
|
self.preparer.format_constraint(cons))
|
||||||
|
self.append("CHECK (%s)" % cons.sqltext)
|
||||||
|
self.define_constraint_deferrability(cons)
|
||||||
|
elif isinstance(cons, UniqueConstraint):
|
||||||
|
if cons.name is not None:
|
||||||
|
self.append("CONSTRAINT %s " %
|
||||||
|
self.preparer.format_constraint(cons))
|
||||||
|
self.append("UNIQUE (%s)" % \
|
||||||
|
(', '.join(self.preparer.quote(c.name, c.quote) for c in cons)))
|
||||||
|
self.define_constraint_deferrability(cons)
|
||||||
|
else:
|
||||||
|
raise exceptions.InvalidConstraintError(cons)
|
||||||
|
|
||||||
|
def _visit_constraint(self, constraint):
|
||||||
|
|
||||||
|
table = self.start_alter_table(constraint)
|
||||||
|
constraint.name = self.get_constraint_name(constraint)
|
||||||
|
self.append("ADD ")
|
||||||
|
self.get_constraint_specification(constraint)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
|
||||||
|
class ANSIConstraintDropper(ANSIConstraintCommon, SchemaDropper):
|
||||||
|
|
||||||
|
def _visit_constraint(self, constraint):
|
||||||
|
self.start_alter_table(constraint)
|
||||||
|
self.append("DROP CONSTRAINT ")
|
||||||
|
constraint.name = self.get_constraint_name(constraint)
|
||||||
|
self.append(self.preparer.format_constraint(constraint))
|
||||||
|
if constraint.cascade:
|
||||||
|
self.cascade_constraint(constraint)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def cascade_constraint(self, constraint):
|
||||||
|
self.append(" CASCADE")
|
||||||
|
|
||||||
|
|
||||||
|
class ANSIDialect(DefaultDialect):
|
||||||
|
columngenerator = ANSIColumnGenerator
|
||||||
|
columndropper = ANSIColumnDropper
|
||||||
|
schemachanger = ANSISchemaChanger
|
||||||
|
constraintgenerator = ANSIConstraintGenerator
|
||||||
|
constraintdropper = ANSIConstraintDropper
|
||||||
190
migrate/changeset/constraint.py
Normal file
190
migrate/changeset/constraint.py
Normal file
@ -0,0 +1,190 @@
|
|||||||
|
"""
|
||||||
|
This module defines standalone schema constraint classes.
|
||||||
|
"""
|
||||||
|
from sqlalchemy import schema
|
||||||
|
|
||||||
|
from migrate.changeset.exceptions import *
|
||||||
|
from migrate.changeset import SQLA_06
|
||||||
|
|
||||||
|
class ConstraintChangeset(object):
|
||||||
|
"""Base class for Constraint classes."""
|
||||||
|
|
||||||
|
def _normalize_columns(self, cols, table_name=False):
|
||||||
|
"""Given: column objects or names; return col names and
|
||||||
|
(maybe) a table"""
|
||||||
|
colnames = []
|
||||||
|
table = None
|
||||||
|
for col in cols:
|
||||||
|
if isinstance(col, schema.Column):
|
||||||
|
if col.table is not None and table is None:
|
||||||
|
table = col.table
|
||||||
|
if table_name:
|
||||||
|
col = '.'.join((col.table.name, col.name))
|
||||||
|
else:
|
||||||
|
col = col.name
|
||||||
|
colnames.append(col)
|
||||||
|
return colnames, table
|
||||||
|
|
||||||
|
def __do_imports(self, visitor_name, *a, **kw):
|
||||||
|
engine = kw.pop('engine', self.table.bind)
|
||||||
|
from migrate.changeset.databases.visitor import (get_engine_visitor,
|
||||||
|
run_single_visitor)
|
||||||
|
visitorcallable = get_engine_visitor(engine, visitor_name)
|
||||||
|
run_single_visitor(engine, visitorcallable, self, *a, **kw)
|
||||||
|
|
||||||
|
def create(self, *a, **kw):
|
||||||
|
"""Create the constraint in the database.
|
||||||
|
|
||||||
|
:param engine: the database engine to use. If this is \
|
||||||
|
:keyword:`None` the instance's engine will be used
|
||||||
|
:type engine: :class:`sqlalchemy.engine.base.Engine`
|
||||||
|
"""
|
||||||
|
# TODO: set the parent here instead of in __init__
|
||||||
|
self.__do_imports('constraintgenerator', *a, **kw)
|
||||||
|
|
||||||
|
def drop(self, *a, **kw):
|
||||||
|
"""Drop the constraint from the database.
|
||||||
|
|
||||||
|
:param engine: the database engine to use. If this is
|
||||||
|
:keyword:`None` the instance's engine will be used
|
||||||
|
:param cascade: Issue CASCADE drop if database supports it
|
||||||
|
:type engine: :class:`sqlalchemy.engine.base.Engine`
|
||||||
|
:type cascade: bool
|
||||||
|
:returns: Instance with cleared columns
|
||||||
|
"""
|
||||||
|
self.cascade = kw.pop('cascade', False)
|
||||||
|
self.__do_imports('constraintdropper', *a, **kw)
|
||||||
|
# the spirit of Constraint objects is that they
|
||||||
|
# are immutable (just like in a DB. they're only ADDed
|
||||||
|
# or DROPped).
|
||||||
|
#self.columns.clear()
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
class PrimaryKeyConstraint(ConstraintChangeset, schema.PrimaryKeyConstraint):
|
||||||
|
"""Construct PrimaryKeyConstraint
|
||||||
|
|
||||||
|
Migrate's additional parameters:
|
||||||
|
|
||||||
|
:param cols: Columns in constraint.
|
||||||
|
:param table: If columns are passed as strings, this kw is required
|
||||||
|
:type table: Table instance
|
||||||
|
:type cols: strings or Column instances
|
||||||
|
"""
|
||||||
|
|
||||||
|
__migrate_visit_name__ = 'migrate_primary_key_constraint'
|
||||||
|
|
||||||
|
def __init__(self, *cols, **kwargs):
|
||||||
|
colnames, table = self._normalize_columns(cols)
|
||||||
|
table = kwargs.pop('table', table)
|
||||||
|
super(PrimaryKeyConstraint, self).__init__(*colnames, **kwargs)
|
||||||
|
if table is not None:
|
||||||
|
self._set_parent(table)
|
||||||
|
|
||||||
|
|
||||||
|
def autoname(self):
|
||||||
|
"""Mimic the database's automatic constraint names"""
|
||||||
|
return "%s_pkey" % self.table.name
|
||||||
|
|
||||||
|
|
||||||
|
class ForeignKeyConstraint(ConstraintChangeset, schema.ForeignKeyConstraint):
|
||||||
|
"""Construct ForeignKeyConstraint
|
||||||
|
|
||||||
|
Migrate's additional parameters:
|
||||||
|
|
||||||
|
:param columns: Columns in constraint
|
||||||
|
:param refcolumns: Columns that this FK reffers to in another table.
|
||||||
|
:param table: If columns are passed as strings, this kw is required
|
||||||
|
:type table: Table instance
|
||||||
|
:type columns: list of strings or Column instances
|
||||||
|
:type refcolumns: list of strings or Column instances
|
||||||
|
"""
|
||||||
|
|
||||||
|
__migrate_visit_name__ = 'migrate_foreign_key_constraint'
|
||||||
|
|
||||||
|
def __init__(self, columns, refcolumns, *args, **kwargs):
|
||||||
|
colnames, table = self._normalize_columns(columns)
|
||||||
|
table = kwargs.pop('table', table)
|
||||||
|
refcolnames, reftable = self._normalize_columns(refcolumns,
|
||||||
|
table_name=True)
|
||||||
|
super(ForeignKeyConstraint, self).__init__(colnames, refcolnames, *args,
|
||||||
|
**kwargs)
|
||||||
|
if table is not None:
|
||||||
|
self._set_parent(table)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def referenced(self):
|
||||||
|
return [e.column for e in self.elements]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def reftable(self):
|
||||||
|
return self.referenced[0].table
|
||||||
|
|
||||||
|
def autoname(self):
|
||||||
|
"""Mimic the database's automatic constraint names"""
|
||||||
|
ret = "%(table)s_%(reftable)s_fkey" % dict(
|
||||||
|
table=self.table.name,
|
||||||
|
reftable=self.reftable.name,)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
|
||||||
|
class CheckConstraint(ConstraintChangeset, schema.CheckConstraint):
|
||||||
|
"""Construct CheckConstraint
|
||||||
|
|
||||||
|
Migrate's additional parameters:
|
||||||
|
|
||||||
|
:param sqltext: Plain SQL text to check condition
|
||||||
|
:param columns: If not name is applied, you must supply this kw\
|
||||||
|
to autoname constraint
|
||||||
|
:param table: If columns are passed as strings, this kw is required
|
||||||
|
:type table: Table instance
|
||||||
|
:type columns: list of Columns instances
|
||||||
|
:type sqltext: string
|
||||||
|
"""
|
||||||
|
|
||||||
|
__migrate_visit_name__ = 'migrate_check_constraint'
|
||||||
|
|
||||||
|
def __init__(self, sqltext, *args, **kwargs):
|
||||||
|
cols = kwargs.pop('columns', [])
|
||||||
|
if not cols and not kwargs.get('name', False):
|
||||||
|
raise InvalidConstraintError('You must either set "name"'
|
||||||
|
'parameter or "columns" to autogenarate it.')
|
||||||
|
colnames, table = self._normalize_columns(cols)
|
||||||
|
table = kwargs.pop('table', table)
|
||||||
|
schema.CheckConstraint.__init__(self, sqltext, *args, **kwargs)
|
||||||
|
if table is not None:
|
||||||
|
if not SQLA_06:
|
||||||
|
self.table = table
|
||||||
|
self._set_parent(table)
|
||||||
|
self.colnames = colnames
|
||||||
|
|
||||||
|
def autoname(self):
|
||||||
|
return "%(table)s_%(cols)s_check" % \
|
||||||
|
dict(table=self.table.name, cols="_".join(self.colnames))
|
||||||
|
|
||||||
|
|
||||||
|
class UniqueConstraint(ConstraintChangeset, schema.UniqueConstraint):
|
||||||
|
"""Construct UniqueConstraint
|
||||||
|
|
||||||
|
Migrate's additional parameters:
|
||||||
|
|
||||||
|
:param cols: Columns in constraint.
|
||||||
|
:param table: If columns are passed as strings, this kw is required
|
||||||
|
:type table: Table instance
|
||||||
|
:type cols: strings or Column instances
|
||||||
|
|
||||||
|
.. versionadded:: 0.5.5
|
||||||
|
"""
|
||||||
|
|
||||||
|
__migrate_visit_name__ = 'migrate_unique_constraint'
|
||||||
|
|
||||||
|
def __init__(self, *cols, **kwargs):
|
||||||
|
self.colnames, table = self._normalize_columns(cols)
|
||||||
|
table = kwargs.pop('table', table)
|
||||||
|
super(UniqueConstraint, self).__init__(*self.colnames, **kwargs)
|
||||||
|
if table is not None:
|
||||||
|
self._set_parent(table)
|
||||||
|
|
||||||
|
def autoname(self):
|
||||||
|
"""Mimic the database's automatic constraint names"""
|
||||||
|
return "%s_%s_key" % (self.table.name, self.colnames[0])
|
||||||
10
migrate/changeset/databases/__init__.py
Normal file
10
migrate/changeset/databases/__init__.py
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
"""
|
||||||
|
This module contains database dialect specific changeset
|
||||||
|
implementations.
|
||||||
|
"""
|
||||||
|
__all__ = [
|
||||||
|
'postgres',
|
||||||
|
'sqlite',
|
||||||
|
'mysql',
|
||||||
|
'oracle',
|
||||||
|
]
|
||||||
63
migrate/changeset/databases/firebird.py
Normal file
63
migrate/changeset/databases/firebird.py
Normal file
@ -0,0 +1,63 @@
|
|||||||
|
"""
|
||||||
|
Firebird database specific implementations of changeset classes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from migrate.changeset import ansisql, exceptions
|
||||||
|
|
||||||
|
# TODO: SQLA 0.6 has not migrated the FB dialect over yet
|
||||||
|
from sqlalchemy.databases import firebird as sa_base
|
||||||
|
FBSchemaGenerator = sa_base.FBSchemaGenerator
|
||||||
|
|
||||||
|
class FBColumnGenerator(FBSchemaGenerator, ansisql.ANSIColumnGenerator):
|
||||||
|
"""Firebird column generator implementation."""
|
||||||
|
|
||||||
|
|
||||||
|
class FBColumnDropper(ansisql.ANSIColumnDropper):
|
||||||
|
"""Firebird column dropper implementation."""
|
||||||
|
|
||||||
|
def visit_column(self, column):
|
||||||
|
table = self.start_alter_table(column)
|
||||||
|
self.append('DROP %s' % self.preparer.format_column(column))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
|
||||||
|
class FBSchemaChanger(ansisql.ANSISchemaChanger):
|
||||||
|
"""Firebird schema changer implementation."""
|
||||||
|
|
||||||
|
def visit_table(self, table):
|
||||||
|
"""Rename table not supported"""
|
||||||
|
raise exceptions.NotSupportedError(
|
||||||
|
"Firebird does not support renaming tables.")
|
||||||
|
|
||||||
|
def _visit_column_name(self, table, column, delta):
|
||||||
|
self.start_alter_table(table)
|
||||||
|
col_name = self.preparer.quote(delta.current_name, table.quote)
|
||||||
|
new_name = self.preparer.format_column(delta.result_column)
|
||||||
|
self.append('ALTER COLUMN %s TO %s' % (col_name, new_name))
|
||||||
|
|
||||||
|
def _visit_column_nullable(self, table, column, delta):
|
||||||
|
"""Changing NULL is not supported"""
|
||||||
|
# TODO: http://www.firebirdfaq.org/faq103/
|
||||||
|
raise exceptions.NotSupportedError(
|
||||||
|
"Firebird does not support altering NULL bevahior.")
|
||||||
|
|
||||||
|
|
||||||
|
class FBConstraintGenerator(ansisql.ANSIConstraintGenerator):
|
||||||
|
"""Firebird constraint generator implementation."""
|
||||||
|
|
||||||
|
|
||||||
|
class FBConstraintDropper(ansisql.ANSIConstraintDropper):
|
||||||
|
"""Firebird constaint dropper implementation."""
|
||||||
|
|
||||||
|
def cascade_constraint(self, constraint):
|
||||||
|
"""Cascading constraints is not supported"""
|
||||||
|
raise exceptions.NotSupportedError(
|
||||||
|
"Firebird does not support cascading constraints")
|
||||||
|
|
||||||
|
|
||||||
|
class FBDialect(ansisql.ANSIDialect):
|
||||||
|
columngenerator = FBColumnGenerator
|
||||||
|
columndropper = FBColumnDropper
|
||||||
|
schemachanger = FBSchemaChanger
|
||||||
|
constraintgenerator = FBConstraintGenerator
|
||||||
|
constraintdropper = FBConstraintDropper
|
||||||
80
migrate/changeset/databases/mysql.py
Normal file
80
migrate/changeset/databases/mysql.py
Normal file
@ -0,0 +1,80 @@
|
|||||||
|
"""
|
||||||
|
MySQL database specific implementations of changeset classes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from migrate.changeset import ansisql, exceptions, SQLA_06
|
||||||
|
from sqlalchemy.databases import mysql as sa_base
|
||||||
|
|
||||||
|
if not SQLA_06:
|
||||||
|
MySQLSchemaGenerator = sa_base.MySQLSchemaGenerator
|
||||||
|
else:
|
||||||
|
MySQLSchemaGenerator = sa_base.MySQLDDLCompiler
|
||||||
|
|
||||||
|
class MySQLColumnGenerator(MySQLSchemaGenerator, ansisql.ANSIColumnGenerator):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class MySQLColumnDropper(ansisql.ANSIColumnDropper):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class MySQLSchemaChanger(MySQLSchemaGenerator, ansisql.ANSISchemaChanger):
|
||||||
|
|
||||||
|
def visit_column(self, delta):
|
||||||
|
table = delta.table
|
||||||
|
colspec = self.get_column_specification(delta.result_column)
|
||||||
|
old_col_name = self.preparer.quote(delta.current_name, table.quote)
|
||||||
|
|
||||||
|
self.start_alter_table(table)
|
||||||
|
|
||||||
|
self.append("CHANGE COLUMN %s " % old_col_name)
|
||||||
|
self.append(colspec)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_index(self, param):
|
||||||
|
# If MySQL can do this, I can't find how
|
||||||
|
raise exceptions.NotSupportedError("MySQL cannot rename indexes")
|
||||||
|
|
||||||
|
|
||||||
|
class MySQLConstraintGenerator(ansisql.ANSIConstraintGenerator):
|
||||||
|
pass
|
||||||
|
|
||||||
|
if SQLA_06:
|
||||||
|
class MySQLConstraintDropper(MySQLSchemaGenerator, ansisql.ANSIConstraintDropper):
|
||||||
|
def visit_migrate_check_constraint(self, *p, **k):
|
||||||
|
raise exceptions.NotSupportedError("MySQL does not support CHECK"
|
||||||
|
" constraints, use triggers instead.")
|
||||||
|
|
||||||
|
else:
|
||||||
|
class MySQLConstraintDropper(ansisql.ANSIConstraintDropper):
|
||||||
|
|
||||||
|
def visit_migrate_primary_key_constraint(self, constraint):
|
||||||
|
self.start_alter_table(constraint)
|
||||||
|
self.append("DROP PRIMARY KEY")
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_migrate_foreign_key_constraint(self, constraint):
|
||||||
|
self.start_alter_table(constraint)
|
||||||
|
self.append("DROP FOREIGN KEY ")
|
||||||
|
constraint.name = self.get_constraint_name(constraint)
|
||||||
|
self.append(self.preparer.format_constraint(constraint))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_migrate_check_constraint(self, *p, **k):
|
||||||
|
raise exceptions.NotSupportedError("MySQL does not support CHECK"
|
||||||
|
" constraints, use triggers instead.")
|
||||||
|
|
||||||
|
def visit_migrate_unique_constraint(self, constraint, *p, **k):
|
||||||
|
self.start_alter_table(constraint)
|
||||||
|
self.append('DROP INDEX ')
|
||||||
|
constraint.name = self.get_constraint_name(constraint)
|
||||||
|
self.append(self.preparer.format_constraint(constraint))
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
|
||||||
|
class MySQLDialect(ansisql.ANSIDialect):
|
||||||
|
columngenerator = MySQLColumnGenerator
|
||||||
|
columndropper = MySQLColumnDropper
|
||||||
|
schemachanger = MySQLSchemaChanger
|
||||||
|
constraintgenerator = MySQLConstraintGenerator
|
||||||
|
constraintdropper = MySQLConstraintDropper
|
||||||
112
migrate/changeset/databases/oracle.py
Normal file
112
migrate/changeset/databases/oracle.py
Normal file
@ -0,0 +1,112 @@
|
|||||||
|
"""
|
||||||
|
Oracle database specific implementations of changeset classes.
|
||||||
|
"""
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
from migrate.changeset import ansisql, exceptions
|
||||||
|
|
||||||
|
from sqlalchemy.databases import oracle as sa_base
|
||||||
|
|
||||||
|
from migrate.changeset import ansisql, exceptions, SQLA_06
|
||||||
|
|
||||||
|
if not SQLA_06:
|
||||||
|
OracleSchemaGenerator = sa_base.OracleSchemaGenerator
|
||||||
|
else:
|
||||||
|
OracleSchemaGenerator = sa_base.OracleDDLCompiler
|
||||||
|
|
||||||
|
|
||||||
|
class OracleColumnGenerator(OracleSchemaGenerator, ansisql.ANSIColumnGenerator):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class OracleColumnDropper(ansisql.ANSIColumnDropper):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class OracleSchemaChanger(OracleSchemaGenerator, ansisql.ANSISchemaChanger):
|
||||||
|
|
||||||
|
def get_column_specification(self, column, **kwargs):
|
||||||
|
# Ignore the NOT NULL generated
|
||||||
|
override_nullable = kwargs.pop('override_nullable', None)
|
||||||
|
if override_nullable:
|
||||||
|
orig = column.nullable
|
||||||
|
column.nullable = True
|
||||||
|
ret = super(OracleSchemaChanger, self).get_column_specification(
|
||||||
|
column, **kwargs)
|
||||||
|
if override_nullable:
|
||||||
|
column.nullable = orig
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def visit_column(self, delta):
|
||||||
|
keys = delta.keys()
|
||||||
|
|
||||||
|
if 'name' in keys:
|
||||||
|
self._run_subvisit(delta,
|
||||||
|
self._visit_column_name,
|
||||||
|
start_alter=False)
|
||||||
|
|
||||||
|
if len(set(('type', 'nullable', 'server_default')).intersection(keys)):
|
||||||
|
self._run_subvisit(delta,
|
||||||
|
self._visit_column_change,
|
||||||
|
start_alter=False)
|
||||||
|
|
||||||
|
def _visit_column_change(self, table, column, delta):
|
||||||
|
# Oracle cannot drop a default once created, but it can set it
|
||||||
|
# to null. We'll do that if default=None
|
||||||
|
# http://forums.oracle.com/forums/message.jspa?messageID=1273234#1273234
|
||||||
|
dropdefault_hack = (column.server_default is None \
|
||||||
|
and 'server_default' in delta.keys())
|
||||||
|
# Oracle apparently doesn't like it when we say "not null" if
|
||||||
|
# the column's already not null. Fudge it, so we don't need a
|
||||||
|
# new function
|
||||||
|
notnull_hack = ((not column.nullable) \
|
||||||
|
and ('nullable' not in delta.keys()))
|
||||||
|
# We need to specify NULL if we're removing a NOT NULL
|
||||||
|
# constraint
|
||||||
|
null_hack = (column.nullable and ('nullable' in delta.keys()))
|
||||||
|
|
||||||
|
if dropdefault_hack:
|
||||||
|
column.server_default = sa.PassiveDefault(sa.sql.null())
|
||||||
|
if notnull_hack:
|
||||||
|
column.nullable = True
|
||||||
|
colspec = self.get_column_specification(column,
|
||||||
|
override_nullable=null_hack)
|
||||||
|
if null_hack:
|
||||||
|
colspec += ' NULL'
|
||||||
|
if notnull_hack:
|
||||||
|
column.nullable = False
|
||||||
|
if dropdefault_hack:
|
||||||
|
column.server_default = None
|
||||||
|
|
||||||
|
self.start_alter_table(table)
|
||||||
|
self.append("MODIFY (")
|
||||||
|
self.append(colspec)
|
||||||
|
self.append(")")
|
||||||
|
|
||||||
|
|
||||||
|
class OracleConstraintCommon(object):
|
||||||
|
|
||||||
|
def get_constraint_name(self, cons):
|
||||||
|
# Oracle constraints can't guess their name like other DBs
|
||||||
|
if not cons.name:
|
||||||
|
raise exceptions.NotSupportedError(
|
||||||
|
"Oracle constraint names must be explicitly stated")
|
||||||
|
return cons.name
|
||||||
|
|
||||||
|
|
||||||
|
class OracleConstraintGenerator(OracleConstraintCommon,
|
||||||
|
ansisql.ANSIConstraintGenerator):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class OracleConstraintDropper(OracleConstraintCommon,
|
||||||
|
ansisql.ANSIConstraintDropper):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class OracleDialect(ansisql.ANSIDialect):
|
||||||
|
columngenerator = OracleColumnGenerator
|
||||||
|
columndropper = OracleColumnDropper
|
||||||
|
schemachanger = OracleSchemaChanger
|
||||||
|
constraintgenerator = OracleConstraintGenerator
|
||||||
|
constraintdropper = OracleConstraintDropper
|
||||||
45
migrate/changeset/databases/postgres.py
Normal file
45
migrate/changeset/databases/postgres.py
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
"""
|
||||||
|
`PostgreSQL`_ database specific implementations of changeset classes.
|
||||||
|
|
||||||
|
.. _`PostgreSQL`: http://www.postgresql.org/
|
||||||
|
"""
|
||||||
|
from migrate.changeset import ansisql, SQLA_06
|
||||||
|
from sqlalchemy.databases import postgres as sa_base
|
||||||
|
|
||||||
|
if not SQLA_06:
|
||||||
|
PGSchemaGenerator = sa_base.PGSchemaGenerator
|
||||||
|
else:
|
||||||
|
PGSchemaGenerator = sa_base.PGDDLCompiler
|
||||||
|
|
||||||
|
|
||||||
|
class PGColumnGenerator(PGSchemaGenerator, ansisql.ANSIColumnGenerator):
|
||||||
|
"""PostgreSQL column generator implementation."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class PGColumnDropper(ansisql.ANSIColumnDropper):
|
||||||
|
"""PostgreSQL column dropper implementation."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class PGSchemaChanger(ansisql.ANSISchemaChanger):
|
||||||
|
"""PostgreSQL schema changer implementation."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class PGConstraintGenerator(ansisql.ANSIConstraintGenerator):
|
||||||
|
"""PostgreSQL constraint generator implementation."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class PGConstraintDropper(ansisql.ANSIConstraintDropper):
|
||||||
|
"""PostgreSQL constaint dropper implementation."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class PGDialect(ansisql.ANSIDialect):
|
||||||
|
columngenerator = PGColumnGenerator
|
||||||
|
columndropper = PGColumnDropper
|
||||||
|
schemachanger = PGSchemaChanger
|
||||||
|
constraintgenerator = PGConstraintGenerator
|
||||||
|
constraintdropper = PGConstraintDropper
|
||||||
132
migrate/changeset/databases/sqlite.py
Normal file
132
migrate/changeset/databases/sqlite.py
Normal file
@ -0,0 +1,132 @@
|
|||||||
|
"""
|
||||||
|
`SQLite`_ database specific implementations of changeset classes.
|
||||||
|
|
||||||
|
.. _`SQLite`: http://www.sqlite.org/
|
||||||
|
"""
|
||||||
|
from UserDict import DictMixin
|
||||||
|
from copy import copy
|
||||||
|
|
||||||
|
from sqlalchemy.databases import sqlite as sa_base
|
||||||
|
|
||||||
|
from migrate.changeset import ansisql, exceptions, SQLA_06
|
||||||
|
|
||||||
|
if not SQLA_06:
|
||||||
|
SQLiteSchemaGenerator = sa_base.SQLiteSchemaGenerator
|
||||||
|
else:
|
||||||
|
SQLiteSchemaGenerator = sa_base.SQLiteDDLCompiler
|
||||||
|
|
||||||
|
class SQLiteCommon(object):
|
||||||
|
|
||||||
|
def _not_supported(self, op):
|
||||||
|
raise exceptions.NotSupportedError("SQLite does not support "
|
||||||
|
"%s; see http://www.sqlite.org/lang_altertable.html" % op)
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteHelper(SQLiteCommon):
|
||||||
|
|
||||||
|
def visit_column(self, delta):
|
||||||
|
if isinstance(delta, DictMixin):
|
||||||
|
column = delta.result_column
|
||||||
|
table = self._to_table(delta.table)
|
||||||
|
else:
|
||||||
|
column = delta
|
||||||
|
table = self._to_table(column.table)
|
||||||
|
table_name = self.preparer.format_table(table)
|
||||||
|
|
||||||
|
# we remove all constraints, indexes so it doesnt recreate them
|
||||||
|
ixbackup = copy(table.indexes)
|
||||||
|
consbackup = copy(table.constraints)
|
||||||
|
table.indexes = set()
|
||||||
|
table.constraints = set()
|
||||||
|
|
||||||
|
self.append('ALTER TABLE %s RENAME TO migration_tmp' % table_name)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
insertion_string = self._modify_table(table, column, delta)
|
||||||
|
|
||||||
|
table.create()
|
||||||
|
self.append(insertion_string % {'table_name': table_name})
|
||||||
|
self.execute()
|
||||||
|
self.append('DROP TABLE migration_tmp')
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
# restore indexes, constraints
|
||||||
|
table.indexes = ixbackup
|
||||||
|
table.constraints = consbackup
|
||||||
|
|
||||||
|
class SQLiteColumnGenerator(SQLiteSchemaGenerator, SQLiteCommon,
|
||||||
|
ansisql.ANSIColumnGenerator):
|
||||||
|
"""SQLite ColumnGenerator"""
|
||||||
|
|
||||||
|
def add_foreignkey(self, constraint):
|
||||||
|
"""Does not support ALTER TABLE ADD FOREIGN KEY"""
|
||||||
|
self._not_supported("ALTER TABLE ADD CONSTRAINT")
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteColumnDropper(SQLiteHelper, ansisql.ANSIColumnDropper):
|
||||||
|
"""SQLite ColumnDropper"""
|
||||||
|
|
||||||
|
def _modify_table(self, table, column, delta):
|
||||||
|
columns = ' ,'.join(map(self.preparer.format_column, table.columns))
|
||||||
|
return 'INSERT INTO %(table_name)s SELECT ' + columns + \
|
||||||
|
' from migration_tmp'
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteSchemaChanger(SQLiteHelper, ansisql.ANSISchemaChanger):
|
||||||
|
"""SQLite SchemaChanger"""
|
||||||
|
|
||||||
|
def _modify_table(self, table, column, delta):
|
||||||
|
column = table.columns[delta.current_name]
|
||||||
|
return 'INSERT INTO %(table_name)s SELECT * from migration_tmp'
|
||||||
|
|
||||||
|
def visit_index(self, index):
|
||||||
|
"""Does not support ALTER INDEX"""
|
||||||
|
self._not_supported('ALTER INDEX')
|
||||||
|
|
||||||
|
|
||||||
|
class SQLiteConstraintGenerator(ansisql.ANSIConstraintGenerator, SQLiteCommon):
|
||||||
|
|
||||||
|
def visit_migrate_primary_key_constraint(self, constraint):
|
||||||
|
tmpl = "CREATE UNIQUE INDEX %s ON %s ( %s )"
|
||||||
|
cols = ', '.join(map(self.preparer.format_column, constraint.columns))
|
||||||
|
tname = self.preparer.format_table(constraint.table)
|
||||||
|
name = self.get_constraint_name(constraint)
|
||||||
|
msg = tmpl % (name, tname, cols)
|
||||||
|
self.append(msg)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_migrate_foreign_key_constraint(self, *p, **k):
|
||||||
|
self._not_supported('ALTER TABLE ADD CONSTRAINT')
|
||||||
|
|
||||||
|
def visit_migrate_unique_constraint(self, *p, **k):
|
||||||
|
self._not_supported('ALTER TABLE ADD CONSTRAINT')
|
||||||
|
|
||||||
|
class SQLiteConstraintDropper(ansisql.ANSIColumnDropper,
|
||||||
|
SQLiteCommon,
|
||||||
|
ansisql.ANSIConstraintCommon):
|
||||||
|
|
||||||
|
def visit_migrate_primary_key_constraint(self, constraint):
|
||||||
|
tmpl = "DROP INDEX %s "
|
||||||
|
name = self.get_constraint_name(constraint)
|
||||||
|
msg = tmpl % (name)
|
||||||
|
self.append(msg)
|
||||||
|
self.execute()
|
||||||
|
|
||||||
|
def visit_migrate_foreign_key_constraint(self, *p, **k):
|
||||||
|
self._not_supported('ALTER TABLE DROP CONSTRAINT')
|
||||||
|
|
||||||
|
def visit_migrate_check_constraint(self, *p, **k):
|
||||||
|
self._not_supported('ALTER TABLE DROP CONSTRAINT')
|
||||||
|
|
||||||
|
def visit_migrate_unique_constraint(self, *p, **k):
|
||||||
|
self._not_supported('ALTER TABLE DROP CONSTRAINT')
|
||||||
|
|
||||||
|
|
||||||
|
# TODO: technically primary key is a NOT NULL + UNIQUE constraint, should add NOT NULL to index
|
||||||
|
|
||||||
|
class SQLiteDialect(ansisql.ANSIDialect):
|
||||||
|
columngenerator = SQLiteColumnGenerator
|
||||||
|
columndropper = SQLiteColumnDropper
|
||||||
|
schemachanger = SQLiteSchemaChanger
|
||||||
|
constraintgenerator = SQLiteConstraintGenerator
|
||||||
|
constraintdropper = SQLiteConstraintDropper
|
||||||
70
migrate/changeset/databases/visitor.py
Normal file
70
migrate/changeset/databases/visitor.py
Normal file
@ -0,0 +1,70 @@
|
|||||||
|
"""
|
||||||
|
Module for visitor class mapping.
|
||||||
|
"""
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
from migrate.changeset import ansisql
|
||||||
|
from migrate.changeset.databases import (sqlite,
|
||||||
|
postgres,
|
||||||
|
mysql,
|
||||||
|
oracle,
|
||||||
|
firebird)
|
||||||
|
|
||||||
|
|
||||||
|
# Map SA dialects to the corresponding Migrate extensions
|
||||||
|
DIALECTS = {
|
||||||
|
"default": ansisql.ANSIDialect,
|
||||||
|
"sqlite": sqlite.SQLiteDialect,
|
||||||
|
"postgres": postgres.PGDialect,
|
||||||
|
"mysql": mysql.MySQLDialect,
|
||||||
|
"oracle": oracle.OracleDialect,
|
||||||
|
"firebird": firebird.FBDialect,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_engine_visitor(engine, name):
|
||||||
|
"""
|
||||||
|
Get the visitor implementation for the given database engine.
|
||||||
|
|
||||||
|
:param engine: SQLAlchemy Engine
|
||||||
|
:param name: Name of the visitor
|
||||||
|
:type name: string
|
||||||
|
:type engine: Engine
|
||||||
|
:returns: visitor
|
||||||
|
"""
|
||||||
|
# TODO: link to supported visitors
|
||||||
|
return get_dialect_visitor(engine.dialect, name)
|
||||||
|
|
||||||
|
|
||||||
|
def get_dialect_visitor(sa_dialect, name):
|
||||||
|
"""
|
||||||
|
Get the visitor implementation for the given dialect.
|
||||||
|
|
||||||
|
Finds the visitor implementation based on the dialect class and
|
||||||
|
returns and instance initialized with the given name.
|
||||||
|
|
||||||
|
Binds dialect specific preparer to visitor.
|
||||||
|
"""
|
||||||
|
|
||||||
|
# map sa dialect to migrate dialect and return visitor
|
||||||
|
sa_dialect_name = getattr(sa_dialect, 'name', 'default')
|
||||||
|
migrate_dialect_cls = DIALECTS[sa_dialect_name]
|
||||||
|
visitor = getattr(migrate_dialect_cls, name)
|
||||||
|
|
||||||
|
# bind preparer
|
||||||
|
visitor.preparer = sa_dialect.preparer(sa_dialect)
|
||||||
|
|
||||||
|
return visitor
|
||||||
|
|
||||||
|
def run_single_visitor(engine, visitorcallable, element, **kwargs):
|
||||||
|
"""Runs only one method on the visitor"""
|
||||||
|
conn = engine.contextual_connect(close_with_result=False)
|
||||||
|
try:
|
||||||
|
visitor = visitorcallable(engine.dialect, conn)
|
||||||
|
if hasattr(element, '__migrate_visit_name__'):
|
||||||
|
fn = getattr(visitor, 'visit_' + element.__migrate_visit_name__)
|
||||||
|
else:
|
||||||
|
fn = getattr(visitor, 'visit_' + element.__visit_name__)
|
||||||
|
fn(element, **kwargs)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
21
migrate/changeset/exceptions.py
Normal file
21
migrate/changeset/exceptions.py
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
"""
|
||||||
|
This module provides exception classes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class Error(Exception):
|
||||||
|
"""
|
||||||
|
Changeset error.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class NotSupportedError(Error):
|
||||||
|
"""
|
||||||
|
Not supported error.
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidConstraintError(Error):
|
||||||
|
"""
|
||||||
|
Invalid constraint error.
|
||||||
|
"""
|
||||||
597
migrate/changeset/schema.py
Normal file
597
migrate/changeset/schema.py
Normal file
@ -0,0 +1,597 @@
|
|||||||
|
"""
|
||||||
|
Schema module providing common schema operations.
|
||||||
|
"""
|
||||||
|
from UserDict import DictMixin
|
||||||
|
import sqlalchemy
|
||||||
|
|
||||||
|
from migrate.changeset import SQLA_06
|
||||||
|
from migrate.changeset.exceptions import *
|
||||||
|
from migrate.changeset.databases.visitor import (get_engine_visitor,
|
||||||
|
run_single_visitor)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
'create_column',
|
||||||
|
'drop_column',
|
||||||
|
'alter_column',
|
||||||
|
'rename_table',
|
||||||
|
'rename_index',
|
||||||
|
'ChangesetTable',
|
||||||
|
'ChangesetColumn',
|
||||||
|
'ChangesetIndex',
|
||||||
|
'ChangesetDefaultClause',
|
||||||
|
'ColumnDelta',
|
||||||
|
]
|
||||||
|
|
||||||
|
DEFAULT_ALTER_METADATA = True
|
||||||
|
|
||||||
|
|
||||||
|
def create_column(column, table=None, *p, **kw):
|
||||||
|
"""Create a column, given the table.
|
||||||
|
|
||||||
|
API to :meth:`ChangesetColumn.create`.
|
||||||
|
"""
|
||||||
|
if table is not None:
|
||||||
|
return table.create_column(column, *p, **kw)
|
||||||
|
return column.create(*p, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def drop_column(column, table=None, *p, **kw):
|
||||||
|
"""Drop a column, given the table.
|
||||||
|
|
||||||
|
API to :meth:`ChangesetColumn.drop`.
|
||||||
|
"""
|
||||||
|
if table is not None:
|
||||||
|
return table.drop_column(column, *p, **kw)
|
||||||
|
return column.drop(*p, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def rename_table(table, name, engine=None, **kw):
|
||||||
|
"""Rename a table.
|
||||||
|
|
||||||
|
If Table instance is given, engine is not used.
|
||||||
|
|
||||||
|
API to :meth:`ChangesetTable.rename`.
|
||||||
|
|
||||||
|
:param table: Table to be renamed.
|
||||||
|
:param name: New name for Table.
|
||||||
|
:param engine: Engine instance.
|
||||||
|
:type table: string or Table instance
|
||||||
|
:type name: string
|
||||||
|
:type engine: obj
|
||||||
|
"""
|
||||||
|
table = _to_table(table, engine)
|
||||||
|
table.rename(name, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def rename_index(index, name, table=None, engine=None, **kw):
|
||||||
|
"""Rename an index.
|
||||||
|
|
||||||
|
If Index instance is given,
|
||||||
|
table and engine are not used.
|
||||||
|
|
||||||
|
API to :meth:`ChangesetIndex.rename`.
|
||||||
|
|
||||||
|
:param index: Index to be renamed.
|
||||||
|
:param name: New name for index.
|
||||||
|
:param table: Table to which Index is reffered.
|
||||||
|
:param engine: Engine instance.
|
||||||
|
:type index: string or Index instance
|
||||||
|
:type name: string
|
||||||
|
:type table: string or Table instance
|
||||||
|
:type engine: obj
|
||||||
|
"""
|
||||||
|
index = _to_index(index, table, engine)
|
||||||
|
index.rename(name, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def alter_column(*p, **k):
|
||||||
|
"""Alter a column.
|
||||||
|
|
||||||
|
Direct API to :class:`ColumnDelta`.
|
||||||
|
|
||||||
|
:param table: Table or table name (will issue reflection).
|
||||||
|
:param engine: Will be used for reflection.
|
||||||
|
:param alter_metadata: Defaults to True. It will alter changes also to objects.
|
||||||
|
:returns: :class:`Columndelta` instance
|
||||||
|
"""
|
||||||
|
|
||||||
|
k.setdefault('alter_metadata', DEFAULT_ALTER_METADATA)
|
||||||
|
|
||||||
|
if 'table' not in k and isinstance(p[0], sqlalchemy.Column):
|
||||||
|
k['table'] = p[0].table
|
||||||
|
if 'engine' not in k:
|
||||||
|
k['engine'] = k['table'].bind
|
||||||
|
|
||||||
|
engine = k['engine']
|
||||||
|
delta = ColumnDelta(*p, **k)
|
||||||
|
|
||||||
|
visitorcallable = get_engine_visitor(engine, 'schemachanger')
|
||||||
|
engine._run_visitor(visitorcallable, delta)
|
||||||
|
|
||||||
|
return delta
|
||||||
|
|
||||||
|
|
||||||
|
def _to_table(table, engine=None):
|
||||||
|
"""Return if instance of Table, else construct new with metadata"""
|
||||||
|
if isinstance(table, sqlalchemy.Table):
|
||||||
|
return table
|
||||||
|
|
||||||
|
# Given: table name, maybe an engine
|
||||||
|
meta = sqlalchemy.MetaData()
|
||||||
|
if engine is not None:
|
||||||
|
meta.bind = engine
|
||||||
|
return sqlalchemy.Table(table, meta)
|
||||||
|
|
||||||
|
|
||||||
|
def _to_index(index, table=None, engine=None):
|
||||||
|
"""Return if instance of Index, else construct new with metadata"""
|
||||||
|
if isinstance(index, sqlalchemy.Index):
|
||||||
|
return index
|
||||||
|
|
||||||
|
# Given: index name; table name required
|
||||||
|
table = _to_table(table, engine)
|
||||||
|
ret = sqlalchemy.Index(index)
|
||||||
|
ret.table = table
|
||||||
|
return ret
|
||||||
|
|
||||||
|
|
||||||
|
class ColumnDelta(DictMixin, sqlalchemy.schema.SchemaItem):
|
||||||
|
"""Extracts the differences between two columns/column-parameters
|
||||||
|
|
||||||
|
May receive parameters arranged in several different ways:
|
||||||
|
|
||||||
|
* **current_column, new_column, \*p, \*\*kw**
|
||||||
|
Additional parameters can be specified to override column
|
||||||
|
differences.
|
||||||
|
|
||||||
|
* **current_column, \*p, \*\*kw**
|
||||||
|
Additional parameters alter current_column. Table name is extracted
|
||||||
|
from current_column object.
|
||||||
|
Name is changed to current_column.name from current_name,
|
||||||
|
if current_name is specified.
|
||||||
|
|
||||||
|
* **current_col_name, \*p, \*\*kw**
|
||||||
|
Table kw must specified.
|
||||||
|
|
||||||
|
:param table: Table at which current Column should be bound to.\
|
||||||
|
If table name is given, reflection will be used.
|
||||||
|
:type table: string or Table instance
|
||||||
|
:param alter_metadata: If True, it will apply changes to metadata.
|
||||||
|
:type alter_metadata: bool
|
||||||
|
:param metadata: If `alter_metadata` is true, \
|
||||||
|
metadata is used to reflect table names into
|
||||||
|
:type metadata: :class:`MetaData` instance
|
||||||
|
:param engine: When reflecting tables, either engine or metadata must \
|
||||||
|
be specified to acquire engine object.
|
||||||
|
:type engine: :class:`Engine` instance
|
||||||
|
:returns: :class:`ColumnDelta` instance provides interface for altered attributes to \
|
||||||
|
`result_column` through :func:`dict` alike object.
|
||||||
|
|
||||||
|
* :class:`ColumnDelta`.result_column is altered column with new attributes
|
||||||
|
|
||||||
|
* :class:`ColumnDelta`.current_name is current name of column in db
|
||||||
|
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
# Column attributes that can be altered
|
||||||
|
diff_keys = ('name', 'type', 'primary_key', 'nullable',
|
||||||
|
'server_onupdate', 'server_default')
|
||||||
|
diffs = dict()
|
||||||
|
__visit_name__ = 'column'
|
||||||
|
|
||||||
|
def __init__(self, *p, **kw):
|
||||||
|
self.alter_metadata = kw.pop("alter_metadata", False)
|
||||||
|
self.meta = kw.pop("metadata", None)
|
||||||
|
self.engine = kw.pop("engine", None)
|
||||||
|
|
||||||
|
# Things are initialized differently depending on how many column
|
||||||
|
# parameters are given. Figure out how many and call the appropriate
|
||||||
|
# method.
|
||||||
|
if len(p) >= 1 and isinstance(p[0], sqlalchemy.Column):
|
||||||
|
# At least one column specified
|
||||||
|
if len(p) >= 2 and isinstance(p[1], sqlalchemy.Column):
|
||||||
|
# Two columns specified
|
||||||
|
diffs = self.compare_2_columns(*p, **kw)
|
||||||
|
else:
|
||||||
|
# Exactly one column specified
|
||||||
|
diffs = self.compare_1_column(*p, **kw)
|
||||||
|
else:
|
||||||
|
# Zero columns specified
|
||||||
|
if not len(p) or not isinstance(p[0], basestring):
|
||||||
|
raise ValueError("First argument must be column name")
|
||||||
|
diffs = self.compare_parameters(*p, **kw)
|
||||||
|
|
||||||
|
self.apply_diffs(diffs)
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return '<ColumnDelta altermetadata=%r, %s>' % (self.alter_metadata,
|
||||||
|
super(ColumnDelta, self).__repr__())
|
||||||
|
|
||||||
|
def __getitem__(self, key):
|
||||||
|
if key not in self.keys():
|
||||||
|
raise KeyError("No such diff key, available: %s" % self.diffs )
|
||||||
|
return getattr(self.result_column, key)
|
||||||
|
|
||||||
|
def __setitem__(self, key, value):
|
||||||
|
if key not in self.keys():
|
||||||
|
raise KeyError("No such diff key, available: %s" % self.diffs )
|
||||||
|
setattr(self.result_column, key, value)
|
||||||
|
|
||||||
|
def __delitem__(self, key):
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def keys(self):
|
||||||
|
return self.diffs.keys()
|
||||||
|
|
||||||
|
def compare_parameters(self, current_name, *p, **k):
|
||||||
|
"""Compares Column objects with reflection"""
|
||||||
|
self.table = k.pop('table')
|
||||||
|
self.result_column = self._table.c.get(current_name)
|
||||||
|
if len(p):
|
||||||
|
k = self._extract_parameters(p, k, self.result_column)
|
||||||
|
return k
|
||||||
|
|
||||||
|
def compare_1_column(self, col, *p, **k):
|
||||||
|
"""Compares one Column object"""
|
||||||
|
self.table = k.pop('table', None) or col.table
|
||||||
|
self.result_column = col
|
||||||
|
if len(p):
|
||||||
|
k = self._extract_parameters(p, k, self.result_column)
|
||||||
|
return k
|
||||||
|
|
||||||
|
def compare_2_columns(self, old_col, new_col, *p, **k):
|
||||||
|
"""Compares two Column objects"""
|
||||||
|
self.process_column(new_col)
|
||||||
|
self.table = k.pop('table', None) or old_col.table or new_col.table
|
||||||
|
self.result_column = old_col
|
||||||
|
|
||||||
|
# set differences
|
||||||
|
# leave out some stuff for later comp
|
||||||
|
for key in (set(self.diff_keys) - set(('type',))):
|
||||||
|
val = getattr(new_col, key, None)
|
||||||
|
if getattr(self.result_column, key, None) != val:
|
||||||
|
k.setdefault(key, val)
|
||||||
|
|
||||||
|
# inspect types
|
||||||
|
if not self.are_column_types_eq(self.result_column.type, new_col.type):
|
||||||
|
k.setdefault('type', new_col.type)
|
||||||
|
|
||||||
|
if len(p):
|
||||||
|
k = self._extract_parameters(p, k, self.result_column)
|
||||||
|
return k
|
||||||
|
|
||||||
|
def apply_diffs(self, diffs):
|
||||||
|
"""Populate dict and column object with new values"""
|
||||||
|
self.diffs = diffs
|
||||||
|
for key in self.diff_keys:
|
||||||
|
if key in diffs:
|
||||||
|
setattr(self.result_column, key, diffs[key])
|
||||||
|
|
||||||
|
self.process_column(self.result_column)
|
||||||
|
|
||||||
|
# create an instance of class type if not yet
|
||||||
|
if 'type' in diffs and callable(self.result_column.type):
|
||||||
|
self.result_column.type = self.result_column.type()
|
||||||
|
|
||||||
|
# add column to the table
|
||||||
|
if self.table and self.alter_metadata:
|
||||||
|
self.result_column.add_to_table(self.table)
|
||||||
|
|
||||||
|
def are_column_types_eq(self, old_type, new_type):
|
||||||
|
"""Compares two types to be equal"""
|
||||||
|
ret = old_type.__class__ == new_type.__class__
|
||||||
|
|
||||||
|
# String length is a special case
|
||||||
|
if ret and isinstance(new_type, sqlalchemy.types.String):
|
||||||
|
ret = (getattr(old_type, 'length', None) == \
|
||||||
|
getattr(new_type, 'length', None))
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def _extract_parameters(self, p, k, column):
|
||||||
|
"""Extracts data from p and modifies diffs"""
|
||||||
|
p = list(p)
|
||||||
|
while len(p):
|
||||||
|
if isinstance(p[0], basestring):
|
||||||
|
k.setdefault('name', p.pop(0))
|
||||||
|
elif isinstance(p[0], sqlalchemy.types.AbstractType):
|
||||||
|
k.setdefault('type', p.pop(0))
|
||||||
|
elif callable(p[0]):
|
||||||
|
p[0] = p[0]()
|
||||||
|
else:
|
||||||
|
break
|
||||||
|
|
||||||
|
if len(p):
|
||||||
|
new_col = column.copy_fixed()
|
||||||
|
new_col._init_items(*p)
|
||||||
|
k = self.compare_2_columns(column, new_col, **k)
|
||||||
|
return k
|
||||||
|
|
||||||
|
def process_column(self, column):
|
||||||
|
"""Processes default values for column"""
|
||||||
|
# XXX: this is a snippet from SA processing of positional parameters
|
||||||
|
if not SQLA_06 and column.args:
|
||||||
|
toinit = list(column.args)
|
||||||
|
else:
|
||||||
|
toinit = list()
|
||||||
|
|
||||||
|
if column.server_default is not None:
|
||||||
|
if isinstance(column.server_default, sqlalchemy.FetchedValue):
|
||||||
|
toinit.append(column.server_default)
|
||||||
|
else:
|
||||||
|
toinit.append(sqlalchemy.DefaultClause(column.server_default))
|
||||||
|
if column.server_onupdate is not None:
|
||||||
|
if isinstance(column.server_onupdate, FetchedValue):
|
||||||
|
toinit.append(column.server_default)
|
||||||
|
else:
|
||||||
|
toinit.append(sqlalchemy.DefaultClause(column.server_onupdate,
|
||||||
|
for_update=True))
|
||||||
|
if toinit:
|
||||||
|
column._init_items(*toinit)
|
||||||
|
|
||||||
|
if not SQLA_06:
|
||||||
|
column.args = []
|
||||||
|
|
||||||
|
def _get_table(self):
|
||||||
|
return getattr(self, '_table', None)
|
||||||
|
|
||||||
|
def _set_table(self, table):
|
||||||
|
if isinstance(table, basestring):
|
||||||
|
if self.alter_metadata:
|
||||||
|
if not self.meta:
|
||||||
|
raise ValueError("metadata must be specified for table"
|
||||||
|
" reflection when using alter_metadata")
|
||||||
|
meta = self.meta
|
||||||
|
if self.engine:
|
||||||
|
meta.bind = self.engine
|
||||||
|
else:
|
||||||
|
if not self.engine and not self.meta:
|
||||||
|
raise ValueError("engine or metadata must be specified"
|
||||||
|
" to reflect tables")
|
||||||
|
if not self.engine:
|
||||||
|
self.engine = self.meta.bind
|
||||||
|
meta = sqlalchemy.MetaData(bind=self.engine)
|
||||||
|
self._table = sqlalchemy.Table(table, meta, autoload=True)
|
||||||
|
elif isinstance(table, sqlalchemy.Table):
|
||||||
|
self._table = table
|
||||||
|
if not self.alter_metadata:
|
||||||
|
self._table.meta = sqlalchemy.MetaData(bind=self._table.bind)
|
||||||
|
|
||||||
|
def _get_result_column(self):
|
||||||
|
return getattr(self, '_result_column', None)
|
||||||
|
|
||||||
|
def _set_result_column(self, column):
|
||||||
|
"""Set Column to Table based on alter_metadata evaluation."""
|
||||||
|
self.process_column(column)
|
||||||
|
if not hasattr(self, 'current_name'):
|
||||||
|
self.current_name = column.name
|
||||||
|
if self.alter_metadata:
|
||||||
|
self._result_column = column
|
||||||
|
else:
|
||||||
|
self._result_column = column.copy_fixed()
|
||||||
|
|
||||||
|
table = property(_get_table, _set_table)
|
||||||
|
result_column = property(_get_result_column, _set_result_column)
|
||||||
|
|
||||||
|
|
||||||
|
class ChangesetTable(object):
|
||||||
|
"""Changeset extensions to SQLAlchemy tables."""
|
||||||
|
|
||||||
|
def create_column(self, column, *p, **kw):
|
||||||
|
"""Creates a column.
|
||||||
|
|
||||||
|
The column parameter may be a column definition or the name of
|
||||||
|
a column in this table.
|
||||||
|
|
||||||
|
API to :meth:`ChangesetColumn.create`
|
||||||
|
|
||||||
|
:param column: Column to be created
|
||||||
|
:type column: Column instance or string
|
||||||
|
"""
|
||||||
|
if not isinstance(column, sqlalchemy.Column):
|
||||||
|
# It's a column name
|
||||||
|
column = getattr(self.c, str(column))
|
||||||
|
column.create(table=self, *p, **kw)
|
||||||
|
|
||||||
|
def drop_column(self, column, *p, **kw):
|
||||||
|
"""Drop a column, given its name or definition.
|
||||||
|
|
||||||
|
API to :meth:`ChangesetColumn.drop`
|
||||||
|
|
||||||
|
:param column: Column to be droped
|
||||||
|
:type column: Column instance or string
|
||||||
|
"""
|
||||||
|
if not isinstance(column, sqlalchemy.Column):
|
||||||
|
# It's a column name
|
||||||
|
try:
|
||||||
|
column = getattr(self.c, str(column))
|
||||||
|
except AttributeError:
|
||||||
|
# That column isn't part of the table. We don't need
|
||||||
|
# its entire definition to drop the column, just its
|
||||||
|
# name, so create a dummy column with the same name.
|
||||||
|
column = sqlalchemy.Column(str(column))
|
||||||
|
column.drop(table=self, *p, **kw)
|
||||||
|
|
||||||
|
def rename(self, name, *args, **kwargs):
|
||||||
|
"""Rename this table.
|
||||||
|
|
||||||
|
:param name: New name of the table.
|
||||||
|
:type name: string
|
||||||
|
:param alter_metadata: If True, table will be removed from metadata
|
||||||
|
:type alter_metadata: bool
|
||||||
|
"""
|
||||||
|
self.alter_metadata = kwargs.pop('alter_metadata', DEFAULT_ALTER_METADATA)
|
||||||
|
engine = self.bind
|
||||||
|
self.new_name = name
|
||||||
|
visitorcallable = get_engine_visitor(engine, 'schemachanger')
|
||||||
|
run_single_visitor(engine, visitorcallable, self, *args, **kwargs)
|
||||||
|
|
||||||
|
# Fix metadata registration
|
||||||
|
if self.alter_metadata:
|
||||||
|
self.name = name
|
||||||
|
self.deregister()
|
||||||
|
self._set_parent(self.metadata)
|
||||||
|
|
||||||
|
def _meta_key(self):
|
||||||
|
return sqlalchemy.schema._get_table_key(self.name, self.schema)
|
||||||
|
|
||||||
|
def deregister(self):
|
||||||
|
"""Remove this table from its metadata"""
|
||||||
|
key = self._meta_key()
|
||||||
|
meta = self.metadata
|
||||||
|
if key in meta.tables:
|
||||||
|
del meta.tables[key]
|
||||||
|
|
||||||
|
|
||||||
|
class ChangesetColumn(object):
|
||||||
|
"""Changeset extensions to SQLAlchemy columns."""
|
||||||
|
|
||||||
|
def alter(self, *p, **k):
|
||||||
|
"""Alter a column's definition: ``ALTER TABLE ALTER COLUMN``.
|
||||||
|
|
||||||
|
May supply a new column object, or a list of properties to
|
||||||
|
change.
|
||||||
|
|
||||||
|
For example; the following are equivalent::
|
||||||
|
|
||||||
|
col.alter(Column('myint', Integer, DefaultClause('foobar')))
|
||||||
|
col.alter('myint', Integer, server_default='foobar', nullable=False)
|
||||||
|
col.alter(DefaultClause('foobar'), name='myint', type=Integer,\
|
||||||
|
nullable=False)
|
||||||
|
|
||||||
|
Column name, type, server_default, and nullable may be changed
|
||||||
|
here.
|
||||||
|
|
||||||
|
Direct API to :func:`alter_column`
|
||||||
|
"""
|
||||||
|
if 'table' not in k:
|
||||||
|
k['table'] = self.table
|
||||||
|
if 'engine' not in k:
|
||||||
|
k['engine'] = k['table'].bind
|
||||||
|
return alter_column(self, *p, **k)
|
||||||
|
|
||||||
|
def create(self, table=None, index_name=None, unique_name=None,
|
||||||
|
primary_key_name=None, *args, **kwargs):
|
||||||
|
"""Create this column in the database.
|
||||||
|
|
||||||
|
Assumes the given table exists. ``ALTER TABLE ADD COLUMN``,
|
||||||
|
for most databases.
|
||||||
|
|
||||||
|
:param table: Table instance to create on.
|
||||||
|
:param index_name: Creates :class:`ChangesetIndex` on this column.
|
||||||
|
:param unique_name: Creates :class:\
|
||||||
|
`~migrate.changeset.constraint.UniqueConstraint` on this column.
|
||||||
|
:param primary_key_name: Creates :class:\
|
||||||
|
`~migrate.changeset.constraint.PrimaryKeyConstraint` on this column.
|
||||||
|
:param alter_metadata: If True, column will be added to table object.
|
||||||
|
:type table: Table instance
|
||||||
|
:type index_name: string
|
||||||
|
:type unique_name: string
|
||||||
|
:type primary_key_name: string
|
||||||
|
:type alter_metadata: bool
|
||||||
|
"""
|
||||||
|
self.alter_metadata = kwargs.pop('alter_metadata', DEFAULT_ALTER_METADATA)
|
||||||
|
self.index_name = index_name
|
||||||
|
self.unique_name = unique_name
|
||||||
|
self.primary_key_name = primary_key_name
|
||||||
|
for cons in ('index_name', 'unique_name', 'primary_key_name'):
|
||||||
|
self._check_sanity_constraints(cons)
|
||||||
|
|
||||||
|
if self.alter_metadata:
|
||||||
|
self.add_to_table(table)
|
||||||
|
engine = self.table.bind
|
||||||
|
visitorcallable = get_engine_visitor(engine, 'columngenerator')
|
||||||
|
engine._run_visitor(visitorcallable, self, *args, **kwargs)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def drop(self, table=None, *args, **kwargs):
|
||||||
|
"""Drop this column from the database, leaving its table intact.
|
||||||
|
|
||||||
|
``ALTER TABLE DROP COLUMN``, for most databases.
|
||||||
|
|
||||||
|
:param alter_metadata: If True, column will be removed from table object.
|
||||||
|
:type alter_metadata: bool
|
||||||
|
"""
|
||||||
|
self.alter_metadata = kwargs.pop('alter_metadata', DEFAULT_ALTER_METADATA)
|
||||||
|
if table is not None:
|
||||||
|
self.table = table
|
||||||
|
engine = self.table.bind
|
||||||
|
if self.alter_metadata:
|
||||||
|
self.remove_from_table(self.table, unset_table=False)
|
||||||
|
visitorcallable = get_engine_visitor(engine, 'columndropper')
|
||||||
|
engine._run_visitor(visitorcallable, self, *args, **kwargs)
|
||||||
|
if self.alter_metadata:
|
||||||
|
self.table = None
|
||||||
|
return self
|
||||||
|
|
||||||
|
def add_to_table(self, table):
|
||||||
|
if table and not self.table:
|
||||||
|
self._set_parent(table)
|
||||||
|
|
||||||
|
def remove_from_table(self, table, unset_table=True):
|
||||||
|
# TODO: remove indexes, primary keys, constraints, etc
|
||||||
|
if unset_table:
|
||||||
|
self.table = None
|
||||||
|
if table.c.contains_column(self):
|
||||||
|
table.c.remove(self)
|
||||||
|
|
||||||
|
# TODO: this is fixed in 0.6
|
||||||
|
def copy_fixed(self, **kw):
|
||||||
|
"""Create a copy of this ``Column``, with all attributes."""
|
||||||
|
return sqlalchemy.Column(self.name, self.type, self.default,
|
||||||
|
key=self.key,
|
||||||
|
primary_key=self.primary_key,
|
||||||
|
nullable=self.nullable,
|
||||||
|
quote=self.quote,
|
||||||
|
index=self.index,
|
||||||
|
unique=self.unique,
|
||||||
|
onupdate=self.onupdate,
|
||||||
|
autoincrement=self.autoincrement,
|
||||||
|
server_default=self.server_default,
|
||||||
|
server_onupdate=self.server_onupdate,
|
||||||
|
*[c.copy(**kw) for c in self.constraints])
|
||||||
|
|
||||||
|
def _check_sanity_constraints(self, name):
|
||||||
|
"""Check if constraints names are correct"""
|
||||||
|
obj = getattr(self, name)
|
||||||
|
if (getattr(self, name[:-5]) and not obj):
|
||||||
|
raise InvalidConstraintError("Column.create() accepts index_name,"
|
||||||
|
" primary_key_name and unique_name to generate constraints")
|
||||||
|
if not isinstance(obj, basestring) and obj is not None:
|
||||||
|
raise InvalidConstraintError(
|
||||||
|
"%s argument for column must be constraint name" % name)
|
||||||
|
|
||||||
|
|
||||||
|
class ChangesetIndex(object):
|
||||||
|
"""Changeset extensions to SQLAlchemy Indexes."""
|
||||||
|
|
||||||
|
__visit_name__ = 'index'
|
||||||
|
|
||||||
|
def rename(self, name, *args, **kwargs):
|
||||||
|
"""Change the name of an index.
|
||||||
|
|
||||||
|
:param name: New name of the Index.
|
||||||
|
:type name: string
|
||||||
|
:param alter_metadata: If True, Index object will be altered.
|
||||||
|
:type alter_metadata: bool
|
||||||
|
"""
|
||||||
|
self.alter_metadata = kwargs.pop('alter_metadata', DEFAULT_ALTER_METADATA)
|
||||||
|
engine = self.table.bind
|
||||||
|
self.new_name = name
|
||||||
|
visitorcallable = get_engine_visitor(engine, 'schemachanger')
|
||||||
|
engine._run_visitor(visitorcallable, self, *args, **kwargs)
|
||||||
|
if self.alter_metadata:
|
||||||
|
self.name = name
|
||||||
|
|
||||||
|
|
||||||
|
class ChangesetDefaultClause(object):
|
||||||
|
"""Implements comparison between :class:`DefaultClause` instances"""
|
||||||
|
|
||||||
|
def __eq__(self, other):
|
||||||
|
if isinstance(other, self.__class__):
|
||||||
|
if self.arg == other.arg:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def __ne__(self, other):
|
||||||
|
return not self.__eq__(other)
|
||||||
5
migrate/versioning/__init__.py
Normal file
5
migrate/versioning/__init__.py
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
"""
|
||||||
|
This package provides functionality to create and manage
|
||||||
|
repositories of database schema changesets and to apply these
|
||||||
|
changesets to databases.
|
||||||
|
"""
|
||||||
368
migrate/versioning/api.py
Normal file
368
migrate/versioning/api.py
Normal file
@ -0,0 +1,368 @@
|
|||||||
|
"""
|
||||||
|
This module provides an external API to the versioning system.
|
||||||
|
|
||||||
|
.. versionchanged:: 0.4.5
|
||||||
|
``--preview_sql`` displays source file when using SQL scripts.
|
||||||
|
If Python script is used, it runs the action with mocked engine and
|
||||||
|
returns captured SQL statements.
|
||||||
|
|
||||||
|
.. versionchanged:: 0.4.5
|
||||||
|
Deprecated ``--echo`` parameter in favour of new
|
||||||
|
:func:`migrate.versioning.util.construct_engine` behavior.
|
||||||
|
"""
|
||||||
|
|
||||||
|
# Dear migrate developers,
|
||||||
|
#
|
||||||
|
# please do not comment this module using sphinx syntax because its
|
||||||
|
# docstrings are presented as user help and most users cannot
|
||||||
|
# interpret sphinx annotated ReStructuredText.
|
||||||
|
#
|
||||||
|
# Thanks,
|
||||||
|
# Jan Dittberner
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import inspect
|
||||||
|
import warnings
|
||||||
|
|
||||||
|
from migrate.versioning import (exceptions, repository, schema, version,
|
||||||
|
script as script_) # command name conflict
|
||||||
|
from migrate.versioning.util import catch_known_errors, construct_engine
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
'help',
|
||||||
|
'create',
|
||||||
|
'script',
|
||||||
|
'script_sql',
|
||||||
|
'make_update_script_for_model',
|
||||||
|
'version',
|
||||||
|
'source',
|
||||||
|
'version_control',
|
||||||
|
'db_version',
|
||||||
|
'upgrade',
|
||||||
|
'downgrade',
|
||||||
|
'drop_version_control',
|
||||||
|
'manage',
|
||||||
|
'test',
|
||||||
|
'compare_model_to_db',
|
||||||
|
'create_model',
|
||||||
|
'update_db_from_model',
|
||||||
|
]
|
||||||
|
|
||||||
|
Repository = repository.Repository
|
||||||
|
ControlledSchema = schema.ControlledSchema
|
||||||
|
VerNum = version.VerNum
|
||||||
|
PythonScript = script_.PythonScript
|
||||||
|
SqlScript = script_.SqlScript
|
||||||
|
|
||||||
|
|
||||||
|
# deprecated
|
||||||
|
def help(cmd=None, **opts):
|
||||||
|
"""%prog help COMMAND
|
||||||
|
|
||||||
|
Displays help on a given command.
|
||||||
|
"""
|
||||||
|
if cmd is None:
|
||||||
|
raise exceptions.UsageError(None)
|
||||||
|
try:
|
||||||
|
func = globals()[cmd]
|
||||||
|
except:
|
||||||
|
raise exceptions.UsageError(
|
||||||
|
"'%s' isn't a valid command. Try 'help COMMAND'" % cmd)
|
||||||
|
ret = func.__doc__
|
||||||
|
if sys.argv[0]:
|
||||||
|
ret = ret.replace('%prog', sys.argv[0])
|
||||||
|
return ret
|
||||||
|
|
||||||
|
@catch_known_errors
|
||||||
|
def create(repository, name, **opts):
|
||||||
|
"""%prog create REPOSITORY_PATH NAME [--table=TABLE]
|
||||||
|
|
||||||
|
Create an empty repository at the specified path.
|
||||||
|
|
||||||
|
You can specify the version_table to be used; by default, it is
|
||||||
|
'migrate_version'. This table is created in all version-controlled
|
||||||
|
databases.
|
||||||
|
"""
|
||||||
|
repo_path = Repository.create(repository, name, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
@catch_known_errors
|
||||||
|
def script(description, repository, **opts):
|
||||||
|
"""%prog script DESCRIPTION REPOSITORY_PATH
|
||||||
|
|
||||||
|
Create an empty change script using the next unused version number
|
||||||
|
appended with the given description.
|
||||||
|
|
||||||
|
For instance, manage.py script "Add initial tables" creates:
|
||||||
|
repository/versions/001_Add_initial_tables.py
|
||||||
|
"""
|
||||||
|
repo = Repository(repository)
|
||||||
|
repo.create_script(description, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
@catch_known_errors
|
||||||
|
def script_sql(database, repository, **opts):
|
||||||
|
"""%prog script_sql DATABASE REPOSITORY_PATH
|
||||||
|
|
||||||
|
Create empty change SQL scripts for given DATABASE, where DATABASE
|
||||||
|
is either specific ('postgres', 'mysql', 'oracle', 'sqlite', etc.)
|
||||||
|
or generic ('default').
|
||||||
|
|
||||||
|
For instance, manage.py script_sql postgres creates:
|
||||||
|
repository/versions/001_postgres_upgrade.sql and
|
||||||
|
repository/versions/001_postgres_postgres.sql
|
||||||
|
"""
|
||||||
|
repo = Repository(repository)
|
||||||
|
repo.create_script_sql(database, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
def version(repository, **opts):
|
||||||
|
"""%prog version REPOSITORY_PATH
|
||||||
|
|
||||||
|
Display the latest version available in a repository.
|
||||||
|
"""
|
||||||
|
repo = Repository(repository)
|
||||||
|
return repo.latest
|
||||||
|
|
||||||
|
|
||||||
|
def db_version(url, repository, **opts):
|
||||||
|
"""%prog db_version URL REPOSITORY_PATH
|
||||||
|
|
||||||
|
Show the current version of the repository with the given
|
||||||
|
connection string, under version control of the specified
|
||||||
|
repository.
|
||||||
|
|
||||||
|
The url should be any valid SQLAlchemy connection string.
|
||||||
|
"""
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
schema = ControlledSchema(engine, repository)
|
||||||
|
return schema.version
|
||||||
|
|
||||||
|
|
||||||
|
def source(version, dest=None, repository=None, **opts):
|
||||||
|
"""%prog source VERSION [DESTINATION] --repository=REPOSITORY_PATH
|
||||||
|
|
||||||
|
Display the Python code for a particular version in this
|
||||||
|
repository. Save it to the file at DESTINATION or, if omitted,
|
||||||
|
send to stdout.
|
||||||
|
"""
|
||||||
|
if repository is None:
|
||||||
|
raise exceptions.UsageError("A repository must be specified")
|
||||||
|
repo = Repository(repository)
|
||||||
|
ret = repo.version(version).script().source()
|
||||||
|
if dest is not None:
|
||||||
|
dest = open(dest, 'w')
|
||||||
|
dest.write(ret)
|
||||||
|
dest.close()
|
||||||
|
ret = None
|
||||||
|
return ret
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade(url, repository, version=None, **opts):
|
||||||
|
"""%prog upgrade URL REPOSITORY_PATH [VERSION] [--preview_py|--preview_sql]
|
||||||
|
|
||||||
|
Upgrade a database to a later version.
|
||||||
|
|
||||||
|
This runs the upgrade() function defined in your change scripts.
|
||||||
|
|
||||||
|
By default, the database is updated to the latest available
|
||||||
|
version. You may specify a version instead, if you wish.
|
||||||
|
|
||||||
|
You may preview the Python or SQL code to be executed, rather than
|
||||||
|
actually executing it, using the appropriate 'preview' option.
|
||||||
|
"""
|
||||||
|
err = "Cannot upgrade a database of version %s to version %s. "\
|
||||||
|
"Try 'downgrade' instead."
|
||||||
|
return _migrate(url, repository, version, upgrade=True, err=err, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade(url, repository, version, **opts):
|
||||||
|
"""%prog downgrade URL REPOSITORY_PATH VERSION [--preview_py|--preview_sql]
|
||||||
|
|
||||||
|
Downgrade a database to an earlier version.
|
||||||
|
|
||||||
|
This is the reverse of upgrade; this runs the downgrade() function
|
||||||
|
defined in your change scripts.
|
||||||
|
|
||||||
|
You may preview the Python or SQL code to be executed, rather than
|
||||||
|
actually executing it, using the appropriate 'preview' option.
|
||||||
|
"""
|
||||||
|
err = "Cannot downgrade a database of version %s to version %s. "\
|
||||||
|
"Try 'upgrade' instead."
|
||||||
|
return _migrate(url, repository, version, upgrade=False, err=err, **opts)
|
||||||
|
|
||||||
|
def test(repository, url, **opts):
|
||||||
|
"""%prog test REPOSITORY_PATH URL [VERSION]
|
||||||
|
|
||||||
|
Performs the upgrade and downgrade option on the given
|
||||||
|
database. This is not a real test and may leave the database in a
|
||||||
|
bad state. You should therefore better run the test on a copy of
|
||||||
|
your database.
|
||||||
|
"""
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
repos = Repository(repository)
|
||||||
|
script = repos.version(None).script()
|
||||||
|
|
||||||
|
# Upgrade
|
||||||
|
print "Upgrading...",
|
||||||
|
script.run(engine, 1)
|
||||||
|
print "done"
|
||||||
|
|
||||||
|
print "Downgrading...",
|
||||||
|
script.run(engine, -1)
|
||||||
|
print "done"
|
||||||
|
print "Success"
|
||||||
|
|
||||||
|
|
||||||
|
def version_control(url, repository, version=None, **opts):
|
||||||
|
"""%prog version_control URL REPOSITORY_PATH [VERSION]
|
||||||
|
|
||||||
|
Mark a database as under this repository's version control.
|
||||||
|
|
||||||
|
Once a database is under version control, schema changes should
|
||||||
|
only be done via change scripts in this repository.
|
||||||
|
|
||||||
|
This creates the table version_table in the database.
|
||||||
|
|
||||||
|
The url should be any valid SQLAlchemy connection string.
|
||||||
|
|
||||||
|
By default, the database begins at version 0 and is assumed to be
|
||||||
|
empty. If the database is not empty, you may specify a version at
|
||||||
|
which to begin instead. No attempt is made to verify this
|
||||||
|
version's correctness - the database schema is expected to be
|
||||||
|
identical to what it would be if the database were created from
|
||||||
|
scratch.
|
||||||
|
"""
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
ControlledSchema.create(engine, repository, version)
|
||||||
|
|
||||||
|
|
||||||
|
def drop_version_control(url, repository, **opts):
|
||||||
|
"""%prog drop_version_control URL REPOSITORY_PATH
|
||||||
|
|
||||||
|
Removes version control from a database.
|
||||||
|
"""
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
schema = ControlledSchema(engine, repository)
|
||||||
|
schema.drop()
|
||||||
|
|
||||||
|
|
||||||
|
def manage(file, **opts):
|
||||||
|
"""%prog manage FILENAME [VARIABLES...]
|
||||||
|
|
||||||
|
Creates a script that runs Migrate with a set of default values.
|
||||||
|
|
||||||
|
For example::
|
||||||
|
|
||||||
|
%prog manage manage.py --repository=/path/to/repository \
|
||||||
|
--url=sqlite:///project.db
|
||||||
|
|
||||||
|
would create the script manage.py. The following two commands
|
||||||
|
would then have exactly the same results::
|
||||||
|
|
||||||
|
python manage.py version
|
||||||
|
%prog version --repository=/path/to/repository
|
||||||
|
"""
|
||||||
|
return Repository.create_manage_file(file, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
def compare_model_to_db(url, model, repository, **opts):
|
||||||
|
"""%prog compare_model_to_db URL MODEL REPOSITORY_PATH
|
||||||
|
|
||||||
|
Compare the current model (assumed to be a module level variable
|
||||||
|
of type sqlalchemy.MetaData) against the current database.
|
||||||
|
|
||||||
|
NOTE: This is EXPERIMENTAL.
|
||||||
|
""" # TODO: get rid of EXPERIMENTAL label
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
print ControlledSchema.compare_model_to_db(engine, model, repository)
|
||||||
|
|
||||||
|
|
||||||
|
def create_model(url, repository, **opts):
|
||||||
|
"""%prog create_model URL REPOSITORY_PATH [DECLERATIVE=True]
|
||||||
|
|
||||||
|
Dump the current database as a Python model to stdout.
|
||||||
|
|
||||||
|
NOTE: This is EXPERIMENTAL.
|
||||||
|
""" # TODO: get rid of EXPERIMENTAL label
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
declarative = opts.get('declarative', False)
|
||||||
|
print ControlledSchema.create_model(engine, repository, declarative)
|
||||||
|
|
||||||
|
|
||||||
|
# TODO: get rid of this? if we don't add back path param
|
||||||
|
@catch_known_errors
|
||||||
|
def make_update_script_for_model(url, oldmodel, model, repository, **opts):
|
||||||
|
"""%prog make_update_script_for_model URL OLDMODEL MODEL REPOSITORY_PATH
|
||||||
|
|
||||||
|
Create a script changing the old Python model to the new (current)
|
||||||
|
Python model, sending to stdout.
|
||||||
|
|
||||||
|
NOTE: This is EXPERIMENTAL.
|
||||||
|
""" # TODO: get rid of EXPERIMENTAL label
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
print PythonScript.make_update_script_for_model(
|
||||||
|
engine, oldmodel, model, repository, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
def update_db_from_model(url, model, repository, **opts):
|
||||||
|
"""%prog update_db_from_model URL MODEL REPOSITORY_PATH
|
||||||
|
|
||||||
|
Modify the database to match the structure of the current Python
|
||||||
|
model. This also sets the db_version number to the latest in the
|
||||||
|
repository.
|
||||||
|
|
||||||
|
NOTE: This is EXPERIMENTAL.
|
||||||
|
""" # TODO: get rid of EXPERIMENTAL label
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
schema = ControlledSchema(engine, repository)
|
||||||
|
schema.update_db_from_model(model)
|
||||||
|
|
||||||
|
|
||||||
|
def _migrate(url, repository, version, upgrade, err, **opts):
|
||||||
|
engine = construct_engine(url, **opts)
|
||||||
|
schema = ControlledSchema(engine, repository)
|
||||||
|
version = _migrate_version(schema, version, upgrade, err)
|
||||||
|
|
||||||
|
changeset = schema.changeset(version)
|
||||||
|
for ver, change in changeset:
|
||||||
|
nextver = ver + changeset.step
|
||||||
|
print '%s -> %s... ' % (ver, nextver)
|
||||||
|
|
||||||
|
if opts.get('preview_sql'):
|
||||||
|
if isinstance(change, PythonScript):
|
||||||
|
print change.preview_sql(url, changeset.step, **opts)
|
||||||
|
elif isinstance(change, SqlScript):
|
||||||
|
print change.source()
|
||||||
|
|
||||||
|
elif opts.get('preview_py'):
|
||||||
|
source_ver = max(ver, nextver)
|
||||||
|
module = schema.repository.version(source_ver).script().module
|
||||||
|
funcname = upgrade and "upgrade" or "downgrade"
|
||||||
|
func = getattr(module, funcname)
|
||||||
|
if isinstance(change, PythonScript):
|
||||||
|
print inspect.getsource(func)
|
||||||
|
else:
|
||||||
|
raise UsageError("Python source can be only displayed"
|
||||||
|
" for python migration files")
|
||||||
|
else:
|
||||||
|
schema.runchange(ver, change, changeset.step)
|
||||||
|
print 'done'
|
||||||
|
|
||||||
|
|
||||||
|
def _migrate_version(schema, version, upgrade, err):
|
||||||
|
if version is None:
|
||||||
|
return version
|
||||||
|
# Version is specified: ensure we're upgrading in the right direction
|
||||||
|
# (current version < target version for upgrading; reverse for down)
|
||||||
|
version = VerNum(version)
|
||||||
|
cur = schema.version
|
||||||
|
if upgrade is not None:
|
||||||
|
if upgrade:
|
||||||
|
direction = cur <= version
|
||||||
|
else:
|
||||||
|
direction = cur >= version
|
||||||
|
if not direction:
|
||||||
|
raise exceptions.KnownError(err % (cur, version))
|
||||||
|
return version
|
||||||
5
migrate/versioning/base/__init__.py
Normal file
5
migrate/versioning/base/__init__.py
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
"""Things that should be imported by all migrate packages"""
|
||||||
|
|
||||||
|
#__all__ = ['logging','log','databases','operations']
|
||||||
|
from logger import logging, log
|
||||||
|
from const import databases, operations
|
||||||
11
migrate/versioning/base/const.py
Normal file
11
migrate/versioning/base/const.py
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
from sqlalchemy.util import OrderedDict
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ['databases', 'operations']
|
||||||
|
|
||||||
|
databases = ('sqlite', 'postgres', 'mysql', 'oracle', 'mssql', 'firebird')
|
||||||
|
|
||||||
|
# Map operation names to function names
|
||||||
|
operations = OrderedDict()
|
||||||
|
operations['upgrade'] = 'upgrade'
|
||||||
|
operations['downgrade'] = 'downgrade'
|
||||||
9
migrate/versioning/base/logger.py
Normal file
9
migrate/versioning/base/logger.py
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
"""Manages logging (to stdout) for our versioning system.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
log=logging.getLogger('migrate.versioning')
|
||||||
|
log.setLevel(logging.WARNING)
|
||||||
|
log.addHandler(logging.StreamHandler())
|
||||||
|
|
||||||
|
__all__ = ['log','logging']
|
||||||
27
migrate/versioning/cfgparse.py
Normal file
27
migrate/versioning/cfgparse.py
Normal file
@ -0,0 +1,27 @@
|
|||||||
|
"""
|
||||||
|
Configuration parser module.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from ConfigParser import ConfigParser
|
||||||
|
|
||||||
|
from migrate.versioning.base import *
|
||||||
|
from migrate.versioning import pathed
|
||||||
|
|
||||||
|
|
||||||
|
class Parser(ConfigParser):
|
||||||
|
"""A project configuration file."""
|
||||||
|
|
||||||
|
def to_dict(self, sections=None):
|
||||||
|
"""It's easier to access config values like dictionaries"""
|
||||||
|
return self._sections
|
||||||
|
|
||||||
|
|
||||||
|
class Config(pathed.Pathed, Parser):
|
||||||
|
"""Configuration class."""
|
||||||
|
|
||||||
|
def __init__(self, path, *p, **k):
|
||||||
|
"""Confirm the config file exists; read it."""
|
||||||
|
self.require_found(path)
|
||||||
|
pathed.Pathed.__init__(self, path)
|
||||||
|
Parser.__init__(self, *p, **k)
|
||||||
|
self.read(path)
|
||||||
75
migrate/versioning/exceptions.py
Normal file
75
migrate/versioning/exceptions.py
Normal file
@ -0,0 +1,75 @@
|
|||||||
|
"""
|
||||||
|
Provide exception classes for :mod:`migrate.versioning`
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class Error(Exception):
|
||||||
|
"""Error base class."""
|
||||||
|
|
||||||
|
|
||||||
|
class ApiError(Error):
|
||||||
|
"""Base class for API errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class KnownError(ApiError):
|
||||||
|
"""A known error condition."""
|
||||||
|
|
||||||
|
|
||||||
|
class UsageError(ApiError):
|
||||||
|
"""A known error condition where help should be displayed."""
|
||||||
|
|
||||||
|
|
||||||
|
class ControlledSchemaError(Error):
|
||||||
|
"""Base class for controlled schema errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidVersionError(ControlledSchemaError):
|
||||||
|
"""Invalid version number."""
|
||||||
|
|
||||||
|
|
||||||
|
class DatabaseNotControlledError(ControlledSchemaError):
|
||||||
|
"""Database should be under version control, but it's not."""
|
||||||
|
|
||||||
|
|
||||||
|
class DatabaseAlreadyControlledError(ControlledSchemaError):
|
||||||
|
"""Database shouldn't be under version control, but it is"""
|
||||||
|
|
||||||
|
|
||||||
|
class WrongRepositoryError(ControlledSchemaError):
|
||||||
|
"""This database is under version control by another repository."""
|
||||||
|
|
||||||
|
|
||||||
|
class NoSuchTableError(ControlledSchemaError):
|
||||||
|
"""The table does not exist."""
|
||||||
|
|
||||||
|
|
||||||
|
class PathError(Error):
|
||||||
|
"""Base class for path errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class PathNotFoundError(PathError):
|
||||||
|
"""A path with no file was required; found a file."""
|
||||||
|
|
||||||
|
|
||||||
|
class PathFoundError(PathError):
|
||||||
|
"""A path with a file was required; found no file."""
|
||||||
|
|
||||||
|
|
||||||
|
class RepositoryError(Error):
|
||||||
|
"""Base class for repository errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidRepositoryError(RepositoryError):
|
||||||
|
"""Invalid repository error."""
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptError(Error):
|
||||||
|
"""Base class for script errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidScriptError(ScriptError):
|
||||||
|
"""Invalid script error."""
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidVersionError(Error):
|
||||||
|
"""Invalid version error."""
|
||||||
220
migrate/versioning/genmodel.py
Normal file
220
migrate/versioning/genmodel.py
Normal file
@ -0,0 +1,220 @@
|
|||||||
|
"""
|
||||||
|
Code to generate a Python model from a database or differences
|
||||||
|
between a model and database.
|
||||||
|
|
||||||
|
Some of this is borrowed heavily from the AutoCode project at:
|
||||||
|
http://code.google.com/p/sqlautocode/
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import migrate
|
||||||
|
import sqlalchemy
|
||||||
|
|
||||||
|
|
||||||
|
HEADER = """
|
||||||
|
## File autogenerated by genmodel.py
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
meta = MetaData()
|
||||||
|
"""
|
||||||
|
|
||||||
|
DECLARATIVE_HEADER = """
|
||||||
|
## File autogenerated by genmodel.py
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
from sqlalchemy.ext import declarative
|
||||||
|
|
||||||
|
Base = declarative.declarative_base()
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class ModelGenerator(object):
|
||||||
|
|
||||||
|
def __init__(self, diff, declarative=False):
|
||||||
|
self.diff = diff
|
||||||
|
self.declarative = declarative
|
||||||
|
|
||||||
|
|
||||||
|
def column_repr(self, col):
|
||||||
|
kwarg = []
|
||||||
|
if col.key != col.name:
|
||||||
|
kwarg.append('key')
|
||||||
|
if col.primary_key:
|
||||||
|
col.primary_key = True # otherwise it dumps it as 1
|
||||||
|
kwarg.append('primary_key')
|
||||||
|
if not col.nullable:
|
||||||
|
kwarg.append('nullable')
|
||||||
|
if col.onupdate:
|
||||||
|
kwarg.append('onupdate')
|
||||||
|
if col.default:
|
||||||
|
if col.primary_key:
|
||||||
|
# I found that PostgreSQL automatically creates a
|
||||||
|
# default value for the sequence, but let's not show
|
||||||
|
# that.
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
kwarg.append('default')
|
||||||
|
ks = ', '.join('%s=%r' % (k, getattr(col, k)) for k in kwarg)
|
||||||
|
|
||||||
|
# crs: not sure if this is good idea, but it gets rid of extra
|
||||||
|
# u''
|
||||||
|
name = col.name.encode('utf8')
|
||||||
|
|
||||||
|
type_ = col.type
|
||||||
|
for cls in col.type.__class__.__mro__:
|
||||||
|
if cls.__module__ == 'sqlalchemy.types' and \
|
||||||
|
not cls.__name__.isupper():
|
||||||
|
if cls is not type_.__class__:
|
||||||
|
type_ = cls()
|
||||||
|
break
|
||||||
|
|
||||||
|
data = {
|
||||||
|
'name': name,
|
||||||
|
'type': type_,
|
||||||
|
'constraints': ', '.join([repr(cn) for cn in col.constraints]),
|
||||||
|
'args': ks and ks or ''}
|
||||||
|
|
||||||
|
if data['constraints']:
|
||||||
|
if data['args']:
|
||||||
|
data['args'] = ',' + data['args']
|
||||||
|
|
||||||
|
if data['constraints'] or data['args']:
|
||||||
|
data['maybeComma'] = ','
|
||||||
|
else:
|
||||||
|
data['maybeComma'] = ''
|
||||||
|
|
||||||
|
commonStuff = """ %(maybeComma)s %(constraints)s %(args)s)""" % data
|
||||||
|
commonStuff = commonStuff.strip()
|
||||||
|
data['commonStuff'] = commonStuff
|
||||||
|
if self.declarative:
|
||||||
|
return """%(name)s = Column(%(type)r%(commonStuff)s""" % data
|
||||||
|
else:
|
||||||
|
return """Column(%(name)r, %(type)r%(commonStuff)s""" % data
|
||||||
|
|
||||||
|
def getTableDefn(self, table):
|
||||||
|
out = []
|
||||||
|
tableName = table.name
|
||||||
|
if self.declarative:
|
||||||
|
out.append("class %(table)s(Base):" % {'table': tableName})
|
||||||
|
out.append(" __tablename__ = '%(table)s'" % {'table': tableName})
|
||||||
|
for col in table.columns:
|
||||||
|
out.append(" %s" % self.column_repr(col))
|
||||||
|
else:
|
||||||
|
out.append("%(table)s = Table('%(table)s', meta," % \
|
||||||
|
{'table': tableName})
|
||||||
|
for col in table.columns:
|
||||||
|
out.append(" %s," % self.column_repr(col))
|
||||||
|
out.append(")")
|
||||||
|
return out
|
||||||
|
|
||||||
|
def toPython(self):
|
||||||
|
"""Assume database is current and model is empty."""
|
||||||
|
out = []
|
||||||
|
if self.declarative:
|
||||||
|
out.append(DECLARATIVE_HEADER)
|
||||||
|
else:
|
||||||
|
out.append(HEADER)
|
||||||
|
out.append("")
|
||||||
|
for table in self.diff.tablesMissingInModel:
|
||||||
|
out.extend(self.getTableDefn(table))
|
||||||
|
out.append("")
|
||||||
|
return '\n'.join(out)
|
||||||
|
|
||||||
|
def toUpgradeDowngradePython(self, indent=' '):
|
||||||
|
''' Assume model is most current and database is out-of-date. '''
|
||||||
|
|
||||||
|
decls = ['meta = MetaData()']
|
||||||
|
for table in self.diff.tablesMissingInModel + \
|
||||||
|
self.diff.tablesMissingInDatabase:
|
||||||
|
decls.extend(self.getTableDefn(table))
|
||||||
|
|
||||||
|
upgradeCommands, downgradeCommands = [], []
|
||||||
|
for table in self.diff.tablesMissingInModel:
|
||||||
|
tableName = table.name
|
||||||
|
upgradeCommands.append("%(table)s.drop()" % {'table': tableName})
|
||||||
|
downgradeCommands.append("%(table)s.create()" % \
|
||||||
|
{'table': tableName})
|
||||||
|
for table in self.diff.tablesMissingInDatabase:
|
||||||
|
tableName = table.name
|
||||||
|
upgradeCommands.append("%(table)s.create()" % {'table': tableName})
|
||||||
|
downgradeCommands.append("%(table)s.drop()" % {'table': tableName})
|
||||||
|
|
||||||
|
pre_command = 'meta.bind(migrate_engine)'
|
||||||
|
|
||||||
|
return (
|
||||||
|
'\n'.join(decls),
|
||||||
|
'\n'.join([pre_command] + ['%s%s' % (indent, line) for line in upgradeCommands]),
|
||||||
|
'\n'.join([pre_command] + ['%s%s' % (indent, line) for line in downgradeCommands]))
|
||||||
|
|
||||||
|
def applyModel(self):
|
||||||
|
"""Apply model to current database."""
|
||||||
|
# Yuck! We have to import from changeset to apply the
|
||||||
|
# monkey-patch to allow column adding/dropping.
|
||||||
|
from migrate.changeset import schema
|
||||||
|
|
||||||
|
def dbCanHandleThisChange(missingInDatabase, missingInModel, diffDecl):
|
||||||
|
if missingInDatabase and not missingInModel and not diffDecl:
|
||||||
|
# Even sqlite can handle this.
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
return not self.diff.conn.url.drivername.startswith('sqlite')
|
||||||
|
|
||||||
|
meta = sqlalchemy.MetaData(self.diff.conn.engine)
|
||||||
|
|
||||||
|
for table in self.diff.tablesMissingInModel:
|
||||||
|
table = table.tometadata(meta)
|
||||||
|
table.drop()
|
||||||
|
for table in self.diff.tablesMissingInDatabase:
|
||||||
|
table = table.tometadata(meta)
|
||||||
|
table.create()
|
||||||
|
for modelTable in self.diff.tablesWithDiff:
|
||||||
|
modelTable = modelTable.tometadata(meta)
|
||||||
|
dbTable = self.diff.reflected_model.tables[modelTable.name]
|
||||||
|
tableName = modelTable.name
|
||||||
|
missingInDatabase, missingInModel, diffDecl = \
|
||||||
|
self.diff.colDiffs[tableName]
|
||||||
|
if dbCanHandleThisChange(missingInDatabase, missingInModel,
|
||||||
|
diffDecl):
|
||||||
|
for col in missingInDatabase:
|
||||||
|
modelTable.columns[col.name].create()
|
||||||
|
for col in missingInModel:
|
||||||
|
dbTable.columns[col.name].drop()
|
||||||
|
for modelCol, databaseCol, modelDecl, databaseDecl in diffDecl:
|
||||||
|
databaseCol.alter(modelCol)
|
||||||
|
else:
|
||||||
|
# Sqlite doesn't support drop column, so you have to
|
||||||
|
# do more: create temp table, copy data to it, drop
|
||||||
|
# old table, create new table, copy data back.
|
||||||
|
#
|
||||||
|
# I wonder if this is guaranteed to be unique?
|
||||||
|
tempName = '_temp_%s' % modelTable.name
|
||||||
|
|
||||||
|
def getCopyStatement():
|
||||||
|
preparer = self.diff.conn.engine.dialect.preparer
|
||||||
|
commonCols = []
|
||||||
|
for modelCol in modelTable.columns:
|
||||||
|
if modelCol.name in dbTable.columns:
|
||||||
|
commonCols.append(modelCol.name)
|
||||||
|
commonColsStr = ', '.join(commonCols)
|
||||||
|
return 'INSERT INTO %s (%s) SELECT %s FROM %s' % \
|
||||||
|
(tableName, commonColsStr, commonColsStr, tempName)
|
||||||
|
|
||||||
|
# Move the data in one transaction, so that we don't
|
||||||
|
# leave the database in a nasty state.
|
||||||
|
connection = self.diff.conn.connect()
|
||||||
|
trans = connection.begin()
|
||||||
|
try:
|
||||||
|
connection.execute(
|
||||||
|
'CREATE TEMPORARY TABLE %s as SELECT * from %s' % \
|
||||||
|
(tempName, modelTable.name))
|
||||||
|
# make sure the drop takes place inside our
|
||||||
|
# transaction with the bind parameter
|
||||||
|
modelTable.drop(bind=connection)
|
||||||
|
modelTable.create(bind=connection)
|
||||||
|
connection.execute(getCopyStatement())
|
||||||
|
connection.execute('DROP TABLE %s' % tempName)
|
||||||
|
trans.commit()
|
||||||
|
except:
|
||||||
|
trans.rollback()
|
||||||
|
raise
|
||||||
97
migrate/versioning/migrate_repository.py
Normal file
97
migrate/versioning/migrate_repository.py
Normal file
@ -0,0 +1,97 @@
|
|||||||
|
"""
|
||||||
|
Script to migrate repository from sqlalchemy <= 0.4.4 to the new
|
||||||
|
repository schema. This shouldn't use any other migrate modules, so
|
||||||
|
that it can work in any version.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
def usage():
|
||||||
|
"""Gives usage information."""
|
||||||
|
print """Usage: %(prog)s repository-to-migrate
|
||||||
|
|
||||||
|
Upgrade your repository to the new flat format.
|
||||||
|
|
||||||
|
NOTE: You should probably make a backup before running this.
|
||||||
|
""" % {'prog': sys.argv[0]}
|
||||||
|
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
def delete_file(filepath):
|
||||||
|
"""Deletes a file and prints a message."""
|
||||||
|
print ' Deleting file: %s' % filepath
|
||||||
|
os.remove(filepath)
|
||||||
|
|
||||||
|
|
||||||
|
def move_file(src, tgt):
|
||||||
|
"""Moves a file and prints a message."""
|
||||||
|
print ' Moving file %s to %s' % (src, tgt)
|
||||||
|
if os.path.exists(tgt):
|
||||||
|
raise Exception(
|
||||||
|
'Cannot move file %s because target %s already exists' % \
|
||||||
|
(src, tgt))
|
||||||
|
os.rename(src, tgt)
|
||||||
|
|
||||||
|
|
||||||
|
def delete_directory(dirpath):
|
||||||
|
"""Delete a directory and print a message."""
|
||||||
|
print ' Deleting directory: %s' % dirpath
|
||||||
|
os.rmdir(dirpath)
|
||||||
|
|
||||||
|
|
||||||
|
def migrate_repository(repos):
|
||||||
|
"""Does the actual migration to the new repository format."""
|
||||||
|
print 'Migrating repository at: %s to new format' % repos
|
||||||
|
versions = '%s/versions' % repos
|
||||||
|
dirs = os.listdir(versions)
|
||||||
|
# Only use int's in list.
|
||||||
|
numdirs = [int(dirname) for dirname in dirs if dirname.isdigit()]
|
||||||
|
numdirs.sort() # Sort list.
|
||||||
|
for dirname in numdirs:
|
||||||
|
origdir = '%s/%s' % (versions, dirname)
|
||||||
|
print ' Working on directory: %s' % origdir
|
||||||
|
files = os.listdir(origdir)
|
||||||
|
files.sort()
|
||||||
|
for filename in files:
|
||||||
|
# Delete compiled Python files.
|
||||||
|
if filename.endswith('.pyc') or filename.endswith('.pyo'):
|
||||||
|
delete_file('%s/%s' % (origdir, filename))
|
||||||
|
|
||||||
|
# Delete empty __init__.py files.
|
||||||
|
origfile = '%s/__init__.py' % origdir
|
||||||
|
if os.path.exists(origfile) and len(open(origfile).read()) == 0:
|
||||||
|
delete_file(origfile)
|
||||||
|
|
||||||
|
# Move sql upgrade scripts.
|
||||||
|
if filename.endswith('.sql'):
|
||||||
|
version, dbms, operation = filename.split('.', 3)[0:3]
|
||||||
|
origfile = '%s/%s' % (origdir, filename)
|
||||||
|
# For instance: 2.postgres.upgrade.sql ->
|
||||||
|
# 002_postgres_upgrade.sql
|
||||||
|
tgtfile = '%s/%03d_%s_%s.sql' % (
|
||||||
|
versions, int(version), dbms, operation)
|
||||||
|
move_file(origfile, tgtfile)
|
||||||
|
|
||||||
|
# Move Python upgrade script.
|
||||||
|
pyfile = '%s.py' % dirname
|
||||||
|
pyfilepath = '%s/%s' % (origdir, pyfile)
|
||||||
|
if os.path.exists(pyfilepath):
|
||||||
|
tgtfile = '%s/%03d.py' % (versions, int(dirname))
|
||||||
|
move_file(pyfilepath, tgtfile)
|
||||||
|
|
||||||
|
# Try to remove directory. Will fail if it's not empty.
|
||||||
|
delete_directory(origdir)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""Main function to be called when using this script."""
|
||||||
|
if len(sys.argv) != 2:
|
||||||
|
usage()
|
||||||
|
migrate_repository(sys.argv[1])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
72
migrate/versioning/pathed.py
Normal file
72
migrate/versioning/pathed.py
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
"""
|
||||||
|
A path/directory class.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions
|
||||||
|
from migrate.versioning.base import *
|
||||||
|
from migrate.versioning.util import KeyedInstance
|
||||||
|
|
||||||
|
|
||||||
|
class Pathed(KeyedInstance):
|
||||||
|
"""
|
||||||
|
A class associated with a path/directory tree.
|
||||||
|
|
||||||
|
Only one instance of this class may exist for a particular file;
|
||||||
|
__new__ will return an existing instance if possible
|
||||||
|
"""
|
||||||
|
parent = None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _key(cls, path):
|
||||||
|
return str(path)
|
||||||
|
|
||||||
|
def __init__(self, path):
|
||||||
|
self.path = path
|
||||||
|
if self.__class__.parent is not None:
|
||||||
|
self._init_parent(path)
|
||||||
|
|
||||||
|
def _init_parent(self, path):
|
||||||
|
"""Try to initialize this object's parent, if it has one"""
|
||||||
|
parent_path = self.__class__._parent_path(path)
|
||||||
|
self.parent = self.__class__.parent(parent_path)
|
||||||
|
log.info("Getting parent %r:%r" % (self.__class__.parent, parent_path))
|
||||||
|
self.parent._init_child(path, self)
|
||||||
|
|
||||||
|
def _init_child(self, child, path):
|
||||||
|
"""Run when a child of this object is initialized.
|
||||||
|
|
||||||
|
Parameters: the child object; the path to this object (its
|
||||||
|
parent)
|
||||||
|
"""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _parent_path(cls, path):
|
||||||
|
"""
|
||||||
|
Fetch the path of this object's parent from this object's path.
|
||||||
|
"""
|
||||||
|
# os.path.dirname(), but strip directories like files (like
|
||||||
|
# unix basename)
|
||||||
|
#
|
||||||
|
# Treat directories like files...
|
||||||
|
if path[-1] == '/':
|
||||||
|
path = path[:-1]
|
||||||
|
ret = os.path.dirname(path)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def require_notfound(cls, path):
|
||||||
|
"""Ensures a given path does not already exist"""
|
||||||
|
if os.path.exists(path):
|
||||||
|
raise exceptions.PathFoundError(path)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def require_found(cls, path):
|
||||||
|
"""Ensures a given path already exists"""
|
||||||
|
if not os.path.exists(path):
|
||||||
|
raise exceptions.PathNotFoundError(path)
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
return self.path
|
||||||
216
migrate/versioning/repository.py
Normal file
216
migrate/versioning/repository.py
Normal file
@ -0,0 +1,216 @@
|
|||||||
|
"""
|
||||||
|
SQLAlchemy migrate repository management.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import string
|
||||||
|
from pkg_resources import resource_string, resource_filename
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions, script, version, pathed, cfgparse
|
||||||
|
from migrate.versioning.template import template
|
||||||
|
from migrate.versioning.base import *
|
||||||
|
|
||||||
|
|
||||||
|
class Changeset(dict):
|
||||||
|
"""A collection of changes to be applied to a database.
|
||||||
|
|
||||||
|
Changesets are bound to a repository and manage a set of
|
||||||
|
scripts from that repository.
|
||||||
|
|
||||||
|
Behaves like a dict, for the most part. Keys are ordered based on step value.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, start, *changes, **k):
|
||||||
|
"""
|
||||||
|
Give a start version; step must be explicitly stated.
|
||||||
|
"""
|
||||||
|
self.step = k.pop('step', 1)
|
||||||
|
self.start = version.VerNum(start)
|
||||||
|
self.end = self.start
|
||||||
|
for change in changes:
|
||||||
|
self.add(change)
|
||||||
|
|
||||||
|
def __iter__(self):
|
||||||
|
return iter(self.items())
|
||||||
|
|
||||||
|
def keys(self):
|
||||||
|
"""
|
||||||
|
In a series of upgrades x -> y, keys are version x. Sorted.
|
||||||
|
"""
|
||||||
|
ret = super(Changeset, self).keys()
|
||||||
|
# Reverse order if downgrading
|
||||||
|
ret.sort(reverse=(self.step < 1))
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def values(self):
|
||||||
|
return [self[k] for k in self.keys()]
|
||||||
|
|
||||||
|
def items(self):
|
||||||
|
return zip(self.keys(), self.values())
|
||||||
|
|
||||||
|
def add(self, change):
|
||||||
|
"""Add new change to changeset"""
|
||||||
|
key = self.end
|
||||||
|
self.end += self.step
|
||||||
|
self[key] = change
|
||||||
|
|
||||||
|
def run(self, *p, **k):
|
||||||
|
"""Run the changeset scripts"""
|
||||||
|
for version, script in self:
|
||||||
|
script.run(*p, **k)
|
||||||
|
|
||||||
|
|
||||||
|
class Repository(pathed.Pathed):
|
||||||
|
"""A project's change script repository"""
|
||||||
|
|
||||||
|
_config = 'migrate.cfg'
|
||||||
|
_versions = 'versions'
|
||||||
|
|
||||||
|
def __init__(self, path):
|
||||||
|
log.info('Loading repository %s...' % path)
|
||||||
|
self.verify(path)
|
||||||
|
super(Repository, self).__init__(path)
|
||||||
|
self.config = cfgparse.Config(os.path.join(self.path, self._config))
|
||||||
|
self.versions = version.Collection(os.path.join(self.path,
|
||||||
|
self._versions))
|
||||||
|
log.info('Repository %s loaded successfully' % path)
|
||||||
|
log.debug('Config: %r' % self.config.to_dict())
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def verify(cls, path):
|
||||||
|
"""
|
||||||
|
Ensure the target path is a valid repository.
|
||||||
|
|
||||||
|
:raises: :exc:`InvalidRepositoryError <migrate.versioning.exceptions.InvalidRepositoryError>`
|
||||||
|
"""
|
||||||
|
# Ensure the existance of required files
|
||||||
|
try:
|
||||||
|
cls.require_found(path)
|
||||||
|
cls.require_found(os.path.join(path, cls._config))
|
||||||
|
cls.require_found(os.path.join(path, cls._versions))
|
||||||
|
except exceptions.PathNotFoundError, e:
|
||||||
|
raise exceptions.InvalidRepositoryError(path)
|
||||||
|
|
||||||
|
# TODO: what are those options?
|
||||||
|
@classmethod
|
||||||
|
def prepare_config(cls, pkg, rsrc, name, **opts):
|
||||||
|
"""
|
||||||
|
Prepare a project configuration file for a new project.
|
||||||
|
"""
|
||||||
|
# Prepare opts
|
||||||
|
defaults = dict(
|
||||||
|
version_table = 'migrate_version',
|
||||||
|
repository_id = name,
|
||||||
|
required_dbs = [])
|
||||||
|
|
||||||
|
defaults.update(opts)
|
||||||
|
|
||||||
|
tmpl = resource_string(pkg, rsrc)
|
||||||
|
ret = string.Template(tmpl).substitute(defaults)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(cls, path, name, **opts):
|
||||||
|
"""Create a repository at a specified path"""
|
||||||
|
cls.require_notfound(path)
|
||||||
|
|
||||||
|
pkg, rsrc = template.get_repository(as_pkg=True)
|
||||||
|
tmplpkg = '.'.join((pkg, rsrc))
|
||||||
|
tmplfile = resource_filename(pkg, rsrc)
|
||||||
|
config_text = cls.prepare_config(tmplpkg, cls._config, name, **opts)
|
||||||
|
|
||||||
|
# Create repository
|
||||||
|
shutil.copytree(tmplfile, path)
|
||||||
|
|
||||||
|
# Edit config defaults
|
||||||
|
fd = open(os.path.join(path, cls._config), 'w')
|
||||||
|
fd.write(config_text)
|
||||||
|
fd.close()
|
||||||
|
|
||||||
|
# Create a management script
|
||||||
|
manager = os.path.join(path, 'manage.py')
|
||||||
|
Repository.create_manage_file(manager, repository=path)
|
||||||
|
|
||||||
|
return cls(path)
|
||||||
|
|
||||||
|
def create_script(self, description, **k):
|
||||||
|
"""API to :meth:`migrate.versioning.version.Collection.create_new_python_version`"""
|
||||||
|
self.versions.create_new_python_version(description, **k)
|
||||||
|
|
||||||
|
def create_script_sql(self, database, **k):
|
||||||
|
"""API to :meth:`migrate.versioning.version.Collection.create_new_sql_version`"""
|
||||||
|
self.versions.create_new_sql_version(database, **k)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def latest(self):
|
||||||
|
"""API to :attr:`migrate.versioning.version.Collection.latest`"""
|
||||||
|
return self.versions.latest
|
||||||
|
|
||||||
|
@property
|
||||||
|
def version_table(self):
|
||||||
|
"""Returns version_table name specified in config"""
|
||||||
|
return self.config.get('db_settings', 'version_table')
|
||||||
|
|
||||||
|
@property
|
||||||
|
def id(self):
|
||||||
|
"""Returns repository id specified in config"""
|
||||||
|
return self.config.get('db_settings', 'repository_id')
|
||||||
|
|
||||||
|
def version(self, *p, **k):
|
||||||
|
"""API to :attr:`migrate.versioning.version.Collection.version`"""
|
||||||
|
return self.versions.version(*p, **k)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def clear(cls):
|
||||||
|
# TODO: deletes repo
|
||||||
|
super(Repository, cls).clear()
|
||||||
|
version.Collection.clear()
|
||||||
|
|
||||||
|
def changeset(self, database, start, end=None):
|
||||||
|
"""Create a changeset to migrate this database from ver. start to end/latest.
|
||||||
|
|
||||||
|
:param database: name of database to generate changeset
|
||||||
|
:param start: version to start at
|
||||||
|
:param end: version to end at (latest if None given)
|
||||||
|
:type database: string
|
||||||
|
:type start: int
|
||||||
|
:type end: int
|
||||||
|
:returns: :class:`Changeset instance <migration.versioning.repository.Changeset>`
|
||||||
|
"""
|
||||||
|
start = version.VerNum(start)
|
||||||
|
|
||||||
|
if end is None:
|
||||||
|
end = self.latest
|
||||||
|
else:
|
||||||
|
end = version.VerNum(end)
|
||||||
|
|
||||||
|
if start <= end:
|
||||||
|
step = 1
|
||||||
|
range_mod = 1
|
||||||
|
op = 'upgrade'
|
||||||
|
else:
|
||||||
|
step = -1
|
||||||
|
range_mod = 0
|
||||||
|
op = 'downgrade'
|
||||||
|
|
||||||
|
versions = range(start + range_mod, end + range_mod, step)
|
||||||
|
changes = [self.version(v).script(database, op) for v in versions]
|
||||||
|
ret = Changeset(start, step=step, *changes)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create_manage_file(cls, file_, **opts):
|
||||||
|
"""Create a project management script (manage.py)
|
||||||
|
|
||||||
|
:param file_: Destination file to be written
|
||||||
|
:param opts: Options that are passed to template
|
||||||
|
"""
|
||||||
|
vars_ = ",".join(["%s='%s'" % var for var in opts.iteritems()])
|
||||||
|
|
||||||
|
pkg, rsrc = template.manage(as_pkg=True)
|
||||||
|
tmpl = resource_string(pkg, rsrc)
|
||||||
|
result = tmpl % dict(defaults=vars_)
|
||||||
|
|
||||||
|
fd = open(file_, 'w')
|
||||||
|
fd.write(result)
|
||||||
|
fd.close()
|
||||||
210
migrate/versioning/schema.py
Normal file
210
migrate/versioning/schema.py
Normal file
@ -0,0 +1,210 @@
|
|||||||
|
"""
|
||||||
|
Database schema version management.
|
||||||
|
"""
|
||||||
|
from sqlalchemy import (Table, Column, MetaData, String, Text, Integer,
|
||||||
|
create_engine)
|
||||||
|
from sqlalchemy.sql import and_
|
||||||
|
from sqlalchemy import exceptions as sa_exceptions
|
||||||
|
from sqlalchemy.sql import bindparam
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions, genmodel, schemadiff
|
||||||
|
from migrate.versioning.repository import Repository
|
||||||
|
from migrate.versioning.util import load_model
|
||||||
|
from migrate.versioning.version import VerNum
|
||||||
|
|
||||||
|
|
||||||
|
class ControlledSchema(object):
|
||||||
|
"""A database under version control"""
|
||||||
|
|
||||||
|
def __init__(self, engine, repository):
|
||||||
|
if isinstance(repository, str):
|
||||||
|
repository = Repository(repository)
|
||||||
|
self.engine = engine
|
||||||
|
self.repository = repository
|
||||||
|
self.meta = MetaData(engine)
|
||||||
|
self.load()
|
||||||
|
|
||||||
|
def __eq__(self, other):
|
||||||
|
"""Compare two schemas by repositories and versions"""
|
||||||
|
return (self.repository is other.repository \
|
||||||
|
and self.version == other.version)
|
||||||
|
|
||||||
|
def load(self):
|
||||||
|
"""Load controlled schema version info from DB"""
|
||||||
|
tname = self.repository.version_table
|
||||||
|
if not hasattr(self, 'table') or self.table is None:
|
||||||
|
try:
|
||||||
|
self.table = Table(tname, self.meta, autoload=True)
|
||||||
|
except (sa_exceptions.NoSuchTableError,
|
||||||
|
AssertionError):
|
||||||
|
# assertionerror is raised if no table is found in oracle db
|
||||||
|
raise exceptions.DatabaseNotControlledError(tname)
|
||||||
|
|
||||||
|
# TODO?: verify that the table is correct (# cols, etc.)
|
||||||
|
result = self.engine.execute(self.table.select(
|
||||||
|
self.table.c.repository_id == str(self.repository.id)))
|
||||||
|
|
||||||
|
try:
|
||||||
|
data = list(result)[0]
|
||||||
|
except IndexError:
|
||||||
|
raise exceptions.DatabaseNotControlledError(tname)
|
||||||
|
|
||||||
|
self.version = data['version']
|
||||||
|
return data
|
||||||
|
|
||||||
|
def drop(self):
|
||||||
|
"""
|
||||||
|
Remove version control from a database.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
self.table.drop()
|
||||||
|
except (sa_exceptions.SQLError):
|
||||||
|
raise exceptions.DatabaseNotControlledError(str(self.table))
|
||||||
|
|
||||||
|
def changeset(self, version=None):
|
||||||
|
"""API to Changeset creation.
|
||||||
|
|
||||||
|
Uses self.version for start version and engine.name
|
||||||
|
to get database name.
|
||||||
|
"""
|
||||||
|
database = self.engine.name
|
||||||
|
start_ver = self.version
|
||||||
|
changeset = self.repository.changeset(database, start_ver, version)
|
||||||
|
return changeset
|
||||||
|
|
||||||
|
def runchange(self, ver, change, step):
|
||||||
|
startver = ver
|
||||||
|
endver = ver + step
|
||||||
|
# Current database version must be correct! Don't run if corrupt!
|
||||||
|
if self.version != startver:
|
||||||
|
raise exceptions.InvalidVersionError("%s is not %s" % \
|
||||||
|
(self.version, startver))
|
||||||
|
# Run the change
|
||||||
|
change.run(self.engine, step)
|
||||||
|
|
||||||
|
# Update/refresh database version
|
||||||
|
self.update_repository_table(startver, endver)
|
||||||
|
self.load()
|
||||||
|
|
||||||
|
def update_repository_table(self, startver, endver):
|
||||||
|
"""Update version_table with new information"""
|
||||||
|
update = self.table.update(and_(self.table.c.version == int(startver),
|
||||||
|
self.table.c.repository_id == str(self.repository.id)))
|
||||||
|
self.engine.execute(update, version=int(endver))
|
||||||
|
|
||||||
|
def upgrade(self, version=None):
|
||||||
|
"""
|
||||||
|
Upgrade (or downgrade) to a specified version, or latest version.
|
||||||
|
"""
|
||||||
|
changeset = self.changeset(version)
|
||||||
|
for ver, change in changeset:
|
||||||
|
self.runchange(ver, change, changeset.step)
|
||||||
|
|
||||||
|
def update_db_from_model(self, model):
|
||||||
|
"""
|
||||||
|
Modify the database to match the structure of the current Python model.
|
||||||
|
"""
|
||||||
|
model = load_model(model)
|
||||||
|
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(
|
||||||
|
model, self.engine, excludeTables=[self.repository.version_table])
|
||||||
|
genmodel.ModelGenerator(diff).applyModel()
|
||||||
|
|
||||||
|
self.update_repository_table(self.version, int(self.repository.latest))
|
||||||
|
|
||||||
|
self.load()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(cls, engine, repository, version=None):
|
||||||
|
"""
|
||||||
|
Declare a database to be under a repository's version control.
|
||||||
|
|
||||||
|
:raises: :exc:`DatabaseAlreadyControlledError`
|
||||||
|
:returns: :class:`ControlledSchema`
|
||||||
|
"""
|
||||||
|
# Confirm that the version # is valid: positive, integer,
|
||||||
|
# exists in repos
|
||||||
|
if isinstance(repository, basestring):
|
||||||
|
repository = Repository(repository)
|
||||||
|
version = cls._validate_version(repository, version)
|
||||||
|
table = cls._create_table_version(engine, repository, version)
|
||||||
|
# TODO: history table
|
||||||
|
# Load repository information and return
|
||||||
|
return cls(engine, repository)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _validate_version(cls, repository, version):
|
||||||
|
"""
|
||||||
|
Ensures this is a valid version number for this repository.
|
||||||
|
|
||||||
|
:raises: :exc:`InvalidVersionError` if invalid
|
||||||
|
:return: valid version number
|
||||||
|
"""
|
||||||
|
if version is None:
|
||||||
|
version = 0
|
||||||
|
try:
|
||||||
|
version = VerNum(version) # raises valueerror
|
||||||
|
if version < 0 or version > repository.latest:
|
||||||
|
raise ValueError()
|
||||||
|
except ValueError:
|
||||||
|
raise exceptions.InvalidVersionError(version)
|
||||||
|
return version
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _create_table_version(cls, engine, repository, version):
|
||||||
|
"""
|
||||||
|
Creates the versioning table in a database.
|
||||||
|
|
||||||
|
:raises: :exc:`DatabaseAlreadyControlledError`
|
||||||
|
"""
|
||||||
|
# Create tables
|
||||||
|
tname = repository.version_table
|
||||||
|
meta = MetaData(engine)
|
||||||
|
|
||||||
|
table = Table(
|
||||||
|
tname, meta,
|
||||||
|
Column('repository_id', String(250), primary_key=True),
|
||||||
|
Column('repository_path', Text),
|
||||||
|
Column('version', Integer), )
|
||||||
|
|
||||||
|
# there can be multiple repositories/schemas in the same db
|
||||||
|
if not table.exists():
|
||||||
|
table.create()
|
||||||
|
|
||||||
|
# test for existing repository_id
|
||||||
|
s = table.select(table.c.repository_id == bindparam("repository_id"))
|
||||||
|
result = engine.execute(s, repository_id=repository.id)
|
||||||
|
if result.fetchone():
|
||||||
|
raise exceptions.DatabaseAlreadyControlledError
|
||||||
|
|
||||||
|
# Insert data
|
||||||
|
engine.execute(table.insert().values(
|
||||||
|
repository_id=repository.id,
|
||||||
|
repository_path=repository.path,
|
||||||
|
version=int(version)))
|
||||||
|
return table
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def compare_model_to_db(cls, engine, model, repository):
|
||||||
|
"""
|
||||||
|
Compare the current model against the current database.
|
||||||
|
"""
|
||||||
|
if isinstance(repository, basestring):
|
||||||
|
repository = Repository(repository)
|
||||||
|
model = load_model(model)
|
||||||
|
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(
|
||||||
|
model, engine, excludeTables=[repository.version_table])
|
||||||
|
return diff
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create_model(cls, engine, repository, declarative=False):
|
||||||
|
"""
|
||||||
|
Dump the current database as a Python model.
|
||||||
|
"""
|
||||||
|
if isinstance(repository, basestring):
|
||||||
|
repository = Repository(repository)
|
||||||
|
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(
|
||||||
|
MetaData(), engine, excludeTables=[repository.version_table])
|
||||||
|
return genmodel.ModelGenerator(diff, declarative).toPython()
|
||||||
214
migrate/versioning/schemadiff.py
Normal file
214
migrate/versioning/schemadiff.py
Normal file
@ -0,0 +1,214 @@
|
|||||||
|
"""
|
||||||
|
Schema differencing support.
|
||||||
|
"""
|
||||||
|
import sqlalchemy
|
||||||
|
from migrate.changeset import SQLA_06
|
||||||
|
|
||||||
|
def getDiffOfModelAgainstDatabase(model, conn, excludeTables=None):
|
||||||
|
"""
|
||||||
|
Return differences of model against database.
|
||||||
|
|
||||||
|
:return: object which will evaluate to :keyword:`True` if there \
|
||||||
|
are differences else :keyword:`False`.
|
||||||
|
"""
|
||||||
|
return SchemaDiff(model, conn, excludeTables)
|
||||||
|
|
||||||
|
|
||||||
|
def getDiffOfModelAgainstModel(oldmodel, model, conn, excludeTables=None):
|
||||||
|
"""
|
||||||
|
Return differences of model against another model.
|
||||||
|
|
||||||
|
:return: object which will evaluate to :keyword:`True` if there \
|
||||||
|
are differences else :keyword:`False`.
|
||||||
|
"""
|
||||||
|
return SchemaDiff(model, conn, excludeTables, oldmodel=oldmodel)
|
||||||
|
|
||||||
|
|
||||||
|
class SchemaDiff(object):
|
||||||
|
"""
|
||||||
|
Differences of model against database.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, model, conn, excludeTables=None, oldmodel=None):
|
||||||
|
"""
|
||||||
|
:param model: Python model's metadata
|
||||||
|
:param conn: active database connection.
|
||||||
|
"""
|
||||||
|
self.model = model
|
||||||
|
self.conn = conn
|
||||||
|
if not excludeTables:
|
||||||
|
# [] can't be default value in Python parameter
|
||||||
|
excludeTables = []
|
||||||
|
self.excludeTables = excludeTables
|
||||||
|
if oldmodel:
|
||||||
|
self.reflected_model = oldmodel
|
||||||
|
else:
|
||||||
|
self.reflected_model = sqlalchemy.MetaData(conn, reflect=True)
|
||||||
|
self.tablesMissingInDatabase, self.tablesMissingInModel, \
|
||||||
|
self.tablesWithDiff = [], [], []
|
||||||
|
self.colDiffs = {}
|
||||||
|
self.compareModelToDatabase()
|
||||||
|
|
||||||
|
def compareModelToDatabase(self):
|
||||||
|
"""
|
||||||
|
Do actual comparison.
|
||||||
|
"""
|
||||||
|
# Setup common variables.
|
||||||
|
cc = self.conn.contextual_connect()
|
||||||
|
if SQLA_06:
|
||||||
|
from sqlalchemy.ext import compiler
|
||||||
|
from sqlalchemy.schema import DDLElement
|
||||||
|
class DefineColumn(DDLElement):
|
||||||
|
def __init__(self, col):
|
||||||
|
self.col = col
|
||||||
|
|
||||||
|
@compiler.compiles(DefineColumn)
|
||||||
|
def compile(elem, compiler, **kw):
|
||||||
|
return compiler.get_column_specification(elem.col)
|
||||||
|
|
||||||
|
def get_column_specification(col):
|
||||||
|
return str(DefineColumn(col).compile(dialect=self.conn.dialect))
|
||||||
|
else:
|
||||||
|
schemagenerator = self.conn.dialect.schemagenerator(
|
||||||
|
self.conn.dialect, cc)
|
||||||
|
def get_column_specification(col):
|
||||||
|
return schemagenerator.get_column_specification(col)
|
||||||
|
|
||||||
|
# For each in model, find missing in database.
|
||||||
|
for modelName, modelTable in self.model.tables.items():
|
||||||
|
if modelName in self.excludeTables:
|
||||||
|
continue
|
||||||
|
reflectedTable = self.reflected_model.tables.get(modelName, None)
|
||||||
|
if reflectedTable:
|
||||||
|
# Table exists.
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.tablesMissingInDatabase.append(modelTable)
|
||||||
|
|
||||||
|
# For each in database, find missing in model.
|
||||||
|
for reflectedName, reflectedTable in \
|
||||||
|
self.reflected_model.tables.items():
|
||||||
|
if reflectedName in self.excludeTables:
|
||||||
|
continue
|
||||||
|
modelTable = self.model.tables.get(reflectedName, None)
|
||||||
|
if modelTable:
|
||||||
|
# Table exists.
|
||||||
|
|
||||||
|
# Find missing columns in database.
|
||||||
|
for modelCol in modelTable.columns:
|
||||||
|
databaseCol = reflectedTable.columns.get(modelCol.name,
|
||||||
|
None)
|
||||||
|
if databaseCol:
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.storeColumnMissingInDatabase(modelTable, modelCol)
|
||||||
|
|
||||||
|
# Find missing columns in model.
|
||||||
|
for databaseCol in reflectedTable.columns:
|
||||||
|
|
||||||
|
# TODO: no test coverage here? (mrb)
|
||||||
|
|
||||||
|
modelCol = modelTable.columns.get(databaseCol.name, None)
|
||||||
|
if modelCol:
|
||||||
|
# Compare attributes of column.
|
||||||
|
modelDecl = \
|
||||||
|
get_column_specification(modelCol)
|
||||||
|
databaseDecl = \
|
||||||
|
get_column_specification(databaseCol)
|
||||||
|
if modelDecl != databaseDecl:
|
||||||
|
# Unfortunately, sometimes the database
|
||||||
|
# decl won't quite match the model, even
|
||||||
|
# though they're the same.
|
||||||
|
mc, dc = modelCol.type.__class__, \
|
||||||
|
databaseCol.type.__class__
|
||||||
|
if (issubclass(mc, dc) \
|
||||||
|
or issubclass(dc, mc)) \
|
||||||
|
and modelCol.nullable == \
|
||||||
|
databaseCol.nullable:
|
||||||
|
# Types and nullable are the same.
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.storeColumnDiff(
|
||||||
|
modelTable, modelCol, databaseCol,
|
||||||
|
modelDecl, databaseDecl)
|
||||||
|
else:
|
||||||
|
self.storeColumnMissingInModel(modelTable, databaseCol)
|
||||||
|
else:
|
||||||
|
self.tablesMissingInModel.append(reflectedTable)
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
''' Summarize differences. '''
|
||||||
|
|
||||||
|
def colDiffDetails():
|
||||||
|
colout = []
|
||||||
|
for table in self.tablesWithDiff:
|
||||||
|
tableName = table.name
|
||||||
|
missingInDatabase, missingInModel, diffDecl = \
|
||||||
|
self.colDiffs[tableName]
|
||||||
|
if missingInDatabase:
|
||||||
|
colout.append(
|
||||||
|
' %s missing columns in database: %s' % \
|
||||||
|
(tableName, ', '.join(
|
||||||
|
[col.name for col in missingInDatabase])))
|
||||||
|
if missingInModel:
|
||||||
|
colout.append(
|
||||||
|
' %s missing columns in model: %s' % \
|
||||||
|
(tableName, ', '.join(
|
||||||
|
[col.name for col in missingInModel])))
|
||||||
|
if diffDecl:
|
||||||
|
colout.append(
|
||||||
|
' %s with different declaration of columns\
|
||||||
|
in database: %s' % (tableName, str(diffDecl)))
|
||||||
|
return colout
|
||||||
|
|
||||||
|
out = []
|
||||||
|
if self.tablesMissingInDatabase:
|
||||||
|
out.append(
|
||||||
|
' tables missing in database: %s' % \
|
||||||
|
', '.join(
|
||||||
|
[table.name for table in self.tablesMissingInDatabase]))
|
||||||
|
if self.tablesMissingInModel:
|
||||||
|
out.append(
|
||||||
|
' tables missing in model: %s' % \
|
||||||
|
', '.join(
|
||||||
|
[table.name for table in self.tablesMissingInModel]))
|
||||||
|
if self.tablesWithDiff:
|
||||||
|
out.append(
|
||||||
|
' tables with differences: %s' % \
|
||||||
|
', '.join([table.name for table in self.tablesWithDiff]))
|
||||||
|
|
||||||
|
if out:
|
||||||
|
out.insert(0, 'Schema diffs:')
|
||||||
|
out.extend(colDiffDetails())
|
||||||
|
return '\n'.join(out)
|
||||||
|
else:
|
||||||
|
return 'No schema diffs'
|
||||||
|
|
||||||
|
def __len__(self):
|
||||||
|
"""
|
||||||
|
Used in bool evaluation, return of 0 means no diffs.
|
||||||
|
"""
|
||||||
|
return len(self.tablesMissingInDatabase) + \
|
||||||
|
len(self.tablesMissingInModel) + len(self.tablesWithDiff)
|
||||||
|
|
||||||
|
def storeColumnMissingInDatabase(self, table, col):
|
||||||
|
if table not in self.tablesWithDiff:
|
||||||
|
self.tablesWithDiff.append(table)
|
||||||
|
missingInDatabase, missingInModel, diffDecl = \
|
||||||
|
self.colDiffs.setdefault(table.name, ([], [], []))
|
||||||
|
missingInDatabase.append(col)
|
||||||
|
|
||||||
|
def storeColumnMissingInModel(self, table, col):
|
||||||
|
if table not in self.tablesWithDiff:
|
||||||
|
self.tablesWithDiff.append(table)
|
||||||
|
missingInDatabase, missingInModel, diffDecl = \
|
||||||
|
self.colDiffs.setdefault(table.name, ([], [], []))
|
||||||
|
missingInModel.append(col)
|
||||||
|
|
||||||
|
def storeColumnDiff(self, table, modelCol, databaseCol, modelDecl,
|
||||||
|
databaseDecl):
|
||||||
|
if table not in self.tablesWithDiff:
|
||||||
|
self.tablesWithDiff.append(table)
|
||||||
|
missingInDatabase, missingInModel, diffDecl = \
|
||||||
|
self.colDiffs.setdefault(table.name, ([], [], []))
|
||||||
|
diffDecl.append((modelCol, databaseCol, modelDecl, databaseDecl))
|
||||||
6
migrate/versioning/script/__init__.py
Normal file
6
migrate/versioning/script/__init__.py
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
from migrate.versioning.script.base import BaseScript
|
||||||
|
from migrate.versioning.script.py import PythonScript
|
||||||
|
from migrate.versioning.script.sql import SqlScript
|
||||||
53
migrate/versioning/script/base.py
Normal file
53
migrate/versioning/script/base.py
Normal file
@ -0,0 +1,53 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
from migrate.versioning.base import log, operations
|
||||||
|
from migrate.versioning import pathed, exceptions
|
||||||
|
|
||||||
|
|
||||||
|
class BaseScript(pathed.Pathed):
|
||||||
|
"""Base class for other types of scripts.
|
||||||
|
All scripts have the following properties:
|
||||||
|
|
||||||
|
source (script.source())
|
||||||
|
The source code of the script
|
||||||
|
version (script.version())
|
||||||
|
The version number of the script
|
||||||
|
operations (script.operations())
|
||||||
|
The operations defined by the script: upgrade(), downgrade() or both.
|
||||||
|
Returns a tuple of operations.
|
||||||
|
Can also check for an operation with ex. script.operation(Script.ops.up)
|
||||||
|
""" # TODO: sphinxfy this and implement it correctly
|
||||||
|
|
||||||
|
def __init__(self, path):
|
||||||
|
log.info('Loading script %s...' % path)
|
||||||
|
self.verify(path)
|
||||||
|
super(BaseScript, self).__init__(path)
|
||||||
|
log.info('Script %s loaded successfully' % path)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def verify(cls, path):
|
||||||
|
"""Ensure this is a valid script
|
||||||
|
This version simply ensures the script file's existence
|
||||||
|
|
||||||
|
:raises: :exc:`InvalidScriptError <migrate.versioning.exceptions.InvalidScriptError>`
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
cls.require_found(path)
|
||||||
|
except:
|
||||||
|
raise exceptions.InvalidScriptError(path)
|
||||||
|
|
||||||
|
def source(self):
|
||||||
|
""":returns: source code of the script.
|
||||||
|
:rtype: string
|
||||||
|
"""
|
||||||
|
fd = open(self.path)
|
||||||
|
ret = fd.read()
|
||||||
|
fd.close()
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def run(self, engine):
|
||||||
|
"""Core of each BaseScript subclass.
|
||||||
|
This method executes the script.
|
||||||
|
"""
|
||||||
|
raise NotImplementedError()
|
||||||
160
migrate/versioning/script/py.py
Normal file
160
migrate/versioning/script/py.py
Normal file
@ -0,0 +1,160 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
from StringIO import StringIO
|
||||||
|
|
||||||
|
import migrate
|
||||||
|
from migrate.versioning import exceptions, genmodel, schemadiff
|
||||||
|
from migrate.versioning.base import operations
|
||||||
|
from migrate.versioning.template import template
|
||||||
|
from migrate.versioning.script import base
|
||||||
|
from migrate.versioning.util import import_path, load_model, construct_engine
|
||||||
|
|
||||||
|
|
||||||
|
class PythonScript(base.BaseScript):
|
||||||
|
"""Base for Python scripts"""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def create(cls, path, **opts):
|
||||||
|
"""Create an empty migration script at specified path
|
||||||
|
|
||||||
|
:returns: :class:`PythonScript instance <migrate.versioning.script.py.PythonScript>`"""
|
||||||
|
cls.require_notfound(path)
|
||||||
|
|
||||||
|
# TODO: Use the default script template (defined in the template
|
||||||
|
# module) for now, but we might want to allow people to specify a
|
||||||
|
# different one later.
|
||||||
|
template_file = None
|
||||||
|
src = template.get_script(template_file)
|
||||||
|
shutil.copy(src, path)
|
||||||
|
|
||||||
|
return cls(path)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def make_update_script_for_model(cls, engine, oldmodel,
|
||||||
|
model, repository, **opts):
|
||||||
|
"""Create a migration script based on difference between two SA models.
|
||||||
|
|
||||||
|
:param repository: path to migrate repository
|
||||||
|
:param oldmodel: dotted.module.name:SAClass or SAClass object
|
||||||
|
:param model: dotted.module.name:SAClass or SAClass object
|
||||||
|
:param engine: SQLAlchemy engine
|
||||||
|
:type repository: string or :class:`Repository instance <migrate.versioning.repository.Repository>`
|
||||||
|
:type oldmodel: string or Class
|
||||||
|
:type model: string or Class
|
||||||
|
:type engine: Engine instance
|
||||||
|
:returns: Upgrade / Downgrade script
|
||||||
|
:rtype: string
|
||||||
|
"""
|
||||||
|
|
||||||
|
if isinstance(repository, basestring):
|
||||||
|
# oh dear, an import cycle!
|
||||||
|
from migrate.versioning.repository import Repository
|
||||||
|
repository = Repository(repository)
|
||||||
|
|
||||||
|
oldmodel = load_model(oldmodel)
|
||||||
|
model = load_model(model)
|
||||||
|
|
||||||
|
# Compute differences.
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstModel(
|
||||||
|
oldmodel,
|
||||||
|
model,
|
||||||
|
engine,
|
||||||
|
excludeTables=[repository.version_table])
|
||||||
|
# TODO: diff can be False (there is no difference?)
|
||||||
|
decls, upgradeCommands, downgradeCommands = \
|
||||||
|
genmodel.ModelGenerator(diff).toUpgradeDowngradePython()
|
||||||
|
|
||||||
|
# Store differences into file.
|
||||||
|
# TODO: add custom templates
|
||||||
|
src = template.get_script(None)
|
||||||
|
f = open(src)
|
||||||
|
contents = f.read()
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
# generate source
|
||||||
|
search = 'def upgrade(migrate_engine):'
|
||||||
|
contents = contents.replace(search, '\n\n'.join((decls, search)), 1)
|
||||||
|
if upgradeCommands:
|
||||||
|
contents = contents.replace(' pass', upgradeCommands, 1)
|
||||||
|
if downgradeCommands:
|
||||||
|
contents = contents.replace(' pass', downgradeCommands, 1)
|
||||||
|
return contents
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def verify_module(cls, path):
|
||||||
|
"""Ensure path is a valid script
|
||||||
|
|
||||||
|
:param path: Script location
|
||||||
|
:type path: string
|
||||||
|
|
||||||
|
:raises: :exc:`InvalidScriptError <migrate.versioning.exceptions.InvalidScriptError>`
|
||||||
|
:returns: Python module
|
||||||
|
"""
|
||||||
|
# Try to import and get the upgrade() func
|
||||||
|
try:
|
||||||
|
module = import_path(path)
|
||||||
|
except:
|
||||||
|
# If the script itself has errors, that's not our problem
|
||||||
|
raise
|
||||||
|
try:
|
||||||
|
assert callable(module.upgrade)
|
||||||
|
except Exception, e:
|
||||||
|
raise exceptions.InvalidScriptError(path + ': %s' % str(e))
|
||||||
|
return module
|
||||||
|
|
||||||
|
def preview_sql(self, url, step, **args):
|
||||||
|
"""Mocks SQLAlchemy Engine to store all executed calls in a string
|
||||||
|
and runs :meth:`PythonScript.run <migrate.versioning.script.py.PythonScript.run>`
|
||||||
|
|
||||||
|
:returns: SQL file
|
||||||
|
"""
|
||||||
|
buf = StringIO()
|
||||||
|
args['engine_arg_strategy'] = 'mock'
|
||||||
|
args['engine_arg_executor'] = lambda s, p = '': buf.write(str(s) + p)
|
||||||
|
engine = construct_engine(url, **args)
|
||||||
|
|
||||||
|
self.run(engine, step)
|
||||||
|
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
def run(self, engine, step):
|
||||||
|
"""Core method of Script file.
|
||||||
|
Exectues :func:`update` or :func:`downgrade` functions
|
||||||
|
|
||||||
|
:param engine: SQLAlchemy Engine
|
||||||
|
:param step: Operation to run
|
||||||
|
:type engine: string
|
||||||
|
:type step: int
|
||||||
|
"""
|
||||||
|
if step > 0:
|
||||||
|
op = 'upgrade'
|
||||||
|
elif step < 0:
|
||||||
|
op = 'downgrade'
|
||||||
|
else:
|
||||||
|
raise exceptions.ScriptError("%d is not a valid step" % step)
|
||||||
|
funcname = base.operations[op]
|
||||||
|
|
||||||
|
func = self._func(funcname)
|
||||||
|
try:
|
||||||
|
func(engine)
|
||||||
|
except TypeError:
|
||||||
|
print "upgrade/downgrade functions must accept engine parameter (since ver 0.5.5)"
|
||||||
|
raise
|
||||||
|
|
||||||
|
@property
|
||||||
|
def module(self):
|
||||||
|
"""Calls :meth:`migrate.versioning.script.py.verify_module`
|
||||||
|
and returns it.
|
||||||
|
"""
|
||||||
|
if not hasattr(self, '_module'):
|
||||||
|
self._module = self.verify_module(self.path)
|
||||||
|
return self._module
|
||||||
|
|
||||||
|
def _func(self, funcname):
|
||||||
|
try:
|
||||||
|
return getattr(self.module, funcname)
|
||||||
|
except AttributeError:
|
||||||
|
msg = "The function %s is not defined in this script"
|
||||||
|
raise exceptions.ScriptError(msg % funcname)
|
||||||
33
migrate/versioning/script/sql.py
Normal file
33
migrate/versioning/script/sql.py
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
from migrate.versioning.script import base
|
||||||
|
|
||||||
|
|
||||||
|
class SqlScript(base.BaseScript):
|
||||||
|
"""A file containing plain SQL statements."""
|
||||||
|
|
||||||
|
# TODO: why is step parameter even here?
|
||||||
|
def run(self, engine, step=None):
|
||||||
|
"""Runs SQL script through raw dbapi execute call"""
|
||||||
|
text = self.source()
|
||||||
|
# Don't rely on SA's autocommit here
|
||||||
|
# (SA uses .startswith to check if a commit is needed. What if script
|
||||||
|
# starts with a comment?)
|
||||||
|
conn = engine.connect()
|
||||||
|
try:
|
||||||
|
trans = conn.begin()
|
||||||
|
try:
|
||||||
|
# HACK: SQLite doesn't allow multiple statements through
|
||||||
|
# its execute() method, but it provides executescript() instead
|
||||||
|
dbapi = conn.engine.raw_connection()
|
||||||
|
if getattr(dbapi, 'executescript', None):
|
||||||
|
dbapi.executescript(text)
|
||||||
|
else:
|
||||||
|
conn.execute(text)
|
||||||
|
trans.commit()
|
||||||
|
except:
|
||||||
|
trans.rollback()
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
164
migrate/versioning/shell.py
Normal file
164
migrate/versioning/shell.py
Normal file
@ -0,0 +1,164 @@
|
|||||||
|
"""The migrate command-line tool."""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import inspect
|
||||||
|
from optparse import OptionParser, BadOptionError
|
||||||
|
|
||||||
|
from migrate.versioning.base import *
|
||||||
|
from migrate.versioning import api, exceptions
|
||||||
|
|
||||||
|
|
||||||
|
alias = dict(
|
||||||
|
s=api.script,
|
||||||
|
vc=api.version_control,
|
||||||
|
dbv=api.db_version,
|
||||||
|
v=api.version,
|
||||||
|
)
|
||||||
|
|
||||||
|
def alias_setup():
|
||||||
|
global alias
|
||||||
|
for key, val in alias.iteritems():
|
||||||
|
setattr(api, key, val)
|
||||||
|
alias_setup()
|
||||||
|
|
||||||
|
|
||||||
|
class PassiveOptionParser(OptionParser):
|
||||||
|
|
||||||
|
def _process_args(self, largs, rargs, values):
|
||||||
|
"""little hack to support all --some_option=value parameters"""
|
||||||
|
|
||||||
|
while rargs:
|
||||||
|
arg = rargs[0]
|
||||||
|
if arg == "--":
|
||||||
|
del rargs[0]
|
||||||
|
return
|
||||||
|
elif arg[0:2] == "--":
|
||||||
|
# if parser does not know about the option
|
||||||
|
# pass it along (make it anonymous)
|
||||||
|
try:
|
||||||
|
opt = arg.split('=', 1)[0]
|
||||||
|
self._match_long_opt(opt)
|
||||||
|
except BadOptionError:
|
||||||
|
largs.append(arg)
|
||||||
|
del rargs[0]
|
||||||
|
else:
|
||||||
|
self._process_long_opt(rargs, values)
|
||||||
|
elif arg[:1] == "-" and len(arg) > 1:
|
||||||
|
self._process_short_opts(rargs, values)
|
||||||
|
elif self.allow_interspersed_args:
|
||||||
|
largs.append(arg)
|
||||||
|
del rargs[0]
|
||||||
|
|
||||||
|
def main(argv=None, **kwargs):
|
||||||
|
"""kwargs are default options that can be overriden with passing
|
||||||
|
--some_option to cmdline
|
||||||
|
"""
|
||||||
|
|
||||||
|
argv = argv or list(sys.argv[1:])
|
||||||
|
commands = list(api.__all__)
|
||||||
|
commands.sort()
|
||||||
|
|
||||||
|
usage = """%%prog COMMAND ...
|
||||||
|
|
||||||
|
Available commands:
|
||||||
|
%s
|
||||||
|
|
||||||
|
Enter "%%prog help COMMAND" for information on a particular command.
|
||||||
|
""" % '\n\t'.join(commands)
|
||||||
|
|
||||||
|
parser = PassiveOptionParser(usage=usage)
|
||||||
|
parser.add_option("-v", "--verbose", action="store_true", dest="verbose")
|
||||||
|
parser.add_option("-d", "--debug", action="store_true", dest="debug")
|
||||||
|
parser.add_option("-f", "--force", action="store_true", dest="force")
|
||||||
|
help_commands = ['help', '-h', '--help']
|
||||||
|
HELP = False
|
||||||
|
|
||||||
|
try:
|
||||||
|
command = argv.pop(0)
|
||||||
|
if command in help_commands:
|
||||||
|
HELP = True
|
||||||
|
command = argv.pop(0)
|
||||||
|
except IndexError:
|
||||||
|
parser.print_help()
|
||||||
|
return
|
||||||
|
|
||||||
|
command_func = getattr(api, command, None)
|
||||||
|
if command_func is None or command.startswith('_'):
|
||||||
|
parser.error("Invalid command %s" % command)
|
||||||
|
|
||||||
|
parser.set_usage(inspect.getdoc(command_func))
|
||||||
|
f_args, f_varargs, f_kwargs, f_defaults = inspect.getargspec(command_func)
|
||||||
|
for arg in f_args:
|
||||||
|
parser.add_option(
|
||||||
|
"--%s" % arg,
|
||||||
|
dest=arg,
|
||||||
|
action='store',
|
||||||
|
type="string")
|
||||||
|
|
||||||
|
# display help of the current command
|
||||||
|
if HELP:
|
||||||
|
parser.print_help()
|
||||||
|
return
|
||||||
|
|
||||||
|
options, args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
# override kwargs with anonymous parameters
|
||||||
|
override_kwargs = dict()
|
||||||
|
for arg in list(args):
|
||||||
|
if arg.startswith('--'):
|
||||||
|
args.remove(arg)
|
||||||
|
if '=' in arg:
|
||||||
|
opt, value = arg[2:].split('=', 1)
|
||||||
|
else:
|
||||||
|
opt = arg[2:]
|
||||||
|
value = True
|
||||||
|
override_kwargs[opt] = value
|
||||||
|
|
||||||
|
# override kwargs with options if user is overwriting
|
||||||
|
for key, value in options.__dict__.iteritems():
|
||||||
|
if value is not None:
|
||||||
|
override_kwargs[key] = value
|
||||||
|
|
||||||
|
# arguments that function accepts without passed kwargs
|
||||||
|
f_required = list(f_args)
|
||||||
|
candidates = dict(kwargs)
|
||||||
|
candidates.update(override_kwargs)
|
||||||
|
for key, value in candidates.iteritems():
|
||||||
|
if key in f_args:
|
||||||
|
f_required.remove(key)
|
||||||
|
|
||||||
|
# map function arguments to parsed arguments
|
||||||
|
for arg in args:
|
||||||
|
try:
|
||||||
|
kw = f_required.pop(0)
|
||||||
|
except IndexError:
|
||||||
|
parser.error("Too many arguments for command %s: %s" % (command,
|
||||||
|
arg))
|
||||||
|
kwargs[kw] = arg
|
||||||
|
|
||||||
|
# apply overrides
|
||||||
|
kwargs.update(override_kwargs)
|
||||||
|
|
||||||
|
# check if all args are given
|
||||||
|
try:
|
||||||
|
num_defaults = len(f_defaults)
|
||||||
|
except TypeError:
|
||||||
|
num_defaults = 0
|
||||||
|
f_args_default = f_args[len(f_args) - num_defaults:]
|
||||||
|
required = list(set(f_required) - set(f_args_default))
|
||||||
|
if required:
|
||||||
|
parser.error("Not enough arguments for command %s: %s not specified" \
|
||||||
|
% (command, ', '.join(required)))
|
||||||
|
|
||||||
|
# handle command
|
||||||
|
try:
|
||||||
|
ret = command_func(**kwargs)
|
||||||
|
if ret is not None:
|
||||||
|
print ret
|
||||||
|
except (exceptions.UsageError, exceptions.KnownError), e:
|
||||||
|
if e.args[0] is None:
|
||||||
|
parser.print_help()
|
||||||
|
parser.error(e.args[0])
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
84
migrate/versioning/template.py
Normal file
84
migrate/versioning/template.py
Normal file
@ -0,0 +1,84 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import sys
|
||||||
|
from pkg_resources import resource_filename
|
||||||
|
|
||||||
|
from migrate.versioning.base import *
|
||||||
|
from migrate.versioning import pathed
|
||||||
|
|
||||||
|
|
||||||
|
class Packaged(pathed.Pathed):
|
||||||
|
"""An object assoc'ed with a Python package"""
|
||||||
|
|
||||||
|
def __init__(self, pkg):
|
||||||
|
self.pkg = pkg
|
||||||
|
path = self._find_path(pkg)
|
||||||
|
super(Packaged, self).__init__(path)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _find_path(cls, pkg):
|
||||||
|
pkg_name, resource_name = pkg.rsplit('.', 1)
|
||||||
|
ret = resource_filename(pkg_name, resource_name)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
|
||||||
|
class Collection(Packaged):
|
||||||
|
"""A collection of templates of a specific type"""
|
||||||
|
|
||||||
|
_default = None
|
||||||
|
|
||||||
|
def get_path(self, file):
|
||||||
|
return os.path.join(self.path, str(file))
|
||||||
|
|
||||||
|
def get_pkg(self, file):
|
||||||
|
return (self.pkg, str(file))
|
||||||
|
|
||||||
|
|
||||||
|
class RepositoryCollection(Collection):
|
||||||
|
_default = 'default'
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptCollection(Collection):
|
||||||
|
_default = 'default.py_tmpl'
|
||||||
|
|
||||||
|
|
||||||
|
class Template(Packaged):
|
||||||
|
"""Finds the paths/packages of various Migrate templates"""
|
||||||
|
|
||||||
|
_repository = 'repository'
|
||||||
|
_script = 'script'
|
||||||
|
_manage = 'manage.py_tmpl'
|
||||||
|
|
||||||
|
def __init__(self, pkg):
|
||||||
|
super(Template, self).__init__(pkg)
|
||||||
|
self.repository = RepositoryCollection('.'.join((self.pkg,
|
||||||
|
self._repository)))
|
||||||
|
self.script = ScriptCollection('.'.join((self.pkg, self._script)))
|
||||||
|
|
||||||
|
def get_item(self, attr, filename=None, as_pkg=None, as_str=None):
|
||||||
|
item = getattr(self, attr)
|
||||||
|
if filename is None:
|
||||||
|
filename = getattr(item, '_default')
|
||||||
|
if as_pkg:
|
||||||
|
ret = item.get_pkg(filename)
|
||||||
|
if as_str:
|
||||||
|
ret = '.'.join(ret)
|
||||||
|
else:
|
||||||
|
ret = item.get_path(filename)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def get_repository(self, filename=None, as_pkg=None, as_str=None):
|
||||||
|
return self.get_item('repository', filename, as_pkg, as_str)
|
||||||
|
|
||||||
|
def get_script(self, filename=None, as_pkg=None, as_str=None):
|
||||||
|
return self.get_item('script', filename, as_pkg, as_str)
|
||||||
|
|
||||||
|
def manage(self, **k):
|
||||||
|
return (self.pkg, self._manage)
|
||||||
|
|
||||||
|
|
||||||
|
template_pkg = 'migrate.versioning.templates'
|
||||||
|
template = Template(template_pkg)
|
||||||
0
migrate/versioning/templates/__init__.py
Normal file
0
migrate/versioning/templates/__init__.py
Normal file
4
migrate/versioning/templates/manage.py_tmpl
Normal file
4
migrate/versioning/templates/manage.py_tmpl
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
from migrate.versioning.shell import main
|
||||||
|
|
||||||
|
main(%(defaults)s)
|
||||||
0
migrate/versioning/templates/repository/__init__.py
Normal file
0
migrate/versioning/templates/repository/__init__.py
Normal file
4
migrate/versioning/templates/repository/default/README
Normal file
4
migrate/versioning/templates/repository/default/README
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
This is a database migration repository.
|
||||||
|
|
||||||
|
More information at
|
||||||
|
http://code.google.com/p/sqlalchemy-migrate/
|
||||||
20
migrate/versioning/templates/repository/default/migrate.cfg
Normal file
20
migrate/versioning/templates/repository/default/migrate.cfg
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
[db_settings]
|
||||||
|
# Used to identify which repository this database is versioned under.
|
||||||
|
# You can use the name of your project.
|
||||||
|
repository_id=${repository_id}
|
||||||
|
|
||||||
|
# The name of the database table used to track the schema version.
|
||||||
|
# This name shouldn't already be used by your project.
|
||||||
|
# If this is changed once a database is under version control, you'll need to
|
||||||
|
# change the table name in each database too.
|
||||||
|
version_table=${version_table}
|
||||||
|
|
||||||
|
# When committing a change script, Migrate will attempt to generate the
|
||||||
|
# sql for all supported databases; normally, if one of them fails - probably
|
||||||
|
# because you don't have that database installed - it is ignored and the
|
||||||
|
# commit continues, perhaps ending successfully.
|
||||||
|
# Databases in this list MUST compile successfully during a commit, or the
|
||||||
|
# entire commit will fail. List the databases your application will actually
|
||||||
|
# be using to ensure your updates to that database work properly.
|
||||||
|
# This must be a list; example: ['postgres','sqlite']
|
||||||
|
required_dbs=${required_dbs}
|
||||||
0
migrate/versioning/templates/script/__init__.py
Normal file
0
migrate/versioning/templates/script/__init__.py
Normal file
11
migrate/versioning/templates/script/default.py_tmpl
Normal file
11
migrate/versioning/templates/script/default.py_tmpl
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
|
||||||
|
def upgrade(migrate_engine):
|
||||||
|
# Upgrade operations go here. Don't create your own engine; bind migrate_engine
|
||||||
|
# to your metadata
|
||||||
|
pass
|
||||||
|
|
||||||
|
def downgrade(migrate_engine):
|
||||||
|
# Operations to reverse the above upgrade go here.
|
||||||
|
pass
|
||||||
132
migrate/versioning/util/__init__.py
Normal file
132
migrate/versioning/util/__init__.py
Normal file
@ -0,0 +1,132 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import warnings
|
||||||
|
from decorator import decorator
|
||||||
|
from pkg_resources import EntryPoint
|
||||||
|
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.engine import Engine
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions
|
||||||
|
from migrate.versioning.util.keyedinstance import KeyedInstance
|
||||||
|
from migrate.versioning.util.importpath import import_path
|
||||||
|
|
||||||
|
|
||||||
|
def load_model(dotted_name):
|
||||||
|
"""Import module and use module-level variable".
|
||||||
|
|
||||||
|
:param dotted_name: path to model in form of string: ``some.python.module:Class``
|
||||||
|
|
||||||
|
.. versionchanged:: 0.5.4
|
||||||
|
|
||||||
|
"""
|
||||||
|
if isinstance(dotted_name, basestring):
|
||||||
|
if ':' not in dotted_name:
|
||||||
|
# backwards compatibility
|
||||||
|
warnings.warn('model should be in form of module.model:User '
|
||||||
|
'and not module.model.User', DeprecationWarning)
|
||||||
|
dotted_name = ':'.join(dotted_name.rsplit('.', 1))
|
||||||
|
return EntryPoint.parse('x=%s' % dotted_name).load(False)
|
||||||
|
else:
|
||||||
|
# Assume it's already loaded.
|
||||||
|
return dotted_name
|
||||||
|
|
||||||
|
def asbool(obj):
|
||||||
|
"""Do everything to use object as bool"""
|
||||||
|
if isinstance(obj, basestring):
|
||||||
|
obj = obj.strip().lower()
|
||||||
|
if obj in ['true', 'yes', 'on', 'y', 't', '1']:
|
||||||
|
return True
|
||||||
|
elif obj in ['false', 'no', 'off', 'n', 'f', '0']:
|
||||||
|
return False
|
||||||
|
else:
|
||||||
|
raise ValueError("String is not true/false: %r" % obj)
|
||||||
|
if obj in (True, False):
|
||||||
|
return bool(obj)
|
||||||
|
else:
|
||||||
|
raise ValueError("String is not true/false: %r" % obj)
|
||||||
|
|
||||||
|
def guess_obj_type(obj):
|
||||||
|
"""Do everything to guess object type from string
|
||||||
|
|
||||||
|
Tries to convert to `int`, `bool` and finally returns if not succeded.
|
||||||
|
|
||||||
|
.. versionadded: 0.5.4
|
||||||
|
"""
|
||||||
|
|
||||||
|
result = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = int(obj)
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if result is None:
|
||||||
|
try:
|
||||||
|
result = asbool(obj)
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if result is not None:
|
||||||
|
return result
|
||||||
|
else:
|
||||||
|
return obj
|
||||||
|
|
||||||
|
@decorator
|
||||||
|
def catch_known_errors(f, *a, **kw):
|
||||||
|
"""Decorator that catches known api errors
|
||||||
|
|
||||||
|
.. versionadded: 0.5.4
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
f(*a, **kw)
|
||||||
|
except exceptions.PathFoundError, e:
|
||||||
|
raise exceptions.KnownError("The path %s already exists" % e.args[0])
|
||||||
|
|
||||||
|
def construct_engine(engine, **opts):
|
||||||
|
""".. versionadded:: 0.5.4
|
||||||
|
|
||||||
|
Constructs and returns SQLAlchemy engine.
|
||||||
|
|
||||||
|
Currently, there are 2 ways to pass create_engine options to :mod:`migrate.versioning.api` functions:
|
||||||
|
|
||||||
|
:param engine: connection string or a existing engine
|
||||||
|
:param engine_dict: python dictionary of options to pass to `create_engine`
|
||||||
|
:param engine_arg_*: keyword parameters to pass to `create_engine` (evaluated with :func:`migrate.versioning.util.guess_obj_type`)
|
||||||
|
:type engine_dict: dict
|
||||||
|
:type engine: string or Engine instance
|
||||||
|
:type engine_arg_*: string
|
||||||
|
:returns: SQLAlchemy Engine
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
keyword parameters override ``engine_dict`` values.
|
||||||
|
|
||||||
|
"""
|
||||||
|
if isinstance(engine, Engine):
|
||||||
|
return engine
|
||||||
|
elif not isinstance(engine, basestring):
|
||||||
|
raise ValueError("you need to pass either an existing engine or a database uri")
|
||||||
|
|
||||||
|
# get options for create_engine
|
||||||
|
if opts.get('engine_dict') and isinstance(opts['engine_dict'], dict):
|
||||||
|
kwargs = opts['engine_dict']
|
||||||
|
else:
|
||||||
|
kwargs = dict()
|
||||||
|
|
||||||
|
# DEPRECATED: handle echo the old way
|
||||||
|
echo = asbool(opts.get('echo', False))
|
||||||
|
if echo:
|
||||||
|
warnings.warn('echo=True parameter is deprecated, pass '
|
||||||
|
'engine_arg_echo=True or engine_dict={"echo": True}',
|
||||||
|
DeprecationWarning)
|
||||||
|
kwargs['echo'] = echo
|
||||||
|
|
||||||
|
# parse keyword arguments
|
||||||
|
for key, value in opts.iteritems():
|
||||||
|
if key.startswith('engine_arg_'):
|
||||||
|
kwargs[key[11:]] = guess_obj_type(value)
|
||||||
|
|
||||||
|
return create_engine(engine, **kwargs)
|
||||||
16
migrate/versioning/util/importpath.py
Normal file
16
migrate/versioning/util/importpath.py
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
def import_path(fullpath):
|
||||||
|
""" Import a file with full path specification. Allows one to
|
||||||
|
import from anywhere, something __import__ does not do.
|
||||||
|
"""
|
||||||
|
# http://zephyrfalcon.org/weblog/arch_d7_2002_08_31.html
|
||||||
|
path, filename = os.path.split(fullpath)
|
||||||
|
filename, ext = os.path.splitext(filename)
|
||||||
|
sys.path.append(path)
|
||||||
|
module = __import__(filename)
|
||||||
|
reload(module) # Might be out of date during tests
|
||||||
|
del sys.path[-1]
|
||||||
|
return module
|
||||||
|
|
||||||
36
migrate/versioning/util/keyedinstance.py
Normal file
36
migrate/versioning/util/keyedinstance.py
Normal file
@ -0,0 +1,36 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
class KeyedInstance(object):
|
||||||
|
"""A class whose instances have a unique identifier of some sort
|
||||||
|
No two instances with the same unique ID should exist - if we try to create
|
||||||
|
a second instance, the first should be returned.
|
||||||
|
"""
|
||||||
|
|
||||||
|
_instances = dict()
|
||||||
|
|
||||||
|
def __new__(cls, *p, **k):
|
||||||
|
instances = cls._instances
|
||||||
|
clskey = str(cls)
|
||||||
|
if clskey not in instances:
|
||||||
|
instances[clskey] = dict()
|
||||||
|
instances = instances[clskey]
|
||||||
|
|
||||||
|
key = cls._key(*p, **k)
|
||||||
|
if key not in instances:
|
||||||
|
instances[key] = super(KeyedInstance, cls).__new__(cls)
|
||||||
|
return instances[key]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _key(cls, *p, **k):
|
||||||
|
"""Given a unique identifier, return a dictionary key
|
||||||
|
This should be overridden by child classes, to specify which parameters
|
||||||
|
should determine an object's uniqueness
|
||||||
|
"""
|
||||||
|
raise NotImplementedError()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def clear(cls):
|
||||||
|
# Allow cls.clear() as well as uniqueInstance.clear(cls)
|
||||||
|
if str(cls) in cls._instances:
|
||||||
|
del cls._instances[str(cls)]
|
||||||
223
migrate/versioning/version.py
Normal file
223
migrate/versioning/version.py
Normal file
@ -0,0 +1,223 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions, pathed, script
|
||||||
|
|
||||||
|
|
||||||
|
class VerNum(object):
|
||||||
|
"""A version number that behaves like a string and int at the same time"""
|
||||||
|
|
||||||
|
_instances = dict()
|
||||||
|
|
||||||
|
def __new__(cls, value):
|
||||||
|
val = str(value)
|
||||||
|
if val not in cls._instances:
|
||||||
|
cls._instances[val] = super(VerNum, cls).__new__(cls)
|
||||||
|
ret = cls._instances[val]
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def __init__(self,value):
|
||||||
|
self.value = str(int(value))
|
||||||
|
if self < 0:
|
||||||
|
raise ValueError("Version number cannot be negative")
|
||||||
|
|
||||||
|
def __add__(self, value):
|
||||||
|
ret = int(self) + int(value)
|
||||||
|
return VerNum(ret)
|
||||||
|
|
||||||
|
def __sub__(self, value):
|
||||||
|
return self + (int(value) * -1)
|
||||||
|
|
||||||
|
def __cmp__(self, value):
|
||||||
|
return int(self) - int(value)
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return "<VerNum(%s)>" % self.value
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
return str(self.value)
|
||||||
|
|
||||||
|
def __int__(self):
|
||||||
|
return int(self.value)
|
||||||
|
|
||||||
|
|
||||||
|
class Collection(pathed.Pathed):
|
||||||
|
"""A collection of versioning scripts in a repository"""
|
||||||
|
|
||||||
|
FILENAME_WITH_VERSION = re.compile(r'^(\d{3,}).*')
|
||||||
|
|
||||||
|
def __init__(self, path):
|
||||||
|
"""Collect current version scripts in repository
|
||||||
|
and store them in self.versions
|
||||||
|
"""
|
||||||
|
super(Collection, self).__init__(path)
|
||||||
|
|
||||||
|
# Create temporary list of files, allowing skipped version numbers.
|
||||||
|
files = os.listdir(path)
|
||||||
|
if '1' in files:
|
||||||
|
# deprecation
|
||||||
|
raise Exception('It looks like you have a repository in the old '
|
||||||
|
'format (with directories for each version). '
|
||||||
|
'Please convert repository before proceeding.')
|
||||||
|
|
||||||
|
tempVersions = dict()
|
||||||
|
for filename in files:
|
||||||
|
match = self.FILENAME_WITH_VERSION.match(filename)
|
||||||
|
if match:
|
||||||
|
num = int(match.group(1))
|
||||||
|
tempVersions.setdefault(num, []).append(filename)
|
||||||
|
else:
|
||||||
|
pass # Must be a helper file or something, let's ignore it.
|
||||||
|
|
||||||
|
# Create the versions member where the keys
|
||||||
|
# are VerNum's and the values are Version's.
|
||||||
|
self.versions = dict()
|
||||||
|
for num, files in tempVersions.items():
|
||||||
|
self.versions[VerNum(num)] = Version(num, path, files)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def latest(self):
|
||||||
|
""":returns: Latest version in Collection"""
|
||||||
|
return max([VerNum(0)] + self.versions.keys())
|
||||||
|
|
||||||
|
def create_new_python_version(self, description, **k):
|
||||||
|
"""Create Python files for new version"""
|
||||||
|
ver = self.latest + 1
|
||||||
|
extra = str_to_filename(description)
|
||||||
|
|
||||||
|
if extra:
|
||||||
|
if extra == '_':
|
||||||
|
extra = ''
|
||||||
|
elif not extra.startswith('_'):
|
||||||
|
extra = '_%s' % extra
|
||||||
|
|
||||||
|
filename = '%03d%s.py' % (ver, extra)
|
||||||
|
filepath = self._version_path(filename)
|
||||||
|
|
||||||
|
if os.path.exists(filepath):
|
||||||
|
raise Exception('Script already exists: %s' % filepath)
|
||||||
|
else:
|
||||||
|
script.PythonScript.create(filepath)
|
||||||
|
|
||||||
|
self.versions[ver] = Version(ver, self.path, [filename])
|
||||||
|
|
||||||
|
def create_new_sql_version(self, database, **k):
|
||||||
|
"""Create SQL files for new version"""
|
||||||
|
ver = self.latest + 1
|
||||||
|
self.versions[ver] = Version(ver, self.path, [])
|
||||||
|
|
||||||
|
# Create new files.
|
||||||
|
for op in ('upgrade', 'downgrade'):
|
||||||
|
filename = '%03d_%s_%s.sql' % (ver, database, op)
|
||||||
|
filepath = self._version_path(filename)
|
||||||
|
if os.path.exists(filepath):
|
||||||
|
raise Exception('Script already exists: %s' % filepath)
|
||||||
|
else:
|
||||||
|
open(filepath, "w").close()
|
||||||
|
self.versions[ver].add_script(filepath)
|
||||||
|
|
||||||
|
def version(self, vernum=None):
|
||||||
|
"""Returns latest Version if vernum is not given.
|
||||||
|
Otherwise, returns wanted version"""
|
||||||
|
if vernum is None:
|
||||||
|
vernum = self.latest
|
||||||
|
return self.versions[VerNum(vernum)]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def clear(cls):
|
||||||
|
super(Collection, cls).clear()
|
||||||
|
|
||||||
|
def _version_path(self, ver):
|
||||||
|
"""Returns path of file in versions repository"""
|
||||||
|
return os.path.join(self.path, str(ver))
|
||||||
|
|
||||||
|
|
||||||
|
class Version(object):
|
||||||
|
"""A single version in a collection """
|
||||||
|
|
||||||
|
def __init__(self, vernum, path, filelist):
|
||||||
|
self.version = VerNum(vernum)
|
||||||
|
|
||||||
|
# Collect scripts in this folder
|
||||||
|
self.sql = dict()
|
||||||
|
self.python = None
|
||||||
|
|
||||||
|
for script in filelist:
|
||||||
|
self.add_script(os.path.join(path, script))
|
||||||
|
|
||||||
|
def script(self, database=None, operation=None):
|
||||||
|
"""Returns SQL or Python Script"""
|
||||||
|
for db in (database, 'default'):
|
||||||
|
# Try to return a .sql script first
|
||||||
|
try:
|
||||||
|
return self.sql[db][operation]
|
||||||
|
except KeyError:
|
||||||
|
continue # No .sql script exists
|
||||||
|
|
||||||
|
# TODO: maybe add force Python parameter?
|
||||||
|
ret = self.python
|
||||||
|
|
||||||
|
assert ret is not None, \
|
||||||
|
"There is no script for %d version" % self.version
|
||||||
|
return ret
|
||||||
|
|
||||||
|
# deprecated?
|
||||||
|
@classmethod
|
||||||
|
def create(cls, path):
|
||||||
|
os.mkdir(path)
|
||||||
|
# create the version as a proper Python package
|
||||||
|
initfile = os.path.join(path, "__init__.py")
|
||||||
|
if not os.path.exists(initfile):
|
||||||
|
# just touch the file
|
||||||
|
open(initfile, "w").close()
|
||||||
|
try:
|
||||||
|
ret = cls(path)
|
||||||
|
except:
|
||||||
|
os.rmdir(path)
|
||||||
|
raise
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def add_script(self, path):
|
||||||
|
"""Add script to Collection/Version"""
|
||||||
|
if path.endswith(Extensions.py):
|
||||||
|
self._add_script_py(path)
|
||||||
|
elif path.endswith(Extensions.sql):
|
||||||
|
self._add_script_sql(path)
|
||||||
|
|
||||||
|
SQL_FILENAME = re.compile(r'^(\d+)_([^_]+)_([^_]+).sql')
|
||||||
|
|
||||||
|
def _add_script_sql(self, path):
|
||||||
|
match = self.SQL_FILENAME.match(os.path.basename(path))
|
||||||
|
|
||||||
|
if match:
|
||||||
|
version, dbms, op = match.group(1), match.group(2), match.group(3)
|
||||||
|
else:
|
||||||
|
raise exceptions.ScriptError("Invalid SQL script name %s" % path)
|
||||||
|
|
||||||
|
# File the script into a dictionary
|
||||||
|
self.sql.setdefault(dbms, {})[op] = script.SqlScript(path)
|
||||||
|
|
||||||
|
def _add_script_py(self, path):
|
||||||
|
if self.python is not None:
|
||||||
|
raise Exception('You can only have one Python script per version,'
|
||||||
|
' but you have: %s and %s' % (self.python, path))
|
||||||
|
self.python = script.PythonScript(path)
|
||||||
|
|
||||||
|
class Extensions:
|
||||||
|
"""A namespace for file extensions"""
|
||||||
|
py = 'py'
|
||||||
|
sql = 'sql'
|
||||||
|
|
||||||
|
def str_to_filename(s):
|
||||||
|
"""Replaces spaces, (double and single) quotes
|
||||||
|
and double underscores to underscores
|
||||||
|
"""
|
||||||
|
|
||||||
|
s = s.replace(' ', '_').replace('"', '_').replace("'", '_')
|
||||||
|
while '__' in s:
|
||||||
|
s = s.replace('__', '_')
|
||||||
|
return s
|
||||||
15
setup.cfg
Normal file
15
setup.cfg
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
[build_sphinx]
|
||||||
|
source-dir = docs
|
||||||
|
build-dir = docs/_build
|
||||||
|
|
||||||
|
[egg_info]
|
||||||
|
tag_svn_revision = 1
|
||||||
|
tag_build = .dev
|
||||||
|
|
||||||
|
[nosetests]
|
||||||
|
#pdb = true
|
||||||
|
#pdb-failures = true
|
||||||
|
#stop = true
|
||||||
|
|
||||||
|
[aliases]
|
||||||
|
release = egg_info -RDb ''
|
||||||
45
setup.py
Normal file
45
setup.py
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
#!/usr/bin/python
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
try:
|
||||||
|
from setuptools import setup, find_packages
|
||||||
|
except ImportError:
|
||||||
|
from ez_setup import use_setuptools
|
||||||
|
use_setuptools()
|
||||||
|
from setuptools import setup, find_packages
|
||||||
|
|
||||||
|
try:
|
||||||
|
import buildutils
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
test_requirements = ['nose >= 0.10']
|
||||||
|
required_deps = ['sqlalchemy >= 0.5', 'decorator']
|
||||||
|
readme_file = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'README'))
|
||||||
|
|
||||||
|
setup(
|
||||||
|
name = "sqlalchemy-migrate",
|
||||||
|
version = "0.5.5",
|
||||||
|
packages = find_packages(exclude=['test*']),
|
||||||
|
include_package_data = True,
|
||||||
|
description = "Database schema migration for SQLAlchemy",
|
||||||
|
long_description = readme_file.read(),
|
||||||
|
install_requires = required_deps,
|
||||||
|
tests_require = test_requirements,
|
||||||
|
extras_require = {
|
||||||
|
'docs' : ['sphinx >= 0.5'],
|
||||||
|
},
|
||||||
|
author = "Evan Rosson",
|
||||||
|
author_email = "evan.rosson@gmail.com",
|
||||||
|
url = "http://code.google.com/p/sqlalchemy-migrate/",
|
||||||
|
maintainer = "Jan Dittberner",
|
||||||
|
maintainer_email = "jan@dittberner.info",
|
||||||
|
license = "MIT",
|
||||||
|
entry_points = """
|
||||||
|
[console_scripts]
|
||||||
|
migrate = migrate.versioning.shell:main
|
||||||
|
migrate-repository = migrate.versioning.migrate_repository:main
|
||||||
|
""",
|
||||||
|
test_suite = "nose.collector",
|
||||||
|
)
|
||||||
0
test/__init__.py
Normal file
0
test/__init__.py
Normal file
0
test/changeset/__init__.py
Normal file
0
test/changeset/__init__.py
Normal file
759
test/changeset/test_changeset.py
Normal file
759
test/changeset/test_changeset.py
Normal file
@ -0,0 +1,759 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
import sqlalchemy
|
||||||
|
from sqlalchemy import *
|
||||||
|
|
||||||
|
from migrate import changeset
|
||||||
|
from migrate.changeset import *
|
||||||
|
from migrate.changeset.schema import ColumnDelta
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class TestAddDropColumn(fixture.DB):
|
||||||
|
"""Test add/drop column through all possible interfaces
|
||||||
|
also test for constraints
|
||||||
|
"""
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
table_name = 'tmp_adddropcol'
|
||||||
|
table_int = 0
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
super(TestAddDropColumn, self)._setup(url)
|
||||||
|
self.meta = MetaData()
|
||||||
|
self.table = Table(self.table_name, self.meta,
|
||||||
|
Column('id', Integer, unique=True),
|
||||||
|
)
|
||||||
|
self.meta.bind = self.engine
|
||||||
|
if self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
self.table.create()
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
if self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
self.meta.clear()
|
||||||
|
super(TestAddDropColumn,self)._teardown()
|
||||||
|
|
||||||
|
def run_(self, create_column_func, drop_column_func, *col_p, **col_k):
|
||||||
|
col_name = 'data'
|
||||||
|
|
||||||
|
def assert_numcols(num_of_expected_cols):
|
||||||
|
# number of cols should be correct in table object and in database
|
||||||
|
self.refresh_table(self.table_name)
|
||||||
|
result = len(self.table.c)
|
||||||
|
|
||||||
|
self.assertEquals(result, num_of_expected_cols),
|
||||||
|
if col_k.get('primary_key', None):
|
||||||
|
# new primary key: check its length too
|
||||||
|
result = len(self.table.primary_key)
|
||||||
|
self.assertEquals(result, num_of_expected_cols)
|
||||||
|
|
||||||
|
assert_numcols(1)
|
||||||
|
if len(col_p) == 0:
|
||||||
|
col_p = [String(40)]
|
||||||
|
col = Column(col_name, *col_p, **col_k)
|
||||||
|
create_column_func(col)
|
||||||
|
assert_numcols(2)
|
||||||
|
col2 = getattr(self.table.c, col_name)
|
||||||
|
self.assertEquals(col2, col)
|
||||||
|
drop_column_func(col2)
|
||||||
|
assert_numcols(1)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_undefined(self):
|
||||||
|
"""Add/drop columns not yet defined in the table"""
|
||||||
|
def add_func(col):
|
||||||
|
return create_column(col, self.table)
|
||||||
|
def drop_func(col):
|
||||||
|
return drop_column(col, self.table)
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_defined(self):
|
||||||
|
"""Add/drop columns already defined in the table"""
|
||||||
|
def add_func(col):
|
||||||
|
self.meta.clear()
|
||||||
|
self.table = Table(self.table_name, self.meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
col,
|
||||||
|
)
|
||||||
|
return create_column(col)
|
||||||
|
def drop_func(col):
|
||||||
|
return drop_column(col)
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_method_bound(self):
|
||||||
|
"""Add/drop columns via column methods; columns bound to a table
|
||||||
|
ie. no table parameter passed to function
|
||||||
|
"""
|
||||||
|
def add_func(col):
|
||||||
|
self.assert_(col.table is None, col.table)
|
||||||
|
self.table.append_column(col)
|
||||||
|
return col.create()
|
||||||
|
def drop_func(col):
|
||||||
|
#self.assert_(col.table is None,col.table)
|
||||||
|
#self.table.append_column(col)
|
||||||
|
return col.drop()
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_method_notbound(self):
|
||||||
|
"""Add/drop columns via column methods; columns not bound to a table"""
|
||||||
|
def add_func(col):
|
||||||
|
return col.create(self.table)
|
||||||
|
def drop_func(col):
|
||||||
|
return col.drop(self.table)
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_tablemethod_obj(self):
|
||||||
|
"""Add/drop columns via table methods; by column object"""
|
||||||
|
def add_func(col):
|
||||||
|
return self.table.create_column(col)
|
||||||
|
def drop_func(col):
|
||||||
|
return self.table.drop_column(col)
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_tablemethod_name(self):
|
||||||
|
"""Add/drop columns via table methods; by column name"""
|
||||||
|
def add_func(col):
|
||||||
|
# must be bound to table
|
||||||
|
self.table.append_column(col)
|
||||||
|
return self.table.create_column(col.name)
|
||||||
|
def drop_func(col):
|
||||||
|
# Not necessarily bound to table
|
||||||
|
return self.table.drop_column(col.name)
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_byname(self):
|
||||||
|
"""Add/drop columns via functions; by table object and column name"""
|
||||||
|
def add_func(col):
|
||||||
|
self.table.append_column(col)
|
||||||
|
return create_column(col.name, self.table)
|
||||||
|
def drop_func(col):
|
||||||
|
return drop_column(col.name, self.table)
|
||||||
|
return self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_drop_column_not_in_table(self):
|
||||||
|
"""Drop column by name"""
|
||||||
|
def add_func(col):
|
||||||
|
return self.table.create_column(col)
|
||||||
|
def drop_func(col):
|
||||||
|
self.table.c.remove(col)
|
||||||
|
return self.table.drop_column(col.name)
|
||||||
|
self.run_(add_func, drop_func)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_fk(self):
|
||||||
|
"""Can create columns with foreign keys"""
|
||||||
|
# create FK's target
|
||||||
|
reftable = Table('tmp_ref', self.meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
)
|
||||||
|
if self.engine.has_table(reftable.name):
|
||||||
|
reftable.drop()
|
||||||
|
reftable.create()
|
||||||
|
|
||||||
|
# create column with fk
|
||||||
|
col = Column('data', Integer, ForeignKey(reftable.c.id))
|
||||||
|
if self.url.startswith('sqlite'):
|
||||||
|
self.assertRaises(changeset.exceptions.NotSupportedError,
|
||||||
|
col.create, self.table)
|
||||||
|
else:
|
||||||
|
col.create(self.table)
|
||||||
|
|
||||||
|
# check if constraint is added
|
||||||
|
for cons in self.table.constraints:
|
||||||
|
if isinstance(cons, sqlalchemy.schema.ForeignKeyConstraint):
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
self.fail('No constraint found')
|
||||||
|
|
||||||
|
# TODO: test on db level if constraints work
|
||||||
|
|
||||||
|
self.assertEqual(reftable.c.id.name, col.foreign_keys[0].column.name)
|
||||||
|
col.drop(self.table)
|
||||||
|
|
||||||
|
if self.engine.has_table(reftable.name):
|
||||||
|
reftable.drop()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported='sqlite')
|
||||||
|
def test_pk(self):
|
||||||
|
"""Can create columns with primary key"""
|
||||||
|
col = Column('data', Integer, nullable=False)
|
||||||
|
self.assertRaises(changeset.exceptions.InvalidConstraintError,
|
||||||
|
col.create, self.table, primary_key_name=True)
|
||||||
|
col.create(self.table, primary_key_name='data_pkey')
|
||||||
|
|
||||||
|
# check if constraint was added (cannot test on objects)
|
||||||
|
self.table.insert(values={'data': 4}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'data': 4}).execute()
|
||||||
|
except (sqlalchemy.exc.IntegrityError,
|
||||||
|
sqlalchemy.exc.ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
col.drop()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported='mysql')
|
||||||
|
def test_check(self):
|
||||||
|
"""Can create columns with check constraint"""
|
||||||
|
col = Column('data',
|
||||||
|
Integer,
|
||||||
|
sqlalchemy.schema.CheckConstraint('data > 4'))
|
||||||
|
col.create(self.table)
|
||||||
|
|
||||||
|
# check if constraint was added (cannot test on objects)
|
||||||
|
self.table.insert(values={'data': 5}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'data': 3}).execute()
|
||||||
|
except (sqlalchemy.exc.IntegrityError,
|
||||||
|
sqlalchemy.exc.ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
col.drop()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported='sqlite')
|
||||||
|
def test_unique(self):
|
||||||
|
"""Can create columns with unique constraint"""
|
||||||
|
self.assertRaises(changeset.exceptions.InvalidConstraintError,
|
||||||
|
Column('data', Integer, unique=True).create, self.table)
|
||||||
|
col = Column('data', Integer)
|
||||||
|
col.create(self.table, unique_name='data_unique')
|
||||||
|
|
||||||
|
# check if constraint was added (cannot test on objects)
|
||||||
|
self.table.insert(values={'data': 5}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'data': 5}).execute()
|
||||||
|
except (sqlalchemy.exc.IntegrityError,
|
||||||
|
sqlalchemy.exc.ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
col.drop(self.table)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_index(self):
|
||||||
|
"""Can create columns with indexes"""
|
||||||
|
self.assertRaises(changeset.exceptions.InvalidConstraintError,
|
||||||
|
Column('data', Integer).create, self.table, index_name=True)
|
||||||
|
col = Column('data', Integer)
|
||||||
|
col.create(self.table, index_name='ix_data')
|
||||||
|
|
||||||
|
# check if index was added
|
||||||
|
self.table.insert(values={'data': 5}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'data': 5}).execute()
|
||||||
|
except (sqlalchemy.exc.IntegrityError,
|
||||||
|
sqlalchemy.exc.ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
Index('ix_data', col).drop(bind=self.engine)
|
||||||
|
col.drop()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_server_defaults(self):
|
||||||
|
"""Can create columns with server_default values"""
|
||||||
|
col = Column('data', String(244), server_default='foobar')
|
||||||
|
col.create(self.table)
|
||||||
|
|
||||||
|
self.table.insert(values={'id': 10}).execute()
|
||||||
|
row = self.table.select(autocommit=True).execute().fetchone()
|
||||||
|
self.assertEqual(u'foobar', row['data'])
|
||||||
|
|
||||||
|
col.drop()
|
||||||
|
|
||||||
|
# TODO: test sequence
|
||||||
|
# TODO: test quoting
|
||||||
|
# TODO: test non-autoname constraints
|
||||||
|
|
||||||
|
|
||||||
|
class TestRename(fixture.DB):
|
||||||
|
"""Tests for table and index rename methods"""
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
meta = MetaData()
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
super(TestRename, self)._setup(url)
|
||||||
|
self.meta.bind = self.engine
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported='firebird')
|
||||||
|
def test_rename_table(self):
|
||||||
|
"""Tables can be renamed"""
|
||||||
|
c_name = 'col_1'
|
||||||
|
table_name1 = 'name_one'
|
||||||
|
table_name2 = 'name_two'
|
||||||
|
index_name1 = 'x' + table_name1
|
||||||
|
index_name2 = 'x' + table_name2
|
||||||
|
|
||||||
|
self.meta.clear()
|
||||||
|
self.column = Column(c_name, Integer)
|
||||||
|
self.table = Table(table_name1, self.meta, self.column)
|
||||||
|
self.index = Index(index_name1, self.column, unique=False)
|
||||||
|
|
||||||
|
if self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
if self.engine.has_table(table_name2):
|
||||||
|
tmp = Table(table_name2, self.meta, autoload=True)
|
||||||
|
tmp.drop()
|
||||||
|
tmp.deregister()
|
||||||
|
del tmp
|
||||||
|
self.table.create()
|
||||||
|
|
||||||
|
def assert_table_name(expected, skip_object_check=False):
|
||||||
|
"""Refresh a table via autoload
|
||||||
|
SA has changed some since this test was written; we now need to do
|
||||||
|
meta.clear() upon reloading a table - clear all rather than a
|
||||||
|
select few. So, this works only if we're working with one table at
|
||||||
|
a time (else, others will vanish too).
|
||||||
|
"""
|
||||||
|
if not skip_object_check:
|
||||||
|
# Table object check
|
||||||
|
self.assertEquals(self.table.name,expected)
|
||||||
|
newname = self.table.name
|
||||||
|
else:
|
||||||
|
# we know the object's name isn't consistent: just assign it
|
||||||
|
newname = expected
|
||||||
|
# Table DB check
|
||||||
|
self.meta.clear()
|
||||||
|
self.table = Table(newname, self.meta, autoload=True)
|
||||||
|
self.assertEquals(self.table.name, expected)
|
||||||
|
|
||||||
|
def assert_index_name(expected, skip_object_check=False):
|
||||||
|
if not skip_object_check:
|
||||||
|
# Index object check
|
||||||
|
self.assertEquals(self.index.name, expected)
|
||||||
|
else:
|
||||||
|
# object is inconsistent
|
||||||
|
self.index.name = expected
|
||||||
|
# TODO: Index DB check
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Table renames
|
||||||
|
assert_table_name(table_name1)
|
||||||
|
rename_table(self.table, table_name2)
|
||||||
|
assert_table_name(table_name2)
|
||||||
|
self.table.rename(table_name1)
|
||||||
|
assert_table_name(table_name1)
|
||||||
|
|
||||||
|
# test by just the string
|
||||||
|
rename_table(table_name1, table_name2, engine=self.engine)
|
||||||
|
assert_table_name(table_name2, True) # object not updated
|
||||||
|
|
||||||
|
# Index renames
|
||||||
|
if self.url.startswith('sqlite') or self.url.startswith('mysql'):
|
||||||
|
self.assertRaises(changeset.exceptions.NotSupportedError,
|
||||||
|
self.index.rename, index_name2)
|
||||||
|
else:
|
||||||
|
assert_index_name(index_name1)
|
||||||
|
rename_index(self.index, index_name2, engine=self.engine)
|
||||||
|
assert_index_name(index_name2)
|
||||||
|
self.index.rename(index_name1)
|
||||||
|
assert_index_name(index_name1)
|
||||||
|
|
||||||
|
# test by just the string
|
||||||
|
rename_index(index_name1, index_name2, engine=self.engine)
|
||||||
|
assert_index_name(index_name2, True)
|
||||||
|
|
||||||
|
finally:
|
||||||
|
if self.table.exists():
|
||||||
|
self.table.drop()
|
||||||
|
|
||||||
|
|
||||||
|
class TestColumnChange(fixture.DB):
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
table_name = 'tmp_colchange'
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
super(TestColumnChange, self)._setup(url)
|
||||||
|
self.meta = MetaData(self.engine)
|
||||||
|
self.table = Table(self.table_name, self.meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
Column('data', String(40), server_default=DefaultClause("tluafed"),
|
||||||
|
nullable=True),
|
||||||
|
)
|
||||||
|
if self.table.exists():
|
||||||
|
self.table.drop()
|
||||||
|
try:
|
||||||
|
self.table.create()
|
||||||
|
except sqlalchemy.exceptions.SQLError, e:
|
||||||
|
# SQLite: database schema has changed
|
||||||
|
if not self.url.startswith('sqlite://'):
|
||||||
|
raise
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
if self.table.exists():
|
||||||
|
try:
|
||||||
|
self.table.drop(self.engine)
|
||||||
|
except sqlalchemy.exceptions.SQLError,e:
|
||||||
|
# SQLite: database schema has changed
|
||||||
|
if not self.url.startswith('sqlite://'):
|
||||||
|
raise
|
||||||
|
super(TestColumnChange, self)._teardown()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_rename(self):
|
||||||
|
"""Can rename a column"""
|
||||||
|
def num_rows(col, content):
|
||||||
|
return len(list(self.table.select(col == content).execute()))
|
||||||
|
# Table content should be preserved in changed columns
|
||||||
|
content = "fgsfds"
|
||||||
|
self.engine.execute(self.table.insert(), data=content, id=42)
|
||||||
|
self.assertEquals(num_rows(self.table.c.data, content), 1)
|
||||||
|
|
||||||
|
# ...as a function, given a column object and the new name
|
||||||
|
alter_column('data', name='data2', table=self.table)
|
||||||
|
self.refresh_table()
|
||||||
|
alter_column(self.table.c.data2, name='atad')
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_('data' not in self.table.c.keys())
|
||||||
|
self.assert_('atad' in self.table.c.keys())
|
||||||
|
self.assertEquals(num_rows(self.table.c.atad, content), 1)
|
||||||
|
|
||||||
|
# ...as a method, given a new name
|
||||||
|
self.table.c.atad.alter(name='data')
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_('atad' not in self.table.c.keys())
|
||||||
|
self.table.c.data # Should not raise exception
|
||||||
|
self.assertEquals(num_rows(self.table.c.data, content), 1)
|
||||||
|
|
||||||
|
# ...as a function, given a new object
|
||||||
|
col = Column('atad', String(40), server_default=self.table.c.data.server_default)
|
||||||
|
alter_column(self.table.c.data, col)
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_('data' not in self.table.c.keys())
|
||||||
|
self.table.c.atad # Should not raise exception
|
||||||
|
self.assertEquals(num_rows(self.table.c.atad, content), 1)
|
||||||
|
|
||||||
|
# ...as a method, given a new object
|
||||||
|
col = Column('data', String(40), server_default=self.table.c.atad.server_default)
|
||||||
|
self.table.c.atad.alter(col)
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_('atad' not in self.table.c.keys())
|
||||||
|
self.table.c.data # Should not raise exception
|
||||||
|
self.assertEquals(num_rows(self.table.c.data,content), 1)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_type(self):
|
||||||
|
"""Can change a column's type"""
|
||||||
|
# Entire column definition given
|
||||||
|
self.table.c.data.alter(Column('data', String(42)))
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_(isinstance(self.table.c.data.type, String))
|
||||||
|
if self.engine.name == 'firebird':
|
||||||
|
self.assertEquals(self.table.c.data.type.length, 42 * 4)
|
||||||
|
else:
|
||||||
|
self.assertEquals(self.table.c.data.type.length, 42)
|
||||||
|
|
||||||
|
# Just the new type
|
||||||
|
self.table.c.data.alter(type=String(43))
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_(isinstance(self.table.c.data.type, String))
|
||||||
|
if self.engine.name == 'firebird':
|
||||||
|
self.assertEquals(self.table.c.data.type.length, 43 * 4)
|
||||||
|
else:
|
||||||
|
self.assertEquals(self.table.c.data.type.length, 43)
|
||||||
|
|
||||||
|
# Different type
|
||||||
|
self.assert_(isinstance(self.table.c.id.type, Integer))
|
||||||
|
self.assertEquals(self.table.c.id.nullable, False)
|
||||||
|
|
||||||
|
if not self.engine.name == 'firebird':
|
||||||
|
self.table.c.id.alter(type=String(20))
|
||||||
|
self.assertEquals(self.table.c.id.nullable, False)
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_(isinstance(self.table.c.id.type, String))
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_default(self):
|
||||||
|
"""Can change a column's server_default value (DefaultClauses only)
|
||||||
|
Only DefaultClauses are changed here: others are managed by the
|
||||||
|
application / by SA
|
||||||
|
"""
|
||||||
|
self.assertEquals(self.table.c.data.server_default.arg, 'tluafed')
|
||||||
|
|
||||||
|
# Just the new default
|
||||||
|
default = 'my_default'
|
||||||
|
self.table.c.data.alter(server_default=DefaultClause(default))
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
#self.assertEquals(self.table.c.data.server_default.arg,default)
|
||||||
|
# TextClause returned by autoload
|
||||||
|
self.assert_(default in str(self.table.c.data.server_default.arg))
|
||||||
|
self.engine.execute(self.table.insert(), id=12)
|
||||||
|
row = self.table.select(autocommit=True).execute().fetchone()
|
||||||
|
self.assertEqual(row['data'], default)
|
||||||
|
|
||||||
|
# Column object
|
||||||
|
default = 'your_default'
|
||||||
|
self.table.c.data.alter(Column('data', String(40), server_default=DefaultClause(default)))
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assert_(default in str(self.table.c.data.server_default.arg))
|
||||||
|
|
||||||
|
# Drop/remove default
|
||||||
|
self.table.c.data.alter(server_default=None)
|
||||||
|
self.assertEqual(self.table.c.data.server_default, None)
|
||||||
|
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
# server_default isn't necessarily None for Oracle
|
||||||
|
#self.assert_(self.table.c.data.server_default is None,self.table.c.data.server_default)
|
||||||
|
self.engine.execute(self.table.insert(), id=11)
|
||||||
|
row = self.table.select(self.table.c.id == 11, autocommit=True).execute().fetchone()
|
||||||
|
self.assert_(row['data'] is None, row['data'])
|
||||||
|
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported='firebird')
|
||||||
|
def test_null(self):
|
||||||
|
"""Can change a column's null constraint"""
|
||||||
|
self.assertEquals(self.table.c.data.nullable, True)
|
||||||
|
|
||||||
|
# Column object
|
||||||
|
self.table.c.data.alter(Column('data', String(40), nullable=False))
|
||||||
|
self.table.nullable = None
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assertEquals(self.table.c.data.nullable, False)
|
||||||
|
|
||||||
|
# Just the new status
|
||||||
|
self.table.c.data.alter(nullable=True)
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
self.assertEquals(self.table.c.data.nullable, True)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_alter_metadata(self):
|
||||||
|
"""Test if alter_metadata is respected"""
|
||||||
|
|
||||||
|
self.table.c.data.alter(Column('data', String(100)))
|
||||||
|
|
||||||
|
self.assert_(isinstance(self.table.c.data.type, String))
|
||||||
|
self.assertEqual(self.table.c.data.type.length, 100)
|
||||||
|
|
||||||
|
# nothing should change
|
||||||
|
self.table.c.data.alter(Column('data', String(200)), alter_metadata=False)
|
||||||
|
self.assert_(isinstance(self.table.c.data.type, String))
|
||||||
|
self.assertEqual(self.table.c.data.type.length, 100)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_alter_returns_delta(self):
|
||||||
|
"""Test if alter constructs return delta"""
|
||||||
|
|
||||||
|
delta = self.table.c.data.alter(Column('data', String(100)))
|
||||||
|
self.assert_('type' in delta)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_alter_all(self):
|
||||||
|
"""Tests all alter changes at one time"""
|
||||||
|
# test for each db separately
|
||||||
|
# since currently some dont support everything
|
||||||
|
|
||||||
|
# test pre settings
|
||||||
|
self.assertEqual(self.table.c.data.nullable, True)
|
||||||
|
self.assertEqual(self.table.c.data.server_default.arg, 'tluafed')
|
||||||
|
self.assertEqual(self.table.c.data.name, 'data')
|
||||||
|
self.assertTrue(isinstance(self.table.c.data.type, String))
|
||||||
|
self.assertTrue(self.table.c.data.type.length, 40)
|
||||||
|
|
||||||
|
kw = dict(nullable=False,
|
||||||
|
server_default='foobar',
|
||||||
|
name='data_new',
|
||||||
|
type=String(50),
|
||||||
|
alter_metadata=True)
|
||||||
|
if self.engine.name == 'firebird':
|
||||||
|
del kw['nullable']
|
||||||
|
self.table.c.data.alter(**kw)
|
||||||
|
|
||||||
|
# test altered objects
|
||||||
|
self.assertEqual(self.table.c.data.server_default.arg, 'foobar')
|
||||||
|
if not self.engine.name == 'firebird':
|
||||||
|
self.assertEqual(self.table.c.data.nullable, False)
|
||||||
|
self.assertEqual(self.table.c.data.name, 'data_new')
|
||||||
|
self.assertEqual(self.table.c.data.type.length, 50)
|
||||||
|
|
||||||
|
self.refresh_table(self.table.name)
|
||||||
|
|
||||||
|
# test post settings
|
||||||
|
if not self.engine.name == 'firebird':
|
||||||
|
self.assertEqual(self.table.c.data_new.nullable, False)
|
||||||
|
self.assertEqual(self.table.c.data_new.name, 'data_new')
|
||||||
|
self.assertTrue(isinstance(self.table.c.data_new.type, String))
|
||||||
|
self.assertTrue(self.table.c.data_new.type.length, 50)
|
||||||
|
|
||||||
|
# insert data and assert default
|
||||||
|
self.table.insert(values={'id': 10}).execute()
|
||||||
|
row = self.table.select(autocommit=True).execute().fetchone()
|
||||||
|
self.assertEqual(u'foobar', row['data_new'])
|
||||||
|
|
||||||
|
|
||||||
|
class TestColumnDelta(fixture.DB):
|
||||||
|
"""Tests ColumnDelta class"""
|
||||||
|
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
table_name = 'tmp_coldelta'
|
||||||
|
table_int = 0
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
super(TestColumnDelta, self)._setup(url)
|
||||||
|
self.meta = MetaData()
|
||||||
|
self.table = Table(self.table_name, self.meta,
|
||||||
|
Column('ids', String(10)),
|
||||||
|
)
|
||||||
|
self.meta.bind = self.engine
|
||||||
|
if self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
self.table.create()
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
if self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
self.meta.clear()
|
||||||
|
super(TestColumnDelta,self)._teardown()
|
||||||
|
|
||||||
|
def mkcol(self, name='id', type=String, *p, **k):
|
||||||
|
return Column(name, type, *p, **k)
|
||||||
|
|
||||||
|
def verify(self, expected, original, *p, **k):
|
||||||
|
self.delta = ColumnDelta(original, *p, **k)
|
||||||
|
result = self.delta.keys()
|
||||||
|
result.sort()
|
||||||
|
self.assertEquals(expected, result)
|
||||||
|
return self.delta
|
||||||
|
|
||||||
|
def test_deltas_two_columns(self):
|
||||||
|
"""Testing ColumnDelta with two columns"""
|
||||||
|
col_orig = self.mkcol(primary_key=True)
|
||||||
|
col_new = self.mkcol(name='ids', primary_key=True)
|
||||||
|
self.verify([], col_orig, col_orig)
|
||||||
|
self.verify(['name'], col_orig, col_orig, 'ids')
|
||||||
|
self.verify(['name'], col_orig, col_orig, name='ids')
|
||||||
|
self.verify(['name'], col_orig, col_new)
|
||||||
|
self.verify(['name', 'type'], col_orig, col_new, type=String)
|
||||||
|
|
||||||
|
# Type comparisons
|
||||||
|
self.verify([], self.mkcol(type=String), self.mkcol(type=String))
|
||||||
|
self.verify(['type'], self.mkcol(type=String), self.mkcol(type=Integer))
|
||||||
|
self.verify(['type'], self.mkcol(type=String), self.mkcol(type=String(42)))
|
||||||
|
self.verify([], self.mkcol(type=String(42)), self.mkcol(type=String(42)))
|
||||||
|
self.verify(['type'], self.mkcol(type=String(24)), self.mkcol(type=String(42)))
|
||||||
|
self.verify(['type'], self.mkcol(type=String(24)), self.mkcol(type=Text(24)))
|
||||||
|
|
||||||
|
# Other comparisons
|
||||||
|
self.verify(['primary_key'], self.mkcol(nullable=False), self.mkcol(primary_key=True))
|
||||||
|
|
||||||
|
# PK implies nullable=False
|
||||||
|
self.verify(['nullable', 'primary_key'], self.mkcol(nullable=True), self.mkcol(primary_key=True))
|
||||||
|
self.verify([], self.mkcol(primary_key=True), self.mkcol(primary_key=True))
|
||||||
|
self.verify(['nullable'], self.mkcol(nullable=True), self.mkcol(nullable=False))
|
||||||
|
self.verify([], self.mkcol(nullable=True), self.mkcol(nullable=True))
|
||||||
|
self.verify([], self.mkcol(server_default=None), self.mkcol(server_default=None))
|
||||||
|
self.verify([], self.mkcol(server_default='42'), self.mkcol(server_default='42'))
|
||||||
|
|
||||||
|
# test server default
|
||||||
|
delta = self.verify(['server_default'], self.mkcol(), self.mkcol('id', String, DefaultClause('foobar')))
|
||||||
|
self.assertEqual(delta['server_default'].arg, 'foobar')
|
||||||
|
|
||||||
|
self.verify([], self.mkcol(server_default='foobar'), self.mkcol('id', String, DefaultClause('foobar')))
|
||||||
|
self.verify(['type'], self.mkcol(server_default='foobar'), self.mkcol('id', Text, DefaultClause('foobar')))
|
||||||
|
|
||||||
|
# test alter_metadata
|
||||||
|
col = self.mkcol(server_default='foobar')
|
||||||
|
self.verify(['type'], col, self.mkcol('id', Text, DefaultClause('foobar')), alter_metadata=True)
|
||||||
|
self.assert_(isinstance(col.type, Text))
|
||||||
|
|
||||||
|
col = self.mkcol()
|
||||||
|
self.verify(['name', 'server_default', 'type'], col, self.mkcol('beep', Text, DefaultClause('foobar')), alter_metadata=True)
|
||||||
|
self.assert_(isinstance(col.type, Text))
|
||||||
|
self.assertEqual(col.name, 'beep')
|
||||||
|
self.assertEqual(col.server_default.arg, 'foobar')
|
||||||
|
|
||||||
|
col = self.mkcol()
|
||||||
|
self.verify(['name', 'server_default', 'type'], col, self.mkcol('beep', Text, DefaultClause('foobar')), alter_metadata=False)
|
||||||
|
self.assertFalse(isinstance(col.type, Text))
|
||||||
|
self.assertNotEqual(col.name, 'beep')
|
||||||
|
self.assertFalse(col.server_default)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_deltas_zero_columns(self):
|
||||||
|
"""Testing ColumnDelta with zero columns"""
|
||||||
|
|
||||||
|
self.verify(['name'], 'ids', table=self.table, name='hey')
|
||||||
|
|
||||||
|
# test reflection
|
||||||
|
self.verify(['type'], 'ids', table=self.table.name, type=String(80), engine=self.engine)
|
||||||
|
self.verify(['type'], 'ids', table=self.table.name, type=String(80), metadata=self.meta)
|
||||||
|
|
||||||
|
# check if alter_metadata is respected
|
||||||
|
self.meta.clear()
|
||||||
|
delta = self.verify(['type'], 'ids', table=self.table.name, type=String(80), alter_metadata=True, metadata=self.meta)
|
||||||
|
self.assert_(self.table.name in self.meta)
|
||||||
|
self.assertEqual(delta.result_column.type.length, 80)
|
||||||
|
self.assertEqual(self.meta.tables.get(self.table.name).c.ids.type.length, 80)
|
||||||
|
|
||||||
|
self.meta.clear()
|
||||||
|
self.verify(['type'], 'ids', table=self.table.name, type=String(80), alter_metadata=False, engine=self.engine)
|
||||||
|
self.assert_(self.table.name not in self.meta)
|
||||||
|
|
||||||
|
self.meta.clear()
|
||||||
|
self.verify(['type'], 'ids', table=self.table.name, type=String(80), alter_metadata=False, metadata=self.meta)
|
||||||
|
self.assert_(self.table.name not in self.meta)
|
||||||
|
|
||||||
|
# test defaults
|
||||||
|
self.meta.clear()
|
||||||
|
self.verify(['server_default'], 'ids', table=self.table.name, server_default='foobar', alter_metadata=True, metadata=self.meta)
|
||||||
|
self.meta.tables.get(self.table.name).c.ids.server_default.arg == 'foobar'
|
||||||
|
|
||||||
|
# test missing parameters
|
||||||
|
self.assertRaises(ValueError, ColumnDelta, table=self.table.name)
|
||||||
|
self.assertRaises(ValueError, ColumnDelta, 'ids', table=self.table.name, alter_metadata=True)
|
||||||
|
self.assertRaises(ValueError, ColumnDelta, 'ids', table=self.table.name, alter_metadata=False)
|
||||||
|
|
||||||
|
def test_deltas_one_column(self):
|
||||||
|
"""Testing ColumnDelta with one column"""
|
||||||
|
col_orig = self.mkcol(primary_key=True)
|
||||||
|
|
||||||
|
self.verify([], col_orig)
|
||||||
|
self.verify(['name'], col_orig, 'ids')
|
||||||
|
# Parameters are always executed, even if they're 'unchanged'
|
||||||
|
# (We can't assume given column is up-to-date)
|
||||||
|
self.verify(['name', 'primary_key', 'type'], col_orig, 'id', Integer, primary_key=True)
|
||||||
|
self.verify(['name', 'primary_key', 'type'], col_orig, name='id', type=Integer, primary_key=True)
|
||||||
|
|
||||||
|
# Change name, given an up-to-date definition and the current name
|
||||||
|
delta = self.verify(['name'], col_orig, name='blah')
|
||||||
|
self.assertEquals(delta.get('name'), 'blah')
|
||||||
|
self.assertEquals(delta.current_name, 'id')
|
||||||
|
|
||||||
|
# check if alter_metadata is respected
|
||||||
|
col_orig = self.mkcol(primary_key=True)
|
||||||
|
self.verify(['name', 'type'], col_orig, name='id12', type=Text, alter_metadata=True)
|
||||||
|
self.assert_(isinstance(col_orig.type, Text))
|
||||||
|
self.assertEqual(col_orig.name, 'id12')
|
||||||
|
|
||||||
|
col_orig = self.mkcol(primary_key=True)
|
||||||
|
self.verify(['name', 'type'], col_orig, name='id12', type=Text, alter_metadata=False)
|
||||||
|
self.assert_(isinstance(col_orig.type, String))
|
||||||
|
self.assertEqual(col_orig.name, 'id')
|
||||||
|
|
||||||
|
# test server default
|
||||||
|
col_orig = self.mkcol(primary_key=True)
|
||||||
|
delta = self.verify(['server_default'], col_orig, DefaultClause('foobar'))
|
||||||
|
self.assertEqual(delta['server_default'].arg, 'foobar')
|
||||||
|
|
||||||
|
delta = self.verify(['server_default'], col_orig, server_default=DefaultClause('foobar'))
|
||||||
|
self.assertEqual(delta['server_default'].arg, 'foobar')
|
||||||
|
|
||||||
|
# no change
|
||||||
|
col_orig = self.mkcol(server_default=DefaultClause('foobar'))
|
||||||
|
delta = self.verify(['type'], col_orig, DefaultClause('foobar'), type=PickleType)
|
||||||
|
self.assert_(isinstance(delta.result_column.type, PickleType))
|
||||||
|
|
||||||
|
# TODO: test server on update
|
||||||
|
# TODO: test bind metadata
|
||||||
278
test/changeset/test_constraint.py
Normal file
278
test/changeset/test_constraint.py
Normal file
@ -0,0 +1,278 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
from sqlalchemy.util import *
|
||||||
|
from sqlalchemy.exc import *
|
||||||
|
|
||||||
|
from migrate.changeset import *
|
||||||
|
from migrate.changeset.exceptions import *
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class CommonTestConstraint(fixture.DB):
|
||||||
|
"""helper functions to test constraints.
|
||||||
|
|
||||||
|
we just create a fresh new table and make sure everything is
|
||||||
|
as required.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
super(CommonTestConstraint, self)._setup(url)
|
||||||
|
self._create_table()
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
if hasattr(self, 'table') and self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
super(CommonTestConstraint, self)._teardown()
|
||||||
|
|
||||||
|
def _create_table(self):
|
||||||
|
self._connect(self.url)
|
||||||
|
self.meta = MetaData(self.engine)
|
||||||
|
self.tablename = 'mytable'
|
||||||
|
self.table = Table(self.tablename, self.meta,
|
||||||
|
Column('id', Integer, nullable=False),
|
||||||
|
Column('fkey', Integer, nullable=False),
|
||||||
|
mysql_engine='InnoDB')
|
||||||
|
if self.engine.has_table(self.table.name):
|
||||||
|
self.table.drop()
|
||||||
|
self.table.create()
|
||||||
|
|
||||||
|
# make sure we start at zero
|
||||||
|
self.assertEquals(len(self.table.primary_key), 0)
|
||||||
|
self.assert_(isinstance(self.table.primary_key,
|
||||||
|
schema.PrimaryKeyConstraint), self.table.primary_key.__class__)
|
||||||
|
|
||||||
|
|
||||||
|
class TestConstraint(CommonTestConstraint):
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
|
||||||
|
def _define_pk(self, *cols):
|
||||||
|
# Add a pk by creating a PK constraint
|
||||||
|
if (self.engine.name in ('oracle', 'firebird')):
|
||||||
|
# Can't drop Oracle PKs without an explicit name
|
||||||
|
pk = PrimaryKeyConstraint(table=self.table, name='temp_pk_key', *cols)
|
||||||
|
else:
|
||||||
|
pk = PrimaryKeyConstraint(table=self.table, *cols)
|
||||||
|
self.assertEquals(list(pk.columns), list(cols))
|
||||||
|
pk.create()
|
||||||
|
self.refresh_table()
|
||||||
|
if not self.url.startswith('sqlite'):
|
||||||
|
self.assertEquals(list(self.table.primary_key), list(cols))
|
||||||
|
|
||||||
|
# Drop the PK constraint
|
||||||
|
#if (self.engine.name in ('oracle', 'firebird')):
|
||||||
|
# # Apparently Oracle PK names aren't introspected
|
||||||
|
# pk.name = self.table.primary_key.name
|
||||||
|
pk.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.assertEquals(len(self.table.primary_key), 0)
|
||||||
|
self.assert_(isinstance(self.table.primary_key, schema.PrimaryKeyConstraint))
|
||||||
|
return pk
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported='sqlite')
|
||||||
|
def test_define_fk(self):
|
||||||
|
"""FK constraints can be defined, created, and dropped"""
|
||||||
|
# FK target must be unique
|
||||||
|
pk = PrimaryKeyConstraint(self.table.c.id, table=self.table, name="pkid")
|
||||||
|
pk.create()
|
||||||
|
|
||||||
|
# Add a FK by creating a FK constraint
|
||||||
|
self.assertEquals(self.table.c.fkey.foreign_keys._list, [])
|
||||||
|
fk = ForeignKeyConstraint([self.table.c.fkey],
|
||||||
|
[self.table.c.id],
|
||||||
|
name="fk_id_fkey",
|
||||||
|
ondelete="CASCADE")
|
||||||
|
self.assert_(self.table.c.fkey.foreign_keys._list is not [])
|
||||||
|
self.assertEquals(list(fk.columns), [self.table.c.fkey])
|
||||||
|
self.assertEquals([e.column for e in fk.elements], [self.table.c.id])
|
||||||
|
self.assertEquals(list(fk.referenced), [self.table.c.id])
|
||||||
|
|
||||||
|
if self.url.startswith('mysql'):
|
||||||
|
# MySQL FKs need an index
|
||||||
|
index = Index('index_name', self.table.c.fkey)
|
||||||
|
index.create()
|
||||||
|
fk.create()
|
||||||
|
|
||||||
|
# test for ondelete/onupdate
|
||||||
|
fkey = self.table.c.fkey.foreign_keys._list[0]
|
||||||
|
self.assertEquals(fkey.ondelete, "CASCADE")
|
||||||
|
# TODO: test on real db if it was set
|
||||||
|
|
||||||
|
self.refresh_table()
|
||||||
|
self.assert_(self.table.c.fkey.foreign_keys._list is not [])
|
||||||
|
|
||||||
|
fk.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.assertEquals(self.table.c.fkey.foreign_keys._list, [])
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_define_pk(self):
|
||||||
|
"""PK constraints can be defined, created, and dropped"""
|
||||||
|
self._define_pk(self.table.c.fkey)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_define_pk_multi(self):
|
||||||
|
"""Multicolumn PK constraints can be defined, created, and dropped"""
|
||||||
|
self._define_pk(self.table.c.id, self.table.c.fkey)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_drop_cascade(self):
|
||||||
|
"""Drop constraint cascaded"""
|
||||||
|
|
||||||
|
pk = PrimaryKeyConstraint('fkey', table=self.table, name="id_pkey")
|
||||||
|
pk.create()
|
||||||
|
self.refresh_table()
|
||||||
|
|
||||||
|
# Drop the PK constraint forcing cascade
|
||||||
|
try:
|
||||||
|
pk.drop(cascade=True)
|
||||||
|
except NotSupportedError:
|
||||||
|
if self.engine.name == 'firebird':
|
||||||
|
pass
|
||||||
|
|
||||||
|
# TODO: add real assertion if it was added
|
||||||
|
|
||||||
|
@fixture.usedb(supported=['mysql'])
|
||||||
|
def test_fail_mysql_check_constraints(self):
|
||||||
|
"""Check constraints raise NotSupported for mysql on drop"""
|
||||||
|
cons = CheckConstraint('id > 3', name="id_check", table=self.table)
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
|
||||||
|
try:
|
||||||
|
cons.drop()
|
||||||
|
except NotSupportedError:
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported=['sqlite', 'mysql'])
|
||||||
|
def test_named_check_constraints(self):
|
||||||
|
"""Check constraints can be defined, created, and dropped"""
|
||||||
|
self.assertRaises(InvalidConstraintError, CheckConstraint, 'id > 3')
|
||||||
|
cons = CheckConstraint('id > 3', name="id_check", table=self.table)
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
|
||||||
|
self.table.insert(values={'id': 4, 'fkey': 1}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'id': 1, 'fkey': 1}).execute()
|
||||||
|
except (IntegrityError, ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
# Remove the name, drop the constraint; it should succeed
|
||||||
|
cons.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.table.insert(values={'id': 2, 'fkey': 2}).execute()
|
||||||
|
self.table.insert(values={'id': 1, 'fkey': 2}).execute()
|
||||||
|
|
||||||
|
|
||||||
|
class TestAutoname(CommonTestConstraint):
|
||||||
|
"""Every method tests for a type of constraint wether it can autoname
|
||||||
|
itself and if you can pass object instance and names to classes.
|
||||||
|
"""
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported=['oracle', 'firebird'])
|
||||||
|
def test_autoname_pk(self):
|
||||||
|
"""PrimaryKeyConstraints can guess their name if None is given"""
|
||||||
|
# Don't supply a name; it should create one
|
||||||
|
cons = PrimaryKeyConstraint(self.table.c.id)
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
if not self.url.startswith('sqlite'):
|
||||||
|
# TODO: test for index for sqlite
|
||||||
|
self.assertEquals(list(cons.columns), list(self.table.primary_key))
|
||||||
|
|
||||||
|
# Remove the name, drop the constraint; it should succeed
|
||||||
|
cons.name = None
|
||||||
|
cons.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.assertEquals(list(), list(self.table.primary_key))
|
||||||
|
|
||||||
|
# test string names
|
||||||
|
cons = PrimaryKeyConstraint('id', table=self.table)
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
if not self.url.startswith('sqlite'):
|
||||||
|
# TODO: test for index for sqlite
|
||||||
|
self.assertEquals(list(cons.columns), list(self.table.primary_key))
|
||||||
|
cons.name = None
|
||||||
|
cons.drop()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported=['oracle', 'sqlite', 'firebird'])
|
||||||
|
def test_autoname_fk(self):
|
||||||
|
"""ForeignKeyConstraints can guess their name if None is given"""
|
||||||
|
cons = PrimaryKeyConstraint(self.table.c.id)
|
||||||
|
cons.create()
|
||||||
|
|
||||||
|
cons = ForeignKeyConstraint([self.table.c.fkey], [self.table.c.id])
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
self.table.c.fkey.foreign_keys[0].column is self.table.c.id
|
||||||
|
|
||||||
|
# Remove the name, drop the constraint; it should succeed
|
||||||
|
cons.name = None
|
||||||
|
cons.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.assertEquals(self.table.c.fkey.foreign_keys._list, list())
|
||||||
|
|
||||||
|
# test string names
|
||||||
|
cons = ForeignKeyConstraint(['fkey'], ['%s.id' % self.tablename], table=self.table)
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
self.table.c.fkey.foreign_keys[0].column is self.table.c.id
|
||||||
|
|
||||||
|
# Remove the name, drop the constraint; it should succeed
|
||||||
|
cons.name = None
|
||||||
|
cons.drop()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported=['oracle', 'sqlite', 'mysql'])
|
||||||
|
def test_autoname_check(self):
|
||||||
|
"""CheckConstraints can guess their name if None is given"""
|
||||||
|
cons = CheckConstraint('id > 3', columns=[self.table.c.id])
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
|
||||||
|
if not self.engine.name == 'mysql':
|
||||||
|
self.table.insert(values={'id': 4, 'fkey': 1}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'id': 1, 'fkey': 2}).execute()
|
||||||
|
except (IntegrityError, ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
# Remove the name, drop the constraint; it should succeed
|
||||||
|
cons.name = None
|
||||||
|
cons.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.table.insert(values={'id': 2, 'fkey': 2}).execute()
|
||||||
|
self.table.insert(values={'id': 1, 'fkey': 3}).execute()
|
||||||
|
|
||||||
|
@fixture.usedb(not_supported=['oracle', 'sqlite'])
|
||||||
|
def test_autoname_unique(self):
|
||||||
|
"""UniqueConstraints can guess their name if None is given"""
|
||||||
|
cons = UniqueConstraint(self.table.c.fkey)
|
||||||
|
cons.create()
|
||||||
|
self.refresh_table()
|
||||||
|
|
||||||
|
self.table.insert(values={'fkey': 4, 'id': 1}).execute()
|
||||||
|
try:
|
||||||
|
self.table.insert(values={'fkey': 4, 'id': 2}).execute()
|
||||||
|
except (sqlalchemy.exc.IntegrityError,
|
||||||
|
sqlalchemy.exc.ProgrammingError):
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
self.fail()
|
||||||
|
|
||||||
|
# Remove the name, drop the constraint; it should succeed
|
||||||
|
cons.name = None
|
||||||
|
cons.drop()
|
||||||
|
self.refresh_table()
|
||||||
|
self.table.insert(values={'fkey': 4, 'id': 2}).execute()
|
||||||
|
self.table.insert(values={'fkey': 4, 'id': 1}).execute()
|
||||||
66
test/fixture/__init__.py
Normal file
66
test/fixture/__init__.py
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
# Append test method name,etc. to descriptions automatically.
|
||||||
|
# Yes, this is ugly, but it's the simplest way...
|
||||||
|
def getDescription(self, test):
|
||||||
|
ret = str(test)
|
||||||
|
if self.descriptions:
|
||||||
|
return test.shortDescription() or ret
|
||||||
|
return ret
|
||||||
|
unittest._TextTestResult.getDescription = getDescription
|
||||||
|
|
||||||
|
|
||||||
|
class Result(unittest._TextTestResult):
|
||||||
|
# test description may be changed as we go; store the description at
|
||||||
|
# exception-time and print later
|
||||||
|
def __init__(self,*p,**k):
|
||||||
|
super(Result,self).__init__(*p,**k)
|
||||||
|
self.desc=dict()
|
||||||
|
def _addError(self,test,err,errs):
|
||||||
|
test,err=errs.pop()
|
||||||
|
errdata=(test,err,self.getDescription(test))
|
||||||
|
errs.append(errdata)
|
||||||
|
|
||||||
|
def addFailure(self,test,err):
|
||||||
|
super(Result,self).addFailure(test,err)
|
||||||
|
self._addError(test,err,self.failures)
|
||||||
|
def addError(self,test,err):
|
||||||
|
super(Result,self).addError(test,err)
|
||||||
|
self._addError(test,err,self.errors)
|
||||||
|
def printErrorList(self, flavour, errors):
|
||||||
|
# Copied from unittest.py
|
||||||
|
#for test, err in errors:
|
||||||
|
for errdata in errors:
|
||||||
|
test,err,desc=errdata
|
||||||
|
self.stream.writeln(self.separator1)
|
||||||
|
#self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
|
||||||
|
self.stream.writeln("%s: %s" % (flavour,desc or self.getDescription(test)))
|
||||||
|
self.stream.writeln(self.separator2)
|
||||||
|
self.stream.writeln("%s" % err)
|
||||||
|
|
||||||
|
class Runner(unittest.TextTestRunner):
|
||||||
|
def _makeResult(self):
|
||||||
|
return Result(self.stream,self.descriptions,self.verbosity)
|
||||||
|
|
||||||
|
def suite(imports):
|
||||||
|
return unittest.TestLoader().loadTestsFromNames(imports)
|
||||||
|
|
||||||
|
def main(imports=None):
|
||||||
|
if imports:
|
||||||
|
global suite
|
||||||
|
suite = suite(imports)
|
||||||
|
defaultTest='fixture.suite'
|
||||||
|
else:
|
||||||
|
defaultTest=None
|
||||||
|
return unittest.TestProgram(defaultTest=defaultTest,\
|
||||||
|
testRunner=Runner(verbosity=1))
|
||||||
|
|
||||||
|
from base import Base
|
||||||
|
from pathed import Pathed
|
||||||
|
from shell import Shell
|
||||||
|
from database import DB,usedb
|
||||||
35
test/fixture/base.py
Normal file
35
test/fixture/base.py
Normal file
@ -0,0 +1,35 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import re
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from nose.tools import raises, eq_
|
||||||
|
|
||||||
|
|
||||||
|
class Base(unittest.TestCase):
|
||||||
|
|
||||||
|
def setup_method(self,func=None):
|
||||||
|
self.setUp()
|
||||||
|
|
||||||
|
def teardown_method(self,func=None):
|
||||||
|
self.tearDown()
|
||||||
|
|
||||||
|
def assertEqualsIgnoreWhitespace(self, v1, v2):
|
||||||
|
"""Compares two strings that should be\
|
||||||
|
identical except for whitespace
|
||||||
|
"""
|
||||||
|
def strip_whitespace(s):
|
||||||
|
return re.sub(r'\s', '', s)
|
||||||
|
|
||||||
|
line1 = strip_whitespace(v1)
|
||||||
|
line2 = strip_whitespace(v2)
|
||||||
|
|
||||||
|
self.assertEquals(line1, line2, "%s != %s" % (v1, v2))
|
||||||
|
|
||||||
|
def ignoreErrors(self, func, *p,**k):
|
||||||
|
"""Call a function, ignoring any exceptions"""
|
||||||
|
try:
|
||||||
|
func(*p,**k)
|
||||||
|
except:
|
||||||
|
pass
|
||||||
144
test/fixture/database.py
Normal file
144
test/fixture/database.py
Normal file
@ -0,0 +1,144 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
from decorator import decorator
|
||||||
|
|
||||||
|
from sqlalchemy import create_engine, Table, MetaData
|
||||||
|
from sqlalchemy.orm import create_session
|
||||||
|
|
||||||
|
from test.fixture.base import Base
|
||||||
|
from test.fixture.pathed import Pathed
|
||||||
|
|
||||||
|
|
||||||
|
def readurls():
|
||||||
|
"""read URLs from config file return a list"""
|
||||||
|
filename = 'test_db.cfg'
|
||||||
|
ret = list()
|
||||||
|
# TODO: remove tmpfile since sqlite can store db in memory
|
||||||
|
tmpfile = Pathed.tmp()
|
||||||
|
fullpath = os.path.join(os.curdir, filename)
|
||||||
|
|
||||||
|
try:
|
||||||
|
fd = open(fullpath)
|
||||||
|
except IOError:
|
||||||
|
raise IOError("""You must specify the databases to use for testing!
|
||||||
|
Copy %(filename)s.tmpl to %(filename)s and edit your database URLs.""" % locals())
|
||||||
|
|
||||||
|
for line in fd:
|
||||||
|
if line.startswith('#'):
|
||||||
|
continue
|
||||||
|
line = line.replace('__tmp__', tmpfile).strip()
|
||||||
|
ret.append(line)
|
||||||
|
fd.close()
|
||||||
|
return ret
|
||||||
|
|
||||||
|
def is_supported(url, supported, not_supported):
|
||||||
|
db = url.split(':', 1)[0]
|
||||||
|
|
||||||
|
if supported is not None:
|
||||||
|
if isinstance(supported, basestring):
|
||||||
|
return supported == db
|
||||||
|
else:
|
||||||
|
return db in supported
|
||||||
|
elif not_supported is not None:
|
||||||
|
if isinstance(not_supported, basestring):
|
||||||
|
return not_supported != db
|
||||||
|
else:
|
||||||
|
return not (db in not_supported)
|
||||||
|
return True
|
||||||
|
|
||||||
|
# we make the engines global, which should make the tests run a bit faster
|
||||||
|
urls = readurls()
|
||||||
|
engines = dict([(url, create_engine(url, echo=True)) for url in urls])
|
||||||
|
|
||||||
|
|
||||||
|
def usedb(supported=None, not_supported=None):
|
||||||
|
"""Decorates tests to be run with a database connection
|
||||||
|
These tests are run once for each available database
|
||||||
|
|
||||||
|
@param supported: run tests for ONLY these databases
|
||||||
|
@param not_supported: run tests for all databases EXCEPT these
|
||||||
|
|
||||||
|
If both supported and not_supported are empty, all dbs are assumed
|
||||||
|
to be supported
|
||||||
|
"""
|
||||||
|
if supported is not None and not_supported is not None:
|
||||||
|
raise AssertionError("Can't specify both supported and not_supported in fixture.db()")
|
||||||
|
|
||||||
|
my_urls = [url for url in urls if is_supported(url, supported, not_supported)]
|
||||||
|
|
||||||
|
@decorator
|
||||||
|
def dec(f, self, *a, **kw):
|
||||||
|
for url in my_urls:
|
||||||
|
self._setup(url)
|
||||||
|
f(self, *a, **kw)
|
||||||
|
self._teardown()
|
||||||
|
return dec
|
||||||
|
|
||||||
|
|
||||||
|
class DB(Base):
|
||||||
|
# Constants: connection level
|
||||||
|
NONE = 0 # No connection; just set self.url
|
||||||
|
CONNECT = 1 # Connect; no transaction
|
||||||
|
TXN = 2 # Everything in a transaction
|
||||||
|
|
||||||
|
level = TXN
|
||||||
|
|
||||||
|
def _engineInfo(self, url=None):
|
||||||
|
if url is None:
|
||||||
|
url = self.url
|
||||||
|
return url
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
self._connect(url)
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
self._disconnect()
|
||||||
|
|
||||||
|
def _connect(self, url):
|
||||||
|
self.url = url
|
||||||
|
self.engine = engines[url]
|
||||||
|
self.meta = MetaData(bind=self.engine)
|
||||||
|
if self.level < self.CONNECT:
|
||||||
|
return
|
||||||
|
#self.conn = self.engine.connect()
|
||||||
|
self.session = create_session(bind=self.engine)
|
||||||
|
if self.level < self.TXN:
|
||||||
|
return
|
||||||
|
self.txn = self.session.begin()
|
||||||
|
|
||||||
|
#self.txn.add(self.engine)
|
||||||
|
|
||||||
|
def _disconnect(self):
|
||||||
|
if hasattr(self, 'txn'):
|
||||||
|
self.txn.rollback()
|
||||||
|
if hasattr(self, 'session'):
|
||||||
|
self.session.close()
|
||||||
|
#if hasattr(self,'conn'):
|
||||||
|
# self.conn.close()
|
||||||
|
|
||||||
|
def _supported(self, url):
|
||||||
|
db = url.split(':',1)[0]
|
||||||
|
func = getattr(self, self._TestCase__testMethodName)
|
||||||
|
if hasattr(func, 'supported'):
|
||||||
|
return db in func.supported
|
||||||
|
if hasattr(func, 'not_supported'):
|
||||||
|
return not (db in func.not_supported)
|
||||||
|
# Neither list assigned; assume all are supported
|
||||||
|
return True
|
||||||
|
def _not_supported(self, url):
|
||||||
|
return not self._supported(url)
|
||||||
|
|
||||||
|
def refresh_table(self, name=None):
|
||||||
|
"""Reload the table from the database
|
||||||
|
Assumes we're working with only a single table, self.table, and
|
||||||
|
metadata self.meta
|
||||||
|
|
||||||
|
Working w/ multiple tables is not possible, as tables can only be
|
||||||
|
reloaded with meta.clear()
|
||||||
|
"""
|
||||||
|
if name is None:
|
||||||
|
name = self.table.name
|
||||||
|
self.meta.clear()
|
||||||
|
self.table = Table(name, self.meta, autoload=True)
|
||||||
77
test/fixture/pathed.py
Normal file
77
test/fixture/pathed.py
Normal file
@ -0,0 +1,77 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import shutil
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
from test.fixture import base
|
||||||
|
|
||||||
|
|
||||||
|
class Pathed(base.Base):
|
||||||
|
# Temporary files
|
||||||
|
|
||||||
|
_tmpdir = tempfile.mkdtemp()
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super(Pathed, self).setUp()
|
||||||
|
self.temp_usable_dir = tempfile.mkdtemp()
|
||||||
|
sys.path.append(self.temp_usable_dir)
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
super(Pathed, self).tearDown()
|
||||||
|
try:
|
||||||
|
sys.path.remove(self.temp_usable_dir)
|
||||||
|
except:
|
||||||
|
pass # w00t?
|
||||||
|
Pathed.purge(self.temp_usable_dir)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _tmp(cls, prefix='', suffix=''):
|
||||||
|
"""Generate a temporary file name that doesn't exist
|
||||||
|
All filenames are generated inside a temporary directory created by
|
||||||
|
tempfile.mkdtemp(); only the creating user has access to this directory.
|
||||||
|
It should be secure to return a nonexistant temp filename in this
|
||||||
|
directory, unless the user is messing with their own files.
|
||||||
|
"""
|
||||||
|
file, ret = tempfile.mkstemp(suffix,prefix,cls._tmpdir)
|
||||||
|
os.close(file)
|
||||||
|
os.remove(ret)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def tmp(cls, *p, **k):
|
||||||
|
return cls._tmp(*p, **k)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def tmp_py(cls, *p, **k):
|
||||||
|
return cls._tmp(suffix='.py', *p, **k)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def tmp_sql(cls, *p, **k):
|
||||||
|
return cls._tmp(suffix='.sql', *p, **k)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def tmp_named(cls, name):
|
||||||
|
return os.path.join(cls._tmpdir, name)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def tmp_repos(cls, *p, **k):
|
||||||
|
return cls._tmp(*p, **k)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def purge(cls, path):
|
||||||
|
"""Removes this path if it exists, in preparation for tests
|
||||||
|
Careful - all tests should take place in /tmp.
|
||||||
|
We don't want to accidentally wipe stuff out...
|
||||||
|
"""
|
||||||
|
if os.path.exists(path):
|
||||||
|
if os.path.isdir(path):
|
||||||
|
shutil.rmtree(path)
|
||||||
|
else:
|
||||||
|
os.remove(path)
|
||||||
|
if path.endswith('.py'):
|
||||||
|
pyc = path + 'c'
|
||||||
|
if os.path.exists(pyc):
|
||||||
|
os.remove(pyc)
|
||||||
56
test/fixture/shell.py
Normal file
56
test/fixture/shell.py
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
|
||||||
|
from test.fixture.pathed import *
|
||||||
|
|
||||||
|
|
||||||
|
class Shell(Pathed):
|
||||||
|
"""Base class for command line tests"""
|
||||||
|
def execute(self, command, *p, **k):
|
||||||
|
"""Return the fd of a command; can get output (stdout/err) and exitcode"""
|
||||||
|
# We might be passed a file descriptor for some reason; if so, just return it
|
||||||
|
if isinstance(command, types.FileType):
|
||||||
|
return command
|
||||||
|
|
||||||
|
# Redirect stderr to stdout
|
||||||
|
# This is a bit of a hack, but I've not found a better way
|
||||||
|
py_path = os.environ.get('PYTHONPATH', '')
|
||||||
|
py_path_list = py_path.split(':')
|
||||||
|
py_path_list.append(os.path.abspath('.'))
|
||||||
|
os.environ['PYTHONPATH'] = ':'.join(py_path_list)
|
||||||
|
fd = os.popen(command + ' 2>&1')
|
||||||
|
|
||||||
|
if py_path:
|
||||||
|
py_path = os.environ['PYTHONPATH'] = py_path
|
||||||
|
else:
|
||||||
|
del os.environ['PYTHONPATH']
|
||||||
|
return fd
|
||||||
|
|
||||||
|
def output_and_exitcode(self, *p, **k):
|
||||||
|
fd=self.execute(*p, **k)
|
||||||
|
output = fd.read().strip()
|
||||||
|
exitcode = fd.close()
|
||||||
|
if k.pop('emit',False):
|
||||||
|
print output
|
||||||
|
return (output, exitcode)
|
||||||
|
|
||||||
|
def exitcode(self, *p, **k):
|
||||||
|
"""Execute a command and return its exit code
|
||||||
|
...without printing its output/errors
|
||||||
|
"""
|
||||||
|
ret = self.output_and_exitcode(*p, **k)
|
||||||
|
return ret[1]
|
||||||
|
|
||||||
|
def assertFailure(self, *p, **k):
|
||||||
|
output,exitcode = self.output_and_exitcode(*p, **k)
|
||||||
|
assert (exitcode), output
|
||||||
|
|
||||||
|
def assertSuccess(self, *p, **k):
|
||||||
|
output,exitcode = self.output_and_exitcode(*p, **k)
|
||||||
|
#self.assert_(not exitcode, output)
|
||||||
|
assert (not exitcode), output
|
||||||
0
test/integrated/__init__.py
Normal file
0
test/integrated/__init__.py
Normal file
16
test/integrated/test_docs.py
Normal file
16
test/integrated/test_docs.py
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
from test import fixture
|
||||||
|
import doctest
|
||||||
|
import os
|
||||||
|
|
||||||
|
# Collect tests for all handwritten docs: doc/*.rst
|
||||||
|
|
||||||
|
dir = ('..','..','docs')
|
||||||
|
absdir = (os.path.dirname(os.path.abspath(__file__)),)+dir
|
||||||
|
dirpath = os.path.join(*absdir)
|
||||||
|
files = [f for f in os.listdir(dirpath) if f.endswith('.rst')]
|
||||||
|
paths = [os.path.join(*(dir+(f,))) for f in files]
|
||||||
|
assert len(paths) > 0
|
||||||
|
suite = doctest.DocFileSuite(*paths)
|
||||||
|
|
||||||
|
def test_docs():
|
||||||
|
suite.debug()
|
||||||
0
test/versioning/__init__.py
Normal file
0
test/versioning/__init__.py
Normal file
21
test/versioning/test_cfgparse.py
Normal file
21
test/versioning/test_cfgparse.py
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
from test import fixture
|
||||||
|
from migrate.versioning import cfgparse
|
||||||
|
from migrate.versioning.repository import *
|
||||||
|
|
||||||
|
class TestConfigParser(fixture.Base):
|
||||||
|
def test_to_dict(self):
|
||||||
|
"""Correctly interpret config results as dictionaries"""
|
||||||
|
parser = cfgparse.Parser(dict(default_value=42))
|
||||||
|
self.assert_(len(parser.sections())==0)
|
||||||
|
parser.add_section('section')
|
||||||
|
parser.set('section','option','value')
|
||||||
|
self.assert_(parser.get('section','option')=='value')
|
||||||
|
self.assert_(parser.to_dict()['section']['option']=='value')
|
||||||
|
|
||||||
|
def test_table_config(self):
|
||||||
|
"""We should be able to specify the table to be used with a repository"""
|
||||||
|
default_text=Repository.prepare_config(template.get_repository(as_pkg=True,as_str=True),
|
||||||
|
Repository._config,'repository_name')
|
||||||
|
specified_text=Repository.prepare_config(template.get_repository(as_pkg=True,as_str=True),
|
||||||
|
Repository._config,'repository_name',version_table='_other_table')
|
||||||
|
self.assertNotEquals(default_text,specified_text)
|
||||||
11
test/versioning/test_database.py
Normal file
11
test/versioning/test_database.py
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
from sqlalchemy import select
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
class TestConnect(fixture.DB):
|
||||||
|
level=fixture.DB.TXN
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_connect(self):
|
||||||
|
"""Connect to the database successfully"""
|
||||||
|
# Connection is done in fixture.DB setup; make sure we can do stuff
|
||||||
|
select(['42'],bind=self.engine).execute()
|
||||||
13
test/versioning/test_genmodel.py
Normal file
13
test/versioning/test_genmodel.py
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from migrate.versioning.genmodel import *
|
||||||
|
from migrate.versioning.exceptions import *
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class TestModelGenerator(fixture.Pathed, fixture.DB):
|
||||||
|
level = fixture.DB.TXN
|
||||||
40
test/versioning/test_keyedinstance.py
Normal file
40
test/versioning/test_keyedinstance.py
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
from test import fixture
|
||||||
|
from migrate.versioning.util.keyedinstance import *
|
||||||
|
|
||||||
|
class TestKeydInstance(fixture.Base):
|
||||||
|
def test_unique(self):
|
||||||
|
"""UniqueInstance should produce unique object instances"""
|
||||||
|
class Uniq1(KeyedInstance):
|
||||||
|
@classmethod
|
||||||
|
def _key(cls,key):
|
||||||
|
return str(key)
|
||||||
|
def __init__(self,value):
|
||||||
|
self.value=value
|
||||||
|
class Uniq2(KeyedInstance):
|
||||||
|
@classmethod
|
||||||
|
def _key(cls,key):
|
||||||
|
return str(key)
|
||||||
|
def __init__(self,value):
|
||||||
|
self.value=value
|
||||||
|
|
||||||
|
a10 = Uniq1('a')
|
||||||
|
|
||||||
|
# Different key: different instance
|
||||||
|
b10 = Uniq1('b')
|
||||||
|
self.assert_(a10 is not b10)
|
||||||
|
|
||||||
|
# Different class: different instance
|
||||||
|
a20 = Uniq2('a')
|
||||||
|
self.assert_(a10 is not a20)
|
||||||
|
|
||||||
|
# Same key/class: same instance
|
||||||
|
a11 = Uniq1('a')
|
||||||
|
self.assert_(a10 is a11)
|
||||||
|
|
||||||
|
# __init__ is called
|
||||||
|
self.assertEquals(a10.value,'a')
|
||||||
|
|
||||||
|
# clear() causes us to forget all existing instances
|
||||||
|
Uniq1.clear()
|
||||||
|
a12 = Uniq1('a')
|
||||||
|
self.assert_(a10 is not a12)
|
||||||
51
test/versioning/test_pathed.py
Normal file
51
test/versioning/test_pathed.py
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
from test import fixture
|
||||||
|
from migrate.versioning.pathed import *
|
||||||
|
|
||||||
|
class TestPathed(fixture.Base):
|
||||||
|
def test_parent_path(self):
|
||||||
|
"""Default parent_path should behave correctly"""
|
||||||
|
filepath='/fgsfds/moot.py'
|
||||||
|
dirpath='/fgsfds/moot'
|
||||||
|
sdirpath='/fgsfds/moot/'
|
||||||
|
|
||||||
|
result='/fgsfds'
|
||||||
|
self.assert_(result==Pathed._parent_path(filepath))
|
||||||
|
self.assert_(result==Pathed._parent_path(dirpath))
|
||||||
|
self.assert_(result==Pathed._parent_path(sdirpath))
|
||||||
|
|
||||||
|
def test_new(self):
|
||||||
|
"""Pathed(path) shouldn't create duplicate objects of the same path"""
|
||||||
|
path='/fgsfds'
|
||||||
|
class Test(Pathed):
|
||||||
|
attr=None
|
||||||
|
o1=Test(path)
|
||||||
|
o2=Test(path)
|
||||||
|
self.assert_(isinstance(o1,Test))
|
||||||
|
self.assert_(o1.path==path)
|
||||||
|
self.assert_(o1 is o2)
|
||||||
|
o1.attr='herring'
|
||||||
|
self.assert_(o2.attr=='herring')
|
||||||
|
o2.attr='shrubbery'
|
||||||
|
self.assert_(o1.attr=='shrubbery')
|
||||||
|
|
||||||
|
def test_parent(self):
|
||||||
|
"""Parents should be fetched correctly"""
|
||||||
|
class Parent(Pathed):
|
||||||
|
parent=None
|
||||||
|
children=0
|
||||||
|
def _init_child(self,child,path):
|
||||||
|
"""Keep a tally of children.
|
||||||
|
(A real class might do something more interesting here)
|
||||||
|
"""
|
||||||
|
self.__class__.children+=1
|
||||||
|
|
||||||
|
class Child(Pathed):
|
||||||
|
parent=Parent
|
||||||
|
|
||||||
|
path='/fgsfds/moot.py'
|
||||||
|
parent_path='/fgsfds'
|
||||||
|
object=Child(path)
|
||||||
|
self.assert_(isinstance(object,Child))
|
||||||
|
self.assert_(isinstance(object.parent,Parent))
|
||||||
|
self.assert_(object.path==path)
|
||||||
|
self.assert_(object.parent.path==parent_path)
|
||||||
196
test/versioning/test_repository.py
Normal file
196
test/versioning/test_repository.py
Normal file
@ -0,0 +1,196 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions
|
||||||
|
from migrate.versioning.repository import *
|
||||||
|
from migrate.versioning.script import *
|
||||||
|
from nose.tools import raises
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class TestRepository(fixture.Pathed):
|
||||||
|
def test_create(self):
|
||||||
|
"""Repositories are created successfully"""
|
||||||
|
path = self.tmp_repos()
|
||||||
|
name = 'repository_name'
|
||||||
|
|
||||||
|
# Creating a repository that doesn't exist should succeed
|
||||||
|
repo = Repository.create(path, name)
|
||||||
|
config_path = repo.config.path
|
||||||
|
manage_path = os.path.join(repo.path, 'manage.py')
|
||||||
|
self.assert_(repo)
|
||||||
|
|
||||||
|
# Files should actually be created
|
||||||
|
self.assert_(os.path.exists(path))
|
||||||
|
self.assert_(os.path.exists(config_path))
|
||||||
|
self.assert_(os.path.exists(manage_path))
|
||||||
|
|
||||||
|
# Can't create it again: it already exists
|
||||||
|
self.assertRaises(exceptions.PathFoundError, Repository.create, path, name)
|
||||||
|
return path
|
||||||
|
|
||||||
|
def test_load(self):
|
||||||
|
"""We should be able to load information about an existing repository"""
|
||||||
|
# Create a repository to load
|
||||||
|
path = self.test_create()
|
||||||
|
repos = Repository(path)
|
||||||
|
|
||||||
|
self.assert_(repos)
|
||||||
|
self.assert_(repos.config)
|
||||||
|
self.assert_(repos.config.get('db_settings', 'version_table'))
|
||||||
|
|
||||||
|
# version_table's default isn't none
|
||||||
|
self.assertNotEquals(repos.config.get('db_settings', 'version_table'), 'None')
|
||||||
|
|
||||||
|
def test_load_notfound(self):
|
||||||
|
"""Nonexistant repositories shouldn't be loaded"""
|
||||||
|
path = self.tmp_repos()
|
||||||
|
self.assert_(not os.path.exists(path))
|
||||||
|
self.assertRaises(exceptions.InvalidRepositoryError, Repository, path)
|
||||||
|
|
||||||
|
def test_load_invalid(self):
|
||||||
|
"""Invalid repos shouldn't be loaded"""
|
||||||
|
# Here, invalid=empty directory. There may be other conditions too,
|
||||||
|
# but we shouldn't need to test all of them
|
||||||
|
path = self.tmp_repos()
|
||||||
|
os.mkdir(path)
|
||||||
|
self.assertRaises(exceptions.InvalidRepositoryError, Repository, path)
|
||||||
|
|
||||||
|
|
||||||
|
class TestVersionedRepository(fixture.Pathed):
|
||||||
|
"""Tests on an existing repository with a single python script"""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super(TestVersionedRepository, self).setUp()
|
||||||
|
Repository.clear()
|
||||||
|
self.path_repos = self.tmp_repos()
|
||||||
|
Repository.create(self.path_repos, 'repository_name')
|
||||||
|
|
||||||
|
def test_version(self):
|
||||||
|
"""We should correctly detect the version of a repository"""
|
||||||
|
repos = Repository(self.path_repos)
|
||||||
|
|
||||||
|
# Get latest version, or detect if a specified version exists
|
||||||
|
self.assertEquals(repos.latest, 0)
|
||||||
|
# repos.latest isn't an integer, but a VerNum
|
||||||
|
# (so we can't just assume the following tests are correct)
|
||||||
|
self.assert_(repos.latest >= 0)
|
||||||
|
self.assert_(repos.latest < 1)
|
||||||
|
|
||||||
|
# Create a script and test again
|
||||||
|
repos.create_script('')
|
||||||
|
self.assertEquals(repos.latest, 1)
|
||||||
|
self.assert_(repos.latest >= 0)
|
||||||
|
self.assert_(repos.latest >= 1)
|
||||||
|
self.assert_(repos.latest < 2)
|
||||||
|
|
||||||
|
# Create a new script and test again
|
||||||
|
repos.create_script('')
|
||||||
|
self.assertEquals(repos.latest, 2)
|
||||||
|
self.assert_(repos.latest >= 0)
|
||||||
|
self.assert_(repos.latest >= 1)
|
||||||
|
self.assert_(repos.latest >= 2)
|
||||||
|
self.assert_(repos.latest < 3)
|
||||||
|
|
||||||
|
def test_source(self):
|
||||||
|
"""Get a script object by version number and view its source"""
|
||||||
|
# Load repository and commit script
|
||||||
|
repo = Repository(self.path_repos)
|
||||||
|
repo.create_script('')
|
||||||
|
|
||||||
|
# Get script object
|
||||||
|
source = repo.version(1).script().source()
|
||||||
|
|
||||||
|
# Source is valid: script must have an upgrade function
|
||||||
|
# (not a very thorough test, but should be plenty)
|
||||||
|
self.assert_(source.find('def upgrade') >= 0)
|
||||||
|
|
||||||
|
def test_latestversion(self):
|
||||||
|
"""Repository.version() (no params) returns the latest version"""
|
||||||
|
repos = Repository(self.path_repos)
|
||||||
|
repos.create_script('')
|
||||||
|
self.assert_(repos.version(repos.latest) is repos.version())
|
||||||
|
self.assert_(repos.version() is not None)
|
||||||
|
|
||||||
|
def test_changeset(self):
|
||||||
|
"""Repositories can create changesets properly"""
|
||||||
|
# Create a nonzero-version repository of empty scripts
|
||||||
|
repos = Repository(self.path_repos)
|
||||||
|
for i in range(10):
|
||||||
|
repos.create_script('')
|
||||||
|
|
||||||
|
def check_changeset(params, length):
|
||||||
|
"""Creates and verifies a changeset"""
|
||||||
|
changeset = repos.changeset('postgres', *params)
|
||||||
|
self.assertEquals(len(changeset), length)
|
||||||
|
self.assertTrue(isinstance(changeset, Changeset))
|
||||||
|
uniq = list()
|
||||||
|
# Changesets are iterable
|
||||||
|
for version, change in changeset:
|
||||||
|
self.assert_(isinstance(change, script.BaseScript))
|
||||||
|
# Changes aren't identical
|
||||||
|
self.assert_(id(change) not in uniq)
|
||||||
|
uniq.append(id(change))
|
||||||
|
return changeset
|
||||||
|
|
||||||
|
# Upgrade to a specified version...
|
||||||
|
cs = check_changeset((0, 10), 10)
|
||||||
|
self.assertEquals(cs.keys().pop(0),0 ) # 0 -> 1: index is starting version
|
||||||
|
self.assertEquals(cs.keys().pop(), 9) # 9 -> 10: index is starting version
|
||||||
|
self.assertEquals(cs.start, 0) # starting version
|
||||||
|
self.assertEquals(cs.end, 10) # ending version
|
||||||
|
check_changeset((0, 1), 1)
|
||||||
|
check_changeset((0, 5), 5)
|
||||||
|
check_changeset((0, 0), 0)
|
||||||
|
check_changeset((5, 5), 0)
|
||||||
|
check_changeset((10, 10), 0)
|
||||||
|
check_changeset((5, 10), 5)
|
||||||
|
|
||||||
|
# Can't request a changeset of higher version than this repository
|
||||||
|
self.assertRaises(Exception, repos.changeset, 'postgres', 5, 11)
|
||||||
|
self.assertRaises(Exception, repos.changeset, 'postgres', -1, 5)
|
||||||
|
|
||||||
|
# Upgrade to the latest version...
|
||||||
|
cs = check_changeset((0,), 10)
|
||||||
|
self.assertEquals(cs.keys().pop(0), 0)
|
||||||
|
self.assertEquals(cs.keys().pop(), 9)
|
||||||
|
self.assertEquals(cs.start, 0)
|
||||||
|
self.assertEquals(cs.end, 10)
|
||||||
|
check_changeset((1,), 9)
|
||||||
|
check_changeset((5,), 5)
|
||||||
|
check_changeset((9,), 1)
|
||||||
|
check_changeset((10,), 0)
|
||||||
|
|
||||||
|
# run changes
|
||||||
|
cs.run('postgres', 'upgrade')
|
||||||
|
|
||||||
|
# Can't request a changeset of higher/lower version than this repository
|
||||||
|
self.assertRaises(Exception, repos.changeset, 'postgres', 11)
|
||||||
|
self.assertRaises(Exception, repos.changeset, 'postgres', -1)
|
||||||
|
|
||||||
|
# Downgrade
|
||||||
|
cs = check_changeset((10, 0),10)
|
||||||
|
self.assertEquals(cs.keys().pop(0), 10) # 10 -> 9
|
||||||
|
self.assertEquals(cs.keys().pop(), 1) # 1 -> 0
|
||||||
|
self.assertEquals(cs.start, 10)
|
||||||
|
self.assertEquals(cs.end, 0)
|
||||||
|
check_changeset((10, 5), 5)
|
||||||
|
check_changeset((5, 0), 5)
|
||||||
|
|
||||||
|
def test_many_versions(self):
|
||||||
|
"""Test what happens when lots of versions are created"""
|
||||||
|
repos = Repository(self.path_repos)
|
||||||
|
for i in range(1001):
|
||||||
|
repos.create_script('')
|
||||||
|
|
||||||
|
# since we normally create 3 digit ones, let's see if we blow up
|
||||||
|
self.assert_(os.path.exists('%s/versions/1000.py' % self.path_repos))
|
||||||
|
self.assert_(os.path.exists('%s/versions/1001.py' % self.path_repos))
|
||||||
|
|
||||||
|
|
||||||
|
# TODO: test manage file
|
||||||
|
# TODO: test changeset
|
||||||
47
test/versioning/test_runchangeset.py
Normal file
47
test/versioning/test_runchangeset.py
Normal file
@ -0,0 +1,47 @@
|
|||||||
|
from test import fixture
|
||||||
|
from migrate.versioning.schema import *
|
||||||
|
from migrate.versioning import script
|
||||||
|
import os,shutil
|
||||||
|
|
||||||
|
class TestRunChangeset(fixture.Pathed,fixture.DB):
|
||||||
|
level=fixture.DB.CONNECT
|
||||||
|
def _setup(self, url):
|
||||||
|
super(TestRunChangeset, self)._setup(url)
|
||||||
|
Repository.clear()
|
||||||
|
self.path_repos=self.tmp_repos()
|
||||||
|
# Create repository, script
|
||||||
|
Repository.create(self.path_repos,'repository_name')
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_changeset_run(self):
|
||||||
|
"""Running a changeset against a repository gives expected results"""
|
||||||
|
repos=Repository(self.path_repos)
|
||||||
|
for i in range(10):
|
||||||
|
repos.create_script('')
|
||||||
|
try:
|
||||||
|
ControlledSchema(self.engine,repos).drop()
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
db=ControlledSchema.create(self.engine,repos)
|
||||||
|
|
||||||
|
# Scripts are empty; we'll check version # correctness.
|
||||||
|
# (Correct application of their content is checked elsewhere)
|
||||||
|
self.assertEquals(db.version,0)
|
||||||
|
db.upgrade(1)
|
||||||
|
self.assertEquals(db.version,1)
|
||||||
|
db.upgrade(5)
|
||||||
|
self.assertEquals(db.version,5)
|
||||||
|
db.upgrade(5)
|
||||||
|
self.assertEquals(db.version,5)
|
||||||
|
db.upgrade(None) # Latest is implied
|
||||||
|
self.assertEquals(db.version,10)
|
||||||
|
self.assertRaises(Exception,db.upgrade,11)
|
||||||
|
self.assertEquals(db.version,10)
|
||||||
|
db.upgrade(9)
|
||||||
|
self.assertEquals(db.version,9)
|
||||||
|
db.upgrade(0)
|
||||||
|
self.assertEquals(db.version,0)
|
||||||
|
self.assertRaises(Exception,db.upgrade,-1)
|
||||||
|
self.assertEquals(db.version,0)
|
||||||
|
#changeset = repos.changeset(self.url,0)
|
||||||
|
db.drop()
|
||||||
203
test/versioning/test_schema.py
Normal file
203
test/versioning/test_schema.py
Normal file
@ -0,0 +1,203 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
from migrate.versioning.schema import *
|
||||||
|
from migrate.versioning import script, exceptions, schemadiff
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class TestControlledSchema(fixture.Pathed, fixture.DB):
|
||||||
|
# Transactions break postgres in this test; we'll clean up after ourselves
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super(TestControlledSchema, self).setUp()
|
||||||
|
path_repos = self.temp_usable_dir + '/repo/'
|
||||||
|
self.repos = Repository.create(path_repos, 'repo_name')
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
self.setUp()
|
||||||
|
super(TestControlledSchema, self)._setup(url)
|
||||||
|
self.cleanup()
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
super(TestControlledSchema, self)._teardown()
|
||||||
|
self.cleanup()
|
||||||
|
self.tearDown()
|
||||||
|
|
||||||
|
def cleanup(self):
|
||||||
|
# drop existing version table if necessary
|
||||||
|
try:
|
||||||
|
ControlledSchema(self.engine, self.repos).drop()
|
||||||
|
except:
|
||||||
|
# No table to drop; that's fine, be silent
|
||||||
|
pass
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.cleanup()
|
||||||
|
super(TestControlledSchema, self).tearDown()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_version_control(self):
|
||||||
|
"""Establish version control on a particular database"""
|
||||||
|
# Establish version control on this database
|
||||||
|
dbcontrol = ControlledSchema.create(self.engine, self.repos)
|
||||||
|
|
||||||
|
# Trying to create another DB this way fails: table exists
|
||||||
|
self.assertRaises(exceptions.DatabaseAlreadyControlledError,
|
||||||
|
ControlledSchema.create, self.engine, self.repos)
|
||||||
|
|
||||||
|
# We can load a controlled DB this way, too
|
||||||
|
dbcontrol0 = ControlledSchema(self.engine, self.repos)
|
||||||
|
self.assertEquals(dbcontrol, dbcontrol0)
|
||||||
|
|
||||||
|
# We can also use a repository path, instead of a repository
|
||||||
|
dbcontrol0 = ControlledSchema(self.engine, self.repos.path)
|
||||||
|
self.assertEquals(dbcontrol, dbcontrol0)
|
||||||
|
|
||||||
|
# We don't have to use the same connection
|
||||||
|
engine = create_engine(self.url)
|
||||||
|
dbcontrol0 = ControlledSchema(engine, self.repos.path)
|
||||||
|
self.assertEquals(dbcontrol, dbcontrol0)
|
||||||
|
|
||||||
|
# Clean up:
|
||||||
|
dbcontrol.drop()
|
||||||
|
|
||||||
|
# Attempting to drop vc from a db without it should fail
|
||||||
|
self.assertRaises(exceptions.DatabaseNotControlledError, dbcontrol.drop)
|
||||||
|
|
||||||
|
# No table defined should raise error
|
||||||
|
self.assertRaises(exceptions.DatabaseNotControlledError,
|
||||||
|
ControlledSchema, self.engine, self.repos)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_version_control_specified(self):
|
||||||
|
"""Establish version control with a specified version"""
|
||||||
|
# Establish version control on this database
|
||||||
|
version = 0
|
||||||
|
dbcontrol = ControlledSchema.create(self.engine, self.repos, version)
|
||||||
|
self.assertEquals(dbcontrol.version, version)
|
||||||
|
|
||||||
|
# Correct when we load it, too
|
||||||
|
dbcontrol = ControlledSchema(self.engine, self.repos)
|
||||||
|
self.assertEquals(dbcontrol.version, version)
|
||||||
|
|
||||||
|
dbcontrol.drop()
|
||||||
|
|
||||||
|
# Now try it with a nonzero value
|
||||||
|
version = 10
|
||||||
|
for i in range(version):
|
||||||
|
self.repos.create_script('')
|
||||||
|
self.assertEquals(self.repos.latest, version)
|
||||||
|
|
||||||
|
# Test with some mid-range value
|
||||||
|
dbcontrol = ControlledSchema.create(self.engine,self.repos, 5)
|
||||||
|
self.assertEquals(dbcontrol.version, 5)
|
||||||
|
dbcontrol.drop()
|
||||||
|
|
||||||
|
# Test with max value
|
||||||
|
dbcontrol = ControlledSchema.create(self.engine, self.repos, version)
|
||||||
|
self.assertEquals(dbcontrol.version, version)
|
||||||
|
dbcontrol.drop()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_version_control_invalid(self):
|
||||||
|
"""Try to establish version control with an invalid version"""
|
||||||
|
versions = ('Thirteen', '-1', -1, '' , 13)
|
||||||
|
# A fresh repository doesn't go up to version 13 yet
|
||||||
|
for version in versions:
|
||||||
|
#self.assertRaises(ControlledSchema.InvalidVersionError,
|
||||||
|
# Can't have custom errors with assertRaises...
|
||||||
|
try:
|
||||||
|
ControlledSchema.create(self.engine,self.repos,version)
|
||||||
|
self.assert_(False, repr(version))
|
||||||
|
except exceptions.InvalidVersionError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_changeset(self):
|
||||||
|
"""Create changeset from controlled schema"""
|
||||||
|
dbschema = ControlledSchema.create(self.engine, self.repos)
|
||||||
|
|
||||||
|
# empty schema doesn't have changesets
|
||||||
|
cs = dbschema.changeset()
|
||||||
|
self.assertEqual(cs, {})
|
||||||
|
|
||||||
|
for i in range(5):
|
||||||
|
self.repos.create_script('')
|
||||||
|
self.assertEquals(self.repos.latest, 5)
|
||||||
|
|
||||||
|
cs = dbschema.changeset(5)
|
||||||
|
self.assertEqual(len(cs), 5)
|
||||||
|
|
||||||
|
# cleanup
|
||||||
|
dbschema.drop()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_upgrade_runchange(self):
|
||||||
|
dbschema = ControlledSchema.create(self.engine, self.repos)
|
||||||
|
|
||||||
|
for i in range(10):
|
||||||
|
self.repos.create_script('')
|
||||||
|
|
||||||
|
self.assertEquals(self.repos.latest, 10)
|
||||||
|
|
||||||
|
dbschema.upgrade(10)
|
||||||
|
|
||||||
|
self.assertRaises(ValueError, dbschema.upgrade, 'a')
|
||||||
|
self.assertRaises(exceptions.InvalidVersionError, dbschema.runchange, 20, '', 1)
|
||||||
|
|
||||||
|
# TODO: test for table version in db
|
||||||
|
|
||||||
|
# cleanup
|
||||||
|
dbschema.drop()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_create_model(self):
|
||||||
|
"""Test workflow to generate create_model"""
|
||||||
|
model = ControlledSchema.create_model(self.engine, self.repos, declarative=False)
|
||||||
|
self.assertTrue(isinstance(model, basestring))
|
||||||
|
|
||||||
|
model = ControlledSchema.create_model(self.engine, self.repos.path, declarative=True)
|
||||||
|
self.assertTrue(isinstance(model, basestring))
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_compare_model_to_db(self):
|
||||||
|
meta = self.construct_model()
|
||||||
|
|
||||||
|
diff = ControlledSchema.compare_model_to_db(self.engine, meta, self.repos)
|
||||||
|
self.assertTrue(isinstance(diff, schemadiff.SchemaDiff))
|
||||||
|
|
||||||
|
diff = ControlledSchema.compare_model_to_db(self.engine, meta, self.repos.path)
|
||||||
|
self.assertTrue(isinstance(diff, schemadiff.SchemaDiff))
|
||||||
|
meta.drop_all(self.engine)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_update_db_from_model(self):
|
||||||
|
dbschema = ControlledSchema.create(self.engine, self.repos)
|
||||||
|
|
||||||
|
meta = self.construct_model()
|
||||||
|
|
||||||
|
dbschema.update_db_from_model(meta)
|
||||||
|
|
||||||
|
# TODO: test for table version in db
|
||||||
|
|
||||||
|
# cleanup
|
||||||
|
dbschema.drop()
|
||||||
|
meta.drop_all(self.engine)
|
||||||
|
|
||||||
|
def construct_model(self):
|
||||||
|
meta = MetaData()
|
||||||
|
|
||||||
|
user = Table('temp_model_schema', meta, Column('id', Integer), Column('user', String(245)))
|
||||||
|
|
||||||
|
return meta
|
||||||
|
|
||||||
|
# TODO: test how are tables populated in db
|
||||||
158
test/versioning/test_schemadiff.py
Normal file
158
test/versioning/test_schemadiff.py
Normal file
@ -0,0 +1,158 @@
|
|||||||
|
import os
|
||||||
|
import sqlalchemy
|
||||||
|
from sqlalchemy import *
|
||||||
|
from test import fixture
|
||||||
|
from migrate.versioning import genmodel, schemadiff
|
||||||
|
from nose.tools import eq_
|
||||||
|
|
||||||
|
|
||||||
|
class TestSchemaDiff(fixture.DB):
|
||||||
|
level=fixture.DB.CONNECT
|
||||||
|
table_name = 'tmp_schemadiff'
|
||||||
|
|
||||||
|
def _setup(self, url):
|
||||||
|
|
||||||
|
super(TestSchemaDiff, self)._setup(url)
|
||||||
|
self.meta = MetaData(self.engine, reflect=True)
|
||||||
|
self.meta.drop_all() # in case junk tables are lying around in the test database
|
||||||
|
self.meta = MetaData(self.engine, reflect=True) # needed if we just deleted some tables
|
||||||
|
self.table = Table(self.table_name,self.meta,
|
||||||
|
Column('id',Integer(),primary_key=True),
|
||||||
|
Column('name',UnicodeText()),
|
||||||
|
Column('data',UnicodeText()),
|
||||||
|
)
|
||||||
|
WANT_ENGINE_ECHO = os.environ.get('WANT_ENGINE_ECHO', 'F') # to get debugging: set this to T and run py.test with --pdb
|
||||||
|
if WANT_ENGINE_ECHO == 'T':
|
||||||
|
self.engine.echo = True
|
||||||
|
|
||||||
|
def _teardown(self):
|
||||||
|
if self.table.exists():
|
||||||
|
#self.table.drop() # bummer, this doesn't work because the list of tables is out of date, but calling reflect didn't work
|
||||||
|
self.meta = MetaData(self.engine, reflect=True)
|
||||||
|
self.meta.drop_all()
|
||||||
|
super(TestSchemaDiff, self)._teardown()
|
||||||
|
|
||||||
|
def _applyLatestModel(self):
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(self.meta, self.engine, excludeTables=['migrate_version'])
|
||||||
|
genmodel.ModelGenerator(diff).applyModel()
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_rundiffs(self):
|
||||||
|
|
||||||
|
# Yuck! We have to import from changeset to apply the monkey-patch to allow column adding/dropping.
|
||||||
|
from migrate.changeset import schema
|
||||||
|
|
||||||
|
def assertDiff(isDiff, tablesMissingInDatabase, tablesMissingInModel, tablesWithDiff):
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(self.meta, self.engine, excludeTables=['migrate_version'])
|
||||||
|
eq_(bool(diff), isDiff)
|
||||||
|
eq_( ([t.name for t in diff.tablesMissingInDatabase], [t.name for t in diff.tablesMissingInModel], [t.name for t in diff.tablesWithDiff]),
|
||||||
|
(tablesMissingInDatabase, tablesMissingInModel, tablesWithDiff) )
|
||||||
|
|
||||||
|
# Model is defined but database is empty.
|
||||||
|
assertDiff(True, [self.table_name], [], [])
|
||||||
|
|
||||||
|
# Check Python upgrade and downgrade of database from updated model.
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(self.meta, self.engine, excludeTables=['migrate_version'])
|
||||||
|
decls, upgradeCommands, downgradeCommands = genmodel.ModelGenerator(diff).toUpgradeDowngradePython()
|
||||||
|
self.assertEqualsIgnoreWhitespace(decls, '''
|
||||||
|
meta = MetaData()
|
||||||
|
tmp_schemadiff = Table('tmp_schemadiff',meta,
|
||||||
|
Column('id',Integer(),primary_key=True,nullable=False),
|
||||||
|
Column('name',UnicodeText(length=None)),
|
||||||
|
Column('data',UnicodeText(length=None)),
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
self.assertEqualsIgnoreWhitespace(upgradeCommands,
|
||||||
|
'''meta.bind(migrate_engine)
|
||||||
|
tmp_schemadiff.create()''')
|
||||||
|
self.assertEqualsIgnoreWhitespace(downgradeCommands,
|
||||||
|
'''meta.bind(migrate_engine)
|
||||||
|
tmp_schemadiff.drop()''')
|
||||||
|
|
||||||
|
# Create table in database, now model should match database.
|
||||||
|
self._applyLatestModel()
|
||||||
|
assertDiff(False, [], [], [])
|
||||||
|
|
||||||
|
# Check Python code gen from database.
|
||||||
|
diff = schemadiff.getDiffOfModelAgainstDatabase(MetaData(), self.engine, excludeTables=['migrate_version'])
|
||||||
|
src = genmodel.ModelGenerator(diff).toPython()
|
||||||
|
src = src.replace(genmodel.HEADER, '')
|
||||||
|
self.assertEqualsIgnoreWhitespace(src, '''
|
||||||
|
tmp_schemadiff = Table('tmp_schemadiff',meta,
|
||||||
|
Column('id',Integer(),primary_key=True,nullable=False),
|
||||||
|
Column('name',Text(length=None,convert_unicode=False,assert_unicode=None)),
|
||||||
|
Column('data',Text(length=None,convert_unicode=False,assert_unicode=None)),
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
|
||||||
|
if not self.engine.name == 'oracle':
|
||||||
|
# Add data, later we'll make sure it's still present.
|
||||||
|
result = self.engine.execute(self.table.insert(), id=1, name=u'mydata')
|
||||||
|
dataId = result.last_inserted_ids()[0]
|
||||||
|
|
||||||
|
# Modify table in model (by removing it and adding it back to model) -- drop column data and add column data2.
|
||||||
|
self.meta.remove(self.table)
|
||||||
|
self.table = Table(self.table_name,self.meta,
|
||||||
|
Column('id',Integer(),primary_key=True),
|
||||||
|
Column('name',UnicodeText(length=None)),
|
||||||
|
Column('data2',Integer(),nullable=True),
|
||||||
|
)
|
||||||
|
assertDiff(True, [], [], [self.table_name])
|
||||||
|
|
||||||
|
# Apply latest model changes and find no more diffs.
|
||||||
|
self._applyLatestModel()
|
||||||
|
assertDiff(False, [], [], [])
|
||||||
|
|
||||||
|
if not self.engine.name == 'oracle':
|
||||||
|
# Make sure data is still present.
|
||||||
|
result = self.engine.execute(self.table.select(self.table.c.id==dataId))
|
||||||
|
rows = result.fetchall()
|
||||||
|
eq_(len(rows), 1)
|
||||||
|
eq_(rows[0].name, 'mydata')
|
||||||
|
|
||||||
|
# Add data, later we'll make sure it's still present.
|
||||||
|
result = self.engine.execute(self.table.insert(), id=2, name=u'mydata2', data2=123)
|
||||||
|
dataId2 = result.last_inserted_ids()[0]
|
||||||
|
|
||||||
|
# Change column type in model.
|
||||||
|
self.meta.remove(self.table)
|
||||||
|
self.table = Table(self.table_name,self.meta,
|
||||||
|
Column('id',Integer(),primary_key=True),
|
||||||
|
Column('name',UnicodeText(length=None)),
|
||||||
|
Column('data2',String(255),nullable=True),
|
||||||
|
)
|
||||||
|
assertDiff(True, [], [], [self.table_name]) # TODO test type diff
|
||||||
|
|
||||||
|
# Apply latest model changes and find no more diffs.
|
||||||
|
self._applyLatestModel()
|
||||||
|
assertDiff(False, [], [], [])
|
||||||
|
|
||||||
|
if not self.engine.name == 'oracle':
|
||||||
|
# Make sure data is still present.
|
||||||
|
result = self.engine.execute(self.table.select(self.table.c.id==dataId2))
|
||||||
|
rows = result.fetchall()
|
||||||
|
self.assertEquals(len(rows), 1)
|
||||||
|
self.assertEquals(rows[0].name, 'mydata2')
|
||||||
|
self.assertEquals(rows[0].data2, '123')
|
||||||
|
|
||||||
|
# Delete data, since we're about to make a required column.
|
||||||
|
# Not even using sqlalchemy.PassiveDefault helps because we're doing explicit column select.
|
||||||
|
self.engine.execute(self.table.delete(), id=dataId)
|
||||||
|
|
||||||
|
if not self.engine.name == 'firebird':
|
||||||
|
# Change column nullable in model.
|
||||||
|
self.meta.remove(self.table)
|
||||||
|
self.table = Table(self.table_name,self.meta,
|
||||||
|
Column('id',Integer(),primary_key=True),
|
||||||
|
Column('name',UnicodeText(length=None)),
|
||||||
|
Column('data2',String(255),nullable=False),
|
||||||
|
)
|
||||||
|
assertDiff(True, [], [], [self.table_name]) # TODO test nullable diff
|
||||||
|
|
||||||
|
# Apply latest model changes and find no more diffs.
|
||||||
|
self._applyLatestModel()
|
||||||
|
assertDiff(False, [], [], [])
|
||||||
|
|
||||||
|
# Remove table from model.
|
||||||
|
self.meta.remove(self.table)
|
||||||
|
assertDiff(True, [], [self.table_name], [])
|
||||||
198
test/versioning/test_script.py
Normal file
198
test/versioning/test_script.py
Normal file
@ -0,0 +1,198 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
from migrate.versioning import exceptions, version, repository
|
||||||
|
from migrate.versioning.script import *
|
||||||
|
from migrate.versioning.util import *
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class TestBaseScript(fixture.Pathed):
|
||||||
|
|
||||||
|
def test_all(self):
|
||||||
|
"""Testing all basic BaseScript operations"""
|
||||||
|
# verify / source / run
|
||||||
|
src = self.tmp()
|
||||||
|
open(src, 'w').close()
|
||||||
|
bscript = BaseScript(src)
|
||||||
|
BaseScript.verify(src)
|
||||||
|
self.assertEqual(bscript.source(), '')
|
||||||
|
self.assertRaises(NotImplementedError, bscript.run, 'foobar')
|
||||||
|
|
||||||
|
|
||||||
|
class TestPyScript(fixture.Pathed, fixture.DB):
|
||||||
|
cls = PythonScript
|
||||||
|
def test_create(self):
|
||||||
|
"""We can create a migration script"""
|
||||||
|
path = self.tmp_py()
|
||||||
|
# Creating a file that doesn't exist should succeed
|
||||||
|
self.cls.create(path)
|
||||||
|
self.assert_(os.path.exists(path))
|
||||||
|
# Created file should be a valid script (If not, raises an error)
|
||||||
|
self.cls.verify(path)
|
||||||
|
# Can't create it again: it already exists
|
||||||
|
self.assertRaises(exceptions.PathFoundError,self.cls.create,path)
|
||||||
|
|
||||||
|
@fixture.usedb(supported='sqlite')
|
||||||
|
def test_run(self):
|
||||||
|
script_path = self.tmp_py()
|
||||||
|
pyscript = PythonScript.create(script_path)
|
||||||
|
pyscript.run(self.engine, 1)
|
||||||
|
pyscript.run(self.engine, -1)
|
||||||
|
|
||||||
|
self.assertRaises(exceptions.ScriptError, pyscript.run, self.engine, 0)
|
||||||
|
self.assertRaises(exceptions.ScriptError, pyscript._func, 'foobar')
|
||||||
|
|
||||||
|
def test_verify_notfound(self):
|
||||||
|
"""Correctly verify a python migration script: nonexistant file"""
|
||||||
|
path = self.tmp_py()
|
||||||
|
self.assertFalse(os.path.exists(path))
|
||||||
|
# Fails on empty path
|
||||||
|
self.assertRaises(exceptions.InvalidScriptError,self.cls.verify,path)
|
||||||
|
self.assertRaises(exceptions.InvalidScriptError,self.cls,path)
|
||||||
|
|
||||||
|
def test_verify_invalidpy(self):
|
||||||
|
"""Correctly verify a python migration script: invalid python file"""
|
||||||
|
path=self.tmp_py()
|
||||||
|
# Create empty file
|
||||||
|
f=open(path,'w')
|
||||||
|
f.write("def fail")
|
||||||
|
f.close()
|
||||||
|
self.assertRaises(Exception,self.cls.verify_module,path)
|
||||||
|
# script isn't verified on creation, but on module reference
|
||||||
|
py = self.cls(path)
|
||||||
|
self.assertRaises(Exception,(lambda x: x.module),py)
|
||||||
|
|
||||||
|
def test_verify_nofuncs(self):
|
||||||
|
"""Correctly verify a python migration script: valid python file; no upgrade func"""
|
||||||
|
path = self.tmp_py()
|
||||||
|
# Create empty file
|
||||||
|
f = open(path, 'w')
|
||||||
|
f.write("def zergling():\n\tprint 'rush'")
|
||||||
|
f.close()
|
||||||
|
self.assertRaises(exceptions.InvalidScriptError, self.cls.verify_module, path)
|
||||||
|
# script isn't verified on creation, but on module reference
|
||||||
|
py = self.cls(path)
|
||||||
|
self.assertRaises(exceptions.InvalidScriptError,(lambda x: x.module),py)
|
||||||
|
|
||||||
|
@fixture.usedb(supported='sqlite')
|
||||||
|
def test_preview_sql(self):
|
||||||
|
"""Preview SQL abstract from ORM layer (sqlite)"""
|
||||||
|
path = self.tmp_py()
|
||||||
|
|
||||||
|
f = open(path, 'w')
|
||||||
|
content = """
|
||||||
|
from migrate import *
|
||||||
|
from sqlalchemy import *
|
||||||
|
|
||||||
|
metadata = MetaData()
|
||||||
|
|
||||||
|
UserGroup = Table('Link', metadata,
|
||||||
|
Column('link1ID', Integer),
|
||||||
|
Column('link2ID', Integer),
|
||||||
|
UniqueConstraint('link1ID', 'link2ID'))
|
||||||
|
|
||||||
|
def upgrade(migrate_engine):
|
||||||
|
metadata.create_all(migrate_engine)
|
||||||
|
"""
|
||||||
|
f.write(content)
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
pyscript = self.cls(path)
|
||||||
|
SQL = pyscript.preview_sql(self.url, 1)
|
||||||
|
self.assertEqualsIgnoreWhitespace("""
|
||||||
|
CREATE TABLE "Link"
|
||||||
|
("link1ID" INTEGER,
|
||||||
|
"link2ID" INTEGER,
|
||||||
|
UNIQUE ("link1ID", "link2ID"))
|
||||||
|
""", SQL)
|
||||||
|
# TODO: test: No SQL should be executed!
|
||||||
|
|
||||||
|
def test_verify_success(self):
|
||||||
|
"""Correctly verify a python migration script: success"""
|
||||||
|
path = self.tmp_py()
|
||||||
|
# Succeeds after creating
|
||||||
|
self.cls.create(path)
|
||||||
|
self.cls.verify(path)
|
||||||
|
|
||||||
|
# test for PythonScript.make_update_script_for_model
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_make_update_script_for_model(self):
|
||||||
|
"""Construct script source from differences of two models"""
|
||||||
|
|
||||||
|
self.setup_model_params()
|
||||||
|
self.write_file(self.first_model_path, self.base_source)
|
||||||
|
self.write_file(self.second_model_path, self.base_source + self.model_source)
|
||||||
|
|
||||||
|
|
||||||
|
source_script = self.pyscript.make_update_script_for_model(
|
||||||
|
engine=self.engine,
|
||||||
|
oldmodel=load_model('testmodel_first:meta'),
|
||||||
|
model=load_model('testmodel_second:meta'),
|
||||||
|
repository=self.repo_path,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertTrue('User.create()' in source_script)
|
||||||
|
self.assertTrue('User.drop()' in source_script)
|
||||||
|
|
||||||
|
#@fixture.usedb()
|
||||||
|
#def test_make_update_script_for_model_equals(self):
|
||||||
|
# """Try to make update script from two identical models"""
|
||||||
|
|
||||||
|
# self.setup_model_params()
|
||||||
|
# self.write_file(self.first_model_path, self.base_source + self.model_source)
|
||||||
|
# self.write_file(self.second_model_path, self.base_source + self.model_source)
|
||||||
|
|
||||||
|
# source_script = self.pyscript.make_update_script_for_model(
|
||||||
|
# engine=self.engine,
|
||||||
|
# oldmodel=load_model('testmodel_first:meta'),
|
||||||
|
# model=load_model('testmodel_second:meta'),
|
||||||
|
# repository=self.repo_path,
|
||||||
|
# )
|
||||||
|
|
||||||
|
# self.assertFalse('User.create()' in source_script)
|
||||||
|
# self.assertFalse('User.drop()' in source_script)
|
||||||
|
|
||||||
|
def setup_model_params(self):
|
||||||
|
self.script_path = self.tmp_py()
|
||||||
|
self.repo_path = self.tmp()
|
||||||
|
self.first_model_path = os.path.join(self.temp_usable_dir, 'testmodel_first.py')
|
||||||
|
self.second_model_path = os.path.join(self.temp_usable_dir, 'testmodel_second.py')
|
||||||
|
|
||||||
|
self.base_source = """from sqlalchemy import *\nmeta = MetaData()\n"""
|
||||||
|
self.model_source = """
|
||||||
|
User = Table('User', meta,
|
||||||
|
Column('id', Integer, primary_key=True),
|
||||||
|
Column('login', Unicode(40)),
|
||||||
|
Column('passwd', String(40)),
|
||||||
|
)"""
|
||||||
|
|
||||||
|
self.repo = repository.Repository.create(self.repo_path, 'repo')
|
||||||
|
self.pyscript = PythonScript.create(self.script_path)
|
||||||
|
|
||||||
|
|
||||||
|
def write_file(self, path, contents):
|
||||||
|
f = open(path, 'w')
|
||||||
|
f.write(contents)
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
|
||||||
|
class TestSqlScript(fixture.Pathed, fixture.DB):
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_error(self):
|
||||||
|
"""Test if exception is raised on wrong script source"""
|
||||||
|
src = self.tmp()
|
||||||
|
|
||||||
|
f = open(src, 'w')
|
||||||
|
f.write("""foobar""")
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
sqls = SqlScript(src)
|
||||||
|
self.assertRaises(Exception, sqls.run, self.engine)
|
||||||
568
test/versioning/test_shell.py
Normal file
568
test/versioning/test_shell.py
Normal file
@ -0,0 +1,568 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import shutil
|
||||||
|
import traceback
|
||||||
|
from types import FileType
|
||||||
|
from StringIO import StringIO
|
||||||
|
|
||||||
|
from sqlalchemy import MetaData,Table
|
||||||
|
|
||||||
|
from migrate.versioning.repository import Repository
|
||||||
|
from migrate.versioning import genmodel, shell, api
|
||||||
|
from migrate.versioning.exceptions import *
|
||||||
|
from test import fixture
|
||||||
|
|
||||||
|
|
||||||
|
class Shell(fixture.Shell):
|
||||||
|
|
||||||
|
_cmd = os.path.join(sys.executable + ' migrate', 'versioning', 'shell.py')
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def cmd(cls, *args):
|
||||||
|
safe_parameters = map(lambda arg: str(arg), args)
|
||||||
|
return ' '.join([cls._cmd] + safe_parameters)
|
||||||
|
|
||||||
|
def execute(self, shell_cmd, runshell=None, **kwargs):
|
||||||
|
"""A crude simulation of a shell command, to speed things up"""
|
||||||
|
|
||||||
|
# If we get an fd, the command is already done
|
||||||
|
if isinstance(shell_cmd, (FileType, StringIO)):
|
||||||
|
return shell_cmd
|
||||||
|
|
||||||
|
# Analyze the command; see if we can 'fake' the shell
|
||||||
|
try:
|
||||||
|
# Forced to run in shell?
|
||||||
|
# if runshell or '--runshell' in sys.argv:
|
||||||
|
if runshell:
|
||||||
|
raise Exception
|
||||||
|
# Remove the command prefix
|
||||||
|
if not shell_cmd.startswith(self._cmd):
|
||||||
|
raise Exception
|
||||||
|
cmd = shell_cmd[(len(self._cmd) + 1):]
|
||||||
|
params = cmd.split(' ')
|
||||||
|
command = params[0]
|
||||||
|
except:
|
||||||
|
return super(Shell, self).execute(shell_cmd)
|
||||||
|
|
||||||
|
# Redirect stdout to an object; redirect stderr to stdout
|
||||||
|
fd = StringIO()
|
||||||
|
orig_stdout = sys.stdout
|
||||||
|
orig_stderr = sys.stderr
|
||||||
|
sys.stdout = fd
|
||||||
|
sys.stderr = fd
|
||||||
|
# Execute this command
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
shell.main(params, **kwargs)
|
||||||
|
except SystemExit, e:
|
||||||
|
# Simulate the exit status
|
||||||
|
fd_close = fd.close
|
||||||
|
def close_():
|
||||||
|
fd_close()
|
||||||
|
return e.args[0]
|
||||||
|
fd.close = close_
|
||||||
|
except Exception, e:
|
||||||
|
# Print the exception, but don't re-raise it
|
||||||
|
traceback.print_exc()
|
||||||
|
# Simulate a nonzero exit status
|
||||||
|
fd_close = fd.close
|
||||||
|
def close_():
|
||||||
|
fd_close()
|
||||||
|
return 2
|
||||||
|
fd.close = close_
|
||||||
|
finally:
|
||||||
|
# Clean up
|
||||||
|
sys.stdout = orig_stdout
|
||||||
|
sys.stderr = orig_stderr
|
||||||
|
fd.seek(0)
|
||||||
|
return fd
|
||||||
|
|
||||||
|
def cmd_version(self, repos_path):
|
||||||
|
fd = self.execute(self.cmd('version', repos_path))
|
||||||
|
result = int(fd.read().strip())
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def cmd_db_version(self, url, repos_path):
|
||||||
|
fd = self.execute(self.cmd('db_version', url, repos_path))
|
||||||
|
txt = fd.read()
|
||||||
|
#print txt
|
||||||
|
ret = int(txt.strip())
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
class TestShellCommands(Shell):
|
||||||
|
"""Tests migrate.py commands"""
|
||||||
|
|
||||||
|
def test_help(self):
|
||||||
|
"""Displays default help dialog"""
|
||||||
|
self.assertSuccess(self.cmd('-h'), runshell=True)
|
||||||
|
self.assertSuccess(self.cmd('--help'), runshell=True)
|
||||||
|
self.assertSuccess(self.cmd('help'), runshell=True)
|
||||||
|
self.assertSuccess(self.cmd('help'))
|
||||||
|
|
||||||
|
self.assertRaises(UsageError, api.help)
|
||||||
|
self.assertRaises(UsageError, api.help, 'foobar')
|
||||||
|
self.assert_(isinstance(api.help('create'), str))
|
||||||
|
|
||||||
|
def test_help_commands(self):
|
||||||
|
"""Display help on a specific command"""
|
||||||
|
for cmd in shell.api.__all__:
|
||||||
|
fd = self.execute(self.cmd('help', cmd))
|
||||||
|
# Description may change, so best we can do is ensure it shows up
|
||||||
|
output = fd.read()
|
||||||
|
self.assertNotEquals(output, '')
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
def test_create(self):
|
||||||
|
"""Repositories are created successfully"""
|
||||||
|
repos = self.tmp_repos()
|
||||||
|
|
||||||
|
# Creating a file that doesn't exist should succeed
|
||||||
|
cmd = self.cmd('create', repos, 'repository_name')
|
||||||
|
self.assertSuccess(cmd)
|
||||||
|
|
||||||
|
# Files should actually be created
|
||||||
|
self.assert_(os.path.exists(repos))
|
||||||
|
|
||||||
|
# The default table should not be None
|
||||||
|
repos_ = Repository(repos)
|
||||||
|
self.assertNotEquals(repos_.config.get('db_settings', 'version_table'), 'None')
|
||||||
|
|
||||||
|
# Can't create it again: it already exists
|
||||||
|
self.assertFailure(cmd)
|
||||||
|
|
||||||
|
def test_script(self):
|
||||||
|
"""We can create a migration script via the command line"""
|
||||||
|
repos = self.tmp_repos()
|
||||||
|
self.assertSuccess(self.cmd('create', repos, 'repository_name'))
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % repos, 'Desc'))
|
||||||
|
self.assert_(os.path.exists('%s/versions/001_Desc.py' % repos))
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % repos, 'More'))
|
||||||
|
self.assert_(os.path.exists('%s/versions/002_More.py' % repos))
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % repos, '"Some Random name"'), runshell=True)
|
||||||
|
self.assert_(os.path.exists('%s/versions/003_Some_Random_name.py' % repos))
|
||||||
|
|
||||||
|
def test_script_sql(self):
|
||||||
|
"""We can create a migration sql script via the command line"""
|
||||||
|
repos = self.tmp_repos()
|
||||||
|
self.assertSuccess(self.cmd('create', repos, 'repository_name'))
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('script_sql', '--repository=%s' % repos, 'mydb'))
|
||||||
|
self.assert_(os.path.exists('%s/versions/001_mydb_upgrade.sql' % repos))
|
||||||
|
self.assert_(os.path.exists('%s/versions/001_mydb_downgrade.sql' % repos))
|
||||||
|
|
||||||
|
# Test creating a second
|
||||||
|
self.assertSuccess(self.cmd('script_sql', '--repository=%s' % repos, 'postgres'))
|
||||||
|
self.assert_(os.path.exists('%s/versions/002_postgres_upgrade.sql' % repos))
|
||||||
|
self.assert_(os.path.exists('%s/versions/002_postgres_downgrade.sql' % repos))
|
||||||
|
|
||||||
|
def test_manage(self):
|
||||||
|
"""Create a project management script"""
|
||||||
|
script = self.tmp_py()
|
||||||
|
self.assert_(not os.path.exists(script))
|
||||||
|
|
||||||
|
# No attempt is made to verify correctness of the repository path here
|
||||||
|
self.assertSuccess(self.cmd('manage', script, '--repository=/path/to/repository'))
|
||||||
|
self.assert_(os.path.exists(script))
|
||||||
|
|
||||||
|
class TestShellRepository(Shell):
|
||||||
|
"""Shell commands on an existing repository/python script"""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
"""Create repository, python change script"""
|
||||||
|
super(TestShellRepository, self).setUp()
|
||||||
|
self.path_repos = repos = self.tmp_repos()
|
||||||
|
self.assertSuccess(self.cmd('create', repos, 'repository_name'))
|
||||||
|
|
||||||
|
def test_version(self):
|
||||||
|
"""Correctly detect repository version"""
|
||||||
|
# Version: 0 (no scripts yet); successful execution
|
||||||
|
fd = self.execute(self.cmd('version','--repository=%s' % self.path_repos))
|
||||||
|
self.assertEquals(fd.read().strip(), "0")
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
# Also works as a positional param
|
||||||
|
fd = self.execute(self.cmd('version', self.path_repos))
|
||||||
|
self.assertEquals(fd.read().strip(), "0")
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
# Create a script and version should increment
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % self.path_repos, 'Desc'))
|
||||||
|
fd = self.execute(self.cmd('version',self.path_repos))
|
||||||
|
self.assertEquals(fd.read().strip(), "1")
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
def test_source(self):
|
||||||
|
"""Correctly fetch a script's source"""
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % self.path_repos, 'Desc'))
|
||||||
|
|
||||||
|
filename = '%s/versions/001_Desc.py' % self.path_repos
|
||||||
|
source = open(filename).read()
|
||||||
|
self.assert_(source.find('def upgrade') >= 0)
|
||||||
|
|
||||||
|
# Version is now 1
|
||||||
|
fd = self.execute(self.cmd('version', self.path_repos))
|
||||||
|
self.assert_(fd.read().strip() == "1")
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
# Output/verify the source of version 1
|
||||||
|
fd = self.execute(self.cmd('source', 1, '--repository=%s' % self.path_repos))
|
||||||
|
result = fd.read()
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
self.assert_(result.strip() == source.strip())
|
||||||
|
|
||||||
|
# We can also send the source to a file... test that too
|
||||||
|
self.assertSuccess(self.cmd('source', 1, filename, '--repository=%s'%self.path_repos))
|
||||||
|
self.assert_(os.path.exists(filename))
|
||||||
|
fd = open(filename)
|
||||||
|
result = fd.read()
|
||||||
|
self.assert_(result.strip() == source.strip())
|
||||||
|
|
||||||
|
class TestShellDatabase(Shell, fixture.DB):
|
||||||
|
"""Commands associated with a particular database"""
|
||||||
|
# We'll need to clean up after ourself, since the shell creates its own txn;
|
||||||
|
# we need to connect to the DB to see if things worked
|
||||||
|
|
||||||
|
level = fixture.DB.CONNECT
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_version_control(self):
|
||||||
|
"""Ensure we can set version control on a database"""
|
||||||
|
path_repos = repos = self.tmp_repos()
|
||||||
|
self.assertSuccess(self.cmd('create', path_repos, 'repository_name'))
|
||||||
|
self.exitcode(self.cmd('drop_version_control', self.url, path_repos))
|
||||||
|
self.assertSuccess(self.cmd('version_control', self.url, path_repos))
|
||||||
|
|
||||||
|
# Clean up
|
||||||
|
self.assertSuccess(self.cmd('drop_version_control',self.url,path_repos))
|
||||||
|
# Attempting to drop vc from a database without it should fail
|
||||||
|
self.assertFailure(self.cmd('drop_version_control',self.url,path_repos))
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_wrapped_kwargs(self):
|
||||||
|
"""Commands with default arguments set by manage.py"""
|
||||||
|
path_repos = repos = self.tmp_repos()
|
||||||
|
self.assertSuccess(self.cmd('create', '--', '--name=repository_name'), repository=path_repos)
|
||||||
|
self.exitcode(self.cmd('drop_version_control'), url=self.url, repository=path_repos)
|
||||||
|
self.assertSuccess(self.cmd('version_control'), url=self.url, repository=path_repos)
|
||||||
|
|
||||||
|
# Clean up
|
||||||
|
self.assertSuccess(self.cmd('drop_version_control'), url=self.url, repository=path_repos)
|
||||||
|
# Attempting to drop vc from a database without it should fail
|
||||||
|
self.assertFailure(self.cmd('drop_version_control'), url=self.url, repository=path_repos)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_version_control_specified(self):
|
||||||
|
"""Ensure we can set version control to a particular version"""
|
||||||
|
path_repos = self.tmp_repos()
|
||||||
|
self.assertSuccess(self.cmd('create', path_repos, 'repository_name'))
|
||||||
|
self.exitcode(self.cmd('drop_version_control', self.url, path_repos))
|
||||||
|
|
||||||
|
# Fill the repository
|
||||||
|
path_script = self.tmp_py()
|
||||||
|
version = 1
|
||||||
|
for i in range(version):
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % path_repos, 'Desc'))
|
||||||
|
|
||||||
|
# Repository version is correct
|
||||||
|
fd = self.execute(self.cmd('version', path_repos))
|
||||||
|
self.assertEquals(fd.read().strip(), str(version))
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
# Apply versioning to DB
|
||||||
|
self.assertSuccess(self.cmd('version_control', self.url, path_repos, version))
|
||||||
|
|
||||||
|
# Test version number
|
||||||
|
fd = self.execute(self.cmd('db_version', self.url, path_repos))
|
||||||
|
self.assertEquals(fd.read().strip(), str(version))
|
||||||
|
self.assertSuccess(fd)
|
||||||
|
|
||||||
|
# Clean up
|
||||||
|
self.assertSuccess(self.cmd('drop_version_control', self.url, path_repos))
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_upgrade(self):
|
||||||
|
"""Can upgrade a versioned database"""
|
||||||
|
# Create a repository
|
||||||
|
repos_name = 'repos_name'
|
||||||
|
repos_path = self.tmp()
|
||||||
|
self.assertSuccess(self.cmd('create', repos_path,repos_name))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 0)
|
||||||
|
|
||||||
|
# Version the DB
|
||||||
|
self.exitcode(self.cmd('drop_version_control', self.url, repos_path))
|
||||||
|
self.assertSuccess(self.cmd('version_control', self.url, repos_path))
|
||||||
|
|
||||||
|
# Upgrades with latest version == 0
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
self.assertSuccess(self.cmd('upgrade', self.url, repos_path))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
self.assertSuccess(self.cmd('upgrade', self.url, repos_path, 0))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
self.assertFailure(self.cmd('upgrade', self.url, repos_path, 1))
|
||||||
|
self.assertFailure(self.cmd('upgrade', self.url, repos_path, -1))
|
||||||
|
|
||||||
|
# Add a script to the repository; upgrade the db
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % repos_path, 'Desc'))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 1)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
|
||||||
|
# Test preview
|
||||||
|
self.assertSuccess(self.cmd('upgrade', self.url, repos_path, 0, "--preview_sql"))
|
||||||
|
self.assertSuccess(self.cmd('upgrade', self.url, repos_path, 0, "--preview_py"))
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('upgrade', self.url, repos_path))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 1)
|
||||||
|
|
||||||
|
# Downgrade must have a valid version specified
|
||||||
|
self.assertFailure(self.cmd('downgrade', self.url, repos_path))
|
||||||
|
self.assertFailure(self.cmd('downgrade', self.url, repos_path, '-1', 2))
|
||||||
|
#self.assertFailure(self.cmd('downgrade', self.url, repos_path, '1', 2))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 1)
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('downgrade', self.url, repos_path, 0))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
|
||||||
|
self.assertFailure(self.cmd('downgrade',self.url, repos_path, 1))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('drop_version_control', self.url, repos_path))
|
||||||
|
|
||||||
|
def _run_test_sqlfile(self, upgrade_script, downgrade_script):
|
||||||
|
# TODO: add test script that checks if db really changed
|
||||||
|
|
||||||
|
repos_path = self.tmp()
|
||||||
|
repos_name = 'repos'
|
||||||
|
self.assertSuccess(self.cmd('create', repos_path, repos_name))
|
||||||
|
self.exitcode(self.cmd('drop_version_control', self.url, repos_path))
|
||||||
|
self.assertSuccess(self.cmd('version_control', self.url, repos_path))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 0)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url,repos_path), 0)
|
||||||
|
|
||||||
|
beforeCount = len(os.listdir(os.path.join(repos_path, 'versions'))) # hmm, this number changes sometimes based on running from svn
|
||||||
|
self.assertSuccess(self.cmd('script_sql', '--repository=%s' % repos_path, 'postgres'))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 1)
|
||||||
|
self.assertEquals(len(os.listdir(os.path.join(repos_path,'versions'))), beforeCount + 2)
|
||||||
|
|
||||||
|
open('%s/versions/001_postgres_upgrade.sql' % repos_path, 'a').write(upgrade_script)
|
||||||
|
open('%s/versions/001_postgres_downgrade.sql' % repos_path, 'a').write(downgrade_script)
|
||||||
|
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
self.assertRaises(Exception, self.engine.text('select * from t_table').execute)
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('upgrade', self.url,repos_path))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url,repos_path), 1)
|
||||||
|
self.engine.text('select * from t_table').execute()
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('downgrade', self.url, repos_path, 0))
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
self.assertRaises(Exception, self.engine.text('select * from t_table').execute)
|
||||||
|
|
||||||
|
# The tests below are written with some postgres syntax, but the stuff
|
||||||
|
# being tested (.sql files) ought to work with any db.
|
||||||
|
@fixture.usedb(supported='postgres')
|
||||||
|
def test_sqlfile(self):
|
||||||
|
upgrade_script = """
|
||||||
|
create table t_table (
|
||||||
|
id serial,
|
||||||
|
primary key(id)
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
downgrade_script = """
|
||||||
|
drop table t_table;
|
||||||
|
"""
|
||||||
|
self.meta.drop_all()
|
||||||
|
self._run_test_sqlfile(upgrade_script, downgrade_script)
|
||||||
|
|
||||||
|
|
||||||
|
@fixture.usedb(supported='postgres')
|
||||||
|
def test_sqlfile_comment(self):
|
||||||
|
upgrade_script = """
|
||||||
|
-- Comments in SQL break postgres autocommit
|
||||||
|
create table t_table (
|
||||||
|
id serial,
|
||||||
|
primary key(id)
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
downgrade_script = """
|
||||||
|
-- Comments in SQL break postgres autocommit
|
||||||
|
drop table t_table;
|
||||||
|
"""
|
||||||
|
self._run_test_sqlfile(upgrade_script,downgrade_script)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_command_test(self):
|
||||||
|
repos_name = 'repos_name'
|
||||||
|
repos_path = self.tmp()
|
||||||
|
|
||||||
|
self.assertSuccess(self.cmd('create', repos_path, repos_name))
|
||||||
|
self.exitcode(self.cmd('drop_version_control', self.url, repos_path))
|
||||||
|
self.assertSuccess(self.cmd('version_control', self.url, repos_path))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 0)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
|
||||||
|
# Empty script should succeed
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % repos_path, 'Desc'))
|
||||||
|
self.assertSuccess(self.cmd('test', repos_path, self.url))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 1)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
|
||||||
|
# Error script should fail
|
||||||
|
script_path = self.tmp_py()
|
||||||
|
script_text="""
|
||||||
|
from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
|
||||||
|
def upgrade():
|
||||||
|
print 'fgsfds'
|
||||||
|
raise Exception()
|
||||||
|
|
||||||
|
def downgrade():
|
||||||
|
print 'sdfsgf'
|
||||||
|
raise Exception()
|
||||||
|
""".replace("\n ","\n")
|
||||||
|
file = open(script_path, 'w')
|
||||||
|
file.write(script_text)
|
||||||
|
file.close()
|
||||||
|
|
||||||
|
self.assertFailure(self.cmd('test', repos_path, self.url, 'blah blah'))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 1)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path),0)
|
||||||
|
|
||||||
|
# Nonempty script using migrate_engine should succeed
|
||||||
|
script_path = self.tmp_py()
|
||||||
|
script_text="""
|
||||||
|
from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
|
||||||
|
meta = MetaData(migrate_engine)
|
||||||
|
account = Table('account',meta,
|
||||||
|
Column('id',Integer,primary_key=True),
|
||||||
|
Column('login',String(40)),
|
||||||
|
Column('passwd',String(40)),
|
||||||
|
)
|
||||||
|
def upgrade():
|
||||||
|
# Upgrade operations go here. Don't create your own engine; use the engine
|
||||||
|
# named 'migrate_engine' imported from migrate.
|
||||||
|
meta.create_all()
|
||||||
|
|
||||||
|
def downgrade():
|
||||||
|
# Operations to reverse the above upgrade go here.
|
||||||
|
meta.drop_all()
|
||||||
|
""".replace("\n ","\n")
|
||||||
|
file = open(script_path, 'w')
|
||||||
|
file.write(script_text)
|
||||||
|
file.close()
|
||||||
|
self.assertSuccess(self.cmd('test', repos_path, self.url))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path), 1)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url, repos_path), 0)
|
||||||
|
|
||||||
|
@fixture.usedb()
|
||||||
|
def test_rundiffs_in_shell(self):
|
||||||
|
# This is a variant of the test_schemadiff tests but run through the shell level.
|
||||||
|
# These shell tests are hard to debug (since they keep forking processes), so they shouldn't replace the lower-level tests.
|
||||||
|
repos_name = 'repos_name'
|
||||||
|
repos_path = self.tmp()
|
||||||
|
script_path = self.tmp_py()
|
||||||
|
old_model_path = self.tmp_named('oldtestmodel.py')
|
||||||
|
model_path = self.tmp_named('testmodel.py')
|
||||||
|
|
||||||
|
# Create empty repository.
|
||||||
|
self.meta = MetaData(self.engine, reflect=True)
|
||||||
|
self.meta.drop_all() # in case junk tables are lying around in the test database
|
||||||
|
self.assertSuccess(self.cmd('create',repos_path,repos_name))
|
||||||
|
self.exitcode(self.cmd('drop_version_control',self.url,repos_path))
|
||||||
|
self.assertSuccess(self.cmd('version_control',self.url,repos_path))
|
||||||
|
self.assertEquals(self.cmd_version(repos_path),0)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url,repos_path),0)
|
||||||
|
|
||||||
|
# Setup helper script.
|
||||||
|
model_module = 'testmodel:meta'
|
||||||
|
self.assertSuccess(self.cmd('manage',script_path,'--repository=%s --url=%s --model=%s' % (repos_path, self.url, model_module)))
|
||||||
|
self.assert_(os.path.exists(script_path))
|
||||||
|
|
||||||
|
# Write old and new model to disk - old model is empty!
|
||||||
|
script_preamble="""
|
||||||
|
from sqlalchemy import *
|
||||||
|
|
||||||
|
meta = MetaData()
|
||||||
|
""".replace("\n ","\n")
|
||||||
|
|
||||||
|
script_text="""
|
||||||
|
""".replace("\n ","\n")
|
||||||
|
open(old_model_path, 'w').write(script_preamble + script_text)
|
||||||
|
|
||||||
|
script_text="""
|
||||||
|
tmp_account_rundiffs = Table('tmp_account_rundiffs',meta,
|
||||||
|
Column('id',Integer,primary_key=True),
|
||||||
|
Column('login',String(40)),
|
||||||
|
Column('passwd',String(40)),
|
||||||
|
)
|
||||||
|
""".replace("\n ","\n")
|
||||||
|
open(model_path, 'w').write(script_preamble + script_text)
|
||||||
|
|
||||||
|
# Model is defined but database is empty.
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s compare_model_to_db' % (sys.executable, script_path))
|
||||||
|
assert "tables missing in database: tmp_account_rundiffs" in output, output
|
||||||
|
|
||||||
|
# Test Deprecation
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s compare_model_to_db --model=testmodel.meta' % (sys.executable, script_path))
|
||||||
|
assert "tables missing in database: tmp_account_rundiffs" in output, output
|
||||||
|
|
||||||
|
# Update db to latest model.
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s update_db_from_model' % (sys.executable, script_path))
|
||||||
|
self.assertEquals(exitcode, None)
|
||||||
|
self.assertEquals(self.cmd_version(repos_path),0)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url,repos_path),0) # version did not get bumped yet because new version not yet created
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s compare_model_to_db' % (sys.executable, script_path))
|
||||||
|
assert "No schema diffs" in output, output
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s create_model' % (sys.executable, script_path))
|
||||||
|
output = output.replace(genmodel.HEADER.strip(), '') # need strip b/c output_and_exitcode called strip
|
||||||
|
assert """tmp_account_rundiffs = Table('tmp_account_rundiffs', meta,
|
||||||
|
Column('id', Integer(), primary_key=True, nullable=False),
|
||||||
|
Column('login', String(length=None, convert_unicode=False, assert_unicode=None)),
|
||||||
|
Column('passwd', String(length=None, convert_unicode=False, assert_unicode=None)),""" in output.strip(), output
|
||||||
|
|
||||||
|
# We're happy with db changes, make first db upgrade script to go from version 0 -> 1.
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s make_update_script_for_model' % (sys.executable, script_path)) # intentionally omit a parameter
|
||||||
|
self.assertEquals('Not enough arguments' in output, True)
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s make_update_script_for_model --oldmodel=oldtestmodel:meta' % (sys.executable, script_path))
|
||||||
|
self.assertEqualsIgnoreWhitespace(output,
|
||||||
|
"""from sqlalchemy import *
|
||||||
|
from migrate import *
|
||||||
|
|
||||||
|
meta = MetaData()
|
||||||
|
tmp_account_rundiffs = Table('tmp_account_rundiffs', meta,
|
||||||
|
Column('id', Integer(), primary_key=True, nullable=False),
|
||||||
|
Column('login', String(length=40, convert_unicode=False, assert_unicode=None)),
|
||||||
|
Column('passwd', String(length=40, convert_unicode=False, assert_unicode=None)),
|
||||||
|
)
|
||||||
|
|
||||||
|
def upgrade(migrate_engine):
|
||||||
|
# Upgrade operations go here. Don't create your own engine; bind migrate_engine
|
||||||
|
# to your metadata
|
||||||
|
meta.bind(migrate_engine)
|
||||||
|
tmp_account_rundiffs.create()
|
||||||
|
|
||||||
|
def downgrade(migrate_engine):
|
||||||
|
# Operations to reverse the above upgrade go here.
|
||||||
|
meta.bind(migrate_engine)
|
||||||
|
tmp_account_rundiffs.drop()""")
|
||||||
|
|
||||||
|
# Save the upgrade script.
|
||||||
|
self.assertSuccess(self.cmd('script', '--repository=%s' % repos_path, 'Desc'))
|
||||||
|
upgrade_script_path = '%s/versions/001_Desc.py' % repos_path
|
||||||
|
open(upgrade_script_path, 'w').write(output)
|
||||||
|
#output, exitcode = self.output_and_exitcode('%s %s test %s' % (sys.executable, script_path, upgrade_script_path)) # no, we already upgraded the db above
|
||||||
|
#self.assertEquals(output, "")
|
||||||
|
output, exitcode = self.output_and_exitcode('%s %s update_db_from_model' % (sys.executable, script_path)) # bump the db_version
|
||||||
|
self.assertEquals(exitcode, None)
|
||||||
|
self.assertEquals(self.cmd_version(repos_path),1)
|
||||||
|
self.assertEquals(self.cmd_db_version(self.url,repos_path),1)
|
||||||
17
test/versioning/test_template.py
Normal file
17
test/versioning/test_template.py
Normal file
@ -0,0 +1,17 @@
|
|||||||
|
from test import fixture
|
||||||
|
from migrate.versioning.repository import *
|
||||||
|
import os
|
||||||
|
|
||||||
|
class TestPathed(fixture.Base):
|
||||||
|
def test_templates(self):
|
||||||
|
"""We can find the path to all repository templates"""
|
||||||
|
path = str(template)
|
||||||
|
self.assert_(os.path.exists(path))
|
||||||
|
def test_repository(self):
|
||||||
|
"""We can find the path to the default repository"""
|
||||||
|
path = template.get_repository()
|
||||||
|
self.assert_(os.path.exists(path))
|
||||||
|
def test_script(self):
|
||||||
|
"""We can find the path to the default migration script"""
|
||||||
|
path = template.get_script()
|
||||||
|
self.assert_(os.path.exists(path))
|
||||||
87
test/versioning/test_util.py
Normal file
87
test/versioning/test_util.py
Normal file
@ -0,0 +1,87 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from sqlalchemy import *
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
from migrate.versioning.util import *
|
||||||
|
|
||||||
|
|
||||||
|
class TestUtil(fixture.Pathed):
|
||||||
|
|
||||||
|
def test_construct_engine(self):
|
||||||
|
"""Construct engine the smart way"""
|
||||||
|
url = 'sqlite://'
|
||||||
|
|
||||||
|
engine = construct_engine(url)
|
||||||
|
self.assert_(engine.name == 'sqlite')
|
||||||
|
|
||||||
|
# keyword arg
|
||||||
|
engine = construct_engine(url, engine_arg_assert_unicode=True)
|
||||||
|
self.assertTrue(engine.dialect.assert_unicode)
|
||||||
|
|
||||||
|
# dict
|
||||||
|
engine = construct_engine(url, engine_dict={'assert_unicode': True})
|
||||||
|
self.assertTrue(engine.dialect.assert_unicode)
|
||||||
|
|
||||||
|
# engine parameter
|
||||||
|
engine_orig = create_engine('sqlite://')
|
||||||
|
engine = construct_engine(engine_orig)
|
||||||
|
self.assertEqual(engine, engine_orig)
|
||||||
|
|
||||||
|
# test precedance
|
||||||
|
engine = construct_engine(url, engine_dict={'assert_unicode': False},
|
||||||
|
engine_arg_assert_unicode=True)
|
||||||
|
self.assertTrue(engine.dialect.assert_unicode)
|
||||||
|
|
||||||
|
# deprecated echo= parameter
|
||||||
|
engine = construct_engine(url, echo='True')
|
||||||
|
self.assertTrue(engine.echo)
|
||||||
|
|
||||||
|
def test_asbool(self):
|
||||||
|
"""test asbool parsing"""
|
||||||
|
result = asbool(True)
|
||||||
|
self.assertEqual(result, True)
|
||||||
|
|
||||||
|
result = asbool(False)
|
||||||
|
self.assertEqual(result, False)
|
||||||
|
|
||||||
|
result = asbool('y')
|
||||||
|
self.assertEqual(result, True)
|
||||||
|
|
||||||
|
result = asbool('n')
|
||||||
|
self.assertEqual(result, False)
|
||||||
|
|
||||||
|
self.assertRaises(ValueError, asbool, 'test')
|
||||||
|
self.assertRaises(ValueError, asbool, object)
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_model(self):
|
||||||
|
"""load model from dotted name"""
|
||||||
|
model_path = os.path.join(self.temp_usable_dir, 'test_load_model.py')
|
||||||
|
|
||||||
|
f = open(model_path, 'w')
|
||||||
|
f.write("class FakeFloat(int): pass")
|
||||||
|
f.close()
|
||||||
|
|
||||||
|
FakeFloat = load_model('test_load_model.FakeFloat')
|
||||||
|
self.assert_(isinstance(FakeFloat(), int))
|
||||||
|
|
||||||
|
FakeFloat = load_model('test_load_model:FakeFloat')
|
||||||
|
self.assert_(isinstance(FakeFloat(), int))
|
||||||
|
|
||||||
|
FakeFloat = load_model(FakeFloat)
|
||||||
|
self.assert_(isinstance(FakeFloat(), int))
|
||||||
|
|
||||||
|
def test_guess_obj_type(self):
|
||||||
|
"""guess object type from string"""
|
||||||
|
result = guess_obj_type('7')
|
||||||
|
self.assertEqual(result, 7)
|
||||||
|
|
||||||
|
result = guess_obj_type('y')
|
||||||
|
self.assertEqual(result, True)
|
||||||
|
|
||||||
|
result = guess_obj_type('test')
|
||||||
|
self.assertEqual(result, 'test')
|
||||||
142
test/versioning/test_version.py
Normal file
142
test/versioning/test_version.py
Normal file
@ -0,0 +1,142 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
from test import fixture
|
||||||
|
from migrate.versioning.version import *
|
||||||
|
|
||||||
|
|
||||||
|
class TestVerNum(fixture.Base):
|
||||||
|
def test_invalid(self):
|
||||||
|
"""Disallow invalid version numbers"""
|
||||||
|
versions = ('-1', -1, 'Thirteen', '')
|
||||||
|
for version in versions:
|
||||||
|
self.assertRaises(ValueError, VerNum, version)
|
||||||
|
|
||||||
|
def test_is(self):
|
||||||
|
"""Two version with the same number should be equal"""
|
||||||
|
a = VerNum(1)
|
||||||
|
b = VerNum(1)
|
||||||
|
self.assert_(a is b)
|
||||||
|
|
||||||
|
self.assertEqual(VerNum(VerNum(2)), VerNum(2))
|
||||||
|
|
||||||
|
def test_add(self):
|
||||||
|
self.assertEqual(VerNum(1) + VerNum(1), VerNum(2))
|
||||||
|
self.assertEqual(VerNum(1) + 1, 2)
|
||||||
|
self.assertEqual(VerNum(1) + 1, '2')
|
||||||
|
self.assert_(isinstance(VerNum(1) + 1, VerNum))
|
||||||
|
|
||||||
|
def test_sub(self):
|
||||||
|
self.assertEqual(VerNum(1) - 1, 0)
|
||||||
|
self.assert_(isinstance(VerNum(1) - 1, VerNum))
|
||||||
|
self.assertRaises(ValueError, lambda: VerNum(0) - 1)
|
||||||
|
|
||||||
|
def test_eq(self):
|
||||||
|
"""Two versions are equal"""
|
||||||
|
self.assertEqual(VerNum(1), VerNum('1'))
|
||||||
|
self.assertEqual(VerNum(1), 1)
|
||||||
|
self.assertEqual(VerNum(1), '1')
|
||||||
|
self.assertNotEqual(VerNum(1), 2)
|
||||||
|
|
||||||
|
def test_ne(self):
|
||||||
|
self.assert_(VerNum(1) != 2)
|
||||||
|
self.assertFalse(VerNum(1) != 1)
|
||||||
|
|
||||||
|
def test_lt(self):
|
||||||
|
self.assertFalse(VerNum(1) < 1)
|
||||||
|
self.assert_(VerNum(1) < 2)
|
||||||
|
self.assertFalse(VerNum(2) < 1)
|
||||||
|
|
||||||
|
def test_le(self):
|
||||||
|
self.assert_(VerNum(1) <= 1)
|
||||||
|
self.assert_(VerNum(1) <= 2)
|
||||||
|
self.assertFalse(VerNum(2) <= 1)
|
||||||
|
|
||||||
|
def test_gt(self):
|
||||||
|
self.assertFalse(VerNum(1) > 1)
|
||||||
|
self.assertFalse(VerNum(1) > 2)
|
||||||
|
self.assert_(VerNum(2) > 1)
|
||||||
|
|
||||||
|
def test_ge(self):
|
||||||
|
self.assert_(VerNum(1) >= 1)
|
||||||
|
self.assert_(VerNum(2) >= 1)
|
||||||
|
self.assertFalse(VerNum(1) >= 2)
|
||||||
|
|
||||||
|
class TestVersion(fixture.Pathed):
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
super(TestVersion, self).setUp()
|
||||||
|
|
||||||
|
def test_str_to_filename(self):
|
||||||
|
self.assertEquals(str_to_filename(''), '')
|
||||||
|
self.assertEquals(str_to_filename('__'), '_')
|
||||||
|
self.assertEquals(str_to_filename('a'), 'a')
|
||||||
|
self.assertEquals(str_to_filename('Abc Def'), 'Abc_Def')
|
||||||
|
self.assertEquals(str_to_filename('Abc "D" Ef'), 'Abc_D_Ef')
|
||||||
|
self.assertEquals(str_to_filename("Abc's Stuff"), 'Abc_s_Stuff')
|
||||||
|
self.assertEquals(str_to_filename("a b"), 'a_b')
|
||||||
|
|
||||||
|
def test_collection(self):
|
||||||
|
"""Let's see how we handle versions collection"""
|
||||||
|
coll = Collection(self.temp_usable_dir)
|
||||||
|
coll.create_new_python_version("foo bar")
|
||||||
|
coll.create_new_sql_version("postgres")
|
||||||
|
coll.create_new_sql_version("sqlite")
|
||||||
|
coll.create_new_python_version("")
|
||||||
|
|
||||||
|
self.assertEqual(coll.latest, 4)
|
||||||
|
self.assertEqual(len(coll.versions), 4)
|
||||||
|
self.assertEqual(coll.version(4), coll.version(coll.latest))
|
||||||
|
|
||||||
|
coll2 = Collection(self.temp_usable_dir)
|
||||||
|
self.assertEqual(coll.versions, coll2.versions)
|
||||||
|
|
||||||
|
#def test_collection_unicode(self):
|
||||||
|
# pass
|
||||||
|
|
||||||
|
def test_create_new_python_version(self):
|
||||||
|
coll = Collection(self.temp_usable_dir)
|
||||||
|
coll.create_new_python_version("foo bar")
|
||||||
|
|
||||||
|
ver = coll.version()
|
||||||
|
self.assert_(ver.script().source())
|
||||||
|
|
||||||
|
def test_create_new_sql_version(self):
|
||||||
|
coll = Collection(self.temp_usable_dir)
|
||||||
|
coll.create_new_sql_version("sqlite")
|
||||||
|
|
||||||
|
ver = coll.version()
|
||||||
|
ver_up = ver.script('sqlite', 'upgrade')
|
||||||
|
ver_down = ver.script('sqlite', 'downgrade')
|
||||||
|
ver_up.source()
|
||||||
|
ver_down.source()
|
||||||
|
|
||||||
|
def test_selection(self):
|
||||||
|
"""Verify right sql script is selected"""
|
||||||
|
|
||||||
|
# Create empty directory.
|
||||||
|
path = self.tmp_repos()
|
||||||
|
os.mkdir(path)
|
||||||
|
|
||||||
|
# Create files -- files must be present or you'll get an exception later.
|
||||||
|
python_file = '001_initial_.py'
|
||||||
|
sqlite_upgrade_file = '001_sqlite_upgrade.sql'
|
||||||
|
default_upgrade_file = '001_default_upgrade.sql'
|
||||||
|
for file_ in [sqlite_upgrade_file, default_upgrade_file, python_file]:
|
||||||
|
filepath = '%s/%s' % (path, file_)
|
||||||
|
open(filepath, 'w').close()
|
||||||
|
|
||||||
|
ver = Version(1, path, [sqlite_upgrade_file])
|
||||||
|
self.assertEquals(os.path.basename(ver.script('sqlite', 'upgrade').path), sqlite_upgrade_file)
|
||||||
|
|
||||||
|
ver = Version(1, path, [default_upgrade_file])
|
||||||
|
self.assertEquals(os.path.basename(ver.script('default', 'upgrade').path), default_upgrade_file)
|
||||||
|
|
||||||
|
ver = Version(1, path, [sqlite_upgrade_file, default_upgrade_file])
|
||||||
|
self.assertEquals(os.path.basename(ver.script('sqlite', 'upgrade').path), sqlite_upgrade_file)
|
||||||
|
|
||||||
|
ver = Version(1, path, [sqlite_upgrade_file, default_upgrade_file, python_file])
|
||||||
|
self.assertEquals(os.path.basename(ver.script('postgres', 'upgrade').path), default_upgrade_file)
|
||||||
|
|
||||||
|
ver = Version(1, path, [sqlite_upgrade_file, python_file])
|
||||||
|
self.assertEquals(os.path.basename(ver.script('postgres', 'upgrade').path), python_file)
|
||||||
14
test_db.cfg
Normal file
14
test_db.cfg
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
# test_db.cfg
|
||||||
|
#
|
||||||
|
# This file contains a list of connection strings which will be used by
|
||||||
|
# database tests. Tests will be executed once for each string in this file.
|
||||||
|
# You should be sure that the database used for the test doesn't contain any
|
||||||
|
# important data. See README for more information.
|
||||||
|
#
|
||||||
|
# The string '__tmp__' is substituted for a temporary file in each connection
|
||||||
|
# string. This is useful for sqlite tests.
|
||||||
|
sqlite:///__tmp__
|
||||||
|
postgres://migrate:UPd2icyw@localhost/migrate_test
|
||||||
|
mysql://migrate:fTP82sjf@localhost/migrate_test
|
||||||
|
oracle://migrate:FdnjJK8s@localhost
|
||||||
|
firebird://migrate:BowV7EEm@localhost//var/db/migrate.gdb
|
||||||
13
test_db.cfg.tmpl
Normal file
13
test_db.cfg.tmpl
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
# test_db.cfg
|
||||||
|
#
|
||||||
|
# This file contains a list of connection strings which will be used by
|
||||||
|
# database tests. Tests will be executed once for each string in this file.
|
||||||
|
# You should be sure that the database used for the test doesn't contain any
|
||||||
|
# important data. See README for more information.
|
||||||
|
#
|
||||||
|
# The string '__tmp__' is substituted for a temporary file in each connection
|
||||||
|
# string. This is useful for sqlite tests.
|
||||||
|
sqlite:///__tmp__
|
||||||
|
postgres://scott:tiger@localhost/test
|
||||||
|
mysql://scott:tiger@localhost/test
|
||||||
|
oracle://scott:tiger@localhost
|
||||||
Loading…
Reference in New Issue
Block a user