Reinout Slot Cms
We’ve got a django-cms site that started out on sqlite. Now we’re adding GISshapefiles so we wanted to move to postgres/postgis. Ok, switch over theDATABASES definition in settings.py, make a fixture and load it,right? Ahem, no.
Migrating django(-cms) from sqlite to postgres¶ Tags: django. We’ve got a django-cms site that started out on sqlite. Now we’re adding GIS shapefiles so we wanted to move to postgres/postgis. Ok, switch over the DATABASES definition in settings.py, make a fixture and load it, right?
- CMS is praised for its 'great mix of associate and partner involvement' and for 'getting things done'. Roman Tarlavski, who leads the group, handles cross-border M&A, structured joint ventures and private equity transactions, while Reinout Slot 's practice encompasses corporate and securities law with an emphasis on private and public equity.
- Cms Reinout Slot, raul martinez gallego poker, apo racing slot, klamath california casino.
There were a couple of problems, partially with our code, partiallydjango-cms, partially django. Some solutions below might not be needed inevery case, but I’m just going to list everything in the hope that it helpssomeone.
South migration dependencies¶
Lots of our code uses south for databasemigrations. But when creating our database from scratch, “migrate” wouldcomplain that a certain table of application A didn’t exist yet when migratingapplication B. Uh oh, a dependency.
Turns out that’s easily solved. In the failing migration step, just add adependency on the missing application’s relevant step:
After that, all migrations run fine.
Small sqldiff limitation¶
We’ve got django-extensions in our projects. One of thehandy extra management commands itprovides is “sqldiff” (so manage.pysqldiff or, as we use buildout,bin/djangosqldiff). (The best extension is “graph_models” which createsa picture of your model structure).
We got some complaints when importing a fixture. In the end it turned out tobe that someone somehow copied an old and wrong fixture instead of a freshone. But we suspected that some older migration had mis-fired. “sqldiff”will tell you the difference between the current database schema inside theactual database and what it should be according to django.
We got a bunch of missing fields in some custom user model inside django-cms!What! Faulty south migrations? What?
In the end, it turned out to be a false alarm. Django-cms’s user modelinherits from django’s own user model and adds one field. In the database thisends up as a small table with a pointer to a corresponding django user item’sID and the one single extra field. But sqldiff complains about all the regulardjango user model fields being missing :-)
Note: I don’t fault django-extensions very much here. They note it is anexperimental feature and we just happened to hit a corner case. Most of thetime it is a real helper!
IntegrityError: duplicate key value violates unique constraint¶
See for instance a stackoverflow questionand especially Mark van Lent’s recent explanation of what can happen.
There are two basic ways, I think, such an IntegrityError can crop up whenloading a fixture:
You’re loading them out of order or with gaps in the primary key IDs. Thiscan mess up the automatic ID generation.
You’ve really already got an item with that ID in the database and thecomplaint is right.
IntegrityError solution 1: natural keys¶
See django ticket #7052. Solved. Solved, that is, whenthe model that gives you a problem supports so-called naturalkeys. Basically, your fixture doesn’t point at a user with primary key “54”,but at a user with username “pietje”, whatever the ID.
This takes away a too-hard dependency on primary key ID numbers. To get itgoing in your fixture, add the --natural flag:
IntegrityError solution 2: make sure there are no duplicates¶
One of our problems was that django’s content_type table was alreadypre-filled with several items after the regular “syncdb”. Quite sensibly. Butit conflicted when loading our fixture. (It might be that the natural keysolution mentioned above would have solved this, but we tried it in adifferent order).
The solution: remove those content types from the table before loading thefixture. The fixture is a full fixture of everything in the database, sothat’s OK.
In your manage.pydbshell or bin/djangodbshell, run:
IntegrityError solution 3: split the fixture¶
Django-cms’s cms.placeholder items continued to give grief no matterwhat. Perhaps they ought to start to support the natural key solution? Notsure if that would help, but I guess so.
The solution? Grab the whole exported fixture and split that all.json intotwo json files, one with only “cms.placeholder” items and one withoutthem. (Make sure the last dict-like item in the json should be without aclosing comma!
all_only_placeholders.json:
all_without_placeholders.json:
After that, load the data:
Closing comment¶
In the end, we got it running. I hope someone gets a good tip out of thisentry when faced with a similar problem.
One django advantage I noticed: everything is visible. You can alwaysinspect the source code. “What exactly is in that one model that’s giving megrief?” “Which migration steps are available?” Etc.
The advantage of open source!
We’ve got a django-cms site that started out on sqlite. Now we’re adding GISshapefiles so we wanted to move to postgres/postgis. Ok, switch over theDATABASES definition in settings.py, make a fixture and load it,right? Ahem, no.
There were a couple of problems, partially with our code, partiallydjango-cms, partially django. Some solutions below might not be needed inevery case, but I’m just going to list everything in the hope that it helpssomeone.
Reinout Slot Cms Definition
South migration dependencies¶
Lots of our code uses south for databasemigrations. But when creating our database from scratch, “migrate” wouldcomplain that a certain table of application A didn’t exist yet when migratingapplication B. Uh oh, a dependency.
Turns out that’s easily solved. In the failing migration step, just add adependency on the missing application’s relevant step:
After that, all migrations run fine.
Small sqldiff limitation¶
We’ve got django-extensions in our projects. One of thehandy extra management commands itprovides is “sqldiff” (so manage.pysqldiff or, as we use buildout,bin/djangosqldiff). (The best extension is “graph_models” which createsa picture of your model structure).
We got some complaints when importing a fixture. In the end it turned out tobe that someone somehow copied an old and wrong fixture instead of a freshone. But we suspected that some older migration had mis-fired. “sqldiff”will tell you the difference between the current database schema inside theactual database and what it should be according to django.
We got a bunch of missing fields in some custom user model inside django-cms!What! Faulty south migrations? What?
In the end, it turned out to be a false alarm. Django-cms’s user modelinherits from django’s own user model and adds one field. In the database thisends up as a small table with a pointer to a corresponding django user item’sID and the one single extra field. But sqldiff complains about all the regulardjango user model fields being missing :-)
Note: I don’t fault django-extensions very much here. They note it is anexperimental feature and we just happened to hit a corner case. Most of thetime it is a real helper!
IntegrityError: duplicate key value violates unique constraint¶
See for instance a stackoverflow questionand especially Mark van Lent’s recent explanation of what can happen.
There are two basic ways, I think, such an IntegrityError can crop up whenloading a fixture:
You’re loading them out of order or with gaps in the primary key IDs. Thiscan mess up the automatic ID generation.
You’ve really already got an item with that ID in the database and thecomplaint is right.
IntegrityError solution 1: natural keys¶
See django ticket #7052. Solved. Solved, that is, whenthe model that gives you a problem supports so-called naturalkeys. Basically, your fixture doesn’t point at a user with primary key “54”,but at a user with username “pietje”, whatever the ID.
This takes away a too-hard dependency on primary key ID numbers. To get itgoing in your fixture, add the --natural flag:
IntegrityError solution 2: make sure there are no duplicates¶
One of our problems was that django’s content_type table was alreadypre-filled with several items after the regular “syncdb”. Quite sensibly. Butit conflicted when loading our fixture. (It might be that the natural keysolution mentioned above would have solved this, but we tried it in adifferent order).
The solution: remove those content types from the table before loading thefixture. The fixture is a full fixture of everything in the database, sothat’s OK.
Reinout Slot Cms App
In your manage.pydbshell or bin/djangodbshell, run:
IntegrityError solution 3: split the fixture¶
Django-cms’s cms.placeholder items continued to give grief no matterwhat. Perhaps they ought to start to support the natural key solution? Notsure if that would help, but I guess so.
Reinout Slot Cms Software
The solution? Grab the whole exported fixture and split that all.json intotwo json files, one with only “cms.placeholder” items and one withoutthem. (Make sure the last dict-like item in the json should be without aclosing comma!
all_only_placeholders.json:
all_without_placeholders.json:
After that, load the data:
Closing comment¶
Reinout Slot Cms Login
In the end, we got it running. I hope someone gets a good tip out of thisentry when faced with a similar problem.
One django advantage I noticed: everything is visible. You can alwaysinspect the source code. “What exactly is in that one model that’s giving megrief?” “Which migration steps are available?” Etc.
The advantage of open source!