Some Major Issues while Migrations in Django

Developing Any Web apps “Data Base” is one of the major part of that . Sometimes its very tedious to connect and managing the DB as per requirement . Python Django gives a super facility for this, it is known as "migration". Migrations are Django’s way of propagating changes we create in our models (adding a field, deleting a model, etc.) into our database schema. They’re designed to be mostly automatic, but we have to know when to form migrations, when to run them, and therefore the common problems we would possibly run into.

There are two defined Python command to perform above functions.

1. python manage.py makemigrations app_name

2. python manage.py migrate app_name

First command generates the python code of your app's model with all the key, relation and field name as follow,

Creating New model :-

operations = [

migrations.CreateModel(

name='NewThing',

fields=[

...

]

]

Altering/Updating model:-

operations = [

migrations.AddField(

model_name='xxxxx',

name='xxxxx',

field=models.ManyToManyField(to='xxxxx'),

),

]

Second python command is responsible the applying and unapplying the above migrations code for actual data in defined DB by the help of Django ORM or Object-relational Mapper.

Some important key issue during executing migrations in Django.

1.Table name already exist : -This happen when the migration files executed an action to make a model but that table already present at same location within the database,So django migration neglect the action and shows the error.

2.The field which will be created already exists : - Some times we tries to create a field that already present in that respective model/table.

3. Django register column and relation as non-existent :- . This happen when the migration adds a key or constraint to a particular field but django couldn’t execute this because the column referenced foreign key doesn’t exist. In this situation we've to switch the table’s field manually.

These above points are some general issue that we find. Except these, some another higher level of root cause are also there as follows:--

1. When two or more than Developer works in same branch of VCS and pull the code of others , in this condition if they didn't run the migrations command in their local system , then it may cause of issue.

2.The field which will be created already exists : - Some times we tries to create a field that already present in that respective model/table.

3. Django register column and relation as non-existent :- . This happen when the migration adds a key or constraint to a particular field but django couldn’t execute this because the column referenced foreign key doesn’t exist. In this situation we've to switch the table’s field manually.

These above points are some general issue that we find. Except these, some another higher level of root cause are also there as follows:--

1. When two or more than Developer works in same branch of VCS and pull the code of others , in this condition if they didn't run the migrations command in their local system , then it may cause of issue.

2. When the developer pushes their code on "DEV" and forget to create migration on dev , at the other end if another developer push the code and create the migration then it may also cause the isuue. To get rid of this issue the best practice is during pushing the code and making a new build always run migrations command for all the installed apps for our project.

3. Deleting relation of the Table :- As per our requirement we make a relation as "Foreign key", "ManyToMany Field" etc and on later if we change the relation and tried to delete the table physically , it may also cause the major issue. So best practice is if you don't have deep knowledge of migrations function then never create,update or delete the table or field directly from the DB.

4. Circular Dependencies :- Some times we create the model of two different apps of the project and create a relation to each other and then make a migration. In this situation it may cause the "Relation Not found" , "Table Not found" .

Some times it does not cause an error during migrations , but if we get migration issue and try to make it fake migrations then ,It cause an issue as "Circular Dependencies " . To overcome from this issue we have to comment out all the relation from the model and make a migrations(makemigrations and migrate both) and after creating respective table, again uncomment the commented relation and make a migrations.

Followings are most importent points for avoiding the migrations issue in Django.

1. Make sure and cordinate with your team when modifying models. this may reduce the changes conflict with each other .

2. Commit migration file into repo. this might be tricky in local development — every developer has got to pull and apply changes from other environments to his local one. However, this may keep the code up-to-date and reduce conflicts within the main file .

3. Squash migration periodically. it'll cut the time utilised in deploying the migration commands. Django runs each migration file, one by one, from starting to the present state of the database schema.

4. When conflict occurs, save the migration files on both test and deploy environments. This way, you'll observe conflict issues.

5. Separate migration command per apps to avoid circular dependency. When running apps, run those with no dependency first, followed by apps that are associated with it.

6. Deploy sequentially from development, staging then production. don't skip any releases which will cause the database state to differ in each stage. you'll not need to treat and migrate each release differently.

Conclusion

The good knowledge working of migrations file is the key of all the above issue and solution. Some times we failed in finding the exact issue and and its solution, in this situation we have a last option that delete our all migrations file from the migration folder of respective app (Note:- Never delete the __init__.py) and drop your DB and perform the makemigrations and migrate again. Simultaneously we have to keep in mind that there is no any circualr dependencies at this time, if there is a relation then first comment those relation field.

There is also an another method in which no need of droping the DB i.e we can preserve our saved data. This method is known as creating –FAKE MIGRATIONS by following steps.

First run the "showmigrations" command so that we can see the list of all migrations:

1. python manage.py showmigrations

Result::

admin

[X] 0001_initial

[X] 0002_logentry_remove_auto_add

auth

[X] 0001_initial

[X] 0002_alter_permission_name_max_length

[X] 0003_alter_user_email_max_length

[X] 0004_alter_user_username_opts

[X] 0005_alter_user_last_login_null

[X] 0006_require_contenttypes_0002

[X] 0007_alter_validators_add_error_messages

contenttypes

[X] 0001_initial

[X] 0002_remove_content_type_name

app1

[X] 0001_initial

[X] 0002_remove_mymodel_i

[X] 0003_mymodel_bio

sessions

[X] 0001_initial

2. Clear the migration history (please note that app1 is the name of my app):

python manage.py migrate --fake app1 zero

Operations to perform:

Unapply all migrations: core

Running migrations:

Rendering model states... DONE

Unapplying core.0003_mymodel_bio... FAKED

Unapplying core.0002_remove_mymodel_i... FAKED

Unapplying core.0001_initial... FAKED

3. python manage.py showmigrations

Result::

admin

[X] 0001_initial

[X] 0002_logentry_remove_auto_add

auth

[X] 0001_initial

[X] 0002_alter_permission_name_max_length

[X] 0003_alter_user_email_max_length

[X] 0004_alter_user_username_opts

[X] 0005_alter_user_last_login_null

[X] 0006_require_contenttypes_0002

[X] 0007_alter_validators_add_error_messages

contenttypes

[X] 0001_initial

[X] 0002_remove_content_type_name

app1

[ ] 0001_initial

[ ] 0002_remove_mymodel_i

[ ] 0003_mymodel_bio

sessions

[X] 0001_initial

4. Remove the actual migration files.

Go to projects apps migration folder and remove everything inside, except for the init file.

5. python manage.py showmigrations

Result::

admin

[X] 0001_initial

[X] 0002_logentry_remove_auto_add

auth

[X] 0001_initial

[X] 0002_alter_permission_name_max_length

[X] 0003_alter_user_email_max_length

[X] 0004_alter_user_username_opts

[X] 0005_alter_user_last_login_null

[X] 0006_require_contenttypes_0002

[X] 0007_alter_validators_add_error_messages

contenttypes

[X] 0001_initial

[X] 0002_remove_content_type_name

app1

(no migrations)

sessions

[X] 0001_initial

6. Create the initial migrations

python manage.py makemigrations

Result::

Migrations for 'app1':

0001_initial.py:

- Create model MyModel

7. Fake the initial migration

In this case the database table already exists so we won’t be able to apply the initial migration. So What we want to do is to fake this all migrations:

python manage.py migrate –fake-initial

Result::

Operations to perform:

Apply all migrations: admin, core, contenttypes, auth, sessions

Running migrations:

Rendering model states... DONE

Applying core.0001_initial... FAKED

8. python manage.py showmigrations

admin

[X] 0001_initial

[X] 0002_logentry_remove_auto_add

auth

[X] 0001_initial

[X] 0002_alter_permission_name_max_length

[X] 0003_alter_user_email_max_length

[X] 0004_alter_user_username_opts

[X] 0005_alter_user_last_login_null

[X] 0006_require_contenttypes_0002

[X] 0007_alter_validators_add_error_messages

contenttypes

[X] 0001_initial

[X] 0002_remove_content_type_name

app1

[X] 0001_initial

sessions

[X] 0001_initial