A Liquibase pitfall
I will tell you about a not very common Liquibase pitfall what happened to me lately.
Liquibase has a property for its changesets: failOnError. You can set it to false, if you expect that your changeset can eventually fail. How it comes?
End of the World is coming
Well, imagine following scenario: you create a new Entity table with the common column: ID. This is auto-generated during insertion (eventually managed by Hibernate) therefore you add a sequence to the database. You commit the changes, it get’s deployed to your colleagues, eventually QA servers and you get a bug ticket mentioning: you made a mistake in the sequence (for example it increments with 20 values instead of 1 — ist a bogus example but can happen 🙂 ).
So you create a new changeset, drop the existing sequence and in the next change you create a new one — and with this move you also delete the old changeset from the repository. Tests pass on your development environment, you check in everything is fine.
But soon you get another ticket: it’s not possible to execute your changes on the pre-live system because your sequence dropping change fails — because there’s no such sequence (because the application was not deployed in the mean time to this environment). How can you fix the problem? Yes, you set failOnError to false. Because Liquibase executes this changeSet and
Hurray you saved the world.
The End of the World
Your code was deployed into production without any problems. You are happy, everyone is happy.
After some time you get a new customer for the same application. This is a regular work: you take your application develop some new features if needed, deploy it to the QA systems and do your work.
And you get a bug report. The application throws an exception because your sequence is missing. You do not get the point, because you did everything fine, and you have at least one System running in production mode with your sequence. Nobody has a clue — so the end of the world came a bit later. You do a quick fix and create the sequence in the database not to break the work of your QA — but you look for the cause.
If you are “lucky” your change was way back so you have to look up the flow of the changes from your commit up to today. But there’s no one involving your sequence. In the meantime the QA server got a new database (or at least the whole schema was dropped) and you get the same error message that your sequence is missing and it leads to malfunction.
So you take the database, open the DatabaseChangeLog table and search for the execution… And it is there, your changeset, which is supposed to be marked ran but… The execution timestamp: it is later than the others. In the database where the long productive version is running is the timestamp before the one, which creates the sequence…
This is a superb question. I asked myself for an hour or so how can this be possible. Than I realized something. In the mean time (between deploying the changeset into production and preparing the new customer’s version) we changed Liquibase from 1.9 to 2.0.
And as it is in 2.0 failOnError=false fails not but it does not save the changeset into the database. So when you run your migration again then the statement is executed too.
So the scenario:
- Drop sequence changeset fails but nothing groovy happens
- Sequence is created
- Re-run the migration
- Drop sequence changeset is executed successfully –> your sequence is dropped
- The system crashes
And worry: if you do the same with only Liquibase 2.0 you will get the same malfunction.
I created a sample Liquibase XML file with 3 changesets where you can try this problem on your own database, you can find it at my gitHub repository for the variations project.
After you run the whole XML file for the first time, only the changeset with the id “pitfall2” will show up in your DatabaseChangeLog table — and you get a wrong_seq named sequence.
Running the changesets for a second time you’ll find a new entry in your table with the id “pitfall”. And your sequence is lost.
And what is with “pitfall3”? It will never fully execute because the dropSequence fails and there stops the execution and there will be no “wrong_seq2” in your database.
Never ever start your chain of changesets with a drop or a delete with failOnError=false. Eventually you could set up a precondition but sometimes this can lead to heavy queries against you database because liquibase looks in the system tables to validate your precondition.