Deployment as it should be

In the development lifecycle of web applications, I think deployment is something that doesn't get nearly enough attention of developers. But where you can test your code, testing deployment is a bit harder. A good system for deploying applications is therefore a useful thing to have. Symfony has a very easy and strong system for deploying applications based on rsync.

Most web developers have a lot of focus for writing quality applications. They ensure their markup and style is displaying correctly in multiple browsers, and they focus on their code, some even write unit tests and functional tests ;)

Additional measures are having seperate development, test, staging and production servers. Having seperate test teams. Have client tests. But all this focusses on the workings and looks of the application.

I've worked for quite a few companies, and I've seen some methods of deploying. The worst was using ftp for deploying an application. Usually this resulted in errors as parts of the application were updated already and other parts weren't. And especially with bigger applications, this causes problems for a longer period of time.

Slightly less bad was deploying from version control (CVS or svn). Usually we'd make an export in a temporary place and then simply move the files to the correct position. This works pretty OK I must say, but when moving away the old version (backup!) uploaded files were also gone and so this required quite a lot of work.

One of the options of symfony's commandline interface is the ability to deploy applications to servers using the symfony sync command. In a configuration file, you can define a list of servers that you want to deploy to. Each server has a snippet in there similar to this:

[production]
host=ftp.leftontheweb.com
port=22
user=skoop
dir=/home/skoop/symfony_sites/www.leftontheweb.com

As you see, this is basically all information you need to deploy an application, aside of course from the password which you wouldn't want to specify in a configuration file if it isn't required. Instead, as soon as you issue the symfony sync command for this server, it will ask you for your password. There are, in the case of the above server, two commands you can issue:

symfony sync production
This will do a dry run, to see if the deployment of the files might cause a problem

symfony sync production go
This will actually send the files over to the specified server.

Now, because the symfony sync command uses rsync, only those files that have changed are deployed to the specified server. Aside from the first deployment, in which case all files need to be sent to the server, this speeds up deployment big time as only a selection of files are transferred.

Aside from easily transferring the necessarily files, symfony also makes sure that the files you don't want to have deployed won't. For this, there is the rsync_exclude.txt file. In this file is a simple list of files (which can include wildcards, so you can exclude whole directories or files with a certain naming convention) that should not be deployed. Good examples of files you'll want listed in this file are the _dev.php development environments, your databases.yml (to ensure you don't overwrite your database configuration with an incorrect one) and your cache and log directories (listed by default).

Now we have transferring files covered. But database changes also need to happen as you deploy of course, because your new code will depend on those. In symfony 1.0, I personally always use batch job which can be called from the commandline. In symfony 1.1 it will be even easier, as the recommended way to do it then is to use actual symfony tasks (which can easily be written, in 1.0 already by the way).

So, in short, deployment will then be:

  • deploy code using symfony sync
  • call the batch job/symfony task to update the database and possibly do other required stuff
  • symfony cc (clear cache) - though this could be done from the above batch job/symfony task as well

And what is left is testing if everything went well.

Of course, having a powerful tool such as this at your fingertips does still require you to keep your head clear and keep thinking about what you are doing, but it definitely makes it much easier and ensures you don't have to think about everything.


Add comment

Comments

gravatar Ivo: Does it support clustered environments? In other words, can 'production' be linked to more than one host? (assuming a cluster that doesn't have a shared storage).

On the one hand, it feels like the same as deploying over ftp, only with a script; I prefer a sandboxed environment that can be updated through source control, but deploying with scripts still beats deploying manually.
January 30, 2008
gravatar left: Ivo: No, a cluster in this case will be a single server. Of course, since symfony is very extendable, the symfony sync task could easily be extended to a symfony cluster-sync task or something that goes over a list of servers instead of one, while still taking advantage of the original sync task. It is not there by default, but would make an excellent plugin :)

there's quite some differences (I tried to point some out) with deploying over ftp, for instance the incremental filetransfer (so only changed files get updated) and related to that the speed. And maybe inherent to the way symfony teaches you to do it, but it requires you to think upfront on what needs to be done. With FTP you can do that however, but still a lot of people don't actually *do* that. But that's less how you do it, more related to the approach to professional deployment I guess.
January 30, 2008
gravatar James Moey: I use SVN for deployment all the time, checkout or export depending on the situation. Sometime SVN+rsync deployment is required. Anyway, I just want to say "Slightly less bad was deploying from version control (CVS or svn)" is incorrect. If you are using CVS or SVN for your project from the beginning which you should do. There are little chance that you overwrite files as all changes are track in the repository. Plus CVS/SVN is not bound to a framework, a tool that you can use for all your project, no matter what environment you are in. So how can a useful tool like that be slightly less bad that FTP? It should be better or equal to symfony sync solution depending on the situation.
January 31, 2008
gravatar Digitalbase: We also work with SVN and a Rsync script to transfer everything to a live server. It does the job as it should be.
However, it's a great thing that Symfony offers a solution to those who don't have a SVN server.
January 31, 2008
gravatar Bert-Jan: I use sync for deployment too but there is a major snag in the rsync protocol: it uses a very primitive way of identifying which files have been changed: ls -l
The trouble this brings is when you develop a project with the help of some others who can sync from their dev server, the timestamps etc. don't match yours anymore. When you make even a tiny modification and sync it, rsync will not recognise any of the files on the prod server and simply upload *everything*. If you sync again after that the incremental principle kicks in because all files are in sync with *your* server. Now when the 'other guys' run a sync *they* have to upload everything.
Quite disappointing to be honest...
January 31, 2008
gravatar Junni: I always do the upload of the files manually. Since I know that my ftp client somethings skips certain subdirectories (especially in de web- and plugindir), I'm going to give the Symfony sync tools a try.
January 31, 2008
gravatar Thierry: Symfony sync is a great tool. Rsync is really nice for syncing files.

Currently I use svn though, also works like a charm. A simple svn up on the server and you are done. Very similar to rsync in that it only does the changed files.
January 31, 2008
gravatar Jay Klehr: Bert-Jan: Look into the rsync -c flag, it does a checksum comparison on the files, instead of a modification timestamp check.
January 31, 2008
gravatar Bert-Jan: Jay: thank you. I just found that out myself before I read you comment here. I also found out you can set the parameters symfony passes to rsync with a 'parameters' attribute in properties.ini. I haven't yet tested if it works as expected.
February 4, 2008
gravatar Jason: great article! good points made by commenters!

if you work on large systems, and are using load-balancing or clustering, a reasonable solution is also to write your own set of symfony deployment tasks for the capistrano system.

then you get atomic releases which can be rolled back instantaneously, hooks for batches, and incremental updating which is not reliant on the filesystem to determine modification (either with timestamp or size with rsync).

mostly all you'll use in capistrano are variable names in strings to parameterize shell commands, so don't be afraid that it's in ruby and not php. I built out my deployment system in about 2 hours, and it works like a charm, with database migration hooks and all.

Cheers!
February 8, 2008
gravatar left: Hi Jason,

Thanks for that. I've seen mentions before of using Capistrano for PHP deployment, I guess I need to dive into that. Yet one more thing to add to my list of things to do ;)
February 8, 2008

Php5_zce_logo

not tested in IE


Upcoming events

I will be speaking 04-09-2009: Symfony Day 2009

Tags

1337 2008 4developers accessibility AdaLovelaceDay09 agavi agile amsterdam apache apple article articles atk atkMetaNode audioscrobbler backwards compatibility bbc beatstad best practices bittorrent book books bughuntday caching cake cal evans cat cerf certificate cfp clear cms community conference conferences continuous integration crisis css custom DbFinderPlugin decorator decorators deployment devdays development directoryindex documentation download dpc dpc09 DPC2008 dreamhost eclipse ed efficiency enterprise event events expertise flickr frameworks freeze frontend fun games germany getting real google googletalk graceful degradation hack hackers howto html http ibuildings icann ide imovie indy internet IPC ipc ipc08 javascript jobeet john peel joomla left on the web lime linux live london loudblog m2ts mac malware mambo marjolein meme meta methodology microsoft movie music mysql namespace namespaces nllgg odmarco open source ORM osx paradiso pear personal pfcongrez photo php phpabstract phpBB phpbb phpbelgium phpbenelux phpgg phpitalia phpnw phpnw08 phptek phptek09 phpuk2009 phpUnderControl phpunit php|architect php|tek podcast politics portability postcrossing presentation presentations public qa recruiting refactoring review rewrite ruby on rails schedule script security seven things simplexml slides software sogeti solar standard standards static steer subversion symfony symfonycamp symfonyUnderControlPlugin talk technology techportal tek09 terratec terrorism testfest testing textpattern tips tld tomas usability usergroup vhost video vinyl virus warp weblogging wiki women world world of warcraft writing xml xpath yara year youtube ZCE zemanta zend zend framework zend server zend studio Zend_Form
© 2004 - 2009 Stefan Koopmanschap + Powered by Symfony, photos powered by Flickr, links powered by Delicious, Shanghai smilies by Iconbuffet. Feeds: rss / atom. Left on the Web v4.4.0