Deployment as it should be

In the development lifecycle of web applications, I think deployment is something that doesn't get nearly enough attention of developers. But where you can test your code, testing deployment is a bit harder. A good system for deploying applications is therefore a useful thing to have. Symfony has a very easy and strong system for deploying applications based on rsync.

Most web developers have a lot of focus for writing quality applications. They ensure their markup and style is displaying correctly in multiple browsers, and they focus on their code, some even write unit tests and functional tests ;)

Additional measures are having seperate development, test, staging and production servers. Having seperate test teams. Have client tests. But all this focusses on the workings and looks of the application.

I've worked for quite a few companies, and I've seen some methods of deploying. The worst was using ftp for deploying an application. Usually this resulted in errors as parts of the application were updated already and other parts weren't. And especially with bigger applications, this causes problems for a longer period of time.

Slightly less bad was deploying from version control (CVS or svn). Usually we'd make an export in a temporary place and then simply move the files to the correct position. This works pretty OK I must say, but when moving away the old version (backup!) uploaded files were also gone and so this required quite a lot of work.

One of the options of symfony's commandline interface is the ability to deploy applications to servers using the symfony sync command. In a configuration file, you can define a list of servers that you want to deploy to. Each server has a snippet in there similar to this:

[production]
host=ftp.leftontheweb.com
port=22
user=skoop
dir=/home/skoop/symfony_sites/www.leftontheweb.com

As you see, this is basically all information you need to deploy an application, aside of course from the password which you wouldn't want to specify in a configuration file if it isn't required. Instead, as soon as you issue the symfony sync command for this server, it will ask you for your password. There are, in the case of the above server, two commands you can issue:

symfony sync production
This will do a dry run, to see if the deployment of the files might cause a problem

symfony sync production go
This will actually send the files over to the specified server.

Now, because the symfony sync command uses rsync, only those files that have changed are deployed to the specified server. Aside from the first deployment, in which case all files need to be sent to the server, this speeds up deployment big time as only a selection of files are transferred.

Aside from easily transferring the necessarily files, symfony also makes sure that the files you don't want to have deployed won't. For this, there is the rsync_exclude.txt file. In this file is a simple list of files (which can include wildcards, so you can exclude whole directories or files with a certain naming convention) that should not be deployed. Good examples of files you'll want listed in this file are the _dev.php development environments, your databases.yml (to ensure you don't overwrite your database configuration with an incorrect one) and your cache and log directories (listed by default).

Now we have transferring files covered. But database changes also need to happen as you deploy of course, because your new code will depend on those. In symfony 1.0, I personally always use batch job which can be called from the commandline. In symfony 1.1 it will be even easier, as the recommended way to do it then is to use actual symfony tasks (which can easily be written, in 1.0 already by the way).

So, in short, deployment will then be:

  • deploy code using symfony sync
  • call the batch job/symfony task to update the database and possibly do other required stuff
  • symfony cc (clear cache) - though this could be done from the above batch job/symfony task as well

And what is left is testing if everything went well.

Of course, having a powerful tool such as this at your fingertips does still require you to keep your head clear and keep thinking about what you are doing, but it definitely makes it much easier and ensures you don't have to think about everything.


Add comment

Comments

gravatar Ivo: Does it support clustered environments? In other words, can 'production' be linked to more than one host? (assuming a cluster that doesn't have a shared storage).

On the one hand, it feels like the same as deploying over ftp, only with a script; I prefer a sandboxed environment that can be updated through source control, but deploying with scripts still beats deploying manually.
January 30, 2008
gravatar left: Ivo: No, a cluster in this case will be a single server. Of course, since symfony is very extendable, the symfony sync task could easily be extended to a symfony cluster-sync task or something that goes over a list of servers instead of one, while still taking advantage of the original sync task. It is not there by default, but would make an excellent plugin :)

there's quite some differences (I tried to point some out) with deploying over ftp, for instance the incremental filetransfer (so only changed files get updated) and related to that the speed. And maybe inherent to the way symfony teaches you to do it, but it requires you to think upfront on what needs to be done. With FTP you can do that however, but still a lot of people don't actually *do* that. But that's less how you do it, more related to the approach to professional deployment I guess.
January 30, 2008
gravatar James Moey: I use SVN for deployment all the time, checkout or export depending on the situation. Sometime SVN+rsync deployment is required. Anyway, I just want to say "Slightly less bad was deploying from version control (CVS or svn)" is incorrect. If you are using CVS or SVN for your project from the beginning which you should do. There are little chance that you overwrite files as all changes are track in the repository. Plus CVS/SVN is not bound to a framework, a tool that you can use for all your project, no matter what environment you are in. So how can a useful tool like that be slightly less bad that FTP? It should be better or equal to symfony sync solution depending on the situation.
January 31, 2008
gravatar Digitalbase: We also work with SVN and a Rsync script to transfer everything to a live server. It does the job as it should be.
However, it's a great thing that Symfony offers a solution to those who don't have a SVN server.
January 31, 2008
gravatar Bert-Jan: I use sync for deployment too but there is a major snag in the rsync protocol: it uses a very primitive way of identifying which files have been changed: ls -l
The trouble this brings is when you develop a project with the help of some others who can sync from their dev server, the timestamps etc. don't match yours anymore. When you make even a tiny modification and sync it, rsync will not recognise any of the files on the prod server and simply upload *everything*. If you sync again after that the incremental principle kicks in because all files are in sync with *your* server. Now when the 'other guys' run a sync *they* have to upload everything.
Quite disappointing to be honest...
January 31, 2008
gravatar Junni: I always do the upload of the files manually. Since I know that my ftp client somethings skips certain subdirectories (especially in de web- and plugindir), I'm going to give the Symfony sync tools a try.
January 31, 2008
gravatar Thierry: Symfony sync is a great tool. Rsync is really nice for syncing files.

Currently I use svn though, also works like a charm. A simple svn up on the server and you are done. Very similar to rsync in that it only does the changed files.
January 31, 2008
gravatar Jay Klehr: Bert-Jan: Look into the rsync -c flag, it does a checksum comparison on the files, instead of a modification timestamp check.
January 31, 2008
gravatar Bert-Jan: Jay: thank you. I just found that out myself before I read you comment here. I also found out you can set the parameters symfony passes to rsync with a 'parameters' attribute in properties.ini. I haven't yet tested if it works as expected.
February 4, 2008
gravatar Jason: great article! good points made by commenters!

if you work on large systems, and are using load-balancing or clustering, a reasonable solution is also to write your own set of symfony deployment tasks for the capistrano system.

then you get atomic releases which can be rolled back instantaneously, hooks for batches, and incremental updating which is not reliant on the filesystem to determine modification (either with timestamp or size with rsync).

mostly all you'll use in capistrano are variable names in strings to parameterize shell commands, so don't be afraid that it's in ruby and not php. I built out my deployment system in about 2 hours, and it works like a charm, with database migration hooks and all.

Cheers!
February 8, 2008
gravatar left: Hi Jason,

Thanks for that. I've seen mentions before of using Capistrano for PHP deployment, I guess I need to dive into that. Yet one more thing to add to my list of things to do ;)
February 8, 2008
gravatar angelia110: Costs For Uggs--What It Costs?
-->Are you lusting after a few (or even more) Ugg boots and if so, you're in good company, as girls and women of all ages, especially with the simple, slipper-like design of these sheepskin ugg boots sale , which are relatively easy and very comfortable to wear. And they appear to be affected. Some men are also jumping on the trend and luxury UGGs? Australia, the brand also offers a range of contemporary styles for them.
-->Could you mind giving everyone loves from ugg classic ? There are so many types of uggs, ugg bailey button,ugg classic tall, ugg classic short, ugg classic cardy. How to choose your favorite? Or do you really want to one uggs regardless its style? Despite their design is awkward and slipper-lile, Uggs is one of the few stations that are of general interest, have argued that cross generational lines.
-->Young people, students and young mothers and the Middle Ages, the original Black Ultra Tall UGG Boots, seem pulled the fleecy-lined boots that are manufactured in Australia, with the best materials. Are you sure that your feet warm in winter without socks, and cool in summer so that is more versatile too? If no, hurry up to take one ugg boots on uggs on sale ! That's why we see people wear them in schools, supermarkets, on the slopes and even in the most popular beaches in the United States and abroad. Many surfers also use uggs to keep their feet warm.
-->What do you really care about? Is its price or quality, or you just following the general trend? You know what are you thinking in your heart!
October 15, 2009
gravatar buddy: I think I will try to recommend this post to my friends and family, cuz it’s really helpful.Ugg Boots Sale
October 27, 2009
gravatar Cheap Kamagra: thanks for the share
November 28, 2009
gravatar Cheap Viagra: nice website good read
November 28, 2009
gravatar buy thesis: Good job done about the post, thank you!

May 5, 2011
gravatar Craigslist Illinois apts: Web Development is very important topic to learn because it provides solid foundation to move forward in the world of IT. We must learn it with special care and understanding. It will help us in the field of technology
June 10, 2011
gravatar orlando wrongful death attorneys: There is so much that goes into this deployment. The ideas that they have for this is so good. Keep up the good work.
November 3, 2011
gravatar buy essays: Thanks for sharing this information, keep up the good work.
December 5, 2011
gravatar City School Lahore: The following article actually established my very own little brown eyes towards the several merchants which usually organizations currently have by means of internet marketing.i like to read informative blogs and this blog is also so good and helpful.thanks for taking time to discus this topic..

January 15, 2012
gravatar discount uggs boots: I am typically to running a blog and i really respect your content. The article has really peaks my interest.
February 1, 2012
gravatar michael kors purses: http://www.michaelkorscheapoutlet.com/michael-kors-purses-c-6.html

February 2, 2012

Php5_zce_logo

Upcoming events

I will be speaking 06-02-2012: D-Day
I will be speaking 17-02-2012: Techademy Trainingday February
I will be speaking 23-02-2012: Zend Webinar: Git for Subversion Users

Tags

1337 2008 2010 2011 4developers access modifiers accessibility AdaLovelaceDay09 advent agavi agile alfred amsterdam apache api apple article articles atk atkMetaNode audioscrobbler automation azure backwards compatibility barcelona barcodes bash bbc bbq beatstad belgium best practices bittorrent blogging blogs boards of canada book books bughuntday bundle caching cake cal evans calendar career cat cerf certificate cfp clear cms cologne common sense communities community components conference conferences contest continuous integration contribute contribution crisis css custom d-day datetime DbFinderPlugin decorator decorators deployment devdays development directoryindex docblox doctrine documentation download dpc dpc09 dpc10 dpc11 DPC2008 dreamhost drupal dv7 eclipse ed editors efficiency enterprise errors event events expertise ezcomponents facebook finland flickr fork framework frameworks freelance freeze frontend fun game games geoip germany getting real git github gnome-do google google calendar googletalk graceful degradation hack hackers hidden gem hiphop howto hp HR html http i386 ibuildings icann ide ideasofmarch idm imovie indy ingewikkeld integration international php conference internet interview ipad IPC ipc ipc08 ipc10 ipc11se iterm2 javascript jenkins jenkins-php job job openings jobeet john peel joomla joomladays kiva kubuntu launcher launchy left on the web libraries library lighttpd lime linktuesday linux live london loudblog m2ts mac magazines malware mambo marjolein mediterra meeting meme meta methodology micro-financing microframework microsoft migration movie music mysql namespace namespaces netbeans netherlands newsfire nllgg nos odmarco open source opinion ORM osx paradiso paris partnership pavilion pear pecl performance personal pfc10 pfc11 pfcongres pfcongrez pfz photo php php5.3 phpabstract phpazure phpBB phpbb phpbelgium phpbenelux phpbnl10 phpday phpdoc phpdocumentor phpgg phpitalia phpnw phpnw08 phpnw11 phpstorm phptek phptek09 phpuk2009 phpUnderControl phpunit php|architect php|tek podcast politics portability postcrossing presentation presentations private projects protected prototype PSR-0 public python qa qr codes re2c recruiting refactoring review rewrite ruby on rails san francisco schedule scifi script security sensio seven things sfdaycgn sflive2011 shell scripting silex simplexml slides smfony software sogeti solar sound speakers spl ssh standard standards star trek static steer strings stylesheets subversion symfony symfony live Symfony2 symfonycamp symfonyday symfonylive symfonyUnderControlPlugin talk talks techademy technology techportal tek09 telecommuting terratec terrorism testfest testing textmate textpattern the right tool timeout tips tld todo tomas tools training twig uncon unet usability usergroup validation vhost video vim vinyl virus warp webinar weblogging webservices wiki windows winphp women wordpress work workshop world world of warcraft wpi writing wunderlist xml xpath xsd yara year youtube zc11 ZCE zemanta zend zend framework zend server zend studio zendcon Zend_Form zite
© 2004 - 2012 Stefan Koopmanschap + Powered by Symfony, photos powered by Flickr, links powered by Delicious, Shanghai smilies by Iconbuffet. Feeds: rss / atom. Left on the Web v4.4.0.1