Is C# uncool?
I’ve been a Microsoft developer since 1998. Back then I was working in a bank—so maybe that explains why I went with ASP instead of Perl. I’ve worked with the hugely expensive Visual Studio 6 development suite (which I could use at work only after convincing my boss that the price tag of £1,200 would be worth it—but could never afford it at home) all the way up to Visual Studio 2010 (a fantastic IDE). But Microsoft has let me down in a lot of areas. This is why I also take on occasional projects in PHP or Ruby.
There’s an interesting article in the New York Times this weekend called “Microsoft Calling. Anyone There?” In the article, it says:
Part of its (Microsoft's) problem may be that its ability to intrigue and attract software developers is also waning, which threatens its ability to steer markets over the long term. When it comes to electronic devices, people writing software have turned their attention to platforms from Apple and Google.
Meanwhile, young technology companies today rely on free, open-source business software rather than Microsoft’s products, so young students, soon to be looking for jobs, have embraced open-source software as well.
I know many developers in my field who will staunchly defend Microsoft development tools for their usability. But even as a big fan and in spite of my love for the C# language, I know that Microsoft has a lot of problems. Here are a few things I don’t like about the Microsoft developer technologies:
- They require you run them on Microsoft machines. Asp.net will only run on IIS which will only run on an expensive Windows server.
- Whenever possible, Microsoft tries to change the web platform by ignoring certain standards. In the late 90s, they were trying to push ActiveX Objects which no one could use—except in a corporate environment. Now, you see XBap projects (xaml based) which are just as awkward.
- Microsoft seems to use some of their products in an effort to push other products on you (I think Apple does this too, btw). For example, the new Office 2010 Suite is filled with “Post to Web” features, but you can only use this if you have a Sharepoint server which (if you are not in a corporate environment) could cost you thousands of dollars per year.
- The barrier for entry into this field is so low that anyone can claim to be a .Net developer with even a tiny bit of experience. For example, Microsoft tools are terrible about including drag and drop interfaces for development. Drag this on here, drag that onto there—hey, I’m a developer!
Still, the Microsoft developer community is large—so that makes it easier to solve a tough development problem when you have one. There is no shortage of documentation.
But, also, Microsoft developers are more likely to work for large companies than for small ones. There is a growing collection of open source projects written in .Net, but the number pales in comparison to the amount of open source projects written in PHP. This leads me to believe that Microsoft developers work more for the money than for the love of coding. I love to code and do it whenever I can, but as someone who interviews often—I realise that is not the norm. There are a few very dedicated and talented C# developers out there, but there’s also a lot who . . . aren’t.
I still feel the most comfortable in C#, but PHP can do some very cool things. PHP stays close to the HTML instead of using some pseudo ASP.Net server control tags which render HTML under the covers. So I will continue to bounce between the two. At least this way, I can know which developers are worthy of hiring for Overpass and which are the draggers and droppers.
Healthy Scepticism of Moving to the Cloud
Back in 2006, I got a call from a former client in the banking industry asking me to call a vendor about a trading system they were building. The bank was thinking about moving from an in-house system on internal servers to a remote web-based solution outside the firewall. I called the vendor and asked some security questions and about how their app would work. The owner of the company was giddy with excitement. He told me they were going sell this software “as a service” where the client would pay for each user, but the application would be hosted on their servers. I had heard about SaaS before (Salesforce.com were big with it back then—and it was hardly difficult to conceive), but it was obviously new to this guy because he kept telling me how their new software was going to work.
The benefit for the bank was they wouldn’t have to worry about applying software patches to internal servers (all internal bank applications are web-based on the LAN already anyway). The benefit for the vendor was they would have a continuous income stream without ever having to produce new versions of their software. This is the promise of the new age of cloud computing. It’s a win-win.
But I’m not a fan. I advised the client against working with the vendor. Their data is too valuable to trust with someone else. A few years earlier, they would never have dreamed about holding this sensitive data outside their data centres, but they have warmed to the idea.
I have a healthy paranoia for cloud computing. My main problems are:
- I no longer control the security of my data (if I have a server, I can back it up and encrypt it myself—but I can’t control what another company does).
- I’m locked into a contract where I pay for as long as I use the software (like leasing a car instead of buying one)—when I can’t afford it anymore, they take away my data.
- Your ISP becomes another failure point in the smooth running of your business.
- I develop a dependence on an external company for my company to work properly. The biggest problem with using a hosted solution is that you find it difficult to get your data back out when you can no longer afford to pay.
I use some hosted services, like Google Apps for Domains, so I’m not totally averse to the idea. The big benefit of not needing an Exchange server to sync up all my mail clients makes it worthwhile. Basecamp is very cool and I used it for a few years, but it always bothered me that I was sending my clients to someone else’s website to work on my projects—so I moved to a self-hosted solution.
While looking for software to help me in running Overpass, I come across a lot of web-based solutions where they ask you to store some pretty sensitive information on their servers. I’ve seen invoicing sites, accounting sites, crm sites, etc where they ask you to hand over information about bank accounts, clients, etc. If you ask the companies that run these sites if they have a self-hosted solution, you get a polite response that they don’t do self-hosted (unless you have a huge enterprise and could pay through the nose) but they assure you that your data is safe with them. I believe them—they wouldn’t do anything sneaky with your data. But I don’t know who will own that company later.
Despite the trend towards “moving into the cloud”, I prefer keeping things in the LAN. Just because everyone is doing it, doesn’t mean it’s safe.
Hiding from competitors
When I came up with my initial ideas for Overpass—how to build a collaboration system where clients could manage an offshore project easier than they could an internal team—some of my colleagues at the time agreed that the idea was great, but that I shouldn’t tell anyone about it so no one steals my idea.
On more than one occasion, I’ve had clients who sold world-class software but were afraid to put screenshots of demo videos on their sites because “the competition may try to steal our ideas.” These are small companies and they stay small.
The truth is that an idea that cannot be shared gains no traction. People often under-estimate the laziness of other companies and competitors.
I had a client say to me, while I was signing an NDA, “We don’t want you learning our industry and then making a competitor product.” But I told him that I had no interest in going into his industry. I’m a software guy. That’s where my passion lies. If I tried to go into his arena, it would suck—because I wouldn’t have the vision he has.
An idea is worth nothing. It’s all in the execution.
Will there be an Apple backlash?
So, for the past 4 days (since the last iTunes upgrade), I’ve been trying to get my 3-year-old sixth gen iPod to sync on Windows 7. This is the second time I’ve had this problem. After syncing for a few hours, I get a cryptic error telling me an unknown error occurred (with an error number -69). Googling the problem leads to to forum after forum of other people making guesses at what the problem is. The best answer seems to be that one song is corrupted and I should sit and watch the whole sync process to find out where it fails and remove that track. I’ve been doing this for days.
The big problem with super-simple interfaces which “just work” is that when they don’t work, there is never any easy way to troubleshoot. There are no error logs. No settings I can tweak. Just forum posts made by people who don’t work at Apple.
I recently gave up my iPhone of 2 years for an Android phone. I was holding out for the announcement on iPhone 4, but decided that the incremental new features (I don’t agree with the “game changer” boasts they are making) were not worth the queues and waiting lists. The real feature I wanted was a multi-tasking, which was available with the new OS—but that feature would not be available on the iPhone 3g (only the 3gs) and I doubt that it was for hardware reasons. I kind of felt like I was getting dicked around as a user—so I got an HTC Desire with Android and it works great.
I would get rid of my iPod if it weren’t for all the DRM encrypted music I bought from them.
And now there are complaints that the iPhone 4 has problems with its reception. Steve Jobs answered all of his raving fans who had this problem by saying “don’t hold it that way’. And there are still supply shortages in the UK. I remember waiting for months after the release until I finally got my hands on a 3g back in 2008. Who didn’t see this coming?
A year ago I was raving about Apple. If I wasn’t a fanboy, I was on my way to becoming one.
Is it worth is anymore?
Apple has really cool stuff. Their design is fantastic. But I get the feeling that with them that a 3-year-old gadget is not worth fixing and should be replaced. I’ve thought about getting a Mac (purely so I could code some iPhone apps—can’t be done on Windows!), but I have this fear that any future upgrade means buying new kit. At least with my PC, I can replace components when they fail.
I have a big problem with developers who overcomplicate their applications and put every possible button on their main interface. But Apple takes this to the other extreme. I spent minutes going through my iPhone settings when I first got it to find out how little I could customise it.
Apple is losing its edge. Lucky for them that Microsoft is not competent enough to take advantage of this.
Will the Apple fans see this? Probably not. They are very proud of their kit. Some are starting to say that Apple is a cult. It’s beginning to a look a lot that way.
Why offshore outsourcing fails
Project outsourcing fails because project owners give away too much control at the outset of the project. That’s it.
My first foray into outsourcing occurred in 2004. I hired a PHP web designer to redesign the Overpass site. I could have done it myself, but I decided to dedicate my time to other things. I found this guy’s website. I looked a the sites in his portfolio and they were impressive. His rates were very cheap. I speak fluent Mandarin, so that wasn’t a barrier either.
I gave the developer some general ideas of what I wanted on the site and some examples of other sites I wanted it to look like (the same way my clients often specify their requirements). I asked him for an IM account I could chat with him but he didn’t have one and said he didn’t believe in IM and preferred to communicate via email. He asked for the money for the project up front—because that is how his company works. I paid a few hundred quid from the Overpass account. He went away for a few days and came back with some screenshot proposals.
What he came back with was awful. It was far worse than the existing site I had put together. He used all the images from my old site (nothing new at all) and swapped the white background for a dark blue background with white text. It looked like the kind of thing you would generate from a template. In fact, I’m sure it was. It didn’t look anything like the sites I had pointed out to him as guidelines. He obviously spent time on it (I don’t think he was scamming me), but what he was doing was awful. I told him I didn’t like it and pointed out some more things I’d like to see on the site. He came back a few days later with something marginally better—but still so far away from what I wanted that I was looking for that I decided to scrap the whole thing. I asked him to translate the site text into Mandarin instead (it was the only thing I could think of to do with the guy)—but he came back and told me it would cost a few hundred pounds more. I asked for the remainder of my money back—minus the cost of his time to date—but as you can guess, that is the last I heard of him.
I made a lot if mistakes in that early project, but I learned a lot.
Outsourcing is like any other contract negotiation. You never assume the other side knows what they are doing and always give yourself an “out” clause. Here are some things I always insist on when working with offshore developers:
- Never pay up front for a project. Keep payment cycles short so as to decrease the risk to you and to them.
- Always give yourself an easy out. If they demand a notice period, never accept anything over one week. One day is better.
- Always insist on owning your own source code. This is very important and I hear more about problems in this area than in any other. When you have access to your source code, you have to option of taking it to another vendor if the project goes wrong. If they own the source code, you rely on them for everything. With access to your code, you can have a third-party developer look it over to ease your mind if you have doubts about quality.
- Always have easy communication with project managers through IM or Skype. Ideally, you should be able to talk to developers too. Most outsource companies will promise you a weekly or even daily update, but that is no substitute for being able to ask questions immediately. You want the questions answered by the troops and not he press secretary!
- Get access to the task list and milestones. If anything is behind schedule, you want to know about it as soon as it occurs.
- When you ask for any changes, find out first how it will affect the project. There’s nothing worse than have a project 5 days late—only to find out that the developers are blaming the delay on an off-hand comment you made about a nice-to-have feature. I always insist that no changes be introduced into a project mid-cycle (but my clients may sometimes insist!).
- Never give developers access to your live data or production environment. Your intellectual property is probably your most valuable asset. Obfuscate your data as much as you can before you hand it over. You may have the most reliable outsource partner in the world but . . . good fences make good neighbours.
- Keep your designers local, when possible. Unlike software functionality (“A textbox here, a button goes down here. . . “) design is a very iterative process. A good database guy can work from anywhere, but you need a very open communication channel with your designer. Not only is design heavily based on cultural influences, it requires more subtle changes (“That colour look a few shades too light.” “This area of the page looks a bit cluttered.”, etc.). When I was working for the BBC, we had teams of developers and a separate team of designers—they had a completely different skill set—working with Photoshop on Macs while we worked with Visual Studio on Windows. Since then, I have rarely seen a jack-of-all trades that I would want on my project.
The fact is, when outsourcing your software development, this is a buyer’s market. The nice thing about having developers all over the world competing for your business, you can usually get what you want in a service contract. But once you tie yourself in, they call the shots. Always make sure you can walk away (and take your code with you).
Now, I hear a lot of talk about fix-priced contracts where you pay upon completion. These are nice when they work out. You can give your requirements to an offshore company, but they will not give you access to the source code. You haven’t paid for it yet. You have to wait until project completion to see anything. I’m not a fan of this approach, although I can see the appeal. But I’ve seen too many projects fail because people through a list of requirements over the fence and expect the completed project to thrown back over exactly how they imagined it would be.
An analogy I like to use is that you are having an extension put on your house.
You could choose a builder who takes payment on project completion at a fixed price. You sign a contract. Your driveway is soon covered in building materials. On day one, a hole is ripped in your wall and covered in plastic sheeting. On day two he starts building the framework extension, but you get the uncomfortable feeling that the builder may not be as competent as you thought when he nearly electrocutes himself cutting through the wall. On day three, he doesn’t show up. In fact, a week passes and he doesn’t show up. You keep calling his mobile, and he apologises and explains that he can’t work on your house today because of an emergency job a few towns away. Meanwhile you are left with a hole in your wall and are practically begging him to return and finish the job. You gave away too much control when you agreed to that fixed-price contract. Technically, he owns all the building materials on your driveway, so you can’t just call someone else to finish it. Even when it is finished, maybe he doesn’t know what he’s doing anyway—you’ve lost all confidence in him.
If you had negotiated your contract on short payment terms, you own the materials (or the source code), and you can get out of the contract easily, you could still get another builder to come in and finish the job. You have more power in the contract.
Software outsourcing works the same way. It fails when you give away too much at the outset.
Thrashing
One of the best management books to come out this year is “Linchpin, Are you Indispensible?” by Seth Godin. I’m a big Godin fan and have been a regular reader of his blog for over 5 years now.
In this book, Seth talks about thrashing. In software development, thrashing is when you are throwing things around and rewriting things trying to come up with a system at the end. We see this a lot. And we do this more than we should.
An example of thrashing would be moving functionality around on a website or changing requirements late in development.
For example, you write an application based on a carefully-planned architecture and database design. Say, you’ve written a personnel system where a manager can log in and make recommendations on his staff’s payroll. You’ve written all error handling, validation, and a pretty-impressive hierarchical user interface so senior managers can view any employees indirectly reporting to them. The design is clean and efficient.
The application goes onto a staging server for UAT. A few key decision makers suddenly appear and are asked for their “opinions” too.
Someone points out that some employees have more than one manager. Some have one manager for payroll and another manager for appraisals (but these managers should have a say in the recommendation too). This becomes a “must have” requirement for the project, which is already is little late.
This is when thrashing begins. All prior testing is moot. The database needs a redesign. So does all the data access code. Every form is affected by the change. Form validation is no longer accurate. But, the project manager knows that the development was completed—“We just need to add this change. We can get some more developers to help out if you need it.” Thrash, thrash, thrash! The result is an application with a shaky architecture.
This example is not rare.
The best way to prevent thrashing is to design everything in advance. This is what Godin suggests. I agree.
I always start projects with extensive flowcharts, screenshots, and specifications. Nothing too formal—just enough so we all know what we are aiming for. I want the app I see in my head to be the same app the project managers and other developers can see in their head.
Once these designs are drafted, the absolutely most important step is to make sure all decision makers see them and make comments then (or forever hold your peace).
They key here is, as Godin points out, do all of your thrashing at the beginning of the project when it is free.
Link or Button? Consistency or Usability?
Link or Button? I’ve had this conversation so many times on so many projects. Here are my views on the matter. From my experience, they are pretty much standard in the industry:
- A link will take you somewhere (or perform a less-than-likely action)
- A button will perform an action
You would never have a button that takes you to a new page. You would never have a link to submit a form. But there are exceptions to this rule.
I had this discussion just the other day. A client wanted buttons and links to be used “Consistently”. I argued that we had to use them with the “Usability” in mind.
As a UI designer, you also need to help your user as much as possible by letting them know what you expect them to do. You see this on a lot of web applications these days.
For example, you have a simple submit form (most forms are more complicated, but I’ll use this for simplicity):
Coming to this page, I already know the email address and have a shopping cart of items to purchase. I use links to modify the form items for minor alterations (because they are unobtrusive). But at the bottom of the form, we have 3 buttons.
In this scenario, I have to ask “What do I expect the user to do?” Do I expect them to cancel? Do I expect them to go back and continue shopping? No. If they got to this page, I expect them to give me their name and submit. But yet I give the “Cancel” button the same weight as the “Submit” button. I’m confusing the User (but it is consistent).
A better approve is to “guide” the user into what I expect him to do while still giving the option to do the other things.
By keeping this less-than-likely options as links, I give importance to the Submit button. When you get to this page, I pretty much expect you to alter your name if necessary and Submit. However, you may not be finished shopping yet, so I give you that option too.
A great example of this principle (often referred to as Fitt’s Law), is an “Advanced Settings” option. If you are going post a blog comment, for example you would have a form as simple as we can make it.
99% of the time, that’s all you want to do. Put in a comment and save it. If I add anything more to the form, it will get confusing and the user will be less likely to fill it out. However, there may be other options that the user might want 1% of the time. The user should still have these available, but not in his face:
I don’t expect you to chose options, but there are things you might want to do with the comment (like make it available to only certain people). But I don’t want that functionality to have the same weight as the “Save” button.
When working with clients, the biggest excuse I get against using this approach is “We need consistency”. Consistency is often used at the expense of usability.
Reading:
Despite what many people think, web usability is a skill like any other. It’s not merely a matter of personal opinion where a developer does something “as long as it works.”
The best book I’ve read on this subject is “Don’t Make me Think” by Steve Krug. It’s generally considered the best book web usability book in the industry and should be required reading for all application developers, designers, and product managers.
Backups 101 Part 2: Remote SVN Backups
This the second post about remote SVN backups. Part 1.
Yesterday I wrote about how to schedule a daily backup of MySQL databases from a remote server. I call it a remote back-up, but it is really a backup to your local PC. Every night, I have batch files which run on my Windows 7 PC that back up crucial data that I would hate to lose if a server failed.
Today I want to talk about SVN Backups.
For Overpass, we use loads of Subversion repositories. In order to check code quality of offshore developers, I require a daily code check-in. It would be devastating to lose this data due to a server failure. My server is backed up, of course, but an entire server restoration is time-consuming.
Like MySQL, SVN has a “dump” facility, but it doesn’t work with remote repositories. So instead, we will sync up a local repository witht he svnsync tool.
To perform this back-up, you will need:
- SVN Client Tools for Command Line
Step 1: Create a local Repository to Sync with remote server
This only needs to be done, so it’s not required in batch file. Just create a local repository using the command line:
svnadmin create c:\svnbackups\myproject
This will create a brand new repository in c:\svnbackups called myproject which we sill use to store contents of our remote repository.
Now, you need to create an empty file in the “hooks” directory of this folder (c:\svnbackups\myproject\hooks) called pre-revprop-change.bat. If you fail to do this, you will be the following error when trying to sync:
svnsync: Repository has not been enabled to accept revision pr
ask the administrator to create a pre-revprop-change hook
Step 2: Initiate the Sync
Next, you need to initate the Sync. This only needs to be done once (so again, not part of the batch file):
svnsync init file:///c:/svnbackups/myproject svn://myserver.com/myproject
The syntax of statement is “svnsync init URLOfLocalRepository URLOfRemoteRepository”. Note the syntax for the “URL” of the local repository (you can’t just use a path – c:\svnbackups).
If you initiate again after the first sync, you will get this error:
svnsync: Cannot initialize a repository with content in it
Step 3: Perform the Sync:
This is the step that needs to be run from time to time so we include it in our batch file we created yesterday.
svnsync sync file:///c:/svnbackups/myproject
When you first run this command, you should see all the remote data transferred to the local repository.
Step 4: Dump the Sync Copy
Now I can run the svn dump off the local repository to store in my daily zip archive:
svnadmin dump c:\svnbackups\myproject > i:\myprojectbackup.bak
Note that with the local copy, we are now dealing with Paths instead of URLs. We refer to it with c:\svnbackups\…
Step 5: Automate
Now let’s include these two last commands in the daily backup script we created yesterday:
rem Location of Archives
SET backupDir=I:\SiteBackupsrem Location for downloads
SET tempDir=I:\regularBackups\temprem create new file name
set DD=%DATE:~0,2%
set MM=%DATE:~3,2%
set YY=%DATE:~8,2%
set YYYY=%DATE:~6,4%
set zipFile=%YYYY%-%MM%-%DD%.ziprem create the data dump
mysqldump -h 111.222.111.222 -u username -password databasename> %tempDir%\blogdump.sql. . . (All other databases listed here)
rem Download SVN databases
svnsync sync file:///c:/svnbackups/myproject
svnadmin dump c:\svnbackups\myproject > %tempDir%\myprojectbackup.bak
. . . (Download SVN databases)
rem zip the files
SET InstallLocationOf7Zip=%ProgramFiles%\7-Zip
SET exe7Zip=%InstallLocationOf7Zip%\7z.exeIF NOT EXIST "%exe7Zip%" (
ECHO 7-Zip is not installed in the location: %dir7Zip%
ECHO Please update the directory where 7-Zip is installed.
GOTO End
)"%exe7Zip%" a -tzip -r -mx5 %backupDir%\%zipFile% %tempDir%
rem remove old files
del %tempDir% /Qrem hibernate the pc
psshutdown.exe -h -t 00exit
Now, the daily backup script will:
- Download MySQL databases
- Sync SVN Repositories
- Dump SVN Repositories to the local temp folder
- Zip up the folder with MySQL and SVN dumps
- Hibernate the PC.
In a future post, I will talk about how to incorporate a full website FTP into a regular backup.
If this post helped you, please leave a comment letting me know. I’d love to hear how it worked out.
Adobe’s Mobile Failings.
A great article on the Business Insider website today on Adobe’s failing in the mobile market. Sounds like a terrible place to work (but familiar for many of us):
Adobe's ineptitude at responding to the changing industry is a byproduct of its increasingly multi-tiered management structure. Former Adobe employees complain of an excessively bureaucratic management process where progress is rendered minimal by an insistence on decisions by committee. Rather than entrusting important decisions to knowledgeable team managers, Adobe now relies on an elaborate web of middle managers who don't always have a close understanding of the matters they decide upon. Even worse, some of those managers have never even used the products that they are assigned to oversee, and their decisions are made not for the sake of the product but for the sake of keeping their own job in the face of Adobe's regular outsourcing of engineering positions.
Along with regular staff cuts has come an apprehension among employees to take responsibility for new ideas. Team members now are afraid to go against the bureaucratic grain by suggesting new ideas lest they get let go at the end of the year. The overly complicated decision-making process also is discouraging to team members hoping to see their ideas for the company come to fruition. Sometimes, a cut-throat attitude takes over as managers might shoot down a great idea from a team member only to turn around and present it as their own to the executives, which only further fragments the spirit of teamwork that traditionally enabled Adobe's engineers to create trailblazing products.
Flash is dead.
Great article if you have a chance to read it.
I wonder if all those Flash heavy websites are thinking of redesigns because of the iPad and iPhone. And what about Silverlight?
Backups 101 Part 1: MySQL Remote Backups on Windows
One of the great things about backups is that you can write an automation script to do everything automatically. Basically, you write you script, schedule it, and forget about it.
I have a dedicated server for the Overpass websites and blogs, so my local PC (and external hard drive) are ideal of off-site backups. I have written scheduled tasks which fire several events while I sleep:
-
- PC wakes from Hibernate at 2am.
- All my databases are downloaded to a local folder.
- All SVN repositories are downloaded to a local folder.
- The local folder is zipped up in an archive with a date stamp.
- PC goes back to a Hibernate state.
I works beautifully. I also have two other weekly scripts which download all website content (for sites like WordPress which store images and content on the server with every post) and a script that clears down very old archives (after 5 days). I have full server back-ups too, but this give me just a bit of extra confidence.
For this post, I would like to talk about backing up MySQL.
One of the features I always found SQL Server to lack is the ability to generate insert scripts. Sure, it's easy to script out tables, views, and stored procs-- but you still have to get tricky with transferring data from one database to another. I know people say "you can just back up the database and restore it to a new server", but they obviously have never worked in a tight change-controlled enterprise environment where you need everyone under the sun to approve this a dedicated dba overlooking everything. This method is overkill if you just want to replace the a record that a user has "accidentally deleted".
MySql has a very handy feature that use frequently called mysqldump. This will create an sql file of all db objects and store them in an sql file -- data structures and data too. Mysqldump will create one script with table create statements (and drop statements) along with insert statements for all of your data.
MySqlDump is fantastic for remote backups.
On Windows, this routine requires:
- MySQL Client tools
- MySQL Administrator
- 7-Zip for archiving
- PsShutdown to Hibernate
- Windows Task Scheduler (you should already have this)
Here's how it's done using my WordPress database and local Windows 7 account:
1. Make sure you can connect to your remote database from your local pc
To do this, you need to have MySql client tools installed on you local pc. Just try to log into our remote site using MySQL Administrator.
If you can connect from you local pc, you can use mysqldump from the command line.
2. Write a batch script to create an archive for each day
I have a folder called "backups" with a date-stamped zip file for each day of backup (ie. 2009-06-08.zip). Here is my code for that batch file (with my db details removed):
rem Location of Archives
SET backupDir=I:\SiteBackupsrem Location for downloads
SET tempDir=I:\regularBackups\temprem create new file name
set DD=%DATE:~0,2%
set MM=%DATE:~3,2%
set YY=%DATE:~8,2%
set YYYY=%DATE:~6,4%
set zipFile=%YYYY%-%MM%-%DD%.ziprem create the data dump
mysqldump -h 111.222.111.222 -u username -password databasename> %tempDir%\blogdump.sql. . . (All other databases listed here)
. . . (Download SVN databases)
rem zip the files
SET InstallLocationOf7Zip=%ProgramFiles%\7-Zip
SET exe7Zip=%InstallLocationOf7Zip%\7z.exeIF NOT EXIST "%exe7Zip%" (
ECHO 7-Zip is not installed in the location: %dir7Zip%
ECHO Please update the directory where 7-Zip is installed.
GOTO End
)"%exe7Zip%" a -tzip -r -mx5 %backupDir%\%zipFile% %tempDir%
rem remove old files
del %tempDir% /Qrem hibernate the pc
psshutdown.exe -h -t 00exit
For the mysqldump line, use the ip address (or host name) of the remote server and user name and password. In the example above, I am connecting to my db called "databasename" on host 111.222.111.222 with the user name "username" and password "password". It is writing out to a file on my local pc called "blogdump.sql" in a folder with a date stamp.
NOTE: the -p switch does not have a space between it and the password. This is bizarre, but that's how it works.
Save to the above code to a file with a .bat extension. I called mine "backup.bat".
3. Create a scheduled task to run this job.
Use Windows Scheduled Tasks to automatically run your batch job. This can be found in Control Panel and is easy to use (Wizard Generated). I run my job at 2am—which means I use Hibernate to shutdown every night instead of a full shutdown. Once a week, I have a similar job that will back up all web code via ftp.
That's it, the majority of my disaster recovery problems taken care of. I have the data stored on my local pc and on the server. It would be easy to run the same script from another pc if I need to, also.
As you can imagine, this daily backup can take a good deal of disk space. Each archive is about 3mb for this site. I usually clear things down every few weeks and delete backups that are over a week old. I don't replace my backup file every day because it could be a few days before I realise anything has gone wrong. I may script this at some point in the future.
If you are not backing up your databases (even if they are hosted with a web host), you should really consider it. It is a trivial task to set this up and could save you in the future.
About Overpass
Recent Tweets
- New Overpass Post: Is C# uncool? - I’ve been a Microsoft developer since 1998. Back then I was working in a bank—so ... http://ow.ly/1822TG
- New Overpass Post: Healthy Scepticism of Moving to the Cloud http://ow.ly/17Y9Ta
- New Overpass Post: Hiding from competitors - When I came up with my initial ideas for Overpass—how to build a collab... http://ow.ly/17Wfp0
- New Overpass Post: Will there be an Apple backlash? - So, for the past 4 days (since the last iTunes upgrade), I’ve ... http://ow.ly/17Vw9m
- New Overpass Post: Why offshore outsourcing fails - Project outsourcing fails because project owners give away too m... http://ow.ly/17UEoq
- Most IT projects fail because the project owner does not have full visibility into the project.
- Now that email marketing is so difficult, someone needs to think about spam filters for Skype, SMS, and Twitter.
- New Overpass Post: Thrashing - One of the best management books to come out this year is “Linchpin, Are you Indispe... http://ow.ly/17bFSt
- New Overpass Post: Link or Button? Consistency or Usability? http://ow.ly/17aUT4
- New Overpass Post: Backups 101 Part 2: Remote SVN Backups - This the second post about remote SVN backups. Part 1. ... http://ow.ly/179moR