Creating Static Websites from Drupal CMS
This is a technical article, but it isn’t *that* technical. For many many years I have been building websites. Sometimes for money, but mostly as a hobby, and I suspect a lot of readers either build or use their own hobby websites too. This article is for you.
Getting There
Back in the mist of time websites were designed on one person’s computer and uploaded as html files to a web server which delivered them to web browsers on request. We used software which made creation of the html much easier and there were lots of strategies for this.
Then Content Management Systems appeared. I remember working on one for a major national newspaper group which cost half a million pounds to license, and another half a million pounds in consultancy. Sadly I didn’t see much of that money. The idea was that people who wrote the stories on the website did not need to know any technical html, but could submit their news through a simple web interface. A workflow could be enforced so that editors got to decide what went where, and it could be published in an eficient way.
This sort of CMS has been developing for the last fifteen years or more. The price has been coming down and free open source CMS have been quite appropriate for some time. This includes my favourite CMS Drupal. The CMS included many dynamic features like forms for data entry, comment systems, forums, and so on.
I am not going to argue with you why Drupal might be good for you as we are different. You might have different priorities than me. Drupal has suited me for 14 years - including several total rewrites. What I am going to describe is a “new” way of doing Drupal which might become the standard for low turnover sites - such as club, society, and hobby websites.
For some time maintainers of dynamic websites have tried to reduce the amount of work required to keep their sites going. You always need to keep the software up to date with respect to security flaws or else find that your website and possibly whole machine are compromised. This is unfortunately easy to ignore until it is too late. Professional sites normally have the resources to employ system administrators whose job it is to update the running software, but hobby sites don’t.
So what might be the solution. Well, one reasonable solution is to go with a cloud based solution such as hosting your website on a blogging platform like Tumblr, Livejournal, or wordpress.com This works for many people and the simplicity for users is attractive to those who want a troublefree life. The problem comes when you want something slightly different from what they offer.
So what has been the solution? In the past I have been using third party systems for things I just did not want to handle myself. I use Paypal for credit card handling. None of my machines ever see a credit card number let alone make claims on those cards. It turned out to be a terrible problem to keep spammers off of the comment areas of the website. The solution was to switch off comments or to use Disqus. I used to run a web banner advertising system for science fiction fans, but nowadays I just use advertising generated by Amazon or Google. This has in fact produced better ads for my sites than I could have done myself. (Or in some cases we just did without).
The end result is that the CMS just concentrates on doing what it did best - content management.
But can we go further? Can we remove the CMS from the site delivery? Can we do without running any code on the server other than the plain and simple web server? Can we go back to the DreamWeaver type operation we did twenty years ago when we just uploaded html? Well, yes.
I had a brief look at JekyllRB - which looked good, but is only really for software developers. It did not really have any kind of CMS. Although you could import from your old CMS I didn’t see any way of repeatedly combining the two.
So with this in mind I considered grabbing a copy of the site as full text html. This is possible with something like httrack and that may work for some small sites. But was there anything better?
My Current Solution
Whilst googling I stumbled across the Drupal module Static. This seems to do 90% of what I want. I used it to turn my corporate Drupal website into a static set of html files without too much trouble.
I wont go into too much detail as to how to use it but here is the cut down recipe:
I have a test version of my site on a secure computer. I modified this one and disabled the live one.
On the “test” version I installed and enabled this drupal module, plus a few others it needed.
I then switched off several dynamic features which weren’t going to work - such as the search form, contact form, and the login form.
I then told it to regenerate a “cached” version of the site which was effectively a bunch of html files
I think rsync’ed the html files to the web server machine and told the web server machine that there was a new directory to serve instead of the old php one.
Of course it wasn’t entirely as simple as that, but you get the idea.
The test site is more accurately a staging site which gets used by all the site contributors. You can do whatever you like to make sure that only those people get access to it. For instance I have mine on a machine at home. However, you could protect it with simple basic authentication implemented in the web server; whatever you need so that malicious people or scripts cannot see the staging website.
I had already used Disqus for comments on my site and these seemed to be unchanged. Note that the URLs had not changed so Disqus did not really care about how the page was generated.
I don’t have a search box on the site. In theory I can use Google for that and either accept that Google will put ads in - or pay them to not to. Or I can go old school and implement my own search engine. Nah.
I have had to remove the tags from my articles. This system (known as Taxonomy in Drupal circles) is quite powerful, and I hope that there will be a way of doing them in a future version of the Static module. I haven’t lost the label information - it is just hidden from users.
I already used a cloud based system for sharing urls with others - it is called “AddToAny” and lets you share articles on Facebook, twitter, and so on…
So now I am on the lookout for new cloud based systems I can incorporate into my site.
You can see my finished site at http://www.owal.co.uk/ but there isn’t much to see. It just looks like any other website developed by an engineer instead of a graphic designer.
(Comments usually limited to LiveJournal friends)
Getting There
Back in the mist of time websites were designed on one person’s computer and uploaded as html files to a web server which delivered them to web browsers on request. We used software which made creation of the html much easier and there were lots of strategies for this.
Then Content Management Systems appeared. I remember working on one for a major national newspaper group which cost half a million pounds to license, and another half a million pounds in consultancy. Sadly I didn’t see much of that money. The idea was that people who wrote the stories on the website did not need to know any technical html, but could submit their news through a simple web interface. A workflow could be enforced so that editors got to decide what went where, and it could be published in an eficient way.
This sort of CMS has been developing for the last fifteen years or more. The price has been coming down and free open source CMS have been quite appropriate for some time. This includes my favourite CMS Drupal. The CMS included many dynamic features like forms for data entry, comment systems, forums, and so on.
I am not going to argue with you why Drupal might be good for you as we are different. You might have different priorities than me. Drupal has suited me for 14 years - including several total rewrites. What I am going to describe is a “new” way of doing Drupal which might become the standard for low turnover sites - such as club, society, and hobby websites.
For some time maintainers of dynamic websites have tried to reduce the amount of work required to keep their sites going. You always need to keep the software up to date with respect to security flaws or else find that your website and possibly whole machine are compromised. This is unfortunately easy to ignore until it is too late. Professional sites normally have the resources to employ system administrators whose job it is to update the running software, but hobby sites don’t.
So what might be the solution. Well, one reasonable solution is to go with a cloud based solution such as hosting your website on a blogging platform like Tumblr, Livejournal, or wordpress.com This works for many people and the simplicity for users is attractive to those who want a troublefree life. The problem comes when you want something slightly different from what they offer.
So what has been the solution? In the past I have been using third party systems for things I just did not want to handle myself. I use Paypal for credit card handling. None of my machines ever see a credit card number let alone make claims on those cards. It turned out to be a terrible problem to keep spammers off of the comment areas of the website. The solution was to switch off comments or to use Disqus. I used to run a web banner advertising system for science fiction fans, but nowadays I just use advertising generated by Amazon or Google. This has in fact produced better ads for my sites than I could have done myself. (Or in some cases we just did without).
The end result is that the CMS just concentrates on doing what it did best - content management.
But can we go further? Can we remove the CMS from the site delivery? Can we do without running any code on the server other than the plain and simple web server? Can we go back to the DreamWeaver type operation we did twenty years ago when we just uploaded html? Well, yes.
I had a brief look at JekyllRB - which looked good, but is only really for software developers. It did not really have any kind of CMS. Although you could import from your old CMS I didn’t see any way of repeatedly combining the two.
So with this in mind I considered grabbing a copy of the site as full text html. This is possible with something like httrack and that may work for some small sites. But was there anything better?
My Current Solution
Whilst googling I stumbled across the Drupal module Static. This seems to do 90% of what I want. I used it to turn my corporate Drupal website into a static set of html files without too much trouble.
I wont go into too much detail as to how to use it but here is the cut down recipe:
I have a test version of my site on a secure computer. I modified this one and disabled the live one.
On the “test” version I installed and enabled this drupal module, plus a few others it needed.
I then switched off several dynamic features which weren’t going to work - such as the search form, contact form, and the login form.
I then told it to regenerate a “cached” version of the site which was effectively a bunch of html files
I think rsync’ed the html files to the web server machine and told the web server machine that there was a new directory to serve instead of the old php one.
Of course it wasn’t entirely as simple as that, but you get the idea.
The test site is more accurately a staging site which gets used by all the site contributors. You can do whatever you like to make sure that only those people get access to it. For instance I have mine on a machine at home. However, you could protect it with simple basic authentication implemented in the web server; whatever you need so that malicious people or scripts cannot see the staging website.
I had already used Disqus for comments on my site and these seemed to be unchanged. Note that the URLs had not changed so Disqus did not really care about how the page was generated.
I don’t have a search box on the site. In theory I can use Google for that and either accept that Google will put ads in - or pay them to not to. Or I can go old school and implement my own search engine. Nah.
I have had to remove the tags from my articles. This system (known as Taxonomy in Drupal circles) is quite powerful, and I hope that there will be a way of doing them in a future version of the Static module. I haven’t lost the label information - it is just hidden from users.
I already used a cloud based system for sharing urls with others - it is called “AddToAny” and lets you share articles on Facebook, twitter, and so on…
So now I am on the lookout for new cloud based systems I can incorporate into my site.
You can see my finished site at http://www.owal.co.uk/ but there isn’t much to see. It just looks like any other website developed by an engineer instead of a graphic designer.
(Comments usually limited to LiveJournal friends)