Playing around with the OTW archive software on Linux
I love fanfiction, I love programming, I love Open Source... so of course I love the Archive of Our Own! I'm not sure if I'm going to sign up as a volunteer, mostly because I always seem to have too little time, but still I'm curious and would like to play around with the software and have a look at the sourcecode on my own. I have been poking around at the OTW websites and at the googlecode page where the project is hosted, but I can't find anything that documents how to start. I find that a bit sad, I mean, I don't need to be taken by the hand, I'm a computer scientist myself, and I really don't want to bother the developer people when I'm not seriously considering joining the team.
So, I've tried on my own, and I got at least so far as to see the front page of the archive software. :) I'm on a Linux box (Debian Squeeze to be precise), and I've tried to write down all the necessary steps that need to be done, however, since I'm using my machine for developing other stuff, it might be that I've already had installed things before which are mandatory and which you will need to install to get the otwarchive to run. Also, I assume that you are able to use the command line, a text editor, and know how to work as root and install packages from your package manager (on Debian and Ubuntu, I recommend aptitude).
Getting the source code
First of all, we need to get the sourcecode, it is located at googlecode and managed by a version control system which is called subversion. To download the sourcecode, you need to install a subversion client. As root, execute:
aptitude install subversion
Now you can download (check out) the source code by typing in the command line (as a normal user):
svn checkout http://otwarchive.googlecode.com/svn/trunk/ otwarchive-read-only
You will see what files will be downloaded, it will take a while until the whole repository is completely downloaded to your hard drive. When the command is finished, you will have a directory otwarchive-read-only which contains the source code. If you want, you can already look around!
In the future, the source code will be changed by the developers, and you, of course, want to have those updates, too. Change to the source directory and run the update command to check for new stuff (as normal user):
cd otwarchive-read-only
svn update
Getting the required software to run the archive
Now for the interesting part: Getting the archive to run on our own machine! There's a couple of packages that need to be installed (as root):
aptitude install ruby rubygems rake ruby-dev libopenssl-ruby sphinxsearch libxslt1-dev
This are the basic Ruby packages (Ruby is the language the archive is written in) and some additional stuff. Also, the archive needs a database as a backend, and it seems that you have the choice between sqlite3 and mysql. sqlite3 is great for testing or for private use of the archive, because it just organises the database in a file; it behaves like a real database but you don't need additional software. However, I ran into a syntax error with the archive software while trying to use sqlite3. mysql is a real database server, which means you need to install an additional program which needs to run in the background. mysql is preferable for production usage, i.e. when you use the archive for real.
So, for the sqlite version install as root:
aptitude install sqlite3-dev
For the mysql version install as root:
aptitude install mysql-server libmysqld-dev libmysqlclient-dev
It should ask you for a password you want to use for the mysql root (not to be confused with the the root of your Linux system), and it will start the mysql server in the background.
Now it's time to get some additional stuff for Ruby. Most important: The Rails framework. Still as root, type:
gem install rails
Depending on whether you want to use sqlite or mysql, type one of the following commands to install the Ruby interface for the database:
gem install sqlite3-ruby
gem install mysql
In the otwarchive-read-only directory, type
rake gems:installThis will get some additional Ruby packages that the archive software needs. Now you have set up all the software you need.
Configuring the archive and starting it for a first test
As normal user, go to the otwarchive-read-only directory. You should see a file config/database.example, open it with an Editor and save it to config/database.yml, and make the following changes to the yml file:
If you use mysql, at the top of the file there's a line where you should enter the root password you had to choose when installing mysql, the line should look something like
password: secretI'm not sure yet whether it is a good idea to let the archive access the database as root, or if it wouldn't be better to create a new mysql user, but this is only for testing, and no-one can access my database anyway since I'm behind a firewall.
If you want to try sqlite, you should comment the whole first part of the file, i.e. add a hash sign (#) to the begin of each line, and uncomment the second part of the file (starting after the line 'The following is an example of how to set up the db for sqlite3'), i.e. remove the first hash sign (#) from each line.
The archive software is preconfigured to use Rails version 2.3.t, but if you have another version, open the file config/environment.rb and comment the line starting with 'RAILS_GEM_VERSION ='. Without this line in effect, the software will just use the newest gems version you have installed.
Now, still as normal user and in the otwarchive-read-only directory, start the archive:
script/server
You should see the following output:
=> Booting WEBrick
=> Rails 2.3.5 application starting on http://0.0.0.0:3000
no admin settings for you
no STI
language/locale not set
no database yet, not initializing user login alphabet
no database yet, not initializing pseud alphabet
Warning: Error loading /home/NNN/otwarchive-read-only/app/models/fandom.rb
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2010-01-07 13:31:53] INFO WEBrick 1.3.1
[2010-01-07 13:31:53] INFO ruby 1.8.7 (2009-06-12) [x86_64-linux]
[2010-01-07 13:31:58] INFO WEBrick::HTTPServer#start: pid=17315 port=3000
Now open up a browser and visit http://localhost:3000/, you should see the archive's front page! The page will show an error message, though, and in the command line, the message
Unknown database 'otwarchive_development'tells that the archive can talk to the database, but that the database isn't set up correctly.
Setting up the database and further configuration
We will fix the database now; stop the server by pressing Ctrl+C in the console. If you have installed mysql, enter (as normal user):
rake db:create:all rake db:schema:load If you restart the server now, the archive should look fully functional! You can create an admin by executing:
chmod +x script/runner ruby script/create_admin.rb After that, you can login to your new admin account using the URL http://localhost:3000/admin/login. BTW, most of the archive configuration can be found in config/config.yml of which one should make a copy called local.yml, it looks quite self-explanatory.
To update your database after you have pulled changes from the SVN, run:
rake db:migrate Last edited 18 May 2010
So, I've tried on my own, and I got at least so far as to see the front page of the archive software. :) I'm on a Linux box (Debian Squeeze to be precise), and I've tried to write down all the necessary steps that need to be done, however, since I'm using my machine for developing other stuff, it might be that I've already had installed things before which are mandatory and which you will need to install to get the otwarchive to run. Also, I assume that you are able to use the command line, a text editor, and know how to work as root and install packages from your package manager (on Debian and Ubuntu, I recommend aptitude).
Getting the source code
First of all, we need to get the sourcecode, it is located at googlecode and managed by a version control system which is called subversion. To download the sourcecode, you need to install a subversion client. As root, execute:
aptitude install subversion
Now you can download (check out) the source code by typing in the command line (as a normal user):
svn checkout http://otwarchive.googlecode.com/svn/trunk/ otwarchive-read-only
You will see what files will be downloaded, it will take a while until the whole repository is completely downloaded to your hard drive. When the command is finished, you will have a directory otwarchive-read-only which contains the source code. If you want, you can already look around!
In the future, the source code will be changed by the developers, and you, of course, want to have those updates, too. Change to the source directory and run the update command to check for new stuff (as normal user):
cd otwarchive-read-only
svn update
Getting the required software to run the archive
Now for the interesting part: Getting the archive to run on our own machine! There's a couple of packages that need to be installed (as root):
aptitude install ruby rubygems rake ruby-dev libopenssl-ruby sphinxsearch libxslt1-dev
This are the basic Ruby packages (Ruby is the language the archive is written in) and some additional stuff. Also, the archive needs a database as a backend, and it seems that you have the choice between sqlite3 and mysql. sqlite3 is great for testing or for private use of the archive, because it just organises the database in a file; it behaves like a real database but you don't need additional software. However, I ran into a syntax error with the archive software while trying to use sqlite3. mysql is a real database server, which means you need to install an additional program which needs to run in the background. mysql is preferable for production usage, i.e. when you use the archive for real.
So, for the sqlite version install as root:
aptitude install sqlite3-dev
For the mysql version install as root:
aptitude install mysql-server libmysqld-dev libmysqlclient-dev
It should ask you for a password you want to use for the mysql root (not to be confused with the the root of your Linux system), and it will start the mysql server in the background.
Now it's time to get some additional stuff for Ruby. Most important: The Rails framework. Still as root, type:
gem install rails
Depending on whether you want to use sqlite or mysql, type one of the following commands to install the Ruby interface for the database:
gem install sqlite3-ruby
gem install mysql
In the otwarchive-read-only directory, type
rake gems:installThis will get some additional Ruby packages that the archive software needs. Now you have set up all the software you need.
Configuring the archive and starting it for a first test
As normal user, go to the otwarchive-read-only directory. You should see a file config/database.example, open it with an Editor and save it to config/database.yml, and make the following changes to the yml file:
If you use mysql, at the top of the file there's a line where you should enter the root password you had to choose when installing mysql, the line should look something like
password: secretI'm not sure yet whether it is a good idea to let the archive access the database as root, or if it wouldn't be better to create a new mysql user, but this is only for testing, and no-one can access my database anyway since I'm behind a firewall.
If you want to try sqlite, you should comment the whole first part of the file, i.e. add a hash sign (#) to the begin of each line, and uncomment the second part of the file (starting after the line 'The following is an example of how to set up the db for sqlite3'), i.e. remove the first hash sign (#) from each line.
The archive software is preconfigured to use Rails version 2.3.t, but if you have another version, open the file config/environment.rb and comment the line starting with 'RAILS_GEM_VERSION ='. Without this line in effect, the software will just use the newest gems version you have installed.
Now, still as normal user and in the otwarchive-read-only directory, start the archive:
script/server
You should see the following output:
=> Booting WEBrick
=> Rails 2.3.5 application starting on http://0.0.0.0:3000
no admin settings for you
no STI
language/locale not set
no database yet, not initializing user login alphabet
no database yet, not initializing pseud alphabet
Warning: Error loading /home/NNN/otwarchive-read-only/app/models/fandom.rb
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2010-01-07 13:31:53] INFO WEBrick 1.3.1
[2010-01-07 13:31:53] INFO ruby 1.8.7 (2009-06-12) [x86_64-linux]
[2010-01-07 13:31:58] INFO WEBrick::HTTPServer#start: pid=17315 port=3000
Now open up a browser and visit http://localhost:3000/, you should see the archive's front page! The page will show an error message, though, and in the command line, the message
Unknown database 'otwarchive_development'tells that the archive can talk to the database, but that the database isn't set up correctly.
Setting up the database and further configuration
We will fix the database now; stop the server by pressing Ctrl+C in the console. If you have installed mysql, enter (as normal user):
rake db:create:all rake db:schema:load If you restart the server now, the archive should look fully functional! You can create an admin by executing:
chmod +x script/runner ruby script/create_admin.rb After that, you can login to your new admin account using the URL http://localhost:3000/admin/login. BTW, most of the archive configuration can be found in config/config.yml of which one should make a copy called local.yml, it looks quite self-explanatory.
To update your database after you have pulled changes from the SVN, run:
rake db:migrate Last edited 18 May 2010