Difference between revisions of "Dreamwidth Scratch Installation"

Revision as of 13:00, 5 April 2010

These are the instructions to install a Dreamwidth system from scratch, assuming you are running Ubuntu 9.04. Please note that the server currently segfaults under Ubuntu 9.10 setups. See Setting up Dreamwidth on Linode for instructions on setting up a system to the point where you can log in and start this. After you set up, you can keep updated with the instructions on Dev Maintenance.

If you have difficulties or problems or suggestions for installation procedures, please report them to Bug 61. Also look at Production Setup: Webserver.

Bringing the system up to date

Before you start installing anything else, you might want to make sure your system is up to date. You should log in as root and update the packages on your system:

apt-get update
apt-get upgrade

If you want man pages on your system:

apt-get install man

Installing necessary packages

This will install necessary packages you'll need to run Dreamwidth (and some optional ones). You'll want to be root when running these commands.

apt-get install mercurial subversion apache2-mpm-prefork \
  libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \
  mysql-server wget unzip links vim libclass-autouse-perl \
  libcompress-zlib-perl libdatetime-perl libdigest-sha1-perl \
  libgd-gd2-perl libhtml-template-perl libwww-perl libmime-lite-perl \
  libnet-dns-perl liburi-perl libxml-simple-perl libclass-accessor-perl \
  libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \
  libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \
  libgnupg-interface-perl libmail-gnupg-perl perlmagick \
  libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \
  libstring-crc32-perl libtext-vcard-perl libxml-atom-perl libxml-rss-perl \
  libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \
  libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \
  libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \
  libtest-simpleunit-perl libtemplate-perl libterm-readkey-perl \
  libextutils-cbuilder-perl

This will download about 123MB of files and use around 446MB of disk space.

Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again.

Check whether your system is using threaded MPMs or pre-fork. You will need the latter.

If you have weird errors saying things like "Package mercurial is not available, but is referred to by another package.", try editing /etc/apt/sources.list and uncommenting the other repositories and then doing an apt-get update. Ran into this problem on a fresh install of jaunty on Linode.

You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay.

Before installing, check if your CPAN shell has 'make' at the correct location. Run which make, then compare the results with running o conf make at the CPAN shell (to run the CPAN shell, type CPAN.)

The perl libraries are:

perl -MCPAN -e 'install Bundle::CPAN'
perl -MCPAN -e 'install GTop' # this will guide you through setting up CPAN
perl -MCPAN -e 'install Unicode::CheckUTF8'
perl -MCPAN -e 'install Captcha::reCAPTCHA'
perl -MCPAN -e 'install IP::Country::Fast'
perl -MCPAN -e 'install Google::Checkout::General::GCO'

You will also want to install and configure Postfix so your DW can send out email:

apt-get install postfix

If you choose not to configure on the install, you can do so later with:

dpkg-reconfigure postfix

Setting up the DW user account

To set up the user account:

adduser username

Fill out the user's info and put them in the sudo group:

usermod -a -G sudo username

So, to use dw as a username:

adduser dw
usermod -a -G sudo dw

Now, we will get the $LJHOME variable set. Log into your user:

su - dw

We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables.

For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file:

export LJHOME=/home/dw

Or whatever your user account's home directory is. Yes, the variable is named LJHOME. We'll live with it for now.  ;-)

Test this. Log out of your user account and log back in, then type:

echo $LJHOME

You should see /home/dw or whatever you set it to. If you don't, then you didn't get the export line in the right place. (Hey, I'm not here to teach basic sysadmin stuff. You should know how to set environment variables.)

Downloading the Dreamwidth code

You should do this as the user you want to run Dreamwidth as. If you are root, you can switch with:

su - dw

We have put together a bootstrap script that will download all of the code. You can obtain this file with your favorite tool (wget, curl, etc):

wget http://hg.dwscoalition.org/dw-free/raw-file/tip/bin/bootstrap.pl

The next step is fairly automatic. Assuming that your $LJHOME environment variable is good to go, you should be able to just do this:

perl bootstrap.pl

The script will start doing some work. It should start out by saying 'seems we need to start at the beginning', which is what you expect. Then you will sit around for half an hour while it checks out various packages and repositories from various locations.

When it's done, it will say so. It will also say to delete the bootstrap script:

rm bootstrap.pl

Do a quick ls in $LJHOME to see if you have a bunch of new directories like cvs, cgi-bin, htdocs, etc. If you do, huzzah! You now have the code checked out.

If you don't, figure out what step didn't work, and try again. All else fails, hit the mailing list or find someone on IRC.

dw-nonfree

If you are doing development on Dreamwidth custom files (what few there are), then you will need to do another step. This is fairly manual, the bootstrap script does not handle this process for you. But assuming that you did the above steps, you can get dw-nonfree code like this:

cd $LJHOME/cvs
hg clone http://hg.dwscoalition.org/dw-nonfree dw-nonfree
cp dw-nonfree/cvs/multicvs-local.conf .

You may want to look at Keeping your site configs from wiping during updates if you want to keep your config files from getting wiped during subsequent updates if you're using dw-nonfree.

Updating to the tip

As of right now, you have the code that existed up until the end of the initial branch with no name, ending in changeset 1defc551af6c from October 13th 2008. However, with the next commit on January 14th 2009, named branches were introduced, and the dw-free repository now uses all named branches. (currently there is only one named branch that is up to date, 'production')

Right now, bootstrap.pl gives you the SVN repository, rather than an HG which is currently used and as such cvsreport.pl fails because it expected HG.

Remove it and then do checkout:

cd $LJHOME/cvs/dw-free
hg update -C tip
cd $LJHOME
bin/cvsreport.pl -sync -cvsonly
rm -rf cvs/perlbal/
rm -rf cvs/js/
bin/cvsreport.pl --checkout
bin/cvsreport.pl -sync -cvsonly

You need to do the last line twice, since the multicvs.conf file has changed since then. Any future updates should now work as described on the Dev Maintenance page. You will also want to delete files that no longer exist in the repository:

cd $LJHOME; for i in `bin/cvsreport.pl -n -1`; do echo "Removing $i" && rm $i; done

Database setup

You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root:

mysql -u root -p  

These MySQL commands wil create your development database:

create database dw;
grant all on dw.* to 'dw'@'localhost' identified by 'somePassword';
quit;

You might want to pick a more appropriate database username/database name/password.

Editing the config files

Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now:

cd $LJHOME
cp doc/config-local.pl.txt etc/config-local.pl
vim etc/config-local.pl

Or use your editor of choice, of course. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation.

Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.

Next, you need to change your local variables in the config-private.pl file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your $DOMAIN and the %DBINFO structure.

cd $LJHOME
cp doc/config-private.pl.txt etc/config-private.pl
chmod go-rwx etc/config-private.pl
vim etc/config-private.pl

In this file, uncomment the lines in the %DBINFO blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example:

   %DBINFO = (
       master => {
           pass => 'mypassword',
       },
   );

Note that there are two DBINFO blocks; get them both.

The base configuration file is under source control, and is already in etc/config.pl. You should not need to change anything very much in this, but you might have to change some stuff. If you find you do have to touch this, please see [1] first. Then you can edit it:

vim etc/config.pl

Make sure things are working with checkconfig.pl

Now, you need to see if everything is working.

If you've setup the files as indicated above, run this command:

cd $LJHOME
bin/checkconfig.pl --no=ljconfig

If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC or the mailing list.

Populate database with initial data

There are a few commands you can now run to install the database. Just run these and watch for errors.

Note: If make_system.pl says it can't give the system user admin privileges, something has gone wrong with your database population, even if there were no errors.

$LJHOME/bin/upgrading/update-db.pl -r --innodb
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
$LJHOME/bin/upgrading/update-db.pl -p

(That step will ask you for a password for the system account. You can change it later by logging in as system, so just give it something for now.)

$LJHOME/bin/upgrading/make_system.pl
$LJHOME/bin/upgrading/texttool.pl load

Configure Apache

This step will need to be done as the root user. Below is the Apache 2 configuration running on the Dreamwidth staging site. Put this in a file named "stage" in /etc/apache2/conf.d:

User dw
Group dw
UseCanonicalName off

StartServers 1
MaxSpareServers 2
MinSpareServers 1
   
DocumentRoot /home/dw/htdocs
   
PerlSetEnv  LJHOME /home/dw
PerlPassEnv LJHOME
   
PerlRequire  /home/dw/cgi-bin/modperl.pl

Then disable the default site:

a2dissite default

You might also have to enable the Perl Apache Request module:

a2enmod apreq

If you don't want this warning:

[Thu Jan 15 01:46:54 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
 ... waiting [Thu Jan 15 01:46:55 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
  ...done.

Then use vim /etc/apache2/ports.conf and put a # in front of the NameVirtualHost line:

# NameVirtualHost *:80

Restart the server:

/etc/init.d/apache2 restart

Now what?

Have a look at your new DW instance

Congratulations! You now have a working (though minimal) Dreamwidth install. If you point your web browser at your server, you should see a bare-looking welcome page.

Further setup

There are lots of other articles on setting up and customizing your DW install in the DW Installation category, including:

• TheSchwartz Setup - TheSchwartz is needed for a handful of features - notably comment posting
• Subdomain setup
• Statistics setup
• Generating documentation
• Allow users to register

There is a list of wanted how-tos at Installation Wanted How-To, in case you need to add something to the list.

Starting development

The list of things that need doing are in Bugzilla. You can keep your code update with the instructions in Dev Maintenance. Instructions on submitting patches are in Dev Patches.