Dreamwidth Scratch Installation

From Dreamwidth Notes
Revision as of 20:28, 18 September 2009 by Shadowspar (Talk | contribs)

Jump to: navigation, search

These are the instructions to install a Dreamwidth system from scratch, assuming you are running Ubuntu. See Setting up Dreamwidth on Linode for instructions on setting up a system to the point where you can log in and start this. After you set up, you can keep updated with the instructions on Dev Maintenance.

If you have difficulties or problems or suggestions for installation procedures, please report them to Bug 61. Also look at Production Setup: Webserver.

Bringing the system up to date

Before you start installing anything else, you might want to make sure your system is up to date. You should log in as root and update the packages on your system: 

apt-get update
apt-get upgrade

If you want man pages on your system: 

apt-get install man

Installing necessary packages

This will install necessary packages you'll need to run Dreamwidth. You'll want to be root when running these commands. 

apt-get install mercurial subversion apache2-mpm-prefork \
  libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \
  mysql-server wget unzip links vim libclass-autouse-perl \
  libcompress-zlib-perl libdatetime-perl libdigest-sha1-perl \
  libgd-gd2-perl libhtml-template-perl libwww-perl libmime-lite-perl \
  libnet-dns-perl liburi-perl libxml-simple-perl libclass-accessor-perl \
  libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \
  libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \
  libgnupg-interface-perl libmail-gnupg-perl perlmagick \
  libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \
  libstring-crc32-perl libtext-vcard-perl libxml-atom-perl libxml-rss-perl \
  libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \
  libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \
  libcaptcha-recaptcha-perl libdbd-sqlite3-perl

This will download about 123MB of files and use around 446MB of disk space.

Check whether your system is using threaded MPMs or pre-fork. You will need the latter.

If you have weird errors saying things like "Package mercurial is not available, but is referred to by another package.", try editing /etc/apt/sources.list and uncommenting the other repositories and then doing an apt-get update. Ran into this problem on a fresh install of jaunty on Linode.

You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay: 

perl -MCPAN -e 'install Bundle::CPAN'
perl -MCPAN -e 'install GTop' # this will guide you through setting up CPAN
perl -MCPAN -e 'install Unicode::CheckUTF8'
perl -MCPAN -e 'install Captcha::reCAPTCHA'
perl -MCPAN -e 'install IP::Country::Fast'

You will also want to install and configure Postfix so your DW can send out email: 

apt-get install postfix

If you choose not to configure on the install, you can do so later with: 

dpkg-reconfigure postfix

Setting up the DW user account

Note: If you still find yourself unable to make sudo commands even after usermod, please see Making your DW user a sudo account for alternatives.
Create a user account that will be used for the code. This can be your own personal account if you wish, but it is recommended that you setup a new user that is only going to be used for running the code. That way if you have any insecure code running (i.e., a bad page, or a patch you're working on doesn't have security implemented yet) you don't have to worry about someone getting access to your personal files.

To set up the user account: 

adduser username

Fill out the user's info and put them in the sudo group: 

usermod -a -G sudo username

So, to use dw as a username: 

adduser dw
usermod -a -G sudo dw

Now, we will get the $LJHOME variable set. Log into your user: 

su - dw

We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables.

For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file: 

export LJHOME=/home/dw

Or whatever your user account's home directory is. Yes, the variable is named LJHOME. We'll live with it for now.  ;-)

Test this. Log out of your user account and log back in, then type: 

echo $LJHOME

You should see /home/dw or whatever you set it to. If you don't, then you didn't get the export line in the right place. (Hey, I'm not here to teach basic sysadmin stuff. You should know how to set environment variables.)

Downloading the Dreamwidth code

You should do this as the user you want to run Dreamwidth as. If you are root, you can switch with: 

su - dw

We have put together a bootstrap script that will download all of the code. You can obtain this file with your favorite tool (wget, curl, etc): 

wget http://hg.dwscoalition.org/dw-free/raw-file/tip/bin/bootstrap.pl

The next step is fairly automatic. Assuming that your $LJHOME environment variable is good to go, you should be able to just do this: 

perl bootstrap.pl

The script will start doing some work. It should start out by saying 'seems we need to start at the beginning', which is what you expect. Then you will sit around for half an hour while it checks out various packages and repositories from various locations.

When it's done, it will say so. It will also say to delete the bootstrap script: 

rm bootstrap.pl

Do a quick ls in $LJHOME to see if you have a bunch of new directories like cvs, cgi-bin, htdocs, etc. If you do, huzzah! You now have the code checked out.

If you don't, figure out what step didn't work, and try again. All else fails, hit the mailing list or find someone on IRC.

dw-nonfree

Warning: This should not be used on installations that are not Dreamwidth development installations. It is currently a project to make sure that nothing in dw-free requires anything in dw-nonfree.

If you are doing development on Dreamwidth custom files (what few there are), then you will need to do another step. This is fairly manual, the bootstrap script does not handle this process for you. But assuming that you did the above steps, you can get dw-nonfree code like this: 

cd $LJHOME/cvs
hg clone http://hg.dwscoalition.org/dw-nonfree dw-nonfree
cp dw-nonfree/cvs/multicvs-local.conf .

You may want to look at Keeping your site configs from wiping during updates if you want to keep your config files from getting wiped during subsequent updates if you're using dw-nonfree.

Updating to the tip

As of right now, you have the code that existed up until the end of the initial branch with no name, ending in changeset 1defc551af6c from October 13th 2008. However, with the next commit on January 14th 2009, named branches were introduced, and the dw-free repository now uses all named branches. (currently there is only one named branch that is up to date, 'production')

Right now, bootstrap.pl gives you the SVN repository, rather than an HG which is currently used and as such cvsreport.pl fails because it expected HG.

Remove it and then do checkout: 

cd $LJHOME/cvs/dw-free
hg update -C tip
cd $LJHOME
rm -rf cvs/perlbal/
bin/cvsreport.pl -sync -cvsonly
bin/cvsreport.pl --checkout
bin/cvsreport.pl -sync -cvsonly


You need to do the last line twice, since the multicvs.conf file has changed since then. Any future updates should now work as described on the Dev Maintenance page. You will also want to delete files that no longer exist in the repository: 

cd $LJHOME; for i in `bin/cvsreport.pl -n -1`; do echo "Removing $i" && rm $i; done

Database setup

You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root: 

mysql -u root -p

These MySQL commands wil create your development database: 

create database dw;
grant all on dw.* to 'dw'@'localhost' identified by 'somePassword';
quit;

You might want to pick a more appropriate database username/database name/password.

Editing the config files

Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl for now: 

cd $LJHOME
cp doc/config-local.pl.txt etc/config-local.pl
vim etc/config-local.pl

Or use your editor of choice, of course. Do a search for the phrase 'CHANGE THIS' which is in three places. You will want to change at least your $DOMAIN and the %DBINFO structure.

The base configuration file is under source control, and is already in etc/config.pl. You should not need to change anything very much in this, but you might have to change some stuff. If you find you do have to touch this, please see [1] first.

Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.

Apart from this, the configuration should be fairly self-explanatory, except for one thing - do not place your database password in the config-local.pl file! Instead, place it in the config-private.pl file: 

cd $LJHOME
cp doc/config-private.pl.txt etc/config-private.pl

(if you didn't already do that above) 

chmod go-rwx etc/config-private.pl
vim etc/config-private.pl

In this file, uncomment the lines in the %DBINFO block, and place your database password in quotes on the "pass => ," line, before the comma. For example: 

   %DBINFO = (
       master => {
           pass => 'mypassword',
       },
   );

As mentioned, there are also settings to check out in etc/config.pl: 

vim etc/config.pl

Make sure things are working with checkconfig.pl

Now, you need to see if everything is working. Before the next step you will need to apply this patch locally DW patch prior to checkconfig

If you've setup the files as indicated above, run this command: 

cd $LJHOME
bin/checkconfig.pl --no=ljconfig

If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC or the mailing list.

Populate database with initial data

There are a few commands you can now run to install the database. Just run these and watch for errors.

Note: If make_system.pl says it can't give the system user admin privileges, something has gone wrong with your database population, even if there were no errors. 

$LJHOME/bin/upgrading/update-db.pl -r --innodb
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
$LJHOME/bin/upgrading/update-db.pl -p
$LJHOME/bin/upgrading/make_system.pl

(That step will ask you for a password for the system account. You can change it later by logging in as system, so just give it something for now.) 

$LJHOME/bin/upgrading/texttool.pl load
$LJHOME/bin/upgrading/copy-emailpass-out-of-user

Configure Apache

This step will need to be done as the root user. Below is the Apache 2 configuration running on the Dreamwidth staging site. Put this in a file named "stage" in /etc/apache2/conf.d: 

User dw
Group dw
UseCanonicalName off

StartServers 1
MaxSpareServers 2
MinSpareServers 1
   
DocumentRoot /home/dw/htdocs
   
PerlSetEnv  LJHOME /home/dw
PerlPassEnv LJHOME
   
PerlRequire  /home/dw/cgi-bin/modperl.pl

Then disable the default site: 

a2dissite default

You might also have to enable the Perl Apache Request module: 

a2enmod apreq

If you don't want this warning: 

[Thu Jan 15 01:46:54 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
 ... waiting [Thu Jan 15 01:46:55 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
  ...done.

Then use vim /etc/apache2/ports.conf and put a # in front of the NameVirtualHost line: 

# NameVirtualHost *:80

Restart the server: 

/etc/init.d/apache2 restart

Now what?

Further setup

There are lots of other articles on setting up and customizing your DW install in the DW Installation category, including:

There is a list of wanted how-tos at Installation Wanted How-To, in case you need to add something to the list.

Starting development

The list of things that need doing are in Bugzilla. You can keep your code update with the instructions in Dev Maintenance. Instructions on submitting patches are in Dev Patches.

Retrieved from "http://wiki.dreamwidth.net/wiki/index.php?title=Dreamwidth_Scratch_Installation&oldid=5869"