Dreamwidth Scratch Installation

From Dreamwidth Notes
Revision as of 00:42, 18 August 2012 by Foxfirefey (Talk | contribs)

Jump to: navigation, search
Needs Update: Much of this is invalid since our GitHub move!

Firstly, you'll need to set up a Linux system; these instructions assume Ubuntu 9.04 or up. Please let us know if you run into difficulties--we want to update this if people are having problems! We recommend coming into the #dreamwidth-dev channel on IRC.

After you set up, you can keep updated with the instructions on Dev Maintenance.

If you have difficulties or problems or suggestions for installation procedures, please report them to Bug 61. Also look at Production Setup: Webserver.

Bringing the system up to date

Before you start installing anything else, you might want to make sure your system is up to date. You should log in as root and update the packages on your system: 

apt-get update
apt-get upgrade

If you want man pages on your system: 

apt-get install man

Installing necessary packages

This will install necessary packages you'll need to run Dreamwidth (and some optional ones). You'll want to be root when running these commands.

If you are using lucid (or a similar/newer version of Ubuntu)

If you are using lucid, you can add exor674's ppa and then install all the needed DW packages by doing 

add-apt-repository ppa:anall/dw
apt-get update
apt-get install mercurial subversion apache2-mpm-prefork \
  libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \
  mysql-server wget unzip links vim dreamwidth-packages gcc

And that should pull in all required packages ( if it misses some Perl packages, feel free to poke [info]exor674

Please note that these packages are unofficial, and you should only install them if you trust Dre, there is no guarantee that there will be no security issues in any of the packages in the ppa, if this concerns you, do not follow these instructions and instead follow the instructions below.

Otherwise

apt-get install mercurial git-core subversion apache2-mpm-prefork \
  libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \
  mysql-server wget unzip links vim libclass-autouse-perl \
  libcompress-zlib-perl libdatetime-perl libdigest-sha1-perl \
  libgd-gd2-perl libhtml-template-perl libwww-perl libmime-lite-perl \
  libnet-dns-perl liburi-perl libxml-simple-perl libclass-accessor-perl \
  libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \
  libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \
  libgnupg-interface-perl libmail-gnupg-perl perlmagick \
  libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \
  libstring-crc32-perl libtext-vcard-perl libxml-atom-perl libxml-rss-perl \
  libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \
  libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \
  libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \
  libtest-simpleunit-perl libtemplate-perl libterm-readkey-perl \
  libextutils-cbuilder-perl gcc libtest-most-perl \
  libbusiness-creditcard-perl liblwpx-paranoidagent-perl \
  libtheschwartz-perl libfile-type-perl

This will download about 123MB of files and use around 446MB of disk space.

Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again.

If you have trouble with libdigest-sha1-perl not being available on Ubuntu 12.04, install dh-make-perl and do dh-make-perl --cpan digest-sha1 instead.

If you can't get libcompress-zlib-perl, then libio-compress-perl is an acceptable substitute.

Then

Check whether your system is using threaded MPMs or pre-fork. You will need the latter.

If you have weird errors saying things like "Package mercurial is not available, but is referred to by another package.", try editing /etc/apt/sources.list and uncommenting the other repositories and then doing an apt-get update. Ran into this problem on a fresh install of jaunty on Linode.

You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay.

Before installing, check if your CPAN shell has 'make' at the correct location. Run which make, then compare the results with running o conf make at the CPAN shell (to run the CPAN shell, type cpan.)

The perl libraries are: 

cpan Bundle::CPAN
cpan GTop   # this will guide you through setting up CPAN
cpan Unicode::CheckUTF8
cpan Captcha::reCAPTCHA
cpan Hash::MultiValue
cpan MogileFS::Client   # this is necessary even if you don't use MogileFS
cpan TheSchwartz::Worker::SendEmail

See Bundle::CPAN if you are having trouble installing Bundle::CPAN

You will also want to install and configure Postfix so your DW can send out email: 

apt-get install postfix

If you choose not to configure on the install, you can do so later with: 

dpkg-reconfigure postfix

Setting up the DW user account

Note: If you still find yourself unable to make sudo commands even after usermod, please see Making your DW user a sudo account for alternatives.
Create a user account that will be used for the code. This can be your own personal account if you wish, but it is recommended that you setup a new user that is only going to be used for running the code. That way if you have any insecure code running (i.e., a bad page, or a patch you're working on doesn't have security implemented yet) you don't have to worry about someone getting access to your personal files.

To set up the user account: 

adduser username

Fill out the user's info and put them in the sudo group: 

usermod -a -G sudo username

So, to use dw as a username: 

adduser dw
usermod -a -G sudo dw

Now, we will get the $LJHOME variable set. Log into your user: 

su - dw

We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables.

For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file: 

export LJHOME=/home/dw

Or whatever your user account's home directory is. Yes, the variable is named LJHOME. We'll live with it for now.  ;-)

Test this. Log out of your user account and log back in, then type: 

echo $LJHOME

You should see /home/dw or whatever you set it to. If you don't, then you didn't get the export line in the right place. (Hey, I'm not here to teach basic sysadmin stuff. You should know how to set environment variables.)

Downloading the Dreamwidth code

You should do this as the user you want to run Dreamwidth as. If you are root, you can switch with: 

su - dw

We have put together a bootstrap script that will download all of the code. You can obtain this file with your favorite tool (wget, curl, etc): 

wget http://hg.dwscoalition.org/dw-free/raw-file/tip/bin/bootstrap.pl

The next step is fairly automatic. Assuming that your $LJHOME environment variable is good to go, you should be able to just do this: 

perl bootstrap.pl

The script will start doing some work. It should start out by saying 'seems we need to start at the beginning', which is what you expect. This may take some time while it checks out various packages and repositories from various locations.

When it's done, it will say so. It will also say to delete the bootstrap script: 

rm bootstrap.pl

Do a quick ls in $LJHOME to see if you have a bunch of new directories like cvs, cgi-bin, htdocs, etc. If you do, huzzah! You now have the code checked out.

If you don't, figure out what step didn't work, and try again. All else fails, hit the mailing list or find someone on IRC.

dw-nonfree

Warning: This should not be used on installations that are not Dreamwidth development installations.

If you are doing development on Dreamwidth custom files (what few there are), then you will need to do another step. This is fairly manual, the bootstrap script does not handle this process for you. But assuming that you did the above steps, you can get dw-nonfree code like this: 

cd $LJHOME/cvs
hg clone http://hg.dwscoalition.org/dw-nonfree dw-nonfree
cp dw-nonfree/cvs/multicvs-local.conf .

You may want to look at Keeping your site configs from wiping during updates if you want to keep your config files from getting wiped during subsequent updates if you're using dw-nonfree.

Updating to the tip

Because all named branches have been closed as of July 19th 2010, you should have the most up-to-date code already checked out, although you may need to do hg update -C tip to get the most recent changes.

Some repositories have changed locations and must be manually removed if they already exist in the cvs directory. 

cd $LJHOME/cvs/dw-free
hg update -C tip
cd $LJHOME
bin/cvsreport.pl -sync -cvsonly
rm -rf cvs/perlbal/
rm -rf cvs/js/
bin/cvsreport.pl --checkout
bin/cvsreport.pl -sync -cvsonly


You need to do the last line twice, since the multicvs.conf file has changed since then. Any future updates should now work as described on the Dev Maintenance page. You will also want to delete files that no longer exist in the repository: 

cd $LJHOME
for i in $(bin/cvsreport.pl -n -1); do 
   echo "Removing $i" && rm $i; 
done

Database setup

You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root: 

mysql -u root -p

These MySQL commands wil create your development database: 

create database dw;
grant all on dw.* to 'dw'@'localhost' identified by 'somePassword';
quit;

You might want to pick a more appropriate database username/database name/password.

Editing the config files

Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now: 

cd $LJHOME
cp doc/config-local.pl.txt etc/config-local.pl
vim etc/config-local.pl

Or use your editor of choice, of course. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation.

Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.

Next, you need to change your local variables in the config-private.pl file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your $DOMAIN and the %DBINFO structure. 

cd $LJHOME
cp doc/config-private.pl.txt etc/config-private.pl
chmod go-rwx etc/config-private.pl
vim etc/config-private.pl

In this file, uncomment the lines in the %DBINFO blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example: 

   %DBINFO = (
       master => {
           pass => 'mypassword',
       },
   );

Note that there are two DBINFO blocks; get them both.

The base configuration file is under source control, and is already in etc/config.pl. You should not need to change anything very much in this, but you might have to change some stuff. If you find you do have to touch this, please see [1] first. Then you can edit it: 

vim etc/config.pl

Make sure things are working with checkconfig.pl

Now, you need to see if everything is working.

If you've setup the files as indicated above, run this command: 

cd $LJHOME
bin/checkconfig.pl --no=ljconfig

If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC or the mailing list.

Populate database with initial data

There are a few commands you can now run to install the database. Just run these and watch for errors.

Note: If make_system.pl says it can't give the system user admin privileges, something has gone wrong with your database population, even if there were no errors. 

$LJHOME/bin/upgrading/update-db.pl -r --innodb
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
$LJHOME/bin/upgrading/update-db.pl -p

(That step will ask you for a password for the System account. You can change it later by logging in as system, so just give it something for now.) 

$LJHOME/bin/upgrading/make_system.pl
$LJHOME/bin/upgrading/texttool.pl load

Configure Apache

This step will need to be done as the root user. Below is the Apache 2 configuration running on the Dreamwidth staging site. Put this in a file named "stage" in /etc/apache2/conf.d: 

User dw
Group dw
UseCanonicalName off

StartServers 1
MaxSpareServers 2
MinSpareServers 1
   
DocumentRoot /home/dw/htdocs
   
PerlSetEnv  LJHOME /home/dw
PerlPassEnv LJHOME
   
PerlRequire  /home/dw/cgi-bin/modperl.pl

Then disable the default site: 

a2dissite default

You might also have to enable the Perl Apache Request module: 

a2enmod apreq

If you don't want this warning: 

[Thu Jan 15 01:46:54 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
 ... waiting [Thu Jan 15 01:46:55 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
  ...done.

Then use vim /etc/apache2/ports.conf and put a # in front of the NameVirtualHost line: 

# NameVirtualHost *:80

Restart the server: 

/etc/init.d/apache2 restart

Now what?

Have a look at your new DW instance

Congratulations! You now have a working (though minimal) Dreamwidth install. If you point your web browser at your server, you should see a bare-looking welcome page.

Further setup

There are lots of other articles on setting up and customizing your DW install in the DW Installation category, including:

There is a list of wanted how-tos at Installation Wanted How-To, in case you need to add something to the list.

Starting development

The list of things that need doing are in Bugzilla. You can keep your code up to date with the instructions in Dev Maintenance. Instructions on submitting patches are in Dev Patches.

Retrieved from "http://wiki.dreamwidth.net/wiki/index.php?title=Dreamwidth_Scratch_Installation&oldid=10683"