Dreamwidth Scratch Installation

Firstly, you'll need to set up a Linux system -- here are the Suggested Server Requirements. These instructions assume Ubuntu 12.04 or up. Please let us know if you run into difficulties--we want to update this if people are having problems! We recommend coming into the #dreamwidth-dev channel on IRC.

After you set up, you can keep updated with the instructions on Dev Maintenance.

If you have difficulties or problems or suggestions for installation procedures, please report them to dw_dev. Also look at Production Setup: Webserver.

Bringing the system up to date

Before you start installing anything else, you might want to make sure your system is up to date. You should update the packages on your system (either in a root shell (sudo bash) or preceding these commands with 'sudo'):

apt-get update
apt-get upgrade

If you want man pages on your system:

apt-get install man

Installing necessary packages

This will install necessary packages you'll need to run Dreamwidth (and some optional ones). You'll want to be root when running these commands.

apt-get install git-core apache2-mpm-prefork \
  libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \
  mysql-server wget unzip links vim libclass-autouse-perl \
  libdatetime-perl libcache-memcached-perl libhash-multivalue-perl \
  libgd-gd2-perl libhtml-template-perl libwww-perl libmime-lite-perl \
  libnet-dns-perl liburi-perl libxml-simple-perl libclass-accessor-perl \
  libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \
  libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \
  libgnupg-interface-perl libmail-gnupg-perl perlmagick \
  libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \
  libstring-crc32-perl libtext-vcard-perl libxml-atom-perl libxml-rss-perl \
  libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \
  libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \
  libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \
  libtemplate-perl libterm-readkey-perl libmime-base64-urlsafe-perl \
  gcc libtest-most-perl libgearman-client-perl \
  libbusiness-creditcard-perl liblwpx-paranoidagent-perl \
  libtheschwartz-perl libfile-type-perl libjson-perl ruby

This will download about 113MB of files and use around 355MB of disk space.

Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again.

Then

Check whether your system is using threaded MPMs or pre-fork. You will need the latter. (Note: if you ran the above command to install Apache, then you have the pre-fork version installed.)

If you have weird errors saying things like "Package mercurial is not available, but is referred to by another package.", try editing /etc/apt/sources.list and uncommenting the other repositories and then doing an apt-get update. Ran into this problem on a fresh install of jaunty on Linode.

You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay.

Before installing, check if your CPAN shell has 'make' at the correct location. Run which make, then compare the results with running o conf make at the CPAN shell (to run the CPAN shell, type cpan.)

The perl libraries are:

cpan Bundle::CPAN   # this will guide you through setting up CPAN
cpan GTop  
cpan Digest::SHA1
cpan Unicode::CheckUTF8
cpan MogileFS::Client   # this is necessary even if you don't use MogileFS
cpan TheSchwartz::Worker::SendEmail
cpan Digest::SHA256
cpan Text::Markdown
cpan Image::ExifTool
cpan Net::OAuth
cpan LWP::UserAgent::Paranoid
cpan Mozilla::CA
cpan Net::SMTPS
cpan XMLRPC::Lite

See Bundle::CPAN if you are having trouble installing Bundle::CPAN

You will also want to install and configure Postfix so your DW can send out email:

apt-get install postfix

If you choose not to configure on the install, you can do so later with:

dpkg-reconfigure postfix

Set Timezone to UTC

Dreamwidth will complain if you aren't running in UTC. If you're not configured for UTC already, you can reset it (on Ubuntu/Debian) using

dpkg-reconfigure tzdata

and choose Etc/UTC.

Setting up the DW user account

Note: If you still find yourself unable to make sudo commands even after usermod, please see Making your DW user a sudo account for alternatives.

Create a user account that will be used for the code. This can be your own personal account if you wish, but it is recommended that you setup a new user that is only going to be used for running the code. That way if you have any insecure code running (i.e., a bad page, or a patch you're working on doesn't have security implemented yet) you don't have to worry about someone getting access to your personal files.

To set up the user account:

adduser username

Fill out the user's info and put them in the sudo group:

usermod -a -G sudo username

So, to use dw as a username:

adduser dw
usermod -a -G sudo dw

Now, we will get the $LJHOME variable set. Log into your user:

su - dw

We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables.

For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file:

export LJHOME=/home/dw/dw

Or wherever it is that you want the root of your source code install to be. Yes, the variable is named LJHOME. We'll live with it for now.  ;-)

Test this. Log out of the dw user account and log back in, then type:

echo $LJHOME

You should see /home/dw/dw or whatever you set it to. If you don't, then something has gone awry -- maybe the export line isn't in the right place..?

Get the Dreamwidth code

Set up GitHub

First, create an account on Github. Then, you will need to fork these two repositories:

• https://github.com/dreamwidth/dw-free
• https://github.com/dreamwidth/dw-nonfree

Go to each of their pages, and click the "Fork" button. This sets up your own copy of Dreamwidth's code which use to make and submit your changes.

Here's a quick overview of how the repositories will work together:

• dw-free is the main repository and goes into $LJHOME
• other repositories will go into $LJHOME/ext
• personal config files go into $LJHOME/ext/local
• All code from dw-free ($LJHOME) and the additional repos under $LJHOME/ext are automatically live when you start your server. There's no need to run any additional syncing steps anymore.

Set up dw-free

Replace the USERNAME sections of the URL with your Github username. This will make your repository an authenticated version that can push changes back to Github, as opposed to only being able to pull them.

# clone a copy of the repository onto your machine
cd ~/
git clone https://USERNAME@github.com/USERNAME/dw-free.git $LJHOME
cd $LJHOME

# and let's make it aware of the dreamwidth repository so we can grab updates later
git remote add dreamwidth https://github.com/dreamwidth/dw-free
git fetch dreamwidth

# we also want to sync up with the main branches
# for Git 1.7 or earlier:
git branch --set-upstream develop dreamwidth/develop
# for Git 1.8 or later:
git branch -u dreamwidth/develop develop

Set up dw-nonfree

Warning: For dev servers only. Clone sites should not use dw-nonfree because it contains Dreamwidth-specific branding and files

cd $LJHOME/ext
git clone https://USERNAME@github.com/USERNAME/dw-nonfree.git
cd dw-nonfree
git remote add dreamwidth https://github.com/dreamwidth/dw-nonfree
git fetch dreamwidth

# for Git 1.7 or earlier:
git branch --set-upstream develop dreamwidth/develop
# for Git 1.8 or later:
git branch -u dreamwidth/develop develop
cd ../..

Database setup

You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root:

mysql -u root -p  

These MySQL commands will create your development database:

create database dw;
grant all on dw.* to 'dw'@'localhost' identified by 'somePassword';

That last line is made up of a few different sections:

• grant all on dw.* - this tells MySQL to grant all permissions to the "dw" database which you created in the first line.
• to 'dw'@'localhost' - this tells MySQL to grant the above permissions to a username called "dw" connecting from "localhost" (which means the same computer as the one MySQL is on). You can change "dw" to another username if you like. NB: this is 'dw'@'localhost', not 'dw@localhost' - either cut and paste, or type carefully!
• identified by 'somePassword' - since this user doesn't exist yet, we can tell MySQL to create it by adding this. This tells MySQL that the "dw" user will use the password "somePassword" to connect. You probably want to change this password!

While you're here, you probably want to go ahead and create a database for TheSchwartz, too, unless you really know that you won't need it:

create database dw_schwartz;
grant all on dw_schwartz.* to 'dw'@'localhost';

(We're leaving out the "identified by" part this time because the user already exists.)

When you're done, exit out of MySQL:

quit;

Editing the config files

Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now:

## copy over your config files, and tell them to take priority
## over config files in any of the repos
cd $LJHOME
mkdir -p ext/local/etc
echo "highest"  > ext/local/.dir_scope
cp doc/config-local.pl.txt ext/local/etc/config-local.pl
vim ext/local/etc/config-local.pl

Or use your editor of choice, of course. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation.

Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.

Next, you need to change your local variables in the config-private.pl file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your $DOMAIN and the %DBINFO structure.

cd $LJHOME
cp doc/config-private.pl.txt ext/local/etc/config-private.pl
chmod go-rwx ext/local/etc/config-private.pl
vim ext/local/etc/config-private.pl

In this file, uncomment the lines in the %DBINFO blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example:

   %DBINFO = (
       master => {
           pass => 'mypassword',
       },
   );

Note that there are two DBINFO blocks; get them both.

If you created the database for TheSchwartz earlier, you'll also want to configure it:

   # Schwartz DB configuration
   @THESCHWARTZ_DBS = (
           {
               dsn => 'dbi:mysql:dw_schwartz;host=localhost',
               user => 'dw',
               pass => ,     # CHANGETHIS
           },
       );

The base configuration file is under source control, and is already in etc/config.pl. You should not need to change anything very much in this, but you might have to change some stuff. If you find you do have to touch this, copy it over first, then you can edit it:

cd $LJHOME
cp etc/config.pl ext/local/etc/config.pl
vim ext/local/etc/config.pl

Note that from here on, when any documentation says to edit etc/config*, you'll likely want to edit ext/local/etc/config-* instead.

Make sure things are working with checkconfig.pl

Now, you need to see if everything is working.

If you've setup the files as indicated above, run this command:

cd $LJHOME
bin/checkconfig.pl --no=ljconfig

If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC, the mailing list, or make a post to dw_dev.

Populate database with initial data

There are a few commands you can now run to install the database. Just run these and watch for errors.

Note: If make_system.pl says it can't give the system user admin privileges, something has gone wrong with your database population, even if there were no errors.

$LJHOME/bin/upgrading/update-db.pl -r --innodb
$LJHOME/bin/upgrading/update-db.pl -r --innodb # at least for now we have to run this twice
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
$LJHOME/bin/upgrading/update-db.pl -p

If you created your database for TheSchwartz earlier, you'll also want to populate it with its default tables:

mysql -u dw -p dw_schwartz < /usr/share/doc/libtheschwartz-perl/schema.sql

Create a system account:

$LJHOME/bin/upgrading/make_system.pl

(That step will ask you for a password for the System account. You can change it later by logging in as system, so just give it something for now.)

And load in the translation strings:

$LJHOME/bin/upgrading/texttool.pl load

Compile Foundation stylesheets

Follow the SCSS instructions for installing compass and sass:

gem install sass --version 3.2.19 
gem install compass --version 0.12.2

Then compile the stylesheets for dw-free:

cd $LJHOME
compass compile

If you are also installing dw-nonfree, you will need to compile in that directory as well:

cd $LJHOME/ext/dw-nonfree
compass compile

Configure Apache

This step will need to be done as the root user. Below is the Apache 2 configuration running on the Dreamwidth staging site. Put this in a file named "stage" (which you may need to create) in /etc/apache2/conf.d:

User dw
Group dw
UseCanonicalName off

StartServers 1
MaxSpareServers 2
MinSpareServers 1
   
DocumentRoot /home/dw/dw/htdocs
   
PerlSetEnv  LJHOME /home/dw/dw
PerlPassEnv LJHOME
   
PerlRequire  /home/dw/dw/cgi-bin/modperl.pl

Then disable the default site:

a2dissite default

You might also have to enable the Perl Apache Request module:

a2enmod apreq

If you don't want this warning:

[Thu Jan 15 01:46:54 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
 ... waiting [Thu Jan 15 01:46:55 2009] [warn] NameVirtualHost *:80 has no VirtualHosts
  ...done.

Then use vim /etc/apache2/ports.conf and put a # in front of the NameVirtualHost line:

# NameVirtualHost *:80

Restart the server:

/etc/init.d/apache2 restart

Now what?

Have a look at your new DW instance

Congratulations! You now have a working (though minimal) Dreamwidth install. If you point your web browser at your server, you should see a bare-looking welcome page.

Further setup

There are lots of other articles on setting up and customizing your DW install in the DW Installation category, including:

• TheSchwartz Setup - TheSchwartz is needed for a handful of features - notably comment posting
• Subdomain setup
• Statistics setup
• Generating documentation
• Allow users to register
• Setting up Search

There is a list of wanted how-tos at Installation Wanted How-To, in case you need to add something to the list.

Starting development

If you haven't already, see Dev Getting Started.