Difference between revisions of "Dreamwidth Scratch Installation"
(→Otherwise: Add liblwpx-paranoidagent-perl to list.) |
(→Further setup: DNS and email) |
||
(90 intermediate revisions by 13 users not shown) | |||
Line 1: | Line 1: | ||
− | Firstly, you'll need to set up a Linux system | + | |
+ | Firstly, you'll need to set up a Linux system -- here are the [[Suggested Server Requirements]]. These instructions assume Ubuntu 16.04. Later versions (as of December 18, 2017) may or may not work, see https://github.com/dreamwidth/dw-free/issues/2248. Please let us know if you run into difficulties--we want to update this if people are having problems! We recommend coming into the #dreamwidth-dev channel on [[IRC]]. | ||
After you set up, you can keep updated with the instructions on [[Dev Maintenance]]. | After you set up, you can keep updated with the instructions on [[Dev Maintenance]]. | ||
− | If you have difficulties or problems or suggestions for installation procedures, please report them to | + | If you have difficulties or problems or suggestions for installation procedures, please report them to <dwuser>dw_dev</dwuser>. Also look at [[Production Setup: Webserver]]. |
− | == | + | == Command convention == |
− | + | In this walkthrough, commands to be run as a regular user are prefixed with <tt>$</tt>. Commands to be run as the super-user are prefixed with <tt>#</tt>. If you are not the super-user, you can run single commands like this: | |
− | + | $ sudo <command goes here> | |
− | + | ||
− | + | Alternately, you can spawn a super-user shell using: | |
− | + | $ sudo -i | |
+ | # <command goes here> | ||
− | + | When done with the super-user shell, exit the shell via: | |
− | + | # exit | |
− | == | + | == Configuring the OS and environment== |
− | + | === Bringing the system up to date === | |
− | + | Before you start installing anything else, you might want to make sure your system is up to date. You should update the packages on your system: | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | # apt-get update | |
+ | # apt-get upgrade | ||
− | + | If you want man pages on your system: | |
− | + | # apt-get install man | |
− | apt-get install | + | === Installing necessary packages === |
+ | |||
+ | This will install necessary packages you'll need to run Dreamwidth (and some optional ones). You'll want to be root when running these commands. | ||
+ | |||
+ | # apt-get install git apache2 apache2-bin apache2-data apache2-utils \ | ||
libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \ | libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \ | ||
− | mysql-server wget unzip links vim libclass-autouse-perl \ | + | mysql-server wget unzip links vim libclass-autouse-perl libdatetime-perl \ |
− | + | libcache-memcached-perl libhash-multivalue-perl libgd-gd2-perl \ | |
− | + | libhtml-template-perl libwww-perl libmime-lite-perl \ | |
− | + | liburi-perl libxml-simple-perl libclass-accessor-perl \ | |
libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \ | libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \ | ||
libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \ | libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \ | ||
libgnupg-interface-perl libmail-gnupg-perl perlmagick \ | libgnupg-interface-perl libmail-gnupg-perl perlmagick \ | ||
libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \ | libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \ | ||
− | libstring-crc32 | + | libstring-crc32-perl libxml-atom-perl libxml-rss-perl \ |
libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \ | libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \ | ||
libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \ | libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \ | ||
libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \ | libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \ | ||
− | + | libtemplate-perl libterm-readkey-perl libmime-base64-urlsafe-perl \ | |
− | + | gcc libtest-most-perl libgearman-client-perl libfile-find-rule-perl \ | |
− | libbusiness-creditcard-perl liblwpx-paranoidagent-perl | + | libbusiness-creditcard-perl liblwpx-paranoidagent-perl libtheschwartz-perl \ |
+ | libfile-type-perl libjson-perl ruby libdbd-mysql-perl libdanga-socket-perl \ | ||
+ | libio-aio-perl libsys-syscall-perl liblog-log4perl-perl libtext-markdown-perl \ | ||
+ | libimage-exiftool-perl libnet-oauth-perl libnet-smtps-perl libxmlrpc-lite-perl | ||
− | This will download about | + | This will download about 106MB of files and use around 511MB of disk space. |
Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again. | Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again. | ||
− | + | You will also want to install and configure Postfix so your DW can send out email: | |
− | If you | + | # apt-get install postfix |
+ | |||
+ | If you choose not to configure on the install, you can do so later with: | ||
+ | |||
+ | # dpkg-reconfigure postfix | ||
+ | |||
+ | === Configure Apache 2 === | ||
+ | |||
+ | [http://perl.apache.org/docs/2.0/user/install/install.html#Prerequisites Check] whether your system is using threaded MPMs or pre-fork. You will need the latter. <em>Note</em>: if you ran the above command to install Apache, then you need to run these commands to switch to the pre-fork model: | ||
+ | |||
+ | # a2dismod mpm_event | ||
+ | # a2enmod mpm_prefork | ||
+ | # service apache2 restart | ||
+ | |||
+ | === Install Perl modules with CPAN === | ||
You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay. | You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay. | ||
− | Before installing, check if your CPAN shell has | + | Before installing, check if your CPAN shell has <tt>make</tt> at the correct location. Run |
− | + | # which make | |
− | + | /usr/bin/make | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | then compare the results with running: | |
− | + | # cpan | |
+ | ... | ||
+ | Would you like to configure as much as possible automatically? [yes] <PRESS ENTER> | ||
+ | ... | ||
+ | cpan[1]> o conf make | ||
+ | make [/usr/bin/make] | ||
+ | ... | ||
+ | cpan[2]> exit | ||
− | + | The value printed should match that from the <tt>which make</tt> command you previously ran. | |
− | + | To install the perl libraries, first set up CPAN: | |
+ | |||
+ | # cpan Bundle::CPAN | ||
+ | |||
+ | When prompted, press Enter to exit tests. Then, run the following command to install the rest of the necessary libraries: | ||
+ | |||
+ | # cpan GTop Digest::SHA1 Unicode::CheckUTF8 MogileFS::Client \ | ||
+ | TheSchwartz::Worker::SendEmail LWP::UserAgent::Paranoid \ | ||
+ | Mozilla::CA List::Util Paws::S3 Net::DNS Text::Fuzzy | ||
+ | |||
+ | (Net::DNS is required to be installed via CPAN because the version included with current versions of Ubuntu is 0.81, which is too old for LWP::UserAgent::Paranoid to work properly.) | ||
+ | |||
+ | Note that the MogileFS::Client is required even if you are not configuring MogileFS. | ||
+ | |||
+ | === Set Timezone to UTC === | ||
+ | |||
+ | Dreamwidth will complain if you aren't running in UTC. If you're not configured for UTC already, you can reset it (on Ubuntu/Debian) using | ||
+ | |||
+ | # dpkg-reconfigure tzdata | ||
+ | |||
+ | and choose Etc/UTC. | ||
+ | |||
+ | === Configuring MySQL/MariaDB === | ||
+ | |||
+ | Newer versions of MySQL and/or MariaDB have imposed stricter requirements than the current Dreamwidth SQL definitions can comply with. These restrictions must be relaxed, or the initial database population scripts will fail. | ||
+ | |||
+ | To do this, run the following 4 commands: | ||
− | + | $ sudo -i | |
+ | # <nowiki>echo "sql_mode = ''" >> /etc/mysql/mysql.conf.d/mysqld.cnf</nowiki> | ||
+ | # service mysql restart | ||
+ | # exit | ||
− | == Setting up the DW user account == | + | === Setting up the DW user account === |
{{Note|text=If you still find yourself unable to make sudo commands even after <code>usermod</code>, please see [[Making your DW user a sudo account]] for alternatives.}} Create a user account that will be used for the code. This can be your own personal account if you wish, but it is recommended that you setup a new user that is only going to be used for running the code. That way if you have any insecure code running (i.e., a bad page, or a patch you're working on doesn't have security implemented yet) you don't have to worry about someone getting access to your personal files. | {{Note|text=If you still find yourself unable to make sudo commands even after <code>usermod</code>, please see [[Making your DW user a sudo account]] for alternatives.}} Create a user account that will be used for the code. This can be your own personal account if you wish, but it is recommended that you setup a new user that is only going to be used for running the code. That way if you have any insecure code running (i.e., a bad page, or a patch you're working on doesn't have security implemented yet) you don't have to worry about someone getting access to your personal files. | ||
Line 90: | Line 140: | ||
To set up the user account: | To set up the user account: | ||
− | adduser <i>username</i> | + | # adduser <i>username</i> |
Fill out the user's info and put them in the sudo group: | Fill out the user's info and put them in the sudo group: | ||
− | usermod -a -G sudo <i>username</i> | + | # usermod -a -G sudo <i>username</i> |
So, to use <code>dw</code> as a username: | So, to use <code>dw</code> as a username: | ||
− | adduser dw | + | # adduser dw |
− | usermod -a -G sudo dw | + | # usermod -a -G sudo dw |
Now, we will get the $LJHOME variable set. Log into your user: | Now, we will get the $LJHOME variable set. Log into your user: | ||
− | su - dw | + | # su - dw |
We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables. | We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables. | ||
− | For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file: | + | For bash, you should look in the home directory of the user account you just created for a file <tt>.profile</tt>. You will want to add one more line to this file: |
− | export LJHOME=/home/dw | + | export LJHOME=/home/dw/dw |
− | Or | + | Or wherever it is that you want the root of your source code install to be. Yes, the variable is named LJHOME. We'll live with it for now. ;-) |
− | Test this. Log out of | + | Test this. Log out of the dw user account and log back in, then type: |
echo $LJHOME | echo $LJHOME | ||
− | You should see /home/dw or whatever you set it to. If you don't, then | + | You should see <tt>/home/dw/dw</tt> or whatever you set it to. If you don't, then something has gone awry -- maybe the export line isn't in the right place..? |
− | == | + | == Get the Dreamwidth code == |
− | + | === Configure git === | |
− | + | Pick up a name and email address you're willing to share with the world (or at least that part of it that pays attention to GitHub and Dreamwidth). Then run the following 2 commands, replacing NAME and EMAIL with what you chose: | |
− | + | $ git config --global user.name "NAME" | |
+ | $ git config --global user.email "EMAIL" | ||
− | + | Optionally, make git use your preferred editor for commit messages (and a few other things). First find the path to your favorite editor (replace EDITOR with the name of your preferred editor, eg nano or vi): | |
− | + | $ which EDITOR | |
− | + | This will print something like: <code>/usr/bin/vi</code> (if your editor of choice is vi). Then, run the following command, replacing /PATH/TO/EDITOR with whatever was output: | |
− | + | $ git config --global core.editor "/PATH/TO/EDITOR" | |
− | + | === Set up GitHub === | |
− | + | First, create an account on Github. Then, you will need to fork these two repositories: | |
− | + | * https://github.com/dreamwidth/dw-free | |
+ | * https://github.com/dreamwidth/dw-nonfree | ||
− | + | Go to each of their pages, and click the "Fork" button. This sets up your own copy of Dreamwidth's code which you will use to make and submit your changes. | |
− | + | Here's a quick overview of how the repositories will work together: | |
− | + | * dw-free is the main repository and goes into $LJHOME | |
+ | * other repositories will go into $LJHOME/ext | ||
+ | * personal config files go into $LJHOME/ext/local | ||
+ | * All code from dw-free ($LJHOME) and the additional repos under $LJHOME/ext are automatically live when you start your server. There's no need to run any additional syncing steps anymore. | ||
− | + | === Set up dw-free === | |
− | + | Replace the USERNAME sections of the URL with your Github username. This will make your repository an authenticated version that can push changes back to Github, as opposed to only being able to pull them. | |
− | + | ||
− | + | ||
− | + | First, clone a copy of the repository onto your machine: | |
− | + | $ cd ~/ | |
+ | $ git clone https://USERNAME@github.com/USERNAME/dw-free.git $LJHOME | ||
+ | $ cd $LJHOME | ||
+ | |||
+ | Now, let's make it aware of the dreamwidth repository so we can grab updates later: | ||
− | + | $ git remote add dreamwidth https://github.com/dreamwidth/dw-free | |
− | + | $ git fetch dreamwidth | |
− | + | ||
+ | Finally, let's set up to sync up with the main repository's master branch: | ||
− | + | $ git branch -u dreamwidth/master master | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
+ | === Set up dw-nonfree === | ||
− | + | {{ Warn|text=For dev servers only. Clone sites should not use dw-nonfree because it contains Dreamwidth-specific branding and files.}} | |
− | + | $ cd $LJHOME/ext | |
+ | $ git clone https://USERNAME@github.com/USERNAME/dw-nonfree.git | ||
+ | $ cd dw-nonfree | ||
+ | $ git remote add dreamwidth https://github.com/dreamwidth/dw-nonfree | ||
+ | $ git fetch dreamwidth | ||
+ | $ git branch -u dreamwidth/master master | ||
+ | $ cd ../.. | ||
== Database setup == | == Database setup == | ||
Line 179: | Line 237: | ||
You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root: | You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root: | ||
− | mysql -u root -p | + | $ mysql -u root -p |
− | These MySQL commands | + | These MySQL commands will create your development database: |
− | create database dw; | + | mysql> create database dw; |
− | grant all on dw.* to 'dw'@'localhost' identified by 'somePassword' | + | mysql> grant all on dw.* to 'dw'@'localhost' identified by 'somePassword'; |
− | + | ||
− | You | + | That last line is made up of a few different sections: |
+ | |||
+ | * '''grant all on dw.*''' - this tells MySQL to grant all permissions to the "dw" database which you created in the first line. | ||
+ | * '''to 'dw'@'localhost'''' - this tells MySQL to grant the above permissions to a username called "dw" connecting from "localhost" (which means the same computer as the one MySQL is on). You can change "dw" to another username if you like. '''NB:''' this is 'dw'@'localhost', not 'dw@localhost' - either cut and paste, or type carefully! | ||
+ | * '''identified by 'somePassword' ''' - since this user doesn't exist yet, we can tell MySQL to create it by adding this. This tells MySQL that the "dw" user will use the password "somePassword" to connect. You probably want to change this password! | ||
+ | |||
+ | While you're here, you probably want to go ahead and create a database for TheSchwartz, too, unless you really know that you won't need it: | ||
+ | |||
+ | mysql> create database dw_schwartz; | ||
+ | mysql> grant all on dw_schwartz.* to 'dw'@'localhost'; | ||
+ | |||
+ | (We're leaving out the "identified by" part this time because the user already exists.) | ||
+ | |||
+ | When you're done, exit out of MySQL: | ||
+ | |||
+ | mysql> quit; | ||
== Editing the config files == | == Editing the config files == | ||
− | Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now | + | Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now. |
− | + | Copy over your config files, and tell them to take priority over config files in any of the repos: | |
− | + | ||
− | + | ||
− | Or use your editor of choice, of course. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation. | + | $ cd $LJHOME |
+ | $ mkdir -p ext/local/etc | ||
+ | $ echo "highest" > ext/local/.dir_scope | ||
+ | $ cp etc/config-local.pl.example ext/local/etc/config-local.pl | ||
+ | $ vim ext/local/etc/config-local.pl | ||
+ | |||
+ | Or use your editor of choice, of course, instead of vim. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation. | ||
Note that the <var>$IS_DEV_SERVER</var> flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1. | Note that the <var>$IS_DEV_SERVER</var> flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1. | ||
+ | |||
+ | Below the 'CHANGE THIS' section (unless this is already present somewhere), add: | ||
+ | |||
+ | $STATDOCS = "$HOME/build/static" | ||
Next, you need to change your local variables in the <tt>config-private.pl</tt> file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your <var>$DOMAIN</var> and the <var>%DBINFO</var> structure. | Next, you need to change your local variables in the <tt>config-private.pl</tt> file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your <var>$DOMAIN</var> and the <var>%DBINFO</var> structure. | ||
− | cd $LJHOME | + | $ cd $LJHOME |
− | cp | + | $ cp etc/config-private.pl.example ext/local/etc/config-private.pl |
− | chmod go-rwx etc/config-private.pl | + | $ chmod go-rwx ext/local/etc/config-private.pl |
− | vim etc/config-private.pl | + | $ vim ext/local/etc/config-private.pl |
In this file, uncomment the lines in the <var>%DBINFO</var> blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example: | In this file, uncomment the lines in the <var>%DBINFO</var> blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example: | ||
Line 218: | Line 298: | ||
Note that there are two DBINFO blocks; get them both. | Note that there are two DBINFO blocks; get them both. | ||
− | + | If you created the database for TheSchwartz earlier, you'll also want to configure it: | |
− | vim etc/config.pl | + | # Schwartz DB configuration |
+ | @THESCHWARTZ_DBS = ( | ||
+ | { | ||
+ | dsn => 'dbi:mysql:dw_schwartz;host=localhost', | ||
+ | user => 'dw', | ||
+ | pass => '', # CHANGETHIS | ||
+ | }, | ||
+ | ); | ||
+ | |||
+ | Near the bottom of the file is the line you will need to change to tell your server which domain to use. | ||
+ | |||
+ | package DW::PRIVATE; | ||
+ | |||
+ | $DOMAIN = "ljsite.com"; | ||
+ | |||
+ | <div style="padding:1em; background-color:#fee;">'''A note about $DOMAIN:''' If you are running a local server for development purposes and don't need or want a hostname, you may be tempted to just set $DOMAIN to localhost, but the server won't function correctly without a domain name. One possible workaround for this is to set it to a nonexistent domain (the default ljsite.com is fine) and add a line in your <code>/etc/hosts</code> file that resolves that domain name to your localhost IP (either <tt>127.0.0.1</tt> or <tt>::1</tt>.) Another workaround is to [http://wiki.dreamwidth.net/wiki/index.php/Subdomain_setup#Local_development_via_dnsmasq install and configure dnsmasq].</div> | ||
+ | |||
+ | <div style="padding:1em; background-color:#fee;">'''SSL:''' If you're running a development server and don't want to fuss with SSL, put this in your local config: | ||
+ | |||
+ | $PROTOCOL = "http"; | ||
+ | </div> | ||
+ | |||
+ | The base configuration file also needs to be copied over, although for most purposes you should not need to change very much in this file. If you forget to make a local copy of this file, you may get a null caps error when trying to populate the database tables. | ||
+ | |||
+ | $ cd $LJHOME | ||
+ | $ cp etc/config.pl.example ext/local/etc/config.pl | ||
+ | $ vim ext/local/etc/config.pl | ||
+ | |||
+ | Note that from here on, when any documentation says to edit <tt>etc/config*</tt>, you'll likely want to edit <tt>ext/local/etc/config-*</tt> instead. | ||
== Make sure things are working with checkconfig.pl == | == Make sure things are working with checkconfig.pl == | ||
Line 228: | Line 336: | ||
If you've setup the files as indicated above, run this command: | If you've setup the files as indicated above, run this command: | ||
− | cd $LJHOME | + | $ cd $LJHOME |
− | bin/checkconfig.pl | + | $ bin/checkconfig.pl |
− | If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC | + | If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC, the mailing list, or make a post to <dwcomm>dw_dev</dwcomm>. |
== Populate database with initial data == | == Populate database with initial data == | ||
Line 239: | Line 347: | ||
<b>Note:</b> If make_system.pl says it can't give the system user admin [[privileges]], something has gone wrong with your database population, even if there were no errors. | <b>Note:</b> If make_system.pl says it can't give the system user admin [[privileges]], something has gone wrong with your database population, even if there were no errors. | ||
− | $LJHOME/bin/upgrading/update-db.pl -r --innodb | + | $ $LJHOME/bin/upgrading/update-db.pl -r --innodb |
− | $LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb | + | $ $LJHOME/bin/upgrading/update-db.pl -r --innodb # at least for now we have to run this twice |
− | $LJHOME/bin/upgrading/update-db.pl -p | + | $ $LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb |
+ | $ $LJHOME/bin/upgrading/update-db.pl -p | ||
+ | |||
+ | If you created your database for TheSchwartz earlier, you'll also want to populate it with its default tables: | ||
+ | |||
+ | $ mysql -u dw -p dw_schwartz < /usr/share/doc/libtheschwartz-perl/schema.sql | ||
+ | |||
+ | |||
+ | Create a system account: | ||
+ | |||
+ | $ $LJHOME/bin/upgrading/make_system.pl | ||
(That step will ask you for a password for the [[System account]]. You can change it later by logging in as system, so just give it something for now.) | (That step will ask you for a password for the [[System account]]. You can change it later by logging in as system, so just give it something for now.) | ||
− | $LJHOME/bin/upgrading/ | + | And load in the translation strings: |
− | $LJHOME/bin/ | + | |
+ | $ $LJHOME/bin/upgrading/texttool.pl load | ||
+ | |||
+ | == Compile Foundation stylesheets and static files == | ||
+ | |||
+ | Follow the [[SCSS]] instructions for installing compass and sass: | ||
+ | |||
+ | $ gem install sass --version 3.4.25 | ||
+ | $ gem install compass --version 1.0.3 | ||
+ | |||
+ | Install Java, which will download about 55.9MB and take up 334MB of disk space: | ||
+ | |||
+ | $ sudo apt-get install default-jre | ||
+ | |||
+ | Then compile the stylesheets and static files for dw-free: | ||
+ | |||
+ | $ $LJHOME/bin/build-static.sh | ||
+ | |||
+ | You will see a warning for the compressor not being present, ignore it. | ||
== Configure Apache == | == Configure Apache == | ||
− | This step will need to be done as the root user. | + | This step will need to be done as the root user. Follow the instructions below for your version of Apache, and make sure to do <code> service apache2 restart</code> when you are done for your config changes to take effect. |
− | + | === Apache 2.4 configuration === | |
− | + | ||
− | + | This is the configuration you need for Ubuntu 16.04 and newer. | |
+ | |||
+ | Add the contents of [https://gist.github.com/kareila/9b1f62cfe4eab2a8bb70 this config file] (name it something like <code>dw.conf</code>), change the hostname, port, and directory if necessary and add to <code>/etc/apache2/sites-available</code>. | ||
+ | |||
+ | Create the apache log directory referenced in the configuration file: | ||
+ | |||
+ | $ mkdir /home/dw/apache_logs | ||
− | + | To activate the site run: | |
− | + | ||
− | + | $ sudo a2ensite dw | |
− | + | ||
− | + | (Replacing dw with whatever you named your config file before the .conf bit) | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
Then disable the default site: | Then disable the default site: | ||
− | + | $ sudo a2dissite 000-default | |
+ | |||
+ | And activate the Perl Requests and Perl modules: | ||
− | + | $ sudo a2enmod apreq2 | |
+ | $ sudo a2enmod perl | ||
− | + | Also activate the <code>mpm_prefork</code> module - you may see an error about needing to disable <code>mpm_event</code> first, in which case do the following: | |
− | + | $ sudo a2dismod mpm_event | |
+ | $ sudo a2enmod mpm_prefork | ||
− | + | Finally, restart apache: | |
− | + | ||
− | + | ||
− | + | $ sudo service apache2 restart | |
− | + | Some errors may come up here, if they do, don't be disheartened! In nearly all cases Apache provides enough information to fix these. | |
− | + | === Apache 2.2 configuration === | |
− | + | [https://github.com/dreamwidth/dw-free/issues/923 Apache 2.2 is no longer supported by Dreamwidth code]. | |
== Now what? == | == Now what? == | ||
Line 300: | Line 439: | ||
* [[TheSchwartz Setup]] - TheSchwartz is needed for a handful of features - notably comment posting | * [[TheSchwartz Setup]] - TheSchwartz is needed for a handful of features - notably comment posting | ||
+ | * [[MogileFS setup]] | ||
* [[Subdomain setup]] | * [[Subdomain setup]] | ||
+ | * [[DNS and email setup]] | ||
* [[Statistics setup]] | * [[Statistics setup]] | ||
* [[Generating documentation]] | * [[Generating documentation]] | ||
* [[Allow users to register]] | * [[Allow users to register]] | ||
+ | * Setting up [[Search]] | ||
+ | * [[Dev Testing]] -- setting up the test db and running the tests | ||
There is a list of wanted how-tos at [[Installation Wanted How-To]], in case you need to add something to the list. | There is a list of wanted how-tos at [[Installation Wanted How-To]], in case you need to add something to the list. | ||
Line 309: | Line 452: | ||
=== Starting development === | === Starting development === | ||
− | + | If you haven't already, see [[Dev Getting Started]]. | |
[[Category: Dreamwidth Installation]] | [[Category: Dreamwidth Installation]] |
Latest revision as of 00:49, 30 January 2024
Firstly, you'll need to set up a Linux system -- here are the Suggested Server Requirements. These instructions assume Ubuntu 16.04. Later versions (as of December 18, 2017) may or may not work, see https://github.com/dreamwidth/dw-free/issues/2248. Please let us know if you run into difficulties--we want to update this if people are having problems! We recommend coming into the #dreamwidth-dev channel on IRC.
After you set up, you can keep updated with the instructions on Dev Maintenance.
If you have difficulties or problems or suggestions for installation procedures, please report them to dw_dev. Also look at Production Setup: Webserver.
Contents
Command convention
In this walkthrough, commands to be run as a regular user are prefixed with $. Commands to be run as the super-user are prefixed with #. If you are not the super-user, you can run single commands like this:
$ sudo <command goes here>
Alternately, you can spawn a super-user shell using:
$ sudo -i # <command goes here>
When done with the super-user shell, exit the shell via:
# exit
Configuring the OS and environment
Bringing the system up to date
Before you start installing anything else, you might want to make sure your system is up to date. You should update the packages on your system:
# apt-get update # apt-get upgrade
If you want man pages on your system:
# apt-get install man
Installing necessary packages
This will install necessary packages you'll need to run Dreamwidth (and some optional ones). You'll want to be root when running these commands.
# apt-get install git apache2 apache2-bin apache2-data apache2-utils \ libapache2-mod-perl2 libapache2-mod-apreq2 libapache2-request-perl \ mysql-server wget unzip links vim libclass-autouse-perl libdatetime-perl \ libcache-memcached-perl libhash-multivalue-perl libgd-gd2-perl \ libhtml-template-perl libwww-perl libmime-lite-perl \ liburi-perl libxml-simple-perl libclass-accessor-perl \ libclass-data-inheritable-perl libclass-trigger-perl libcrypt-dh-perl \ libmath-bigint-gmp-perl liburi-fetch-perl libgd-graph-perl \ libgnupg-interface-perl libmail-gnupg-perl perlmagick \ libproc-processtable-perl libsoap-lite-perl librpc-xml-perl \ libstring-crc32-perl libxml-atom-perl libxml-rss-perl \ libimage-size-perl libunicode-maputf8-perl libgtop2-dev build-essential \ libnet-openid-consumer-perl libnet-openid-server-perl libyaml-perl \ libcaptcha-recaptcha-perl libdbd-sqlite3-perl libtest-simple-perl \ libtemplate-perl libterm-readkey-perl libmime-base64-urlsafe-perl \ gcc libtest-most-perl libgearman-client-perl libfile-find-rule-perl \ libbusiness-creditcard-perl liblwpx-paranoidagent-perl libtheschwartz-perl \ libfile-type-perl libjson-perl ruby libdbd-mysql-perl libdanga-socket-perl \ libio-aio-perl libsys-syscall-perl liblog-log4perl-perl libtext-markdown-perl \ libimage-exiftool-perl libnet-oauth-perl libnet-smtps-perl libxmlrpc-lite-perl
This will download about 106MB of files and use around 511MB of disk space.
Check whether these packages have actually installed. Later on in the process, if you find you are having inexplicable problems, try installing these again.
You will also want to install and configure Postfix so your DW can send out email:
# apt-get install postfix
If you choose not to configure on the install, you can do so later with:
# dpkg-reconfigure postfix
Configure Apache 2
Check whether your system is using threaded MPMs or pre-fork. You will need the latter. Note: if you ran the above command to install Apache, then you need to run these commands to switch to the pre-fork model:
# a2dismod mpm_event # a2enmod mpm_prefork # service apache2 restart
Install Perl modules with CPAN
You will also want to install some perl libraries with CPAN. Defaults during CPAN's setup should be okay.
Before installing, check if your CPAN shell has make at the correct location. Run
# which make /usr/bin/make
then compare the results with running:
# cpan ... Would you like to configure as much as possible automatically? [yes] <PRESS ENTER> ... cpan[1]> o conf make make [/usr/bin/make] ... cpan[2]> exit
The value printed should match that from the which make command you previously ran.
To install the perl libraries, first set up CPAN:
# cpan Bundle::CPAN
When prompted, press Enter to exit tests. Then, run the following command to install the rest of the necessary libraries:
# cpan GTop Digest::SHA1 Unicode::CheckUTF8 MogileFS::Client \ TheSchwartz::Worker::SendEmail LWP::UserAgent::Paranoid \ Mozilla::CA List::Util Paws::S3 Net::DNS Text::Fuzzy
(Net::DNS is required to be installed via CPAN because the version included with current versions of Ubuntu is 0.81, which is too old for LWP::UserAgent::Paranoid to work properly.)
Note that the MogileFS::Client is required even if you are not configuring MogileFS.
Set Timezone to UTC
Dreamwidth will complain if you aren't running in UTC. If you're not configured for UTC already, you can reset it (on Ubuntu/Debian) using
# dpkg-reconfigure tzdata
and choose Etc/UTC.
Configuring MySQL/MariaDB
Newer versions of MySQL and/or MariaDB have imposed stricter requirements than the current Dreamwidth SQL definitions can comply with. These restrictions must be relaxed, or the initial database population scripts will fail.
To do this, run the following 4 commands:
$ sudo -i # echo "sql_mode = ''" >> /etc/mysql/mysql.conf.d/mysqld.cnf # service mysql restart # exit
Setting up the DW user account
usermod
, please see Making your DW user a sudo account for alternatives.To set up the user account:
# adduser username
Fill out the user's info and put them in the sudo group:
# usermod -a -G sudo username
So, to use dw
as a username:
# adduser dw # usermod -a -G sudo dw
Now, we will get the $LJHOME variable set. Log into your user:
# su - dw
We will assume that you use the bash shell. If you do not, then this section does not apply. You will have to find the shell specific way of setting environment variables.
For bash, you should look in the home directory of the user account you just created for a file .profile. You will want to add one more line to this file:
export LJHOME=/home/dw/dw
Or wherever it is that you want the root of your source code install to be. Yes, the variable is named LJHOME. We'll live with it for now. ;-)
Test this. Log out of the dw user account and log back in, then type:
echo $LJHOME
You should see /home/dw/dw or whatever you set it to. If you don't, then something has gone awry -- maybe the export line isn't in the right place..?
Get the Dreamwidth code
Configure git
Pick up a name and email address you're willing to share with the world (or at least that part of it that pays attention to GitHub and Dreamwidth). Then run the following 2 commands, replacing NAME and EMAIL with what you chose:
$ git config --global user.name "NAME" $ git config --global user.email "EMAIL"
Optionally, make git use your preferred editor for commit messages (and a few other things). First find the path to your favorite editor (replace EDITOR with the name of your preferred editor, eg nano or vi):
$ which EDITOR
This will print something like: /usr/bin/vi
(if your editor of choice is vi). Then, run the following command, replacing /PATH/TO/EDITOR with whatever was output:
$ git config --global core.editor "/PATH/TO/EDITOR"
Set up GitHub
First, create an account on Github. Then, you will need to fork these two repositories:
Go to each of their pages, and click the "Fork" button. This sets up your own copy of Dreamwidth's code which you will use to make and submit your changes.
Here's a quick overview of how the repositories will work together:
- dw-free is the main repository and goes into $LJHOME
- other repositories will go into $LJHOME/ext
- personal config files go into $LJHOME/ext/local
- All code from dw-free ($LJHOME) and the additional repos under $LJHOME/ext are automatically live when you start your server. There's no need to run any additional syncing steps anymore.
Set up dw-free
Replace the USERNAME sections of the URL with your Github username. This will make your repository an authenticated version that can push changes back to Github, as opposed to only being able to pull them.
First, clone a copy of the repository onto your machine:
$ cd ~/ $ git clone https://USERNAME@github.com/USERNAME/dw-free.git $LJHOME $ cd $LJHOME
Now, let's make it aware of the dreamwidth repository so we can grab updates later:
$ git remote add dreamwidth https://github.com/dreamwidth/dw-free $ git fetch dreamwidth
Finally, let's set up to sync up with the main repository's master branch:
$ git branch -u dreamwidth/master master
Set up dw-nonfree
$ cd $LJHOME/ext $ git clone https://USERNAME@github.com/USERNAME/dw-nonfree.git $ cd dw-nonfree $ git remote add dreamwidth https://github.com/dreamwidth/dw-nonfree $ git fetch dreamwidth $ git branch -u dreamwidth/master master $ cd ../..
Database setup
You should have a local MySQL installation and know the root login to the database. To sign into MySQL as root:
$ mysql -u root -p
These MySQL commands will create your development database:
mysql> create database dw; mysql> grant all on dw.* to 'dw'@'localhost' identified by 'somePassword';
That last line is made up of a few different sections:
- grant all on dw.* - this tells MySQL to grant all permissions to the "dw" database which you created in the first line.
- to 'dw'@'localhost' - this tells MySQL to grant the above permissions to a username called "dw" connecting from "localhost" (which means the same computer as the one MySQL is on). You can change "dw" to another username if you like. NB: this is 'dw'@'localhost', not 'dw@localhost' - either cut and paste, or type carefully!
- identified by 'somePassword' - since this user doesn't exist yet, we can tell MySQL to create it by adding this. This tells MySQL that the "dw" user will use the password "somePassword" to connect. You probably want to change this password!
While you're here, you probably want to go ahead and create a database for TheSchwartz, too, unless you really know that you won't need it:
mysql> create database dw_schwartz; mysql> grant all on dw_schwartz.* to 'dw'@'localhost';
(We're leaving out the "identified by" part this time because the user already exists.)
When you're done, exit out of MySQL:
mysql> quit;
Editing the config files
Next you need to configure the site configuration scripts. This is probably the most tricky part of the whole process, since there are so many things you can tweak. However, you can get by with just tweaking etc/config-local.pl and etc/config-private.pl for now.
Copy over your config files, and tell them to take priority over config files in any of the repos:
$ cd $LJHOME $ mkdir -p ext/local/etc $ echo "highest" > ext/local/.dir_scope $ cp etc/config-local.pl.example ext/local/etc/config-local.pl $ vim ext/local/etc/config-local.pl
Or use your editor of choice, of course, instead of vim. Do a search for the phrase 'CHANGE THIS' which occurs once, before the block of human readable text you should use to describe your development installation.
Note that the $IS_DEV_SERVER flag is set to 1 in the template given. Be sure to set this to 0 for a production site, as there are big security issues involved with leaving the flag set to 1.
Below the 'CHANGE THIS' section (unless this is already present somewhere), add:
$STATDOCS = "$HOME/build/static"
Next, you need to change your local variables in the config-private.pl file. This is where your passwords are configured, as well as many of the variables which define your domain. Do a search for the phrase 'CHANGETHIS'(note the slightly different spacing). You will want to change at least your $DOMAIN and the %DBINFO structure.
$ cd $LJHOME $ cp etc/config-private.pl.example ext/local/etc/config-private.pl $ chmod go-rwx ext/local/etc/config-private.pl $ vim ext/local/etc/config-private.pl
In this file, uncomment the lines in the %DBINFO blocks (if they are commented out), and place your database password in quotes on the "pass => ," line, before the comma. For example:
%DBINFO = ( master => { pass => 'mypassword', }, );
Note that there are two DBINFO blocks; get them both.
If you created the database for TheSchwartz earlier, you'll also want to configure it:
# Schwartz DB configuration @THESCHWARTZ_DBS = ( { dsn => 'dbi:mysql:dw_schwartz;host=localhost', user => 'dw', pass => , # CHANGETHIS }, );
Near the bottom of the file is the line you will need to change to tell your server which domain to use.
package DW::PRIVATE; $DOMAIN = "ljsite.com";
/etc/hosts
file that resolves that domain name to your localhost IP (either 127.0.0.1 or ::1.) Another workaround is to install and configure dnsmasq.$PROTOCOL = "http";
The base configuration file also needs to be copied over, although for most purposes you should not need to change very much in this file. If you forget to make a local copy of this file, you may get a null caps error when trying to populate the database tables.
$ cd $LJHOME $ cp etc/config.pl.example ext/local/etc/config.pl $ vim ext/local/etc/config.pl
Note that from here on, when any documentation says to edit etc/config*, you'll likely want to edit ext/local/etc/config-* instead.
Make sure things are working with checkconfig.pl
Now, you need to see if everything is working.
If you've setup the files as indicated above, run this command:
$ cd $LJHOME $ bin/checkconfig.pl
If you installed everything given at the top of this page, you should find you have no missing modules. Congratulations! If you don't have all the modules, this is where you need some systems specific knowledge for your system. You will need to install whatever modules are missing. If you get well and truly stuck, find someone on IRC, the mailing list, or make a post to dw_dev.
Populate database with initial data
There are a few commands you can now run to install the database. Just run these and watch for errors.
Note: If make_system.pl says it can't give the system user admin privileges, something has gone wrong with your database population, even if there were no errors.
$ $LJHOME/bin/upgrading/update-db.pl -r --innodb $ $LJHOME/bin/upgrading/update-db.pl -r --innodb # at least for now we have to run this twice $ $LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb $ $LJHOME/bin/upgrading/update-db.pl -p
If you created your database for TheSchwartz earlier, you'll also want to populate it with its default tables:
$ mysql -u dw -p dw_schwartz < /usr/share/doc/libtheschwartz-perl/schema.sql
Create a system account:
$ $LJHOME/bin/upgrading/make_system.pl
(That step will ask you for a password for the System account. You can change it later by logging in as system, so just give it something for now.)
And load in the translation strings:
$ $LJHOME/bin/upgrading/texttool.pl load
Compile Foundation stylesheets and static files
Follow the SCSS instructions for installing compass and sass:
$ gem install sass --version 3.4.25 $ gem install compass --version 1.0.3
Install Java, which will download about 55.9MB and take up 334MB of disk space:
$ sudo apt-get install default-jre
Then compile the stylesheets and static files for dw-free:
$ $LJHOME/bin/build-static.sh
You will see a warning for the compressor not being present, ignore it.
Configure Apache
This step will need to be done as the root user. Follow the instructions below for your version of Apache, and make sure to do service apache2 restart
when you are done for your config changes to take effect.
Apache 2.4 configuration
This is the configuration you need for Ubuntu 16.04 and newer.
Add the contents of this config file (name it something like dw.conf
), change the hostname, port, and directory if necessary and add to /etc/apache2/sites-available
.
Create the apache log directory referenced in the configuration file:
$ mkdir /home/dw/apache_logs
To activate the site run:
$ sudo a2ensite dw
(Replacing dw with whatever you named your config file before the .conf bit)
Then disable the default site:
$ sudo a2dissite 000-default
And activate the Perl Requests and Perl modules:
$ sudo a2enmod apreq2 $ sudo a2enmod perl
Also activate the mpm_prefork
module - you may see an error about needing to disable mpm_event
first, in which case do the following:
$ sudo a2dismod mpm_event $ sudo a2enmod mpm_prefork
Finally, restart apache:
$ sudo service apache2 restart
Some errors may come up here, if they do, don't be disheartened! In nearly all cases Apache provides enough information to fix these.
Apache 2.2 configuration
Apache 2.2 is no longer supported by Dreamwidth code.
Now what?
Have a look at your new DW instance
Congratulations! You now have a working (though minimal) Dreamwidth install. If you point your web browser at your server, you should see a bare-looking welcome page.
Further setup
There are lots of other articles on setting up and customizing your DW install in the DW Installation category, including:
- TheSchwartz Setup - TheSchwartz is needed for a handful of features - notably comment posting
- MogileFS setup
- Subdomain setup
- DNS and email setup
- Statistics setup
- Generating documentation
- Allow users to register
- Setting up Search
- Dev Testing -- setting up the test db and running the tests
There is a list of wanted how-tos at Installation Wanted How-To, in case you need to add something to the list.
Starting development
If you haven't already, see Dev Getting Started.