Summer of Code 2010

Welcome to Dreamwidth!

An Introduction to Dreamwidth Development

If you're totally new to Dreamwidth, you can find out more about us by reading our Guiding Principles, our Diversity Statement, and our About Us guide. We're a blogging platform and social network, forked in 2008 from the code originally developed by LiveJournal.com and considerably extended. We've developed a strong mentoring program that has brought dozens of new programmers into the world of open source, and we'd love to include you as a part of that.

Getting started in Dreamwidth development is documented in the 'Category:Development' category on this Wiki. We offer free hosted development environments, if you don't have to install the server code yourself, so you can get straight to hacking, or if you prefer, you can install from scratch. (Our system documentation being perpetually behind, we really do recommend that you use our hosted development environment service, but if you prefer to install to your own space, we'll give you as much of a hand as you need -- we have many available mentors who are used to doing a step-by-step install.)

As a project, we have a pretty well-developed network of mentoring, coaching, and tutoring, so if you aren't already involved in Dreamwidth development, our first step will be to introduce you to our development community. We're a close-knit team full of lots of enthusiasm! You'll be assigned an official mentor as part of the Google Summer of Code program, but everyone on the project will be available to answer questions, walk you through things, or give you a few tips and hints.

You can browse or download our code at our Mercurial repositories: the dw-free branch is the branch where main development happens, while the other branches are backend utilities or site-specific branding. Technical and development discussion takes place on Dreamwidth itself: dw_dev for general development discussion, dw_dev_training for questions, tips, and useful tricks, and changelog for our project changelog. (There's also dw_news for general site announcements.)

If you'd like an account creation code for Dreamwidth, so you can explore the site, just email Denise (denise@dreamwidth.org) or Mark (mark@dreamwidth.org), the project owners.

General Skill Requirements

Most of the projects outlined here are going to be user-facing improvements to the web site. Unless otherwise stated, you will want to have some knowledge of modern HTML and CSS in order to create the pages that the users of the site will be interacting with. We're in the process of converting our entire site to use the Template Toolkit templating system, but many of our existing pages use a custom templating system with HTML/CSS resting on top of it.

Perl knowledge is also very helpful given that nearly all of the code you write will be in this language. You aren't expected to be a master, and we certainly hope that you learn a few tricks in your time working with us. Similarly, if you've never coded in Perl in your life, you'll probably do just fine if you have experience with Python or a similar language and a desire to learn -- we regularly provide Perl coaching and training for people who want to get started with us.

Other than that, we don't have high requirements. There are additional skills that would be of benefit to you: MySQL, the architecture of a web site in general, Apache, Linux (Ubuntu), scalability, memcached, Gearman, etc etc. None of these are required, however.

Summer of Code Project Ideas

This page serves as a beginning ideas list for the Google Summer of Code project -- it isn't the sole list of projects that might interest a new developer who's looking for a summer-long project, but it's a list of things that we think are fairly standalone and would work well as modular projects. If you don't see something here to strike your fancy, you might also want to look over our unassigned 'effort-major' bugs, or our list of unassigned user suggestions. (Or, if you're already a Dreamwidth user and have an idea for something in particular you want to work on, we'd love to hear about it.)

Most of these ideas are flexible -- so you'll have lots of leeway no matter what you choose. Still, if you're interested in a more self-guided experience, you might want to choose something that's less specified, while if you're the type who prefers to work to a spec, you should pick a project that has a functional spec written. (Or, if you're really interested in a project that doesn't have a functional spec yet, but you work better with one, just let us know. We'll get you one.)

Likewise, if you want to work on a project that has both backend and frontend elements, but you're only interested in the backend bits, come talk to us before you write your proposal, and we'll see what we can work out. We may be able to find you a partner who can work with you on frontend presentation.

Some of these projects are more standalone than others -- we've marked the ones that are more tied in to how the site's culture works, or things we think might require significant user input. Those projects would be a good choice for someone who's interested in getting experience gathering, working with, and incorporating end-user feedback. (We'll do as much of the end-user communication for you as you'd like, or we'll coach and guide you through the process if you'd like to get experience doing it yourself. You can check the dw_biz and dw_suggestions communities for some examples of our user feedback.)

Project:

Description:

Code area to look at:

Difficulty level:

Skills needed:

Spec available:

Mentor help needed:

Clients & Protocols

LiveJournal has always had, and Dreamwidth would like to have as well, desktop or mobile applications that connect to the web site and allow you to write posts or do other actions that normally you need the site to do. One of our most requested features is for the ability to use the site without having to access it via a web browser.

This is a fairly broad category. It would be a valid project to do a mobile client (iPhone, Android, etc), or similarly, one for the desktop (Mac, Windows, Linux). We've made several changes to the LiveJournal client protocol in order to update it, as we've proceeded along our fork, so many of the existing LiveJournal clients work imperfectly for Dreamwidth, or not at all. We'd love to have a set of DW-native clients.

Likewise, we'd love to implement some more of the more widely popular standard blogging APIs: Blogger, MetaWeblog, Atom, etc, as part of our drive towards interoperability. We'd love to hear your ideas on any of these.

Here are some sample clients and protocols projects:

Desktop clients

Project: Desktop client

Description: Program a downloadable client (for the platform of your choice) that will allow users to interact with their journal remotely, without needing a web browser. Ideally, a client program will allow users to post entries to their journal, read their "reading page", and manage the settings of their account (to the extent available through the protocol), but at a bare minimum, the client program should allow the user to post entries to their journal.

Code area to look at: The LiveJournal client protocol contains all of the functions you'd need to implement.

Difficulty level: Medium

Skills needed: Desktop software development, proficiency in application language of your choice

Spec available: No formal spec, but the LiveJournal download page contains examples of client programs written for LiveJournal. Clients written for Dreamwidth should function in roughly the same way.

Mentor help needed: Your mentor will help guide you through the client protocol -- specifically, the changes Dreamwidth has made to the LiveJournal client protocol -- and help you to determine what further changes need to be made to the client protocol to best serve client authors. (There are also several LiveJournal client authors who have expressed interest in adapting their clients to work with Dreamwidth.)

Mobile clients

Project: Mobile client

Description: Program a mobile/handheld client (for the platform of your choice -- we're particularly interested in an iPhone application, but any mobile platform would be awesome) that will allow users to interact with their journal through their mobile device. Ideally, a client program will allow users to post entries to their journal, read their "reading page", and manage the settings of their account (to the extent available through the protocol), but at a bare minimum, the client program should allow the user to post entries to their journal.

Code area to look at: The LiveJournal client protocol contains all of the functions you'd need to implement.

Difficulty level: Medium

Skills needed: Mobile platform software development, proficiency in the mobile programming language of your choice

Spec available: No formal spec, but the LiveJournal download page contains examples of client programs written for LiveJournal. Clients written for Dreamwidth should function in roughly the same way.

Mentor help needed: Your mentor will help guide you through the client protocol -- specifically, the changes Dreamwidth has made to the LiveJournal client protocol -- and help you to determine what further changes need to be made to the client protocol to best serve client authors.

API extensions

Project: API Extensions

Description: In addition to the existing LiveJournal protocol, implement and integrate one (or more) additional open blogging APIs: Blogger, MetaWeblog, Atom, etc. We have rudimentary Blogger and Atom support, but no MetaWeblog support.

Code area to look at: The LiveJournal protocol is where the existing protocol lives -- all transactions with the site go through the protocol, even web-based transactions. Additional APIs would need to be implemented in a way that would tie in to the existing protocol. We have existing bare-bones implementations of the Blogger and Atom APIs, but they aren't very fully-featured and could use a lot of expansion.

Difficulty level: Medium

Skills needed: Backend programming knowledge, Perl, familiarity with chosen API

Spec available: No Dreamwidth-specific spec, but the specs for various APIs are widely available.

Mentor help needed: Your mentor will answer questions about the existing client protocol and help you make decisions about how to translate other APIs' features to Dreamwidth's feature set.

OAuth

Project: Implement OAuth Support

Description: Right now the various APIs that Dreamwidth supports require users to provide their password to third party tools and web sites. This is not up to date with modern best practices, so we'd like to add support for OAuth.

Code area to look at: The crossposting system that we've developed (in which Dreamwidth acts as a native client for other blogging platforms) is in cgi-bin/DW/External.

Difficulty level: Medium

Skills needed: In addition to our general requirements, this project requires you to read and understand OAuth and be able to implement it on Dreamwidth in Perl -- you can of course use modules distributed on CPAN.

Spec available: No spec on the Dreamwidth side, but the OAuth spec is available.

Mentor help needed: Your mentor will explain and walk you through the existing crossposting functionality and be available to answer questions about the details of OAuth implementation.

Wordpress or Movable Type integration

Project: Wordpress or Movable Type import/crosspost

Description: Dreamwidth allows importing archives from, and crossposting entries to, LiveJournal-based platforms already. We would like to add support for Wordpress and Movable Type so people don't need to rely on third-party plugins/extensions/scripts, and so people can post from Dreamwidth if they choose.

Code area to look at: The crossposting system that we've developed (in which Dreamwidth acts as a native client for other blogging platforms) is in cgi-bin/DW/External. Currently, the system only speaks to LJ-based sites, but Wordpress support would go there. Code for the importer is in a few different places, but primarily cgi-bin/DW/Worker.

Difficulty level: Low-Medium, depending on how much detail you want to get into. Both the crossposting system framework and the importing system framework have had a lot of bugs worked out of them over the past year. Adding WP or MT support would plug into this existing framework and expand it.

Skills needed: Our basic requirements, plus familiarity with (or desire to learn) the other platform.

Spec available: Not currently

Mentor help needed: Your mentor will explain and walk you through the existing crossposting and importing functionality and be available to answer questions about the details of WP/MT integration.

Facebook Connect

Project: Implement support for Facebook functionality.

Description: We would like the ability to support some of the Facebook Connect functionality. For example, allow our users to opt-in to sharing content they post on Facebook, or giving people the ability to post something interesting they see on Dreamwidth to Facebook (Share This type functionality), or even live chat functionality.

There are a lot of potential modes of integration. This project will require some independent research and thought into what makes the most sense for Dreamwidth's particular userbase and use cases, so it would be a good choice for someone who's particularly interested in working with end-users to create and implement a spec.

Code area to look at: Depends on the particular implementation you choose, it's likely you might wind up touching everything. The crossposting system might be a good place to start looking, though.

Difficulty level: Medium to High

Skills needed: Knowledge of Facebook and an interest in connecting it with Dreamwidth. Given the breadth of this project it would help to have experience with Perl and MySQL already.

Spec available: No; there has been some some discussion in the dw_suggestions (Crossposting of non-public items to Facebook, Tool to post links to DW entries on Facebook), so there's some initial user feedback, but this would require a lot of careful thought.

Mentor help needed: Your mentor will help walk you through the process of gathering and processing user feedback (or take point, if you're less comfortable), as well as helping you form a spec that takes site culture into consideration.

ping.fm support

Project: Implement ping.fm support for Dreamwidth.

Description: To make it easier for our users to post, we'd like to support the ping.fm API. Whether it's just ping.fm API support or a more tight integration with ping.fm's services is up to you.

Code area to look at: The LiveJournal protocol is where the existing protocol lives -- all transactions with the site go through the protocol, even web-based transactions. Additional APIs would need to be implemented in a way that would tie in to the existing protocol.

Difficulty: Medium-high, depending on how detailed you get with your implementation.

Skills needed: Our basic requirements, plus a good grasp of how ping.fm works (or should work). You'll be making a lot of the integration choices yourself, since we're less familiar with the ping.fm service, so you should be comfortable with working without specs.

Spec available: Not yet, although this is bug 1675 in our Bugzilla.

Mentor help needed: Your mentor will help walk you through the process of gathering and processing user feedback (or take point, if you're less comfortable), as well as helping you form a spec that takes site culture into consideration.

Media Hosting

Project: Implement a system for storing and displaying user submitted media (photos, video, audio, etc).

Description: Dreamwidth has a lot of functionality for manipulating, storing, and displaying text. One of our serious downsides right now is not having the ability for users to store other media on the site and share it with the people that subscribe to them. This is something that we consider a high priority project for us, and we'll provide a lot of support to make this happen.

This project will be considered successful if, at the end, users are able to upload images to the site, post those images in their journals, and other users can view them. Other types of media (audio, video, etc) and other functionality (automatic rotation, resizing, transcoding, etc) will be considered as bonus objectives that we will plan but may not implement.

Code area to look at: We don't currently have any media hosting, so all of this would be written from scratch.

Difficulty level: High, you're implementing an entirely new feature on the site and will have to touch many parts of our codebase.

Skills needed: For this project you should already be familiar with Perl, SQL, and have an interest in media. You will be working with us to implement everything from backend storage (using the MogileFS system) to image manipulation (using distributed tasks in Gearman) to front-end management pages.

Another option might be to take an existing free-software image hosting/gallery system and integrate it with our code. This variation of the project would involve more backend work (MogileFS storage, Gearman distributed tasks, etc) and less front-end management.

Spec available: Partial; we've solicited user feedback on the project, but haven't written a formal spec yet.

Mentor help needed: Your mentor will help guide you through the design stages of the feature, as well as teaching you about how to design for high availability and maximizing database/webserver performance for production code.

Calendar Functionality

Project: Implement calendar/event support.

Description: Our users want the ability to maintain a calendar of events on Dreamwidth. These events can be located in the real world (groups of people who want to schedule a party, f.ex) or they could be entirely online (a fiction challenge). The basic idea of a calendaring system is well understood, but Dreamwidth has some special requirements around security, support for online only events, etc.

We also want the ability for people to integrate their local calendaring application (iCal, Google Calendar, whatever) with their Dreamwidth account. Bonus points if the user can schedule events through their application of choice and have those events show up on the web site.

Difficulty level: High, you will be working with Dreamwidth at it most basic level.

Skills needed: It would help if you are already familiar with calendaring, but that is certainly not required. It would also be useful given the breadth of the project that you area already familiar with Perl, MySQL, and have some web development experience.

Production Monitoring

Project: Implement additional tools for monitoring of the production environment.

Description: All sites require production monitoring: keeping an eye on what is going on inside the servers. Disk space, CPU/memory usage, backups, RAID health status, etc etc. Dreamwidth has need of someone with the ability to code and an interest in the production end of the stack to implement some new tools for monitoring.

We use industry standard environments such as Nagios and Cacti, so most of the code written will be in the form of Nagios/Cacti plugins. However, we have a few of our own tools that we use that do things such as analyzing the packets sent to the MySQL servers to look at the queries being executed, or looking at the HTTP requests being sent to our load balancers live.

Difficulty level: Low to Medium

Skills needed: You must be interested in production environments. (Future sysadmins?) You should also have at least a rudimentary understanding of Linux servers and really strong attention to detail. Difficulty varies by the exact tasks you work on.

WAI-ARIA Improvements

Project: Implement support for WAI-ARIA to improve site accessibility.

Description: Dreamwidth has a strongly stated goal of supporting accessibility whenever it can. We would love if someone with a passion for this area would be interested in helping us to achieve a better level of accessibility for all of our site's functionality. In particular, we will determine what the most popular and hardest to use pages are and then you will work to modernize and make those pages functional for more people.

Difficulty level: Medium

Skills needed: Interest in what makes a site accessible or not and how to get it there. Understanding accessibility is an ongoing process but the skills you learn will help make you a better web developer for the rest of your career.

Account/Usage Statistics

Project: Implement usage tracking and statistics.

Description: Dreamwidth has a few tools for tracking information about account types and site usage, but we're still far behind where we need to be. For example, we would love to know exactly how many accounts are created per day, where they were referred in from, how many accounts are purchased, account turnover, average size of posts, usage of various post features (tags, cuts, images, etc), and many more.

There are a thousand things we can track, and most of the infrastructure for recording the data is already in place. What we lack most is connecting new data points to the statistics system and then a visualization frontend so we can look at the data. We want to be able to select data to view, pick a timeframe, and then view the resulting graph.

An extension to this project would be nightly/weekly/etc reports. I.e., using the administrative interface, configure that we want to see reports with certain data at certain intervals. (This is a bonus objective.)

Difficulty level: Medium to High, depends on how fancy the frontend is.

Skills needed: You should be interested in data, statistics, and graphing. If you already have experience with GD or a similar graphing library, so much the better.