Difference between revisions of "ESN"

Latest revision as of 19:05, 23 April 2016

ESN stands for Event-Subscription-Notification. This is a complex system, and this page only scratches the surface. You should be prepared to do some digging on your own or ask questions. If you do, please think of the next person having the same question and revise this to include the information you were looking for.

Events

An event is anything happening on Dreamwidth that a user may want to be notified of when it happens, like "someone replied to a comment I wrote" or "there's a new entry posted in (journal)".

Events are defined in LJ::Event (base class) and LJ::Event::* (subclasses for specific event types). They have some data attached, including the journal the event is about and up to 2 additional numeric arguments. See the comments at the beginning of LJ::Event for more on event types, subclasses, and what the journal and additional arguments are used for.

Each event class is responsible for providing the notification email subject, plain text, and HTML text, by implementing the as_email_subject, as_email_string, and as_email_html methods. These methods are called by LJ::NotificationMethod::Email (see below) as needed. (as_email_html won't get called if the user doesn't want HTML email.)

Subscriptions

Subscriptions are how someone tells Dreamwidth which events they're interested in. Which events or event types a user may subscribe to may be restricted by the user's account level, but beyond that, any user can subscribe to any available event if they have access to the relevant data (eg, can view the journal/entry/birthdate/etc...).

Subscriptions are defined in LJ::Subscription (base class) and LJ::Subscription::Pending (used by the user interface for things the user hasn't (yet) subscribed to). Unlike for events, there are no subclasses for different things subscribed to. Data for each subscription includes the journal of interest, the type of event, the notification method (see below), and 2 additional (numerical) arguments, which aren't necessarily equal or even related to the ones for events that may match the subscription.

Notification methods

Notification methods are the possible ways you can get notified when an event that matches one of your subscriptions occurs. They're implemented in LJ::NotificationMethod (base class) and classes for specific notification methods:

• LJ::NotificationMethod::Email notifies users by sending them email. It handles both plain-text and HTML email as appropriate, by calling the event's as_email_subject, as_email_string, and as_email_html methods. (There are hooks available, but Dreamwidth doesn't use them. See LJ::NotificationMethod::Email->notify for more details.)

• LJ::NotificationMethod::Inbox notifies users by placing a message in their site inbox.

• LJ::NotificationMethod::DebugLog isn't offered to users, because it doesn't actually send them anything. Instead, it saves all notification data to a database table, which is useful for developers. See LJ::NotificationMethod::DebugLog->notify for more.

Firing events and delivering notifications

To fire an event, use LJ::Event->fire_job. This queues a LJ::Worker::FiredEvent TheSchwartz job. The bulk of processing fired events takes place in LJ::ESN, in 4 stages:

1. LJ::Worker::FiredEvent finds relevant subscriptions, either by queuing a LJ::Worker::FindSubsByCluster subjob for each cluster (see next stage), or (if there aren't too many subscriptions and no cluster is down) by directly removing duplicate or redundant subscriptions and skipping to the last stage, LJ::Worker::ProcessSub.
2. LJ::Worker::FindSubsByCluster finds relevant subscriptions for a specified cluster, either by queuing a LJ::Worker::FilterSubs job for each group of up to $LJ::ESN::MAX_FILTER_SET (currently 5000) that doesn't split subscriptions by the same user, or by directly removing duplicate or redundant subscriptions and skipping to the last stage, LJ::Worker::ProcessSub.
3. LJ::Worker::FilterSubs checks for race conditions (user halfway through cluster move, with some subscriptions left on old cluster), then removes duplicate or redundant subscriptions and passes the remaining ones to the last stage, by queuing one job for each.
4. LJ::Worker::ProcessSub processes a subscription by creating a notification of the appropriate subclass and having it talk to the user.

Some environment variables and testing flags are available to prevent skipping stages 2 or 3: $ENV{FORCE_P1_P2} || $LJ::_T_ESN_FORCE_P1_P2 (stage 2), $ENV{FORCE_P2_P3} || $LJ::_T_ESN_FORCE_P2_P3 (stage 3). Note that setting the later and not the former may result in stage 3 being skipped, if stage 1 skips stage 2. Also, $ENV{DEBUG} forces output of trace messages. (All of those have to be set to a true value to work - it's not enough for the environment variables to exist or the testing flags to have a defined value.)

There are 2 sets of workers for processing events. The first set includes:

• bin/worker/esn-fired-event,
• bin/worker/esn-cluster-subs,
• bin/worker/esn-filter-subs,
• bin/worker/esn-process-sub.

The second set includes:

• bin/worker/esn-fired-event-mass,
• bin/worker/esn-cluster-subs-mass,
• bin/worker/esn-filter-subs-mass,
• bin/worker/esn-process-sub-mass.

Both set of workers handle the same events. The only difference is that the -mass workers use the "mass" role for accessing the queue tables, thus allowing tuning specific to huge, infrequent events like official newsposts.

If your site doesn't generate a lot of events (which would most likely be the case for a developer setup), you may want to use bin/worker/process-esn instead of the first set and bin/worker/process-esn-mass instead of the second set. Each of those replaces all 4 in the corresponding set.

Previewing notifications

On a development server or Dreamhack (one that has $LJ::IS_DEV_SERVER set to a true value, usually 1), it's possible to see what a notification email or inbox message would look like by using http://www.testuser.hack.dreamwidth.net/admin/eventoutput (replacing www.testuser.hack.dreamwidth.net with your server's domain name).