View source for Export Journal Notes

Design document for journal export function

Written by vlion (Paul Nathan)

DRAFT * DRAFT * DRAFT

COMMENTS APPRECIATED.

Contributors: 
* exor674[2]
* Karen Wolf[3]


DRAFT * DRAFT * DRAFT

= Introduction. =

This is a technical document aimed to implement 'journal export to
pdf'[3]. It is written with the understanding that the use cases in
Design Personas[1] are valid representations of common cases of users.

This document will be split into several high-level areas. We will describe development notes, the backend, then the work flow, then the method by which the
backend will implement the work flow, then graphic design considerations.

DRAFT * DRAFT * DRAFT

= Development process =

This is essentially a gigantic reporting application drawing upon DW's
code.

To minimize complexity, the principle of YAGNI[8] will be applied to the backend routines.

We will output to LaTex[5] for these reasons.
* Open Source
* Stable[6]
* Outputs to PDF, PS, DVI, and others[4]
* Established typesetting system
* Well-documented

Several streams of effort will need to be applied:
* Graphic design of the typesetting style.
* Programmatic effort to retrieve the requested information
* Programmatic effort to display the interface
* Programmatic effort to spawn the job


DRAFT * DRAFT * DRAFT

= Backend =

The high-level data-flow architecture of this system will look as follows:

DB -> Filter -> Reformatter -> LaTeX Generator -> LaTeX -> Output
Interface->Filter

The DB->Output line is hereafter referred to as the Pipeline

The high-level control flow architecture of this system will look as
follows:

Interface->Asynch Job start->Pipeline->notify users

The information block will be generically referred to as the database(DB),
whether it is an actual SQL request, a memcached request, or the Perl
module interfacing to the database.

The html interface will pass in the parameters to the asynchronous job
and the pipeline will start.

The Filter module will receive parameters from the Interface via the
asynch starter, request information from the database, do any required
cleaning, (OPTIONAL: form it into a data-structure), and pass the
information on to the Reformatter.

The Reformatter will analyse the information and determine if it needs
reformatting. Example cases might be a super-deep comment thread, or
images that need to be generated/resized. When the information is
prepared, it is passed to the LaTeX Generator.

The LaTeX Generator builds a LaTeX file with the
journal/comment/picture information inside. It is probable that
Dreamwidth will need to implement custom environments to accomodate
threaded comments. Links will be added as a bibliography section.
When the Latex Generator is finished, it writes several files.  out to
the file system and begins the LaTeX process to generate the
final file.  BibTeX may be used for references, in which cases the usual
multipass LaTeX-BibTeX-LaTeX sequence will be used in that process.

When the final file is prepared, it is zipped and moved to a location and the user is
notified via DW-message and email where the location is.

Security considerations:
* Can an unauthorized user peek at the running process of another user?
* Can an unauthorized user peek at the temporary files generated by Latex?
* Can an unauthorized user view the final file?

DRAFT * DRAFT * DRAFT

= Workflow =

The 'export' function will be accessed via the 'create' menu found in
$username.dreamwidth.org.

The Export will bring up an interface that will contain these questions:

* Selection of (journal|community) to export
* Selection of date range to export
* Option to export userpics
* Option to export images found in the journal
* Output format selection (html|pdf|latex| ??? )
* Output paper formats(numbered pages, font, paper size, ???)
* The Export should also have the ability to export by tag, security-level, user-list. 

The user will select the options they choose and press the big export
button. 

The user will be given an estimate of job completion time in the next page. At this point, the Exporter will be finished for some time for this user(5 minutes?). Refreshing will not restart the job. 

When the job completes, the user will receive an email at their registered email account, as well as a notification in the inbox. 

They will have a link to download the exported file (probably in ZIP). The file will be living on Dreamwidth's servers for some length of time(30 days?).

DRAFT * DRAFT * DRAFT

= Workflow<->Backend =
DRAFT * DRAFT * DRAFT

= Graphic Design =

* Links will be added as a bibliography

DRAFT * DRAFT * DRAFT

= Questions to be answered. =

* Deep(Horizontal) comment threads present a presentation issue that must be answered
  Suggestions requested.
* Option to have advanced users upload their own style?

DRAFT * DRAFT * DRAFT

= References =

[1] http://wiki.dwscoalition.org/notes/Design_Personas

[2] http://wiki.dwscoalition.org/notes/User:Exor674/Export_Braindump

[3] http://bugs.dwscoalition.org/attachment.cgi?id=112

[4] http://en.wikibooks.org/wiki/LaTeX/Export_To_Other_Formats

[5] http://www.latex-project.org/

[6] http://en.wikipedia.org/wiki/LaTeX

[7] http://dw-dev.dreamwidth.org/30837.html

[8] http://c2.com/xp/YouArentGonnaNeedIt.html