Dev Finding Things
Contents
Finding code corresponding to a URL
To find the code responsible for a particular page (for Template Toolkit pages), see the Routing and Template Cookbook and in particular finding the handler for a URL.
Grepping through code
Basic grep instructions that will recursively search directories for a given term:
# case insensitive rgrep -i [search term] [path] # case sensitive rgrep [search term] [path] # example -- search for "community" in dw-nonfree rgrep -i "community" $LJHOME/ext/dw-nonfree
To output the results of that search into a file in your home directory:
rgrep -i [search term] * > ~/filename.txt
To browse through all files containing a search term (instead of just getting a list of matching lines):
less `rgrep -il [search term] *`
An example of grep
Let's say that I want to change something in the navstrip, but I'm not sure where the navstrip lives. The easiest way to find something that you're looking at in the browser, but can't identify in the code, is to use grep, and to search for a translation string that you see on the page.
The easiest way to see what a translation string is named is to add ?uselang=debug to the end of the page URL. So, in this case, I'd go to a journal that was showing me the navstrip:
http://zorkian.dreamwidth.org/?uselang=debug
Usually this will show you the translation strings in the wild (see http://www.dreamwidth.org/openid/?uselang=debug for instance). It doesn't seem to work on styled journal pages, though, so instead I'll pick a piece of text that looks very recognizeable, in this case the text that says "you have mutual access and mutual subscriptions". I'll do a search for that with rgrep:
rgrep -i "you have mutual access and mutual subscriptions" * | more
The output I get looks like this:
bin/upgrading/en.dat:web.controlstrip.status.mutualtrust_mutualwatch= [[user]] and you have mutual access and mutual subscriptions
That gives me the name of the string: 'web.controlstrip.status.mutualtrust_mutualwatch'. Using that, I can then grep for it, too:
rgrep -i "web.controlstrip.status.mutualtrust_mutualwatch" * | more
which gives me:
bin/upgrading/en.dat:web.controlstrip.status.mutualtrust_mutualwatch=
[[user]] and you have mutual access and mutual subscriptions
cgi-bin/weblib.pl: 'mutualtrust_mutualwatch' => BML::ml('web.
controlstrip.status.mutualtrust_mutualwatch', {'user' => $journal_display}),
One of them is where the string is defined; the other is where it's used. So, now I know that if I'm looking for navstrip-related code, I probably need to look in cgi-bin/weblib.pl.
Finding filenames that match a pattern
find is a powerful *nix tool for recursively searching directories. Its syntax is:
find [starting-path] [what-to-find]
So, for example:
# starting at the current directory, find files exactly named "stats.bml" > find . -name stats.bml ./htdocs/stats.bml # do a case-insensitive search for files named "user.pm" > find . -iname user.pm ./cgi-bin/DW/External/User.pm ./cgi-bin/LJ/User.pm # you can also use wildcards if you quote them -- # eg, find every .bml file that still exists > find . -iname '*.bml' ./htdocs/tools/opml.bml ./htdocs/tools/emailmanage.bml ./htdocs/tools/userpicfactory.bml ./htdocs/tools/search.bml ./htdocs/tools/tellafriend.bml ...many more... ./htdocs/imgupload.bml # finally, you can also use find as a recursive directory lister -- # entering no search terms means it shows everything shade> find test test test/bin test/bin/ipl test/examples test/examples/moveuclusterd_tests.pl test/examples/example.test.pl test/MANIFEST ...etc...
Directory Structure
An outline of a live site's directory structure can be found in Directory Structure.
Managing the database schema
The database schema is managed by editing update-db-general.pl. Update-db-general.pl is organized roughly like this so:
- mark which tables are clustered (versus global)
- table creation, post-create table population, table deletion
- table alteration
Functions you may need to use:
- creating a new table
register_tablecreate( "tablename", <<'EOC'); CREATE TABLE tablename ( .... ) EOC
- drop a table
- remove the relevant register_tablecreate code
- drop the table if it exists already (for older installations)
register_tabledrop( "tablename" );
- adding a column to an existing table
- add the column to the register_tablecreate code, for new installations
- add this to the bottom of register_alter, for existing installations
# check that the column does not yet exist
unless ( column_type( "tablename", "columname" ) ) {
do_alter("tablename", "ALTER TABLE tablename ADD ..." );
}
- modifying a column in an existing table / modifying data in an existing table
Similar to adding a column, but instead of checking whether a table exists, you may want to check against index_name, table_relevant, column_type, e.g.,
if ( index_name( ... ) ) {}
- initialize a table with data from another table on table creation
post_create( "tablename", /*sql statements/code. Look at other examples in the file */ );
To update your database with your changes, run
$LJHOME/bin/upgrading/update-db.pl -r --innodb
Properties
Properties are listed in bin/upgrading/proplists.dat, broken down into three sections and listed alphabetically within those sections:
- userproplist (user properties)
- talkproplist (comment properties)
- logproplist (entry properties)
When removing, remove the property from the file, and also do a grep over the codebase to remove any code which uses this prop.
Translation strings
These are where all the static text gets defined. There are some oddities to working with them: see English-stripping.
Translation strings are defined in a few different files:
- bin/upgrading/en.dat - central location for strings for the backend code, e.g., things in cgi-bin, hooks, widgets
- htdocs/page.bml.text - scattered throughout htdocs; partner to htdocs/page.bml
There is a file, deadphrases.dat, which was meant for listing phrases that are no longer in use. However, everyone just forgets to use it anyway ;-) Ignore this file.
After making your changes, run
$LJHOME/bin/upgrading/texttool.pl load
to actually load the new/modified strings into your live environment.
Hooks
Hook implementations can be found in cgi-bin/LJ/Hooks.pm and cgi-bin/DW/Hooks.pm.
Some hook documentation can be found in doc/raw/hooks.txt, but it is very outdated and incomplete, and probably should be ignored right now.
Memcache
A list of memcache keys can be found in doc/raw/memcache-keys.txt. You should update this if you add new memcache keys or change existing behavior.
Styles
- styles/s2layers.dat, ext/dw-nonfree/styles/s2layers.dat
Defines system layers + layer types + layer parents.
- styles/$stylename/layout.s2, ext/dw-nonfree/styles/$stylename/layout.s2, styles/$stylename/themes.s2, ext/dw-nonfree/styles/$stylename/themes.s2
Contains actual layout and theme layers.
- cgi-bin/LJ/S2Themes.pm, ext/dw-nonfree/cgi-bin/LJ/S2Themes-local.pm
Handles some logic surrounding themes, contains list of default themes for each style, defines available subcategories for properties and sorts default ones.
- cgi-bin/LJ/S2Theme/$stylename, ext/dw-nonfree/cgi-bin/LJ/S2Theme/$stylename
Contains logic for /customize/ and /customize/options.bml pages, defines supported page set-ups and arranges custom properties into subcategories.
