* import r26.1 livejournal

This commit is contained in:
mark 2008-04-05 01:38:13 -07:00
parent 340c2b117d
commit e7bfae8e48
3678 changed files with 470063 additions and 2 deletions

379
LICENSE Normal file
View File

@ -0,0 +1,379 @@
This repository was originally based off of code retrieved from
Six Apart's public "livejournal" svn repository. The original code
carries this copyright:
The code in LiveJournal.org's "livejournal" cvs repository are
Copyright (C) 1994-2005 LiveJournal.com, Inc., a subsidiary of Six
Apart, Ltd.
The code modifications made to the original code and that are
contained in this repository carry the following copyright:
Original code copyright (C) 1994-2005 LiveJournal.com, Inc., a
subsidiary of Six Apart, Ltd. Modifications copyright (C)
2008 by the Open Source Social Network Foundation, LLC.
Additionally, some files were originally committed to this repository
that were authored by the OSSNF. These files carry this copyright:
Copyright (C) 2008 by the Open Source Social Network Foundation, LLC.
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301, USA.
The text of the GNU General Public License follows:
-------
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.
51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Library General
Public License instead of this License.

15
README
View File

@ -1,3 +1,14 @@
This is a temporary file, just holding open the directory.
Please see the LICENSE file for the license of this code. Note that all code
committed to this repository MUST be licensed under the GPL and have proper
copyright notices tagged at the top of the file.
Edit, edit, and more!
For more information on how to use this software, please harass someone to
actually write out documentation here. :-)
If you just want to get started with a fresh installation, you can get things
rolling along:
perl bin/bootstra.pl
This will check out the various repositories that we use and put them in the
appropriate place.

29
bin/bootstrap.pl Executable file
View File

@ -0,0 +1,29 @@
#!/usr/bin/perl
use strict;
# check for svn in a known place
die "Expected svn in /usr/bin ... not found!\n"
unless -e '/usr/bin/svn';
# get the right directory
my $LJHOME = $ENV{'LJHOME'};
die "Must set the \$LJHOME environment variable before running this.\n"
unless -d $LJHOME;
chdir( $LJHOME )
or die "Couldn't chdir to \$LJHOME directory.\n";
# more than likely we don't have vcv, so let's get it
die "Did you already bootstrap? cvs/vcv exists.\n"
if -d "$LJHOME/cvs/vcv";
# so now get it
system( '/usr/bin/svn co http://code.sixapart.com/svn/vcv/trunk/ cvs/vcv' );
die "Unable to checkout vcv from Six Apart SVN repository.\n"
unless -d "$LJHOME/cvs/vcv" && -e "$LJHOME/cvs/vcv/bin/vcv";
# now get vcv to do the rest for us
system( 'cvs/vcv/bin/vcv --conf=cvs/multicvs.conf --checkout' );
# finished :-)
print "Done! We hope. :-)\n";

15
bin/build-usersearch Executable file
View File

@ -0,0 +1,15 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
use Carp;
BEGIN {
require 'ljlib.pl';
}
use LJ::UserSearch::MetaUpdater;
my $filename = shift || "$ENV{LJHOME}/var/usersearch.data";
$SIG{__DIE__} = sub { Carp::croak( @_ ) };
LJ::UserSearch::MetaUpdater::update_file($filename);

279
bin/checkconfig.pl Executable file
View File

@ -0,0 +1,279 @@
#!/usr/bin/perl
#
use strict;
use Getopt::Long;
my $debs_only = 0;
my ($only_check, $no_check, $opt_nolocal);
my %dochecks; # these are the ones we'll actually do
my @checks = ( # put these in the order they should be checked in
"modules",
"env",
"database",
"ljconfig",
);
foreach my $check (@checks) { $dochecks{$check} = 1; }
sub usage {
die "Usage: checkconfig.pl
checkconfig.pl --needed-debs
checkconfig.pl --only=<check> | --no=<check>
Checks are:
" . join(', ', @checks);
}
usage() unless GetOptions(
'needed-debs' => \$debs_only,
'only=s' => \$only_check,
'no=s' => \$no_check,
'nolocal' => \$opt_nolocal,
);
if ($debs_only) {
$dochecks{ljconfig} = 0;
$dochecks{database} = 0;
}
usage() if $only_check && $no_check;
%dochecks = ( $only_check => 1)
if $only_check;
# dependencies
if ($dochecks{ljconfig}) {
$dochecks{env} = 1;
}
$dochecks{$no_check} = 0
if $no_check;
my @errors;
my $err = sub {
return unless @_;
die "\nProblem:\n" . join('', map { " * $_\n" } @_);
};
my %modules = (
"DateTime" => { 'deb' => 'libdatetime-perl' },
"DBI" => { 'deb' => 'libdbi-perl', },
"DBD::mysql" => { 'deb' => 'libdbd-mysql-perl', },
"Class::Autouse" => { 'deb' => 'libclass-autouse-perl', },
"Digest::MD5" => { 'deb' => 'libmd5-perl', },
"Digest::SHA1" => { 'deb' => 'libdigest-sha1-perl', },
"HTML::Template" => { 'deb' => 'libhtml-template-perl' },
"Image::Size" => { 'deb' => 'libimage-size-perl', },
"MIME::Lite" => { 'deb' => 'libmime-lite-perl', },
"MIME::Words" => { 'deb' => 'libmime-perl', },
"Compress::Zlib" => { 'deb' => 'libcompress-zlib-perl', },
"Net::DNS" => {
'deb' => 'libnet-dns-perl',
},
"URI::URL" => { 'deb' => 'liburi-perl' },
"HTML::Tagset" => { 'deb' => 'libhtml-tagset-perl' },
"HTML::Parser" => { 'deb' => 'libhtml-parser-perl', },
"LWP::Simple" => { 'deb' => 'libwww-perl', },
"LWP::UserAgent" => { 'deb' => 'libwww-perl', },
"GD" => { 'deb' => 'libgd-gd2-perl' },
"GD::Graph" => {
'deb' => 'libgd-graph-perl',
'opt' => 'Required for making graphs for the statistics page.',
},
"Mail::Address" => { 'deb' => 'libmailtools-perl', },
"Proc::ProcessTable" => {
'deb' => 'libproc-process-perl',
'opt' => "Better reliability for starting daemons necessary for high-traffic installations.",
},
"RPC::XML" => {
'deb' => 'librpc-xml-perl',
'opt' => 'Required for outgoing XML-RPC support',
},
"SOAP::Lite" => {
'deb' => 'libsoap-lite-perl',
'opt' => 'Required for XML-RPC support.',
},
"Unicode::MapUTF8" => { 'deb' => 'libunicode-maputf8-perl', },
"XML::RSS" => {
'deb' => 'libxml-rss-perl',
'opt' => 'Required for retrieving RSS off of other sites (syndication).',
},
"XML::Simple" => {
'deb' => 'libxml-simple-perl',
'ver' => 2.12,
},
"String::CRC32" => {
'deb' => 'libstring-crc32-perl',
'opt' => 'Required for palette-altering of PNG files. Only necessary if you plan to make your own S2 styles that use PNGs, not GIFs.',
},
"IO::WrapTie" => { 'deb' => 'libio-stringy-perl' },
"XML::Atom" => {
'deb' => 'libxml-atom-perl',
'opt' => 'Required for Atom API support.',
},
"Math::BigInt::GMP" => {
'deb' => 'libmath-bigint-gmp-perl',
'opt' => 'Aides Crypt::DH so it is not crazy slow.',
},
"URI::Fetch" => {
'deb' => 'liburi-fetch-perl',
'opt' => 'Required for OpenID support.',
},
"Crypt::DH" => {
'deb' => 'libcrypt-dh-perl',
'opt' => 'Required for OpenID support.',
},
"Unicode::CheckUTF8" => {},
"Digest::HMAC_SHA1" => {
'deb' => 'libdigest-hmac-perl',
},
"Image::Magick" => {
'deb' => 'perlmagick',
'opt' => "Required for the userpic factory.",
},
"Class::Accessor" => {
'deb' => 'libclass-accessor-perl',
'opt' => "Required for TheSchwartz job submission.",
},
"Class::Trigger" => {
'deb' => 'libclass-trigger-perl',
'opt' => "Required for TheSchwartz job submission.",
},
"Class::Data::Inheritable" => {
'deb' => 'libclass-data-inheritable-perl',
'opt' => "Required for TheSchwartz job submission.",
},
"GnuPG::Interface" => {
'deb' => 'libgnupg-interface-perl',
'opt' => "Required for email posting.",
},
"Mail::GnuPG" => {
'deb' => 'libmail-gnupg-perl',
'opt' => "Required for email posting.",
},
"Text::vCard" => {
'deb' => 'libtext-vcard-perl',
'opt' => "Used to generate user vCards.",
},
"IP::Country::Fast" => {
'opt' => "Required for country lookup with IP address.",
},
"GTop" => {
'opt' => "Required for Apache per-request database logging.",
},
);
sub check_modules {
print "[Checking for Perl Modules....]\n"
unless $debs_only;
my @debs;
foreach my $mod (sort keys %modules) {
my $rv = eval "use $mod;";
if ($@) {
my $dt = $modules{$mod};
unless ($debs_only) {
if ($dt->{'opt'}) {
print STDERR "Missing optional module $mod: $dt->{'opt'}\n";
} else {
push @errors, "Missing perl module: $mod";
}
}
push @debs, $dt->{'deb'} if $dt->{'deb'};
next;
}
my $ver_want = $modules{$mod}{ver};
my $ver_got = $mod->VERSION;
if ($ver_want && $ver_got && $ver_got < $ver_want) {
push @errors, "Out of date module: $mod (need $ver_want, $ver_got installed)";
}
}
if (@debs && -e '/etc/debian_version') {
if ($debs_only) {
print join(' ', @debs);
} else {
print STDERR "\n# apt-get install ", join(' ', @debs), "\n\n";
}
}
$err->(@errors);
}
sub check_env {
print "[Checking LJ Environment...]\n"
unless $debs_only;
$err->("\$LJHOME environment variable not set.")
unless $ENV{'LJHOME'};
$err->("\$LJHOME directory doesn't exist ($ENV{'LJHOME'})")
unless -d $ENV{'LJHOME'};
# before ljconfig.pl is called, we want to call the site-local checkconfig,
# otherwise ljconfig.pl might load ljconfig-local.pl, which maybe load
# new modules to implement site-specific hooks.
my $local_config = "$ENV{'LJHOME'}/bin/checkconfig-local.pl";
$local_config .= ' --needed-debs' if $debs_only;
if (!$opt_nolocal && -e $local_config) {
my $good = eval { require $local_config; };
exit 1 unless $good;
}
$err->("No ljconfig.pl file found at $ENV{'LJHOME'}/etc/ljconfig.pl")
unless -e "$ENV{'LJHOME'}/etc/ljconfig.pl";
eval { require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl"; };
$err->("Failed to load ljlib.pl: $@") if $@;
}
sub check_database {
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $dbh = LJ::get_dbh("master");
unless ($dbh) {
$err->("Couldn't get master database handle.");
}
foreach my $c (@LJ::CLUSTERS) {
my $dbc = LJ::get_cluster_master($c);
next if $dbc;
$err->("Couldn't get db handle for cluster \#$c");
}
if (%LJ::MOGILEFS_CONFIG && $LJ::MOGILEFS_CONFIG{hosts}) {
print "[Checking MogileFS client.]\n";
my $mog = LJ::mogclient();
die "Couldn't create MogileFS client." unless $mog;
}
}
sub check_ljconfig {
# if we're a developer running this, make sure we didn't add any
# new configuration directives without first documenting them:
$ENV{READ_LJ_SOURCE} = 1 if $LJ::IS_DEV_SERVER;
# check for beta features cap
unless (LJ::class_bit(LJ::BetaFeatures->cap_name)) {
print STDERR "Warning: BetaFeatures module cannot be used unless '" . LJ::BetaFeatures->cap_name . "' cap is configured.";
}
require LJ::ConfCheck;
my @errs = LJ::ConfCheck::config_errors();
local $" = ",\n\t";
$err->("Config errors: @errs") if @errs;
}
foreach my $check (@checks) {
next unless $dochecks{$check};
my $cn = "check_".$check;
no strict 'refs';
&$cn;
}
unless ($debs_only) {
print "All good.\n";
print "NOTE: checkconfig.pl doesn't check everything yet\n";
}

38
bin/cvsreport.pl Executable file
View File

@ -0,0 +1,38 @@
#!/usr/bin/perl
#
# This is now just a wrapper around the non-LJ-specific multicvs.pl
#
use strict;
unless (-d $ENV{'LJHOME'}) {
die "\$LJHOME not set.\n";
}
if (defined $ENV{'FBHOME'} && $ENV{'PWD'} =~ /^$ENV{'FBHOME'}/i) {
die "You are running this LJ script while working in FBHOME" unless $ENV{FBHOME} eq $ENV{LJHOME};
}
# be paranoid in production, force --these
my @paranoia;
eval { require "$ENV{LJHOME}/etc/ljconfig.pl"; };
if ($LJ::IS_LJCOM_PRODUCTION || $LJ::IS_LJCOM_BETA) {
@paranoia = ('--these');
}
# strip off paths beginning with LJHOME
# (useful if you tab-complete filenames)
$_ =~ s!\Q$ENV{'LJHOME'}\E/?!! foreach (@ARGV);
my @extra;
my $vcv_exe = "multicvs.pl";
if (-e "$ENV{LJHOME}/bin/vcv") {
$vcv_exe = "vcv";
@extra = ("--headserver=code.sixapart.com:10000");
}
exec("$ENV{'LJHOME'}/bin/$vcv_exe",
"--conf=$ENV{'LJHOME'}/cvs/multicvs.conf",
@extra,
@paranoia,
@ARGV);

369
bin/dbcheck.pl Executable file
View File

@ -0,0 +1,369 @@
#!/usr/bin/perl
#
use strict;
use DBI;
use Getopt::Long;
my $help = 0;
my $opt_fh = 0;
my $opt_fix = 0;
my $opt_start = 0;
my $opt_stop = 0;
my $opt_err = 0;
my $opt_all = 0;
my $opt_tablestatus;
my $opt_checkreport = 0;
my $opt_verbose;
my $opt_rates;
my @opt_run;
exit 1 unless GetOptions('help' => \$help,
'flushhosts' => \$opt_fh,
'start' => \$opt_start,
'stop' => \$opt_stop,
'checkreport' => \$opt_checkreport,
'rates' => \$opt_rates,
'fix' => \$opt_fix,
'run=s' => \@opt_run,
'onlyerrors' => \$opt_err,
'all' => \$opt_all,
'tablestatus' => \$opt_tablestatus,
'verbose' => \$opt_verbose,
);
unless (-d $ENV{'LJHOME'}) {
die "\$LJHOME not set.\n";
}
if ($help) {
die ("Usage: dbcheck.pl [opts] [[cmd] args...]\n" .
" --all Check all hosts, even those with no weight assigned.\n" .
" --help Get this help\n" .
" --flushhosts Send 'FLUSH HOSTS' to each db as root.\n".
" --fix Fix (once) common problems.\n".
" --checkreport Show tables that haven't been checked in a while.\n".
" --stop Stop replication.\n".
" --start Start replication.\n".
" --run <sql> Run arbitrary SQL.\n".
" --onlyerrors Will be silent unless there are errors.\n".
" --tablestatus Show warnings about full/sparse tables.\n".
"\n".
"Commands\n".
" (none) Shows replication status.\n".
" queries <host> Shows active queries on host, sorted by running time.\n"
);
}
require "$ENV{'LJHOME'}/cgi-bin/ljdb.pl";
debug("Connecting to master...");
my $dbh = LJ::DB::dbh_by_role("master");
die "Can't get master db handle\n" unless $dbh;
my %dbinfo; # dbid -> hashref
my %name2id; # name -> dbid
my $sth;
my $masterid = 0;
my %subclust; # id -> name of parent (pork-85 -> "pork")
$sth = $dbh->prepare("SELECT dbid, name, masterid, rootfdsn FROM dbinfo");
$sth->execute;
while ($_ = $sth->fetchrow_hashref) {
if ($_->{name} =~ /(.+)\-\d\d$/) {
$subclust{$_->{dbid}} = $1;
next;
}
next unless $_->{'dbid'};
$dbinfo{$_->{'dbid'}} = $_;
$name2id{$_->{'name'}} = $_->{'dbid'};
}
my %role; # rolename -> dbid -> [ norm, curr ]
my %rolebyid; # dbid -> rolename -> [ norm, curr ]
$sth = $dbh->prepare("SELECT dbid, role, norm, curr FROM dbweights");
$sth->execute;
while ($_ = $sth->fetchrow_hashref) {
my $id = $_->{dbid};
if ($subclust{$id}) {
$id = $name2id{$subclust{$id}};
}
next unless defined $dbinfo{$id};
$dbinfo{$id}->{'totalweight'} += $_->{'curr'};
$role{$_->{role}}->{$id} = [ $_->{norm}, $_->{curr} ];
$rolebyid{$id}->{$_->{role}} = [ $_->{norm}, $_->{curr} ];
}
check_report() if $opt_checkreport;
rate_report() if $opt_rates;
my %root_handle; # name -> $db
my $root_handle = sub {
my $name = shift;
return $root_handle{$name} if exists $root_handle{$name};
debug("Connecting to '$name' ...");
$LJ::DB_TIMEOUT = 1;
my $db = LJ::DB::root_dbh_by_name($name);
debug(" ($name: failed to connect)") unless $db;
return $root_handle{$name} = $db;
};
my @errors;
my %master_status; # dbid -> [ $file, $pos ]
my $check_master_status = sub {
my $dbid = shift;
my $d = $dbinfo{$dbid};
die "Bogus DB: $dbid" unless $d;
my $db = $root_handle->($d->{name});
next unless $db;
my ($masterfile, $masterpos) = $db->selectrow_array("SHOW MASTER STATUS");
$master_status{$dbid} = [ $masterfile, $masterpos ];
};
my $check = sub {
my $dbid = shift;
my $d = $dbinfo{$dbid};
die "Bogus DB: $dbid" unless $d;
# calculate roles to show
my $roles;
{
my %drole; # display role -> 1
foreach my $role (grep { $role{$_}{$dbid}[1] } keys %{$rolebyid{$dbid}}) {
my $drole = $role;
$drole =~ s/cluster(\d+)\d/cluster${1}0/;
$drole{$drole} = 1;
}
$roles = join(", ", sort keys %drole);
}
my $db = $root_handle->($d->{name});
unless ($db) {
printf("%4d %-18s %4s %16s %14s ($roles)\n",
$dbid,
$d->{name},
$d->{masterid} ? $d->{masterid} : "",
) unless $opt_err;
push @errors, "Can't connect to $d->{'name'}";
return 0;
}
my $tzone;
(undef, $tzone) = $db->selectrow_array("show variables like 'timezone'");
$tzone ||= "???";
$sth = $db->prepare("SHOW PROCESSLIST");
$sth->execute;
my $pcount_total = 0;
my $pcount_busy = 0;
while (my $r = $sth->fetchrow_hashref) {
next if $r->{'State'} =~ /waiting for/i;
next if $r->{'State'} eq "Reading master update";
next if $r->{'State'} =~ /^(Has (sent|read) all)|(Sending binlog)/;
$pcount_total++;
$pcount_busy++ if $r->{'State'};
}
my @master_logs;
my $log_count = 0;
if ($master_status{$dbid} && $master_status{$dbid}->[1]) {
$sth = $db->prepare("SHOW MASTER LOGS");
$sth->execute;
while (my ($log) = $sth->fetchrow_array) {
push @master_logs, $log;
$log_count++;
}
}
my $ss = $db->selectrow_hashref("show slave status");
if ($ss) {
foreach my $k (sort keys %$ss) {
$ss->{lc $k} = $ss->{$k};
}
}
my $diff;
if ($ss) {
if ($ss->{'slave_io_running'} eq "Yes" && $ss->{'slave_sql_running'} eq "Yes") {
if ($ss->{'master_log_file'} eq $ss->{'relay_master_log_file'}) {
$diff = $ss->{'read_master_log_pos'} - $ss->{'exec_master_log_pos'};
} else {
$diff = "XXXXXXX";
push @errors, "Wrong log file: $d->{name}";
}
} else {
$diff = "XXXXXXX";
$ss->{last_error} =~ s/[^\n\r\t\x20-\x7e]//g;
push @errors, "Slave not running: $d->{name}: $ss->{last_error}";
}
my $ms = $master_status{$d->{masterid}} || [];
#print " master: [@$ms], slave at: [$ss->{master_log_file}, $ss->{read_master_log_pos}]\n";
if ($ss->{master_log_file} ne $ms->[0] || $ss->{read_master_log_pos} < $ms->[1] - 20_000) {
push @errors, "$d->{name}: Relay log behind: master=[@$ms], $d->{name}=[$ss->{master_log_file}, $ss->{read_master_log_pos}]";
}
} else {
$diff = "-"; # not applicable
}
my $extra_version = "";
my $ver = $db->selectrow_array('SELECT VERSION()');
if ($ver) {
$ver =~ s/^(\d\.\d+\.\d+).*$/$1/;
$extra_version = $ver;
} else {
$extra_version = "unknown";
}
#print "$dbid of $d->{masterid}: $d->{name} ($roles)\n";
printf("%4d %-18s %4s repl:%7s %4s conn:%4d/%4d $tzone \%s ($roles)\n",
$dbid,
$d->{name},
$d->{masterid} ? $d->{masterid} : "",
$diff,
$log_count ? sprintf("<%2s>", $log_count) : "",
$pcount_busy, $pcount_total,
$extra_version) unless $opt_err;
};
$check_master_status->($_) foreach (sorted_dbids());
$check->($_) foreach (sorted_dbids());
if (@errors) {
if ($opt_err) {
my %ignore;
open(EX, "$ENV{'HOME'}/.dbcheck.ignore");
while (<EX>) {
s/\s+$//;
$ignore{$_} = 1;
}
close EX;
@errors = grep { ! $ignore{$_} } @errors;
}
print STDERR "\nERRORS:\n" if @errors;
foreach (@errors) {
print STDERR " * $_\n";
}
}
my $sorted_cache;
sub sorted_dbids {
return @$sorted_cache if $sorted_cache;
$sorted_cache = [ _sorted_dbids() ];
return @$sorted_cache;
}
sub _sorted_dbids {
my @ids;
my %added; # dbid -> 1
my $add = sub {
my $dbid = shift;
$added{$dbid} = 1;
push @ids, $dbid;
};
my $masterid = (keys %{$role{'master'}})[0];
$add->($masterid);
# then slaves
foreach my $id (sort { $dbinfo{$a}->{name} cmp $dbinfo{$b}->{name} }
grep { ! $added{$_} && $rolebyid{$_}->{slave} } keys %dbinfo) {
$add->($id);
}
# now, figure out which remaining are associated with cluster roles (user clusters)
my %minclust; # dbid -> minimum cluster number associated
my %is_master; # dbid -> bool (is cluster master)
foreach my $dbid (grep { ! $added{$_} } keys %dbinfo) {
foreach my $role (keys %{ $rolebyid{$dbid} || {} }) {
next unless $role =~ /^cluster(\d+)(.*)/;
$minclust{$dbid} = $1 if ! $minclust{$dbid} || $1 < $minclust{$dbid};
$is_master{$dbid} ||= $2 eq "" || $2 eq "a" || $2 eq "b";
}
}
# then misc
foreach my $id (sort { $dbinfo{$a}->{name} cmp $dbinfo{$b}->{name} }
grep { ! $added{$_} && ! $minclust{$_} } keys %dbinfo) {
$add->($id);
}
# then clusters, in order
foreach my $id (sort { $minclust{$a} <=> $minclust{$b} ||
$is_master{$b} <=> $is_master{$a} }
grep { ! $added{$_} && $minclust{$_} } keys %dbinfo) {
$add->($id);
}
return @ids;
}
sub check_report {
foreach my $dbid (sort { $dbinfo{$a}->{name} cmp $dbinfo{$b}->{name} }
keys %dbinfo) {
my $d = $dbinfo{$dbid};
die "Bogus DB: $dbid" unless $d;
my $db = $root_handle->($d->{name});
unless ($db) {
print "$d->{name}\t?\t?\t?\n";
next;
}
my $dbs = $db->selectcol_arrayref("SHOW DATABASES");
foreach my $dbname (@$dbs) {
$db->do("USE $dbname");
my $ts = $db->selectall_hashref("SHOW TABLE STATUS", "Name");
foreach my $tn (sort keys %$ts) {
my $v = $ts->{$tn};
my $ut = $v->{Check_time} || "0000-00-00 00:00:00";
$ut =~ s/ /,/;
print "$d->{name}\t$dbname\t$tn\t$ut\t$v->{Type}-$v->{Row_format}\t$v->{Rows}\n";
}
}
}
exit 0;
}
use Time::HiRes ();
sub rate_report {
my %prev; # dbid -> [ time, questions ]
while (1) {
print "\n";
my $sum = 0;
foreach my $dbid (sorted_dbids()) {
my $d = $dbinfo{$dbid};
die "Bogus DB: $dbid" unless $d;
my $db = $root_handle->($d->{name});
next unless $db;
my (undef, $qs) = $db->selectrow_array("SHOW STATUS LIKE 'Questions'");
my $now = Time::HiRes::time();
my $cur = [ $now, $qs ];
if (my $old = $prev{$dbid}) {
my $dt = $now - $old->[0];
my $qnew = $qs - $old->[1];
my $rate = ($qnew / $dt);
$sum += $rate;
printf "%20s: %7.01f q/s\n", $d->{name}, $rate;
}
$prev{$dbid} ||= $cur;
}
printf "%20s: %7.01f q/s\n", "SUM", $sum;
sleep 1;
}
}
sub debug {
return unless $opt_verbose;
warn $_[0], "\n";
}

157
bin/deleteusers.pl Executable file
View File

@ -0,0 +1,157 @@
#!/usr/bin/perl
#
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $dbh = LJ::get_dbh("master");
$dbh->{'RaiseError'} = 1;
$dbh->{'PrintError'} = 1;
my $sth;
$sth = $dbh->prepare("SELECT userid FROM user WHERE statusvis='D' AND statusvisdate < DATE_SUB(NOW(), INTERVAL 60 DAY) LIMIT 1000");
$sth->execute;
my @delusers;
while (my $duid = $sth->fetchrow_array) {
push @delusers, $duid;
}
print "Users to delete: ", scalar(@delusers), "\n";
# Get hashref mapping {userid => $u} for all users to be deleted
my $user = LJ::load_userids(@delusers);
LJ::load_props($dbh, "talk");
my $p_delposter = LJ::get_prop("talk", "deleted_poster");
die "No 'deleted_poster' talkprop?" unless $p_delposter;
my $ids;
my $lastbreak = time();
my $pause = sub {
if (time() - $lastbreak > 3) { print "pause.\n"; sleep(1); $lastbreak = time(); }
};
# FIXME: This will soon need to be changed to use methods of the $u
# object rather than global LJ:: functions, but this should work
# for now.
my $runsql = sub {
my $db = $dbh;
if (ref $_[0]) { $db = shift; }
my $user = shift;
my $sql = shift;
print " ($user) $sql\n";
$db->do($sql);
};
my $czero = 0;
foreach my $uid (@delusers)
{
my $du = $user->{$uid};
my $user = $du->{'user'};
print "$du->{'user'} ($du->{'userid'}) @ $du->{'statusvisdate'}";
if ($du->{clusterid} == 0) {
print " (on clusterid 0; skipping)\n";
$czero++;
next;
}
print " (cluster $du->{'clusterid'})...\n";
$pause->();
# get a db handle for the cluster master.
LJ::start_request(); # might've been awhile working with last handle, we don't want to be given an expired one.
my $dbcm = LJ::get_cluster_master($du);
$dbcm->{'RaiseError'} = 1;
$dbcm->{'PrintError'} = 1;
# make all the user's comments posted now be owned by posterid 0 (anonymous)
# but with meta-data saying who used to own it
# ..... hm, with clusters this is a pain. let's not.
# delete memories
print " memories\n";
while (($ids = $dbh->selectcol_arrayref("SELECT memid FROM memorable WHERE userid=$uid LIMIT 100")) && @{$ids})
{
my $in = join(",", @$ids);
print " id: $in\n";
$runsql->($dbh, $user, "DELETE FROM memkeyword WHERE memid IN ($in)");
$runsql->($dbh, $user, "DELETE FROM memorable WHERE memid IN ($in)");
}
# delete todos
print " todos\n";
while (($ids = $dbh->selectcol_arrayref("SELECT todoid FROM todo WHERE journalid=$uid LIMIT 100")) && @{$ids})
{
my $in = join(",", @$ids);
print " id: $in\n";
$runsql->($dbh, $user, "DELETE FROM tododep WHERE todoid IN ($in)");
$runsql->($dbh, $user, "DELETE FROM todokeyword WHERE todoid IN ($in)");
$runsql->($dbh, $user, "DELETE FROM todo WHERE todoid IN ($in)");
}
# delete userpics
{
print " userpics\n";
if ($du->{'dversion'} > 6) {
$ids = $dbcm->selectcol_arrayref("SELECT picid FROM userpic2 WHERE userid=$uid");
} else {
$ids = $dbh->selectcol_arrayref("SELECT picid FROM userpic WHERE userid=$uid");
}
my $in = join(",",@$ids);
if ($in) {
print " userpics: $in\n";
$runsql->($dbcm, $user, "DELETE FROM userpicblob2 WHERE userid=$uid AND picid IN ($in)");
if ($du->{'dversion'} > 6) {
$runsql->($dbcm, $user, "DELETE FROM userpic2 WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM userpicmap2 WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM userkeywords WHERE userid=$uid");
} else {
$runsql->($dbh, $user, "DELETE FROM userpic WHERE userid=$uid");
$runsql->($dbh, $user, "DELETE FROM userpicmap WHERE userid=$uid");
}
}
}
# delete posts
print " posts\n";
while (($ids = $dbcm->selectall_arrayref("SELECT jitemid, anum FROM log2 WHERE journalid=$uid LIMIT 100")) && @{$ids})
{
foreach my $idanum (@$ids) {
my ($id, $anum) = ($idanum->[0], $idanum->[1]);
print " deleting $id (a=$anum) ($uid; $du->{'user'})\n";
LJ::delete_entry($du, $id, 0, $anum);
$pause->();
}
}
# misc:
$runsql->($user, "DELETE FROM userusage WHERE userid=$uid");
$runsql->($user, "DELETE FROM friends WHERE userid=$uid");
$runsql->($user, "DELETE FROM friends WHERE friendid=$uid");
$runsql->($user, "DELETE FROM friendgroup WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM friendgroup2 WHERE userid=$uid");
$runsql->($user, "DELETE FROM memorable WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM memorable2 WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM userkeywords WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM memkeyword2 WHERE userid=$uid");
$runsql->($user, "DELETE FROM userbio WHERE userid=$uid");
$runsql->($dbcm, $user, "DELETE FROM userbio WHERE userid=$uid");
$runsql->($user, "DELETE FROM userinterests WHERE userid=$uid");
$runsql->($user, "DELETE FROM userprop WHERE userid=$uid");
$runsql->($user, "DELETE FROM userproplite WHERE userid=$uid");
$runsql->($user, "DELETE FROM txtmsg WHERE userid=$uid");
$runsql->($user, "DELETE FROM overrides WHERE user='$du->{'user'}'");
$runsql->($user, "DELETE FROM priv_map WHERE userid=$uid");
$runsql->($user, "DELETE FROM infohistory WHERE userid=$uid");
$runsql->($user, "DELETE FROM reluser WHERE userid=$uid");
$runsql->($user, "DELETE FROM reluser WHERE targetid=$uid");
$runsql->($user, "DELETE FROM userlog WHERE userid=$uid");
$runsql->($user, "UPDATE user SET statusvis='X', statusvisdate=NOW(), password='' WHERE userid=$uid");
}
if ($czero) {
print "\nWARNING: There are $czero users on cluster zero pending deletion.\n";
print " These users must be upgraded before they can be expunged with this tool.\n";
}

27
bin/dev/entrydump.pl Executable file
View File

@ -0,0 +1,27 @@
#!/usr/bin/perl
#
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
require 'ljlib.pl';
use LJ::Entry;
my $url = shift;
LJ::no_cache(sub {
my $entry = LJ::Entry->new_from_url($url);
print "entry = $entry\n";
use Data::Dumper;
print Dumper($entry->props, clean($entry->event_orig), clean($entry->event_raw));
});
sub clean {
my $txt = shift;
$txt =~ s/[^\x20-\x7f]/"[" . sprintf("%02x", ord($&)) . "]"/eg;
return $txt;
}

27
bin/dev/inject_sms.pl Executable file
View File

@ -0,0 +1,27 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
use LJ::SMS::Message;
require "ljlib.pl";
my ($user, $msg) = @ARGV[0,1];
my $u = LJ::load_user($user);
my $ljmsg = LJ::SMS::Message->new
( owner => $u,
from => $u,
type => 'incoming',
body_text => $msg,
);
warn LJ::D($ljmsg);
warn "Enqueue\n";
LJ::SMS->enqueue_as_incoming($ljmsg);

217
bin/dumpsql.pl Executable file
View File

@ -0,0 +1,217 @@
#!/usr/bin/perl
#
# <LJDEP>
# lib: cgi-bin/ljlib.pl
# </LJDEP>
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
require "$ENV{'LJHOME'}/cgi-bin/ljviews.pl";
my $dbh = LJ::get_db_writer();
sub header_text {
return <<"HEADER";
# This file is automatically generated from MySQL by \$LJHOME/bin/dumpsql.pl
# Don't submit a diff against a hand-modified file - dump and diff instead.
HEADER
}
# what tables don't we want to export the auto_increment columns from
# because they already have their own unique string, which is what matters
my %skip_auto = (
"priv_list" => "privcode",
"supportcat" => "catkey",
"ratelist" => "name",
);
# get tables to export
my %tables = ();
my $sth = $dbh->prepare("SELECT tablename, redist_mode, redist_where ".
"FROM schematables WHERE redist_mode NOT IN ('off')");
$sth->execute;
while (my ($table, $mode, $where) = $sth->fetchrow_array) {
$tables{$table}->{'mode'} = $mode;
$tables{$table}->{'where'} = $where;
}
my %output; # {general|local} -> [ [ $alphasortkey, $SQL ]+ ]
# dump each table.
foreach my $table (sort keys %tables)
{
next if $table =~ /^(user|talk|log)proplist$/;
my $where;
if ($tables{$table}->{'where'}) {
$where = "WHERE $tables{$table}->{'where'}";
}
my $sth = $dbh->prepare("DESCRIBE $table");
$sth->execute;
my @cols = ();
my $skip_auto = 0;
while (my $c = $sth->fetchrow_hashref) {
if ($c->{'Extra'} =~ /auto_increment/ && $skip_auto{$table}) {
$skip_auto = 1;
} else {
push @cols, $c;
}
}
# DESCRIBE table can be different between developers
@cols = sort { $a->{'Field'} cmp $b->{'Field'} } @cols;
my $cols = join(", ", map { $_->{'Field'} } @cols);
my $sth = $dbh->prepare("SELECT $cols FROM $table $where");
$sth->execute;
my $sql;
while (my @r = $sth->fetchrow_array)
{
my %vals;
my $i = 0;
foreach (map { $_->{'Field'} } @cols) {
$vals{$_} = $r[$i++];
}
my $scope = "general";
$scope = "local" if (defined $vals{'scope'} &&
$vals{'scope'} eq "local");
my $verb = "INSERT IGNORE";
$verb = "REPLACE" if ($tables{$table}->{'mode'} eq "replace" &&
! $skip_auto);
$sql = "$verb INTO $table ";
$sql .= "($cols) ";
$sql .= "VALUES (" . join(", ", map { db_quote($_) } @r) . ");\n";
my $uniqc = $skip_auto{$table};
my $skey = $uniqc ? $vals{$uniqc} : $sql;
push @{$output{$scope}}, [ "$table.$skey.1", $sql ];
if ($skip_auto) {
# for all the *proplist tables, there might be new descriptions
# or columns, but we can't do a REPLACE, because that'd mess
# with their auto_increment ids, so we do insert ignore + update
my $where = "$uniqc=" . db_quote($vals{$uniqc});
delete $vals{$uniqc};
$sql = "UPDATE $table SET ";
$sql .= join(",", map { "$_=" . db_quote($vals{$_}) } sort keys %vals);
$sql .= " WHERE $where;\n";
push @{$output{$scope}}, [ "$table.$skey.2", $sql ];
}
}
}
# don't use $dbh->quote because it's changed between versions
# and developers sending patches can't generate concise patches
# it used to not quote " in a single quoted string, but later it does.
# so we'll implement the new way here.
sub db_quote {
my $s = shift;
return "NULL" unless defined $s;
$s =~ s/\\/\\\\/g;
$s =~ s/\"/\\\"/g;
$s =~ s/\'/\\\'/g;
$s =~ s/\n/\\n/g;
$s =~ s/\r/\\r/g;
return "'$s'";
}
foreach my $k (keys %output) {
my $file = $k eq "general" ? "base-data.sql" : "base-data-local.sql";
print "Dumping $file\n";
my $ffile = "$ENV{'LJHOME'}/bin/upgrading/$file";
open (F, ">$ffile") or die "Can't write to $ffile\n";
print F header_text();
foreach (sort { $a->[0] cmp $b->[0] } @{$output{$k}}) {
print F $_->[1];
}
close F;
}
# dump proplists, etc
print "Dumping proplists.dat\n";
open (my $plg, ">$ENV{LJHOME}/bin/upgrading/proplists.dat") or die;
open (my $pll, ">$ENV{LJHOME}/bin/upgrading/proplists-local.dat") or die;
foreach my $table ('userproplist', 'talkproplist', 'logproplist', 'usermsgproplist') {
my $sth = $dbh->prepare("DESCRIBE $table");
$sth->execute;
my @cols = ();
while (my $c = $sth->fetchrow_hashref) {
die "Where is the 'Extra' column?" unless exists $c->{'Extra'}; # future-proof
next if $c->{'Extra'} =~ /auto_increment/;
push @cols, $c;
}
@cols = sort { $a->{'Field'} cmp $b->{'Field'} } @cols;
my $cols = join(", ", map { $_->{'Field'} } @cols);
my $pri_key = "name"; # for now they're all 'name'. might add more tables.
$sth = $dbh->prepare("SELECT $cols FROM $table ORDER BY $pri_key");
$sth->execute;
while (my @r = $sth->fetchrow_array) {
my %vals;
my $i = 0;
foreach (map { $_->{'Field'} } @cols) {
$vals{$_} = $r[$i++];
}
my $scope = $vals{'scope'} && $vals{'scope'} eq "local" ? "local" : "general";
my $fh = $scope eq "local" ? $pll : $plg;
print $fh "$table.$vals{$pri_key}:\n";
foreach my $c (map { $_->{'Field'} } @cols) {
next if $c eq $pri_key;
next if $c eq "scope"; # implied by filenamea
print $fh " $c: $vals{$c}\n";
}
print $fh "\n";
}
}
# now dump school related information
print "Dumping schools.dat\n";
open(F, ">$ENV{LJHOME}/bin/upgrading/schools.dat") or die;
$sth = $dbh->prepare('SELECT name, country, state, city, url FROM schools');
$sth->execute;
while (my @row = $sth->fetchrow_array) {
my $line = '"' . join('","', map { $_ || "" } @row) . '"';
print F "$line\n";
}
close F;
# and do S1 styles (ugly schema)
print "Dumping s1styles.dat\n";
require "$ENV{'LJHOME'}/bin/upgrading/s1style-rw.pl";
my $ss = {};
my $pubstyles = LJ::S1::get_public_styles({ 'formatdata' => 1});
foreach my $s (values %$pubstyles) {
my $uniq = "$s->{'type'}/$s->{'styledes'}";
$ss->{$uniq}->{$_} = $s->{$_} foreach keys %$s;
}
s1styles_write($ss);
# and dump mood info
print "Dumping moods.dat\n";
open (F, ">$ENV{'LJHOME'}/bin/upgrading/moods.dat") or die;
$sth = $dbh->prepare("SELECT moodid, mood, parentmood FROM moods ORDER BY moodid");
$sth->execute;
while (@_ = $sth->fetchrow_array) {
print F "MOOD @_\n";
}
$sth = $dbh->prepare("SELECT moodthemeid, name, des FROM moodthemes WHERE is_public='Y' ORDER BY name");
$sth->execute;
while (my ($id, $name, $des) = $sth->fetchrow_array) {
$name =~ s/://;
print F "MOODTHEME $name : $des\n";
my $std = $dbh->prepare("SELECT moodid, picurl, width, height FROM moodthemedata ".
"WHERE moodthemeid=$id ORDER BY moodid");
$std->execute;
while (@_ = $std->fetchrow_array) {
print F "@_\n";
}
}
close F;
print "Done.\n";

179
bin/evwatch Executable file
View File

@ -0,0 +1,179 @@
#!/usr/bin/perl
use strict;
use warnings;
use lib "$ENV{LJHOME}/cgi-bin";
require 'ljlib.pl';
$|++;
use Errno qw(EAGAIN EWOULDBLOCK);
use LJ::Blockwatch;
use IO::Socket::INET;
use Time::HiRes qw(tv_interval);
my %files;
my %filetimes;
my $last_time_checked = time();
my %time_averages;
my %sockets; # fileno -> IO::Socket::INET socket
my %socket_destinations; # fileno -> "hostname:port"
#############################################################################
# This block handles initial connections to the nodes we want to listen to.
# The list is hard coded in the @destinations list for the moment.
#############################################################################
{
my %connecting_sockets;
my @destinations = qw(localhost:7600 127.0.0.1:7600);
foreach my $dest (@destinations) {
my $sock = IO::Socket::INET->new( PeerHost => $dest, Blocking => 0 ) or die "Couldn't connect: $!";
$connecting_sockets{$sock->fileno} = $sock;
$socket_destinations{$sock->fileno} = $dest;
}
sleep 3;
my $win = '';
foreach my $fd (keys %connecting_sockets) {
vec($win, $fd, 1) = 1;
}
select(undef, my $wout = $win, undef, 0);
while (my ($fd, $sock) = each %connecting_sockets) {
if (vec($wout, $fd, 1)) {
$sockets{$fd} = $sock;
$sock->write("evwatch\n");
}
}
}
die "Nothing allowed us to connect" unless keys %sockets;
my %socket_buffers = map { ($_, '') } keys %sockets; # fileno -> buffer
#############################################################################
# This block handles listening to each of the sockets for reading and handing
# the incoming data off to sub process_line anytime there has been a full
# line read.
#############################################################################
while (1) {
my $rin = '';
foreach my $fd (keys %sockets) {
vec($rin, $fd, 1) = 1;
}
select(my $rout = $rin, undef, undef, undef);
# Read data from the sockets that are ready
SOCK: foreach my $fd (keys %sockets) {
my $sock = $sockets{$fd};
my $bufref = \$socket_buffers{$fd};
if (vec($rout, $fd, 1)) {
READ: while (1) {
my $length = sysread($sock, my $read_buffer, 1024);
if ($length) {
$$bufref .= $read_buffer;
next READ; # Read again, till we get a read error.
}
if ($! == EAGAIN || $! == EWOULDBLOCK) {
last READ; # We've read all we can on this loop.
}
# Other errors mean we just close the connection and move on.
delete $sockets{$fd};
delete $socket_buffers{$fd};
next SOCK;
}
my $dest = $socket_destinations{$fd};
while ($$bufref =~ s/(.*?)\r?\n//) {
my $line = $1;
next unless $line;
my ($filename, $time, $utime, $direction, $event) = split /,/, $line;
process_line("${dest}${filename}", $time, $utime, $direction, $event);
}
}
}
}
#############################################################################
# Process a line of incoming data, arguments are:
# label - hostname and filename concatted together
# time, utime - pair of integers that report when this event happened
# direction - boolean indicating the direction of this event
# begin is 0
# end is 1
# event - integer representing the event that occurred
#############################################################################
sub process_line {
my ($label, $time, $utime, $direction, $event) = @_;
my $filename = $label;
my $current_time = time();
$filetimes{$filename} = $current_time;
my $filedata = $files{$filename} ||= {};
my $eventdata = $filedata->{$event} ||= [];
if ($direction) { # backing out one operation
my $start_times = pop @$eventdata;
delete $filedata->{$event} unless @$eventdata;
return unless $start_times;
my $interval = tv_interval($start_times, [$time, $utime]);
my $average = \$time_averages{$event};
if (defined $$average) {
$$average *= .95;
$$average += ($interval * .05);
} else {
$$average = $interval;
}
} else { # adding an event
push @$eventdata, [$time, $utime];
}
if ($last_time_checked + 1 <= $current_time) {
$last_time_checked = $current_time;
foreach my $key (keys %filetimes) {
if ($filetimes{$key} < $current_time - 10) {
print "Removing $key.\n";
delete $filetimes{$key};
delete $files{$key};
}
}
dump_stats();
}
}
sub dump_stats {
while (my ($filename, $filedata) = each %files) {
next unless keys %$filedata;
print "For '$filename'\n";
while (my ($event, $times) = each %$filedata) {
my $event_name = LJ::Blockwatch->get_event_name($event);
print " $event_name has " . @$times . " outstanding.\n";
}
} continue { print "\n"; }
foreach my $event (map {$_->[1]}
sort {$a->[0] <=> $b->[0]}
map { [$time_averages{$_}, $_] }
keys %time_averages) {
my $time = $time_averages{$event};
my $event_name = LJ::Blockwatch->get_event_name($event);
printf "$time\t$event_name\n";
}
print "\n";
}

53
bin/fakesms-djabberd Executable file
View File

@ -0,0 +1,53 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
use lib "$ENV{LJHOME}/src/djabberd/lib";
require 'ljconfig.pl';
use DJabberd;
use DJabberd::Authen::AllowedUsers;
use DJabberd::Authen::StaticPassword;
use DJabberd::Delivery::Local;
use DJabberd::FakeSMS;
use Getopt::Long;
use vars qw($DEBUG);
$DEBUG = 0;
my ($daemonize);
Getopt::Long::GetOptions(
'd|daemon' => \$daemonize,
'debug=i' => \$DEBUG,
);
use FindBin qw($Bin);
use DJabberd::VHost;
my $vhost = DJabberd::VHost->new(
server_name => $LJ::DOMAIN,
s2s => 0,
plugins => [
DJabberd::Authen::AllowedUsers->new(policy => "deny",
allowedusers => [qr/^\+?\d+/, 'sms']),
DJabberd::Authen::StaticPassword->new(password => "smstest"),
DJabberd::Delivery::FakeSMS->new(),
DJabberd::Delivery::Local->new(),
DJabberd::RosterStorage::FakeSMS->new(),
DJabberd::PresenceChecker::FakeSMS->new(),
],
);
my $server = DJabberd->new(
daemonize => $daemonize,
);
$server->add_vhost($vhost);
# incoming
$server->start_simple_server(5224);
$server->run;

144
bin/fingerd.pl Executable file
View File

@ -0,0 +1,144 @@
#!/usr/bin/perl
#
# finger server.
#
# accepts two optional arguments, host and port.
# doesn't daemonize.
#
#
# <LJDEP>
# lib: Socket::, Text::Wrap, cgi-bin/ljlib.pl
# </LJDEP>
my $bindhost = shift @ARGV;
my $port = shift @ARGV;
unless ($bindhost) {
$bindhost = "0.0.0.0";
}
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
use Socket;
use Text::Wrap;
$SIG{'INT'} = sub {
print "Interrupt caught!\n";
close FH;
close CL;
exit;
};
my $proto = getprotobyname('tcp');
socket(FH, PF_INET, SOCK_STREAM, $proto) || die $!;
$port ||= 79;
my $localaddr = inet_aton($bindhost);
my $sin = sockaddr_in($port, $localaddr);
setsockopt (FH,SOL_SOCKET,SO_REUSEADDR,1) or
die "setsockopt() failed: $!\n";
bind (FH, $sin) || die $!;
listen(FH, 10);
while (LJ::start_request())
{
accept(CL, FH) || die $!;
my $line = <CL>;
chomp $line;
$line =~ s/\0//g;
$line =~ s/\s//g;
if ($line eq "") {
print CL "Welcome to the $LJ::SITENAME finger server!
You can make queries in the following form:
\@$LJ::DOMAIN - this help message
user\@$LJ::DOMAIN - their userinfo
";
close CL;
next;
}
my $dbr = LJ::get_dbh("slave", "master");
if ($line =~ /^(\w{1,15})$/) {
# userinfo!
my $user = $1;
my $quser = $dbr->quote($user);
my $sth = $dbr->prepare("SELECT user, has_bio, caps, userid, name, email, bdate, allow_infoshow FROM user WHERE user=$quser");
$sth->execute;
my $u = $sth->fetchrow_hashref;
unless ($u) {
print CL "\nUnknown user ($user)\n";
close CL;
next;
}
my $bio;
if ($u->{'has_bio'} eq "Y") {
$sth = $dbr->prepare("SELECT bio FROM userbio WHERE userid=$u->{'userid'}");
$sth->execute;
($bio) = $sth->fetchrow_array;
}
delete $u->{'has_bio'};
$u->{'accttype'} = LJ::name_caps($u->{'caps'});
if ($u->{'allow_infoshow'} eq "Y") {
LJ::load_user_props($dbr, $u, "opt_whatemailshow",
"country", "state", "city", "zip",
"aolim", "icq", "url", "urlname",
"yahoo", "msn");
} else {
$u->{'opt_whatemailshow'} = "N";
}
delete $u->{'allow_infoshow'};
if ($u->{'opt_whatemailshow'} eq "L") {
delete $u->{'email'};
}
if ($LJ::USER_EMAIL && LJ::get_cap($u, "useremail")) {
if ($u->{'email'}) { $u->{'email'} .= ", "; }
$u->{'email'} .= "$user\@$LJ::USER_DOMAIN";
}
if ($u->{'opt_whatemailshow'} eq "N") {
delete $u->{'email'};
}
delete $u->{'opt_whatemailshow'};
my $max = 1;
foreach (keys %$u) {
if (length($_) > $max) { $max = length($_); }
}
$max++;
delete $u->{'caps'};
print CL "\nUserinfo for $user...\n\n";
foreach my $k (sort keys %$u) {
printf CL "%${max}s : %s\n", $k, $u->{$k};
}
if ($bio) {
$bio =~ s/^\s+//;
$bio =~ s/\s+$//;
print CL "\nBio:\n\n";
$Text::Wrap::columns = 77;
print CL Text::Wrap::wrap(" ", " ", $bio);
}
print CL "\n\n";
close CL;
next;
}
print CL "Unsupported/unimplemented query type: $line\n";
print CL "length: ", length($line), "\n";
close CL;
next;
}

166
bin/gens2editlib.pl Executable file
View File

@ -0,0 +1,166 @@
#!/usr/bin/perl
# ----------------------------------------------------------------------------
# S2 editor library generation script 08/03/2005
#
# Generates s2library.js from the core layer definitions.
# ----------------------------------------------------------------------------
use strict;
use warnings;
use Getopt::Long;
use Pod::Usage;
my ($filename, $layerid, $query, $outputpath, $help);
GetOptions(
'help|h' => \$help,
'filename|f=s' => \$filename,
'layerid|l=i' => $layerid,
'query|q' => \$query,
'output|o=s' => \$outputpath) or pod2usage(1);
pod2usage(0) if $help;
my $home = $ENV{LJHOME} or die "You'll have to set \$LJHOME first.\n";
require "$home/cgi-bin/ljlib.pl";
require "$home/cgi-bin/LJ/S2.pm";
$outputpath ||= "$home/htdocs/customize/advanced/s2edit/s2library.js";
my $info;
if ($filename) {
local $/ = undef;
open F, $filename or die $!;
eval <F>;
die $@ if $@;
close F;
$info = S2::get_layer_all(defined($layerid) ? $layerid : 1);
} elsif ($query) {
my $pub = LJ::S2::get_public_layers();
my $id = $pub->{core1};
$id = $id ? $id->{'s2lid'} : 0;
die "Couldn't locate a core 1 layer.\n" unless $id;
my $dbr = LJ::get_db_reader();
my $rv = S2::load_layers_from_db($dbr, $id);
$info = S2::get_layer_all($id);
} else {
pod2usage(1);
}
open LIB, ">$outputpath" or die "Failed to open $outputpath for writing: $!\n";
select LIB;
print "// Automatically generated by gens2editlib.pl\n";
print "// Do not edit!\n\n";
# ----------------------------------------------------------------------------
# Classes
# ----------------------------------------------------------------------------
print "// Classes\n";
print "var s2classlib = new Array(";
my %classes = %{$info->{'class'}};
my @orderedClasses = map { $_->[0] } sort { $a->[1] cmp $b->[1] } map
{ [ $_, lc $_ ] } keys %classes;
my $cma = 0;
foreach my $className (@orderedClasses) {
my $class = $classes{$className};
my (%methods, %members);
my $c = $class;
do {
%methods = (%methods, %{$c->{funcs}});
%members = (%members, %{$c->{vars}});
$c = ($c->{'parent'} ? $classes{$c->{'parent'}} : undef);
} while ($c);
print "," if $cma++;
print "\n\t{\n\t\tname: '$className',\n";
print "\t\tmembers: new Array(";
my $cm = 0;
foreach my $memberName (sort keys %members) {
print "," if $cm++;
print "\n\t\t\t{ ";
print "name: '$memberName', ";
print "type: '$members{$memberName}->{type}'";
print " }";
}
print "),\n";
print "\t\tmethods: new Array(";
$cm = 0;
foreach my $methodName (sort keys %methods) {
print "," if $cm++;
print "\n\t\t\t{ ";
print "name: '$methodName', ";
print "type: '$methods{$methodName}->{returntype}'";
print " }";
}
print ")\n";
print "\t}";
}
print ");\n\n";
# ----------------------------------------------------------------------------
# Functions
# ----------------------------------------------------------------------------
print "// Functions\n";
my $global = $info->{'global'};
print "var s2funclib = new Array(";
my $cm = 0;
foreach my $func (sort keys %$global) {
print "," if $cm++;
print "\n\t{ name: '$func', type: '$global->{$func}{returntype}' }";
}
print ");\n\n";
# ----------------------------------------------------------------------------
# Properties
# ----------------------------------------------------------------------------
print "// Properties\n";
my $props = $info->{'prop'};
print "var s2proplib = new Array(";
$cm = 0;
foreach my $prop (sort keys %$props) {
print "," if $cm++;
print "\n\t{ name: '$prop', type: '$props->{$prop}{type}' }";
}
print ");\n";
print "\n// End\n";
__END__
=head1 NAME
gens2editlib.pl - generate the LJ S2 editor library file for core layer 1
=head1 SYNOPSIS
gens2editlib.pl [-q] [-f core1.pl] [-l layerid] [-h] [-o ./s2library.js]
Options consist of:
-f, --file specify path to layer compiled with s2compile.pl
-h, --help print this help message
-l, --layerid specify layer ID number with -f option; defaults to 1
-q, --query query local database to obtain S2 core layer
-o, --output specify where to put the generated JavaScript; defaults to
$LJHOME/htdocs/customize/advanced/s2edit/s2library.js
You must specify either the -q or the -f option.
=cut

105
bin/incoming-mail-inject.pl Executable file
View File

@ -0,0 +1,105 @@
#!/usr/bin/perl
use strict;
use warnings;
BEGIN {
$ENV{LJHOME} ||= "/home/lj";
}
use lib "$ENV{LJHOME}/cgi-bin";
require "$ENV{LJHOME}/cgi-bin/ljlib.pl";
use Class::Autouse qw(
LJ::IncomingEmailHandle
);
my $sclient = LJ::theschwartz() or die "No schwartz config.\n";
my $tempfail = sub {
my $msg = shift;
warn "Failure: $msg\n" if $msg;
# makes postfix do temporary failure:
exit(75);
};
# below this size, we put in database directly. if over,
# we put in mogile.
sub IN_MEMORY_THRES () {
return
$ENV{T_MAILINJECT_THRES_SIZE} ||
768 * 1024;
}
my $buf;
my $msg = ''; # in-memory message
my $rv;
my $len = 0;
my $ieh;
my $ignore_message = 0; # bool: to ignore rest of message.
eval {
while ($rv = sysread(STDIN, $buf, 1024*64)) {
next if $ignore_message;
$len += $rv;
if ($ieh) {
$ieh->append($buf);
} else {
$msg .= $buf;
}
if ($len > IN_MEMORY_THRES && ! $ieh) {
if (should_ignore($msg)) {
$ignore_message = 1;
next;
}
# allocate a mogile filehandle once we cross the line of
# what's too big to store in memory and in a schwartz arg
$ieh = LJ::IncomingEmailHandle->new;
$ieh->append($msg);
undef $msg; # no longer used.
}
}
$tempfail->("Error reading: $!") unless defined $rv;
if ($ieh) {
$ieh->closetemp;
$tempfail->("Size doesn't match") unless $ieh->tempsize == $len;
$ieh->insert_into_mogile;
}
};
# just shut postfix up
if ($ignore_message || should_ignore($msg)) {
exit(0);
}
$tempfail->($@) if $@;
my $h = $sclient->insert(TheSchwartz::Job->new(funcname => "LJ::Worker::IncomingEmail",
arg => ($ieh ? $ieh->id : $msg)));
exit 0 if $h;
exit(75); # temporary error
# it pays to get rid of as many bounces and gibberish now, before we
# have to put it in the database, mogile, allocate ids, run workers,
# move disks around, etc.. so these are just quick & dirty checks.
sub should_ignore {
my $msg = shift;
return 0 unless $msg;
return 1 if $msg =~ /^Return-Path:\s+<>/im;
my ($subject) = $msg =~ /^Subject: (.+)/im;
if ($subject) {
return 1 if
$subject =~ /auto.?(response|reply)/i
|| $subject =~ /^(Undelive|Mail System Error - |ScanMail Message: |\+\s*SPAM|Norton AntiVirus)/i
|| $subject =~ /^(Mail Delivery Problem|Mail delivery failed)/i
|| $subject =~ /^failure notice$/i
|| $subject =~ /\[BOUNCED SPAM\]/i
|| $subject =~ /^Symantec AVF /i
|| $subject =~ /Attachment block message/i
|| $subject =~ /Use this patch immediately/i
|| $subject =~ /^don\'t be late! ([\w\-]{1,15})$/i
|| $subject =~ /^your account ([\w\-]{1,15})$/i
|| $subject =~ /Message Undeliverable/i;
}
return 0;
}

45
bin/lj-repo-own Executable file
View File

@ -0,0 +1,45 @@
#!/usr/bin/perl
#
use strict;
use Getopt::Long;
my $anon = 0;
my $user = "";
sub usage () {
die "
Usage:
lj-repo-own --anonymous
lj-repo-own --user=<danga_username>
";
}
usage() unless GetOptions(
"anonymous" => \$anon,
"user=s" => \$user,
);
usage() if $anon && $user;
usage() unless $anon || $user =~ /^\w+$/;
my $src = $anon ?
":pserver:anonymous\@cvs.livejournal.org:" :
"$user\@cvs.danga.com:";
chdir "$ENV{LJHOME}" or die;
my @files = `find cvs -type f -path '*CVS/Root'`;
chomp @files;
foreach my $f (@files) {
open(R, $f) or die;
my $line = <R>;
close R;
print "in $f\tfrom $line ";
$line =~ s/.+:/$src/;
print " to $line\n";
open(W, ">$f") or die;
print W $line;
close W;
}

35
bin/lj-upgrade Executable file
View File

@ -0,0 +1,35 @@
#!/usr/bin/perl
use strict;
use IO::Socket::INET;
unless ($ENV{LJHOME}) {
die "\$LJHOME not set.";
}
chdir "$ENV{LJHOME}" or die "Failed to chdir to \$LJHOME";
system("cvsreport.pl", "-u", "-c", "-s")
and die "Failed to run cvsreport.pl with update.";
system("cvsreport.pl", "-c", "-s")
and die "Failed to run cvsreport.pl second time.";
system("bin/upgrading/update-db.pl", "-r", "-p")
and die "Failed to run update-db.pl with -r/-p";
system("bin/upgrading/update-db.pl", "-r", "--cluster=all")
and die "Failed to run update-db.pl on all clusters";
system("bin/upgrading/texttool.pl", "load")
and die "Failed to run texttool.pl load";
print "Restarting apache...\n";
my $sock = IO::Socket::INET->new(PeerAddr => "127.0.0.1:7600")
or die "Couldn't connect to webnoded (port 7600)\n";
print $sock "apr\r\n";
while (my $ln = <$sock>) {
print "$ln";
last if $ln =~ /^OK/;
}

631
bin/ljblockwatcher.pl Executable file
View File

@ -0,0 +1,631 @@
#!/usr/bin/perl
##############################################################################
=head1 NAME
dbreportd - Report database latencies at regular intervals.
=head1 SYNOPSIS
$ dbreportd OPTIONS
=head1 OPTIONS
=over 4
=item -c, --clearscreen
Clear the screen and home the cursor before printing each report, like top. May
not work on all terminals.
=item -d, --debug
Turn on debugging information. May be specified more than once for (potentially)
increased levels of debugging.
=item -h, --help
Output a help message and exit.
=item -i, --interval=SECONDS
Set the number of seconds between reports to SECONDS. Defaults to 3 second
intervals.
=item -p, --port=PORT
Set the port to listen on for reports. This is set in ljconfig.pl, but can be
overridden here.
=item -V, --version
Output version information and exit.
=back
=head1 REQUIRES
I<Token requires line>
=head1 DESCRIPTION
None yet.
=head1 AUTHOR
Michael Granger <ged@FaerieMUD.org>
Copyright (c) 2004 Danga Interactive. All rights reserved.
This program is Open Source software. You may use, modify, and/or redistribute
this software under the terms of the Perl Artistic License. (See
http://language.perl.com/misc/Artistic.html)
THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND
FITNESS FOR A PARTICULAR PURPOSE.
=cut
# :TODO: Change param order in received msgs
##############################################################################
package dbreportd;
use strict;
use warnings qw{all};
###############################################################################
### I N I T I A L I Z A T I O N
###############################################################################
BEGIN {
# Turn STDOUT buffering off
$| = 1;
# Versioning stuff and custom includes
use vars qw{$VERSION $RCSID};
$VERSION = do { my @r = (q$Revision: 3794 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };
$RCSID = q$Id: ljblockwatcher.pl 3794 2004-03-09 22:25:05Z deveiant $;
# Define some constants
use constant TRUE => 1;
use constant FALSE => 0;
# How many time samples to keep around to determine average latency
use constant SAMPLE_DEPTH => 10;
# How many samples to show in the "top <n> slow queries"
use constant TOP_QUERY_SIZE => 5;
# ANSI vt100 escape codes for various things
use constant VT100_CLEARSCREEN => "\e[2J";
use constant VT100_HOME => "\e[0;0H";
# Modules
use Getopt::Long qw{GetOptions};
use Pod::Usage qw{pod2usage};
use IO::Socket::INET qw{};
use IO::Select qw{};
use Time::HiRes qw{usleep};
use Data::Dumper qw{};
# Turn on option bundling (-vid)
Getopt::Long::Configure( "bundling" );
}
our $Debug = FALSE;
### Main body
MAIN: {
my (
$helpFlag, # User requested help?
$versionFlag, # User requested version info?
$interval, # Interval between generated reports
$port, # Port number to listen on
$msg, # The message buffer for reports
$sock, # UDP socket
$selector, # IO::Select object
$lastReport, # time() of last report output
$host, # Message host
$time, # Message time
$notes, # Message notes
$type, # Operation type (currently unused)
%buffers, # SampleBuffers keyed by host
$clearscreenFlag, # Clear the screen before every report?
);
# Print the program header and read in command line options
GetOptions(
'd|debug+' => \$Debug,
'h|help' => \$helpFlag,
'i|interval=i' => \$interval,
'V|version' => \$versionFlag,
'p|port=i' => \$port,
'c|clearscreen' => \$clearscreenFlag,
) or abortWithUsage();
# If the -h flag or -V flag was given, just show the help or version,
# respectively, and exit.
helpMode() and exit if $helpFlag;
versionMode() and exit if $versionFlag;
# Either get the port from the command line or a default
$port ||= 4774;
# Set the interval to a default if it wasn't specified
$interval = 3 if !defined $interval;
# Open a receiving UDP socket
print VT100_CLEARSCREEN, VT100_HOME if $clearscreenFlag;
print "Setting up listener on port $port\n";
$sock = new IO::Socket::INET (
Proto => 'udp',
LocalPort => $port
) or die "Failed to open receiving socket: $!";
$selector = new IO::Select;
$selector->add( $sock );
$lastReport = time();
%buffers = ();
# Print reports every couple of seconds
while ( 1 ) {
if ( $selector->can_read($interval) ) {
# Get the message and split it back into four parts
my $addr = $sock->recv( $msg, 1024, 0 );
print ">>> Message: $msg\n" if $Debug;
( $host, $type, $time, $notes ) = split( /\x3/, $msg, 4 );
# Add the time and notes to the table of hosts
$buffers{ $host } ||= new SampleBuffer ( $host, depth => SAMPLE_DEPTH );
$buffers{ $host }->add( $type, $time, $notes );
} else {
sleep 0.5;
}
} continue {
if ( (time() - $lastReport) > $interval ) {
print VT100_CLEARSCREEN, VT100_HOME if $clearscreenFlag;
print_report( values %buffers );
$lastReport = time();
}
};
}
### FUNCTION: print_report( @buffers )
### Given a list of SampleBuffer objects, print a table with the ones with the
### highest average times.
sub print_report {
my @buffers = @_;
my (
$row, # Row count for display
@top, # Top 5 slowest average buffers
%top, # ^-- Hash of same
@sbuffers, # Buffer objects sorted by hostname
@wbuffers, # Buffer objects sorted by worst op
$fmt, # printf format for report rows
$prefix, # Line prefixes
);
if ( @buffers ) {
# Pick the 5 slowest operations
@top =
map { $_->[0] }
sort { $b->[1] <=> $a->[1] }
map { [$_->host, $_->average_time] }
@buffers;
$row = 0;
%top = ();
foreach my $host ( @top[0 ... TOP_QUERY_SIZE] ) {
last unless defined $host;
$top{$host} = ++$row unless exists $top{$host};
}
#print Data::Dumper->Dumpxs( [\%top], ['top'] ), "\n";
# Make an array of sorted buffer objects by worst average time
@sbuffers =
map { $_->[0] }
sort { $b->[1] <=> $a->[1] }
map { [$_, $_->average_time] }
@buffers;
# Make an array of sorted buffer objects by worst time
@wbuffers =
map { $_->[0] }
sort { $b->[1] <=> $a->[1] }
map { [$_, $_->worst_time] }
@buffers;
# Output all hosts with the average worst operation times
$fmt = "%-2s%25s %0.5fs";
$row = 0;
header( "Average longest blocking operations, by host" );
foreach my $buf (@sbuffers) {
$row++;
if ( exists $top{$buf->host} && $top{$buf->host} <= 3 ) {
$prefix = '+';
} else {
$prefix = ' ';
}
printf "$fmt\n", $prefix, $buf->host, $buf->average_time;
}
print "\n";
# Output the worst operations with their notes
$row = 0;
$fmt = "%0.5fs: '%s' [%s/%s]\n";
header( "%d worst blockers", TOP_QUERY_SIZE );
foreach my $buf (@wbuffers[0 ... TOP_QUERY_SIZE]) {
last unless defined $buf;
$row++;
my $sample = $buf->worst_sample;
printf( $fmt,
$sample->time,
$sample->notes || "(none)",
$sample->type,
$buf->host );
}
print "\n";
# Print the raw buffer objects if debugging
if ( $Debug ) {
header( "Raw buffers" );
foreach my $buf ( @buffers ) {
local $Data::Dumper::Indent = 0;
local $Data::Dumper::Terse = TRUE;
print Data::Dumper->Dumpxs( [$buf], ['buf'] ), "\n";
}
}
print "\n";
}
else {
print "No hosts reporting.\n";
}
}
### FUNCTION: header( $fmt, @args )
### Printf the given message as a header.
sub header {
my ( $fmt, @args ) = @_;
my $msg = sprintf( $fmt, @args );
chomp( $msg );
print "$msg\n", '-' x 75, "\n";
}
### FUNCTION: helpMode()
### Exit normally after printing the usage message
sub helpMode {
pod2usage( -verbose => 1, -exitval => 0 );
}
### FUNCTION: versionMode()
### Exit normally after printing version information
sub versionMode {
print STDERR "dbreportd $VERSION\n";
exit;
}
### FUNCTION: abortWithUsage()
### Abort the program showing usage message.
sub abortWithUsage {
if ( @_ ) {
pod2usage( -verbose => 1, -exitval => 1, -msg => join('', @_) );
} else {
pod2usage( -verbose => 1, -exitval => 1 );
}
}
### FUNCTION: abort( @messages )
### Print the specified messages to the terminal and exit with a non-zero status.
sub abort {
my $msg = @_ ? join '', @_ : "unknown error";
print STDERR $msg, "\n";
exit 1;
}
#####################################################################
### T I M E S A M P L E C L A S S
#####################################################################
package Sample;
use strict;
BEGIN {
use vars qw{$AUTOLOAD};
use Carp qw{croak confess};
use Data::Dumper ();
$Data::Dumper::Terse = 1;
$Data::Dumper::Indent = 1;
}
### METHOD: new( $host )
### Create a new sample buffer for the given host
sub new {
my $proto = shift;
my $class = ref $proto || $proto;
my $self = bless {
type => 'db',
time => 0.0,
notes => '',
}, $class;
if ( @_ && (@_ % 2 == 0) ) {
my %args = @_;
foreach my $meth ( keys %args ) {
$self->$meth( $args{$meth} );
}
}
return $self;
}
### FUNCTION: blessed( $var )
### Returns a true value if the given value is a blessed reference.
sub blessed {
my $arg = shift;
return ref $arg && UNIVERSAL::isa( $arg, 'UNIVERSAL' );
}
### (PROXY) METHOD: AUTOLOAD( @args )
### Proxy method to build (non-translucent) object accessors.
sub AUTOLOAD {
my $self = shift;
confess "Cannot be called as a function" unless $self && blessed $self;
( my $name = $AUTOLOAD ) =~ s{.*::}{};
### Build an accessor for extant attributes
if ( exists $self->{$name} ) {
### Define an accessor for this attribute
my $method = sub {
my $closureSelf = shift or confess "Cannot be called as a function";
$closureSelf->{$name} = shift if @_;
return $closureSelf->{$name};
};
### Install the new method in the symbol table
NO_STRICT_REFS: {
no strict 'refs';
*{$AUTOLOAD} = $method;
}
### Now jump to the new method after sticking the self-ref back onto the
### stack
unshift @_, $self;
goto &$AUTOLOAD;
}
### Try to delegate to our parent's version of the method
my $parentMethod = "SUPER::$name";
return $self->$parentMethod( @_ );
}
sub DESTROY {}
sub END {}
#####################################################################
### S A M P L E B U F F E R C L A S S
#####################################################################
### Class for tracking latencies for a given host
package SampleBuffer;
use strict;
BEGIN {
use Carp qw{croak confess};
use vars qw{$AUTOLOAD};
}
### METHOD: new( $host )
### Create a new sample buffer for the given host
sub new {
my $proto = shift;
my $class = ref $proto || $proto;
my $host = shift or die "No hostname given";
my $self = bless {
host => $host,
samples => {},
depth => 10,
}, $class;
if ( @_ && (@_ % 2 == 0) ) {
my %args = @_;
foreach my $meth ( keys %args ) {
$self->$meth( $args{$meth} );
}
}
return $self;
}
### METHOD: add( $type, $time, $notes )
### Add the specified I<time> to the sample buffer for the given I<type> with
### the given I<notes>.
sub add {
my $self = shift or confess "Cannot be called as a function";
my ( $type, $time, $notes ) = @_;
$self->{samples}{ $type } ||= [];
my $slist = $self->{samples}{ $type };
my $sample = new Sample ( type => $type, time => $time, notes => $notes );
unshift @$slist, $sample;
pop @$slist if @$slist > $self->{depth};
return scalar @$slist;
}
### METHOD: samples( [$type] )
### Fetch a list of the samples of the given I<type> in the buffer, or a list of
### all samples if I<type> is not specified.
sub samples {
my $self = shift or confess "Cannot be used as a function";
my $type = shift;
my @samples = ();
# Gather the samples that are going to be used to make the average, either
# the specific kind requested, or all of 'em.
if ( $type ) {
# Regexp filter
if ( ref $type eq 'Regexp' ) {
foreach my $key ( keys %{$self->{samples}} ) {
push @samples, @{$self->{samples}{ $key }}
if $type =~ $key;
}
}
# Any other filter just gets string-equalled.
else {
@samples = @{$self->{samples}{ $type }};
}
} else {
foreach my $type ( keys %{$self->{samples}} ) {
push @samples, @{$self->{samples}{ $type }};
}
}
return @samples;
}
### METHOD: average_time( [$type] )
### Return the average of all the times currently in the buffer for the given
### I<type>. If I<type> isn't given, returns the overall average.
sub average_time {
my $self = shift or confess "Cannot be used as a function";
my $type = shift;
my ( $time, $count ) = ( 0, 0 );
# Now add and count all the time from each target sample
foreach my $sample ( $self->samples($type) ) {
$time += $sample->time;
$count++;
}
return $time / $count;
}
### METHOD: worst_sample( [$type] )
### Return the worst sample in the buffer for the given I<type>. If no type is
### given, return the worst overall sample.
sub worst_sample {
my $self = shift or confess "Cannot be used as a function";
my $type = shift;
return () unless %{$self->{samples}};
my @samples =
map { $_->[0] }
sort { $a->[1] <=> $b->[1] }
map { [$_, $_->time] } $self->samples( $type );
return $samples[-1];
}
### METHOD: worst_time( [$type] )
### Return the worst time in the buffer for the given I<type>. If no I<type> is
### given, returns the worst overall time.
sub worst_time {
my $self = shift or confess "Cannot be used as a function";
my $type = shift;
my $samp = $self->worst_sample( $type ) or return ();
return $samp->time;
}
### METHOD: worst_notes( [$type] )
### Return the notes from the worst sample in the buffer for the given
### I<type>. If I<type> is not specified, returns the notes for the worst
### overall sample.
sub worst_notes {
my $self = shift or confess "Cannot be used as a function";
my $type = shift;
my $samp = $self->worst_sample( $type ) or return ();
return $samp->notes;
}
### FUNCTION: blessed( $var )
### Returns a true value if the given value is a blessed reference.
sub blessed {
my $arg = shift;
return ref $arg && UNIVERSAL::isa( $arg, 'UNIVERSAL' );
}
### (PROXY) METHOD: AUTOLOAD( @args )
### Proxy method to build (non-translucent) object accessors.
sub AUTOLOAD {
my $self = shift;
confess "Cannot be called as a function" unless $self && blessed $self;
( my $name = $AUTOLOAD ) =~ s{.*::}{};
### Build an accessor for extant attributes
if ( exists $self->{$name} ) {
### Define an accessor for this attribute
my $method = sub {
my $closureSelf = shift or confess "Cannot be called as a function";
$closureSelf->{$name} = shift if @_;
return $closureSelf->{$name};
};
### Install the new method in the symbol table
NO_STRICT_REFS: {
no strict 'refs';
*{$AUTOLOAD} = $method;
}
### Now jump to the new method after sticking the self-ref back onto the
### stack
unshift @_, $self;
goto &$AUTOLOAD;
}
### Try to delegate to our parent's version of the method
my $parentMethod = "SUPER::$name";
return $self->$parentMethod( @_ );
}
sub DESTROY {}
sub END {}

110
bin/ljdb Executable file
View File

@ -0,0 +1,110 @@
#!/usr/bin/perl
#
# ljdb connects to master
# ljdb --help
# ljdb --user=bob
# ljdb --user=bob --slave
# ljdb --role=slave
# ljdb --role=slow
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
require 'ljdb.pl';
use Getopt::Long;
my ($user, $role, $inactive, $help);
usage() unless
GetOptions(
'help' => \$help,
'inactive' => \$inactive,
'role=s' => \$role,
'user=s' => \$user,
);
usage() if $help;
sub usage {
die "Usage:
ljdb (connects to master)
ljdb bob (implies --user=bob --inactive)
ljdb --help
ljdb --user=bob
ljdb --user=bob --inactive
ljdb --role=slave
ljdb --role=slow
";
}
if (@ARGV) {
if ($ARGV[0] =~ /^\w{1,15}$/) {
$user = shift;
$inactive = 1;
} else {
usage();
}
}
usage() if $role && ($user || $inactive);
print "For more usage options, see: ljdb --help\n";
if (!$role && $user) {
die "Bogus username" unless $user =~ /^\w{1,15}$/;
my $dbs = LJ::DB::dbh_by_role('slave', 'master');
my ($userid, $cid) = $dbs->selectrow_array('SELECT userid, clusterid FROM user WHERE user = ?', undef, $user);
die "no such user\n" unless $userid && $cid;
$role = "cluster" . $cid;
print "user: $user / userid: $userid / clusterid: $cid";
if (my $ab = $LJ::CLUSTER_PAIR_ACTIVE{$cid}) {
print " / active=$ab\n";
if ($inactive) {
$role .= "b" if $ab eq 'a';
$role .= "a" if $ab eq 'b';
} else {
$role .= $ab;
}
} else {
# type must be master/slave
$role .= "slave" if $inactive && grep { $_->{role}{"${role}slave"} } values %LJ::DBINFO;
}
print "\n";
}
$role ||= "master";
# find a database (not necessarily an alive one) that matches the role
# you need. FIXME: capture mysql's output and try and reconnect to
# another one if it fails?
my $db;
my $dbname;
foreach my $key (keys %LJ::DBINFO) {
my $rec = $LJ::DBINFO{$key};
if ($key eq "master") { $rec->{role}{master} = 1; };
if ($rec->{role}{$role}) {
$dbname = $key;
$db = $rec;
last;
}
}
die "no database record for role $role\n" unless $db;
if ($db->{_fdsn}) {
$db->{_fdsn} =~ /^DBI:mysql:(\w+):host=(.+?)\|(\w+)\|(.+)/
or die "Bogus _fdsn format for $dbname: $db->{_fdsn}\n";
print "found: $1, $2, $3, $4\n";
$db->{dbname} = $1;
$db->{host} = $2;
$db->{user} = $3;
$db->{pass} = $4;
}
my $database = $db->{dbname} || "livejournal";
print "...connecting to $dbname, $db->{host}, db: $database, user: $db->{user}\n\n";
exec("mysql", "--host=$db->{host}", "--user=$db->{user}", "--password=$db->{pass}", "-A", $database);

151
bin/ljmaint.pl Executable file
View File

@ -0,0 +1,151 @@
#!/usr/bin/perl
#
# <LJDEP>
# lib: Fcntl::, cgi-bin/ljlib.pl
# file: bin/maint/taskinfo.txt, bin/maint/taskinfo-local.txt
# </LJDEP>
use strict;
use vars qw(%maint %maintinfo $VERBOSE);
unless (-d $ENV{'LJHOME'}) {
die "\$LJHOME not set.\n";
}
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
if ($LJ::DISABLED{ljmaint_tasks}) {
print "ljmaint.pl tasks disabled, exiting\n";
exit 0;
}
my $MAINT = "$LJ::HOME/bin/maint";
load_tasks();
$VERBOSE = 1; # 0=quiet, 1=normal, 2=verbose
if (@ARGV)
{
## check the correctness of the taskinfo files
if ($ARGV[0] eq "--check") {
foreach my $task (keys %maintinfo)
{
my %loaded;
my $source = $maintinfo{$task}->{'source'};
unless (-e "$MAINT/$source") {
print "$task references missing file $source\n";
next;
}
unless ($loaded{$source}++) {
require "$MAINT/$source";
}
unless (ref $maint{$task} eq "CODE") {
print "$task is missing code in $source\n";
}
}
exit 0;
}
if ($ARGV[0] =~ /^-v(.?)/) {
if ($1 eq "") { $VERBOSE = 2; }
else { $VERBOSE = $1; }
shift @ARGV;
}
my @targv;
my $hit_colon = 0;
my $exit_status = 0;
foreach my $arg (@ARGV)
{
if ($arg eq ';') {
$hit_colon = 1;
$exit_status = 1 unless
run_task(@targv);
@targv = ();
next;
}
push @targv, $arg;
}
if ($hit_colon) {
# new behavior: task1 arg1 arg2 ; task2 arg arg2
$exit_status = 1 unless
run_task(@targv);
} else {
# old behavior: task1 task2 task3 (no args, ever)
foreach my $task (@targv) {
$exit_status = 1 unless
run_task($task);
}
}
exit($exit_status);
}
else
{
print "Available tasks: \n";
foreach (sort keys %maintinfo) {
print " $_ - $maintinfo{$_}->{'des'}\n";
}
}
sub run_task
{
my $task = shift;
return unless ($task);
my @args = @_;
print "Running task '$task':\n\n" if ($VERBOSE >= 1);
unless ($maintinfo{$task}) {
print "Unknown task '$task'\n";
return;
}
$LJ::LJMAINT_VERBOSE = $VERBOSE;
require "$MAINT/$maintinfo{$task}->{'source'}";
my $opts = $maintinfo{$task}{opts} || {};
my $lock = undef;
my $lockname = "mainttask-$task";
if ($opts->{'locking'} eq "per_host") {
$lockname .= "-$LJ::SERVER_NAME";
}
unless ($opts->{no_locking} ||
($lock = LJ::locker()->trylock($lockname))
) {
print "Task '$task' already running ($DDLockClient::Error). Quitting.\n" if $VERBOSE >= 1;
exit 0;
}
eval {
$maint{$task}->(@args);
};
if ( $@ ) {
print STDERR "ERROR> task $task died: $@\n\n";
return 0;
}
return 1;
}
sub load_tasks
{
foreach my $filename (qw(taskinfo.txt taskinfo-local.txt))
{
my $file = "$MAINT/$filename";
open (F, $file) or next;
my $source;
while (my $l = <F>) {
next if ($l =~ /^\#/);
if ($l =~ /^(\S+):\s*/) {
$source = $1;
next;
}
if ($l =~ /^\s*(\w+)\s*-\s*(.+?)\s*$/) {
$maintinfo{$1}->{'des'} = $2;
$maintinfo{$1}->{'source'} = $source;
}
}
close (F);
}
}

236
bin/ljsysban.pl Executable file
View File

@ -0,0 +1,236 @@
#!/usr/bin/perl
#
use strict;
use Getopt::Long;
# parse input options
my ($list, $add, $modify, $banid, $status, $bandate, $banuntil, $banlength, $what, $value, $note);
exit 1 unless GetOptions('list' => \$list,
'add' => \$add,
'modify' => \$modify,
'banid=s' => \$banid,
'status=s' => \$status,
'bandate=s' => \$bandate,
'banuntil=s' => \$banuntil,
'banlength=s' => \$banlength,
'what=s' => \$what,
'value=s' => \$value,
'note=s' => \$note,
);
# did they give valid input?
my $an_opt = ($what || $value || $status || $bandate || $banuntil || $note);
unless (($list && (($banid && ! $an_opt) || (! $banid && $an_opt)) ||
($add && $what && $value) ||
($modify && $banid && $an_opt))) {
die "Usage: ljsysban.pl [opts]\n\n" .
" --list { <--banid=?> | or one of:\n" .
" [--what=? --status=? --bandate=datetime --banuntil=datetime\n" .
" --value=? --note=?]\n" .
" }\n\n" .
" --add <--what=? --value=?\n" .
" [--status=? --bandate=datetime { --banuntil=datetime | --banlength=duration } --note=?]>\n\n" .
" --modify <--banid=?>\n" .
" [--status=? --bandate=datetime { --banuntil=datetime |\n" .
" --banlength=duration } --value=? --note=?]\n\n" .
"datetime in format 'YYYY-MM-DD HH:MM:SS', duration in format 'N[dhms]' e.g. '5d' or '3h'.\n\n" .
"examples:\n" .
" ljsysban.pl --list --what=ip --value=127.0.0.1\n" .
" ljsysban.pl --add --what=email --value=test\@test.com --banuntil='2006-06-01 00:00:00' --note='test'\n" .
" ljsysban.pl --add --what=uniq --value=jd87fdnef8jf8jef --banlength=3d --note='3 day ban'\n\n";
}
# now load in the beast
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $dbh = LJ::get_db_writer();
# list bands
if ($list) {
my $where;
if ($banid) {
$where = "banid=" . $dbh->quote($banid);
} else {
my @where = ();
push @where, ("what=" . $dbh->quote($what)) if $what;
push @where, ("value=" . $dbh->quote($value)) if $value;
push @where, ("status=" . $dbh->quote($status)) if $status;
push @where, ("bandate=" . $dbh->quote($bandate)) if $bandate;
push @where, ("banuntil=" . $dbh->quote($banuntil)) if $banuntil;
push @where, ("note=" . $dbh->quote($note)) if $note;
$where = join(" AND ", @where);
}
my $sth = $dbh->prepare("SELECT * FROM sysban WHERE $where ORDER BY bandate ASC");
$sth->execute;
my $ct;
while (my $ban = $sth->fetchrow_hashref) {
print "> banid: $ban->{'banid'}, status: $ban->{'status'}, ";
print "bandate: " . ($ban->{'bandate'} ? $ban->{'bandate'} : "BOT") . ", ";
print "banuntil: " . ($ban->{'banuntil'} ? $ban->{'banuntil'} : "EOT") . "\n";
print "> what: $ban->{'what'}, value: $ban->{'value'}\n";
print "> note: $ban->{'note'}\n" if $ban->{'note'};
print "\n";
$ct++;
}
print "\n\tNO MATCHES\n\n" unless $ct;
exit;
}
# verify ban length and convert to banuntil as necessary
if ($banlength) {
die "--banlength must be of format N[dmhs] such as 3d, 5h, 60m, 35s.\n"
unless $banlength =~ /^(\d+)([dhms])$/i;
my ($num, $type) = ($1, lc $2);
$banlength = "DATE_ADD(NOW(), INTERVAL $num " .
{ 'd' => "DAY", 'h' => "HOUR", 'm' => "MINUTE", 's' => "SECOND" }->{$type} . ")";
$banuntil = $dbh->selectrow_array("SELECT $banlength");
die $dbh->errstr if $dbh->err;
}
# add new ban
if ($add) {
$status = ($status eq 'expired' ? 'expired' : 'active');
$dbh->do("INSERT INTO sysban (status, what, value, note, bandate, banuntil)" .
"VALUES (?, ?, ?, ?, " .
($bandate ? $dbh->quote($bandate) : 'NOW()') . ", " .
($banuntil ? $dbh->quote($banuntil) : 'NULL') . ")",
undef, $status, $what, $value, $note);
die $dbh->errstr if $dbh->err;
my $insertid = $dbh->{'mysql_insertid'};
if ($what eq 'ip') {
LJ::procnotify_add("ban_ip", { 'ip' => $value,
'exptime' => LJ::mysqldate_to_time($banuntil) });
LJ::MemCache::delete("sysban:ip");
}
if ($what eq 'uniq') {
LJ::procnotify_add("ban_uniq", { 'uniq' => $value,
'exptime' => LJ::mysqldate_to_time($banuntil) });
LJ::MemCache::delete("sysban:uniq");
}
if ($what eq 'contentflag') {
LJ::procnotify_add("ban_contentflag", { 'username' => $value,
'exptime' => LJ::mysqldate_to_time($banuntil) });
LJ::MemCache::delete("sysban:contentflag");
}
# log in statushistory
LJ::statushistory_add(0, 0, 'sysban_add',
"banid=$insertid; status=$status; " .
"bandate=" . ($bandate || LJ::mysql_time()) . "; " .
"banuntil=" . ($banuntil || 'NULL') . "; " .
"what=$what; value=$value; " .
"note=$note;");
print "CREATED: banid=$insertid\n";
exit;
}
# modify existing ban
if ($modify) {
# load selected ban
my $ban = $dbh->selectrow_hashref("SELECT * FROM sysban WHERE banid=?", undef, $banid);
die $dbh->errstr if $dbh->err;
my @set = ();
# ip/uniq ban and we're going to change the value
if (($value && $value ne $ban->{'value'}) ||
$banuntil && $banuntil ne $ban->{'banuntil'} ||
($status && $status ne $ban->{'status'} && $status eq 'expired')) {
if ($ban->{'what'} eq 'ip') {
LJ::procnotify_add("unban_ip", { 'ip' => $value || $ban->{'value'}});
LJ::MemCache::delete("sysban:ip");
}
if ($ban->{'what'} eq 'uniq') {
LJ::procnotify_add("unban_uniq", { 'uniq' => $value || $ban->{'value'} });
LJ::MemCache::delete("sysban:uniq");
}
if ($ban->{'what'} eq 'contentflag') {
LJ::procnotify_add("unban_contentflag", { 'username' => $value || $ban->{'value'} });
LJ::MemCache::delete("sysban:contentflag");
}
}
# what - must have a value
if ($what && $what ne $ban->{'what'}) {
$ban->{'what'} = $what;
push @set, "what=" . $dbh->quote($ban->{'what'});
}
# ip/uniq ban and we are going to change the value
if (($value && $value ne $ban->{'value'}) ||
$banuntil && $banuntil ne $ban->{'banuntil'} ||
($status && $status ne $ban->{'status'} && $status eq 'active')) {
my $new_banuntil = $banuntil || $ban->{'banuntil'};
if ($ban->{'what'} eq 'ip') {
LJ::procnotify_add("ban_ip", { 'ip' => $value || $ban->{'value'},
'exptime' => LJ::mysqldate_to_time($new_banuntil) });
LJ::MemCache::delete("sysban:ip");
}
if ($ban->{'what'} eq 'uniq') {
LJ::procnotify_add("ban_uniq", { 'uniq' => $value || $ban->{'value'},
'exptime' => LJ::mysqldate_to_time($new_banuntil) });
LJ::MemCache::delete("sysban:uniq");
}
if ($ban->{'what'} eq 'contentflag') {
LJ::procnotify_add("ban_contentflag", { 'username' => $value || $ban->{'value'},
'exptime' => LJ::mysqldate_to_time($new_banuntil) });
LJ::MemCache::delete("sysban:contentflag");
}
}
# value - must have a value
if ($value && $value ne $ban->{'value'}) {
$ban->{'value'} = $value;
push @set, "value=" . $dbh->quote($ban->{'value'});
}
# status - must have a value
if ($status && $status ne $ban->{'status'}) {
$ban->{'status'} = ($status eq 'expired' ? 'expired' : 'active');
push @set, "status=" . $dbh->quote($ban->{'status'});
}
# banuntil
if ($banuntil && $banuntil ne $ban->{'banuntil'}) {
$ban->{'banuntil'} = ($banuntil && $banuntil ne 'NULL') ? $banuntil : 0;
push @set, "banuntil=" . ($ban->{'banuntil'} ? $dbh->quote($ban->{'banuntil'}) : 'NULL');
}
# bandate - must have a value
if ($bandate && $bandate ne $ban->{'bandate'}) {
$ban->{'bandate'} = $bandate;
push @set, "bandate=" . $dbh->quote($ban->{'bandate'});
}
# note - can be changed to blank
if (defined $note && $note ne $ban->{'note'}) {
$ban->{'note'} = $note;
push @set, "note=" . $dbh->quote($ban->{'note'});
}
# do update
$dbh->do("UPDATE sysban SET " . join(", ", @set) . " WHERE banid=?", undef, $ban->{'banid'});
# log in statushistory
my $msg; map { $msg .= " " if $msg;
$msg .= "$_=$ban->{$_};" } qw(banid status bandate banuntil what value note);
LJ::statushistory_add(0, 0, 'sysban_mod', $msg);
print "MODIFIED: banid=$banid\n";
exit;
}

46
bin/ljtalkd Executable file
View File

@ -0,0 +1,46 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
use Danga::Socket;
use IO::Socket::INET;
require 'ljlib.pl';
die "This is currently for dev servers only.\n" unless $LJ::IS_DEV_SERVER;
die "No gearmand servers configured in \@LJ::GEARMAN_SERVERS.\n" unless @LJ::GEARMAN_SERVERS;
my $sock = IO::Socket::INET->new(PeerAddr => $LJ::GEARMAN_SERVERS[0])
or die "First gearmand server in \@LJ::GEARMAN_SERVERS ($LJ::GEARMAN_SERVERS[0]) isn't responding.\n";
print $sock "workers\r\n";
my $found_ljtalk = 0;
while (<$sock>) {
$found_ljtalk = 1 if (/ljtalk_auth_check/ && /ljtalk_bot_talk/);
last if /^\./;
}
my $worker_pid;
END {
if ($worker_pid) {
kill 9, $worker_pid;
print "killing worker pid $worker_pid\n";
}
}
unless ($found_ljtalk) {
$worker_pid = fork;
die "Fork error" unless defined $worker_pid;
if (!$worker_pid) {
exec("$LJ::HOME/bin/worker/ljtalk-gm");
}
print "Gearman worker ljtalk-gm started w/ pid $worker_pid.\n";
}
chdir("$LJ::HOME/cvs/djabberd/DJabberd") or die "failed to chdir";
print "Starting djabberd.\n";
$ENV{LJ_DOMAIN} = $LJ::DOMAIN;
system("perl",
"-I", "$ENV{LJHOME}/cgi-bin",
'-I', '../../gearman/api/perl/Gearman-Client-Async/lib/',
'-I', '../DJabberd-LiveJournal/lib',
'-I', 'lib',
'./djabberd', '--conf=../DJabberd-LiveJournal/conf/ljtalk.conf');

1151
bin/ljubackup.pl Executable file

File diff suppressed because it is too large Load Diff

1336
bin/ljumover.pl Executable file

File diff suppressed because it is too large Load Diff

3
bin/ljwhich Executable file
View File

@ -0,0 +1,3 @@
#!/bin/sh
$LJHOME/bin/cvsreport.pl --which $1

110
bin/logsummarize.pl Executable file
View File

@ -0,0 +1,110 @@
#!/usr/bin/perl
#
use strict;
unless (-d $ENV{'LJHOME'}) {
die "\$LJHOME not set.\n";
}
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $db = LJ::get_dbh("logs");
unless ($db) {
die "No 'logs' db handle found.\n";
}
$db->{'InactiveDestroy'} = 1;
my $sth;
my %table;
$sth = $db->prepare("SHOW TABLES LIKE 'access%'");
$sth->execute;
while (my $t = $sth->fetchrow_array) {
$table{$t} = 1;
}
my @need_summary = sort grep { /^access\d{8,8}$/ } keys %table;
pop @need_summary; # don't summarize the current day yet.
my $nsum_total = @need_summary;
my $nsum_ct = 0;
use constant F_SERVER => 0;
use constant F_LANG => 1;
use constant F_METHOD => 2;
use constant F_VHOST => 3;
use constant F_URI => 4;
use constant F_STATUS => 5;
use constant F_CTYPE => 6;
use constant F_BYTES => 7;
use constant F_BROWSER => 8;
use constant F_REF => 9;
foreach my $table (@need_summary)
{
$nsum_ct++;
print "Summarizing $table ($nsum_ct/$nsum_total)\n";
my $row_total = $db->selectrow_array("SELECT COUNT(*) FROM $table");
print " rows = $row_total\n";
my $sth = $db->prepare("SELECT server, langpref, method, vhost, uri, ".
" status, ctype, bytes, browser, ref ".
"FROM $table");
$sth->{'mysql_use_result'} = 1;
$sth->execute;
my ($r, $row_ct);
my %st;
while ($r = $sth->fetchrow_arrayref) {
$row_ct++;
if ($row_ct % 10000 == 0) {
printf " $row_ct/$row_total (%%%.02f)\n", 100*$row_ct/$row_total;
}
next if ($r->[F_URI] =~ m!^/userpic!);
$st{'count'}->{'total'}++;
$st{'count'}->{'bytes'} += $r->[F_BYTES];
$st{'http_meth'}->{$r->[F_METHOD]}++;
$st{'http_status'}->{$r->[F_STATUS]}++;
$st{'browser'}->{$r->[F_BROWSER]}++;
$st{'host'}->{$r->[F_VHOST]}++;
if ($r->[F_URI] =~ s!^/(users/|~)\w+/?!/users/*/!) {
$r->[F_URI] =~ s!day/\d\d\d\d/\d\d/\d\d!day!;
$r->[F_URI] =~ s!calendar/\d\d\d\d!calendar!;
}
if ($r->[F_VHOST] =~ /^(www\.)livejournal\.com$/) {
$st{'uri'}->{$r->[F_URI]}++;
} else {
$r->[F_URI] =~ s!day/\d\d\d\d/\d\d/\d\d!day!;
$r->[F_URI] =~ s!calendar/\d\d\d\d!calendar!;
$st{'uri'}->{"user:" . $r->[F_URI]}++;
}
my $ref = $r->[F_REF];
if ($ref =~ m!^http://([^/]+)!) {
$ref = $1;
$st{'referer'}->{$ref}++ unless ($ref =~ /livejournal\.com$/);
}
}
my $tabledate = $table;
$tabledate =~ s/^access//;
print " Writing stats file.\n";
open (S, "| gzip -c > $ENV{'LJHOME'}/var/stats-$tabledate.gz") or die "Can't open stats file";
foreach my $cat (sort keys %st) {
print "Writing cat: $cat\n";
foreach my $k (sort { $st{$cat}->{$b} <=> $st{$cat}->{$a} } keys %{$st{$cat}}) {
print S "$cat\t$k\t$st{$cat}->{$k}\n"
or die "Failed writing to stats-$tabledate.gz. Device full?\n";
}
}
close S;
$db->do("DROP TABLE $table");
}

384
bin/maint/captcha.pl Normal file
View File

@ -0,0 +1,384 @@
#!/usr/bin/perl
use strict;
use vars qw(%maint %maintinfo);
use LJ::Captcha::Generate;
use LJ::Blob qw{};
use File::Temp qw{tempdir};
use File::Path qw{rmtree};
use File::Spec qw{};
our ( $FakeUserId, $ClusterId, $Digits, $DigitCount,
$ExpireThresUser, $ExpireThresNoUser, $TmpRoot );
# Data for code-generation
$Digits = "abcdefghknpqrstuvxz23456789";
$DigitCount = length( $Digits );
# Maximum age of answered captchas. this is just
# for double-click protection.
$ExpireThresUser = 2 * 60; # two minutes
# 24 hours for captchas which were given out but not answered.
# (they might leave their browser window open or something)
$ExpireThresNoUser = 24 * 3600; # 1 day
# parent directory under which temporary files and directories
# should be created... anything placed in this directory will
# be automatically cleaned
$TmpRoot = "/tmp";
#####################################################################
### F U N C T I O N S
#####################################################################
### Read a file in as a scalar and return it
sub readfile ($) {
my ( $filename ) = @_;
open my $fh, "<$filename" or die "open: $filename: $!";
local $/ = undef;
my $data = <$fh>;
return $data;
}
### Generate an n-character challenge code
sub gencode ($) {
my ( $digits ) = @_;
my $code = '';
for ( 1..$digits ) {
$code .= substr( $Digits, int(rand($DigitCount)), 1 );
}
return $code;
}
#####################################################################
### M A I N T E N A N C E T A S K S
#####################################################################
$maintinfo{gen_audio_captchas}{opts}{locking} = "per_host";
$maint{gen_audio_captchas} = sub {
my (
$u, # Fake user record for Blob::put
$dbh, # Database handle (writer)
$count, # Count of currently-extant audio challenges
$need, # How many we need to still create
$make, # how many we're actually going to create this round
$tmpdir, # Temporary working directory
$code, # The generated challenge code
$wav, # Wav file
$data, # Wav file data
$err, # Error-message ref for Blob::put calls
$capid, # Captcha row id
$anum, # Deseries-ifier value
);
print "-I- Generating new audio captchas...\n";
# fail if we're not doing uploads right now
die "Unable to generate captchas: media uploads disabled.\n"
if $LJ::DISABLE_MEDIA_UPLOADS;
$dbh = LJ::get_dbh({raw=>1}, "master") or die "Failed to get_db_writer()";
$dbh->do("SET wait_timeout=28800");
# Count how many challenges there are currently
$count = $dbh->selectrow_array(q{
SELECT COUNT(*)
FROM captchas
WHERE
type = 'audio'
AND issuetime = 0
});
my $MaxItems = $LJ::CAPTCHA_AUDIO_PREGEN || 500;
# If there are enough, don't generate any more
print "Current count is $count of $MaxItems...";
if ( $count >= $MaxItems ) {
print "already have enough.\n";
return;
} else {
$make = $need = $MaxItems - $count;
$make = $LJ::CAPTCHA_AUDIO_MAKE
if defined $LJ::CAPTCHA_AUDIO_MAKE && $make > $LJ::CAPTCHA_AUDIO_MAKE;
print "generating $make new audio challenges.\n";
}
# Clean up any old audio directories lying about from failed generations
# before. In theory, File::Temp::tempdir() is supposed to clean them up
# itself, but it doesn't appear to be doing so.
foreach my $olddir ( glob "$TmpRoot/audio_captchas_*" ) {
# If it's been more than an hour since it's been changed from the
# starting time of the script, kill it
if ( (-M $olddir) * 24 > 1 ) {
print "cleaning up old working temp directory ($olddir).\n";
rmtree( $olddir ) or die "rmtree: $olddir: $!";
}
}
# Load the system user for Blob::put() and create an auto-cleaning temp
# directory for audio generation
$u = LJ::load_user( "system" )
or die "Couldn't load the system user.";
$tmpdir = tempdir( "audio_captchas_XXXXXX", CLEANUP => 0, DIR => $TmpRoot );
# target location
my $location = $LJ::CAPTCHA_MOGILEFS ? 'mogile' : 'blob';
# Generate the challenges
for ( my $i = 0; $i < $make; $i++ ) {
print "Generating audio $i...";
( $wav, $code ) = LJ::Captcha::Generate->generate_audio( $tmpdir );
$data = readfile( $wav );
unlink $wav or die "unlink: $wav: $!";
# Generate the capid + anum
print "generating new capid/anum...";
$capid = LJ::alloc_global_counter( 'C' );
die "Couldn't allocate capid" unless $capid;
$anum = int( rand 65_535 );
# Insert the blob
print "uploading (capid = $capid, anum = $anum)...";
if ($location eq 'mogile') {
my $mogfs = LJ::mogclient(); # force load
die "Requested to store captchas on MogileFS, but it's not loaded.\n"
unless $mogfs;
my $fh = $mogfs->new_file("captcha:$capid", 'captcha')
or die("Unable to contact MogileFS server for storage: " .
$mogfs->last_tracker . ": ".
$mogfs->errstr . "\n");
$fh->print($data);
$fh->close
or die "Unable to save captcha to MogileFS server: $@\n";
} else {
LJ::Blob::put( $u, 'captcha_audio', 'wav', $capid, $data, \$err )
or die "Error uploading to media server: $err";
}
# Insert the captcha into the DB. If it fails for some reason, delete
# the just-uploaded file from the media storage system too.
print "inserting (code = $code)...";
my $rval = eval {
$dbh->do(q{
INSERT INTO captchas( capid, type, location, answer, anum )
VALUES ( ?, 'audio', ?, ?, ? )
}, undef, $capid, $location, $code, $anum);
};
if ( !$rval || $@ ) {
my $err = $@ || $dbh->errstr;
if ( $location eq 'mogile' ) {
LJ::mogclient()->delete( "captcha:$capid" );
} else {
LJ::Blob::delete( $u, 'captcha_audio', 'wav', $capid );
}
die "audio captcha insert error on ($capid, $location, $code, $anum): $err";
}
print "done.\n";
}
print "cleaning up working temporary directory ($tmpdir).\n";
rmtree( $tmpdir ) or die "Failed directory cleanup: $!";
print "done. Created $make new audio captchas.\n";
return 1;
};
$maintinfo{gen_image_captchas}{opts}{locking} = "per_host";
$maint{gen_image_captchas} = sub {
my (
$u, # Fake user record for Blob::put
$dbh, # Database handle (writer)
$count, # Count of currently-extant audio challenges
$need, # How many we need to still create
$code, # The generated challenge code
$png, # PNG data
$err, # Error-message ref for Blob::put calls
$capid, # Captcha row id
$anum, # Deseries-ifier value
);
print "-I- Generating new image captchas...\n";
# fail if we're not doing uploads right now
die "Unable to generate captchas: media uploads disabled.\n"
if $LJ::DISABLE_MEDIA_UPLOADS;
$dbh = LJ::get_dbh({raw=>1}, "master") or die "Failed to get_db_writer()";
$dbh->do("SET wait_timeout=28800");
# Count how many challenges there are currently
$count = $dbh->selectrow_array(q{
SELECT COUNT(*)
FROM captchas
WHERE
type = 'image'
AND issuetime = 0
});
my $MaxItems = $LJ::CAPTCHA_IMAGE_PREGEN || 1000;
# If there are enough, don't generate any more
print "Current count is $count of $MaxItems...";
if ( $count >= $MaxItems ) {
print "already have enough.\n";
return;
} else {
$need = $MaxItems - $count;
print "generating $need new image challenges.\n";
}
# Load system user for Blob::put()
$u = LJ::load_user( "system" )
or die "Couldn't load the system user.";
$dbh = LJ::get_db_writer() or die "Failed to get_db_writer()";
# target location
my $location = $LJ::CAPTCHA_MOGILEFS ? 'mogile' : 'blob';
# Generate the challenges
for ( my $i = 0; $i < $need; $i++ ) {
print "Generating image $i...";
$code = gencode( 7 );
( $png ) = LJ::Captcha::Generate->generate_visual( $code );
# Generate the capid + anum
print "generating new capid/anum...";
$capid = LJ::alloc_global_counter( 'C' );
die "Couldn't allocate capid" unless $capid;
$anum = int( rand 65_535 );
# Insert the blob
print "uploading (capid = $capid, anum = $anum)...";
if ($location eq 'mogile') {
my $mogfs = LJ::mogclient(); # force load
die "Requested to store captchas on MogileFS, but it's not loaded.\n"
unless $mogfs;
my $fh = $mogfs->new_file("captcha:$capid", 'captcha')
or die("Unable to contact MogileFS server for storage: " .
$mogfs->last_tracker . ": ".
$mogfs->errstr . "\n");
$fh->print($png);
$fh->close
or die "Unable to save captcha to MogileFS server: $@\n";
} else {
LJ::Blob::put( $u, 'captcha_image', 'png', $capid, $png, \$err )
or die "Error uploading to media server: $err";
}
# Insert the captcha into the DB. If it fails for some reason, delete
# the just-uploaded file from the media storage system too.
print "inserting (code = $code)...";
my $rval = eval {
$dbh->do(q{
INSERT INTO captchas( capid, type, location, answer, anum )
VALUES ( ?, 'image', ?, ?, ? )
}, undef, $capid, $location, $code, $anum);
};
if ( !$rval || $@ ) {
my $err = $@ || $dbh->errstr;
if ( $location eq 'mogile' ) {
LJ::mogclient()->delete( "captcha:$capid" );
} else {
LJ::Blob::delete( $u, 'captcha_image', 'png', $capid );
}
die "image captcha insert error on ($capid, $location, $code, $anum): $err";
}
print "done.\n";
}
print "done. Created $need new image captchas.\n";
return 1;
};
$maint{clean_captchas} = sub {
my (
$u, # System user
$expired, # arrayref of arrayrefs of expired captchas
$dbh, # Database handle (writer)
$sql, # SQL statement
$sth, # Statement handle
$count, # Deletion count
$err, # Error message reference for Blob::delete calls
);
print "-I- Cleaning captchas.\n";
# fail if we're not doing uploads right now
die "Unable to clean captchas: media uploads disabled.\n"
if $LJ::DISABLE_MEDIA_UPLOADS;
# Find captchas to delete
$sql = q{
SELECT
capid, type, location
FROM captchas
WHERE
( issuetime <> 0 AND issuetime < ? )
OR
( userid > 0
AND ( issuetime <> 0 AND issuetime < ? )
)
LIMIT 2500
};
$dbh = LJ::get_db_writer()
or die "No master DB handle";
$expired = $dbh->selectall_arrayref( $sql, undef,
time() - $ExpireThresNoUser,
time() - $ExpireThresUser );
die "selectall_arrayref: $sql: ", $dbh->errstr if $dbh->err;
if ( @$expired ) {
print "found ", scalar @$expired, " captchas to delete...\n";
} else {
print "Done: No captchas to delete.\n";
return;
}
# Prepare deletion statement
$sql = q{ DELETE FROM captchas WHERE capid = ? };
$sth = $dbh->prepare( $sql );
# Fetch system user
$u = LJ::load_user( "system" )
or die "Couldn't load the system user.";
# Now delete each one from the DB and the media server
foreach my $captcha ( @$expired ) {
my ( $capid, $type, $location ) = @$captcha;
$location ||= 'blob';
print "Deleting captcha $capid ($type, $location)\n";
my $ext = $type eq 'audio' ? 'wav' : 'png';
if ($location eq 'mogile') {
my $mogfs = LJ::mogclient(); # force load
die "Requested to delete captchas from MogileFS, but it's not loaded.\n"
unless $mogfs;
$mogfs->delete("captcha:$capid")
or die "Unable to delete captcha from MogileFS server for capid = $capid.\n";
} else {
LJ::Blob::delete( $u, "captcha_$type", $ext, $capid, \$err )
or die "Failed to delete $type file from media server for ".
"capid = $capid: $err";
}
$sth->execute( $capid )
or die "execute: $sql ($capid): ", $sth->errstr;
$count++;
}
print "Done: deleted $count expired captchas.\n";
return 1;
};

377
bin/maint/clean_caches.pl Normal file
View File

@ -0,0 +1,377 @@
#!/usr/bin/perl
#
$maint{'clean_caches'} = sub
{
my $dbh = LJ::get_db_writer();
my $sth;
my $verbose = $LJ::LJMAINT_VERBOSE;
print "-I- Cleaning authactions.\n";
$dbh->do("DELETE FROM authactions WHERE datecreate < DATE_SUB(NOW(), INTERVAL 30 DAY)");
print "-I- Cleaning faquses.\n";
$dbh->do("DELETE FROM faquses WHERE dateview < DATE_SUB(NOW(), INTERVAL 7 DAY)");
print "-I- Cleaning duplock.\n";
$dbh->do("DELETE FROM duplock WHERE instime < DATE_SUB(NOW(), INTERVAL 1 HOUR)");
print "-I- Cleaning commenturl.\n";
$dbh->do("DELETE FROM commenturls WHERE timecreate < UNIX_TIMESTAMP() - 86400*30 LIMIT 50000");
if ($LJ::COPPA_CHECK && $LJ::UNIQ_COOKIES) {
print "-I- Cleaning underage uniqs.\n";
$dbh->do("DELETE FROM underage WHERE timeof < (UNIX_TIMESTAMP() - 86400*90) LIMIT 2000");
}
print "-I- Cleaning captcha sessions.\n";
foreach my $c (@LJ::CLUSTERS) {
my $dbcm = LJ::get_cluster_master($c);
next unless $dbcm;
$dbcm->do("DELETE FROM captcha_session WHERE sesstime < UNIX_TIMESTAMP()-86400");
}
print "-I- Cleaning blobcache.\n";
$dbh->do("DELETE FROM blobcache WHERE dateupdate < NOW() - INTERVAL 30 DAY");
print "-I- Cleaning old anonymous comment IP logs.\n";
my $count;
foreach my $c (@LJ::CLUSTERS) {
my $dbcm = LJ::get_cluster_master($c);
next unless $dbcm;
# 432,000 seconds is 5 days
$count += $dbcm->do('DELETE FROM tempanonips WHERE reporttime < (UNIX_TIMESTAMP() - 432000)');
}
print " deleted $count\n";
print "-I- Cleaning old random users.\n";
my $count;
foreach my $c (@LJ::CLUSTERS) {
my $dbcm = LJ::get_cluster_master($c);
next unless $dbcm;
my $secs = $LJ::RANDOM_USER_PERIOD * 24 * 60 * 60;
while (my $deleted = $dbcm->do("DELETE FROM random_user_set WHERE posttime < (UNIX_TIMESTAMP() - $secs) LIMIT 1000")) {
$count += $deleted;
last if $deleted != 1000;
sleep 10;
}
}
print " deleted $count\n";
print "-I- Cleaning diresearchres.\n";
# need insert before delete so master logs delete and slaves actually do it
$dbh->do("INSERT INTO dirsearchres2 VALUES (MD5(NOW()), DATE_SUB(NOW(), INTERVAL 31 MINUTE), '')");
$dbh->do("DELETE FROM dirsearchres2 WHERE dateins < DATE_SUB(NOW(), INTERVAL 30 MINUTE)");
# clean incoming emails older than 7 days from Mogile...
my $mogc = LJ::mogclient();
if ($mogc) {
print "-I- Cleaning incoming email temporary handles.\n";
$sth = $dbh->prepare("SELECT ieid FROM incoming_email_handle WHERE timerecv < UNIX_TIMESTAMP() - 86400*7 LIMIT 10000");
$sth->execute;
while (my ($id) = $sth->fetchrow_array) {
if ($mogc->delete("ie:$id")) {
$dbh->do("DELETE FROM incoming_email_handle WHERE ieid=?", undef, $id);
}
}
}
print "-I- Cleaning meme.\n";
do {
$sth = $dbh->prepare("DELETE FROM meme WHERE ts < DATE_SUB(NOW(), INTERVAL 7 DAY) LIMIT 250");
$sth->execute;
if ($dbh->err) { print $dbh->errstr; }
print " deleted ", $sth->rows, "\n";
} while ($sth->rows && ! $sth->err);
print "-I- Cleaning old pending comments.\n";
$count = 0;
foreach my $c (@LJ::CLUSTERS) {
my $dbcm = LJ::get_cluster_master($c);
next unless $dbcm;
# 3600 seconds is one hour
my $time = time() - 3600;
$count += $dbcm->do('DELETE FROM pendcomments WHERE datesubmit < ? LIMIT 2000', undef, $time);
}
print " deleted $count\n";
# move rows from talkleft_xfp to talkleft
print "-I- Moving talkleft_xfp.\n";
my $xfp_count = $dbh->selectrow_array("SELECT COUNT(*) FROM talkleft_xfp");
print " rows found: $xfp_count\n";
if ($xfp_count) {
my @xfp_cols = qw(userid posttime journalid nodetype nodeid jtalkid publicitem);
my $xfp_cols = join(",", @xfp_cols);
my $xfp_cols_join = join(",", map { "t.$_" } @xfp_cols);
my %insert_vals;
my %delete_vals;
# select out 1000 rows from random clusters
$sth = $dbh->prepare("SELECT u.clusterid,u.user,$xfp_cols_join " .
"FROM talkleft_xfp t, user u " .
"WHERE t.userid=u.userid LIMIT 1000");
$sth->execute();
my $row_ct = 0;
while (my $row = $sth->fetchrow_hashref) {
my %qrow = map { $_, $dbh->quote($row->{$_}) } @xfp_cols;
push @{$insert_vals{$row->{'clusterid'}}},
("(" . join(",", map { $qrow{$_} } @xfp_cols) . ")");
push @{$delete_vals{$row->{'clusterid'}}},
("(userid=$qrow{'userid'} AND " .
"journalid=$qrow{'journalid'} AND " .
"nodetype=$qrow{'nodetype'} AND " .
"nodeid=$qrow{'nodeid'} AND " .
"posttime=$qrow{'posttime'} AND " .
"jtalkid=$qrow{'jtalkid'})");
$row_ct++;
}
foreach my $clusterid (sort keys %insert_vals) {
my $dbcm = LJ::get_cluster_master($clusterid);
unless ($dbcm) {
print " cluster down: $clusterid\n";
next;
}
print " cluster $clusterid: " . scalar(@{$insert_vals{$clusterid}}) .
" rows\n" if $verbose;
$dbcm->do("INSERT INTO talkleft ($xfp_cols) VALUES " .
join(",", @{$insert_vals{$clusterid}})) . "\n";
if ($dbcm->err) {
print " db error (insert): " . $dbcm->errstr . "\n";
next;
}
# no error, delete from _xfp
$dbh->do("DELETE FROM talkleft_xfp WHERE " .
join(" OR ", @{$delete_vals{$clusterid}})) . "\n";
if ($dbh->err) {
print " db error (delete): " . $dbh->errstr . "\n";
next;
}
}
print " rows remaining: " . ($xfp_count - $row_ct) . "\n";
}
# move clustered active_user stats from each cluster to the global active_user_summary table
print "-I- Migrating active_user records.\n";
$count = 0;
foreach my $cid (@LJ::CLUSTERS) {
next unless $cid;
my $dbcm = LJ::get_cluster_master($cid);
unless ($dbcm) {
print " cluster down: $clusterid\n";
next;
}
unless ($dbcm->do("LOCK TABLES active_user WRITE")) {
print " db error (lock): " . $dbcm->errstr . "\n";
next;
}
# We always want to keep at least an hour worth of data in the
# clustered table for duplicate checking. We won't select out
# any rows for this hour or the full hour before in order to avoid
# extra rows counted in hour-boundary edge cases
my $now = time();
# one hour from the start of this hour (
my $before_time = $now - 3600 - ($now % 3600);
my $time_str = LJ::mysql_time($before_time, 'gmt');
# now extract parts from the modified time
my ($yr, $mo, $day, $hr) =
$time_str =~ /^(\d\d\d\d)-(\d\d)-(\d\d) (\d\d)/;
# Building up all this sql is pretty ghetto but otherwise it
# becomes unwieldy with tons of code duplication and more places
# for this fairly-complicated where condition to break. So we'll
# build a nice where clause which uses bind vars and then create
# an array to go inline in the spot where those bind vars should
# be within the larger query
my $where = "WHERE year=? AND month=? AND day=? AND hour<? OR " .
"year=? AND month=? AND day<? OR " .
"year=? AND month<? OR " .
"year<?";
my @where_vals = ($yr, $mo, $day, $hr,
$yr, $mo, $day,
$yr, $mo,
$yr );
# don't need to check for distinct userid in the count here
# because y,m,d,h,uid is the primary key so we know it's
# unique for this hour anyway
my $sth = $dbcm->prepare
("SELECT type, year, month, day, hour, COUNT(userid) " .
"FROM active_user $where GROUP BY 1,2,3,4,5");
$sth->execute(@where_vals);
if ($dbcm->err) {
print " db error (select): " . $dbcm->errstr . "\n";
next;
}
my %counts = ();
my $total_ct = 0;
while (my ($type, $yr, $mo, $day, $hr, $ct) = $sth->fetchrow_array) {
$counts{"$yr-$mo-$day-$hr-$type"} += $ct;
$total_ct += $ct;
}
print " cluster $cid: $total_ct rows\n" if $verbose;
# Note: We can experience failures on both sides of this
# transaction. Either our delete can succeed then
# insert fail or vice versa. Luckily this data is
# for statistical purposes so we can just live with
# the possibility of a small skew.
unless ($dbcm->do("DELETE FROM active_user $where", undef, @where_vals)) {
print " db error (delete): " . $dbcm->errstr . "\n";
next;
}
# at this point if there is an error we will ignore it and try
# to insert the count data above anyway
my $rv = $dbcm->do("UNLOCK TABLES")
or print " db error (unlock): " . $dbcm->errstr . "\n";
# nothing to insert, why bother?
next unless %counts;
# insert summary into active_user_summary table
my @bind = ();
my @vals = ();
while (my ($hkey, $ct) = each %counts) {
# yyyy, mm, dd, hh, cid, type, ct
push @bind, "(?, ?, ?, ?, ?, ?, ?)";
my ($yr, $mo, $day, $hr, $type) = split(/-/, $hkey);
push @vals, ($yr, $mo, $day, $hr, $cid, $type, $ct);
}
my $bind = join(",", @bind);
$dbh->do("INSERT IGNORE INTO active_user_summary (year, month, day, hour, clusterid, type, count) " .
"VALUES $bind", undef, @vals);
if ($dbh->err) {
print " db error (insert): " . $dbh->errstr . "\n";
# something's badly b0rked, don't try any other clusters for now
last;
}
# next cluster
}
# move clustered recentaction summaries from their respective clusters
# to the global actionhistory table
print "-I- Migrating recentactions.\n";
foreach my $cid (@LJ::CLUSTERS) {
next unless $cid;
my $dbcm = LJ::get_cluster_master($cid);
unless ($dbcm) {
print " cluster down: $clusterid\n";
next;
}
unless ($dbcm->do("LOCK TABLES recentactions WRITE")) {
print " db error (lock): " . $dbcm->errstr . "\n";
next;
}
my $sth = $dbcm->prepare
("SELECT what, COUNT(*) FROM recentactions GROUP BY 1");
$sth->execute;
if ($dbcm->err) {
print " db error (select): " . $dbcm->errstr . "\n";
next;
}
my %counts = ();
my $total_ct = 0;
while (my ($what, $ct) = $sth->fetchrow_array) {
$counts{$what} += $ct;
$total_ct += $ct;
}
print " cluster $cid: $total_ct rows\n" if $verbose;
# Note: We can experience failures on both sides of this
# transaction. Either our delete can succeed then
# insert fail or vice versa. Luckily this data is
# for statistical purposes so we can just live with
# the possibility of a small skew.
unless ($dbcm->do("DELETE FROM recentactions")) {
print " db error (delete): " . $dbcm->errstr . "\n";
next;
}
# at this point if there is an error we will ignore it and try
# to insert the count data above anyway
$dbcm->do("UNLOCK TABLES")
or print " db error (unlock): " . $dbcm->errstr . "\n";
# nothing to insert, why bother?
next unless %counts;
# TEMPORARY
# We want to move from using one letter, or prefixed with _ to
# mean local, to actual readable strings. Instead of fighting
# a race when modifying the recentactions table, do it here instead.
# This could should be removable after the code is pushed live and this
# job has run.
# David (1/11/06);
my %whatmap = (
'P' => 'post',
'post' => 'post',
'_F' => 'phonepost',
'phonepost' => 'phonepost',
'_M' => 'phonepost_mp3',
'phonepost_mp3' => 'phonepost_mp3',
);
# insert summary into global actionhistory table
my @bind = ();
my @vals = ();
while (my ($what, $ct) = each %counts) {
push @bind, "(UNIX_TIMESTAMP(),?,?,?)";
# TEMPORARY
my $cwhat = defined $whatmap{$what} ? $whatmap{$what} : $what;
push @vals, $cid, $cwhat, $ct;
}
my $bind = join(",", @bind);
$dbh->do("INSERT INTO actionhistory (time, clusterid, what, count) " .
"VALUES $bind", undef, @vals);
if ($dbh->err) {
print " db error (insert): " . $dbh->errstr . "\n";
# something's badly b0rked, don't try any other clusters for now
last;
}
# next cluster
}
};
1;

89
bin/maint/generic.pl Normal file
View File

@ -0,0 +1,89 @@
#!/usr/bin/perl
#
$maint{joinmail} = sub {
# this needs to be resumeable, so that it can run once every 10 or 15 minutes to digest things
# that are a day old but haven't been sent. also, the first query down there needs to include
# the right authaction type in the WHERE clause, and NOT do a GROUP BY.
print "Returning without running... I'm disabled right now.\n";
return 1;
my $dbr = LJ::get_db_reader();
# get all information
my $pending = $dbr->selectall_arrayref("SELECT userid, COUNT(arg1) FROM authactions " .
"WHERE used = 'N' AND datecreate > DATE_SUB(NOW(), INTERVAL 1 DAY)" .
"GROUP BY userid") || [];
# get userids of communities
my @commids;
push @commids, $_->[0]+0 foreach @$pending;
my $cus = LJ::load_userids(@commids);
# now let's get the maintainers of these
my $in = join ',', @commids;
my $maintrows = $dbr->selectall_arrayref("SELECT userid, targetid FROM reluser WHERE userid IN ($in) AND type = 'A'") || [];
my @maintids;
my %maints;
foreach (@$maintrows) {
push @{$maints{$_->[0]}}, $_->[1];
push @maintids, $_->[1];
}
my $mus = LJ::load_userids(@maintids);
# tell the maintainers that they got new people.
foreach my $row (@$pending) {
my $cuser = $cus->{$row->[0]}{user};
print "$cuser: $row->[1] invites: ";
my %email; # see who we emailed on this comm
foreach my $mid (@{$maints{$row->[0]}}) {
print "$mid ";
next if $email{$mus->{$mid}{email}}++;
LJ::load_user_props($mus->{$mid}, 'opt_communityjoinemail');
next unless $mus->{$mid}{opt_communityjoinemail} eq 'D'; # Daily or Digest
my $body = "Dear $mus->{$mid}{user},\n\n" .
"Over the past day or so, $row->[1] request(s) to join the \"$cuser\" community have " .
"been received. To look at the currently pending membership requests, please visit the pending " .
"membership page:\n\n" .
"\t$LJ::SITEROOT/community/pending.bml?comm=$cuser\n\n" .
"You may also ignore this email. Outstanding requests to join will expire after a period of 30 days.\n\n" .
"If you wish to no longer receive these emails, visit the community management page and " .
"set the relevant options:\n\n\t$LJ::SITEROOT/community/manage.bml\n\n" .
"Regards,\n$LJ::SITENAME Team\n";
LJ::send_mail({
to => $mus->{$mid}{email},
from => $LJ::COMMUNITY_EMAIL,
fromname => $LJ::SITENAME,
charset => 'utf-8',
subject => "$cuser Membership Requests",
body => $body,
wrap => 76,
});
}
print "\n";
}
};
$maint{clean_spamreports} = sub {
my $dbh = LJ::get_db_writer();
my ($len, $ct);
print "-I- Deleting old spam reports.\n";
$len = 86400 * 90; # 90 days
$ct = $dbh->do("DELETE FROM spamreports WHERE reporttime < UNIX_TIMESTAMP() - $len")+0;
print " Deleted $ct reports.\n";
if ($LJ::CLOSE_OLD_SPAMREPORTS) {
print "-I- Closing stale spam reports.\n";
$len = $LJ::CLOSE_OLD_SPAMREPORTS * 86400;
$ct = $dbh->do("UPDATE spamreports SET state='closed' "
. "WHERE state = 'open' AND reporttime < UNIX_TIMESTAMP() - $len")+0;
print " Closed $ct reports.\n";
}
};
1;

566
bin/maint/stats.pl Normal file
View File

@ -0,0 +1,566 @@
#!/usr/bin/perl
#
use strict;
use vars qw(%maint);
require "$ENV{'LJHOME'}/cgi-bin/statslib.pl";
# filled in by ljmaint.pl, 0=quiet, 1=normal, 2=verbose
$LJ::Stats::VERBOSE = $LJ::LJMAINT_VERBOSE >= 2 ? 1 : 0;
$maint{'genstats'} = sub
{
my @which = @_ || qw(users countries
states gender clients
pop_interests meme popfaq
schools);
# popular faq items
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "popfaq",
'statname' => "pop_faq",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
my $sth = $db->prepare("SELECT faqid, COUNT(*) FROM faquses WHERE " .
"faqid<>0 GROUP BY 1 ORDER BY 2 DESC LIMIT 50");
$sth->execute;
die $db->errstr if $db->err;
my %ret;
while (my ($id, $count) = $sth->fetchrow_array) {
$ret{$id} = $count;
}
return \%ret;
},
});
# popular interests
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "pop_interests",
'statname' => "pop_interests",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
return {} if $LJ::DISABLED{'interests-popular'};
# see what the previous min was, then subtract 20% of max from it
my ($prev_min, $prev_max) = $db->selectrow_array("SELECT MIN(statval), MAX(statval) " .
"FROM stats WHERE statcat='pop_interests'");
my $stat_min = int($prev_min - (0.2*$prev_max));
$stat_min = 1 if $stat_min < 1;
my $sth = $db->prepare("SELECT interest, intcount FROM interests WHERE intcount>? " .
"ORDER BY intcount DESC, interest ASC LIMIT 400");
$sth->execute($stat_min);
die $db->errstr if $db->err;
my %ret;
while (my ($int, $count) = $sth->fetchrow_array) {
$ret{$int} = $count;
}
return \%ret;
},
});
# popular memes
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "meme",
'statname' => "popmeme",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
return {} if $LJ::DISABLED{'meme'};
my $sth = $db->prepare("SELECT url, count(*) FROM meme " .
"GROUP BY 1 ORDER BY 2 DESC LIMIT 100");
$sth->execute;
die $db->errstr if $db->err;
my %ret;
while (my ($url, $count) = $sth->fetchrow_array) {
$ret{$url} = $count;
}
return \%ret;
},
});
# clients
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "clients",
'statname' => "client",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
return {} if $LJ::DISABLED{'clientversionlog'};
my $usertotal = $db->selectrow_array("SELECT MAX(userid) FROM user");
my $blocks = LJ::Stats::num_blocks($usertotal);
my %ret;
foreach my $block (1..$blocks) {
my ($low, $high) = LJ::Stats::get_block_bounds($block);
$db = $db_getter->(); # revalidate connection
my $sth = $db->prepare("SELECT c.client, COUNT(*) AS 'count' FROM clients c, clientusage cu " .
"WHERE c.clientid=cu.clientid AND cu.userid BETWEEN $low AND $high " .
"AND cu.lastlogin > DATE_SUB(NOW(), INTERVAL 30 DAY) GROUP BY 1 ORDER BY 2");
$sth->execute;
die $db->errstr if $db->err;
while ($_ = $sth->fetchrow_hashref) {
$ret{$_->{'client'}} += $_->{'count'};
}
print LJ::Stats::block_status_line($block, $blocks);
}
return \%ret;
},
});
# user table analysis
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "users",
'statname' => ["account", "newbyday", "age", "userinfo"],
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
my $usertotal = $db->selectrow_array("SELECT MAX(userid) FROM user");
my $blocks = LJ::Stats::num_blocks($usertotal);
my %ret; # return hash, (statname => { arg => val } since 'statname' is arrayref above
# iterate over user table in batches
foreach my $block (1..$blocks) {
my ($low, $high) = LJ::Stats::get_block_bounds($block);
# user query: gets user,caps,age,status,allow_getljnews
$db = $db_getter->(); # revalidate connection
my $sth = $db->prepare
("SELECT user, caps, " .
"FLOOR((TO_DAYS(NOW())-TO_DAYS(bdate))/365.25) AS 'age', " .
"status, allow_getljnews " .
"FROM user WHERE userid BETWEEN $low AND $high");
$sth->execute;
die $db->errstr if $db->err;
while (my $rec = $sth->fetchrow_hashref) {
# account types
my $capnameshort = LJ::name_caps_short($rec->{'caps'});
$ret{'account'}->{$capnameshort}++;
# ages
$ret{'age'}->{$rec->{'age'}}++
if $rec->{'age'} > 4 && $rec->{'age'} < 110;
# users receiving news emails
$ret{'userinfo'}->{'allow_getljnews'}++
if $rec->{'status'} eq "A" && $rec->{'allow_getljnews'} eq "Y";
}
# userusage query: gets timeupdate,datereg,nowdate
my $sth = $db->prepare
("SELECT DATE_FORMAT(timecreate, '%Y-%m-%d') AS 'datereg', " .
"DATE_FORMAT(NOW(), '%Y-%m-%d') AS 'nowdate', " .
"UNIX_TIMESTAMP(timeupdate) AS 'timeupdate' " .
"FROM userusage WHERE userid BETWEEN $low AND $high");
$sth->execute;
die $db->errstr if $db->err;
while (my $rec = $sth->fetchrow_hashref) {
# date registered
$ret{'newbyday'}->{$rec->{'datereg'}}++
unless $rec->{'datereg'} eq $rec->{'nowdate'};
# total user/activity counts
$ret{'userinfo'}->{'total'}++;
if (my $time = $rec->{'timeupdate'}) {
my $now = time();
$ret{'userinfo'}->{'updated'}++;
$ret{'userinfo'}->{'updated_last30'}++ if $time > $now-60*60*24*30;
$ret{'userinfo'}->{'updated_last7'}++ if $time > $now-60*60*24*7;
$ret{'userinfo'}->{'updated_last1'}++ if $time > $now-60*60*24*1;
}
}
print LJ::Stats::block_status_line($block, $blocks);
}
return \%ret;
},
});
LJ::Stats::register_stat
({ 'type' => "clustered",
'jobname' => "countries",
'statname' => "country",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
my $cid = shift;
return undef unless $db && $cid;
my $upc = LJ::get_prop("user", "country");
die "Can't find country userprop. Database populated?\n" unless $upc;
my $usertotal = $db->selectrow_array("SELECT MAX(userid) FROM userproplite2");
my $blocks = LJ::Stats::num_blocks($usertotal);
my %ret;
foreach my $block (1..$blocks) {
my ($low, $high) = LJ::Stats::get_block_bounds($block);
$db = $db_getter->(); # revalidate connection
my $sth = $db->prepare("SELECT u.value, COUNT(*) AS 'count' FROM userproplite2 u " .
"LEFT JOIN clustertrack2 c ON u.userid=c.userid " .
"WHERE u.upropid=? AND u.value<>'' AND u.userid=c.userid " .
"AND u.userid BETWEEN $low AND $high " .
"AND (c.clusterid IS NULL OR c.clusterid=?)" .
"GROUP BY 1 ORDER BY 2");
$sth->execute($upc->{'id'}, $cid);
die "clusterid: $cid, " . $db->errstr if $db->err;
while ($_ = $sth->fetchrow_hashref) {
$ret{$_->{'value'}} += $_->{'count'};
}
print LJ::Stats::block_status_line($block, $blocks);
}
return \%ret;
},
});
LJ::Stats::register_stat
({ 'type' => "clustered",
'jobname' => "states",
'statname' => "stateus",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
my $cid = shift;
return undef unless $db && $cid;
my $upc = LJ::get_prop("user", "country");
die "Can't find country userprop. Database populated?\n" unless $upc;
my $ups = LJ::get_prop("user", "state");
die "Can't find state userprop. Database populated?\n" unless $ups;
my $usertotal = $db->selectrow_array("SELECT MAX(userid) FROM userproplite2");
my $blocks = LJ::Stats::num_blocks($usertotal);
my %ret;
foreach my $block (1..$blocks) {
my ($low, $high) = LJ::Stats::get_block_bounds($block);
$db = $db_getter->(); # revalidate connection
my $sth = $db->prepare("SELECT ua.value, COUNT(*) AS 'count' " .
"FROM userproplite2 ua, userproplite2 ub " .
"WHERE ua.userid=ub.userid AND ua.upropid=? AND " .
"ub.upropid=? and ub.value='US' AND ub.value<>'' " .
"AND ua.userid BETWEEN $low AND $high " .
"GROUP BY 1 ORDER BY 2");
$sth->execute($ups->{'id'}, $upc->{'id'});
die $db->errstr if $db->err;
while ($_ = $sth->fetchrow_hashref) {
$ret{$_->{'value'}} += $_->{'count'};
}
print LJ::Stats::block_status_line($block, $blocks);
}
return \%ret;
},
});
LJ::Stats::register_stat
({ 'type' => "clustered",
'jobname' => "gender",
'statname' => "gender",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
my $cid = shift;
return undef unless $db && $cid;
my $upg = LJ::get_prop("user", "gender");
die "Can't find gender userprop. Database populated?\n" unless $upg;
my $usertotal = $db->selectrow_array("SELECT MAX(userid) FROM userproplite2");
my $blocks = LJ::Stats::num_blocks($usertotal);
my %ret;
foreach my $block (1..$blocks) {
my ($low, $high) = LJ::Stats::get_block_bounds($block);
$db = $db_getter->(); # revalidate connection
my $sth = $db->prepare("SELECT value, COUNT(*) AS 'count' FROM userproplite2 up " .
"LEFT JOIN clustertrack2 c ON up.userid=c.userid " .
"WHERE up.upropid=? AND up.userid BETWEEN $low AND $high " .
"AND (c.clusterid IS NULL OR c.clusterid=?) GROUP BY 1");
$sth->execute($upg->{'id'}, $cid);
die "clusterid: $cid, " . $db->errstr if $db->err;
while ($_ = $sth->fetchrow_hashref) {
$ret{$_->{'value'}} += $_->{'count'};
}
print LJ::Stats::block_status_line($block, $blocks);
}
return \%ret;
},
});
# schools
LJ::Stats::register_stat
({
'type' => "global",
'jobname' => "schools",
'statname' => "schools",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
# We want to make sure the country detail information is always showing the current
# top 10. Thus we must delete old stat data from the table before doing these new
# calculations. If we don't, then a country which is no longer in the top 10 will
# still be written into the stats.txt file with stale data. Maybe someday we'll
# write an API to make this seem less hacky.
my $dbh = LJ::Stats::get_db("dbh");
$dbh->do("DELETE FROM stats WHERE statcat='schools'");
my @approved_total = $db->selectrow_array("SELECT COUNT(*) FROM schools");
my @pending_total = $db->selectrow_array("SELECT COUNT(*) FROM schools_pending");
my %ret;
my $approved_counts = $db->selectall_hashref("SELECT country, COUNT(*) as count FROM schools GROUP BY country ORDER BY count DESC LIMIT 10", 'country');
my $pending_counts = $db->selectall_hashref("SELECT country, COUNT(*) as count FROM schools_pending GROUP BY country ORDER BY count DESC", 'country');
$ret{'approved'} = $approved_total[0];
$ret{'pending'} = $pending_total[0];
foreach (sort { $approved_counts->{$b}->{'count'} <=> $approved_counts->{$a}->{'count'} } keys %$approved_counts) {
$ret{"approved_$_"} = $approved_counts->{$_}->{'count'};
}
foreach (sort { $pending_counts->{$b}->{'count'} <=> $pending_counts->{$a}->{'count'} } keys %$pending_counts) {
$ret{"pending_$_"} = $pending_counts->{$_}->{'count'};
}
return \%ret;
},
});
# run stats
LJ::Stats::run_stats(@which);
#### dump to text file
print "-I- Dumping to a text file.\n";
{
my $dbh = LJ::Stats::get_db("dbh");
my $sth = $dbh->prepare
("SELECT statcat, statkey, statval FROM stats ORDER BY 1, 2");
$sth->execute;
die $dbh->errstr if $dbh->err;
open (OUT, ">$LJ::HTDOCS/stats/stats.txt");
while (my @row = $sth->fetchrow_array) {
next if grep { $row[0] eq $_ } @LJ::PRIVATE_STATS;
print OUT join("\t", @row), "\n";
}
close OUT;
}
print "-I- Done.\n";
};
$maint{'genstats_size'} = sub {
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "size-accounts",
'statname' => "size",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
# not that this isn't a total of current accounts (some rows may have
# been deleted), but rather a total of accounts ever created
my $size = $db->selectrow_array("SELECT MAX(userid) FROM user");
return { 'accounts' => $size };
},
});
LJ::Stats::register_stat
({ 'type' => "clustered",
'jobname' => "size-accounts_active",
'statname' => "size",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
my @intervals = qw(1 7 30);
my $max_age = 86400 * $intervals[-1];
my $sth = $db->prepare
("SELECT FLOOR((UNIX_TIMESTAMP()-timeactive)/86400), COUNT(*) " .
"FROM clustertrack2 " .
"WHERE timeactive > UNIX_TIMESTAMP()-$max_age GROUP BY 1");
$sth->execute;
my %ret = ();
while (my ($days, $active) = $sth->fetchrow_array) {
# which day interval does this fall in?
# -- in last day, in last 7, in last 30?
foreach my $int (@intervals) {
$ret{$int} += $active if $days < $int;
}
}
return { map { ("accounts_active_$_" => $ret{$_}+0) } @intervals };
},
});
print "-I- Generating account size stats.\n";
LJ::Stats::run_stats("size-accounts", "size-accounts_active");
print "-I- Done.\n";
};
$maint{'genstats_weekly'} = sub
{
LJ::Stats::register_stat
({ 'type' => "global",
'jobname' => "supportrank",
'statname' => "supportrank",
'handler' =>
sub {
my $db_getter = shift;
return undef unless ref $db_getter eq 'CODE';
my $db = $db_getter->();
return undef unless $db;
my %supportrank;
my $rank = 0;
my $lastpoints = 0;
my $buildup = 0;
my $sth = $db->prepare
("SELECT userid, SUM(points) AS 'points' " .
"FROM supportpoints " .
"GROUP BY 1 ORDER BY 2 DESC");
$sth->execute;
die $db->errstr if $db->err;
while ($_ = $sth->fetchrow_hashref) {
if ($lastpoints != $_->{'points'}) {
$lastpoints = $_->{'points'};
$rank += (1 + $buildup);
$buildup = 0;
} else {
$buildup++;
}
$supportrank{$_->{'userid'}} = $rank;
}
# move old 'supportrank' stat to supportrank_prev
# no API for this :-/
{
my $dbh = LJ::Stats::get_db("dbh");
$dbh->do("DELETE FROM stats WHERE statcat='supportrank_prev'");
$dbh->do("UPDATE stats SET statcat='supportrank_prev' WHERE statcat='supportrank'");
}
return \%supportrank;
}
});
print "-I- Generating weekly stats.\n";
LJ::Stats::run_stats('supportrank');
print "-I- Done.\n";
};
$maint{'memeclean'} = sub
{
my $dbh = LJ::get_db_writer();
print "-I- Cleaning memes.\n";
my $sth = $dbh->prepare("SELECT statkey FROM stats WHERE statcat='popmeme'");
$sth->execute;
die $dbh->errstr if $dbh->err;
while (my $url = $sth->fetchrow_array) {
my $copy = $url;
LJ::run_hooks("canonicalize_url", \$copy);
unless ($copy) {
my $d = $dbh->quote($url);
$dbh->do("DELETE FROM stats WHERE statcat='popmeme' AND statkey=$d");
print " deleting: $url\n";
}
}
print "-I- Done.\n";
};
1;

122
bin/maint/statspics.pl Normal file
View File

@ -0,0 +1,122 @@
#!/usr/bin/perl
#
use GD::Graph::bars;
$maint{'genstatspics'} = sub
{
my $dbh = LJ::get_db_writer();
my $sth;
### get posts by day data from summary table
print "-I- new accounts by day.\n";
$sth = $dbh->prepare("SELECT DATE_FORMAT(statkey, '%m-%d') AS 'day', statval AS 'new' FROM stats WHERE statcat='newbyday' ORDER BY statkey DESC LIMIT 60");
$sth->execute;
if ($dbh->err) { die $dbh->errstr; }
my @data;
my $i;
my $max;
while ($_ = $sth->fetchrow_hashref)
{
my $val = $_->{'new'};
unshift @{$data[0]}, ($i++ % 5 == 0 ? $_->{'day'} : "");
unshift @{$data[1]}, $val;
if ($val > $max) { $max = $val; }
}
# posts by day graph
my $g = GD::Graph::bars->new(520, 350);
$g->set(
x_label => 'Day',
y_label => 'Accounts',
title => 'New accounts per day',
tranparent => 0,
y_max_value => $max,
);
my $gd = $g->plot(\@data);
open(IMG, ">$LJ::HTDOCS/stats/newbyday.png") or die $!;
binmode IMG;
print IMG $gd->png;
close IMG;
unless ($LJ::DISABLED{'stats-postsbyday'}) {
print "-I- posts in last 60 days.\n";
### suck the data in
$sth = $dbh->prepare("SELECT DATE_FORMAT(statkey, '%m-%d') AS 'day', statval AS 'posts' FROM stats WHERE statcat='postsbyday' ORDER BY statkey DESC LIMIT 60");
$sth->execute;
if ($dbh->err) { die $dbh->errstr; }
### analyze the last 60 days data
my @data;
my $i;
my $max;
while ($_ = $sth->fetchrow_hashref)
{
my $val = $_->{'posts'};
unshift @{$data[0]}, ($i++ % 5 == 0 ? $_->{'day'} : "");
unshift @{$data[1]}, $val;
if ($val > $max) { $max = $val; }
}
# posts by day graph
my $g = GD::Graph::bars->new(520, 350);
$g->set(
x_label => 'Day',
y_label => 'Posts',
title => 'Posts per day',
tranparent => 0,
y_max_value => $max,
);
my $gd = $g->plot(\@data);
open(IMG, ">$LJ::HTDOCS/stats/postsbyday.png") or die $!;
binmode IMG;
print IMG $gd->png;
close IMG;
print "-I- posts by week.\n";
### suck the data in
$sth = $dbh->prepare("SELECT DATE_FORMAT(statkey, '%X-%V') AS 'week', SUM(statval) AS 'posts' FROM stats WHERE statcat='postsbyday' AND DATE_FORMAT(statkey, '%X-%V') <> DATE_FORMAT(NOW(), '%X-%V') AND statkey>'1999-06-01' GROUP BY 1 ORDER BY statkey DESC");
$sth->execute;
if ($dbh->err) { die $dbh->errstr; }
### analyze the last 60 days data
my @data;
my $i;
my $max;
while ($_ = $sth->fetchrow_hashref)
{
my $val = $_->{'posts'};
unshift @{$data[0]}, ($i++ % 10 == 0 ? $_->{'week'} : "");
unshift @{$data[1]}, $val;
if ($val > $max) { $max = $val; }
}
# posts by week graph
my $g = GD::Graph::bars->new(520, 350);
$g->set(
x_label => 'Week',
y_label => 'Posts',
title => 'Posts per week',
tranparent => 0,
y_max_value => $max,
);
my $gd = $g->plot(\@data);
open(IMG, ">$LJ::HTDOCS/stats/postsbyweek.png") or die $!;
binmode IMG;
print IMG $gd->png;
close IMG;
}
print "-I- done.\n";
};
1;

100
bin/maint/synsuck.pl Normal file
View File

@ -0,0 +1,100 @@
#!/usr/bin/perl
#
use strict;
use vars qw(%maint %maintinfo);
use lib "$ENV{'LJHOME'}/cgi-bin"; # extra XML::Encoding files in cgi-bin/XML/*
use LJ::SynSuck;
$maintinfo{'synsuck'}{opts}{locking} = "per_host";
$maint{'synsuck'} = sub
{
my $maxcount = shift || 0;
my $verbose = $LJ::LJMAINT_VERBOSE;
my %child_jobs; # child pid => [ userid, lock ]
# get the next user to be processed
my @all_users;
my $get_next_user = sub {
return shift @all_users if @all_users;
# need to get some more rows
my $dbh = LJ::get_db_writer();
my $current_jobs = join(",", map { $dbh->quote($_->[0]) } values %child_jobs);
my $in_sql = " AND u.userid NOT IN ($current_jobs)" if $current_jobs;
my $sth = $dbh->prepare("SELECT u.user, s.userid, s.synurl, s.lastmod, " .
" s.etag, s.numreaders, s.checknext " .
"FROM user u, syndicated s " .
"WHERE u.userid=s.userid AND u.statusvis='V' " .
"AND s.checknext < NOW()$in_sql " .
"LIMIT 500");
$sth->execute;
while (my $urow = $sth->fetchrow_hashref) {
push @all_users, $urow;
}
return undef unless @all_users;
return shift @all_users;
};
# fork and manage child processes
my $max_threads = $LJ::SYNSUCK_MAX_THREADS || 1;
print "[$$] PARENT -- using $max_threads workers\n" if $verbose;
my $threads = 0;
my $userct = 0;
my $keep_forking = 1;
while ( $maxcount == 0 || $userct < $maxcount ) {
if ($threads < $max_threads && $keep_forking) {
my $urow = $get_next_user->();
unless ($urow) {
$keep_forking = 0;
next;
}
my $lockname = "synsuck-user-" . $urow->{user};
my $lock = LJ::locker()->trylock($lockname);
next unless $lock;
print "Got lock on '$lockname'. Running\n" if $verbose;
# spawn a new process
if (my $pid = fork) {
# we are a parent, nothing to do?
$child_jobs{$pid} = [$urow->{'userid'}, $lock];
$threads++;
$userct++;
} else {
# handles won't survive the fork
LJ::disconnect_dbs();
LJ::SynSuck::update_feed($urow, $verbose);
exit 0;
}
# wait for child(ren) to die
} else {
my $child = wait();
last if $child == -1;
delete $child_jobs{$child};
$threads--;
}
}
# Now wait on any remaining children so we don't leave zombies behind.
while ( %child_jobs ) {
my $child = wait();
last if $child == -1;
delete $child_jobs{ $child };
$threads--;
}
print "[$$] $userct users processed\n" if $verbose;
return;
};
# Local Variables:
# mode: perl
# c-basic-indent: 4
# indent-tabs-mode: nil
# End:

26
bin/maint/taskinfo.txt Normal file
View File

@ -0,0 +1,26 @@
stats.pl:
genstats - Generates the nightly statistics
genstats_size - Generates the site size stats
genstats_weekly - Generates the weekly statistics
memeclean - Removes things from meme summary that are excluded by new URL cleaner rules
statspics.pl:
genstatspics - Makes a bunch of graphs to show on the statistics page.
clean_caches.pl:
clean_caches - removes old cache files
synsuck.pl:
synsuck - Polls needed remote, syndicated RSS/etc and updates journals.
captcha.pl:
gen_audio_captchas - Generate any needed new audio challenges.
gen_image_captchas - Generate any needed new graphical challenges.
clean_captchas - Purge old challenges from the database.
generic.pl:
joinmail - Generates daily email digests for community join requests
clean_spamreports - Clean out data from the spamreports table older than 90 days.
comm_promo_list.pl:
comm_promo_list - Generates data needed for community promos

1095
bin/moveucluster.pl Executable file

File diff suppressed because it is too large Load Diff

2751
bin/moveuclusterd.pl Executable file

File diff suppressed because it is too large Load Diff

258
bin/qbufferd.pl Executable file
View File

@ -0,0 +1,258 @@
#!/usr/bin/perl
#
# <LJDEP>
# lib: Proc::ProcessTable, cgi-bin/ljlib.pl
# </LJDEP>
use strict;
use Getopt::Long
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
require "$ENV{'LJHOME'}/cgi-bin/supportlib.pl";
require "$ENV{'LJHOME'}/cgi-bin/ljcmdbuffer.pl";
my $opt_foreground;
my $opt_debug;
my $opt_stop;
exit 1 unless GetOptions('foreground' => \$opt_foreground,
'debug' => \$opt_debug,
'stop' => \$opt_stop,
);
if ($LJ::DISABLED{qbufferd_jobs}) {
print "qbufferd.pl jobs disabled, exiting\n";
exit 0;
}
BEGIN {
$LJ::OPTMOD_PROCTABLE = eval "use Proc::ProcessTable; 1;";
}
my $DELAY = $LJ::QBUFFERD_DELAY || 15;
my $pidfile = $LJ::QBUFFERD_PIDFILE || "$ENV{'LJHOME'}/var/qbufferd.pid";
my $pid;
if (-e $pidfile) {
open (PID, $pidfile);
chomp ($pid = <PID>);
close PID;
if ($opt_stop) {
if (kill 15, $pid) {
print "Shutting down qbufferd.\n";
} else {
print "qbufferd not running?\n";
}
exit;
}
if ($LJ::OPTMOD_PROCTABLE) {
my $processes = Proc::ProcessTable->new()->table;
if (grep { $_->cmndline =~ /perl.+qbufferd/ && $_->pid != $$ } @$processes) {
exit;
}
} else {
if (kill 0, $pid) {
# seems to still be running (at least something is with that pid)
exit;
}
}
}
if ($opt_stop) {
print "qbufferd not running?\n";
exit;
}
$SIG{'INT'} = \&stop_qbufferd;
$SIG{'TERM'} = \&stop_qbufferd;
$SIG{'HUP'} = sub {
# nothing. maybe later make a HUP force a flush?
};
if (!$opt_foreground && ($pid = fork))
{
unless (open (PID, ">$pidfile")) {
kill 15, $pid;
die "Couldn't write PID file. Exiting.\n";
}
print PID $pid, "\n";
close PID;
print "qbufferd started with pid $pid\n";
if (-s $pidfile) { print "pid file written ($pidfile)\n"; }
exit;
}
# Close filehandles unless running in --debug or --foreground mode.
unless ( $opt_debug || $opt_foreground ) {
close STDIN && open STDIN, "</dev/null";
close STDOUT && open STDOUT, "+>&STDIN";
close STDERR && open STDERR, "+>&STDIN";
}
# fork off a separate qbufferd process for all specified
# job types in @LJ::QBUFFERD_ISOLATE, and then
# another process for all other job types. The current process
# will keep tabs on all of those
my %isolated;
my %pids; # job -> pid
my %jobs; # pid -> job
my $working = 0; # 1 for processes that do actual work
my $my_job;
foreach my $job (@LJ::QBUFFERD_ISOLATE) {
$isolated{$job} = 1;
}
foreach my $job (@LJ::QBUFFERD_ISOLATE, "_others_") {
if (my $child = fork) {
# parent.
$pids{$job} = $child;
$jobs{$child} = $job;
next;
} else {
# child.
$0 .= " [$job]";
$my_job = $job;
$working = 1;
last;
}
}
# at this point, $my_job is either the specialized 'cmd' to run, or
# '_others_' to mean everything besides stuff with their own processes.
# $working is 1 for nonempty values of $my_job .
sub stop_qbufferd
{
# stop children
unless ($working) {
foreach my $job (keys %pids) {
my $child = $pids{$job};
print "Killing child pid $child job: $job\n" if $opt_debug;
kill 15, $child;
}
unlink $pidfile;
}
print "Quitting: " . ($working ? "job $my_job" : "parent") . "\n" if $opt_debug;
exit;
}
while(not $working) {
# controlling process's cycle
my $pid;
$pid = wait();
print "Child exited, pid $pid, job $jobs{$pid}\n" if $opt_debug;
if ($jobs{$pid}) {
my $job = $jobs{$pid};
print "Restarting job $job\n" if $opt_debug;
delete $pids{$job};
delete $jobs{$pid};
if (my $child = fork) {
# parent.
$pids{$job} = $child;
$jobs{$child} = $job;
} else {
# child.
$0 .= " [$job]";
$my_job = $job;
$working = 1; # go work
}
}
}
# the actual work begins here
my @all_jobs = qw(delitem weblogscom send_mail support_notify dirty);
foreach my $hook (keys %LJ::HOOKS) {
next unless $hook =~ /^cmdbuf:(\w+):run$/;
push @all_jobs, $1;
}
while (LJ::start_request())
{
my $cycle_start = time();
print "Starting cycle. Job $my_job\n" if $opt_debug;
# syndication (checks RSS that need to be checked)
if ($my_job eq "synsuck") {
system("$ENV{'LJHOME'}/bin/ljmaint.pl", "-v0", "synsuck");
print "Sleeping. Job $my_job\n" if $opt_debug;
my $elapsed = time() - $cycle_start;
sleep ($DELAY-$elapsed) if $elapsed < $DELAY;
next;
}
# do main cluster updates
my $dbh = LJ::get_dbh("master");
unless ($dbh) {
sleep 10;
next;
}
# keep track of what commands we've run the start hook for
my %started;
# handle clusters
foreach my $c (@LJ::QBUFFERD_CLUSTERS ? @LJ::QBUFFERD_CLUSTERS : @LJ::CLUSTERS) {
print "Cluster: $c Job: $my_job\n" if $opt_debug;
my $db = LJ::get_cluster_master($c);
next unless $db;
my @check_jobs = ($my_job);
if ($my_job eq "_others_") { @check_jobs = grep { ! $isolated{$_} } @all_jobs; }
foreach my $cmd (@check_jobs) {
my $have_jobs = $db->selectrow_array("SELECT cbid FROM cmdbuffer WHERE cmd=? LIMIT 1",
undef, $cmd);
next unless $have_jobs;
print " Starting $cmd...\n" if $opt_debug;
unless ($started{$cmd}++) {
LJ::Cmdbuffer::flush($dbh, undef, "$cmd:start");
}
LJ::Cmdbuffer::flush($dbh, $db, $cmd);
print " Finished $cmd.\n" if $opt_debug;
# monitor process size and job counts to suicide if necessary
my $size = 0;
if (open(S, "/proc/$$/status")) {
my $file;
{ local $/ = undef; $file = <S>; }
$size = $1 if $file =~ /VmSize:.+?(\d+)/;
close S;
}
# is it our time to go?
my $kill_job_ct = LJ::Cmdbuffer::get_property($cmd, 'kill_job_ct') || 0;
my $kill_mem_size = LJ::Cmdbuffer::get_property($cmd, 'kill_mem_size') || 0;
if ($kill_job_ct && $started{$cmd} >= $kill_job_ct ||
$kill_mem_size && $size >= $kill_mem_size)
{
# trigger reload of current child process
print "Job suicide: $cmd. (size=$size, rpcs=" . ($started{dirty}+0) . ")\n"
if $opt_debug;
# run end hooks before dying
foreach my $cmd (keys %started) {
LJ::Cmdbuffer::flush($dbh, undef, "$cmd:finish");
}
exit 0;
}
}
}
# run the end hook for all commands we've run
foreach my $cmd (keys %started) {
LJ::Cmdbuffer::flush($dbh, undef, "$cmd:finish");
}
print "Sleeping. Job $my_job\n" if $opt_debug;
my $elapsed = time() - $cycle_start;
sleep ($DELAY-$elapsed) if $elapsed < $DELAY;
};

152
bin/renameuser.pl Executable file
View File

@ -0,0 +1,152 @@
#!/usr/bin/perl
#
# <LJDEP>
# lib: cgi-bin/ljlib.pl
# </LJDEP>
use strict;
use Getopt::Long;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
sub usage {
die "Usage: [--swap --force] <from_user> <to_user>\n";
}
my %args = ( swap => 0, force => 0 );
usage() unless
GetOptions('swap' => \$args{swap},
'force' => \$args{force},
);
my $error;
my $from = shift @ARGV;
my $to = shift @ARGV;
usage() unless $from =~ /^\w{1,15}$/ && $to =~ /^\w{1,15}$/;
my $dbh = LJ::get_db_writer();
unless ($args{swap}) {
if (rename_user($from, $to)) {
print "Success. Renamed $from -> $to.\n";
} else {
print "Failed: $error\n";
}
exit;
}
### check that emails/passwords match, and that at least one is verified
unless ($args{force}) {
my @acct = grep { $_ } LJ::no_cache(sub {
return (LJ::load_user($from),
LJ::load_user($to));
});
unless (@acct == 2) {
print "Both accounts aren't valid.\n";
exit 1;
}
unless (lc($acct[0]->raw_email) eq lc($acct[1]->raw_email)) {
print "Email addresses don't match.\n";
print " " . $acct[0]->raw_email . "\n";
print " " . $acct[1]->raw_email . "\n";
exit 1;
}
unless ($acct[0]->password eq $acct[1]->password) {
print "Passwords don't match.\n";
exit 1;
}
unless ($acct[0]->{'status'} eq "A" || $acct[1]->{'status'} eq "A") {
print "At least one account isn't verified.\n";
exit 1;
}
}
my $swapnum = 0;
print "Swapping 1/3...\n";
until ($swapnum == 10 || rename_user($from, "lj_swap_$swapnum")) {
$swapnum++;
}
if ($swapnum == 10) {
print "Couldn't find a swap position?\n";
exit 1;
}
print "Swapping 2/3...\n";
unless (rename_user($to, $from)) {
print "Swap failed in the middle, from $to -> $from failed.\n";
exit 1;
}
print "Swapping 3/3...\n";
unless (rename_user("lj_swap_$swapnum", $to)) {
print "Swap failed in the middle, from lj_swap_$swapnum -> $to failed.\n";
exit 1;
}
# check for circular 'renamedto' references
{
# if the fromuser had redirection on, make sure it points to the new $to user
my $fromu = LJ::load_user($from, 'force');
LJ::load_user_props($fromu, 'renamedto');
if ($fromu->{renamedto} && $fromu->{renamedto} ne $to) {
print "Setting redirection: $from => $to\n";
unless (LJ::set_userprop($fromu, 'renamedto' => $to)) {
print "Error setting 'renamedto' userprop for $from\n";
exit 1;
}
}
# if the $to user had redirection, they shouldn't anymore
my $tou = LJ::load_user($to, 'force');
LJ::load_user_props($tou, 'renamedto');
if ($tou->{renamedto}) {
print "Removing redirection for user: $to\n";
unless (LJ::set_userprop($tou, 'renamedto' => undef)) {
print "Error setting 'renamedto' userprop for $to\n";
exit 1;
}
}
}
print "Swapped.\n";
exit 0;
sub rename_user
{
my $from = shift;
my $to = shift;
my $qfrom = $dbh->quote(LJ::canonical_username($from));
my $qto = $dbh->quote(LJ::canonical_username($to));
print "Renaming $from -> $to\n";
my $u = LJ::load_user($from, 'force');
unless ($u) {
$error = "Invalid source user: $from";
return 0;
}
foreach my $table (qw(user useridmap overrides style))
{
$dbh->do("UPDATE $table SET user=$qto WHERE user=$qfrom");
if ($dbh->err) {
$error = $dbh->errstr;
return 0;
}
}
# from user is now invalidated
LJ::memcache_kill($u->{userid}, "userid");
LJ::MemCache::delete("uidof:$from");
LJ::MemCache::delete("uidof:$to");
LJ::procnotify_add("rename_user", { 'user' => $u->{'user'},
'userid' => $u->{'userid'} });
$dbh->do("INSERT INTO renames (renid, token, payid, userid, fromuser, touser, rendate) ".
"VALUES (NULL,'[manual]',0,$u->{userid},$qfrom,$qto,NOW())");
return 1;
}

207
bin/statserv.pl Executable file
View File

@ -0,0 +1,207 @@
#!/usr/bin/perl -w
# LiveJournal statistics server. Sits on a UDP port and journals
# information on the incoming hit rate, manages site bans, etc.
# Loosely based on the ljrpcd code to save typing ;)
# <LJDEP>
# lib: IO::Socket Proc::ProcessTable IO::Handle DBI
#
# </LJDEP>
use strict;
use IO::Socket;
use IO::Handle;
use Proc::ProcessTable;
use DBI;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
# Max message length and port to bind.
my $MAXLEN = 512;
my $PORTNO = 6200;
my $PIDFILE = '/home/lj/var/statserv.pid';
my $LOGDIR = '/home/lj/logs/';
# Maximum amount of hits they can use in five minutes.
my %maxes = ( 'ip' => 15, 'guest' => 20, 'user' => 25 );
# Pid and pidfile.
my $pid;
my $is_parent = 1;
# Socket. Needs to be here for the HUP stuff.
my $sock;
# Cache hash.
my %caches = ();
# Cache array.
my @events = ();
# Exceptions hash (IP range or username as keys)
# If you want some host (such as a big stupid random proxy) to
# be more lenient with the number of hits it can make in five minutes,
# put the value in here. If value is -1, then there is no limit.
my %except = ();
# In case we're shot, unlink the pidfile.
$SIG{TERM} = sub {
unlink($PIDFILE);
exit 1;
};
# Local network bind to.
my $MYNET = '10.0';
if (-e $PIDFILE) {
open (PID, $PIDFILE);
my $tpid;
chomp ($tpid = <PID>);
close PID;
my $processes = Proc::ProcessTable->new()->table;
if (grep { $_->cmndline =~ /statserv/ } @$processes) {
print "Process exists already, quitting.\n";
exit 1;
}
}
print "LiveJournal Statistics Daemon starting up into the background...\n";
if ($pid = fork) {
# Parent, log pid and exit.
open(PID, ">$PIDFILE") or die "Couldn't open $PIDFILE for writing: $!\n";
print PID $pid;
close(PID);
print "Closing ($pid) wrote to $PIDFILE\n";
$is_parent = 1;
exit;
} else {
# This is the child.
my($cmdmsg, $remaddr, $remhost);
# HUP signal handler.
$SIG{HUP} = \&restart_request;
# SIGUSR handler.
$SIG{USR1} = sub { open_logfile(); };
open_logfile();
$sock = IO::Socket::INET->new(LocalPort => "$PORTNO", Proto => 'udp') or die "socket: $@";
# Main loop.
while ($sock->recv($cmdmsg, $MAXLEN)) {
my ($port, $ipaddr) = sockaddr_in($sock->peername);
my $ip_addr = inet_ntoa($ipaddr);
# Make sure it's from around here.
if ($ip_addr !~ m/^$MYNET/) {
print "Got message from an invalid host.\n";
next;
}
# Quick command parsing, since there isn't much to it.
if ($cmdmsg =~ s/^cmd:\s//) {
handle_request($cmdmsg);
next;
}
}
die "recv: $!\n";
}
# Sub to restart the daemon.
sub restart_request {
$sock->close;
unlink($PIDFILE);
exec($0);
}
# Handle the request. This updates the appropriate caches,
# and may set a ban.
# Requests look like:
# cmd: cachename : ip_addr : type : url
# type can be: ip, guest, or user
# If type is "ip" then cachename can be anything. I suggest
# it be set to "ip" as well. If just to save space.
sub handle_request {
my $cmd = shift;
my $now = time();
# Clear expired events.
clean_events($now);
# As of now, we don't care about the URL, really.
if ($cmd =~ m/^(\w+)\s:\s([\d\.]+)\s:\s(\w+)/) {
my $user = $1;
my $ip_addr = $2;
my $type = $3;
# If there was no cookie of any kind, the type
# name is set to "ip" - in this case we up the
# cache number for the IP range.
if ($type eq "ip") {
# This regex is dumb, but the data we have is trustable.
$user = $ip_addr;
$user =~ s/(\d+)\.(\d+)\.(\d+)\.(\d+)/$1\.$2\.$3\./;
}
unless (exists $caches{$user}) {
$caches{$user} = { 'numhit' => 0, 'type' => $type };
}
push @events, [ $user, $now ];
$caches{$user}->{'numhit'}++;
# Now we check to see if they have hit too fast, and ban if so.
if (should_ban($user)) {
# FIXME: For final operation, this should be replaced with
# a call to set_ban(). This is also going to spam a ton,
# but with the "spiffy" algorithm I can't easily nuke a user.
print "Would have banned user $user. Hits: " . $caches{$user}->{'numhit'} . "\n";
}
# After this, "add_stat($user, $type, $url)" should run.
} else {
print "Got a mal-formed request: $cmd\n";
}
}
# Returns 1 if the passed "user" should be banned, 0 if not.
sub should_ban {
my $user = shift;
my $max = $except{$user} || $maxes{$caches{$user}->{'type'}} || 0;
# If it doesn't have a defined class, do we really want it around?
return 1 unless ($max);
return 1 if ($caches{$user}->{'numhit'} > $max);
return 0;
}
# Removes old events, and decrements caches.
sub clean_events {
my $now = shift;
while (@events && $events[0]->[1] < $now - 360) {
my $deadevt = shift @events;
if (--$caches{$deadevt->[0]}->{'numhits'} < 1) {
delete $caches{$deadevt->[0]};
}
}
}
# Placeholder. Sets a ban in the database.
sub set_ban {
}
# Placeholder. Runs various stats collections.
sub add_stat {
}
# Opens a new tagged logfile. Also sets it to the default
# filehandle, sets autoflush, and returns the new handle.
sub open_logfile {
my $now = time();
my $logname = $LOGDIR . "statserv-" . $now . "\.log\n";
my $logfh = new IO::Handle;
open($logfh, ">> $logname") or die "Couldn't open $logname: $!\n";
my $oldfh = select($logfh);
# Make sure the old one is closed.
close($oldfh);
# Set autoflush and return.
$| = 1;
return $logfh;
}

33
bin/truncate-cluster.pl Executable file
View File

@ -0,0 +1,33 @@
#!/usr/bin/perl
#
use strict;
require "$ENV{LJHOME}/cgi-bin/ljlib.pl";
my $cid = shift;
die "Usage: truncate-cluster.pl <clusterid>\n" unless $cid =~ /^\d+$/;
my $dbh = LJ::get_db_writer();
my $ct = $dbh->selectrow_array("SELECT COUNT(*) FROM user WHERE clusterid=?", undef, $cid);
die $dbh->errstr if $dbh->err;
if ($ct > 0) {
die "Can't truncate a cluster with users. Cluster \#$cid has $ct users.\n";
}
my $cm = LJ::get_cluster_master({raw=>1}, $cid);
die "Can't get handle to cluster \#$cid\n" unless $cm;
my $size;
foreach my $table (sort (@LJ::USER_TABLES, @LJ::USER_TABLES_LOCAL)) {
my $ts = $cm->selectrow_hashref("SHOW TABLE STATUS like '$table'");
die "Can't get table status for $table\n" unless $ts;
print "Size of $table = $ts->{'Data_length'}\n";
next unless $ts->{'Data_length'};
$cm->do("TRUNCATE TABLE $table");
die $cm->errstr if $cm->err;
$size += $ts->{'Data_length'};
}
print "Total size truncated (excluding indexes): $size\n";

99
bin/trunk-update.pl Executable file
View File

@ -0,0 +1,99 @@
#!/usr/bin/perl
use strict;
use IO::Socket::INET;
unless ($ENV{LJHOME}) {
die "\$LJHOME not set.";
}
chdir "$ENV{LJHOME}" or die "Failed to chdir to \$LJHOME";
my $cvsreport = "$ENV{LJHOME}/bin/cvsreport.pl";
die "cvsreport.pl missing or unexecutable" unless -x $cvsreport;
require "$ENV{LJHOME}/cgi-bin/ljlib.pl";
die "NO DO NOT RUN THIS IN PRODUCTION" if $LJ::IS_LJCOM_PRODUCTION;
update_svn();
my @files = get_updated_files();
sync();
new_phrases() if grep { /en.+\.dat/ } @files;
update_db() if grep { /\.sql/ } @files;
bap() if grep { /cgi-bin.+\.[pl|pm]/ } @files;
my $updatedfilepath = "$ENV{LJHOME}/logs/trunk-last-updated.txt";
my $updatedfh;
open($updatedfh, ">$updatedfilepath") or return "Could not open file $updatedfilepath: $!\n";
print $updatedfh time();
close $updatedfh;
if (@files) {
exit 0;
}
exit 1;
sub update_svn {
system($cvsreport, "-u", "--checkout")
and die "Failed to run cvsreport.pl with update.";
}
sub get_updated_files {
my @files = ();
open(my $cr, '-|', $cvsreport, '-c', '-1') or die "Could not run cvsreport.pl";
while (my $line = <$cr>) {
$line =~ s/\s+$//;
push @files, $line;
}
close($cr);
return @files;
}
sub sync {
system($cvsreport, "-c", "-s")
and die "Failed to run cvsreport.pl sync second time.";
}
sub update_db {
foreach (1..10) {
my $res = system("bin/upgrading/update-db.pl", "-r", "-p");
last if $res == 0;
if ($res & 127 == 9) {
warn "Killed by kernel (ran out of memory) sleeping and retrying";
sleep 60;
next;
}
die "Unknown exit state of `update-db.pl -r -p`: $res";
}
system("bin/upgrading/update-db.pl", "-r", "--cluster=all")
and die "Failed to run update-db.pl on all clusters";
}
sub new_phrases {
my @langs = @_;
system("bin/upgrading/texttool.pl", "load", @langs)
and die "Failed to run texttool.pl load @langs";
}
sub bap {
print "Restarting apache...\n";
my $sock = IO::Socket::INET->new(PeerAddr => "127.0.0.1:7600")
or die "Couldn't connect to webnoded (port 7600)\n";
print $sock "apr\r\n";
while (my $ln = <$sock>) {
print "$ln";
last if $ln =~ /^OK/;
}
}

1054
bin/upgrading/base-data.sql Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,72 @@
#!/usr/bin/perl
use strict;
use Getopt::Long;
# load libraries now
use lib "$ENV{'LJHOME'}/cgi-bin";
use LJ::Blob;
use Image::Size ();
require "ljlib.pl";
my $opt_fast;
exit 1 unless
GetOptions("fast" => \$opt_fast,
);
my $clusterid = shift;
die "Usage: blobify_userpics.pl <clusterid>\n"
unless $clusterid;
my $db = LJ::get_cluster_master($clusterid);
die "Invalid/down cluster: $clusterid\n" unless $db;
print "Getting count.\n";
my $total = $db->selectrow_array("SELECT COUNT(*) FROM userpicblob2");
my $done = 0;
my $loop = 1;
while ($loop) {
$loop = 0;
LJ::start_request(); # shrink caches
print "Getting 200.\n";
my $sth = $db->prepare("SELECT userid, picid, imagedata FROM userpicblob2 LIMIT 200");
$sth->execute;
while (my ($uid, $picid, $image) = $sth->fetchrow_array) {
$loop = 1;
my $u = LJ::load_userid($uid);
die "Can't find userid: $uid" unless $u;
# sometimes expunges don't expunge all the way.
if ($u->{'statusvis'} eq "X") {
$db->do("DELETE FROM userpicblob2 WHERE userid=$uid AND picid=$picid");
next;
}
my ($sx, $sy, $fmt) = Image::Size::imgsize(\$image);
die "Unknown format" unless $fmt eq "GIF" || $fmt eq "JPG" || $fmt eq "PNG";
$fmt = lc($fmt);
my $err;
my $rv = LJ::Blob::put($u, "userpic", $fmt, $picid, $image, \$err);
die "Error putting file: $u->{'user'}/$picid\n" unless $rv;
unless ($opt_fast) {
# extra paranoid!
my $get = LJ::Blob::get($u, "userpic", $fmt, $picid);
die "Re-fetch didn't match" unless $get eq $image;
}
$db->do("DELETE FROM userpicblob2 WHERE userid=$uid AND picid=$picid");
$done++;
printf " Moved $uid/$picid.$fmt ($done/$total = %.2f%%)\n", ($done / $total * 100);
}
}
my $end_ct = $db->selectrow_array("SELECT COUNT(*) FROM userpicblob2");
if ($end_ct == 0) {
$db->do("TRUNCATE TABLE userpicblob2");
}
print "Done.\n";

View File

@ -0,0 +1,77 @@
#!/usr/bin/perl
use strict;
my $clusterid = shift;
die "Usage: compress_cluster <clusterid>\n"
unless $clusterid;
# load libraries now
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
# force this option on, since that's the point of the tool
$LJ::COMPRESS_TEXT = 1;
my $db = LJ::get_cluster_master($clusterid);
die "Invalid/down cluster: $clusterid\n" unless $db;
# table, column, [ prikey1, prikey2 ]
foreach my $t (['logtext2', 'event', ['journalid', 'jitemid']],
['talktext2', 'body', ['journalid', 'jtalkid']]) {
my ($table, $col, $key) = @$t;
my ($pk1, $pk2) = @$key; # 2 sections of primary key
my $total = $db->selectrow_array("SELECT COUNT(*) FROM $table");
print "Processing table: $table [$total total rows]\n";
$db->do("HANDLER $table OPEN");
my $ct = 0;
my $modct = 0;
my $bytes_pre;
my $bytes_post;
my $stats = sub {
printf("%6.2f%% done (mod=%.2f%%, size=%.2f%%),\n",
($ct / $total) * 100,
($modct / $ct) * 100,
($bytes_post / $bytes_pre) * 100);
};
my $loop = 1;
while ($loop) {
my $sth = $db->prepare("HANDLER $table READ `PRIMARY` NEXT LIMIT 100");
$sth->execute;
$loop = 0;
while (my $row = $sth->fetchrow_hashref) {
$loop = 1;
# print status
$stats->() if (++$ct % 1000 == 0);
# try to compress the text
my $orig_len = length($row->{$col});
$bytes_pre += $orig_len;
my $new_text = LJ::text_compress($row->{$col});
my $new_len = length($new_text);
$bytes_post += $new_len;
# do nothing if the "compressed" and uncompressed sizes are the same
next if $new_text eq $row->{$col};
# update this row since it compressed
$db->do("UPDATE $table SET $col=? WHERE $pk1=? AND $pk2=? AND $col=?",
undef, $new_text, $row->{$pk1}, $row->{$pk2}, $row->{$col});
$modct++;
}
}
$stats->();
$db->do("HANDLER $table CLOSE");
print "$ct rows processed, $modct modified\n\n";
}

View File

@ -0,0 +1,44 @@
#!/usr/bin/perl
#
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
require 'ljlib.pl';
my $dbh = LJ::get_db_writer();
my $max = $dbh->selectrow_array("SELECT MAX(userid) FROM user");
print "max = $max\n";
my $base = shift || 1;
my $lastcheck = 0;
my $size = 1000;
while ($base <= $max) {
my $upper = $base + $size - 1;
my $rv = $dbh->do("INSERT IGNORE INTO email (userid, email) SELECT userid, email FROM user WHERE userid BETWEEN $base AND $upper");
my $rv2 = $dbh->do("INSERT IGNORE INTO password (userid, password) SELECT userid, password FROM user WHERE userid BETWEEN $base AND $upper");
$base += 1000;
my $busy = 0;
my $now = time();
if ($lastcheck != $now) {
$lastcheck = $now;
my $sth = $dbh->prepare("SHOW PROCESSLIST");
$sth->execute;
while (my $row = $sth->fetchrow_hashref) {
next if $row->{Command} eq "Sleep";
next if $row->{Command} eq "Connect";
$busy++;
}
}
print "$base = $rv / $rv2 b=$busy\n";
if ($busy > 100) {
sleep 1;
} elsif ($busy > 50) {
select undef, undef, undef, 0.25;
}
}

41
bin/upgrading/d1d2-single.pl Executable file
View File

@ -0,0 +1,41 @@
#!/usr/bin/perl
#
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $dbh = LJ::get_dbh("master");
my $user = shift @ARGV;
my $where = "dversion=1 LIMIT 1";
if ($user) {
$where = "user=" . $dbh->quote($user);
}
my $u = $dbh->selectrow_hashref("SELECT * FROM user WHERE $where");
unless ($u) {
die "No users with dversion==1 to convert. Done.\n" unless $user;
die "User not found.\n";
}
die "User not dversion 1\n" unless $u->{'dversion'} == 1;
my $dbch = LJ::get_cluster_master($u);
die "Can't connect to cluster master.\n" unless $dbch;
print "$u->{'user'}:\n";
my @pics = @{$dbh->selectcol_arrayref("SELECT picid FROM userpic WHERE ".
"userid=$u->{'userid'}")};
foreach my $picid (@pics) {
print " $picid...\n";
my $imagedata = $dbh->selectrow_array("SELECT imagedata FROM userpicblob ".
"WHERE picid=$picid");
$imagedata = $dbh->quote($imagedata);
$dbch->do("REPLACE INTO userpicblob2 (userid, picid, imagedata) VALUES ".
"($u->{'userid'}, $picid, $imagedata)");
}
$dbh->do("UPDATE user SET dversion=2 WHERE userid=$u->{'userid'}");
print "Done.\n";

320
bin/upgrading/d4d5-global.pl Executable file
View File

@ -0,0 +1,320 @@
#!/usr/bin/perl
#
use strict;
$| = 1;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $BLOCK_MOVE = 5000; # user rows to get at a time before moving
my $BLOCK_INSERT = 25; # rows to insert at a time when moving users
my $BLOCK_UPDATE = 1000; # users to update at a time if they had no data to move
# used for keeping stats notes
my %stats = (); # { 'action' => { 'pass' => { 'stat' => 'value' } } }
# rows come order by user, keep last username so we can avoid dups
my $lastuser;
# what pass are we on?
my $pass;
my $get_db_writer = sub {
return LJ::get_dbh({raw=>1}, "master");
};
my $get_db_slow = sub {
return LJ::get_dbh({raw=>1}, "slow", "master");
};
my $get_cluster_master = sub {
my $cid = shift;
return LJ::get_dbh({raw=>1}, "cluster$cid");
};
# percentage complete
my $status = sub {
my ($ct, $tot, $units, $pass, $user) = @_;
my $len = length($tot);
my $passtxt = $pass ? "[Pass: $pass] " : '';
my $usertxt = $user ? " Moving user: $user" : '';
return sprintf(" $passtxt\[%6.2f%%: %${len}d/%${len}d $units]$usertxt\n",
($ct / $tot) * 100, $ct, $tot);
};
my $header = sub {
my $size = 50;
return "\n" .
("#" x $size) . "\n" .
"# $_[0] " . (" " x ($size - length($_[0]) - 4)) . "#\n" .
("#" x $size) . "\n\n";
};
# mover function
my $move_user = sub {
my $user = shift;
# if the current user is the same as the last,
# we have a duplicate so skip it
return 0 if $user eq $lastuser;
$lastuser = $user;
my $u = LJ::load_user($user);
return 0 unless $u->{'dversion'} == 4;
# update user count for this pass
$stats{'move'}->{$pass}->{'user_ct'}++;
# print status
print $status->($stats{'move'}->{$pass}->{'ct'},
$stats{'move'}->{$pass}->{'total'}, "rows", $pass, $user);
# ignore expunged users
if ($u->{'statusvis'} eq "X") {
LJ::update_user($u, { 'dversion' => 5 })
or die "error updating dversion";
$u->{'dversion'} = 5; # update local copy in memory
return 1;
}
# get a handle for every user to revalidate our connection?
my $dbh = $get_db_writer->()
or die "Can't connect to global master";
my $dbslo = $get_db_slow->()
or die "Can't connect to global slow master";
my $dbcm = $get_cluster_master->($u->{'clusterid'})
or die "Can't connect to cluster master ($u->{'clusterid'})";
# be careful, we're moving data
foreach my $db ($dbh, $dbslo, $dbcm) {
$db->do("SET wait_timeout=28800");
$db->{'RaiseError'} = 1;
}
my @map = (['style' => 's1style',
qw(styleid userid styledes type formatdata is_public
is_embedded is_colorfree opt_cache has_ads lastupdate) ],
['overrides' => 's1overrides',
qw(userid override) ]
);
# in moving, s1stylecache gets special cased because its
# name hasn't changed, only location. so if there is only
# one cluster, then there's no point in physically moving it
if (@LJ::CLUSTERS > 1) {
push @map, ['s1stylecache' => 's1stylecache',
qw(styleid cleandate type opt_cache vars_stor vars_cleanver) ];
}
# user 'system' is a special case. if we encounter this user we'll swap $dbcm
# to secretly be a $dbh. because the 'system' user uses the clustered tables
# on the global dbs, the queries will still work, we just need to misdirect them
$dbcm = $dbh if $u->{'user'} eq 'system';
# styleids to delete since s1stylemap isn't keyed on user
my @delete_styleids = ();
# update tables
foreach my $tableinf (@map) {
# find src and dest table names
my $src_table = shift @$tableinf;
my $dest_table = shift @$tableinf;
# if this is the style table, replace into stylemap here,
# so that if this process is killed, style and stylemap
# won't get too out of sync
my $do_stylemap = $src_table eq 'style' && $dest_table eq 's1style';
# find what columns this table has
my @cols = @$tableinf;
my $cols = join(",", @cols);
my $bind_row = "(" . join(",", map { "?" } @cols) . ")";
my (@bind, @vals);
my (@map_bind, @map_vals);
# flush rows to destination table
my $flush = sub {
return unless @bind;
# insert data
my $bind = join(",", @bind);
$dbcm->do("REPLACE INTO $dest_table ($cols) VALUES $bind", undef, @vals);
# insert new styles into s1stylemap
if ($do_stylemap) {
my $map_bind = join(",", @map_bind);
$dbh->do("REPLACE INTO s1stylemap (styleid, userid) VALUES $map_bind",
undef, @map_vals);
}
# reset values
@bind = ();
@vals = ();
};
# s1stylecache is the only table keyed on styleid, not user
my $where = "user=" . $dbh->quote($u->{'user'});
if ($src_table eq "s1stylecache") {
my $ids = $dbh->selectcol_arrayref("SELECT styleid FROM style WHERE user=?",
undef, $u->{'user'});
my $ids_in = join(",", map { $dbh->quote($_) } @$ids) || "0";
$where = "styleid IN ($ids_in)";
}
# select from source table and build data for insert
my $sth = $dbh->prepare("SELECT * FROM $src_table WHERE $where");
$sth->execute();
while (my $row = $sth->fetchrow_hashref) {
# so that when we look for userid later, it'll be there
$row->{'userid'} = $u->{'userid'};
# build data for insert
push @bind, $bind_row;
push @vals, $row->{$_} foreach @cols;
# special case: insert new s1styles into s1stylemap
if ($do_stylemap) {
push @map_bind, "(?,?)";
push @map_vals, ($row->{'styleid'}, $u->{'userid'});
push @delete_styleids, $row->{'styleid'};
}
# increment the count for this pass, style or overrides
$stats{'move'}->{$pass}->{'ct'}++
unless $src_table eq 's1stylecache';
# flush if we've reached our insert limit
$flush->() if @bind > $BLOCK_INSERT;
}
$flush->();
}
# haven't died yet? everything is still going okay
# update dversion
LJ::update_user($u, { 'dversion' => 5 })
or die "error updating dversion";
$u->{'dversion'} = 5; # update local copy in memory
return 1;
};
my $dbh = $get_db_writer->();
die "Could not connect to global master" unless $dbh;
$dbh->{'RaiseError'} = 1;
my $dbslo = $get_db_slow->();
die "Could not connect to global slow master" unless $dbslo;
$dbslo->{'RaiseError'} = 1;
my $ts = $dbslo->selectrow_hashref("SHOW TABLE STATUS LIKE 'overrides'");
if ($ts->{'Type'} eq 'ISAM') {
die "This script isn't efficient with ISAM tables. Please convert to MyISAM with:\n" .
" mysql> ALTER TABLE overrides TYPE=MyISAM;\n\n" .
"Then re-run this script.\n";
}
print $header->("Moving user data");
# first pass should get everything, second pass will
# get changes made during the first
foreach my $p (1..2) {
# this is strange. perl bug?
$pass = $p;
# get totals from overrides and style so we can do a percentage bar
# there will be overlaps in this when users have both styles and
# overrides, but we'll fix those up as we go
foreach (qw(style overrides)) {
$stats{'move'}->{$pass}->{'total'} += $dbslo->selectrow_array("SELECT COUNT(*) FROM $_");
}
print "Processing $stats{'move'}->{$p}->{'total'} total rows\n";
# 2 passes, so we catch people with styles & overrides,
# styles w/o overrides, and overrides w/o styles
foreach my $table (qw(style overrides)) {
$lastuser = '';
# loop until we have no more users to convert
my $ct;
do {
# get blocks of $BLOCK_MOVE users at a time
my $sth = $dbslo->prepare("SELECT user FROM $table WHERE user>? " .
"ORDER BY user LIMIT $BLOCK_MOVE");
$sth->execute($lastuser);
$ct = 0;
while (my $user = $sth->fetchrow_array) {
$move_user->($user);
$ct++;
}
} while $ct;
}
print $stats{'move'}->{$p}->{'user_ct'}+0 . " users moved\n\n";
}
# now we're confident that all users have had their data moved if
# necessary, so we can just unconditionally change dversions.
print $header->("Updating remaining users");
$stats{'update'}->{'total'} = $dbslo->selectrow_array("SELECT COUNT(*) FROM user WHERE dversion=4");
print "Converting $stats{'update'}->{'total'} users\n";
# now update dversions for users who had no data to move
if ($stats{'update'}->{'total'}) {
my $get_users = sub {
my $sth = $dbslo->prepare("SELECT userid, user FROM user " .
"WHERE dversion=4 LIMIT $BLOCK_UPDATE");
$sth->execute();
my @rows; # [ userid, user ]
while ( my ($userid, $user) = $sth->fetchrow_array) {
push @rows, [ $userid, $user ];
}
return @rows;
};
while (my @rows = $get_users->()) {
# update database
my $bind = join(",", map { "?" } @rows);
my @vals = map { $_->[0] } @rows;
$dbh->do("UPDATE user SET dversion=5 WHERE userid IN ($bind)",
undef, @vals);
# update memcache
foreach (@rows) {
LJ::MemCache::delete([$_->[0], "userid:" . $_->[0]]);
LJ::MemCache::delete([$_->[1], "user:" . $_->[1]]);
}
$stats{'update'}->{'ct'} += @rows;
print $status->($stats{'update'}->{'ct'}, $stats{'update'}->{'total'}, "users");
}
}
my $zeropad = sub {
return sprintf("%d", $_[0]);
};
# calculate total move stats
foreach (1..2) {
$stats{'move'}->{'total_ct'} += $stats{'move'}->{$_}->{'ct'};
$stats{'move'}->{'total_user_ct'} += $stats{'move'}->{$_}->{'user_ct'};
}
print $header->("Dversion 4->5 conversion completed");
print " Rows moved: " . $zeropad->($stats{'move'}->{'total_ct'}) . "\n";
print " Users moved: " . $zeropad->($stats{'move'}->{'total_user_ct'}) . "\n";
print "Users updated: " . $zeropad->($stats{'update'}->{'ct'}) . "\n\n";

349
bin/upgrading/d5d6-mkf.pl Executable file
View File

@ -0,0 +1,349 @@
#!/usr/bin/perl
#
use strict;
use Getopt::Long;
$| = 1;
use lib ("$ENV{LJHOME}/cgi-bin");
require "ljlib.pl";
use LJ::User;
use constant DEBUG => 0; # turn on for debugging (mostly db handle crap)
my $BLOCK_MOVE = 5000; # user rows to get at a time before moving
my $BLOCK_INSERT = 25; # rows to insert at a time when moving users
my $BLOCK_UPDATE = 1000; # users to update at a time if they had no data to move
# get options
my %opts;
exit 1 unless
GetOptions("lock=s" => \$opts{locktype},);
# if no locking, notify them about it
die "ERROR: Lock must be of types 'ddlockd' or 'none'\n"
if $opts{locktype} && $opts{locktype} !~ m/^(?:ddlockd|none)$/;
# used for keeping stats notes
my %stats = (); # { 'stat' => 'value' }
my %handle;
# database handle retrieval sub
my $get_db_handles = sub {
# figure out what cluster to load
my $cid = shift(@_) + 0;
my $dbh = $handle{0};
unless ($dbh) {
$dbh = $handle{0} = LJ::get_dbh({ raw => 1 }, "master");
print "Connecting to master ($dbh)...\n";
eval {
$dbh->do("SET wait_timeout=28800");
};
$dbh->{'RaiseError'} = 1;
}
my $dbcm;
$dbcm = $handle{$cid} if $cid;
if ($cid && ! $dbcm) {
$dbcm = $handle{$cid} = LJ::get_cluster_master({ raw => 1 }, $cid);
print "Connecting to cluster $cid ($dbcm)...\n";
return undef unless $dbcm;
eval {
$dbcm->do("SET wait_timeout=28800");
};
$dbcm->{'RaiseError'} = 1;
}
# return one or both, depending on what they wanted
return $cid ? ($dbh, $dbcm) : $dbh;
};
# percentage complete
my $status = sub {
my ($ct, $tot, $units, $user) = @_;
my $len = length($tot);
my $usertxt = $user ? " Moving user: $user" : '';
return sprintf(" \[%6.2f%%: %${len}d/%${len}d $units]$usertxt\n",
($ct / $tot) * 100, $ct, $tot);
};
my $header = sub {
my $size = 50;
return "\n" .
("#" x $size) . "\n" .
"# $_[0] " . (" " x ($size - length($_[0]) - 4)) . "#\n" .
("#" x $size) . "\n\n";
};
my $zeropad = sub {
return sprintf("%d", $_[0]);
};
# mover function
my $move_user = sub {
my $u = shift;
# make sure our user is of the proper dversion
return 0 unless $u->{'dversion'} == 5;
# at this point, try to get a lock for this user
my $lock;
if ($opts{locktype} eq 'ddlockd') {
$lock = LJ::locker()->trylock("d5d6-$u->{user}");
return 1 unless $lock;
}
# get a handle for every user to revalidate our connection?
my ($dbh, $dbcm) = $get_db_handles->($u->{clusterid});
return 0 unless $dbh;
if ($dbcm) {
# assign this dbcm to the user
$u->set_dbcm($dbcm)
or die "unable to set database for $u->{user}: dbcm=$dbcm\n";
}
# verify dversion hasn't changed on us (done by another job?)
my $dversion = $dbh->selectrow_array("SELECT dversion FROM user WHERE userid = $u->{userid}");
return 1 unless $dversion == 5;
# ignore expunged users
if ($u->{'statusvis'} eq "X" || $u->{'clusterid'} == 0) {
LJ::update_user($u, { dversion => 6 })
or die "error updating dversion";
$u->{dversion} = 6; # update local copy in memory
return 1;
}
return 0 unless $dbcm;
# step 1: get all friend groups and move those. safe to just grab with no limit because
# there are limits to how many friend groups you can have (30).
my $rows = $dbh->selectall_arrayref('SELECT groupnum, groupname, sortorder, is_public ' .
'FROM friendgroup WHERE userid = ?', undef, $u->{userid});
if (@$rows) {
# got some rows, create an update statement
my (@bind, @vars);
foreach my $row (@$rows) {
push @bind, "($u->{userid}, ?, ?, ?, ?)";
push @vars, $_ foreach @$row;
}
my $bind = join ',', @bind;
eval {
$u->do("INSERT INTO friendgroup2 (userid, groupnum, groupname, sortorder, is_public) " .
"VALUES $bind", undef, @vars);
};
}
# general purpose flusher for use below
my (@bind, @vars);
my $flush = sub {
return unless @bind;
my ($table, $cols) = @_;
# insert data into cluster master
my $bind = join(",", @bind);
$u->do("REPLACE INTO $table ($cols) VALUES $bind", undef, @vars);
die "error in flush $table: " . $u->errstr . "\n" if $u->err;
# reset values
@bind = ();
@vars = ();
};
# step 1.5: see if the user has any data already? clear it if so.
my $counter = $dbcm->selectrow_array("SELECT max FROM counter WHERE journalid = ? AND area = 'R'",
undef, $u->{userid});
$counter += 0;
if ($counter > 0) {
# yep, so we need to delete stuff, real data first
foreach my $table (qw(memorable2 memkeyword2 userkeywords)) {
$u->do("DELETE FROM $table WHERE userid = ?", undef, $u->{userid});
die "error in clear: " . $u->errstr . "\n" if $u->err;
}
# delete counters used (including memcache of such)
$u->do("DELETE FROM counter WHERE journalid = ? AND area IN ('R', 'K')", undef, $u->{userid});
die "error in clear: " . $u->errstr . "\n" if $u->err;
LJ::MemCache::delete([$u->{userid}, "auc:$u->{userid}:R"]);
LJ::MemCache::delete([$u->{userid}, "auc:$u->{userid}:K"]);
}
# step 2: get all of their memories and move them, creating the oldmemid -> newmemid mapping
# that we can use in later steps to migrate keywords
my %bindings; # ( oldid => newid )
my $sth = $dbh->prepare('SELECT memid, journalid, jitemid, des, security ' .
'FROM memorable WHERE userid = ?');
$sth->execute($u->{userid});
while (my $row = $sth->fetchrow_hashref()) {
# got a row, good
my $newid = LJ::alloc_user_counter($u, 'R');
die "Error: unable to allocate type 'R' counter for $u->{user}($u->{userid})\n"
unless $newid;
$bindings{$row->{memid}} = $newid;
# push data
push @bind, "($u->{userid}, ?, ?, ?, ?, ?)";
push @vars, ($newid, map { $row->{$_} } qw(journalid jitemid des security));
# flush if necessary
$flush->('memorable2', 'userid, memid, journalid, ditemid, des, security')
if @bind > $BLOCK_INSERT;
}
$flush->('memorable2', 'userid, memid, journalid, ditemid, des, security');
# step 3: get the list of keywords that these memories all use
my %kwmap;
if (%bindings) {
my $memids = join ',', map { $_+0 } keys %bindings;
my $rows = $dbh->selectall_arrayref("SELECT memid, kwid FROM memkeyword WHERE memid IN ($memids)");
push @{$kwmap{$_->[1]}}, $_->[0] foreach @$rows; # kwid -> [ memid, memid, memid ... ]
}
# step 4: get the actual keywords associated with these keyword ids
my %kwidmap;
if (%kwmap) {
my $kwids = join ',', map { $_+0 } keys %kwmap;
my $rows = $dbh->selectall_arrayref("SELECT kwid, keyword FROM keywords WHERE kwid IN ($kwids)");
%kwidmap = map { $_->[0] => $_->[1] } @$rows; # kwid -> keyword
}
# step 5: now migrate all keywords into userkeywords table
my %mappings;
while (my ($kwid, $keyword) = each %kwidmap) {
# reallocate counter
my $newkwid = LJ::alloc_user_counter($u, 'K');
die "Error: unable to allocate type 'K' counter for $u->{user}($u->{userid})\n"
unless $newkwid;
$mappings{$kwid} = $newkwid;
# push data
push @bind, "($u->{userid}, ?, ?)";
push @vars, ($newkwid, $keyword);
# flush if necessary
$flush->('userkeywords', 'userid, kwid, keyword')
if @bind > $BLOCK_INSERT;
}
$flush->('userkeywords', 'userid, kwid, keyword');
# step 6: now we have to do some mapping conversions and put new data into memkeyword2 table
while (my ($oldkwid, $oldmemids) = each %kwmap) {
foreach my $oldmemid (@$oldmemids) {
# get new data
my ($newkwid, $newmemid) = ($mappings{$oldkwid}, $bindings{$oldmemid});
# push data
push @bind, "($u->{userid}, ?, ?)";
push @vars, ($newmemid, $newkwid);
# flush?
$flush->('memkeyword2', 'userid, memid, kwid')
if @bind > $BLOCK_INSERT;
}
}
$flush->('memkeyword2', 'userid, memid, kwid');
# delete memcache keys that hold old data
LJ::MemCache::delete([$u->{userid}, "memkwid:$u->{userid}"]);
# haven't died yet? everything is still going okay, so update dversion
LJ::update_user($u, { 'dversion' => 6 })
or die "error updating dversion";
$u->{'dversion'} = 6; # update local copy in memory
return 1;
};
# get dbh handle
my $dbh = LJ::get_db_writer(); # just so we can get users...
die "Could not connect to global master" unless $dbh;
# get user count
my $total = $dbh->selectrow_array("SELECT COUNT(*) FROM user WHERE dversion = 5");
$stats{'total_users'} = $total+0;
# print out header and total we're moving
print $header->("Moving user data");
print "Processing $stats{'total_users'} total users with the old dversion\n";
# loop until we have no more users to convert
my $ct;
while (1) {
# get blocks of $BLOCK_MOVE users at a time
my $sth = $dbh->prepare("SELECT * FROM user WHERE dversion = 5 LIMIT $BLOCK_MOVE");
$sth->execute();
$ct = 0;
my %us;
my %fast; # people to move fast
while (my $u = $sth->fetchrow_hashref()) {
$us{$u->{userid}} = $u;
$ct++;
$fast{$u->{userid}} = 1;
}
# jump out if we got nothing
last unless $ct;
# now that we have %us, we can see who has data
my $ids = join ',', map { $_+0 } keys %us;
my $has_memorable = $dbh->selectcol_arrayref("SELECT DISTINCT userid FROM memorable WHERE userid IN ($ids)");
my $has_fgroups = $dbh->selectcol_arrayref("SELECT DISTINCT userid FROM friendgroup WHERE userid IN ($ids)");
my %uids = ( map { $_ => 1 } (@$has_memorable, @$has_fgroups) );
# these people actually have data to migrate; don't move them fast
delete $fast{$_} foreach keys %uids;
# now see who we can do in a fast way
my @fast_ids = map { $_+0 } keys %fast;
if (@fast_ids) {
print "Converting ", scalar(@fast_ids), " users quickly...\n";
# update stats for counting and print
$stats{'fast_moved'} += @fast_ids;
print $status->($stats{'slow_moved'}+$stats{'fast_moved'}, $stats{'total_users'}, "users");
# block update
LJ::update_user(\@fast_ids, { dversion => 6 });
}
my $slow_todo = scalar keys %uids;
print "Of $BLOCK_MOVE, $slow_todo have to be slow-converted...\n";
my @ids = randlist(keys %uids);
foreach my $id (@ids) {
# this person has memories, move them the slow way
die "Userid $id in \$has_memorable, but not in \%us...fatal error\n" unless $us{$id};
# now move the user
bless $us{$id}, 'LJ::User';
if ($move_user->($us{$id})) {
$stats{'slow_moved'}++;
print $status->($stats{'slow_moved'}+$stats{'fast_moved'}, $stats{'total_users'}, "users", $us{$id}{user});
}
}
}
# ...done?
print $header->("Dversion 5->6 conversion completed");
print " Users moved: " . $zeropad->($stats{'slow_moved'}) . "\n";
print "Users updated: " . $zeropad->($stats{'fast_moved'}) . "\n\n";
# helper function to randomize stuff
sub randlist
{
my @rlist = @_;
my $size = scalar(@rlist);
my $i;
for ($i=0; $i<$size; $i++) {
unshift @rlist, splice(@rlist, $i+int(rand()*($size-$i)), 1);
}
return @rlist;
}

352
bin/upgrading/d6d7-userpics.pl Executable file
View File

@ -0,0 +1,352 @@
#!/usr/bin/perl
#
use strict;
use Getopt::Long;
$| = 1;
use lib "$ENV{'LJHOME'}/cgi-bin/";
require "ljlib.pl";
use LJ::Blob;
use LJ::User;
use constant DEBUG => 0; # turn on for debugging (mostly db handle crap)
my $BLOCK_MOVE = 5000; # user rows to get at a time before moving
my $BLOCK_INSERT = 25; # rows to insert at a time when moving users
my $BLOCK_UPDATE = 1000; # users to update at a time if they had no data to move
# get options
my %opts;
exit 1 unless
GetOptions("lock=s" => \$opts{locktype},
"user=s" => \$opts{user},
"total=i" => \$opts{total},);
# if no locking, notify them about it
die "ERROR: Lock must be of types 'ddlockd' or 'none'\n"
if $opts{locktype} && $opts{locktype} !~ m/^(?:ddlockd|none)$/;
# used for keeping stats notes
my %stats = (); # { 'stat' => 'value' }
my %handle;
# database handle retrieval sub
my $get_db_handles = sub {
# figure out what cluster to load
my $cid = shift(@_) + 0;
my $dbh = $handle{0};
unless ($dbh) {
$dbh = $handle{0} = LJ::get_dbh({ raw => 1 }, "master");
print "Connecting to master ($dbh)...\n";
eval {
$dbh->do("SET wait_timeout=28800");
};
$dbh->{'RaiseError'} = 1;
}
my $dbcm;
$dbcm = $handle{$cid} if $cid;
if ($cid && ! $dbcm) {
$dbcm = $handle{$cid} = LJ::get_cluster_master({ raw => 1 }, $cid);
print "Connecting to cluster $cid ($dbcm)...\n";
return undef unless $dbcm;
eval {
$dbcm->do("SET wait_timeout=28800");
};
$dbcm->{'RaiseError'} = 1;
}
# return one or both, depending on what they wanted
return $cid ? ($dbh, $dbcm) : $dbh;
};
# percentage complete
my $status = sub {
my ($ct, $tot, $units, $user) = @_;
my $len = length($tot);
my $usertxt = $user ? " Moving user: $user" : '';
return sprintf(" \[%6.2f%%: %${len}d/%${len}d $units]$usertxt\n",
($ct / $tot) * 100, $ct, $tot);
};
my $header = sub {
my $size = 50;
return "\n" .
("#" x $size) . "\n" .
"# $_[0] " . (" " x ($size - length($_[0]) - 4)) . "#\n" .
("#" x $size) . "\n\n";
};
my $zeropad = sub {
return sprintf("%d", $_[0]);
};
# mover function
my $move_user = sub {
my $u = shift;
# make sure our user is of the proper dversion
return 0 unless $u->{'dversion'} == 6;
# at this point, try to get a lock for this user
my $lock;
if ($opts{locktype} eq 'ddlockd') {
$lock = LJ::locker()->trylock("d6d7-$u->{user}");
return 1 unless $lock;
}
# get a handle for every user to revalidate our connection?
my ($dbh, $dbcm) = $get_db_handles->($u->{clusterid});
return 0 unless $dbh;
# assign this dbcm to the user
if ($dbcm) {
$u->set_dbcm($dbcm)
or die "unable to set database for $u->{user}: dbcm=$dbcm\n";
}
# verify dversion hasn't changed on us (done by another job?)
my $dversion = $dbh->selectrow_array("SELECT dversion FROM user WHERE userid = $u->{userid}");
return 1 unless $dversion == 6;
# ignore expunged users
if ($u->{'statusvis'} eq "X" || $u->{'clusterid'} == 0) {
LJ::update_user($u, { dversion => 7 })
or die "error updating dversion";
$u->{dversion} = 7; # update local copy in memory
return 1;
}
return 0 unless $dbcm;
# step 0.5: delete all the bogus userblob rows for this user
# This is due to the auto_increment for the blobid overflowing
# and thus all entries recieving an id of max id for a mediumint.
# This is lame.
my $domainid = LJ::get_blob_domainid('userpic');
$u->do("DELETE FROM userblob WHERE journalid=$u->{userid} AND domain=$domainid AND blobid>=16777216");
die "error in delete: " . $u->errstr . "\n" if $u->err;
# step 1: get all user pictures and move those. safe to just grab with no limit
# since users can only have a limited number of them
my $rows = $dbh->selectall_arrayref('SELECT picid, userid, contenttype, width, height, state, picdate, md5base64 ' .
'FROM userpic WHERE userid = ?', undef, $u->{userid}) || [];
if (@$rows) {
# got some rows, create an update statement
my (@bind, @vars, @blobids, @blobbind, @picinfo);
foreach my $row (@$rows) {
my $picid = $row->[0];
push @bind, "(?, ?, ?, ?, ?, ?, ?, ?)";
$row->[2] = {'image/gif' => 'G',
'image/jpeg' => 'J',
'image/png' => 'P'}->{$row->[2]};
push @vars, @$row;
# [picid, fmt]
my $fmt = {'G' => 'gif',
'J' => 'jpg',
'P' => 'png'}->{$row->[2]};
push @picinfo, [$picid, $fmt];
# picids
push @blobids, $picid;
push @blobbind, "?";
}
my $bind = join ',', @bind;
$u->do("REPLACE INTO userpic2 (picid, userid, fmt, width, height, state, picdate, md5base64) " .
"VALUES $bind", undef, @vars);
die "error in userpic2 replace: " . $u->errstr . "\n" if $u->err;
# step 1.5: insert missing rows into the userblob table
my $blobbind = join ',', @blobbind;
my $blobrows = $dbcm->selectall_hashref("SELECT blobid FROM userblob WHERE journalid=$u->{userid} AND domain=$domainid " .
"AND blobid IN ($blobbind)", 'blobid', undef, @blobids) || {};
my (@insertbind, @insertvars);
foreach my $pic (@picinfo) {
my ($picid, $fmt) = @$pic;
unless ($blobrows->{$picid}) {
push @insertbind, "(?, ?, ?, ?)";
my $blob = LJ::Blob::get($u, "userpic", $fmt, $picid);
my $length = length($blob);
push @insertvars, $u->{'userid'}, $domainid, $picid, $length;
}
}
if (@insertbind) {
my $insertbind = join ',', @insertbind;
$u->do("INSERT IGNORE INTO userblob (journalid, domain, blobid, length) " .
"VALUES $insertbind", undef, @insertvars);
die "error in userblob insert: " . $u->errstr . "\n" if $u->err;
}
}
# general purpose flusher for use below
my (@bind, @vars);
my $flush = sub {
return unless @bind;
my ($table, $cols) = @_;
# insert data into cluster master
my $bind = join(",", @bind);
$u->do("REPLACE INTO $table ($cols) VALUES $bind", undef, @vars);
die "error in flush $table: " . $u->errstr . "\n" if $u->err;
# reset values
@bind = ();
@vars = ();
};
# step 2: get the mapping of all of their keywords
my $kwrows = $dbh->selectall_arrayref('SELECT picid, kwid FROM userpicmap WHERE userid=?',
undef, $u->{'userid'});
my %kwmap;
if (@$kwrows) {
push @{$kwmap{$_->[1]}}, $_->[0] foreach @$kwrows; # kwid -> [ picid, picid, picid ... ]
}
# step 3: get the actual keywords associated with these keyword ids
my %kwidmap;
if (%kwmap) {
my $kwids = join ',', map { $_+0 } keys %kwmap;
my $rows = $dbh->selectall_arrayref("SELECT kwid, keyword FROM keywords WHERE kwid IN ($kwids)");
%kwidmap = map { $_->[0] => $_->[1] } @$rows; # kwid -> keyword
}
# step 4: now migrate all keywords into userkeywords table
my %mappings;
while (my ($kwid, $keyword) = each %kwidmap) {
# reallocate counter
my $newkwid = LJ::get_keyword_id($u, $keyword);
die "Error: unable to get keyword id for $u->{user}($u->{userid}), keyword '$keyword'\n"
unless $newkwid;
$mappings{$kwid} = $newkwid;
}
# step 5: now we have to do some mapping conversions and put new data into userpicmap2 table
while (my ($oldkwid, $picids) = each %kwmap) {
foreach my $picid (@$picids) {
# get new data
my $newkwid = $mappings{$oldkwid};
# push data
push @bind, "($u->{userid}, ?, ?)";
push @vars, ($picid, $newkwid);
# flush?
$flush->('userpicmap2', 'userid, picid, kwid')
if @bind > $BLOCK_INSERT;
}
}
$flush->('userpicmap2', 'userid, picid, kwid');
# delete memcache keys that hold old data
LJ::MemCache::delete([$u->{userid}, "upicinf:$u->{userid}"]);
# haven't died yet? everything is still going okay, so update dversion
LJ::update_user($u, { 'dversion' => 7 })
or die "error updating dversion";
$u->{'dversion'} = 7; # update local copy in memory
return 1;
};
# get dbh handle
my $dbh = LJ::get_db_writer(); # just so we can get users...
die "Could not connect to global master" unless $dbh;
# get user count
my $total = $opts{total} || $dbh->selectrow_array("SELECT COUNT(*) FROM user WHERE dversion = 6");
$stats{'total_users'} = $total+0;
# print out header and total we're moving
print $header->("Moving user data");
print "Processing $stats{'total_users'} total users with the old dversion\n";
# loop until we have no more users to convert
my $ct;
while (1) {
# get users to move
my $sth;
if ($opts{user}) {
$sth = $dbh->prepare("SELECT * FROM user WHERE user = ? AND dversion = 6");
$sth->execute($opts{user});
} else {
$sth = $dbh->prepare("SELECT * FROM user WHERE dversion = 6 LIMIT $BLOCK_MOVE");
$sth->execute();
}
# get blocks of $BLOCK_MOVE users at a time
$ct = 0;
my (%us, %fast);
while (my $u = $sth->fetchrow_hashref()) {
$us{$u->{userid}} = $u;
$fast{$u->{userid}} = 1;
$ct++;
}
# jump out if we got nothing
last unless $ct;
# now that we have %us, we can see who has data
my $ids = join ',', map { $_+0 } keys %us;
my $has_upics = $dbh->selectcol_arrayref("SELECT DISTINCT userid FROM userpic WHERE userid IN ($ids)");
my %uids = ( map { $_ => 1 } (@$has_upics) );
# remove folks that have userpics from the fast list
delete $fast{$_} foreach keys %uids;
# now see who we can do in a fast way
my @fast_ids = map { $_+0 } keys %fast;
if (@fast_ids) {
# update stats for counting and print
$stats{'fast_moved'} += @fast_ids;
print $status->($stats{'slow_moved'}+$stats{'fast_moved'}, $stats{'total_users'}, "users");
# block update
LJ::update_user(\@fast_ids, { dversion => 7 });
}
my $slow_todo = scalar keys %uids;
print "Of $BLOCK_MOVE, $slow_todo have to be slow-converted...\n";
my @ids = randlist(keys %uids);
foreach my $id (@ids) {
# this person has userpics, move them the slow way
die "Userid $id in \$has_upics, but not in \%us...fatal error\n" unless $us{$id};
# now move the user
bless $us{$id}, 'LJ::User';
if ($move_user->($us{$id})) {
$stats{'slow_moved'}++;
print $status->($stats{'slow_moved'}+$stats{'fast_moved'}, $stats{'total_users'}, "users", $us{$id}{user});
}
}
}
# ...done?
print $header->("Dversion 6->7 conversion completed");
print " Users moved: " . $zeropad->($stats{'slow_moved'}) . "\n";
print "Users updated: " . $zeropad->($stats{'fast_moved'}) . "\n\n";
# helper function to randomize stuff
sub randlist
{
my @rlist = @_;
my $size = scalar(@rlist);
my $i;
for ($i=0; $i<$size; $i++) {
unshift @rlist, splice(@rlist, $i+int(rand()*($size-$i)), 1);
}
return @rlist;
}

76
bin/upgrading/d7d8-polls.pl Executable file
View File

@ -0,0 +1,76 @@
#!/usr/bin/perl
#
# Goes over every user, updating their dversion to 8 and
# migrating whatever polls they have to their user cluster
use strict;
use lib "$ENV{'LJHOME'}/cgi-bin/";
require "ljlib.pl";
use LJ::Poll;
use Term::ReadLine;
my $BLOCK_SIZE = 10_000; # get users in blocks of 10,000
my $VERBOSE = 0; # print out extra info
my $dbh = LJ::get_db_writer()
or die "Could not connect to global master";
my $term = new Term::ReadLine 'd7-d8 migrator';
my $line = $term->readline("Do you want to update to dversion 8 (clustered polls)? [N/y] ");
unless ($line =~ /^y/i) {
print "Not upgrading to dversion 8\n\n";
exit;
}
print "\n--- Upgrading users to dversion 8 (clustered polls) ---\n\n";
# get user count
my $total = $dbh->selectrow_array("SELECT COUNT(*) FROM user WHERE dversion = 7");
print "\tTotal users at dversion 7: $total\n\n";
my $migrated = 0;
foreach my $cid (@LJ::CLUSTERS) {
my $udbh = LJ::get_cluster_master($cid)
or die "Could not get cluster master handle for cluster $cid";
while (1) {
my $sth = $dbh->prepare("SELECT userid FROM user WHERE dversion=7 AND clusterid=? LIMIT $BLOCK_SIZE");
$sth->execute($cid);
die $sth->errstr if $sth->err;
my $count = $sth->rows;
print "\tGot $count users on cluster $cid with dversion=7\n";
last unless $count;
while (my ($userid) = $sth->fetchrow_array) {
my $u = LJ::load_userid($userid)
or die "Invalid userid: $userid";
# lock while upgrading
my $lock = LJ::locker()->trylock("d7d8-$userid");
unless ($lock) {
print STDERR "Could not get a lock for user " . $u->user . ".\n";
next;
}
my $ok = eval { $u->upgrade_to_dversion_8 };
$lock->release;
die $@ if $@;
print "\tMigrated user " . $u->user . "... " . ($ok ? 'ok' : 'ERROR') . "\n"
if $VERBOSE;
$migrated++ if $ok;
}
print "\t - Migrated $migrated users so far\n\n";
# make sure we don't end up running forever for whatever reason
last if $migrated > $total;
}
}
print "--- Done migrating $migrated of $total users to dversion 8 ---\n";

View File

@ -0,0 +1,592 @@
# these get removed if found by texttool.pl
#
general crumb.createjournal
general crumb.modify
general crumb.siteopts
general /allpics.bml.edit
general /allpics.bml.nopics.text
general /allpics.bml.nopics.text.other
general /allpics.bml.pics
general /changeemail.bml.label.password
general /community/join.bml.label.authpost
general /community/join.bml.label.closedcomm
general /community/manage.bml.actmembers
general /community/manage.bml.actsettings
general /community/manage.bml.commlist.actmoderate
general /community/members.bml.manage
general /community/search.bml.contasmember
general /community/search.bml.userlikes
general /community/settings.bml.label.admconsole
general /community/settings.bml.manage
general /create.bml.aolnotice.head
general /create.bml.aolnotice.text
general /create.bml.birthday.question
general /create.bml.birthday.question_1
general /create.bml.birthday.question_2
general /create.bml.create.text
general /create.bml.email.text3
general /create.bml.error.seebelow
general /create.bml.password.secure
general /create.bml.password.secure2
general /create.bml.proceed.head
general /create.bml.proceed.text
general /create.bml.title
general /create.bml.tos.error
general /create.bml.tos.haveread
general /create.bml.tos.p1.2
general /create.bml.username.charsallowed
general /create.bml.username.text
general /editinfo.bml.allowshocontact.email.if_domainaddy
general /editinfo.bml.allowshowcontact.email.reason
general /editinfo.bml.newemail.body
general /editinfo.bml.settings.about
general /editinfo.bml.showsubjicons.about
general /editinfo.bml.showsubjicons.header
general /editinfo.bml.tm.about
general /editinfo.bml.schools.title
general /editinfo.bml.topicdir.about
general /editinfo.bml.topicdir.header
general /editpics.bml.error.toomanypics2
general /editpics.bml.error.toomanypics3
general /editpics.bml.restriction.imagesize
entryform.htmlokay.norich
entryform.htmlokay.rich
general editform.userpics
general /friends/add.bml.colors.hover
general /friends/add.bml.confirm.header
general /friends/add.bml.confirm.syn.title
general /friends/add.bml.confirm.text.nobold
general /friends/add.bml.confirm.title
general /friends/add.bml.error3.text
general /friends/add.bml.groups.text
general /freinds/add.bml.remove.text
general /friends/edit.bml.btn.proceed
general /friends/edit.bml.edit.head
general /friends/edit.bml.edit.text
general /friends/index.bml.invite.about
general /friends/nudge.bml.success.text
general /editjournal.bml.enterlogin
general /editjournal.bml.lostinfo.head
general /editjournal.bml.lostinfo.text
general /editjournal_do.bml.error.mustlogin
general /editjournal_do.bml.success.edit
general /friends/editgroups.bml.login.header
general /friends/editgroups.bml.login.text
general /interests.bml.findsim.account.notallowed
general /interests.bml.findsim.account.notloggedin
general /interests.bml.findsim.btn.find
general /interests.bml.findsim.head
general /interests.bml.findsim.simtouser
general /interests.bml.findsim.text
general /interests.bml.interests.findsim1
general /interests.bml.interests.findsim2
general /login.bml.error.forgotpass
general /login.bml.error.noaccount
general /login.bml.loggedin.head
general /login.bml.loggedin.suggest3
general /login.bml.loggedin.text
general /login.bml.login.forget
general /login.bml.login.text1
general /login.bml.login.text2
general /login.bml.login.text3
general /login.bml.title
general /login.bml.whylogin.benefit1
general /login.bml.whylogin.benefit2
general /login.bml.whylogin.benefit3
general /login.bml.whylogin.head
general /login.bml.whylogin.text
general /logout.bml.loggedout.success
general /manage/comments/index.bml.disablecomment2
general /manage/index.bml.information.editinfo.about
general /manage/index.bml.information.setlang.about
general /manage/index.bml.information.setscheme.about
general /manage/index.bml.information.siteopts.about
general /manage/index.bml.userpictures
general /manage/index.bml.userpictures.edit.about
general /manage/index.bml.userpictures.header
general /manage/index.bml.information.emailpost
general /manage/index.bml.information.emailpost.about
general /manage/profile/index.bml.birthday
general /manage/profile/index.bml.intro
general /manage/profile/index.bml.im
general /manage/profile/index.bml.email
general /manage/settings/index.bml.bdayreminder.note
general /manage/siteopts.bml.btn.lang
general /manage/siteopts.bml.btn.scheme
general /manage/siteopts.bml.head.lang
general /manage/siteopts.bml.title
general /manage/siteopts.bml.scheme.preview
general /modify.bml.btn.proceed
general /modify.bml.journalstatus.about
general /modify.bml.journalstatus.head
general /modify.bml.journalstatus.select.activated
general /modify.bml.journalstatus.select.deleted
general /modify.bml.journalstatus.select.head
general /modify.bml.journalstatus.select.suspended
general /modify.bml.lostinfo.head
general /modify.bml.lostinfo.text
general /modify.bml.modify.head
general /modify.bml.modify.text
general /modify_do.bml.title
general /multisearch.bml.region.text
general /register.bml.new.body
general /register.bml.new.modify2
general /setlang.bml.select
general /setlang.bml.switch
general /setlang.bml.title
general /setscheme.bml.current
general /setscheme.bml.default
general /setscheme.bml.noschemes
general /setscheme.bml.select
general /setscheme.bml.switch
general /setscheme.bml.title
general /support/faqbrowse.bml.backfaq
general /support/faqbrowse.bml.backfaqcat
general /support/faqbrowse.bml.backsupport
general /support/faqbrowse.bml.title
general /talkpost.bml.label.picturetouse
general /talkpost_do.bml.opt.spellcheck
general /talkread.bml.pageofpages
general /tools/memadd.bml.body.added.body
general /update.bml.update.about
general /update.bml.login.success
general /update.bml.htmlokay
general /uploadpic.bml.success.text
general /uploadpic.bml.btn.proceed
general /uploadpic.bml.error.badurl
general /uploadpic.bml.error.databasedown
general /uploadpic.bml.error.filetoolarge
general /uploadpic.bml.error.imagetoolarge
general /uploadpic.bml.error.invalidauth
general /uploadpic.bml.error.invalidimage
general /uploadpic.bml.error.toomanypics
general /uploadpic.bml.error.unknownmode
general /uploadpic.bml.error.urlerror
general /uploadpic.bml.fromfile
general /uploadpic.bml.fromfile.key
general /uploadpic.bml.fromurl
general /uploadpic.bml.fromurl.key
general /uploadpic.bml.header
general /uploadpic.bml.imagesize.by
general /uploadpic.bml.kilobytes
general /uploadpic.bml.makedefault
general /uploadpic.bml.makedefault.key
general /uploadpic.bml.restriction.fileformat
general /uploadpic.bml.restriction.filesize
general /uploadpic.bml.restriction.imagesize
general /uploadpic.bml.success.editpics
general /uploadpic.bml.success.header
general /uploadpic.bml.success.preview
general /uploadpic.bml.success.upload
general /uploadpic.bml.text
general /uploadpic.bml.title
general /uploadpic_do.bml.error.urlerror
general uploadpic.error.toolarge
general /userinfo.bml.canpost
general /userinfo.bml.closedmembership.body
general /userinfo.bml.fbpictures
general /userinfo.bml.label.gizmo2
general /userinfo.bml.label.viewlabel
general /userinfo.bml.openmembership.body
general /userinfo.bml.posthere
general /userinfo.bml.rpmembership.body
general /userinfo.bml.syn.lastnew
general /register.bml.error.alreadyvalidated
general /tools/memories.bml.entry
general /tools/memories.bml.entries
general /create.bml.tos.p1
general /create.bml.email.text
general /create.bml.email.note
general /syn/index.bml.table.cost
general /syn/index.bml.quota.text
general /syn/index.bml.quota.your
general /syn/index.bml.quota.numbers
general /syn/index.bml.invalid.overquota
general /syn/index.bml.invalid.overquota.text
general /syn/index.bml.invalid.overquota.title
general /syn/index.bml.invalid.overquota.username
general /syn/index.bml.accounttype.notallowed
general /syn/index.bml.add.other.title
general /syn/index.bml.add.other.text
general /syn/index.bml.notused
general /userinfo.bml.syn.cost
general bml.needlogin.body
general bml.needlogin.body2
general /userinfo.bml.label.frsyndication
general /syn/index.bml.invalid.address
general /community/members.bml.key.name
general /community/members.bml.success.addusers
general /userinfo.bml.label.totalfriends
general /community/settings.bml.label.closedmemb
general /talkscreen.bml.title
general /community/members.bml.success.message
general /community/transfer.bml.error.banned
general /create.bml.age.check.question
general /create.bml.age.check.yes
general /create.bml.age.check2.question
general /create.bml.age.check2.yes
general /create.bml.age.head
general /community/members.bml.reinvited
general /community/membres.bml.success.invited
general /customize/index.bml.s1
general /changepassword.bml.email.body
general Success
general /schools/index.bml.addschool.input.name.rule1
general /schools/index.bml.addschool.input.name.rule2
general /schools/index.bml.addschool.input.name.rule3
general /schools/index.bml.addschool.input.name.rule4
general /schools/index.bml.addschool.input.name.rule5
general /schools/index.bml.addschool.input.name.campus
general /schools/index.bml.addschool.input.name.capitalize
general /schools/index.bml.addschool.input.name.fullname
general /schools/index.bml.addschool.input.name.noabbrev
general /schools/index.bml.addschool.input.name.nothe
general /schools/index.bml.intro.dontseeschool
general /schools/index.bml.intro.dontseeschool2
general /community/join.bml.label.loginfirst
general /community/join.bml.label.membernow
general /community/leave.bml.label.logoutfirst
general /community/leave.bml.label.removed
general /community/settings.bml.label.commcreate
general /community/settings.bml.label.createtext
general /community/settings.bml.label.maintainer.login
general /friends/add.bml.error1.header
general /friends/add.bml.error1.text
general /friends/add.bml.error2.text
general /friends/filter.bml.error.nogroups
general /manage/index.bml.login
general /portal/index.bml.notloggedin
general /portal/index.bml.notloggedintitle
general /syn/index.bml.loginrequired.text
general /syn/index.bml.loginrequired.title
general /tools/emailmanage.bml.notvalidated.text
general /tools/memadd.bml.error.login
general /directory.bml.error.accounttype
general /support/append_request.bml.back.requests
general /support/append_request.bml.back.support
general /support/append_request.bml.login.required
general /support/encodings.bml.edit.text
general /support/encodings.bml.groups.text
general /support/encodings.bml.still.text
general /editinfo.bml.error.invalidbio
general /editinfo.bml.error.invalidints
general /editinfo.bml.error.invalidname
general /editinfo.bml.persinfo.disclaimer
general /editinfo.bml.success.message
general /editinfo.bml.tm.details
general /editinfo.bml.userpic.edit
general /userinfo.bml.body.leave
general /userinfo.bml.error.notloggedin
general /userinfo.bml.membership.body
general /interests.bml.addint
general /interests.bml.error.enmasse.mustlogin
general /interests.bml.match
general /interests.bml.matches
general /interests.bml.morestuff
general /interests.bml.results.goback
general /interests.bml.results.message
general /login.bml.error.mustenterusername
general /login.bml.links.link1
general /login.bml.links.link2
general /modify_do.bml.overrides.note
general /modify_do.bml.journaloptions.about
general /multisearch.bml.region.bodytext
general /register.bml.new.editinfo
general /register.bml.new.login
general /register.bml.new.modify
general /register.bml.new.update
general /talkpost_do.bml.error.badpassword
general /talkpost_do.bml.error.badusername
general /talkpost_do.bml.error.noverify
general /update.bml.currmood
general /update.bml.update.alternate
general /editpics.bml.curpics.desc
general error.noremote
general bml.needlogin.body3
general bml.needlogin.body4
general /allpics.bml.edit2
general /allpics.bml.nopics.text2
general /community/index.bml.main
general /community/manage.bml.commlist.actmembers
general /community/manage.bml.commlist.actsettings
general /community/manage.bml.commlist.moderation.num
general /community/manage.bml.create.text
general /community/members.bml.settings
general /community/members.bml.success.return
general /community/settings.bml.members
general /editinfo.bml.allowshowcontact.email.withdomainaddr
general /customize/index.bml.s2.advanced.permitted
general /editinfo.bml.login.enterinfo
general /editinfo.bml.login.forgot.header
general /editinfo.bml.login.forgot.recover
general /editjournal_do.bml.currmood
general /editjournal_do.bml.picture
general /friends/add.bml.add.text
general /friends/add.bml.add.text.community
general /friends/add.bml.add.text.feed
general /friends/add.bml.confirm.text1.community
general /friends/edit_do.bml.success.text
general /interests.bml.nointerests.text
general /modify_do.bml.overrides.about
general /modify_do.bml.success.text
general /poll/index.bml.gotocreate
general /support/append_request.bml.successlinks
general /talkmulti.bml.deleted.body
general /talkmulti.bml.screened.body
general /talkmulti.bml.unscreened.body
general /talkpost_do.bml.success.message
general /talkpost_do.bml.success.screened.comm
general /talkpost_do.bml.success.screened.comm.anon
general /talkpost_do.bml.success.screened.user
general /talkpost_do.bml.success.screened.user.anon
general /tools/memadd.bml.login.forgot.header
general /tools/memadd.bml.login.forgot.recover
general /update.bml.loggedinas
general /update.bml.picture
general /update.bml.simple
general /update.bml.update.success
general /userinfo.bml.label.clientsused
general /userinfo.bml.sendmessage.body
general /userinfo.bml.syndinfo.body
general /userinfo.bml.userinfo.body
general lostinfo.text
general /customize/index.bml.s2.advanced.denied
general /styles/create.bml.createstyle.text
general /styles/index.bml.about
general /support/see_overrides.bml.header
general /poll/create.bml.error.accttype
general /editpics.bml.noneupload
general /allpics.bml.edit3
general birthday.link
general birthday.text
general controlstrip.link
general controlstrip.text
general feeds.link
general feeds.text
general ljfeedback.text
general ljspotlight.text
general polls.link
general polls.text
general cprod.friendsfriends.text.v1
general cprod.friendsfriends.text.v2
general cprod.friendsfriends.text.v3
general cprod.friendsfriends.text.v4
general cprod.friendsfriends.text.v5
general cprod.friendsfriends.link.v1
general cprod.friendsfriends.link.v2
general cprod.friendsfriends.link.v3
general cprod.friendsfriends.link.v4
general cprod.friendsfriends.link.v5
general /tools/textmessage.bml.enter.user.text
general cprod.friendsfriendsinline.link2.v1
general cprod.friendsfriendsinline.link2.v2
general cprod.friendsfriendsinline.link2.v3
general cprod.friendsfriendsinline.link2.v4
general cprod.friendsfriendsinline.link2.v5
general cprod.friendsfriendsinline.text2.v1
general cprod.friendsfriendsinline.text2.v2
general cprod.friendsfriendsinline.text2.v3
general cprod.friendsfriendsinline.text2.v4
general cprod.friendsfriendsinline.text2.v5
general cprod.links.text2.v1
general cprod.textmessaging.text2.v1
general web.controlstrip.links.changecommsettings
general /community/join.bml.label.addtofriends
general /community/join.bml.label.allowposting
general /community/join.bml.label.auth
general /community/join.bml.label.expls
general /community/join.bml.label.membernow2
general /community/join.bml.label.sure
general /userinfo.bml.monitor.comm
general cprod.directory.text.v1
general cprod.directory.text.v2
general cprod.directory.text.v3
general cprod.directory.text.v4
general cprod.directory.text.v5
general /customize/index.bml.error.disallowed_theme_layer
general /customize/index.bml.error.not_your_layout
general /community/join.bml.button.join
general /community/join.bml.label.addtofriends.note
general /community/join.bml.label.addtofriends2
general /community/join.bml.label.membernow3
general /community/leave.bml.label.removed2
general /community/leave.bml.label.removed.stopwatch
general /community/join.bml.label.membernow4
general /community/leave.bml.label.removed3
general /community/leave.bml.label.removed.stopwatch2
general /community/join.bml.label.membernow5
general /community/leave.bml.label.removed4
general /community/leave.bml.label.removed.stopwatch3
general /customize/preview.bml.unavailable
general /manage/profile/index.bml.fn.interests
general /manage/profile/index.bml.fn.interests2
general /manage/profile/index.bml.interest.line1
general /manage/profile/index.bml.interest.line2
general /manage/profile/index.bml.interest.line3
general /manage/profile/index.bml.interest.line4
general /manage/profile/index.bml.interest.line5
general /manage/profile/index.bml.pop_interests
general /manage/profile/index.bml.pop_interests2
general /manage/profile/index.bml.pop_interests3
general /manage/profile/index.bml.pop_interests4
general /manage/profile/index.bml.pop_interests5
general /manage/profile/index.bml.pop_interests6
general /manage/profile/index.bml.pop_interests7
general /manage/profile/index.bml.pop_interests8
general /manage/profile/index.bml.pop_interests9
general /manage/profile/index.bml.pop_interests10
general /manage/profile/index.bml.pop_interests11
general /manage/profile/index.bml.pop_interests12
general /manage/profile/index.bml.pop_interests13
general /manage/profile/index.bml.pop_interests14
general /manage/profile/index.bml.pop_interests15
general /manage/profile/index.bml.pop_interests16
general /manage/profile/index.bml.pop_interests17
general /manage/profile/index.bml.pop_interests18
general /manage/profile/index.bml.pop_interests19
general /manage/profile/index.bml.pop_interests20
general /manage/profile/index.bml.pop_interests21
general /friends/invite.bml.error.needcreatelink
general email.newacct3.body
general /manage/profile/index.bml.fn.imservices
general /manage/profile/index.bml.pop_interests1.v2
general /manage/profile/index.bml.pop_interests2.v2
general /manage/profile/index.bml.pop_interests3.v2
general /manage/profile/index.bml.pop_interests4.v2
general /manage/profile/index.bml.pop_interests5.v2
general /manage/profile/index.bml.pop_interests6.v2
general /manage/profile/index.bml.pop_interests7.v2
general /manage/profile/index.bml.pop_interests8.v2
general /manage/profile/index.bml.pop_interests9.v2
general /manage/profile/index.bml.pop_interests10.v2
general /manage/profile/index.bml.pop_interests11.v2
general /manage/profile/index.bml.pop_interests12.v2
general /manage/profile/index.bml.pop_interests13.v2
general /manage/profile/index.bml.pop_interests14.v2
general /manage/profile/index.bml.pop_interests15.v2
general /manage/profile/index.bml.pop_interests16.v2
general /manage/profile/index.bml.pop_interests17.v2
general /manage/profile/index.bml.pop_interests18.v2
general /manage/profile/index.bml.pop_interests19.v2
general /manage/profile/index.bml.pop_interests20.v2
general /manage/profile/index.bml.pop_interests21.v2
general /htdocs/register.bml.new.customize
general /htdocs/register.bml.new.editinfo2
general /htdocs/register.bml.new.header
general /htdocs/register.bml.new.login2
general /htdocs/register.bml.new.update2
general /htdocs/register.bml.new.customize2
general /htdocs/register.bml.new.editinfo3
general /htdocs/register.bml.new.modify2
general /htdocs/register.bml.new.search
general /htdocs/register.bml.new.update3
general /htdocs/register.bml.new.userpics
general widget.qotd.title
general widget.sitemessages.title
general widget.themechooser.btn.layout_filter
general widget.themechooser.layout_filter.label
general /manage/settings/index.bml.safesearch
general /manage/settings/index.bml.safesearch.select.concepts
general /manage/settings/index.bml.safesearch.select.explicit
general widget.verticalentries.title
general widget.verticalentries.nocomments
general widget.customizetheme.linkslist.manage
general widget.customizetheme.linkslist

2487
bin/upgrading/en.dat Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,48 @@
#!/usr/bin/perl
# This script goes through all of the files in your include directory
# (LJHOME/htdocs/inc) and then imports ones that are specified by your
# ljconfig.pl file (%LJ::FILEEDIT_VIA_DB) into your database if the file
# on disk is newer than the one in the database.
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
# create list of files to check
my $dir = "$ENV{'LJHOME'}/htdocs/inc";
print "searching for files to check against database...";
opendir DIR, $dir
or die "Unable to open $ENV{'LJHOME'}/htdocs/inc for searching.\n";
my @files = grep { $LJ::FILEEDIT_VIA_DB ||
$LJ::FILEEDIT_VIA_DB{$_} } readdir(DIR);
my $count = scalar(@files);
print $count+0 . " found.\n";
# now iterate through and check times
my $dbh = LJ::get_db_writer();
foreach my $file (@files) {
my $path = "$dir/$file";
next unless -f $path;
# now get filetime
my $ftimedisk = (stat($path))[9];
my $ftimedb = $dbh->selectrow_array("SELECT updatetime
FROM includetext WHERE incname=?", undef, $file)+0;
# check
if ($ftimedisk > $ftimedb) {
# load file
open FILE, "<$path";
my $content = join("", <FILE>);
close FILE;
# now do SQL
print "$file newer on disk...updating database...";
$dbh->do("REPLACE INTO includetext (incname, inctext, updatetime)" .
"VALUES (?,?,UNIX_TIMESTAMP())", undef, $file, $content);
print $dbh->err ? "error: " . $dbh->errstr . ".\n" : "done.\n";
} else {
print "$file newer in database, ignored.\n";
}
}
print "all done.\n";

54
bin/upgrading/make_system.pl Executable file
View File

@ -0,0 +1,54 @@
#!/usr/bin/perl
#
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $dbh = LJ::get_dbh("master");
print "
This tool will create your LiveJournal 'system' account and
set its password. Or, if you already have a system user, it'll change
its password to whatever you specify.
";
print "Enter password for the 'system' account: ";
my $pass = <STDIN>;
chomp $pass;
print "\n";
print "Creating system account...\n";
unless (LJ::create_account({ 'user' => 'system',
'name' => 'System Account',
'password' => $pass }))
{
print "Already exists.\nModifying 'system' account...\n";
my $id = LJ::get_userid("system");
$dbh->do("UPDATE password SET password=? WHERE userid=?",
undef, $pass, $id);
}
my $u = LJ::load_user("system");
unless ($u) {
print "ERROR: can't find newly-created system account.\n";
exit 1;
}
print "Giving 'system' account 'admin' priv on all areas...\n";
if (LJ::check_priv($u, "admin", "*")) {
print "Already has it.\n";
} else {
my $sth = $dbh->prepare("INSERT INTO priv_map (userid, prlid, arg) ".
"SELECT $u->{'userid'}, prlid, '*' ".
"FROM priv_list WHERE privcode='admin'");
$sth->execute;
if ($dbh->err || $sth->rows == 0) {
print "Couldn't grant system account admin privs\n";
exit 1;
}
}
print "Done.\n\n";

View File

@ -0,0 +1,323 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
require 'ljlib.pl';
use LJ::Blob;
use LJ::User;
use Getopt::Long;
use IPC::Open3;
use Digest::MD5;
# this script is a migrater that will move phone posts from an old storage method
# into mogilefs.
# the basic theory is that we iterate over all clusters, find all phoneposts that
# aren't in mogile right now, and put them there
# determine
my ($one, $besteffort, $dryrun, $user, $verify, $verbose, $clusters, $purge);
my $rv = GetOptions("best-effort" => \$besteffort,
"one" => \$one,
"dry-run" => \$dryrun,
"user=s" => \$user,
"verify" => \$verify,
"verbose" => \$verbose,
"purge-old" => \$purge,
"clusters=s" => \$clusters,);
unless ($rv) {
die <<ERRMSG;
This script supports the following command line arguments:
--clusters=X[-Y]
Only handle clusters in this range. You can specify a single
number, or a range of two numbers with a dash.
--user=username
Only move this particular user.
--one
Only move one user. (But it moves all their phone posts.) This is
used for testing.
--verify
If specified, this option will reload the phonepost from MogileFS and
make sure it's been stored successfully.
--dry-run
If on, do not update the database. This mode will put the phonepost
in MogileFS and give you paths to examine the phone posts and make
sure everything is okay. It will not update the phonepost2 table,
though.
--best-effort
Normally, if a problem is encountered (null phonepost, md5 mismatch,
connection failure, etc) the script will die to make sure
everything goes well. With this flag, we don't die and instead
just print to standard error.
--purge-old
Sometimes we run into data that is for users that have since
moved to a different cluster. Normally we ignore it, but
with this option, we'll clean that data up as we find it.
--verbose
Be very chatty.
ERRMSG
}
# make sure ljconfig is setup right (or so we hope)
die "Please define a 'phoneposts' class in your \%LJ::MOGILEFS_CONFIG\n"
unless defined $LJ::MOGILEFS_CONFIG{classes}->{phoneposts};
die "Unable to find MogileFS object (\%LJ::MOGILEFS_CONFIG not setup?)\n"
unless $LJ::MogileFS;
# setup stderr if we're in best effort mode
if ($besteffort) {
my $oldfd = select(STDERR);
$| = 1;
select($oldfd);
}
# operation modes
if ($user) {
# move a single user
my $u = LJ::load_user($user);
die "No such user: $user\n" unless $u;
handle_userid($u->{userid}, $u->{clusterid});
} else {
# parse the clusters
my @clusters;
if ($clusters) {
if ($clusters =~ /^(\d+)(?:-(\d+))?$/) {
my ($min, $max) = map { $_ + 0 } ($1, $2 || $1);
push @clusters, $_ foreach $min..$max;
} else {
die "Error: --clusters argument not of right format.\n";
}
} else {
@clusters = @LJ::CLUSTERS;
}
# now iterate over the clusters to pick
my $ctotal = scalar(@clusters);
my $ccount = 0;
foreach my $cid (sort { $a <=> $b } @clusters) {
# status report
$ccount++;
print "\nChecking cluster $cid...\n\n";
# get a handle
my $dbcm = get_db_handle($cid);
# get all userids
print "Getting userids...\n";
my $limit = $one ? 'LIMIT 1' : '';
my $userids = $dbcm->selectcol_arrayref
("SELECT DISTINCT userid FROM phonepostentry WHERE (location='blob' OR location IS NULL) $limit");
my $total = scalar(@$userids);
# iterate over userids
my $count = 0;
print "Beginning iteration over userids...\n";
foreach my $userid (@$userids) {
# move this phonepost
my $extra = sprintf("[%6.2f%%, $ccount of $ctotal] ", (++$count/$total*100));
handle_userid($userid, $cid, $extra);
}
# don't hit up more clusters
last if $one;
}
}
print "\n";
print "Updater terminating.\n";
#############################################################################
### helper subs down here
# take a userid and move their phone posts
sub handle_userid {
my ($userid, $cid, $extra) = @_;
# load user to move and do some sanity checks
my $u = LJ::load_userid($userid);
unless ($u) {
LJ::end_request();
LJ::start_request();
$u = LJ::load_userid($userid);
}
die "ERROR: Unable to load userid $userid\n"
unless $u;
# if they're expunged, they might have data somewhere if they were
# copy-moved from A to B, then expunged on B. now we're on A and
# need to delete it ourselves (if purge-old is on)
if ($u->{clusterid} == 0 && $u->{statusvis} eq "X") {
return unless $purge;
# if we get here, the user has indicated they want data purged, get handle
my $to_purge_dbcm = get_db_handle($cid);
my $ct = $to_purge_dbcm->do("DELETE FROM phonepostentry WHERE userid = ?", undef, $u->{userid});
print "\tnotice: purged $ct old rows.\n\n"
if $verbose;
return;
}
# get a handle
my $dbcm = get_db_handle($u->{clusterid});
# print that we're doing this user
print "$extra$u->{user}($u->{userid})\n";
# if a user has been moved to another cluster, but the source data from
# phonepostentry wasn't deleted, we need to ignore the user or purge their data
if ($u->{clusterid} != $cid) {
return unless $purge;
# verify they have some rows on the new side
my $count = $dbcm->selectrow_array
("SELECT COUNT(*) FROM phonepostentry WHERE userid = ?",
undef, $u->{userid});
return unless $count;
# if we get here, the user has indicated they want data purged, get handle
my $to_purge_dbcm = get_db_handle($cid);
# delete the old data
if ($dryrun) {
print "\tnotice: need to delete phonepostentry rows.\n\n"
if $verbose;
} else {
my $ct = $to_purge_dbcm->do("DELETE FROM phonepostentry WHERE userid = ?", undef, $u->{userid});
print "\tnotice: purged $ct old rows.\n\n"
if $verbose;
}
# nothing else to do here
return;
}
# get all their photos that aren't in mogile already
my $rows = $dbcm->selectall_arrayref
("SELECT filetype, blobid FROM phonepostentry WHERE userid = ? ".
"AND (location = 'blob' OR location IS NULL)",
undef, $u->{userid});
return unless @$rows;
# if a user has been moved to another cluster, but the source data from
# phonepost2 wasn't deleted, we need to ignore the user
return unless $u->{clusterid} == $cid;
# now we have a userid and blobids, get the photos from the blob server
foreach my $row (@$rows) {
my ($filetype, $blobid) = @$row;
print "\tstarting move for blobid $blobid\n"
if $verbose;
my $format = { 0 => 'mp3', 1 => 'ogg', 2 => 'wav' }->{$filetype};
my $data = LJ::Blob::get($u, "phonepost", $format, $blobid);
# get length
my $len = length($data);
if (! $len) {
my $has_blob = $dbcm->selectrow_array("SELECT COUNT(*) FROM userblob WHERE ".
"journalid=? AND domain=? AND blobid=?",
undef, $u->{userid},
LJ::get_blob_domainid("phonepost"),
$blobid);
if (! $has_blob) {
$dbcm->do("UPDATE phonepostentry SET location='none' ".
"WHERE userid=? AND blobid=?", undef, $u->{userid}, $blobid);
print "\tnote: changed phonepost entry location to 'none'\n\n"
if $verbose;
next;
}
}
if ($besteffort && !$len) {
print STDERR "empty_phonepost userid=$u->{userid} blobid=$blobid\n";
print "\twarning: empty phonepost.\n\n"
if $verbose;
next;
}
die "Error: data from blob empty ($u->{user}, 'phonepost', $format, $blobid)\n"
unless $len;
# get filehandle to Mogile and put the file there
print "\tdata length = $len bytes, uploading to MogileFS...\n"
if $verbose;
my $fh = $LJ::MogileFS->new_file("pp:$u->{userid}:$blobid", 'phoneposts');
if ($besteffort && !$fh) {
print STDERR "new_file_failed userid=$u->{userid} blobid=$blobid\n";
print "\twarning: failed in call to new_file\n\n"
if $verbose;
next;
}
die "Unable to get filehandle to save file to MogileFS\n"
unless $fh;
# now save the file and close the handles
$fh->print($data);
my $rv = $fh->close;
if ($besteffort && !$rv) {
print STDERR "close_failed userid=$u->{userid} blobid=$blobid reason=$@\n";
print "\twarning: failed in call to cloes: $@\n\n"
if $verbose;
next;
}
die "Unable to save file to MogileFS: $@\n"
unless $rv;
# extra verification
if ($verify) {
my $data2 = $LJ::MogileFS->get_file_data("pp:$u->{userid}:$blobid");
my $eq = ($data2 && $$data2 eq $data) ? 1 : 0;
if ($besteffort && !$eq) {
print STDERR "verify_failed userid=$u->{userid} blobid=$blobid\n";
print "\twarning: verify failed; phone post not updated\n\n"
if $verbose;
next;
}
die "\tERROR: phone post NOT stored successfully, content mismatch\n"
unless $eq;
print "\tverified length = " . length($$data2) . " bytes...\n"
if $verbose;
}
# done moving this phone post
unless ($dryrun) {
print "\tupdating database for this phone post...\n"
if $verbose;
$dbcm->do("UPDATE phonepostentry SET location = 'mogile' WHERE userid = ? AND blobid = ?",
undef, $u->{userid}, $blobid);
}
# get the paths so the user can verify if they want
if ($verbose) {
my @paths = $LJ::MogileFS->get_paths("pp:$u->{userid}:$blobid", 1);
print "\tverify mogile path: $_\n" foreach @paths;
print "\tphone post update complete.\n\n";
}
}
}
# a sub to get a cluster handle and set it up for our use
sub get_db_handle {
my $cid = shift;
my $dbcm = LJ::get_cluster_master({ raw => 1 }, $cid);
unless ($dbcm) {
print STDERR "handle_unavailable clusterid=$cid\n";
die "ERROR: unable to get raw handle to cluster $cid\n";
}
eval {
$dbcm->do("SET wait_timeout = 28800");
die $dbcm->errstr if $dbcm->err;
};
die "Couldn't set wait_timeout on $cid: $@\n" if $@;
$dbcm->{'RaiseError'} = 1;
return $dbcm;
}

318
bin/upgrading/migrate-userpics.pl Executable file
View File

@ -0,0 +1,318 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{LJHOME}/cgi-bin";
require 'ljlib.pl';
use LJ::Blob;
use LJ::User;
use Getopt::Long;
use IPC::Open3;
use Digest::MD5;
# this script is a migrater that will move userpics from an old storage method
# into mogilefs.
# the basic theory is that we iterate over all clusters, find all userpics that
# aren't in mogile right now, and put them there
# determine
my ($one, $besteffort, $dryrun, $user, $verify, $verbose, $clusters, $purge);
my $rv = GetOptions("best-effort" => \$besteffort,
"one" => \$one,
"dry-run" => \$dryrun,
"user=s" => \$user,
"verify" => \$verify,
"verbose" => \$verbose,
"purge-old" => \$purge,
"clusters=s" => \$clusters,);
unless ($rv) {
die <<ERRMSG;
This script supports the following command line arguments:
--clusters=X[-Y]
Only handle clusters in this range. You can specify a
single number, or a range of two numbers with a dash.
--user=username
Only move this particular user.
--one
Only move one user. (But it moves all their pictures.)
This is used for testing.
--verify
If specified, this option will reload the userpic from
MogileFS and make sure it's been stored successfully.
--dry-run
If on, do not update the database. This mode will put the
userpic in MogileFS and give you paths to examine the picture
and make sure everything is okay. It will not update the
userpic2 table, though.
--best-effort
Normally, if a problem is encountered (null userpic, md5
mismatch, connection failure, etc) the script will die to
make sure everything goes well. With this flag, we don't
die and instead just print to standard error.
--purge-old
Sometimes we run into data that is for users that have since
moved to a different cluster. Normally we ignore it, but
with this option, we'll clean that data up as we find it.
--verbose
Be very chatty.
ERRMSG
}
# make sure ljconfig is setup right (or so we hope)
die "Please define a 'userpics' class in your \%LJ::MOGILEFS_CONFIG\n"
unless defined $LJ::MOGILEFS_CONFIG{classes}->{userpics};
die "Unable to find MogileFS object (\%LJ::MOGILEFS_CONFIG not setup?)\n"
unless $LJ::MogileFS;
# setup stderr if we're in best effort mode
if ($besteffort) {
my $oldfd = select(STDERR);
$| = 1;
select($oldfd);
}
# operation modes
if ($user) {
# move a single user
my $u = LJ::load_user($user);
die "No such user: $user\n" unless $u;
handle_userid($u->{userid}, $u->{clusterid});
} else {
# parse the clusters
my @clusters;
if ($clusters) {
if ($clusters =~ /^(\d+)(?:-(\d+))?$/) {
my ($min, $max) = map { $_ + 0 } ($1, $2 || $1);
push @clusters, $_ foreach $min..$max;
} else {
die "Error: --clusters argument not of right format.\n";
}
} else {
@clusters = @LJ::CLUSTERS;
}
# now iterate over the clusters to pick
my $ctotal = scalar(@clusters);
my $ccount = 0;
foreach my $cid (sort { $a <=> $b } @clusters) {
# status report
$ccount++;
print "\nChecking cluster $cid...\n\n";
# get a handle
my $dbcm = get_db_handle($cid);
# get all userids
print "Getting userids...\n";
my $limit = $one ? 'LIMIT 1' : '';
my $userids = $dbcm->selectcol_arrayref
("SELECT DISTINCT userid FROM userpic2 WHERE location <> 'mogile' OR location IS NULL $limit");
my $total = scalar(@$userids);
# iterate over userids
my $count = 0;
print "Beginning iteration over userids...\n";
foreach my $userid (@$userids) {
# move this userpic
my $extra = sprintf("[%6.2f%%, $ccount of $ctotal] ", (++$count/$total*100));
handle_userid($userid, $cid, $extra);
}
# don't hit up more clusters
last if $one;
}
}
print "\n";
print "Updater terminating.\n";
#############################################################################
### helper subs down here
# take a userid and move their pictures. returns 0 on error, 1 on successful
# move of a user's pictures, and 2 meaning the user isn't ready for moving
# (dversion < 7, etc)
sub handle_userid {
my ($userid, $cid, $extra) = @_;
# load user to move and do some sanity checks
my $u = LJ::load_userid($userid);
unless ($u) {
LJ::end_request();
LJ::start_request();
$u = LJ::load_userid($userid);
}
die "ERROR: Unable to load userid $userid\n"
unless $u;
# if they're expunged, they might have data somewhere if they were
# copy-moved from A to B, then expunged on B. now we're on A and
# need to delete it ourselves (if purge-old is on)
if ($u->{clusterid} == 0 && $u->{statusvis} eq "X") {
return unless $purge;
# if we get here, the user has indicated they want data purged, get handle
my $to_purge_dbcm = get_db_handle($cid);
my $ct = $to_purge_dbcm->do("DELETE FROM userpic2 WHERE userid = ?", undef, $u->{userid});
print "\tnotice: purged $ct old rows.\n\n"
if $verbose;
return;
}
# get a handle
my $dbcm = get_db_handle($u->{clusterid});
# print that we're doing this user
print "$extra$u->{user}($u->{userid})\n";
# if a user has been moved to another cluster, but the source data from
# userpic2 wasn't deleted, we need to ignore the user or purge their data
if ($u->{clusterid} != $cid) {
return unless $purge;
# verify they have some rows on the new side
my $count = $dbcm->selectrow_array
("SELECT COUNT(*) FROM userpic2 WHERE userid = ?",
undef, $u->{userid});
return unless $count;
# if we get here, the user has indicated they want data purged, get handle
my $to_purge_dbcm = get_db_handle($cid);
# delete the old data
if ($dryrun) {
print "\tnotice: need to delete userpic2 rows.\n\n"
if $verbose;
} else {
my $ct = $to_purge_dbcm->do("DELETE FROM userpic2 WHERE userid = ?", undef, $u->{userid});
print "\tnotice: purged $ct old rows.\n\n"
if $verbose;
}
# nothing else to do here
return;
}
# get all their photos that aren't in mogile already
my $picids = $dbcm->selectall_arrayref
("SELECT picid, md5base64, fmt FROM userpic2 WHERE userid = ? AND (location <> 'mogile' OR location IS NULL)",
undef, $u->{userid});
return unless @$picids;
# now we have a userid and picids, get the photos from the blob server
foreach my $row (@$picids) {
my ($picid, $md5, $fmt) = @$row;
print "\tstarting move for picid $picid\n"
if $verbose;
my $format = { G => 'gif', J => 'jpg', P => 'png' }->{$fmt};
my $data = LJ::Blob::get($u, "userpic", $format, $picid);
# get length
my $len = length($data);
if ($besteffort && !$len) {
print STDERR "empty_userpic userid=$u->{userid} picid=$picid\n";
print "\twarning: empty userpic.\n\n"
if $verbose;
next;
}
die "Error: data from blob empty ($u->{user}, 'userpic', $format, $picid)\n"
unless $len;
# verify the md5 of this picture with what's in the database
my $blobmd5 = Digest::MD5::md5_base64($data);
if ($besteffort && ($md5 ne $blobmd5)) {
print STDERR "md5_mismatch userid=$u->{userid} picid=$picid dbmd5=$md5 blobmd5=$blobmd5\n";
print "\twarning: md5 mismatch; database=$md5, blobserver=$blobmd5\n\n"
if $verbose;
next;
}
die "\tError: data from blobserver md5 mismatch: database=$md5, blobserver=$blobmd5\n"
unless $md5 eq $blobmd5;
print "\tverified md5; database=$md5, blobserver=$blobmd5\n"
if $verbose;
# get filehandle to Mogile and put the file there
print "\tdata length = $len bytes, uploading to MogileFS...\n"
if $verbose;
my $fh = $LJ::MogileFS->new_file($u->mogfs_userpic_key($picid), 'userpics');
if ($besteffort && !$fh) {
print STDERR "new_file_failed userid=$u->{userid} picid=$picid\n";
print "\twarning: failed in call to new_file\n\n"
if $verbose;
next;
}
die "Unable to get filehandle to save file to MogileFS\n"
unless $fh;
# now save the file and close the handles
$fh->print($data);
my $rv = $fh->close;
if ($besteffort && !$rv) {
print STDERR "close_failed userid=$u->{userid} picid=$picid reason=$@\n";
print "\twarning: failed in call to cloes: $@\n\n"
if $verbose;
next;
}
die "Unable to save file to MogileFS: $@\n"
unless $rv;
# extra verification
if ($verify) {
my $data2 = $LJ::MogileFS->get_file_data($u->mogfs_userpic_key($picid));
my $eq = ($data2 && $$data2 eq $data) ? 1 : 0;
if ($besteffort && !$eq) {
print STDERR "verify_failed userid=$u->{userid} picid=$picid\n";
print "\twarning: verify failed; picture not updated\n\n"
if $verbose;
next;
}
die "\tERROR: picture NOT stored successfully, content mismatch\n"
unless $eq;
print "\tverified length = " . length($$data2) . " bytes...\n"
if $verbose;
}
# done moving this picture
unless ($dryrun) {
print "\tupdating database for this picture...\n"
if $verbose;
$dbcm->do("UPDATE userpic2 SET location = 'mogile' WHERE userid = ? AND picid = ?",
undef, $u->{userid}, $picid);
}
# get the paths so the user can verify if they want
if ($verbose) {
my @paths = $LJ::MogileFS->get_paths($u->mogfs_userpic_key($picid), 1);
print "\tverify mogile path: $_\n" foreach @paths;
print "\tverify site url: $LJ::SITEROOT/userpic/$picid/$u->{userid}\n";
print "\tpicture update complete.\n\n";
}
}
}
# a sub to get a cluster handle and set it up for our use
sub get_db_handle {
my $cid = shift;
my $dbcm = LJ::get_cluster_master({ raw => 1 }, $cid);
unless ($dbcm) {
print STDERR "handle_unavailable clusterid=$cid\n";
die "ERROR: unable to get raw handle to cluster $cid\n";
}
eval {
$dbcm->do("SET wait_timeout = 28800");
die $dbcm->errstr if $dbcm->err;
};
die "Couldn't set wait_timeout on $cid: $@\n" if $@;
$dbcm->{'RaiseError'} = 1;
return $dbcm;
}

162
bin/upgrading/migrate-userprop.pl Executable file
View File

@ -0,0 +1,162 @@
#!/usr/bin/perl
use strict;
use lib "$ENV{'LJHOME'}/cgi-bin";
require 'ljlib.pl';
# so output happens quickly
$| = 1;
# make sure we got a parameter
my $propname = shift;
die "ERROR: no property specified\n" unless $propname;
# verify it's a valid property
my $prop = LJ::get_prop('user', $propname);
# see if we know how to handle this parameter
if ($propname eq 'external_foaf_url' && $prop->{cldversion} == 0) {
# this one is simple; we're moving this one to the clusters 1000 users at a time
print "Beginning initial data migration...\n";
cluster_property($prop);
# update the property to be indexed
print "Updating property data in userproplist...\n";
update_property($prop, { indexed => 0, cldversion => 4 });
# strongly recommend a restart
print "\n";
print "* " x 38 . "\n";
print "WARNING: It is recommended that you restart your web nodes now to cause the\n";
print " updated property to start to take effect. Please press enter when\n";
print " this is done.\n";
print "* " x 38 . "\n";
readline STDIN;
# now let's hope they restarted and let's migrate anybody who is still stuck
print "Beginning final data migration...\n";
cluster_property($prop);
# done
print "Finished migrating external_foaf_url property.\n";
} else {
# don't know how to handle it
die "ERROR: don't know how to handle '$propname' (has it already been handled?)\n";
}
##############################################################################
### helper subs
sub cluster_property {
my $prop = shift;
# some state tracking information
my (%dbcms); # ( clusterid => dbcm )
my (%to_write); # ( clusterid => [ [ userid, value ], [ userid, value ], ... ]
# note: livejournal has only about 7500 external_foaf_urls... those should
# be moved in less time than any database handle will time out, so I've made
# the decision not to worry about handle timeouts right now. all other sites
# are probably no more than this size, so it should be fine for everybody.
# setup our flushing sub that we'll need later
my $flush = sub {
my $cid = shift;
# get the ref from to_write etc
my $aref = $to_write{$cid};
delete $to_write{$cid};
# get handle to database if needed
my $dbcm = $dbcms{$cid} || ($dbcms{$cid} = LJ::get_cluster_master($cid));
# notice that we're flushing data
print "\tflushing " . scalar(@$aref) . " items to cluster $cid...";
# now construct SQL
my $repstr = join(', ', map { "($_->[0], $prop->{upropid}, " .
$dbcm->quote($_->[1]) . ")" } @$aref);
$dbcm->do("REPLACE INTO userproplite2 (userid, upropid, value) VALUES $repstr");
die "ERROR: database: " . $dbcm->errstr . "\n" if $dbcm->err;
# done, status update
print "flushed\n";
};
# start our main loop
while (1) {
# data storage for each loop
my (%users, %values); # ( userid => user object or value )
# clear our handles
$LJ::DBIRole->flush_cache();
# get main database handle
my $dbh = LJ::get_db_writer();
# select up to 1000 userid:value tuples
print "Getting values...";
my $vals = $dbh->selectall_arrayref
('SELECT userid, value FROM userprop WHERE upropid = ? LIMIT 1000',
undef, $prop->{upropid});
die "ERROR: database: " . $dbh->errstr . "\n" if $dbh->err;
print "got " . scalar(@$vals) . " values.\n";
# short circuit if we have 0
return 1 if scalar @$vals == 0;
# get the userids to load
my @to_load;
foreach my $row (@$vals) {
my ($userid, $value) = @$row;
$values{$userid} = $value;
push @to_load, $userid;
}
# now load the users in one big grab
print "Loading users...";
LJ::load_userids_multiple([ map { $_ => \$users{$_} } @to_load ]);
print "loaded.\n";
# now push data onto the cluster lists
while (my ($userid, $value) = each %values) {
my $cid = $users{$userid}->{clusterid};
# clusterid 0 means the user is expunged or somesuch, so we
# don't weant to migrate their settings anywhere and should
# just delete it.
next unless $cid;
# now push this onto the to_write array
$to_write{$cid} ||= [];
push @{$to_write{$cid}}, [ $userid, $value ];
# now, flush this list if it's large (100 or more)
$flush->($cid) if scalar @{$to_write{$cid}} >= 100;
}
# now flush everything that's left
$flush->($_) foreach keys %to_write;
# now delete from the global for items that we've written
print "Deleting " . scalar(keys %values) . " items from global...";
my $instr = join(',', map { $_ + 0 } keys %values);
$dbh->do("DELETE FROM userprop WHERE upropid = $prop->{upropid} AND userid IN ($instr)");
die "ERROR: database: " . $dbh->errstr . "\n" if $dbh->err;
print "deleted.\n";
# last if we had less than 1000 this time
last if scalar @$vals < 1000;
}
}
sub update_property {
my ($prop, $sets) = @_;
die "ERROR: nothing to set\n" unless %$sets;
# now make the updates they want
my $dbh = LJ::get_db_writer();
my $updstr = join(', ', map { "$_ = " . $dbh->quote($sets->{$_}) } keys %$sets);
$dbh->do("UPDATE userproplist SET $updstr WHERE upropid = $prop->{upropid}");
die "ERROR: database: " . $dbh->errstr . "\n" if $dbh->err;
}

107
bin/upgrading/moods.dat Normal file
View File

@ -0,0 +1,107 @@
MOOD 1 aggravated 2
MOOD 2 angry 0
MOOD 3 annoyed 2
MOOD 4 anxious 46
MOOD 5 bored 25
MOOD 6 confused 0
MOOD 7 crappy 25
MOOD 8 cranky 2
MOOD 9 depressed 25
MOOD 10 discontent 25
MOOD 11 energetic 0
MOOD 12 enraged 2
MOOD 14 exhausted 74
MOOD 15 happy 0
MOOD 16 high 41
MOOD 18 hungry 74
MOOD 19 infuriated 2
MOOD 20 irate 2
MOOD 21 jubilant 15
MOOD 22 lonely 25
MOOD 23 moody 2
MOOD 25 sad 0
MOOD 26 satisfied 53
MOOD 28 stressed 2
MOOD 30 thoughtful 0
MOOD 31 tired 14
MOOD 32 touched 15
MOOD 33 lazy 61
MOOD 34 drunk 74
MOOD 41 excited 15
MOOD 42 relieved 26
MOOD 43 hopeful 70
MOOD 44 amused 15
MOOD 46 scared 0
MOOD 47 frustrated 2
MOOD 49 sleepy 31
MOOD 51 groggy 31
MOOD 52 hyper 11
MOOD 53 relaxed 15
MOOD 54 restless 74
MOOD 55 disappointed 25
MOOD 56 curious 6
MOOD 61 okay 0
MOOD 68 calm 53
MOOD 70 optimistic 15
MOOD 71 pessimistic 38
MOOD 74 uncomfortable 25
MOOD 79 embarrassed 46
MOOD 80 envious 10
MOOD 81 sympathetic 25
MOOD 82 sick 74
MOOD 83 hot 74
MOOD 84 cold 74
MOOD 85 worried 25
MOOD 86 loved 15
MOOD 87 awake 0
MOOD 88 working 0
MOOD 89 productive 88
MOOD 90 accomplished 88
MOOD 91 busy 88
MOOD 93 full 26
MOOD 95 grumpy 2
MOOD 96 weird 66
MOOD 98 ecstatic 15
MOOD 101 contemplative 30
MOOD 102 nerdy 0
MOOD 103 geeky 102
MOOD 104 cynical 2
MOOD 106 crazy 66
MOOD 107 creative 88
MOOD 109 pleased 15
MOOD 112 irritated 2
MOOD 113 blank 78
MOOD 114 apathetic 78
MOOD 115 dorky 102
MOOD 116 impressed 15
MOOD 117 naughty 36
MOOD 119 dirty 74
MOOD 120 giddy 66
MOOD 121 surprised 15
MOOD 122 shocked 121
MOOD 123 rejected 25
MOOD 124 numb 25
MOOD 125 cheerful 15
MOOD 126 good 15
MOOD 127 distressed 4
MOOD 128 intimidated 46
MOOD 130 devious 0
MOOD 131 thankful 15
MOOD 132 grateful 15
MOOD 133 jealous 25
MOOD 134 nervous 46
MOODTHEME Classic Smileys : The classic little yellow (and other color) smiley faces.
2 /img/mood/classic/angry.gif 15 15
6 /img/mood/classic/confused.gif 19 17
11 /img/mood/classic/energetic.gif 17 17
15 /img/mood/classic/smile.gif 15 15
25 /img/mood/classic/sad.gif 15 15
30 /img/mood/classic/thinking.gif 22 21
31 /img/mood/classic/tired.gif 19 17
34 /img/mood/classic/drunk.gif 15 15
46 /img/mood/classic/shock.gif 15 15
61 /img/mood/classic/blah.gif 15 15
66 /img/mood/classic/wink.gif 15 15
79 /img/mood/classic/blush.gif 15 15
82 /img/mood/classic/sick.gif 15 15
130 /img/mood/classic/devious.gif 15 15

886
bin/upgrading/move0cluster.pl Executable file
View File

@ -0,0 +1,886 @@
#!/usr/bin/perl
#
# Moves a user between clusters.
#
use strict;
use Getopt::Long;
my $opt_del = 0;
my $opt_destdel = 0;
my $opt_useslow = 0;
my $opt_slowalloc = 0;
my $opt_verbose = 1;
my $opt_movemaster = 0;
my $opt_prelocked = 0;
my $opt_expungedel = 0;
my $opt_ignorebit = 0;
exit 1 unless GetOptions('delete' => \$opt_del,
'destdelete' => \$opt_destdel,
'useslow' => \$opt_useslow, # use slow db role for read
'slowalloc' => \$opt_slowalloc, # see note below
'verbose=i' => \$opt_verbose,
'movemaster|mm' => \$opt_movemaster,
'prelocked' => \$opt_prelocked,
'expungedel' => \$opt_expungedel,
'ignorebit' => \$opt_ignorebit,
);
my $optv = $opt_verbose;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
require "$ENV{'LJHOME'}/cgi-bin/ljcmdbuffer.pl";
my $dbh = LJ::get_dbh({raw=>1}, "master");
die "No master db available.\n" unless $dbh;
$dbh->do("SET wait_timeout=28800");
my $dbr = $dbh;
if ($opt_useslow) {
$dbr = LJ::get_dbh({raw=>1}, "slow");
unless ($dbr) { die "Can't get slow db from which to read.\n"; }
}
my $user = LJ::canonical_username(shift @ARGV);
my $dclust = shift @ARGV;
# tables which are dumbly copied by sucking all into memory first.
# use smarter mover code for anything that shouldn't all go into memory.
# also, to use this you have to define the primary keys below
my @manual_move = qw(loginstall ratelog sessions userproplite2
sessions_data userbio userpicblob2 userpropblob
s1usercache modlog modblob counter
s1style s1overrides links userblob clustertrack2);
sub usage {
die "Usage:\n movecluster.pl <user> <destination cluster #>\n";
}
usage() unless defined $user;
usage() unless defined $dclust;
die "Failed to get move lock.\n"
unless ($dbh->selectrow_array("SELECT GET_LOCK('moveucluster-$user', 10)"));
my $u = $dbh->selectrow_hashref("SELECT * FROM user WHERE user=?", undef, $user);
die "Non-existent user $user.\n" unless $u;
die "Can't move back to legacy cluster 0\n" unless $dclust;
my $dbch = LJ::get_cluster_master({raw=>1}, $dclust);
die "Undefined or down cluster \#$dclust\n" unless $dbch;
$dbch->do("SET wait_timeout=28800");
my $separate_cluster = LJ::use_diff_db("master", "cluster$dclust");
$dbh->{'RaiseError'} = 1;
$dbch->{'RaiseError'} = 1;
my $sclust = $u->{'clusterid'};
if ($sclust == $dclust) {
die "User '$user' is already on cluster $dclust\n";
}
if ($sclust) {
verify_movable_tables();
}
# original cluster db handle.
my $dbo;
if ($sclust) {
$dbo = $opt_movemaster ?
LJ::get_dbh({raw=>1}, "cluster$u->{clusterid}movemaster") :
LJ::get_cluster_master({raw=>1}, $u);
die "Can't get source cluster handle.\n" unless $dbo;
$dbo->{'RaiseError'} = 1;
$dbo->do("SET wait_timeout=28800");
}
my $userid = $u->{'userid'};
# find readonly cap class, complain if not found
my $readonly_bit = undef;
foreach (keys %LJ::CAP) {
if ($LJ::CAP{$_}->{'_name'} eq "_moveinprogress" &&
$LJ::CAP{$_}->{'readonly'} == 1) {
$readonly_bit = $_;
last;
}
}
unless (defined $readonly_bit) {
die "Won't move user without %LJ::CAP capability class named '_moveinprogress' with readonly => 1\n";
}
# make sure a move isn't already in progress
if ($opt_prelocked) {
unless (($u->{'caps'}+0) & (1 << $readonly_bit)) {
die "User '$user' should have been prelocked.\n";
}
} else {
if (($u->{'caps'}+0) & (1 << $readonly_bit)) {
die "User '$user' is already in the process of being moved? (cap bit $readonly_bit set)\n"
unless $opt_ignorebit;
}
}
if ($opt_expungedel && $u->{'statusvis'} eq "D" &&
LJ::mysqldate_to_time($u->{'statusvisdate'}) < time() - 86400*31) {
print "Expunging user '$u->{'user'}'\n";
$dbh->do("INSERT INTO clustermove (userid, sclust, dclust, timestart, timedone) ".
"VALUES (?,?,?,UNIX_TIMESTAMP(),UNIX_TIMESTAMP())", undef,
$userid, $sclust, 0);
LJ::update_user($userid, { clusterid => 0,
statusvis => 'X',
raw => "caps=caps&~(1<<$readonly_bit), statusvisdate=NOW()" });
exit 0;
}
print "Moving '$u->{'user'}' from cluster $sclust to $dclust\n" if $optv >= 1;
# mark that we're starting the move
$dbh->do("INSERT INTO clustermove (userid, sclust, dclust, timestart) ".
"VALUES (?,?,?,UNIX_TIMESTAMP())", undef, $userid, $sclust, $dclust);
my $cmid = $dbh->{'mysql_insertid'};
# set readonly cap bit on user
LJ::update_user($userid, { raw => "caps=caps|(1<<$readonly_bit)" })
unless $opt_prelocked;
$dbh->do("SELECT RELEASE_LOCK('moveucluster-$user')");
unless ($opt_prelocked) {
# wait a bit for writes to stop if journal is somewhat active (last week update)
my $secidle = $dbh->selectrow_array("SELECT UNIX_TIMESTAMP()-UNIX_TIMESTAMP(timeupdate) ".
"FROM userusage WHERE userid=$userid");
if ($secidle) {
sleep(2) unless $secidle > 86400*7;
sleep(1) unless $secidle > 86400;
}
}
# make sure slow is caught up:
if ($opt_useslow)
{
my $ms = $dbh->selectrow_hashref("SHOW MASTER STATUS");
my $loop = 1;
while ($loop) {
my $ss = $dbr->selectrow_hashref("SHOW SLAVE STATUS");
$loop = 0;
unless ($ss->{'Log_File'} gt $ms->{'File'} ||
($ss->{'Log_File'} eq $ms->{'File'} && $ss->{'Pos'} >= $ms->{'Position'}))
{
$loop = 1;
print "Waiting for slave ($ss->{'Pos'} < $ms->{'Position'})\n";
sleep 1;
}
}
}
my $last = time();
my $stmsg = sub {
my $msg = shift;
my $now = time();
return if ($now < $last + 1);
$last = $now;
print $msg;
};
my %bufcols = (); # db|table -> scalar "(foo, anothercol, lastcol)" or undef or ""
my %bufrows = (); # db|table -> [ []+ ]
my %bufdbmap = (); # scalar(DBI hashref) -> DBI hashref
my $flush_buffer = sub {
my $dbandtable = shift;
my ($db, $table) = split(/\|/, $dbandtable);
$db = $bufdbmap{$db};
return unless exists $bufcols{$dbandtable};
my $sql = "REPLACE INTO $table $bufcols{$dbandtable} VALUES ";
$sql .= join(", ",
map { my $r = $_;
"(" . join(", ",
map { $db->quote($_) } @$r) . ")" }
@{$bufrows{$dbandtable}});
$db->do($sql);
delete $bufrows{$dbandtable};
delete $bufcols{$dbandtable};
};
my $flush_all = sub {
foreach (keys %bufcols) {
$flush_buffer->($_);
}
};
my $replace_into = sub {
my $db = ref $_[0] ? shift : $dbch; # optional first arg
my ($table, $cols, $max, @vals) = @_;
my $dbandtable = scalar($db) . "|$table";
$bufdbmap{$db} = $db;
if (exists $bufcols{$dbandtable} && $bufcols{$dbandtable} ne $cols) {
$flush_buffer->($dbandtable);
}
$bufcols{$dbandtable} = $cols;
push @{$bufrows{$dbandtable}}, [ @vals ];
if (scalar @{$bufrows{$dbandtable}} > $max) {
$flush_buffer->($dbandtable);
}
};
# assume never tried to move this user before. however, if reported crap
# in the oldids table, we'll revert to slow alloc_id functionality,
# where we do a round-trip to $dbh for everything and see if every id
# has been remapped already. otherwise we do it in perl and batch
# updates to the oldids table, which is the common/fast case.
my $first_move = ! $opt_slowalloc;
my %alloc_data;
my %alloc_arealast;
my $alloc_id = sub {
my ($area, $orig) = @_;
# fast version
if ($first_move) {
my $id = $alloc_data{$area}->{$orig} = ++$alloc_arealast{$area};
$replace_into->($dbh, "oldids", "(area, oldid, userid, newid)", 250,
$area, $orig, $userid, $id);
return $id;
}
# slow version
$dbh->{'RaiseError'} = 0;
$dbh->do("INSERT INTO oldids (area, oldid, userid, newid) ".
"VALUES ('$area', $orig, $userid, NULL)");
my $id;
if ($dbh->err) {
$id = $dbh->selectrow_array("SELECT newid FROM oldids WHERE area='$area' AND oldid=$orig");
} else {
$id = $dbh->{'mysql_insertid'};
}
$dbh->{'RaiseError'} = 1;
$alloc_data{$area}->{$orig} = $id;
return $id;
};
my $bufread;
if ($sclust == 0)
{
# do bio stuff
{
my $bio = $dbr->selectrow_array("SELECT bio FROM userbio WHERE userid=$userid");
my $bytes = length($bio);
$dbch->do("REPLACE INTO dudata (userid, area, areaid, bytes) VALUES ($userid, 'B', 0, $bytes)");
if ($separate_cluster) {
$bio = $dbh->quote($bio);
$dbch->do("REPLACE INTO userbio (userid, bio) VALUES ($userid, $bio)");
}
}
my @itemids = reverse @{$dbr->selectcol_arrayref("SELECT itemid FROM log ".
"WHERE ownerid=$u->{'userid'} ".
"ORDER BY ownerid, rlogtime")};
$bufread = make_buffer_reader("itemid", \@itemids);
my $todo = @itemids;
my $done = 0;
my $stime = time();
print "Total: $todo\n";
# moving time, journal item at a time, and everything recursively under it
foreach my $itemid (@itemids) {
movefrom0_logitem($itemid);
$done++;
my $percent = $done/$todo;
my $elapsed = time() - $stime;
my $totaltime = $elapsed * (1 / $percent);
my $timeremain = int($totaltime - $elapsed);
$stmsg->(sprintf "$user: copy $done/$todo (%.2f%%) +${elapsed}s -${timeremain}s\n", 100*$percent);
}
$flush_all->();
# update their memories. in particular, any memories of their own
# posts need to to be updated from the (0, globalid) to
# (journalid, jitemid) format, to make the memory filter feature
# work. (it checks the first 4 bytes only, not joining the
# globalid on the clustered log table)
print "Fixing memories.\n";
my @fix = @{$dbh->selectall_arrayref("SELECT memid, jitemid FROM memorable WHERE ".
"userid=$u->{'userid'} AND journalid=0")};
foreach my $f (@fix) {
my ($memid, $newid) = ($f->[0], $alloc_data{'L'}->{$f->[1]});
next unless $newid;
my ($newid2, $anum) = $dbch->selectrow_array("SELECT jitemid, anum FROM log2 ".
"WHERE journalid=$u->{'userid'} AND ".
"jitemid=$newid");
if ($newid2 == $newid) {
my $ditemid = $newid * 256 + $anum;
print "UPDATE $memid TO $ditemid\n";
$dbh->do("UPDATE memorable SET journalid=$u->{'userid'}, jitemid=$ditemid ".
"WHERE memid=$memid");
}
}
# fix polls
print "Fixing polls.\n";
@fix = @{$dbh->selectall_arrayref("SELECT pollid, itemid FROM poll ".
"WHERE journalid=$u->{'userid'}")};
foreach my $f (@fix) {
my ($pollid, $newid) = ($f->[0], $alloc_data{'L'}->{$f->[1]});
next unless $newid;
my ($newid2, $anum) = $dbch->selectrow_array("SELECT jitemid, anum FROM log2 ".
"WHERE journalid=$u->{'userid'} AND ".
"jitemid=$newid");
if ($newid2 == $newid) {
my $ditemid = $newid * 256 + $anum;
print "UPDATE $pollid TO $ditemid\n";
$dbh->do("UPDATE poll SET itemid=$ditemid WHERE pollid=$pollid");
}
}
# move userpics
print "Copying over userpics.\n";
my @pics = @{$dbr->selectcol_arrayref("SELECT picid FROM userpic WHERE ".
"userid=$u->{'userid'}")};
foreach my $picid (@pics) {
print " picid\#$picid...\n";
my $imagedata = $dbr->selectrow_array("SELECT imagedata FROM userpicblob ".
"WHERE picid=$picid");
$imagedata = $dbh->quote($imagedata);
$dbch->do("REPLACE INTO userpicblob2 (userid, picid, imagedata) VALUES ".
"($u->{'userid'}, $picid, $imagedata)");
}
$dbh->do("UPDATE userusage SET lastitemid=0 WHERE userid=$userid");
my $dversion = 2;
# if everything's good (nothing's died yet), then delete all from source
if ($opt_del)
{
# before we start deleting, record they've moved servers.
LJ::update_user($userid, { dversion => $dversion, clusterid => $dclust });
$done = 0;
$stime = time();
foreach my $itemid (@itemids) {
deletefrom0_logitem($itemid);
$done++;
my $percent = $done/$todo;
my $elapsed = time() - $stime;
my $totaltime = $elapsed * (1 / $percent);
my $timeremain = int($totaltime - $elapsed);
$stmsg->(sprintf "$user: delete $done/$todo (%.2f%%) +${elapsed}s -${timeremain}s\n", 100*$percent);
}
# delete bio from source, if necessary
if ($separate_cluster) {
$dbh->do("DELETE FROM userbio WHERE userid=$userid");
}
# delete source userpics
print "Deleting cluster0 userpics...\n";
foreach my $picid (@pics) {
print " picid\#$picid...\n";
$dbh->do("DELETE FROM userpicblob WHERE picid=$picid");
}
# unset read-only bit (marks the move is complete, also, and not aborted mid-delete)
LJ::update_user($userid, { raw => "caps=caps&~(1<<$readonly_bit)" });
$dbh->do("UPDATE clustermove SET sdeleted='1', timedone=UNIX_TIMESTAMP() ".
"WHERE cmid=?", undef, $cmid);
}
else
{
# unset readonly and move to new cluster in one update
LJ::update_user($userid, { dversion => $dversion, clusterid => $dclust,
raw => "caps=caps&~(1<<$readonly_bit)" });
$dbh->do("UPDATE clustermove SET sdeleted='0', timedone=UNIX_TIMESTAMP() ".
"WHERE cmid=?", undef, $cmid);
}
exit 0;
}
elsif ($sclust > 0)
{
print "Moving away from cluster $sclust\n" if $optv;
while (my $cmd = $dbo->selectrow_array("SELECT cmd FROM cmdbuffer WHERE journalid=$userid")) {
my $dbcm = LJ::get_cluster_master($sclust);
print "Flushing cmdbuffer for cmd: $cmd\n" if $optv > 1;
LJ::Cmdbuffer::flush($dbh, $dbcm, $cmd, $userid);
}
my $pri_key = {
# flush this first:
'cmdbuffer' => 'journalid',
# this is populated as we do log/talk
'dudata' => 'userid',
# manual
'loginstall' => 'userid',
'ratelog' => 'userid',
'sessions' => 'userid',
'sessions_data' => 'userid',
'userbio' => 'userid',
'userpicblob2' => 'userid',
'userproplite2' => 'userid',
's1usercache' => 'userid',
'modlog' => 'journalid',
'modblob' => 'journalid',
'counter' => 'journalid',
'userblob' => 'journalid',
'userpropblob' => 'userid',
'clustertrack2' => 'userid',
# log
'log2' => 'journalid',
'logsec2' => 'journalid',
'logprop2' => 'journalid',
'logtext2' => 'journalid',
# talk
'talk2' => 'journalid',
'talkprop2' => 'journalid',
'talktext2' => 'journalid',
# no primary key... move up by posttime
'talkleft' => 'userid',
# s1 styles
's1style' => 'userid',
's1overrides' => 'userid',
# link lists
'links' => 'journalid',
};
# ask the local mods if they have any tables to move
my @local_tables;
my $local_tables = LJ::run_hook("moveucluster_local_tables");
if ($local_tables) {
while (my ($tab, $key) = each %$local_tables) {
push @local_tables, $tab;
$pri_key->{$tab} = $key;
}
}
my @existing_data;
print "Checking for existing data on target cluster...\n" if $optv > 1;
foreach my $table (sort keys %$pri_key) {
my $pri = $pri_key->{$table};
my $is_there = $dbch->selectrow_array("SELECT $pri FROM $table WHERE $pri=$userid LIMIT 1");
next unless $is_there;
if ($opt_destdel) {
while ($dbch->do("DELETE FROM $table WHERE $pri=$userid LIMIT 500") > 0) {
print " deleted from $table\n" if $optv;
}
} else {
push @existing_data, $table;
}
}
if (@existing_data) {
die " Existing data in tables: @existing_data\n";
}
my %pendreplace; # "logprop2:(col,col)" => { 'values' => [ [a, b, c], [d, e, f] ],
# 'bytes' => 3043, 'recs' => 35 }
my $flush = sub {
my $dest = shift;
return 1 unless $pendreplace{$dest};
my ($table, $cols) = split(/:/, $dest);
my $vals;
foreach my $v (@{$pendreplace{$dest}->{'values'}}) {
$vals .= "," if $vals;
$vals .= "(" . join(',', map { $dbch->quote($_) } @$v) . ")";
}
print " flushing write to $table\n" if $optv > 1;
$dbch->do("REPLACE INTO $table $cols VALUES $vals");
die $dbh->errstr if $dbch->err;
delete $pendreplace{$dest};
return 1;
};
my $write = sub {
my $dest = shift;
my @values = @_;
my $new_bytes = 0; foreach (@values) { $new_bytes += length($_); }
push @{$pendreplace{$dest}->{'values'}}, \@values;
$pendreplace{$dest}->{'bytes'} += $new_bytes;
$pendreplace{$dest}->{'recs'}++;
if ($pendreplace{$dest}->{'bytes'} > 1024*500 ||
$pendreplace{$dest}->{'recs'} > 500) { $flush->($dest); }
};
my @styleids = ();
# manual moving (dumb copies)
foreach my $table (@manual_move, @local_tables) {
next if ($table eq "modlog" || $table eq "modblob") && $u->{journaltype} eq "P";
print " moving $table ...\n" if $optv > 1;
my @cols;
my $sth = $dbo->prepare("DESCRIBE $table");
$sth->execute;
my $styleidcolnum = -1;
my $colnum = 0;
while ($_ = $sth->fetchrow_hashref) {
push @cols, $_->{'Field'};
$styleidcolnum = $colnum if $_->{'Field'} eq 'styleid';
$colnum++;
}
my $cols = join(',', @cols);
my $dest = "$table:($cols)";
my $pri = $pri_key->{$table};
$sth = $dbo->prepare("SELECT $cols FROM $table WHERE $pri=$userid");
$sth->execute;
while (my @vals = $sth->fetchrow_array) {
$write->($dest, @vals);
if ($styleidcolnum > -1 && $table eq 's1style') {
push @styleids, $vals[$styleidcolnum];
}
}
}
# size of bio
my $bio_size = $dbch->selectrow_array("SELECT LENGTH(bio) FROM userbio WHERE userid=$userid");
$write->("dudata:(userid,area,areaid,bytes)", $userid, 'B', 0, $bio_size) if $bio_size;
# journal items
{
my $maxjitem = $dbo->selectrow_array("SELECT MAX(jitemid) FROM log2 WHERE journalid=$userid");
my $load_amt = 1000;
my ($lo, $hi) = (1, $load_amt);
my $sth;
my $cols = "security,allowmask,journalid,jitemid,posterid,eventtime,logtime,compressed,anum,replycount,year,month,day,rlogtime,revttime"; # order matters. see indexes below
while ($lo <= $maxjitem) {
print " log ($lo - $hi, of $maxjitem)\n" if $optv > 1;
# log2/logsec2
$sth = $dbo->prepare("SELECT $cols FROM log2 ".
"WHERE journalid=$userid AND jitemid BETWEEN $lo AND $hi");
$sth->execute;
while (my @vals = $sth->fetchrow_array) {
$write->("log2:($cols)", @vals);
if ($vals[0] eq "usemask") {
$write->("logsec2:(journalid,jitemid,allowmask)",
$userid, $vals[3], $vals[1]);
}
}
# logprop2
$sth = $dbo->prepare("SELECT journalid,jitemid,propid,value ".
"FROM logprop2 WHERE journalid=$userid AND jitemid BETWEEN $lo AND $hi");
$sth->execute;
while (my @vals = $sth->fetchrow_array) {
$write->("logprop2:(journalid,jitemid,propid,value)", @vals);
}
# logtext2
$sth = $dbo->prepare("SELECT journalid,jitemid,subject,event ".
"FROM logtext2 WHERE journalid=$userid AND jitemid BETWEEN $lo AND $hi");
$sth->execute;
while (my @vals = $sth->fetchrow_array) {
my $size = length($vals[2]) + length($vals[3]);
LJ::text_compress(\$vals[3]);
$write->("logtext2:(journalid,jitemid,subject,event)", @vals);
$write->("dudata:(userid,area,areaid,bytes)", $userid, 'L', $vals[1], $size);
}
$hi += $load_amt; $lo += $load_amt;
}
}
# comments
{
my $maxtalkid = $dbo->selectrow_array("SELECT MAX(jtalkid) FROM talk2 WHERE journalid=$userid");
my $load_amt = 1000;
my ($lo, $hi) = (1, $load_amt);
my $sth;
my %cols = ('talk2' => 'journalid,jtalkid,nodetype,nodeid,parenttalkid,posterid,datepost,state',
'talkprop2' => 'journalid,jtalkid,tpropid,value',
'talktext2' => 'journalid,jtalkid,subject,body');
while ($lo <= $maxtalkid) {
print " talk ($lo - $hi, of $maxtalkid)\n" if $optv > 1;
foreach my $table (keys %cols) {
$sth = $dbo->prepare("SELECT $cols{$table} FROM $table ".
"WHERE journalid=$userid AND jtalkid BETWEEN $lo AND $hi");
$sth->execute;
while (my @vals = $sth->fetchrow_array) {
LJ::text_compress(\$vals[3]) if $table eq "talktext2";
$write->("$table:($cols{$table})", @vals);
}
}
$hi += $load_amt; $lo += $load_amt;
}
}
# talkleft table.
{
# no primary key... delete all of target first.
while ($dbch->do("DELETE FROM talkleft WHERE userid=$userid LIMIT 1000") > 0) {
print " deleted from talkleft\n" if $optv > 1;
}
my $last_max = 0;
my $cols = "userid,posttime,journalid,nodetype,nodeid,jtalkid,publicitem";
while (defined $last_max) {
print " talkleft: $last_max\n" if $optv > 1;
my $sth = $dbo->prepare("SELECT $cols FROM talkleft WHERE userid=$userid ".
"AND posttime > $last_max ORDER BY posttime LIMIT 1000");
$sth->execute;
undef $last_max;
while (my @vals = $sth->fetchrow_array) {
$write->("talkleft:($cols)", @vals);
$last_max = $vals[1];
}
}
}
# flush remaining items
foreach (keys %pendreplace) { $flush->($_); }
# unset readonly and move to new cluster in one update
LJ::update_user($userid, { clusterid => $dclust, raw => "caps=caps&~(1<<$readonly_bit)" });
print "Moved.\n" if $optv;
# delete from source cluster
if ($opt_del) {
print "Deleting from source cluster...\n" if $optv;
foreach my $table (sort keys %$pri_key) {
my $pri = $pri_key->{$table};
while ($dbo->do("DELETE FROM $table WHERE $pri=$userid LIMIT 500") > 0) {
print " deleted from $table\n" if $optv;
}
}
# s1stylecache table
if (@styleids) {
my $styleids_in = join(",", map { $dbo->quote($_) } @styleids);
if ($dbo->do("DELETE FROM s1stylecache WHERE styleid IN ($styleids_in)") > 0) {
print " deleted from s1stylecache\n" if $optv;
}
}
} else {
# at minimum, we delete the clustertrack2 row so it doesn't get
# included in a future ljumover.pl query from that cluster.
$dbo->do("DELETE FROM clustertrack2 WHERE userid=$userid");
}
$dbh->do("UPDATE clustermove SET sdeleted=?, timedone=UNIX_TIMESTAMP() ".
"WHERE cmid=?", undef, $opt_del ? 1 : 0, $cmid);
exit 0;
}
sub deletefrom0_logitem
{
my $itemid = shift;
# delete all the comments
my $talkids = $dbh->selectcol_arrayref("SELECT talkid FROM talk ".
"WHERE nodetype='L' AND nodeid=$itemid");
my $talkidin = join(",", @$talkids);
if ($talkidin) {
foreach my $table (qw(talktext talkprop talk)) {
$dbh->do("DELETE FROM $table WHERE talkid IN ($talkidin)");
}
}
$dbh->do("DELETE FROM logsec WHERE ownerid=$userid AND itemid=$itemid");
foreach my $table (qw(logprop logtext log)) {
$dbh->do("DELETE FROM $table WHERE itemid=$itemid");
}
}
sub movefrom0_logitem
{
my $itemid = shift;
my $item = $bufread->(100, "SELECT * FROM log", $itemid);
my $itemtext = $bufread->(50, "SELECT itemid, subject, event FROM logtext", $itemid);
return 1 unless $item && $itemtext; # however that could happen.
# we need to allocate a new jitemid (journal-specific itemid) for this item now.
my $jitemid = $alloc_id->('L', $itemid);
unless ($jitemid) {
die "ERROR: could not allocate a new jitemid\n";
}
$dbh->{'RaiseError'} = 1;
$item->{'jitemid'} = $jitemid;
$item->{'anum'} = int(rand(256));
# copy item over:
$replace_into->("log2", "(journalid, jitemid, posterid, eventtime, logtime, ".
"compressed, security, allowmask, replycount, year, month, day, ".
"rlogtime, revttime, anum)",
50, map { $item->{$_} } qw(ownerid jitemid posterid eventtime
logtime compressed security allowmask replycount
year month day rlogtime revttime anum));
$replace_into->("logtext2", "(journalid, jitemid, subject, event)", 10,
$userid, $jitemid, map { $itemtext->{$_} } qw(subject event));
# add disk usage info! (this wasn't in cluster0 anywhere)
my $bytes = length($itemtext->{'event'}) + length($itemtext->{'subject'});
$replace_into->("dudata", "(userid, area, areaid, bytes)", 50, $userid, 'L', $jitemid, $bytes);
# add the logsec item, if necessary:
if ($item->{'security'} ne "public") {
$replace_into->("logsec2", "(journalid, jitemid, allowmask)", 50,
map { $item->{$_} } qw(ownerid jitemid allowmask));
}
# copy its logprop over:
while (my $lp = $bufread->(50, "SELECT itemid, propid, value FROM logprop", $itemid)) {
next unless $lp->{'value'};
$replace_into->("logprop2", "(journalid, jitemid, propid, value)", 50,
$userid, $jitemid, $lp->{'propid'}, $lp->{'value'});
}
# copy its talk shit over:
my %newtalkids = (0 => 0); # 0 maps back to 0 still
my $talkids = $dbr->selectcol_arrayref("SELECT talkid FROM talk ".
"WHERE nodetype='L' AND nodeid=$itemid");
my @talkids = sort { $a <=> $b } @$talkids;
my $treader = make_buffer_reader("talkid", \@talkids);
foreach my $t (@talkids) {
movefrom0_talkitem($t, $jitemid, \%newtalkids, $item, $treader);
}
}
sub movefrom0_talkitem
{
my $talkid = shift;
my $jitemid = shift;
my $newtalkids = shift;
my $logitem = shift;
my $treader = shift;
my $item = $treader->(100, "SELECT *, UNIX_TIMESTAMP(datepost) AS 'datepostunix' FROM talk", $talkid);
my $itemtext = $treader->(50, "SELECT talkid, subject, body FROM talktext", $talkid);
return 1 unless $item && $itemtext; # however that could happen.
# abort if this is a stranded entry. (shouldn't happen, anyway. even if it does, it's
# not like we're losing data: the UI (talkread.bml) won't show it anyway)
return unless defined $newtalkids->{$item->{'parenttalkid'}};
# we need to allocate a new jitemid (journal-specific itemid) for this item now.
my $jtalkid = $alloc_id->('T', $talkid);
unless ($jtalkid) {
die "ERROR: could not allocate a new jtalkid\n";
}
$newtalkids->{$talkid} = $jtalkid;
$dbh->{'RaiseError'} = 1;
# copy item over:
$replace_into->("talk2", "(journalid, jtalkid, parenttalkid, nodeid, ".
"nodetype, posterid, datepost, state)", 50,
$userid, $jtalkid, $newtalkids->{$item->{'parenttalkid'}},
$jitemid, 'L', map { $item->{$_} } qw(posterid datepost state));
$replace_into->("talktext2", "(journalid, jtalkid, subject, body)",
20, $userid, $jtalkid, map { $itemtext->{$_} } qw(subject body));
# copy its logprop over:
while (my $lp = $treader->(50, "SELECT talkid, tpropid, value FROM talkprop", $talkid)) {
next unless $lp->{'value'};
$replace_into->("talkprop2", "(journalid, jtalkid, tpropid, value)", 50,
$userid, $jtalkid, $lp->{'tpropid'}, $lp->{'value'});
}
# note that poster commented here
if ($item->{'posterid'}) {
my $pub = $logitem->{'security'} eq "public" ? 1 : 0;
my ($table, $db) = ("talkleft_xfp", $dbh);
($table, $db) = ("talkleft", $dbch) if $userid == $item->{'posterid'};
$replace_into->($db, $table, "(userid, posttime, journalid, nodetype, ".
"nodeid, jtalkid, publicitem)", 50,
$item->{'posterid'}, $item->{'datepostunix'}, $userid,
'L', $jitemid, $jtalkid, $pub);
}
}
sub make_buffer_reader
{
my $pricol = shift;
my $valsref = shift;
my %bfd; # buffer read data. halfquery -> { 'rows' => { id => [] },
# 'remain' => [], 'loaded' => { id => 1 } }
return sub
{
my ($amt, $hq, $pid) = @_;
if (not defined $bfd{$hq}->{'loaded'}->{$pid})
{
if (not exists $bfd{$hq}->{'remain'}) {
$bfd{$hq}->{'remain'} = [ @$valsref ];
}
my @todo;
for (1..$amt) {
next unless @{$bfd{$hq}->{'remain'}};
my $id = shift @{$bfd{$hq}->{'remain'}};
push @todo, $id;
$bfd{$hq}->{'loaded'}->{$id} = 1;
}
if (@todo) {
my $sql = "$hq WHERE $pricol IN (" . join(",", @todo) . ")";
my $sth = $dbr->prepare($sql);
$sth->execute;
while (my $r = $sth->fetchrow_hashref) {
push @{$bfd{$hq}->{'rows'}->{$r->{$pricol}}}, $r;
}
}
}
return shift @{$bfd{$hq}->{'rows'}->{$pid}};
};
}
# this function needs to die loudly if moveucluster.pl is unable to move
# any type of table that exists on this installation
sub verify_movable_tables {
my %table; # tablename -> unhandled flag
# first, assume everything's unhandled
foreach my $t (@LJ::USER_TABLES, @LJ::USER_TABLES_LOCAL) {
$table{$t} = 1;
}
# now, clear things we know how to move
foreach my $t (qw(cmdbuffer dudata log2 logsec2 logprop2 logtext2
talk2 talkprop2 talktext2 talkleft
), @manual_move) {
delete $table{$t};
}
# local stuff
my $local_tables = LJ::run_hook("moveucluster_local_tables");
if ($local_tables) {
while (my ($tab, $key) = each %$local_tables) {
delete $table{$tab};
}
}
# things we list as active tables but don't use yet
delete $table{"events"};
# things we don't move because it doesn't really matter
delete $table{"s1stylecache"};
delete $table{"captcha_session"};
if (%table) {
die "ERROR. Won't try to move user, because this mover script can't move all live tables for this user. List of tables without mover code available: \n -- " . join("\n -- ", sort keys %table), "\n";
}
}
1; # return true;

View File

@ -0,0 +1,84 @@
#!/usr/bin/perl
#
# This script converts from dversion 3 to dversion 4,
# which makes most userprops clustered
#
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $fromver = shift;
die "Usage: pop-clusterprops.pl <fromdversion>\n\t(where fromdversion is one of: 3)\n"
unless $fromver == 3;
my $dbh = LJ::get_db_writer();
my $todo = $dbh->selectrow_array("SELECT COUNT(*) FROM user WHERE dversion=$fromver");
my $done = 0;
unless ($todo) {
print "Nothing to convert.\n";
exit 0;
}
sub get_some {
my @list;
my $sth = $dbh->prepare("SELECT * FROM user WHERE dversion=$fromver LIMIT 200");
$sth->execute;
push @list, $_ while $_ = $sth->fetchrow_hashref;
@list;
}
my $tover = $fromver + 1;
print "Converting $todo users from data version $fromver to $tover...\n";
my @props;
my $sth = $dbh->prepare("SELECT upropid FROM userproplist WHERE cldversion=?");
$sth->execute($tover);
push @props, $_ while $_ = $sth->fetchrow_array;
my $in = join(',', @props);
die "No values?" unless $in;
my $start = time();
while (my @list = get_some()) {
LJ::start_request();
my %cluster; # clusterid -> [ $u* ]
foreach my $u (@list) {
push @{$cluster{$u->{'clusterid'}}}, $u;
}
foreach my $cid (keys %cluster) {
my $dbcm = LJ::get_cluster_master($cid);
next unless $dbcm;
my $uid_in = join(',', map { $_->{'userid'} } @{$cluster{$cid}});
my @vals;
foreach my $table (qw(userprop userproplite)) {
$sth = $dbh->prepare("SELECT userid, upropid, value FROM $table ".
"WHERE userid IN ($uid_in) AND upropid IN ($in)");
$sth->execute();
while (my ($uid, $pid, $v) = $sth->fetchrow_array) {
push @vals, "($uid,$pid," . $dbh->quote($v) . ")";
}
}
if (@vals) {
my $sql = "REPLACE INTO userproplite2 VALUES " . join(',', @vals);
$dbcm->do($sql);
if ($dbcm->err) {
die "Error: " . $dbcm->errstr . "\n\n(Do you need to --runsql on your clusters first?)\n";
}
$dbh->do("DELETE FROM userprop WHERE userid IN ($uid_in) AND upropid IN ($in)");
$dbh->do("DELETE FROM userproplite WHERE userid IN ($uid_in) AND upropid IN ($in)");
}
$dbh->do("UPDATE user SET dversion=$tover WHERE userid IN ($uid_in) AND dversion=$fromver");
$done += scalar @{$cluster{$cid}};
}
my $perc = $done/$todo;
my $elapsed = time() - $start;
my $total_time = $elapsed / $perc;
my $min_remain = int(($total_time - $elapsed) / 60);
printf "%d/%d complete (%.02f%%) minutes_remain=%d\n", $done, $todo, ($perc*100), $min_remain;
}

68
bin/upgrading/pop-weekuu.pl Executable file
View File

@ -0,0 +1,68 @@
#!/usr/bin/perl
#
# This script converts from dversion 2 (clustered + userpicblobs clustered)
# to dversion 3, which adds weekuserusage population.
#
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
die "This script is no longer useful.\n";
my $dbh = LJ::get_db_writer();
my $todo = $dbh->selectrow_array("SELECT COUNT(*) FROM user WHERE dversion=2");
my $done = 0;
unless ($todo) {
print "Nothing to convert.\n";
exit 0;
}
sub get_some {
my @list;
my $sth = $dbh->prepare("SELECT * FROM user WHERE dversion=2 LIMIT 200");
$sth->execute;
push @list, $_ while $_ = $sth->fetchrow_hashref;
@list;
}
print "Converting $todo users from data version 2 to 3...\n";
my $start = time();
while (my @list = get_some()) {
LJ::start_request();
foreach my $u (@list) {
my $dbcm = LJ::get_cluster_master($u);
next unless $dbcm;
my %week;
my $sth = $dbcm->prepare("SELECT rlogtime FROM log2 ".
"WHERE journalid=? AND rlogtime < 2147483647");
$sth->execute($u->{'userid'});
while (my $t = $sth->fetchrow_array) {
my ($week, $uafter, $ubefore) = LJ::weekuu_parts($t);
if (! $week{$week}) {
$week{$week} = [ $uafter, $ubefore ];
} elsif ($ubefore < $week{$week}->[1]) {
$week{$week}->[1] = $ubefore;
}
}
if (%week) {
my $sql = "REPLACE INTO weekuserusage (wknum,userid,uafter,ubefore) VALUES " .
join(",", map { "($_,$u->{'userid'},$week{$_}->[0],$week{$_}->[1])" } keys %week);
my $rv = $dbh->do($sql);
die $dbh->errstr if $dbh->err;
next unless $rv; # error? try later.
}
$dbh->do("UPDATE user SET dversion=3 WHERE userid=?", undef, $u->{'userid'});
$done++;
}
my $perc = $done/$todo;
my $elapsed = time() - $start;
my $total_time = $elapsed / $perc;
my $min_remain = int(($total_time - $elapsed) / 60);
printf "%d/%d complete (%.02f%%) minutes_remain=%d\n", $done, $todo, ($perc*100), $min_remain;
}

View File

@ -0,0 +1,89 @@
#! /usr/bin/perl
use strict;
require "$ENV{'LJHOME'}/cgi-bin/ljlib.pl";
my $dbslo = LJ::get_dbh("slow")
or die "cannot connect to slow role";
my $clear = $ARGV[0] eq '--clear' ? 1 : 0;
my $limit = 5000;
if ($clear) {
print "Clearing 'birthdays' on all clusters\n";
foreach my $cid (@LJ::CLUSTERS) {
my $dbcm = LJ::get_cluster_master($cid)
or die "no cluster: $cid\n";
$dbcm->do("DELETE FROM birthdays")
or die "unable to delete from birthdays for cluster: $cid\n";
}
}
my $last_max_uid = 0;
print "Querying clusters for current max\n";
foreach my $cid (@LJ::CLUSTERS) {
my $dbcr = LJ::get_cluster_def_reader($cid)
or die "no cluster: $cid\n";
my $cluster_max_uid = $dbcr->selectrow_array
("SELECT MAX(userid) FROM birthdays");
$last_max_uid = $cluster_max_uid if $cluster_max_uid > $last_max_uid;
}
my $uids_done = 0;
my $max_uid = $dbslo->selectrow_array("SELECT MAX(userid) FROM user")+0;
print "Populating userids from $last_max_uid through $max_uid\n";
# scary, i know... but we'll last out if we ever get less than $limit uids
my $start_time = time();
while (1) {
# Let's call start_request
# -- so our in-process $u caches don't get unreasonable
# -- so we revalidate our database handles
LJ::start_request();
$dbslo = LJ::get_dbh("slow")
or die "cannot connect to slow role";
# load user rows from slow
my $sth = $dbslo->prepare
("SELECT * FROM user WHERE userid>? AND statusvis!='X' AND journaltype='P' ORDER BY userid LIMIT $limit");
$sth->execute($last_max_uid);
die $dbslo->errstr if $dbslo->err;
# construct user objects from them since we have the full data around
my %user_rows = (); # uid => $row
while (my $row = $sth->fetchrow_hashref) {
$user_rows{$row->{userid}} = LJ::User->new_from_row($row);
}
last unless %user_rows;
# now update each one
while (my ($uid, $u) = each %user_rows) {
$u->set_next_birthday;
$last_max_uid = $uid if $uid > $last_max_uid;
$uids_done++;
}
# update max userid every so often for our pretty status display
if ($uids_done % 10_000 == 0) {
$max_uid = $dbslo->selectrow_array("SELECT MAX(userid) FROM user")+0;
}
printf ("[%.2f] $uids_done - current id $last_max_uid - %.2f hours\n",
100*($last_max_uid / $max_uid), ($max_uid - $last_max_uid) / ($uids_done / (time() - $start_time)) / 3600
);
# we're done if we got less than the limit
last if scalar (keys %user_rows) < $limit;
}
print "All done!\n";
1;

1331
bin/upgrading/proplists.dat Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,85 @@
#!/usr/bin/perl
#
# Library to read/write s1styles.dat
#
sub s1styles_read
{
my $ss = {};
open (F, "$ENV{'LJHOME'}/bin/upgrading/s1styles.dat");
my $uniq;
my $entry;
my $read_entry = 0;
my $line = 0;
while (<F>)
{
$line++;
if ($read_entry && $entry)
{
if ($_ eq ".\n") {
chop $entry->{'formatdata'}; # we added a newline
$read_entry = 0;
undef $entry;
next;
}
s!^\.!!;
$entry->{'formatdata'} .= $_;
next;
}
if (m!^Style:\s*(\w+?)/(.+?)\s*$!) {
$uniq = "$1/$2";
die "Repeat style in s1styles.dat at line $line!"
if exists $ss->{$uniq};
$entry = $ss->{$uniq} = {
'type' => $1,
'styledes' => $2,
};
$read_entry = 0;
next;
}
if ($entry && $_ eq "\n") {
$read_entry = 1;
next;
}
next unless $entry;
if (/^(\w+):\s*(.+?)\s*$/) {
$entry->{$1} = $2;
next;
}
die "s1styles.dat:$line: bogus line\n" if /\S/;
}
close F;
return $ss;
}
sub s1styles_write
{
my $ss = shift;
open (F, ">$ENV{'LJHOME'}/bin/upgrading/s1styles.dat")
or die "Can't open s1styles.dat for writing.\n";
foreach my $uniq (sort keys %$ss) {
my $s = $ss->{$uniq};
print F "Style: $uniq\n";
foreach (qw(is_public is_embedded is_colorfree opt_cache lastupdate)) {
next unless exists $s->{$_};
print F "$_: $s->{$_}\n";
}
my $formatdata = $s->{'formatdata'};
$formatdata =~ s/\r//g; # die, DOS line endings!
$formatdata =~ s/\n\./\n\.\./g;
print F "\n$formatdata\n.\n\n";
}
close F;
}
1;

8312
bin/upgrading/s1styles.dat Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,83 @@
################################################################
# base filename layer type parent
core1 core -
i18nc/be1 i18nc core1
i18nc/da1 i18nc core1
i18nc/de1 i18nc core1
i18nc/en1 i18nc core1
i18nc/en_GB1 i18nc core1
i18nc/eo1 i18nc core1
i18nc/fi1 i18nc core1
i18nc/fr1 i18nc core1
i18nc/is1 i18nc core1
i18nc/ja1 i18nc core1
i18nc/lv1 i18nc core1
i18nc/ru1 i18nc core1
i18nc/uk1 i18nc core1
classic/layout layout core1
classic/en i18n classic/layout
classic/themes theme+ classic/layout
cleansimple/layout layout core1
cleansimple/en i18n cleansimple/layout
cleansimple/themes theme+ cleansimple/layout
digitalmultiplex/layout layout core1
digitalmultiplex/en i18n digitalmultiplex/layout
digitalmultiplex/themes theme+ digitalmultiplex/layout
disjointed/layout layout core1
disjointed/themes theme+ disjointed/layout
haven/layout layout core1
haven/themes theme+ haven/layout
generator/layout layout core1
generator/en i18n generator/layout
generator/themes theme+ generator/layout
magazine/layout layout core1
magazine/en i18n magazine/layout
magazine/themes theme+ magazine/layout
notepad/layout layout core1
notepad/en i18n notepad/layout
notepad/themes theme+ notepad/layout
punquin/layout layout core1
punquin/en i18n punquin/layout
punquin/themes theme+ punquin/layout
refriedpaper/layout layout core1
refriedpaper/themes theme+ refriedpaper/layout
tabularindent/layout layout core1
tabularindent/en i18n tabularindent/layout
tabularindent/themes theme+ tabularindent/layout
sturdygesture/layout layout core1
sturdygesture/themes theme+ sturdygesture/layout
variableflow/layout layout core1
variableflow/themes theme+ variableflow/layout
s1shortcomings/layout layout core1
sixhtml/layout layout core1
sixhtml/themes theme+ sixhtml/layout
stylecontest/layout layout(sixhtml/layout) core1
stylecontest/themes theme+ stylecontest/layout
deardiary/layout layout core1
deardiary/themes theme+ deardiary/layout
lickable/layout layout core1
lickable/themes theme+ lickable/layout
hostedcomments/layout layout core1
# include local file
s2layers-local.dat INCLUDE

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

View File

@ -0,0 +1,16 @@
# -*-s2-*-
layerinfo type = "i18n";
layerinfo name = "English";
layerinfo redist_uniq = "classic/en";
set text_meta_music = "Current Music";
set text_meta_mood = "Current Mood";
set text_permalink = "Link";
set text_post_comment = "Comment on This";
set text_post_comment_friends = "Comment on This";
set text_read_comments = "1 comment // # comments";
set text_read_comments_friends = "1 comment // # comments";

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,424 @@
#NEWLAYER: classic/standard
layerinfo type = theme;
layerinfo name = "Standard";
layerinfo redist_uniq = "classic/standard";
#NEWLAYER: classic/calmfire
layerinfo type = theme;
layerinfo name = "Calm Fire";
layerinfo redist_uniq = "classic/calmfire";
set body_bgcolor = "#9e0610";
set main_bgcolor = "#7d7d7d";
set main_fgcolor = "#ffffff";
set headerbar_bgcolor = "#ff6c1d";
set headerbar_fgcolor = "#000000";
set metabar_bgcolor = "#ff9364";
set metabar_fgcolor = "#000000";
set page_title_color = "#8b1a1a";
set page_subtitle_color = "#c00000";
set link_color = "#ff1010";
set vlink_color = "#f10707";
set alink_color = "#ff1d1d";
set comment_bar_one_bgcolor = "#ff6c1d";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#ff9364";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/shrinkvio
layerinfo type = theme;
layerinfo name = "Shrinking Violet";
layerinfo redist_uniq = "classic/shrinkvio";
set body_bgcolor = "#ad22e7";
set main_bgcolor = "#ffffff";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#5d0383";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#5d0343";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#d9a1f1";
set page_subtitle_color = "#d9a1f1";
set link_color = "#2e053f";
set vlink_color = "#611627";
set alink_color = "#ff00c0";
set comment_bar_one_bgcolor = "#5d0383";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#5d0343";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/pistmint
layerinfo type = theme;
layerinfo name = "Pistachio Mint";
layerinfo redist_uniq = "classic/pistmint";
set body_bgcolor = "#133422";
set main_bgcolor = "#a7c4b4";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#096d36";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#096d36";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#096d36";
set page_subtitle_color = "#096d36";
set link_color = "#8afabc";
set vlink_color = "#1da65a";
set alink_color = "#f9f5f5";
set comment_bar_one_bgcolor = "#096d36";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#096d36";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/mexicanfood
layerinfo type = theme;
layerinfo name = "Mexican Food";
layerinfo redist_uniq = "classic/mexicanfood";
set body_bgcolor = "#ff0000";
set main_bgcolor = "#f8ff3e";
set main_fgcolor = "#f15601";
set headerbar_bgcolor = "#bdbf3e";
set headerbar_fgcolor = "#ff0000";
set metabar_bgcolor = "#ffc664";
set metabar_fgcolor = "#000000";
set page_title_color = "#ff0000";
set page_subtitle_color = "#e15a18";
set link_color = "#f49e08";
set vlink_color = "#b05403";
set alink_color = "#ff7405";
set comment_bar_one_bgcolor = "#bdbf3e";
set comment_bar_one_fgcolor = "#ff0000";
set comment_bar_two_bgcolor = "#ffc664";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/ashfire
layerinfo type = theme;
layerinfo name = "Ash and Fire";
layerinfo redist_uniq = "classic/ashfire";
set body_bgcolor = "#b5b5b5";
set main_bgcolor = "#ffb6af";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#e75454";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#ff9696";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#ff1106";
set page_subtitle_color = "#f06c88";
set link_color = "#f70208";
set vlink_color = "#b0161d";
set alink_color = "#d70106";
set comment_bar_one_bgcolor = "#e75454";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ff9696";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/desktop
layerinfo type = theme;
layerinfo name = "Classic Desktop";
layerinfo redist_uniq = "classic/desktop";
set body_bgcolor = "#00545c";
set main_bgcolor = "#ffffff";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#ff7b05";
set headerbar_fgcolor = "#ffeddd";
set metabar_bgcolor = "#ffffff";
set metabar_fgcolor = "#000000";
set page_title_color = "#ff7b05";
set page_subtitle_color = "#ffeddd";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#5a76ff";
set comment_bar_one_bgcolor = "#ff7b05";
set comment_bar_one_fgcolor = "#ffeddd";
set comment_bar_two_bgcolor = "#ffffff";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/satinhandshake
layerinfo type = theme;
layerinfo name = "Satin Handshake";
layerinfo redist_uniq = "classic/satinhandshake";
set body_bgcolor = "#480c0c";
set main_bgcolor = "#d06464";
set main_fgcolor = "#00001d";
set headerbar_bgcolor = "#aaaaaa";
set headerbar_fgcolor = "#000000";
set metabar_bgcolor = "#dddddd";
set metabar_fgcolor = "#000000";
set page_title_color = "#9d0404";
set page_subtitle_color = "#9d0404";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#ff00c0";
set comment_bar_one_bgcolor = "#aaaaaa";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#dddddd";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/deepmelodrama
layerinfo type = theme;
layerinfo name = "Deep MeloDrama";
layerinfo redist_uniq = "classic/deepmelodrama";
set body_bgcolor = "#872d89";
set main_bgcolor = "#719cff";
set main_fgcolor = "#8e48b2";
set comment_bar_one_bgcolor = "#3794b3";
set comment_bar_one_fgcolor = "#84b8e7";
set metabar_bgcolor = "#65b2c1";
set metabar_fgcolor = "#f5d3ff";
set page_title_color = "#8e48b2";
set page_subtitle_color = "#65b2c1";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#dfd3ff";
set comment_bar_one_bgcolor = "#3794b3";
set comment_bar_one_fgcolor = "#84b8e7";
set comment_bar_two_bgcolor = "#65b2c1";
set comment_bar_two_fgcolor = "#f5d3ff";
#NEWLAYER: classic/everwhite
layerinfo type = theme;
layerinfo name = "Everwhite";
layerinfo redist_uniq = "classic/everwhite";
set body_bgcolor = "#ffffff";
set main_bgcolor = "#ffffff";
set main_fgcolor = "#000000";
set comment_bar_one_bgcolor = "#ffffff";
set comment_bar_one_fgcolor = "#000000";
set metabar_bgcolor = "#ffffff";
set metabar_fgcolor = "#000000";
set page_title_color = "#000000";
set page_subtitle_color = "#ff0000";
set link_color = "#e60000";
set vlink_color = "#c10602";
set alink_color = "#ff0600";
set comment_bar_one_bgcolor = "#ffffff";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#ffffff";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/everblue
layerinfo type = theme;
layerinfo name = "Everblue with Greys";
layerinfo redist_uniq = "classic/everblue";
set body_bgcolor = "#0f0c6d";
set main_bgcolor = "#ffffff";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#000000";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#aaaaaa";
set metabar_fgcolor = "#000000";
set page_title_color = "#000000";
set page_subtitle_color = "#aaaaaa";
set link_color = "#2f00f2";
set vlink_color = "#060667";
set alink_color = "#6691ff";
set comment_bar_one_bgcolor = "#000000";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#aaaaaa";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/brownleather
layerinfo type = theme;
layerinfo name = "Brown Leather Coat";
layerinfo redist_uniq = "classic/brownleather";
set body_bgcolor = "#d2b48c";
set main_bgcolor = "#ffebcd";
set main_fgcolor = "#8b4513";
set headerbar_bgcolor = "#d48050";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#d48014";
set metabar_fgcolor = "#d48014";
set page_title_color = "#d48014";
set page_subtitle_color = "#ffe1a1";
set link_color = "#000050";
set vlink_color = "#867a55";
set alink_color = "#fffab3";
set comment_bar_one_bgcolor = "#d48014";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#d48050";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/bruise
layerinfo type = theme;
layerinfo name = "Bruise";
layerinfo redist_uniq = "classic/bruise";
set body_bgcolor = "#000000";
set main_bgcolor = "#bcbcbc";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#1114a0";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#21c2f1";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#1114a0";
set page_subtitle_color = "#21c2f1";
set link_color = "#0000cc";
set vlink_color = "#000088";
set alink_color = "#0000ff";
set comment_bar_one_bgcolor = "#1114a0";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#21c2f1";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/ranchhand
layerinfo type = theme;
layerinfo name = "Ranch Hand";
layerinfo redist_uniq = "classic/ranchhand";
set body_bgcolor = "#2999c2";
set main_bgcolor = "#cfe0ff";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#54442c";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#bababa";
set metabar_fgcolor = "#000000";
set page_title_color = "#9d995d";
set page_subtitle_color = "#704400";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#6a20ff";
set comment_bar_one_bgcolor = "#54442c";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#bababa";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: classic/victim
layerinfo type = theme;
layerinfo name = "Victim";
layerinfo redist_uniq = "classic/victim";
set body_bgcolor = "#2cd0ff";
set main_bgcolor = "#505050";
set main_fgcolor = "#ffffff";
set headerbar_bgcolor = "#166bac";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#2098f3";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#26b6ff";
set page_subtitle_color = "#353535";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#ff00c0";
set comment_bar_one_bgcolor = "#166bac";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#2098f3";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/forest
layerinfo type = theme;
layerinfo name = "Forest";
layerinfo redist_uniq = "classic/forest";
set body_bgcolor = "#778e64";
set main_bgcolor = "#9b9ba5";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#72784c";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#72781c";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#a0ac62";
set page_subtitle_color = "#73777a";
set link_color = "#3811e1";
set vlink_color = "#310cbb";
set alink_color = "#4e7bef";
set comment_bar_one_bgcolor = "#72784c";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#72781c";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/drone
layerinfo type = theme;
layerinfo name = "Drone";
layerinfo redist_uniq = "classic/drone";
set body_bgcolor = "#395f82";
set main_bgcolor = "#f9fcfe";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#904094";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#f56efc";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#ff93ff";
set page_subtitle_color = "#eeeeff";
set link_color = "#395f82";
set vlink_color = "#395f82";
set alink_color = "#5266ce";
set comment_bar_one_bgcolor = "#904094";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#f56efc";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/lowercurtain
layerinfo type = theme;
layerinfo name = "Lower the Curtain";
layerinfo redist_uniq = "classic/lowercurtain";
set body_bgcolor = "#000000";
set main_bgcolor = "#6b6b6b";
set main_fgcolor = "#ffffff";
set headerbar_bgcolor = "#363636";
set headerbar_fgcolor = "#f0f5fb";
set metabar_bgcolor = "#c6c6c6";
set metabar_fgcolor = "#222222";
set page_title_color = "#363636";
set page_subtitle_color = "#c5c8ca";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#3314ba";
set comment_bar_one_bgcolor = "#363636";
set comment_bar_one_fgcolor = "#f0f5fb";
set comment_bar_two_bgcolor = "#c6c6c6";
set comment_bar_two_fgcolor = "#222222";
#NEWLAYER: classic/sunny
layerinfo type = theme;
layerinfo name = "Sunny Day";
layerinfo redist_uniq = "classic/sunny";
set body_bgcolor = "#55e0f9";
set main_bgcolor = "#e38202";
set main_fgcolor = "#ffffff";
set headerbar_bgcolor = "#fff500";
set headerbar_fgcolor = "#e38202";
set metabar_bgcolor = "#fff5c5";
set metabar_fgcolor = "#e38202";
set page_title_color = "#efe052";
set page_subtitle_color = "#ffba03";
set link_color = "#df0d12";
set vlink_color = "#ac1b25";
set alink_color = "#fe3b3b";
set comment_bar_one_bgcolor = "#fff500";
set comment_bar_one_fgcolor = "#e38202";
set comment_bar_two_bgcolor = "#fff5c5";
set comment_bar_two_fgcolor = "#e38202";
#NEWLAYER: classic/valentine
layerinfo type = theme;
layerinfo name = "Be Mine";
layerinfo redist_uniq = "classic/valentine";
set body_bgcolor = "#6f104a";
set main_bgcolor = "#f2bce9";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#ff37ff";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#df2096";
set metabar_fgcolor = "#ffffff";
set page_title_color = "#ae1774";
set page_subtitle_color = "#df2096";
set link_color = "#ffffff";
set vlink_color = "#a51014";
set alink_color = "#ed8188";
set comment_bar_one_bgcolor = "#ff37ff";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#df2096";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: classic/stripes
layerinfo type = theme;
layerinfo name = "Stripes";
layerinfo redist_uniq = "classic/stripes";
set body_bgcolor = "#ffffff";
set main_bgcolor = "#ffffff";
set main_fgcolor = "#000000";
set headerbar_bgcolor = "#e7212a";
set headerbar_fgcolor = "#ffffff";
set metabar_bgcolor = "#ffffff";
set metabar_fgcolor = "#000000";
set page_title_color = "#ffffff";
set page_subtitle_color = "#ffcfdc";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#ffafc1";
set comment_bar_one_bgcolor = "#e7212a";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ffffff";
set comment_bar_two_fgcolor = "#000000";

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

View File

@ -0,0 +1,11 @@
# -*-s2-*-
layerinfo type = "i18n";
layerinfo name = "English";
layerinfo redist_uniq = "cleansimple/en";
set text_meta_music = "music";
set text_meta_mood = "mood";
set text_post_comment = "Comment on this";
set text_post_comment_friends = "Comment on this";

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,425 @@
#NEWLAYER: cleansimple/standard
layerinfo type = theme;
layerinfo name = "Standard";
layerinfo redist_uniq = "cleansimple/standard";
#NEWLAYER: cleansimple/purpleblues
layerinfo type = theme;
layerinfo name = "Purples and Blues";
layerinfo redist_uniq = "cleansimple/purpleblues";
set body_bgcolor = "#660099";
set entry_bgcolor = "#9966cc";
set entry_fgcolor = "#ffff00";
set meta_color = "#003399";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#00ffff";
set topbar_bgcolor = "#c0c0ff";
set topbar_fgcolor = "#000000";
set navbar_bgcolor = "#eeeeff";
set navbar_fgcolor = "#000000";
set comment_bar_one_bgcolor = "#c0c0ff";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#eeeeff";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/flesh
layerinfo type = "theme";
layerinfo name = "Flesh";
layerinfo redist_uniq = "cleansimple/flesh";
set body_bgcolor = "#eeeeff";
set entry_bgcolor = "#ffffff";
set entry_fgcolor = "#000000";
set meta_color = "#ff0000";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#00ffff";
set topbar_bgcolor = "#fa83d7";
set topbar_fgcolor = "#000000";
set navbar_bgcolor = "#f677ad";
set navbar_fgcolor = "#000000";
set comment_bar_one_bgcolor = "#fa83d7";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#f677ad";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/shrinkvio
layerinfo type = theme;
layerinfo name = "Shrinking Violet";
layerinfo redist_uniq = "cleansimple/shrinkvio";
set body_bgcolor = "#ad22e7";
set entry_bgcolor = "#ffffff";
set entry_fgcolor = "#000000";
set meta_color = "#381a45";
set topbar_bgcolor = "#5d0383";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#d9a1f1";
set navbar_fgcolor = "#000000";
set link_color = "#2e053f";
set vlink_color = "#611627";
set alink_color = "#ff00c0";
set comment_bar_one_bgcolor = "#5d0383";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#d9a1f1";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/pistmint
layerinfo type = theme;
layerinfo name = "Pistachio Mint";
layerinfo redist_uniq = "cleansimple/pistmint";
set body_bgcolor = "#133422";
set entry_bgcolor = "#a7c4b4";
set entry_fgcolor = "#000000";
set meta_color = "#096d36";
set topbar_bgcolor = "#096d36";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#094f36";
set navbar_fgcolor = "#000000";
set link_color = "#8afabc";
set vlink_color = "#1da65a";
set alink_color = "#f9f5f5";
set comment_bar_one_bgcolor = "#096d36";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#094f36";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/mexicanfood
layerinfo type = theme;
layerinfo name = "Mexican Food";
layerinfo redist_uniq = "cleansimple/mexicanfood";
set body_bgcolor = "#ff0000";
set entry_bgcolor = "#f8ff3e";
set entry_fgcolor = "#f15601";
set meta_color = "#f50701";
set topbar_bgcolor = "#bdbf3e";
set topbar_fgcolor = "#ff0000";
set navbar_bgcolor = "#e15a18";
set navbar_fgcolor = "#ffffff";
set link_color = "#f49e08";
set vlink_color = "#b05403";
set alink_color = "#ff7405";
set comment_bar_one_bgcolor = "#bdbf3e";
set comment_bar_two_fgcolor = "#ff0000";
set comment_bar_two_bgcolor = "#e15a18";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: cleansimple/ashfire
layerinfo type = theme;
layerinfo name = "Ash and Fire";
layerinfo redist_uniq = "cleansimple/ashfire";
set body_bgcolor = "#b5b5b5";
set entry_bgcolor = "#ffb6af";
set entry_fgcolor = "#000000";
set meta_color = "#d90308";
set topbar_bgcolor = "#e75454";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#f06c88";
set navbar_fgcolor = "#000000";
set link_color = "#f70208";
set vlink_color = "#b0161d";
set alink_color = "#d70106";
set comment_bar_one_bgcolor = "#e75454";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#f06c88";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/desktop
# for layout: 13 (cleansimple/layout)
layerinfo type = theme;
layerinfo name = "Classic Desktop";
layerinfo redist_uniq = "cleansimple/desktop";
set body_bgcolor = "#00545c";
set entry_bgcolor = "#ffffff";
set entry_fgcolor = "#000000";
set meta_color = "#000000";
set topbar_bgcolor = "#ff7b05";
set topbar_fgcolor = "#ffeddd";
set navbar_bgcolor = "#ffeddd";
set navbar_fgcolor = "#000000";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#5a76ff";
set comment_bar_one_bgcolor = "#ff7b05";
set comment_bar_one_fgcolor = "#ffeddd";
set comment_bar_two_bgcolor = "#ffeddd";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/satinhandshake
layerinfo type = theme;
layerinfo name = "Satin Handshake";
layerinfo redist_uniq = "cleansimple/satinhandshake";
set body_bgcolor = "#480c0c";
set entry_bgcolor = "#d06464";
set entry_fgcolor = "#00001d";
set meta_color = "#000000";
set topbar_bgcolor = "#aaaaaa";
set topbar_fgcolor = "#000000";
set navbar_bgcolor = "#9d0404";
set navbar_fgcolor = "#000000";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#ff00c0";
set comment_bar_one_bgcolor = "#aaaaaa";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#9d0404";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/deepmelodrama
layerinfo type = theme;
layerinfo name = "Deep MeloDrama";
layerinfo redist_uniq = "cleansimple/deepmelodrama";
set body_bgcolor = "#872d89";
set entry_bgcolor = "#719cff";
set entry_fgcolor = "#8e48b2";
set meta_color = "#8e48b2";
set topbar_bgcolor = "#3794b3";
set topbar_fgcolor = "#84b8e7";
set navbar_bgcolor = "#65b2c1";
set navbar_fgcolor = "#f5d3ff";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#dfd3ff";
set comment_bar_one_bgcolor = "#3794b3";
set comment_bar_one_fgcolor = "#84b8e7";
set comment_bar_two_bgcolor = "#65b2c1";
set comment_bar_two_fgcolor = "#f5d3ff";
#NEWLAYER: cleansimple/everwhite
layerinfo type = theme;
layerinfo name = "Everwhite";
layerinfo redist_uniq = "cleansimple/everwhite";
set body_bgcolor = "#ffffff";
set entry_bgcolor = "#ffffff";
set entry_fgcolor = "#000000";
set meta_color = "#000000";
set topbar_bgcolor = "#ffffff";
set topbar_fgcolor = "#000000";
set navbar_bgcolor = "#ffffff";
set navbar_fgcolor = "#000000";
set link_color = "#e60000";
set vlink_color = "#c10602";
set alink_color = "#ff0600";
set comment_bar_one_bgcolor = "#ffffff";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#ffffff";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/everblue
layerinfo type = theme;
layerinfo name = "Everblue with Greys";
layerinfo redist_uniq = "cleansimple/everblue";
set body_bgcolor = "#0f0c6d";
set entry_bgcolor = "#ffffff";
set entry_fgcolor = "#000000";
set meta_color = "#000000";
set topbar_bgcolor = "#000000";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#aaaaaa";
set navbar_fgcolor = "#000000";
set link_color = "#2f00f2";
set vlink_color = "#060667";
set alink_color = "#6691ff";
set comment_bar_one_bgcolor = "#000000";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#aaaaaa";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/brownleather
layerinfo type = theme;
layerinfo name = "Brown Leather Coat";
layerinfo redist_uniq = "cleansimple/brownleather";
set body_bgcolor = "#d2b48c";
set entry_bgcolor = "#ffebcd";
set entry_fgcolor = "#8b4513";
set meta_color = "#000000";
set topbar_bgcolor = "#d48014";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#ffe1a1";
set navbar_fgcolor = "#000000";
set link_color = "#000050";
set vlink_color = "#867a55";
set alink_color = "#fffab3";
set comment_bar_one_bgcolor = "#d48014";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ffe1a1";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/bruise
layerinfo type = theme;
layerinfo name = "Bruise";
layerinfo redist_uniq = "cleansimple/bruise";
set body_bgcolor = "#000000";
set entry_bgcolor = "#bcbcbc";
set entry_fgcolor = "#000000";
set meta_color = "#000000";
set topbar_bgcolor = "#1114a0";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#21c2f1";
set navbar_fgcolor = "#0000ff";
set link_color = "#0000cc";
set vlink_color = "#000088";
set alink_color = "#0000ff";
set comment_bar_one_bgcolor = "#1114a0";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#21c2f1";
set comment_bar_two_fgcolor = "#0000ff";
#NEWLAYER: cleansimple/ranchhand
layerinfo type = theme;
layerinfo name = "Ranch Hand";
layerinfo redist_uniq = "cleansimple/ranchhand";
set body_bgcolor = "#2999c2";
set entry_bgcolor = "#cfe0ff";
set entry_fgcolor = "#000000";
set meta_color = "#060667";
set topbar_bgcolor = "#54442c";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#704400";
set navbar_fgcolor = "#bababa";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#6a20ff";
set comment_bar_one_bgcolor = "#54442c";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#704400";
set comment_bar_two_fgcolor = "#bababa";
#NEWLAYER: cleansimple/victim
layerinfo type = theme;
layerinfo name = "Victim";
layerinfo redist_uniq = "cleansimple/victim";
set body_bgcolor = "#2cd0ff";
set entry_bgcolor = "#505050";
set entry_fgcolor = "#ffffff";
set meta_color = "#000000";
set topbar_bgcolor = "#166bac";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#353535";
set navbar_fgcolor = "#ffffff";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#ff00c0";
set comment_bar_one_bgcolor = "#166bac";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#353535";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: cleansimple/forest
layerinfo type = theme;
layerinfo name = "Forest";
layerinfo redist_uniq = "cleansimple/forest";
set body_bgcolor = "#778e64";
set entry_bgcolor = "#9b9ba5";
set entry_fgcolor = "#000000";
set meta_color = "#ffffff";
set topbar_bgcolor = "#72784c";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#73777a";
set navbar_fgcolor = "#000000";
set link_color = "#3811e1";
set vlink_color = "#310cbb";
set alink_color = "#4e7bef";
set comment_bar_one_bgcolor = "#72784c";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#73777a";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/drone
layerinfo type = theme;
layerinfo name = "Drone";
layerinfo redist_uniq = "cleansimple/drone";
set body_bgcolor = "#395f82";
set entry_bgcolor = "#f9fcfe";
set entry_fgcolor = "#000000";
set meta_color = "#000000";
set topbar_bgcolor = "#904094";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#eeeeff";
set navbar_fgcolor = "#000000";
set link_color = "#395f82";
set vlink_color = "#395f82";
set alink_color = "#5266ce";
set comment_bar_one_bgcolor = "#904094";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#eeeeff";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/lowercurtain
layerinfo type = theme;
layerinfo name = "Lower the Curtain";
layerinfo redist_uniq = "cleansimple/lowercurtain";
set body_bgcolor = "#000000";
set entry_bgcolor = "#6b6b6b";
set entry_fgcolor = "#ffffff";
set meta_color = "#ffffff";
set topbar_bgcolor = "#363636";
set topbar_fgcolor = "#f0f5fb";
set navbar_bgcolor = "#c5c8ca";
set navbar_fgcolor = "#000000";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#3314ba";
set comment_bar_one_bgcolor = "#363636";
set comment_bar_one_fgcolor = "#f0f5fb";
set comment_bar_two_bgcolor = "#c5c8ca";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/sunny
layerinfo type = theme;
layerinfo name = "Sunny Day";
layerinfo redist_uniq = "cleansimple/sunny";
set body_bgcolor = "#55e0f9";
set entry_bgcolor = "#e38202";
set entry_fgcolor = "#ffffff";
set meta_color = "#000000";
set topbar_bgcolor = "#ffba03";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#ffba55";
set navbar_fgcolor = "#ffffff";
set link_color = "#df0d12";
set vlink_color = "#ac1b25";
set alink_color = "#fe3b3b";
set comment_bar_one_bgcolor = "#ffba03";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ffba55";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: cleansimple/valentine
layerinfo type = theme;
layerinfo name = "Be Mine";
layerinfo redist_uniq = "cleansimple/valentine";
set body_bgcolor = "#6f104a";
set entry_bgcolor = "#f2bce9";
set entry_fgcolor = "#000000";
set meta_color = "#ff24ab";
set topbar_bgcolor = "#ff37ff";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#df2096";
set navbar_fgcolor = "#000000";
set link_color = "#ffffff";
set vlink_color = "#a51014";
set alink_color = "#ed8188";
set comment_bar_one_bgcolor = "#ff37ff";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#df2096";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: cleansimple/stripes
layerinfo type = theme;
layerinfo name = "Stripes";
layerinfo redist_uniq = "cleansimple/stripes";
set body_bgcolor = "#ffffff";
set entry_bgcolor = "#ffffff";
set entry_fgcolor = "#000000";
set meta_color = "#ff0000";
set topbar_bgcolor = "#e7212a";
set topbar_fgcolor = "#ffffff";
set navbar_bgcolor = "#ffcfdc";
set navbar_fgcolor = "#000000";
set link_color = "#000050";
set vlink_color = "#500050";
set alink_color = "#ffafc1";
set comment_bar_one_bgcolor = "#e7212a";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ffcfdc";
set comment_bar_two_fgcolor = "#000000";

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

View File

@ -0,0 +1,183 @@
#NEWLAYER: deardiary/royalty
layerinfo "type" = "theme";
layerinfo "name" = "Royalty";
layerinfo "redist_uniq" = "deardiary/royalty";
#NEWLAYER: deardiary/redrock
layerinfo "type" = "theme";
layerinfo "name" = "Red Rock";
layerinfo "redist_uniq" = "deardiary/redrock";
set clr_sidebar_bg = "#990000";
set clr_sidebar_fg = "#ff9933";
set clr_page_fg = "#dddddd";
set clr_link_visited = "#ee7711";
set clr_link_normal = "#ff9933";
set clr_title_pattern = "#ff9933";
set clr_page_bg = "#333333";
set clr_title_bg = "#660000";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/wired
layerinfo "type" = "theme";
layerinfo "name" = "Wired";
layerinfo "redist_uniq" = "deardiary/wired";
set clr_sidebar_bg = "#0183fd";
set title_pattern = "wires.png";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#0063c0";
set clr_link_normal = "#0183fd";
set clr_title_pattern = "#faf49a";
set clr_page_bg = "#e3f1ff";
set clr_title_bg = "#0063c0";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/everred
layerinfo "type" = "theme";
layerinfo "name" = "Ever Red";
layerinfo "redist_uniq" = "deardiary/everred";
set clr_sidebar_bg = "#e10b52";
set title_pattern = "bubblewrap.png";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#c0003b";
set clr_link_normal = "#e10b52";
set clr_title_pattern = "#d5fdf9";
set clr_page_bg = "#ffffff";
set clr_title_bg = "#c0003b";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/cellular
layerinfo "type" = "theme";
layerinfo "name" = "Cellular";
layerinfo "redist_uniq" = "deardiary/cellular";
set clr_sidebar_bg = "#3d9aff";
set title_pattern = "explosion.png";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#017fff";
set clr_link_normal = "#3d9aff";
set clr_title_pattern = "#ff8312";
set clr_page_bg = "#ffffff";
set clr_title_bg = "#69e8f9";
set clr_title_fg = "#000000";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/deepblue
layerinfo "type" = "theme";
layerinfo "name" = "Deep Blue";
layerinfo "redist_uniq" = "deardiary/deepblue";
set clr_sidebar_bg = "#336699";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#FFFFFF";
set clr_link_visited = "#9FBFDD";
set clr_link_normal = "#9FBFDD";
set clr_title_pattern = "#9FBFDD";
set clr_page_bg = "#003366";
set clr_title_bg = "#6699CC";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/unsaturates
layerinfo "type" = "theme";
layerinfo "name" = "Unsaturates";
layerinfo "redist_uniq" = "deardiary/unsaturates";
set clr_sidebar_bg = "#838383";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#838383";
set clr_link_normal = "#c0c0c0";
set clr_title_pattern = "#ffffff";
set clr_page_bg = "#ffffff";
set clr_title_bg = "#c0c0c0";
set clr_title_fg = "#000000";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/nature
layerinfo "type" = "theme";
layerinfo "name" = "Back To Nature";
layerinfo "redist_uniq" = "deardiary/nature";
set clr_sidebar_bg = "#66cc66";
set title_pattern = "nature.png";
set clr_sidebar_fg = "#000000";
set clr_page_fg = "#000000";
set clr_link_visited = "#007800";
set clr_link_normal = "#007800";
set clr_title_pattern = "#d6ead6";
set clr_page_bg = "#d6ead6";
set clr_title_bg = "#cc9966";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/slinkypink
layerinfo "type" = "theme";
layerinfo "name" = "Slinky Pink";
layerinfo "redist_uniq" = "deardiary/slinkypink";
set clr_sidebar_bg = "#cc6699";
set title_pattern = "ramblings.png";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#ad437a";
set clr_link_normal = "#ad437a";
set clr_title_pattern = "#d5d5c3";
set clr_page_bg = "#ffc5e3";
set clr_title_bg = "#993366";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#000000";
#NEWLAYER: deardiary/striking
layerinfo "type" = "theme";
layerinfo "name" = "Striking";
layerinfo "redist_uniq" = "deardiary/striking";
set title_separator = false;
set clr_sidebar_bg = "#cc0000";
set title_pattern = "letters.png";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#cc0000";
set clr_link_normal = "#cc0000";
set clr_title_pattern = "#ff0f0f";
set clr_page_bg = "#ffffff";
set clr_title_bg = "#000000";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#cc0000";
#NEWLAYER: deardiary/regal
layerinfo "type" = "theme";
layerinfo "name" = "Regal";
layerinfo "redist_uniq" = "deardiary/regal";
set title_separator = false;
set clr_sidebar_bg = "#a000ce";
set clr_sidebar_fg = "#ffffff";
set clr_page_fg = "#000000";
set clr_link_visited = "#a000ce";
set clr_link_normal = "#a000ce";
set clr_title_pattern = "#a000ce";
set clr_page_bg = "#ffffff";
set clr_title_bg = "#000000";
set clr_title_fg = "#ffffff";
set clr_title_separator = "#a000ce";

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

View File

@ -0,0 +1,10 @@
# -*-s2-*-
layerinfo type = "i18n";
layerinfo name = "English";
layerinfo redist_uniq = "digitalmultiplex/en";
set text_meta_music = "Music";
set text_meta_mood = "Mood";
set text_sidebar_link_separator = "&nbsp;&nbsp;&raquo;&nbsp;&nbsp;";

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,4 @@
#NEWLAYER: digitalmultiplex/classic
layerinfo type = "theme";
layerinfo name = "Classic";
layerinfo redist_uniq = "digitalmultiplex/classic";

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,169 @@
#NEWLAYER: disjointed/blackblu
layerinfo "type" = "theme";
layerinfo "name" = "Blackblu";
layerinfo "redist_uniq" = "disjointed/blackblu";
layerinfo "author_name" = "VahnYughi @ LiveJournal.com";
set main_bgcolor = "#000000";
set under_bgcolor = "#c0c0c0";
set under_fgcolor = "#000000";
set over_bgcolor = "#0000ff";
set over_fgcolor = "#ffffff";
set bordercolor = "#000000";
set color_comment_bar = "#c0c0c0";
set comment_screened_bgcolor = "#ffaaaa";
set comment_screened_fgcolor = "#000000";
#NEWLAYER: disjointed/blues
layerinfo "type" = "theme";
layerinfo "name" = "Blues";
layerinfo "redist_uniq" = "disjointed/blues";
layerinfo "author_name" = "Jaybeda1 @ LiveJournal.com";
set main_bgcolor = "#ffffff";
set under_bgcolor = "#336699";
set under_fgcolor = "#99cccc";
set over_bgcolor = "#99cccc";
set over_fgcolor = "#336699";
set bordercolor = "#000000";
set color_comment_bar = "#336699";
set comment_screened_bgcolor = "#996633";
set comment_screened_fgcolor = "#99cccc";
#NEWLAYER: disjointed/dante
layerinfo "type" = "theme";
layerinfo "name" = "Dante's Cheese Grater";
layerinfo "redist_uniq" = "disjointed/dante";
layerinfo "author_name" = "31337_n1nj4 @ LiveJournal.com";
set main_bgcolor = "#000000";
set under_bgcolor = "#333333";
set under_fgcolor = "#00ff99";
set over_bgcolor = "#555555";
set over_fgcolor = "#00ff99";
set bordercolor = "#000000";
set color_comment_bar = "#6d6d6d";
set comment_screened_bgcolor = "#333333";
set comment_screened_fgcolor = "#00ff99";
#NEWLAYER: disjointed/deepmelodrama
layerinfo "type" = "theme";
layerinfo "name" = "Deep Melorama";
layerinfo "redist_uniq" = "disjointed/deepmelodrama";
layerinfo "author_name" = "Giapet @ LiveJournal.com";
set main_bgcolor = "#872d89";
set under_bgcolor = "#719cff";
set under_fgcolor = "#9348a1";
set over_bgcolor = "#3794b3";
set over_fgcolor = "#a4b8ff";
set bordercolor = "#000000";
set color_comment_bar = "#65b2c1";
set comment_screened_bgcolor = "#21677b";
set comment_screened_fgcolor = "#a4b8ff";
set main_link = "#f5d3ff";
set main_alink = "#e2ffe3";
set main_vlink = "#e2ffe3";
#NEWLAYER: disjointed/greenhues
layerinfo "type" = "theme";
layerinfo "name" = "Green Hues";
layerinfo "redist_uniq" = "disjointed/greenhues";
layerinfo "author_name" = "EduThePenguin @ LiveJournal.com";
set main_bgcolor = "#2b412c";
set under_bgcolor = "#90d599";
set under_fgcolor = "#32442e";
set over_bgcolor = "#086023";
set over_fgcolor = "#ffffff";
set bordercolor = "#000000";
set color_comment_bar = "#6f8270";
set comment_screened_bgcolor = "#698d14";
set comment_screened_fgcolor = "#32442e";
set main_link = "#ffffff";
set main_alink = "#ffffff";
set main_vlink = "#ffffff";
#NEWLAYER: disjointed/monotonegrey
layerinfo "type" = "theme";
layerinfo "name" = "Monotone Grey";
layerinfo "redist_uniq" = "disjointed/monotonegrey";
layerinfo "author_name" = "MasterSlacker @ LiveJournal.com";
set main_bgcolor = "#999999";
set under_bgcolor = "#bbbbbb";
set under_fgcolor = "#000000";
set over_bgcolor = "#dddddd";
set over_fgcolor = "#000000";
set bordercolor = "#000000";
set color_comment_bar = "#bbbbbb";
set comment_screened_bgcolor = "#dddddd";
set comment_screened_fgcolor = "#000000";
#NEWLAYER: disjointed/periwinkle
layerinfo "type" = "theme";
layerinfo "name" = "Periwinkle";
layerinfo "redist_uniq" = "disjointed/periwinkle";
#NEWLAYER: disjointed/satinhandshake
layerinfo "type" = "theme";
layerinfo "name" = "Satin Handshake";
layerinfo "redist_uniq" = "disjointed/satinhandshake";
layerinfo "author_name" = "Giapet @ LiveJournal.com";
set main_bgcolor = "#480c0c";
set under_bgcolor = "#d06464";
set under_fgcolor = "#000000";
set over_bgcolor = "#9d0404";
set over_fgcolor = "#ffffff";
set bordercolor = "#000000";
set color_comment_bar = "#d06464";
set comment_screened_bgcolor = "#480c0c";
set comment_screened_fgcolor = "#000000";
set main_link = "#ffffff";
set main_alink = "#333333";
set main_vlink = "#6f2222";
#NEWLAYER: disjointed/subduedblues
layerinfo "type" = "theme";
layerinfo "name" = "Subdued Blues";
layerinfo "redist_uniq" = "disjointed/subduedblues";
layerinfo "author_name" = "MasterSlacker @ LiveJournal.com";
set main_bgcolor = "#314667";
set under_bgcolor = "#aabbdd";
set under_fgcolor = "#000000";
set over_bgcolor = "#6b7da6";
set over_fgcolor = "#000000";
set bordercolor = "#d4ddee";
set color_comment_bar = "#aabbdd";
set comment_screened_bgcolor = "#6b7da6";
set comment_screened_fgcolor = "#000000";
set main_link = "#000000";
set main_alink = "#000000";
set main_vlink = "#000000";
#NEWLAYER: disjointed/subduedpurples
layerinfo "type" = "theme";
layerinfo "name" = "Subdued Purples";
layerinfo "redist_uniq" = "disjointed/subduedpurples";
layerinfo "author_name" = "MasterSlacker @ LiveJournal.com";
set main_bgcolor = "#563976";
set under_bgcolor = "#bfaad7";
set under_fgcolor = "#000000";
set over_bgcolor = "#dcd1e9";
set over_fgcolor = "#000000";
set bordercolor = "#000000";
set color_comment_bar = "#bfaad7";
set comment_screened_bgcolor = "#dcd1e9";
set comment_screened_fgcolor = "#000000";
set main_link = "#000000";
set main_alink = "#000000";
set main_vlink = "#000000";
#NEWLAYER: disjointed/xcolibur
layerinfo "type" = "theme";
layerinfo "name" = "XColibur";
layerinfo "redist_uniq" = "disjointed/xcolibur";
layerinfo "author_name" = "Brandon_Scott @ LiveJournal.com";
set main_bgcolor = "#ffffff";
set under_bgcolor = "#003366";
set under_fgcolor = "#ffffff";
set over_bgcolor = "#a7c7e8";
set over_fgcolor = "#000000";
set bordercolor = "#ffffff";
set color_comment_bar = "#ffffff";
set comment_screened_bgcolor = "#aaaaaa";
set comment_screened_fgcolor = "#ffffff";

View File

@ -0,0 +1,17 @@
# -*-s2-*-
layerinfo type = "i18n";
layerinfo name = "English";
layerinfo redist_uniq = "generator/en";
set text_meta_music = "music";
set text_meta_mood = "mood";
set text_permalink = "link";
set text_post_comment = "post comment";
set text_post_comment_friends = "post comment";
set text_read_comments = "1 comment // # comments";
set text_read_comments_friends = "1 comment // # comments";

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.9 KiB

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,374 @@
#NEWLAYER: generator/nautical
layerinfo "type" = "theme";
layerinfo "name" = "Nautical";
layerinfo "redist_uniq" = "generator/nautical";
#NEWLAYER: generator/mintchoc
layerinfo "type" = "theme";
layerinfo "name" = "Mint Chocolate Chip";
layerinfo "redist_uniq" = "generator/mintchoc";
layerinfo "author_name" = "Andrea Hartmann";
layerinfo "author_email" = "mullenkamp@livejournal.com";
set entry_back = "#ddffee";
set entry_text = "#330000";
set page_link = "#993333";
set page_vlink = "#663333";
set page_alink = "#000000";
set page_back = "#ccffdd";
set stronger_back = "#663333";
set stronger_text = "#ccffdd";
set weak_back = "#88ffdd";
set weak_text = "#663333";
set comment_bar_one_bgcolor = "#88ffdd";
set comment_bar_one_fgcolor = "#663333";
set comment_bar_two_fgcolor = "#663333";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/purple
layerinfo "type" = "theme";
layerinfo "name" = "Purple";
layerinfo "redist_uniq" = "generator/purple";
layerinfo "author_name" = "Andrea Hartmann";
layerinfo "author_email" = "mullenkamp@livejournal.com";
set entry_back = "#9933cc";
set entry_text = "#ddccff";
set page_link = "#ffffff";
set page_vlink = "#330033";
set page_alink = "#ffccff";
set page_back = "#bb55ee";
set stronger_back = "#660099";
set stronger_text = "#f5eeff";
set weak_back = "#bb88dd";
set weak_text = "#ddccff";
set comment_bar_one_bgcolor = "#660099";
set comment_bar_one_fgcolor = "#f5eeff";
set comment_bar_two_bgcolor = "#bb88dd";
set comment_bar_two_fgcolor = "#ddccff";
#NEWLAYER: generator/ocean
layerinfo "type" = "theme";
layerinfo "name" = "Deep Blue Xenogears Ocean";
layerinfo "redist_uniq" = "generator/ocean";
layerinfo "author_name" = "Andrea Hartmann";
layerinfo "author_email" = "mullenkamp@livejournal.com";
set entry_back = "#1100dd";
set entry_text = "#ccccff";
set page_link = "#ffffff";
set page_vlink = "#ddddff";
set page_alink = "#ffffff";
set page_back = "#3333ff";
set stronger_back = "#aabbff";
set stronger_text = "#0033cc";
set weak_back = "#3399ff";
set weak_text = "#0033cc";
set comment_bar_one_bgcolor = "#aabbff";
set comment_bar_one_fgcolor = "#0033cc";
set comment_bar_two_bgcolor = "#3399ff";
set comment_bar_two_fgcolor = "#0033cc";
#NEWLAYER: generator/harvest
layerinfo "type" = "theme";
layerinfo "name" = "Harvest";
layerinfo "redist_uniq" = "generator/harvest";
layerinfo "author_name" = "Andrea Hartmann";
layerinfo "author_email" = "mullenkamp@livejournal.com";
set entry_back = "#ff9900";
set entry_text = "#663333";
set page_link = "#ffeeaa";
set page_vlink = "#993300";
set page_alink = "#ffcc88";
set page_back = "#ffaa11";
set stronger_back = "#ffcc33";
set stronger_text = "#663333";
set weak_back = "#cc6600";
set weak_text = "#ffcc88";
set comment_bar_one_bgcolor = "#ffcc33";
set comment_bar_one_fgcolor = "#663333";
set comment_bar_two_bgcolor = "#cc6600";
set comment_bar_two_fgcolor = "#ffcc88";
#NEWLAYER: generator/jeweled
layerinfo "type" = "theme";
layerinfo "name" = "Jeweled";
layerinfo "redist_uniq" = "generator/jeweled";
layerinfo "author_name" = "Andrea Hartmann";
layerinfo "author_email" = "mullenkamp@livejournal.com";
set entry_back = "#aa6699";
set entry_text = "#330066";
set page_link = "#ccddee";
set page_vlink = "#cc99cc";
set page_alink = "#cceecc";
set page_back = "#881188";
set stronger_back = "#006699";
set stronger_text = "#ceecc";
set weak_back = "#008888";
set weak_text = "#002222";
set comment_bar_one_bgcolor = "#006699";
set comment_bar_one_fgcolor = "#ceecc";
set comment_bar_two_bgcolor = "#008888";
set comment_bar_two_fgcolor = "#002222";
#NEWLAYER: generator/darkgreens
layerinfo type = "theme";
layerinfo name = "Dark Greens";
layerinfo redist_uniq = "generator/darkgreens";
layerinfo author_name = "Ryan Fitzpatrick";
set entry_back = "#020202";
set entry_text = "#fbfbfb";
set page_link = "#9aff8d";
set page_vlink = "#e3ee39";
set page_alink = "#ff0220";
set page_back = "#117218";
set stronger_back = "#38e616";
set stronger_text = "#020100";
set weak_back = "#020100";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#38e616";
set comment_bar_one_fgcolor = "#020100";
set comment_bar_two_bgcolor = "#117218";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/satin
layerinfo type = theme;
layerinfo name = "Satin Handshake";
layerinfo redist_uniq = "generator/satin";
set entry_back = "#d06464";
set entry_text = "#000000";
set page_link = "#ffffff";
set page_vlink = "#6f2222";
set page_alink = "#333333";
set page_back = "#480c0c";
set stronger_back = "#9d0404";
set stronger_text = "#ffffff";
set weak_back = "#d06464";
set weak_text = "#ffffff";
set comment_bar_one_bgcolor = "#9d0404";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#d06464";
set comment_bar_two_fgcolor = "#ffffff";
#NEWLAYER: generator/sunset
layerinfo type = theme;
layerinfo name = "Sunset";
layerinfo redist_uniq = "generator/sunset";
set entry_back = "#ff7301";
set entry_text = "#fdb54f";
set page_link = "#ffe65e";
set page_vlink = "#ff9879";
set page_alink = "#ffb866";
set page_back = "#070494";
set stronger_back = "#f8a402";
set stronger_text = "#ffd510";
set weak_back = "#f51700";
set weak_text = "#f77603";
set comment_bar_one_bgcolor = "#f8a402";
set comment_bar_one_fgcolor = "#ffd510";
set comment_bar_two_bgcolor = "#f51700";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/redbliss
layerinfo type = theme;
layerinfo name = "Red Bliss";
layerinfo redist_uniq = "generator/redbliss";
set entry_back = "#ffcccc";
set entry_text = "#000000";
set page_link = "#ffffff";
set page_vlink = "#ffcccc";
set page_alink = "#00ffff";
set page_back = "#ff3300";
set stronger_back = "#cc3333";
set stronger_text = "#ffffff";
set weak_back = "#cc3333";
set weak_text = "#fafafa";
set comment_bar_one_bgcolor = "#cc3333";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ff3333";
set comment_bar_two_fgcolor = "#fafafa";
#NEWLAYER: generator/pastelneons
layerinfo type = theme;
layerinfo name = "Pastel Neons";
layerinfo redist_uniq = "generator/pastelneons";
set entry_back = "#ffffff";
set entry_text = "#1a6b6a";
set page_link = "#330f42";
set page_vlink = "#033339";
set page_alink = "#9923ca";
set page_back = "#00ffff";
set stronger_back = "#60d2cb";
set stronger_text = "#000000";
set weak_back = "#d385f3";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#60d2cb";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#d385f3";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/classdesk
layerinfo type = "theme";
layerinfo name = "Classic Desktop";
layerinfo redist_uniq = "generator/classdesk";
set page_link = "#ff7b05";
set page_vlink = "#ce4900";
set page_alink = "#ff9e2b";
set page_back = "#007782";
set stronger_back = "#00545c";
set stronger_text = "#ffffff";
set weak_back = "#ffeddd";
set weak_text = "#ffffff";
set comment_bar_one_bgcolor = "#00545c";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#ffeddd";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/everwhite
layerinfo type = theme;
layerinfo name = "Everwhite";
layerinfo redist_uniq = "generator/everwhite";
set entry_back = "#ffffff";
set entry_text = "#000000";
set page_link = "#ff0000";
set page_vlink = "#ff0000";
set page_alink = "#ffffff";
set page_back = "#ffffff";
set stronger_back = "#ffffff";
set stronger_text = "#ff0000";
set weak_back = "#ffffff";
set weak_text = "#ffffff";
set comment_bar_one_bgcolor = "#dddddd";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#aaaaaa";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/everblue
layerinfo type = theme;
layerinfo name = "Everblue with Greys";
layerinfo redist_uniq = "generator/everblue";
set entry_back = "#ffffff";
set entry_text = "#000000";
set page_link = "#0000ff";
set page_vlink = "#060667";
set page_alink = "#ffffff";
set page_back = "#0f0c6d";
set stronger_back = "#000000";
set stronger_text = "#ffffff";
set weak_back = "#aaaaaa";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#000000";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#aaaaaa";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/purplesky
layerinfo type = theme;
layerinfo name = "Purple Sky";
layerinfo redist_uniq = "generator/purplesky";
set entry_back = "#24d8df";
set entry_text = "#0424e4";
set page_link = "#a917c4";
set page_vlink = "#9911e8";
set page_alink = "#ea09ff";
set page_back = "#99e3e2";
set stronger_back = "#8802fa";
set stronger_text = "#ffffff";
set weak_back = "#89e5e9";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#8802fa";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#89e5e9";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/bruise
layerinfo type = theme;
layerinfo name = "Bruise";
layerinfo redist_uniq = "generator/bruise";
set entry_back = "#bcbcbc";
set entry_text = "#000000";
set page_link = "#0000ff";
set page_vlink = "#0000ff";
set page_alink = "#0000ff";
set page_back = "#000000";
set stronger_back = "#1114a0";
set stronger_text = "#ffffff";
set weak_back = "#21c2f1";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#1114a0";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#21c2f1";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/elegant
layerinfo type = theme;
layerinfo name = "Elegant";
layerinfo redist_uniq = "generator/elegant";
set entry_back = "#ffffff";
set entry_text = "#000000";
set page_link = "#e91822";
set page_vlink = "#bf1418";
set page_alink = "#f71e28";
set page_back = "#ffffff";
set stronger_back = "#777777";
set stronger_text = "#000000";
set weak_back = "#e6e6e6";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#777777";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#e6e6e6";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/bananapeel
layerinfo type = theme;
layerinfo name = "Banana Peel";
layerinfo redist_uniq = "generator/bananapeel";
set entry_back = "#f2f688";
set entry_text = "#ee2914";
set page_link = "#db9129";
set page_vlink = "#c58f1b";
set page_alink = "#ffc518";
set page_back = "#ffffff";
set stronger_back = "#f0f905";
set stronger_text = "#000000";
set weak_back = "#dcdf02";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#f0f905";
set comment_bar_one_fgcolor = "#000000";
set comment_bar_two_bgcolor = "#dcdf02";
set comment_bar_two_fgcolor = "#000000";
#NEWLAYER: generator/melodrama
layerinfo type = theme;
layerinfo name = "Deep MeloDrama";
layerinfo redist_uniq = "generator/melodrama";
set entry_back = "#719cff";
set entry_text = "#9348a1";
set page_link = "#f5d3ff";
set page_vlink = "#e2ffe3";
set page_alink = "#e2ffe3";
set page_back = "#872d89";
set stronger_back = "#3794b3";
set stronger_text = "#a4b8ff";
set weak_back = "#65b2c1";
set weak_text = "#98bac5";
set comment_bar_one_bgcolor = "#3794b3";
set comment_bar_one_fgcolor = "#a4b8ff";
set comment_bar_two_bgcolor = "#65b2c1";
set comment_bar_two_fgcolor = "#872d89";
#NEWLAYER: generator/iceburg
layerinfo type = theme;
layerinfo name = "Iceburg";
layerinfo redist_uniq = "generator/iceburg";
set entry_back = "#ffffff";
set entry_text = "#000000";
set page_link = "#0000ff";
set page_vlink = "#0000ff";
set page_alink = "#387bf7";
set page_back = "#c6c2f5";
set stronger_back = "#7c6ccd";
set stronger_text = "#ffffff";
set weak_back = "#eef0fd";
set weak_text = "#000000";
set comment_bar_one_bgcolor = "#7c6ccd";
set comment_bar_one_fgcolor = "#ffffff";
set comment_bar_two_bgcolor = "#eef0fd";
set comment_bar_two_fgcolor = "#000000";

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,396 @@
#NEWLAYER: haven/indigoblue
layerinfo "type" = "theme";
layerinfo "name" = "Indigo Blue";
layerinfo "redist_uniq" = "haven/indigoblue";
#NEWLAYER: haven/bluemonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Nautical Blue";
layerinfo "redist_uniq" = "haven/bluemonochromatic";
set color_scheme_base = "#1575C7";
set color_scheme = "monochromatic";
#NEWLAYER: haven/blueanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Blue Teal";
layerinfo "redist_uniq" = "haven/blueanalogous";
set color_scheme_base = "#1575C7";
set color_scheme = "analogous";
#NEWLAYER: haven/bluecomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Tan Blues";
layerinfo "redist_uniq" = "haven/bluecomplementary";
set color_scheme_base = "#1575C7";
set color_scheme = "complementary";
#NEWLAYER: haven/bluesplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Nautical Birch";
layerinfo "redist_uniq" = "haven/bluesplit_complementary";
set color_scheme_base = "#1575C7";
set color_scheme = "split_complementary";
#NEWLAYER: haven/bluedouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Multi Olive";
layerinfo "redist_uniq" = "haven/bluedouble_complementary";
set color_scheme_base = "#1575C7";
set color_scheme = "double_complementary";
#NEWLAYER: haven/bluetriadic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Flourescents";
layerinfo "redist_uniq" = "haven/bluetriadic";
set color_scheme_base = "#1575C7";
set color_scheme = "triadic";
#NEWLAYER: haven/bluetetradic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Orange Blues";
layerinfo "redist_uniq" = "haven/bluetetradic";
set color_scheme_base = "#1575C7";
set color_scheme = "tetradic";
#NEWLAYER: haven/orangemonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Orange Sorbet";
layerinfo "redist_uniq" = "haven/orangemonochromatic";
set color_scheme_base = "#F3904E";
set color_scheme = "monochromatic";
#NEWLAYER: haven/orangeanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Earthy Rose";
layerinfo "redist_uniq" = "haven/orangeanalogous";
set color_scheme_base = "#F3904E";
set color_scheme = "analogous";
#NEWLAYER: haven/orangecomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Nautical Orange";
layerinfo "redist_uniq" = "haven/orangecomplementary";
set color_scheme_base = "#F3904E";
set color_scheme = "complementary";
#NEWLAYER: haven/orangesplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Blue Oranges";
layerinfo "redist_uniq" = "haven/orangesplit_complementary";
set color_scheme_base = "#F3904E";
set color_scheme = "split_complementary";
#NEWLAYER: haven/orangedouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Multi Blue Tans";
layerinfo "redist_uniq" = "haven/orangedouble_complementary";
set color_scheme_base = "#F3904E";
set color_scheme = "double_complementary";
#NEWLAYER: haven/orangetriadic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Light Purple";
layerinfo "redist_uniq" = "haven/orangetriadic";
set color_scheme_base = "#F3904E";
set color_scheme = "triadic";
#NEWLAYER: haven/orangetetradic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Blue Orange";
layerinfo "redist_uniq" = "haven/orangetetradic";
set color_scheme_base = "#F3904E";
set color_scheme = "tetradic";
#NEWLAYER: haven/greenmonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Forest Green";
layerinfo "redist_uniq" = "haven/greenmonochromatic";
set color_scheme_base = "#48C754";
set color_scheme = "monochromatic";
#NEWLAYER: haven/greenanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Fresh Green";
layerinfo "redist_uniq" = "haven/greenanalogous";
set color_scheme_base = "#48C754";
set color_scheme = "analogous";
#NEWLAYER: haven/greencomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Purple Greens";
layerinfo "redist_uniq" = "haven/greencomplementary";
set color_scheme_base = "#48C754";
set color_scheme = "complementary";
#NEWLAYER: haven/greensplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Rose Garden";
layerinfo "redist_uniq" = "haven/greensplit_complementary";
set color_scheme_base = "#48C754";
set color_scheme = "split_complementary";
#NEWLAYER: haven/greendouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Mauve Greens";
layerinfo "redist_uniq" = "haven/greendouble_complementary";
set color_scheme_base = "#48C754";
set color_scheme = "double_complementary";
#NEWLAYER: haven/greentriadic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Rose";
layerinfo "redist_uniq" = "haven/greentriadic";
set color_scheme_base = "#48C754";
set color_scheme = "triadic";
#NEWLAYER: haven/greentetradic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Purple Greens";
layerinfo "redist_uniq" = "haven/greentetradic";
set color_scheme_base = "#48C754";
set color_scheme = "tetradic";
#NEWLAYER: haven/violetmonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Purple Power";
layerinfo "redist_uniq" = "haven/violetmonochromatic";
set color_scheme_base = "#9D1EC7";
set color_scheme = "monochromatic";
#NEWLAYER: haven/violetanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Mauve Blue";
layerinfo "redist_uniq" = "haven/violetanalogous";
set color_scheme_base = "#9D1EC7";
set color_scheme = "analogous";
#NEWLAYER: haven/violetcomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Hyper Green";
layerinfo "redist_uniq" = "haven/violetcomplementary";
set color_scheme_base = "#9D1EC7";
set color_scheme = "complementary";
#NEWLAYER: haven/violetsplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Mint Purple";
layerinfo "redist_uniq" = "haven/violetsplit_complementary";
set color_scheme_base = "#9D1EC7";
set color_scheme = "split_complementary";
#NEWLAYER: haven/violetdouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Multi Cyber Green";
layerinfo "redist_uniq" = "haven/violetdouble_complementary";
set color_scheme_base = "#9D1EC7";
set color_scheme = "double_complementary";
#NEWLAYER: haven/violettriadic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Teal";
layerinfo "redist_uniq" = "haven/violettriadic";
set color_scheme_base = "#9D1EC7";
set color_scheme = "triadic";
#NEWLAYER: haven/violettetradic
layerinfo "type" = "theme";
layerinfo "name" = "Bright Green Purple";
layerinfo "redist_uniq" = "haven/violettetradic";
set color_scheme_base = "#9D1EC7";
set color_scheme = "tetradic";
#NEWLAYER: haven/yellowmonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Pale Yellows";
layerinfo "redist_uniq" = "haven/yellowmonochromatic";
set color_scheme_base = "#F3EAA0";
set color_scheme = "monochromatic";
#NEWLAYER: haven/yellowanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Olive Tans";
layerinfo "redist_uniq" = "haven/yellowanalogous";
set color_scheme_base = "#F3EAA0";
set color_scheme = "analogous";
#NEWLAYER: haven/yellowcomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Standard";
layerinfo "redist_uniq" = "haven/yellowcomplementary";
set color_scheme_base = "#F3EAA0";
set color_scheme = "complementary";
#NEWLAYER: haven/yellowsplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Purple Tans";
layerinfo "redist_uniq" = "haven/yellowsplit_complementary";
set color_scheme_base = "#F3EAA0";
set color_scheme = "split_complementary";
#NEWLAYER: haven/yellowdouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Multi Purple";
layerinfo "redist_uniq" = "haven/yellowdouble_complementary";
set color_scheme_base = "#F3EAA0";
set color_scheme = "double_complementary";
#NEWLAYER: haven/yellowtriadic
layerinfo "type" = "theme";
layerinfo "name" = "Pastels";
layerinfo "redist_uniq" = "haven/yellowtriadic";
set color_scheme_base = "#F3EAA0";
set color_scheme = "triadic";
#NEWLAYER: haven/yellowtetradic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Dark Blue";
layerinfo "redist_uniq" = "haven/yellowtetradic";
set color_scheme_base = "#F3EAA0";
set color_scheme = "tetradic";
#NEWLAYER: haven/redmonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Pink Reds";
layerinfo "redist_uniq" = "haven/redmonochromatic";
set color_scheme_base = "#F35951";
set color_scheme = "monochromatic";
#NEWLAYER: haven/redanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Rose Tans";
layerinfo "redist_uniq" = "haven/redanalogous";
set color_scheme_base = "#F35951";
set color_scheme = "analogous";
#NEWLAYER: haven/redcomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Salmon Teal";
layerinfo "redist_uniq" = "haven/redcomplementary";
set color_scheme_base = "#F35951";
set color_scheme = "complementary";
#NEWLAYER: haven/redsplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Red Blues";
layerinfo "redist_uniq" = "haven/redsplit_complementary";
set color_scheme_base = "#F35951";
set color_scheme = "split_complementary";
#NEWLAYER: haven/reddouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Multi Light Blue";
layerinfo "redist_uniq" = "haven/reddouble_complementary";
set color_scheme_base = "#F35951";
set color_scheme = "double_complementary";
#NEWLAYER: haven/redtriadic
layerinfo "type" = "theme";
layerinfo "name" = "Light Blue Greens";
layerinfo "redist_uniq" = "haven/redtriadic";
set color_scheme_base = "#F35951";
set color_scheme = "triadic";
#NEWLAYER: haven/redtetradic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Aqua";
layerinfo "redist_uniq" = "haven/redtetradic";
set color_scheme_base = "#F35951";
set color_scheme = "tetradic";
#NEWLAYER: haven/indigomonochromatic
layerinfo "type" = "theme";
layerinfo "name" = "Real Blue";
layerinfo "redist_uniq" = "haven/indigomonochromatic";
set color_scheme_base = "#6F60C7";
set color_scheme = "monochromatic";
#NEWLAYER: haven/indigoanalogous
layerinfo "type" = "theme";
layerinfo "name" = "Purple Blues";
layerinfo "redist_uniq" = "haven/indigoanalogous";
set color_scheme_base = "#6F60C7";
set color_scheme = "analogous";
#NEWLAYER: haven/indigocomplementary
layerinfo "type" = "theme";
layerinfo "name" = "Olive Blues";
layerinfo "redist_uniq" = "haven/indigocomplementary";
set color_scheme_base = "#6F60C7";
set color_scheme = "complementary";
#NEWLAYER: haven/indigosplit_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Sage Blues";
layerinfo "redist_uniq" = "haven/indigosplit_complementary";
set color_scheme_base = "#6F60C7";
set color_scheme = "split_complementary";
#NEWLAYER: haven/indigodouble_complementary
layerinfo "type" = "theme";
layerinfo "name" = "Multi Green";
layerinfo "redist_uniq" = "haven/indigodouble_complementary";
set color_scheme_base = "#6F60C7";
set color_scheme = "double_complementary";
#NEWLAYER: haven/indigotriadic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Mint";
layerinfo "redist_uniq" = "haven/indigotriadic";
set color_scheme_base = "#6F60C7";
set color_scheme = "triadic";
#NEWLAYER: haven/indigotetradic
layerinfo "type" = "theme";
layerinfo "name" = "Multi Olive Blues";
layerinfo "redist_uniq" = "haven/indigotetradic";
set color_scheme_base = "#6F60C7";
set color_scheme = "tetradic";

View File

@ -0,0 +1,367 @@
layerinfo type = "layout";
layerinfo name = "Hosted Comments";
layerinfo redist_uniq = "hostedcomments/layout";
layerinfo des = "A style with only comments";
function print_custom_stylesheet() {}
property bool no_show_control_strip;
set no_show_control_strip = true;
function print_stylesheet () {
print_custom_stylesheet();
}
function num_comments_in_thread (Comment[] comments) : int {
var int total = 0;
foreach var Comment c ($comments) {
$total = $total + 1;
if (size $c.replies > 0) {
$total = $total + num_comments_in_thread($c.replies);
}
}
return $total;
}
function container_open (string name) {
println safe """<div id="$name"><div id="$name-inner">""";
}
function container_close () {
"</div></div>\n";
}
function Page::print () {
"""<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
""";
$this->print_head();
$this->print_stylesheets();
print safe """<title>""" + $this->title() + """</title>\n""";
"""
</head>
<body>""";
$this->print_body();
"""</body>
</html>""";
}
function EntryPage::print_body() {
if ($.entry.comments.enabled and $.comment_pages.total_subitems > 0) {
"""<a id="comments"></a>
<div id="comments">
<div class="comments-inner">
<div class="comments-body">
<div class="comments-nav">
( """; $this->print_reply_link({"target" => "topcomment", "linktext" => $*text_post_comment}); """ )""";
$this->print_reply_container({"target" => "topcomment"});
"""</div>""";
if($.multiform_on) {
$this->print_multiform_start();
}
if(not $.comment_pages.all_subitems_displayed) {
$.comment_pages->print();
}
foreach var Comment c ($.comments) {
"""<div style="margin-left: 0px">""";
$this->print_comment($c);
"""</div>""";
}
if(not $.comment_pages.all_subitems_displayed) {
$.comment_pages->print();
}
if ($.multiform_on) {
print """<div style="text-align: center">"""; $this->print_multiform_actionline(); print "</div>";
$this->print_multiform_end();
}
"""
</div>
</div>
</div>
""";
}
}
function lang_posted_by_date_and_time(EntryLite e, bool showposter, bool showdate, bool showtime) : string {
var string posted = "";
if ($showposter) {
var string posterstr = (defined $e.poster ? ""+$e.poster : """<i class="noposter">$*text_poster_anonymous</i>""");
if (not $showtime and not $showdate) {
return $posted;
}
}
if ($showdate and $showtime) {
if ($e.depth > 0) { # If $e is a comment
return $posted + $e->time_display("med", "");
} else {
return $posted + $e->time_display($*lang_fmt_date_med + " at %%h%%:%%min%% %%A%%M", "none");
}
}
if ($showdate and not $showtime) {
return $posted + $e->time_display("med", "none");
}
if (not $showdate and $showtime) {
return $e->time_display("none", "%%h%%:%%min%% %%A%%M");
}
return "";
}
function Comment::print_linkbar() {
var Link link;
var string sep = """<span class="separator">|</span>""";
var string url = "";
var string text = "";
var string{} link_caption = {
# TODO: These should really be separate properties
"delete_comment" => $*text_multiform_opt_delete,
"freeze_thread" => $*text_multiform_opt_freeze,
"unfreeze_thread" => $*text_multiform_opt_unfreeze,
"screen_comment" => $*text_multiform_opt_screen,
"unscreen_comment" => $*text_multiform_opt_unscreen,
"watch_comments" => $*text_multiform_opt_track,
"unwatch_comments" => $*text_multiform_opt_untrack,
};
foreach var string link_key ($.link_keyseq) {
$link = $this->get_link($link_key);
if (defined $link) {
$url = $link.url;
$text = $link_caption{$link_key} != "" ? $link_caption{$link_key} : $link.caption;
print safe """ $sep <a href="$url">$text</a>""";
}
}
}
function CommentInfo::print_readlink {
var Page p = get_page();
var string show_screened = "";
if ($.screened) {
$show_screened = " <b>$*text_month_screened_comments</b>";
}
print safe "<a href=\"$.read_url\">"+
get_plural_phrase($.count, $p.view == "friends" ?
"text_read_comments_friends" : "text_read_comments")+
"$show_screened</a>";
}
function CommentInfo::print() {
if ($.show_readlink) {
"""<li class="asset-meta-comments item asset-meta-no-comments">"""; $this->print_readlink(); "</li>\n";
}
if ($.show_postlink) {
"""<li class="asset-meta-comments item asset-meta-no-comments">"""; $this->print_postlink(); "</li>\n";
}
}
function Entry::print_linkbar() {
var Link link;
var string url = "";
var string text = "";
print """<div class="asset-meta">\n""";
print """<ul class="asset-meta-list">\n""";
$.comments->print();
var string{} link_caption = {
"edit_entry" => $*text_edit_entry,
"edit_tags" => $*text_edit_tags,
"mem_add" => $*text_mem_add,
"watch_comments" => $*text_watch_comments,
"unwatch_comments" => $*text_unwatch_comments,
};
foreach var string link_key ($.link_keyseq) {
$link = $this->get_link($link_key);
if (defined $link) {
$url = $link.url;
$text = $link_caption{$link_key} != "" ? $link_caption{$link_key} : $link.caption;
print safe """<li class="asset-meta-comments item asset-meta-no-comments"><a href="$url">$text</a></li>""";
}
}
print """<li class="asset-meta-comments item asset-meta-no-comments"><a href="$.permalink_url">$*text_permalink</a></li>""";
println "\n</ul>\n</div>\n";
}
function Entry::print_metadata() {
if (size $.metadata) {
"""<div class="lj-currents">\n<ul>\n""";
foreach var string m ($.metadata) {
var string text = lang_metadata_title($m);
var string val = $.metadata{$m};
if ($m == "mood") {
if(($.mood_icon)) {
$val = " $.mood_icon " + $val;
}
}
print safe """<li><span class="entryMetadata-label">${text}:</span><span class="entryMetadata-content">$val</span></li>""";
}
"</ul>\n</div>\n";
}
if ((size($.tags) > 0) and ($*tags_aware)) {
var int tcount = 0;
"""<div class="asset-tags"><h4 class="asset-tags-header page-header-4">Tags:</h4>\n<ul class="asset-tags-list">\n """;
foreach var Tag t ($this.tags) {
"""<li class="item"><a rel="tag" href="$t.url">$t.name</a>""";
$tcount++;
if ($tcount != size $.tags) {
",";
}
"</li>";
}
"\n</ul>\n</div>\n";
}
}
function EntryPage::print_comment(Comment e) {
var string barc = "comment-" + ($e.depth % 2 ? "odd" : "even");
var string screenbarc = "";
var string borderwidth = "";
if ($e.screened) {
$screenbarc = "style=\"border-width: 3px; border-style: dashed\"";
}
var int num = 0;
if (not $e.full) {
# Collapsed mode
var string id = $e.dom_id ? " id=\"comment-$e.talkid\"" : "";
"""
<a name='$e.anchor'></a>
<div class="collapsed-comment"$id>
<div class="comment-inner">
<div class="comment-meta">""";
var string subject = $e.subject != "" ? $e.subject : """<i class="nosubject">$*text_nosubject</i>""";
print safe """<a href="$e.permalink_url" class="collapsed-comment-link">$subject</a> <span class="separator">-</span> """;
var string poster = (defined $e.poster ? ""+$e.poster : """<i class="noposter">$*text_poster_anonymous</i>""");
print safe """$poster <span class="separator">-</span> """;
print lang_posted_by_date_and_time($e, false, true, true);
if (size($e.replies) > 0) {
foreach var Comment c ($e.replies) {
$this->print_comment($c);
}
}
"""
</div>
</div>
</div>\n""";
return;
}
var string id = $e.dom_id ? " id=\"comment-$e.talkid\"" : "";
"""<a name='$e.anchor'></a>
<div $screenbarc class="$barc comment"$id>
<div class="comment-inner">
<div class="comment-meta">""";
if (defined $e.userpic) {
"""<div class="user-icon">$e.userpic</div>""";
}
var string poster = defined $e.poster ? $e.poster->as_string() : "<i>$*text_poster_anonymous</i>";
print safe """<span class="commenter-name">$poster """;
if ($e.metadata{"poster_ip"} != "") {
"(" + $e.metadata{"poster_ip"} + ") ";
}
"""wrote:</span><br />""";
var string datetime = lang_posted_by_date_and_time($e, true, true, true);
"""<div class="comment-date"><abbr class="datetime">$datetime</abbr></div>""";
if (defined $e.subject_icon) {
"""<div class="comment-subject-icon">$e.subject_icon</div>""";
}
if ($e.subject != "") {
"""<div class="comment-subject">$e.subject</div>""";
}
"""<div class="comment-body">""";
$e->print_text();
"""</div>
<div class="comment-links">""";
var string sep = """<span class="separator">|</span>""";
"""<a class="permalink" href="$e.permalink_url">$*text_permalink</a>""";
if ($e.frozen) {
print safe " $sep $*text_comment_frozen";
} else {
" $sep "; $e->print_reply_link({"linktext" => $*text_comment_reply});
}
if ($e.parent_url) { print """ $sep <a href="$e.parent_url">$*text_comment_parent</a>"""; }
if ($e.thread_url) { print """ $sep <a href="$e.thread_url">$*text_comment_thread</a>"""; }
$e->print_linkbar();
if ($this.multiform_on) {
println safe """ <label for="ljcomsel_$e.talkid">$*text_multiform_check</label>""";
$e->print_multiform_check();
}
"</div>\n</div>\n</div>\n</div>\n";
if (not $e.frozen) {
$e->print_reply_container({"class" => "quickreply"});
}
if (size($e.replies) > 0) {
foreach var Comment c ($e.replies) {
"""<div style="margin-left: 35px">""";
$this->print_comment($c);
"""</div>""";
}
}
}
function ReplyPage::print_body
{
if (not $.entry.comments.enabled) {
print "<h2>$*text_reply_nocomments_header</h2><p>$*text_reply_nocomments</p>";
return;
}
print "\n<hr />\n";
container_open("comments");
"""
<h2 class="comments-header page-header2">$*text_replyform_header</h2>
<div class="comments-body">
<div class="comments-nav">
( <a href="$.entry.permalink_url">$*text_reply_back</a> )
</div>
<a name="replyform"></a>
<div class="replyform">""";
$.form->print();
"""
<div class="comments-nav">
( <a href="$.entry.permalink_url">$*text_reply_back</a> )
</div>
</div>
</div>""";
container_close();
}

View File

@ -0,0 +1,200 @@
#-*-s2-*- ;; -*- coding: utf-8 -*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/be1";
layerinfo "name" = "Belarusian";
layerinfo "langcode" = "be";
layerinfo "author_name" = "mirritil";
set lang_current = "be";
# Short date format. All numeric.
set lang_fmt_date_short = "%%d%%.%%m%%.%%yy%%";
# Medium date format. Abbreviated month name, no day of the week.
set lang_fmt_date_med = "%%dd%% %%mon%% %%yyyy%%";
# Medium date format with day of week. Abbreviated month name and abbreviated day of the week.
set lang_fmt_date_med_day = "%%dd%% %%mon%% %%yyyy%%, %%da%%";
# Long date format. With full month name, but no day of the week.
set lang_fmt_date_long = "%%dd%% %%month%% %%yyyy%%";
# Long date format. With full month name and full day of the week.
set lang_fmt_date_long_day = "%%dd%% %%month%% %%yyyy%%, %%day%%";
# Time format.
set lang_fmt_time_short = "%%HH%%:%%min%%";
# Short month format.
set lang_fmt_month_short = "%%mm%%.%%yyyy%%";
# Medium month format.
set lang_fmt_month_med = "%%month%% %%yyyy%%";
# Long month format.
set lang_fmt_month_long = "%%mon%% %%yyyy%%";
# Months of the year. Indexed from 1 (January) to 12 (December).
set lang_monthname_long = [ "", "студзеня", "лютага", "сакавіка", "красавіка", "траўня", "чэрвеня", "ліпеня", "жніўня", "верасьня", "кастрычніка", "лістапада", "сьнежня" ];
# Months of the year, in their short forms. Indexed from 1 (Jan) to 12 (Dec).
set lang_monthname_short = [ "", "Стд", "Лют", "Сак", "Крс", "Тра", "Чэр", "Ліп", "Жнв", "Врс", "Кст", "Ліс", "Снж" ];
# Days of the week. Indexed from 1 (Sunday) to 7 (Saturday).
set lang_dayname_long = [ "", "нядзеля", "панядзелак", "аўторак", "серада", "чацьвер", "пятніца", "субота" ];
# Days of the week, in their short forms. Indexed from 1 (Sun) to 7 (Sat).
set lang_dayname_short = ["", "Няд", "Пан", "Аўт", "Сер", "Чац", "Пят", "Суб"];
set text_meta_music = "Цяпер грае";
set text_meta_mood = "Настрой";
set text_post_comment = "Пракамэнтаваць";
set text_max_comments = "Дасягнутая максымальная колькасьць камэнтароў";
set text_read_comments="# камэнтар // # камэнтара // # камэнтароў";
set text_post_comment_friends = "Пракамэнтаваць";
set text_read_comments_friends = "# камэнтар // # камэнтара // # камэнтароў";
set text_tag_uses = "# раз // # раза // # разоў";
set text_skiplinks_back = "Папярэднія #";
set text_skiplinks_forward = "Наступныя #";
set text_skiplinks_forward_words = "Наперад";
set text_view_recent = "Апошнія запісы";
set text_view_friends = "Сябры";
set text_view_friends_comm = "Удзельнікі";
set text_view_friends_filter = "Сябры (фільтар)";
set text_view_friendsfriends = "Сябры сяброў";
set text_view_friendsfriends_filter = "Сябры сяброў (фільтар)";
set text_view_archive = "Архіў";
set text_view_userinfo = "Профіль карыстальніка";
set text_view_month = "Праглядзець назвы";
set text_nosubject = "(бяз тэмы)";
set text_noentries_recent = "Запісаў няма";
set text_noentries_day = "Няма запісаў за гэты дзень";
set text_permalink = "Спасылка";
set text_month_screened_comments = "Ёсьць схаваныя камэнтары";
# Коментарі
set text_multiform_check = "Выбраць:";
set text_multiform_des = "Апэрацыя над усімі выбранымі камэнтарамі:";
set text_multiform_btn = "Выканаць";
set text_multiform_opt_unscreen = "Паказаць";
set text_multiform_opt_screen = "Схаваць";
set text_multiform_opt_unfreeze = "Размарозіць";
set text_multiform_opt_freeze = "Замарозіць";
set text_multiform_opt_delete = "Выдаліць";
set text_multiform_opt_deletespam = "Выдаліць як спам";
set text_multiform_conf_delete = "Выдаліць выбраныя камэнтары?";
set text_comment_posted = "Камэнтар пасьпяхова дасланы.";
set text_comment_from = "Ад:";
set text_comment_date = "Дата:";
set text_comment_ipaddr = "IP-адрас:";
set text_comment_reply = "Адказаць";
set text_comment_frozen = "Галіна замарожаная";
set text_comment_parent = "Бацькоўскі";
set text_comment_thread = "Галіна";
set text_tags = "Цэтлікі: #";
set reg_firstdayofweek = "monday";
set text_day_prev = "Папярэдні дзень";
set text_day_next = "Наступны дзень";
set text_poster_anonymous = "(Ананімна)";
set text_reply_back = "Чытаць камэнтары";
set text_reply_nocomments_header = "Камэнтары адключаныя:";
set text_reply_nocomments = "Камэнтары да гэтага запісу адключаныя.";
set text_website_default_name = "Мой сайт";
set text_icon_alt_protected = "[абаронены запіс]";
set text_icon_alt_private = "[прыватны запіс]";
### Additional text
set text_meta_location = "Месцазнаходжаньне";
set text_tags_page_header = "Бачныя цэтлікі";
set text_tag_uses = "# раз // # раза // # разоў";
set text_view_memories = "Запамінальныя запісы";
set text_entry_prev = "Папярэдні запіс";
set text_entry_next = "Наступны запіс";
set text_edit_entry = "Рэдагаваць запіс";
set text_edit_tags = "Зьмяніць цэтлікі";
set text_tell_friend = "Паведаміць сябру";
set text_mem_add = "Дадаць да запамінаў";
set text_watch_comments = "Сачыць за камэнтарамі";
set text_unwatch_comments = "Не сачыць за камэнтарамі";
set text_month_form_btn = "Глядзець";
set text_replyform_header = "Зьмяшчэньне камэнтара";
set text_multiform_opt_track = "Сачыць за камэнтарамі";
set text_multiform_opt_untrack = "Не сачыць за камэнтарамі";
### Functions
function lang_page_of_pages (int pg, int pgs) [notags] : string {
return "Старонка $pg з $pgs";
}
# Three forms, special cases for numbers ending in 1 and 2, 3, 4, except those ending in 1[1-4]
function lang_map_plural (int n) : int {
if ($n%10 == 1 and $n%100 != 11) { return 0; }
if ($n%10 >= 2 and $n%10 <= 4 and ($n%100 < 10 or $n%100>=20)) { return 1; }
return 2;
}
function lang_ordinal(int num) [notags] : string
"Make an ordinal number from a cardinal number"
{
return $num+"";
# if ($num%10 == 3 and $num%100 != 13) { return $num+"-є"; }
# return $num+"-е";
}
function lang_viewname(string viewid) [notags] : string {
if ($viewid == "recent") { return $*text_view_recent; }
if ($viewid == "archive") { return $*text_view_archive; }
if ($viewid == "friends") { return $*text_view_friends; }
if ($viewid == "day") { return "Дзень"; }
if ($viewid == "month") { return "Месяц"; }
if ($viewid == "userinfo") { return $*text_view_userinfo; }
if ($viewid == "entry") { return "Чытаць камэнтары"; }
if ($viewid == "reply") { return "Пракамэнтаваць"; }
if ($viewid == "tags") { return "Цэтлікі"; }
return "Невядомы тып старонкі";
}
function ReplyPage::view_title() : string {
return "Пракамэнтаваць";
}
function server_sig() {
"""Распрацавана <a href="$*SITEROOT/">$*SITENAME</a>""";
}
function Page::print_entry_poster(Entry e) {
$e.poster->print();
if ($.view == "friends" and not $e.poster->equals($e.journal)) {
" напісаў у ";
$e.journal->print();
}
}
function lang_user_wrote(UserLite u) : string {
if (defined $u) {
return $u->as_string()+" напісаў:";
}
else {
return "Ананім напісаў";
}
}
function lang_at_datetime(DateTime d) : string {
return " " + $d->date_format("long") + " у " + $d->time_format();
}

View File

@ -0,0 +1,161 @@
#-*-s2-*- ;; -*- coding: utf-8 -*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/da1";
layerinfo "name" = "Danish";
layerinfo "langcode" = "da";
layerinfo "author_name" = "LiveJournal Danish Translation Team";
layerinfo "author_email" = "lj_dansk@livejournal.com";
layerinfo "source_viewable" = 1;
set lang_current = "da";
#[[ date and time l12n ]]
# Kort datoformat
set lang_fmt_date_short = "%%d%%/%%m%%/%%yy%%";
# Mellem dato
set lang_fmt_date_med = "%%dayord%% %%mon%%., %%yyyy%%";
# Mellem dato med forkortet ugedag
set lang_fmt_date_med_day = "%%da%%, d. %%dayord%% %%mon%%., %%yyyy%%";
# Lang dato
set lang_fmt_date_long = "%%dayord%% %%month%%, %%yyyy%%";
# Lang dato med ugedag
set lang_fmt_date_long_day = "%%day%%, d. %%dayord%% %%month%%, %%yyyy%%";
# Tidsformat
set lang_fmt_time_short = "%%HH%%:%%min%%";
# Kort månedsformat (samme som engelsk)
set lang_fmt_month_short = "%%m%%/%%yy%%";
# Mellem måned (samme som engelsk)
set lang_fmt_month_med = "%%mon%% %%yyyy%%";
# Lang måned (samme som engelsk)
set lang_fmt_month_long = "%%month%% %%yyyy%%";
# Årets måneder, lang
set lang_monthname_long = [ "", "Januar", "Februar", "Marts",
"April", "Maj", "Juni",
"Juli", "August", "September",
"Oktober", "November", "December" ];
# Årets måneder, kort
set lang_monthname_short = [ "", "Jan", "Feb", "Mar",
"Apr", "Maj", "Jun",
"Jul", "Aug", "Sep",
"Okt", "Nov", "Dec" ];
# Ugens dage, lang
set lang_dayname_long = [ "", "Søndag", "Mandag", "Tirsdag", "Onsdag",
"Torsdag", "Fredag", "Lørdag" ];
# Ugens dage, kort
set lang_dayname_short = [ "", "Søn", "Man", "Tirs", "Ons",
"Tors", "Fre", "Lør" ];
set reg_firstdayofweek = "monday";
#[[ texttranslation ]]
# Currents
set text_meta_music = "Nuværende musik";
set text_meta_mood = "Nuværende humør";
# Comments
set text_post_comment = "Skriv kommentar";
set text_read_comments = "1 kommentar // # kommentarer";
set text_post_comment_friends = "Skriv kommentar";
set text_read_comments_friends = "1 kommentar // # kommentarer";
# Skiplinks
set text_skiplinks_back="Forrige #";
set text_skiplinks_forward="Næste #";
# Views
set text_view_recent = "Seneste poster";
set text_view_friends = "Venner";
set text_view_archive = "Arkiv";
set text_view_userinfo = "Brugerinfo";
set text_view_month = "Vis emner"; # "Vis overskrifter"?
# Misc. texts
set text_nosubject = "(intet emne)";
set text_noentries_recent = "Der er ingen poster at vise.";
set text_noentries_day = "Der blev ikke skrevet nogle poster denne dag.";
set text_permalink = "Link";
set text_month_screened_comments = "m. skærmede";
set text_multiform_check = "Vælg:";
set text_multiform_des = "Massehandling på valgte kommentarer:";
set text_multiform_btn = "Udfør handling";
set text_multiform_opt_unscreen = "Afskærm";
set text_multiform_opt_screen = "Skærm";
set text_multiform_opt_delete = "Slet";
set text_multiform_conf_delete = "Slet valgte kommentarer?";
set text_day_prev = "Forrige dag";
set text_day_next = "Næste dag";
set text_comment_from = "Fra:";
set text_comment_date = "Dato:";
set text_comment_ipaddr = "IP adresse:";
set text_comment_reply = "Svar";
set text_comment_parent = "Forælder";
set text_comment_thread = "Tråd";
set text_reply_back = "Læs kommentarer";
set text_reply_nocomments_header = "Kommentarer slået fra:";
set text_reply_nocomments = "Kommentarer til denne post er blevet slået fra.";
set text_poster_anonymous = "(Anonym)";
set text_website_default_name = "Mit websted";
#[[ function translations ]]
# Samme som engelsk:
#function lang_map_plural (int n) : int {
# if ($n == 1) { return 0; } # singular
# return 1; # plural
#}
function lang_page_of_pages (int pg, int pgs) [notags] : string {
return "Side $pg af $pgs";
}
function lang_ordinal(int num) [notags] : string {
return $num+".";
}
function lang_user_wrote(UserLite u) : string
"Returns text describing that the user wrote something. i18nc layers should override this."
{
if (defined $u) {
return $u->as_string()+" skrev";
}
else {
return "En anonym bruger skrev";
}
}
function lang_at_datetime(DateTime d) : string
"Returns a string saying \"at {the data and time given}\". Used in the core implementation of EntryPage and ReplyPage. i18nc layers should override this."
{
# return "d. 1. Januar, 2004, kl. 23:01";
return "d. " + $d->date_format("long") + ", kl. " + $d->time_format();
}
### Ovenstående skal testes i brug ASAP. ###
function lang_viewname(string viewid) [notags] : string
"Get some words representing a view"
{
if ($viewid == "recent") { return $*text_view_recent; }
if ($viewid == "archive") { return $*text_view_archive; }
if ($viewid == "friends") { return $*text_view_friends; }
if ($viewid == "day") { return "Dag"; }
if ($viewid == "month") { return "Måned"; }
if ($viewid == "userinfo") { return $*text_view_userinfo; }
if ($viewid == "entry") { return "Læs kommentarer"; }
if ($viewid == "reply") { return "Skriv kommentar"; }
return "Ukendt visningstype";
}
function server_sig() {
"""Kørt af <a href="$*SITEROOT/">$*SITENAME</a>""";
}
function ReplyPage::view_title() : string {
return "Skriv kommentar";
}
function Page::print_entry_poster(Entry e) {
$e.poster->print();
if ($.view == "friends" and not $e.poster->equals($e.journal)) {
" skrev i ";
$e.journal->print();
}
}

View File

@ -0,0 +1,147 @@
#-*-s2-*- ;; -*- coding: utf-8 -*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/de1";
layerinfo "name" = "German";
layerinfo "langcode" = "de";
layerinfo "author_name" = "Timwi";
layerinfo "author_email" = "timwi@livejournal.com";
layerinfo "source_viewable" = 1;
set lang_current = "de";
# Kurzes Datumsformat
set lang_fmt_date_short = "%%d%%.%%m%%.%%yy%%";
# Mittellanges Datumsformat
set lang_fmt_date_med = "%%dayord%% %%mon%% %%yyyy%%";
# Mittellanges Datumsformat mit Wochentag
set lang_fmt_date_med_day = "%%da%%, %%dayord%% %%mon%% %%yyyy%%";
# Langes Datumsformat
set lang_fmt_date_long = "%%dayord%% %%month%% %%yyyy%%";
# Langes Datumsformat mit Wochentag
set lang_fmt_date_long_day = "%%day%%, %%dayord%% %%month%% %%yyyy%%";
# Zeitformat
set lang_fmt_time_short = "%%HH%%:%%min%%";
# Kurzes Monatsformat
#set lang_fmt_month_short = "%%mon%% %%yy%%";
# Mittleres Monatsformat
#set lang_fmt_month_med = "%%mon%% %%yyyy%%";
# Langes Monatsformat
#set lang_fmt_month_long = "%%month%% %%yyyy%%";
# Monatsnamen
set lang_monthname_long = [ "", "Januar", "Februar", "März",
"April", "Mai", "Juni",
"Juli", "August", "September",
"Oktober", "November", "Dezember" ];
# Monatsabkürzungen
set lang_monthname_short = [ "", "Jan", "Feb", "Mär",
"Apr", "Mai", "Jun",
"Jul", "Aug", "Sep",
"Okt", "Nov", "Dez" ];
# Wochentagnamen
set lang_dayname_long = [ "", "Sonntag", "Montag", "Dienstag", "Mittwoch",
"Donnerstag", "Freitag", "Samstag" ];
# Wochentagabkürzungen
set lang_dayname_short = [ "", "So", "Mo", "Di", "Mi",
"Do", "Fr", "Sa" ];
set reg_firstdayofweek = "monday";
#[[ texttranslation ]]
# Currents
set text_meta_music = "Aktuelle Musik";
set text_meta_mood = "Aktuelle Stimmung";
# Comments
set text_post_comment = "Kommentar hinterlassen";
set text_read_comments = "1 Kommentar // # Kommentare";
set text_post_comment_friends = "Kommentar hinterlassen";
set text_read_comments_friends = "1 Kommentar // # Kommentare";
# Skiplinks
set text_skiplinks_back="Vorherige #";
set text_skiplinks_forward="Nächste #";
# Views
set text_view_recent = "Neueste Einträge";
set text_view_friends = "Freunde";
set text_view_archive = "Archiv";
set text_view_userinfo = "Benutzerprofil";
set text_view_month = "Monatsansicht"; # "Vis overskrifter"?
# Misc. texts
set text_nosubject = "(kein Betreff)";
set text_noentries_recent = "Keine Einträge.";
set text_noentries_day = "An diesem Tag wurden keine Einträge gemacht.";
set text_permalink = "Link";
set text_month_screened_comments = "zzgl. verdeckte";
set text_multiform_check = "Auswählen:";
set text_multiform_des = "Alle ausgewählten Kommentare:";
set text_multiform_btn = "Ausführen";
set text_multiform_opt_unscreen = "Aufdecken";
set text_multiform_opt_screen = "Verdecken";
set text_multiform_opt_delete = "Löschen";
set text_multiform_conf_delete = "Bist du dir sicher, dass du die ausgewählten Kommentare löschen möchtest?";
set text_day_prev = "Vorheriger Tag";
set text_day_next = "Nächster Tag";
set text_comment_from = "Von:";
set text_comment_date = "Datum:";
set text_comment_ipaddr = "IP-Adresse:";
set text_comment_reply = "Darauf antworten";
set text_comment_parent = "Kommentar davor";
set text_comment_thread = "Nachfolgende Kommentare";
set text_reply_back = "Kommentare lesen";
set text_reply_nocomments_header = "Kommentarfunktion deaktiviert:";
set text_reply_nocomments = "Für diesen Eintrag wurde die Kommentarfunktion deaktiviert.";
set text_website_default_name = "Meine Webseite";
set text_poster_anonymous = "(Anonym)";
#[[ function translations ]]
function lang_page_of_pages (int pg, int pgs) [notags] : string {
return "Seite $pg von $pgs";
}
function lang_ordinal(int num) [notags] : string {
return $num+".";
}
function lang_viewname(string viewid) [notags] : string
"Get some words representing a view"
{
if ($viewid == "recent") { return $*text_view_recent; }
if ($viewid == "archive") { return $*text_view_archive; }
if ($viewid == "friends") { return $*text_view_friends; }
if ($viewid == "day") { return "Tag"; }
if ($viewid == "month") { return "Monat"; }
if ($viewid == "userinfo") { return $*text_view_userinfo; }
if ($viewid == "entry") { return "Kommentare lesen"; }
if ($viewid == "reply") { return "Kommentar hinterlassen"; }
return "Unbekannte Ansicht";
}
function ReplyPage::view_title() : string {
return "Kommentar hinterlassen";
}
function server_sig() {
"""Gehostet von <a href="$*SITEROOT/">$*SITENAME</a>""";
}
function Page::print_entry_poster(Entry e) {
$e.poster->print();
if ($.view == "friends" and not $e.poster->equals($e.journal)) {
" schrieb in ";
$e.journal->print();
}
}
function lang_user_wrote(UserLite u) : string "Returns text describing that the user wrote something. i18nc layers should override this." {
if (defined $u) {
return $u->as_string()+" schrieb";
}
else {
return "Ein anonymer Benutzer schrieb";
}
}
function lang_at_datetime(DateTime d) : string "Returns a string saying \"at {the date and time given}\". Used in the core implementation of EntryPage and ReplyPage. i18nc layers should override this." {
return "am " + $d->date_format("long") + " um " + $d->time_format();
}

View File

@ -0,0 +1,9 @@
#-*-s2-*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/en1";
layerinfo "name" = "English";
layerinfo "langcode" = "en";
# Note: this file doesn't actually override anything, since the core
# is in English

View File

@ -0,0 +1,21 @@
# -*-s2-*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/en_GB1";
layerinfo "name" = "English (UK)";
layerinfo "langcode" = "en_GB";
layerinfo "author_name" = "Smigs";
layerinfo "author_email" = "smigs@livejournal.com";
set lang_current = "en_GB";
# date formats
set lang_fmt_date_short = "%%d%%/%%m%%/%%yy%%";
set lang_fmt_date_med = "%%dayord%% %%mon%%, %%yyyy%%";
set lang_fmt_date_med_day = "%%da%%, %%dayord%% %%mon%%. %%yyyy%%";
set lang_fmt_date_long = "%%dayord%% %%month%% %%yyyy%%";
set lang_fmt_date_long_day = "%%day%%, the %%dayord%% %%month%% %%yyyy%%";
set lang_fmt_time_short = "%%HH%%:%%min%%";
set lang_fmt_month_short = "%%mm%%/%%yy%%";
set lang_fmt_month_med = "%%mon%% %%yyyy%%";
set lang_fmt_month_long = "%%month%% %%yyyy%%";
propgroup colors = "Colours";

View File

@ -0,0 +1,139 @@
#-*-s2-*- ;; -*- coding: utf-8 -*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/eo1";
layerinfo "name" = "Esperanto";
layerinfo "langcode" = "eo";
layerinfo "author_name" = "Timwi, Amuzulo";
layerinfo "author_email" = "timwi@livejournal.com, amuzulo@livejournal.com";
layerinfo "source_viewable" = 1;
set lang_current = "eo";
set lang_fmt_date_short = "%%yyyy%%-%%mm%%-%%dd%%";
set lang_fmt_date_med = "%%dayord%% de %%mon%% %%yyyy%%";
set lang_fmt_date_med_day = "%%da%%, la %%dayord%% de %%mon%% %%yyyy%%";
set lang_fmt_date_long = "la %%dayord%% de %%month%% %%yyyy%%";
set lang_fmt_date_long_day = "%%day%%, la %%dayord%% de %%month%% %%yyyy%%";
set lang_fmt_time_short = "%%HH%%:%%min%%";
set lang_fmt_month_short = "%%mon%% %%yy%%";
set lang_fmt_month_med = "%%mon%% %%yyyy%%";
set lang_fmt_month_long = "%%month%% %%yyyy%%";
set lang_monthname_long = [ "", "januaro", "februaro", "marto",
"aprilo", "majo", "junio",
"julio", "aŭgusto", "septembro",
"oktobro", "novembro", "decembro" ];
set lang_monthname_short = [ "", "jan", "feb", "mar",
"apr", "maj", "jun",
"jul", "aŭg", "sep",
"okt", "nov", "dec" ];
set lang_dayname_long = [ "", "dimanĉo", "lundo", "mardo", "merkredo",
"ĵaŭdo", "vendredo", "sabato" ];
set lang_dayname_short = [ "", "di", "lu", "ma", "me",
"ĵa", "ve", "sa" ];
set reg_firstdayofweek = "monday";
#[[ texttranslation ]]
# Currents
set text_meta_music = "Nuna muziko";
set text_meta_mood = "Nuna humoro";
# Comments
set text_post_comment = "Afiŝu novan komenton";
set text_read_comments = "1 komento // # komentoj";
set text_post_comment_friends = "Afiŝu novan komenton";
set text_read_comments_friends = "1 komento // # komentoj";
# Skiplinks
set text_skiplinks_back="# antaŭaj komentoj";
set text_skiplinks_forward="# sekvontaj komentoj";
# Views
set text_view_recent = "Lastatempaj enskriboj";
set text_view_friends = "Geamikoj";
set text_view_archive = "Arĥivo";
set text_view_userinfo = "Uzantinformoj";
set text_view_month = "Monataj temoj";
# Misc. texts
set text_nosubject = "(neniu temo)";
set text_noentries_recent = "Neniuj enskriboj.";
set text_noentries_day = "Ekzistas neniuj enskriboj en tiu tago.";
set text_permalink = "Ligilo";
set text_month_screened_comments = "+ kaŝitoj";
set text_multiform_check = "Elektu:";
set text_multiform_des = "Amasagado por elektitaj komentoj:";
set text_multiform_btn = "Agu";
set text_multiform_opt_unscreen = "Malkaŝu";
set text_multiform_opt_screen = "Kaŝu";
set text_multiform_opt_delete = "Forigu";
set text_multiform_conf_delete = "Ĉu vi certas ke vi viŝas forigi la elektitajn komentojn?";
set text_day_prev = "Antaŭa tago";
set text_day_next = "Sekvonta tago";
set text_comment_from = "De:";
set text_comment_date = "Dato:";
set text_comment_ipaddr = "IP-adreso:";
set text_comment_reply = "Respondu al ĉi tiu";
set text_comment_parent = "Patro";
set text_comment_thread = "Fadeno";
set text_reply_back = "Legu komentojn";
set text_reply_nocomments_header = "Komentoj malebligitaj:";
set text_reply_nocomments = "La uzanto malebligis komentojn por ĉi tiu enskribo.";
set text_website_default_name = "Mia TTT-ejo";
set text_poster_anonymous = "(sennoma)";
#[[ function translations ]]
function lang_page_of_pages (int pg, int pgs) [notags] : string {
return "Paĝo $pg da $pgs";
}
function lang_ordinal(int num) [notags] : string {
return $num + "-a";
}
function lang_viewname(string viewid) [notags] : string
"Get some words representing a view"
{
if ($viewid == "recent") { return $*text_view_recent; }
if ($viewid == "archive") { return $*text_view_archive; }
if ($viewid == "friends") { return $*text_view_friends; }
if ($viewid == "day") { return "Tago"; }
if ($viewid == "month") { return "Monato"; }
if ($viewid == "userinfo") { return $*text_view_userinfo; }
if ($viewid == "entry") { return "Legu komentojn"; }
if ($viewid == "reply") { return "Afiŝu komenton"; }
return "Nekonata vido";
}
function ReplyPage::view_title() : string {
return "Afiŝu komenton";
}
function server_sig() {
"""Funkciigita de <a href="$*SITEROOT/">$*SITENAME</a>""";
}
function Page::print_entry_poster(Entry e) {
$e.poster->print();
if ($.view == "friends" and not $e.poster->equals($e.journal)) {
" skribis en ";
$e.journal->print();
}
}
function lang_user_wrote(UserLite u) : string "Returns text describing that the user wrote something. i18nc layers should override this." {
if (defined $u) {
return $u->as_string()+" skribis";
}
else {
return "Sennoma uzanto skribis";
}
}
function lang_at_datetime(DateTime d) : string "Returns a string saying \"at {the date and time given}\". Used in the core implementation of EntryPage and ReplyPage. i18nc layers should override this." {
return "je " + $d->date_format("long") + " je " + $d->time_format();
}

View File

@ -0,0 +1,171 @@
#-*-s2-*- ;; -*- coding: utf-8 -*-
layerinfo "type" = "i18nc";
layerinfo "redist_uniq" = "i18nc/fi1";
layerinfo "name" = "Finnish";
layerinfo "langcode" = "fi";
layerinfo "author_name" = "shiningkianna, zell_d";
layerinfo "source_viewable" = 1;
set lang_current = "fi";
# Ajat ja päiväykset
# Viikonpäivät
set lang_dayname_long = [ "", "sunnuntai", "maanantai", "tiistai", "keskiviikko", "torstai", "perjantai", "lauantai" ];
set lang_dayname_short = [ "", "su", "ma", "ti", "ke", "to", "pe", "la" ];
# Pitkä päiväys
set lang_fmt_date_long = "%%dayord%% %%month%%ta %%yyyy%%";
# Pitkä päiväys viikonpäivällä
set lang_fmt_date_long_day = "%%day%%, %%dayord%% %%month%%ta %%yyyy%%";
# Keskipitkä päiväys
set lang_fmt_date_med = "%%d%%. %%mon%%. %%yyyy%%";
# Keskipitkä päiväys viikonpäivällä
set lang_fmt_date_med_day = "%%da%%, %%dayord%% %%mon%%. %%yyyy%%";
# Lyhyt päiväys
set lang_fmt_date_short = "%%d%%.%%m%%.%%yyyy%%";
# Pitkä kuukausipäiväys
set lang_fmt_month_long = "%%month%% %%yyyy%%";
# Keskipitkä kuukausipäiväys
set lang_fmt_month_med = "%%mon%% %%yyyy%%";
# Lyhyt kuukausipäiväys
set lang_fmt_month_short = "%%m%%/%%yyyy%%";
# Aika
set lang_fmt_time_short ="%%HH%%:%%min%%";
# Kuukaudet
set lang_monthname_long = [ "", "tammikuu", "helmikuu", "maaliskuu", "huhtikuu", "toukokuu", "kesäkuu", "heinäkuu", "elokuu", "syyskuu", "lokakuu", "marraskuu", "joulukuu" ];
set lang_monthname_short = [ "", "tammik", "helmik", "maalisk", "huhtik", "toukok", "kesäk", "heinäk", "elok", "syysk", "lokak", "marrask", "jouluk" ];
# Viikko alkaa maanantaista
set reg_firstdayofweek = "monday";
# Tekstit
# Tämänhetkinen musiikki ja mieliala
set text_meta_mood = "Mieliala";
set text_meta_music = "Musiikki";
# Kommentit
set text_post_comment = "Jätä vastaus tähän";
set text_post_comment_friends = "Jätä vastaus tähän";
set text_read_comments = "1 kommentti // # kommenttia";
set text_read_comments_friends = "1 kommentti // # kommenttia";
# Linkit, joilla hypätään viestien yli
set text_skiplinks_back = "Edelliset #";
set text_skiplinks_forward = "Seuraavat #";
# Näkymät
set text_view_archive = "Arkisto";
set text_view_friends = "Kaverit";
set text_view_friends_comm = "Jäsenet";
set text_view_friends_filter = "Kaverit (mukautettu suodatin)";
set text_view_friendsfriends = "Kavereiden kaverit";
set text_view_friendsfriends_filter = "Kavereiden kaverit (mukautettu suodatin)";
set text_view_month = "Otsikot";
set text_view_recent = "Merkinnät";
set text_view_userinfo = "Käyttäjätiedot";
# Sekalaisia tekstejä
set text_comment_date = "Päiväys:";
set text_comment_from = "Lähettäjä:";
set text_comment_frozen = "Jäädytetty";
set text_comment_ipaddr = "IP-osoite:";
set text_comment_parent = "Ylempi";
set text_comment_reply = "Vastaa";
set text_comment_thread = "Viestiketju";
set text_day_next = "Seuraava päivä";
set text_day_prev = "Edellinen päivä";
set text_max_comments = "Maksimimäärä kommentteja saatu";
set text_month_screened_comments = "ja peitettyjä kommentteja";
set text_multiform_btn = "Muokkaa";
set text_multiform_check = "Valitse:";
set text_multiform_conf_delete = "Poista valitut kommentit?";
set text_multiform_des = "Muokkaa kaikkia valittuja kommentteja:";
set text_multiform_opt_delete = "Poista";
set text_multiform_opt_freeze = "Jäädytä";
set text_multiform_opt_screen = "Peitä";
set text_multiform_opt_unfreeze = "Poista jäädytys";
set text_multiform_opt_unscreen = "Poista peitto";
set text_noentries_day = "Kyseisenä päivänä ei tehty merkintöjä";
set text_noentries_recent = "Ei merkintöjä";
set text_nosubject = "(ei otsikkoa)";
set text_permalink = "Linkki";
set text_poster_anonymous = "(tuntematon)";
set text_reply_back = "Lue kommentteja";
set text_reply_nocomments = "Kommentointi on estetty tämän merkinnän kohdalla";
set text_reply_nocomments_header = "Kommentointi estetty:";
set text_website_default_name = "Kotisivut";
# Funktiot
# Antaa eri tuloksen riippuen siitä tuleeko tekstiä käsitellä yksikkönä vai monikkona
function lang_map_plural (int n) : int {
if ($n == 1) { return 0; } # singular
return 1; # plural
}
# Palauttaa tekstin "Sivu X/Y", esim. Sivu 3/5
function lang_page_of_pages (int pg, int pgs) [notags] : string {
return "Sivu $pg/$pgs";
}
# Tekee numerosta järjestysnumeron, eli laittaa numeron perään pisteen
function lang_ordinal(int num) : string {
return $num+".";
}
# Palauttaa tekstin joka kertoo millainen näkymä on kyseessä
function lang_viewname(string viewid) [notags] : string "Get some words representing a view" {
if ($viewid == "recent") { return $*text_view_recent; }
if ($viewid == "archive") { return $*text_view_archive; }
if ($viewid == "friends") { return $*text_view_friends; }
if ($viewid == "day") { return "Päivä"; }
if ($viewid == "month") { return "Kuukausi"; }
if ($viewid == "userinfo") { return $*text_view_userinfo; }
if ($viewid == "entry") { return "Lue kommentteja"; }
if ($viewid == "reply") { return "Jätä kommentti"; }
return "Tuntematon näkymä";
}
# Vastaussivun otsikko
function ReplyPage::view_title() : string {
return "Jätä kommentti";
}
# Kirjoittaa palvelimen allekirjoituksen,
# Esim. "Sivun tarjoaa LiveJournal.com"
function server_sig() {
"""Sivun tarjoaa <a href="$*SITEROOT/">$*SITENAME</a>""";
}
# Kirjoittaa kaverisivulla tekstin, joka kertoo missä yhteisössä joku kirjoitti jotakin,
# Esim. "Esimerkkilähettäjä kirjoitti yhteisössä esimerkkiyhteisö"
function Page::print_entry_poster(Entry e) {
$e.poster->print();
if ($.view == "friends" and not $e.poster->equals($e.journal)) {
" kirjoitti yhteisössä ";
$e.journal->print();
}
}
# Palauttaa tekstin joka kertoo että joku kirjoitti,
# Esim. "Esimerkkikäyttäjä kirjoitti" tai "Tuntematon käyttäjä kirjoitti"
function lang_user_wrote(UserLite u) : string "Returns text describing that the user wrote something. i18nc layers should override this." {
if (defined $u) {
return $u->as_string()+" kirjoitti";
}
else {
return "Tuntematon käyttäjä kirjoitti";
}
}
# Palauttaa tekstin joka ilmoittaa päiväyksen ja kellonajan,
# Esim. "3. lokakuuta 2004 kello 14:45"
function lang_at_datetime(DateTime d) : string "Returns a string saying \"at {the date and time given}\". Used in the core implementation of EntryPage and ReplyPage. i18nc layers should override this." {
return $d->date_format("long") + " kello " + $d->time_format();
}

Some files were not shown because too many files have changed in this diff Show More