Sigh….. I waited for someone to use that heading so I wouldn’t have to reveal myself as a closet Star Trek fan. Stories have been bandied about by all about a new report by scientists at the National Research Council which basically says that life elsewhere may have little in common with ours. My first thought was that they are right, aliens are probably very different from us, for example they are probably intelligent.

Then I put aside my cynicism and decided to wait and see how long it would be before someone used the lyrics from “Star Trekkin“. Sad to say, it turned out that it only occurred to me (or nobody else would admit to it).

All I can say in regards to the recent report, is that I’ll be terribly disappointed if, after watching years of sci-fi, it turns out that the only life out there is intelligent slime.

Shockingly, a Motorola phone has killed a 22 yo man in China after it exploded (or at least the battery did) in his pocket and pushed broken ribs into his heart. The man was welding at the time and the temperature is the most likely cause of the explosion. China is apparently rife with counterfeit cheap batteries and this may have played a part in the incident. Read more here.

I’ve been busy writing the new version of the Statistical hit counter, and in doing so, I write helper scripts to show me what new browser strings, screen resolutions etc are out there now that perhaps should be counted. In doing so, I’ve noticed an odd new trend and I believe I have an explanation for it. Of the past 6781 unique hits we’ve had, 2470 have apparently had JavaScript turned off, the beta counter that has been runing behind htmlfixit has been tracking JS and non JS (including bots) for about 2 years now.

Think about that, of 6781 hits, 2470 of them didn’t have JavaScript (not counting known search engine spiders either). I didn’t believe the number could be that high, but I’ve manually reviewed the stats data myself and it’s legit. If my maths is correct, that’s over 35% of all traffic having no JavaScript. Everyone else I’ve read about has claimed figures of more than 80% having JS turned on, so why are my figures so off?

After some thought, I realized or at least theorized as to what the cause is and the answer is something I imagine other CMS or forum systems (systems in which strangers can post or comment) could testify to. Automated Comment spamming programs, I wrote a quick program to print out the details of those browsers that did not support JavaScript, (or had it turned off) and group them by popularity. The program is here, and the list is many and varied, which is to be expected because spammer submit bots can pretend to be anything at all, so copying popular browser strings would be common sense to them. I don’t believe all of these are faked, this is a techie oriented site and many of use are security conscious enough to turn off JavaScript, I just don’t believe anywhere near 35% of us are.

We have some systems in place here in our CMS, a blacklist based on IP or domain and a system that works in much the same way that SpamAssassin does (except HTTP not email) by grading comments on known criteria and blocking comments below a certain threshold. These systems stop allot of bad hits (in fact we’ve had not at all get though in ages, wish they’d stop trying), but most of the site is not in the CMS and therefore gets bad hits. (like the forum which had 80 odd spiders in it at once last week.) I’m now looking at the possibility of using mod_security (which we already have) in conjunction with a blacklist of known spammer IP’s to block them before they ever get a full connection to apache. My concern with this is the risk of blocking normal nice non-scummy people as well, so it has to be done carefully.

Interestingly enough, a bit of Googling shows me that I am not alone in this thinking, and even better I’ve found a mod_security rules filter that looks like it will really make a difference just in the pattern checks alone. I am going to try this now and I’ll let you know how it goes. (Assuming we haven’t accidentally blocked you from accessing our server.) If all goes well, and I’ll know how successful the measure is by how many comment blocks are listed in our spam log. I’ll let you know how it goes and what exactly I did.

Update: We are now running the latest mod_security, with the latest gotroot rules and comment spam blacklists. In the past few days we’ve seen a 50% drop in comment spamming attempts. I’d imagine that blocking these hordes of comment spamming bots will show a significant drop in bandwidth usage also. I was required to make a couple of exclusion rules so that all our various programs work as planned without being blocked by mod_security, but other than that it has been a straight forward and easy (and worthwhile) experience. The more people that use this fantastic tool, the more the blacklists will be effective and up to date. If you have control over your own server, you could do allot worse than to install mod security.

Analyzing the traffic details reported by my Advanced Stats counter, I’ve noticed some interesting trends regarding the various search engines. HTMLfixIT is now getting roughly 98.5% of our search engine traffic from Google. Now we have worked hard to create a site that would do well with search engines, and we have loads of links from other peoples sites to our own, so it isn’t surprising that we do well with Google. What is surprising, is that we do far better from Google than Yahoo, Microsoft and the others despite the fact that their search engine algorithms are growing closer and that now days we usually rank the same or similar for many of our search terms in all engines. Lets take this brief example search string:
‘column count doesn’
Which is the start of a MySQL error that we explain in one of our tutes. The full string is:
Invalid Query: Column count doesn’t match value count at row 1 MySQL
but we don’t need the full string as the partial works rather well to demonstrate my point. and the addition of the ‘ in doesn’t seems to confuse things with some of them. (though it doesn’t change the conclusions I’m drawing here as I’ve tried it with many tech search terms that bring us traffic from Google.)

That string ranks the following in the top 5 search engines.
Google Number 3
Yahoo Number 1
Microsoft Number 1
Altavista Number 1
AOL Number 10

Now for the above term, Google has sent us 213 unique visitors in the past 5 days, Yahoo has sent us 2 visitors for that term and the others have sent us none at all. It sounds like an odd string to use for this example, but I’ve tried it in what is possibly the most searched error message on the Internet as well. “premature end of script headers” and it the same trend is found.

Now I have a friend who has a great many domain names and the vast majority of them are not tech based. They are mostly spiritual in nature, (meditation, psychic readings etc). This friends sites get far far more traffic from the other engines then does htmlfixit, and in fact their ratio of traffic from Google to the other engines is much better also.

So what can we conclude from this?
Well if htmlfixit can rank higher on MSN and Yahoo (for example) than on Google for a popular tech search term, and still get a hundred times more traffic for that search term from Google. Then the logical conclusion is that Google is the search engine preferred by tech oriented people.
That makes sense really, for example; Firefox and Safari both use Google as their homepage, and it is the tech aware users who are installing these browsers (on Windows anyway) on their machines. The people using Microsoft search are likely doing so because it’s the default search engine setup on Windows and they don’t know any better. Yahoo seems to be more about entertainment and eye candy, so they would likely draw the entertainment seeking, less serious searching, younger market.

More proof of the non techie users of Microsoft search is that the most technical search term anyone had used to find us was: “anti virus for windows 98se” which is somewhat lower on the “tech” ranking than CGI or MySQL error rankings. Yahoo is definitely heaps better than Microsoft at this stage, but they are still light years behind Google for tech terms.

As a test, I’m going to see if I can prove this conclusion by using a couple of pre-existing sub-domains and create non tech hotspots for certain keywords. Then after a year or so has gone by, I will analyze the traffic and see how much better the non Google engines do for non techie search terms.

I don’t think I’ve said anything revolutionary in the above, if I’d been asked a week ago what search engine techies preferred, I’d have said Google anyway, but I wasn’t expecting the ratio to be that high and growing. A year ago, about 94% of our search engine traffic was Google, now it’s 98.5%. Since we are talking about many thousands of referrals it’s quite a significant increase.

Recently I had to create a website for a customer that contained quite a long video clip. After much searching around and some experimentation with wmv and divx I decided to use Flash. After all, if it’s good enough for YouTube…… Flash can achieve very good compression rates while maintaining “watchability” and was supported by a larger percentage of the population than any other format, which was exactly what I needed. The video I had to work with was 1.8 gigabytes, so the compression I needed was going to have to be pretty extreme.

Next step was to find a free (preferably) tool to do the encoding, which I figured would mean I’d lose the very latest codecs, (and right I was) but I was hopeful I’d still achieve the results I needed. (and I did.)

After much searching I came across Super which is a video processing/encoding tool that offers flash encoding as one of its benefits. This is a well designed, functional piece of software and I got what I needed from it without once needing to read any documentation. However the design of the Super pages at the eRightsoft site is deplorable beyond measure and I can’t believe it was designed by the same people that created the tool. I could not download the tool from their website no matter how hard I tried, the page just kept looping. I had no proxy, no software firewall , nothing was blocking my referrer and I tried 3 browsers (IE, Firefox and Safari), then I gave up and downloaded it from Afterdawn.

I was thinking about writing a tutorial detailing the steps I took making my videos, but it really was too easy and not really worthy of one. The only tip I could really give is to right click on a blank part of the main window to see all the config settings.
The site I worked on was finished on time (2 days) and I had a perfectly watchable 10MB flash video embedded in the page as well as offering Xvid, DivX and WMV higher quality versions available for download. Perfect!

Both a new Harry Potter book and a new Harry Potter movie will shortly be released. Since Potter mayhem is at an all time high, why it’s time to write a computer virus to take advantage of all the people prepared to risk their computer to get an insight to it all.

The Hairy-A worm will create users from the Potter novels on your machine in addition to putting down a word document HarryPotter-TheDeathlyHallows.doc containing a message that Harry is dead. Worse it copies itself to USB storage devices and hops a ride to all your mates/school etc. Oh and it redirects the Internet Explorer home page to an Amazon HP spoof book page.

Read all the details at Sophos. If it’s all the same to you lot, I’ll wait for the book. 🙂

Checking my inbox this morning I found myself reading a “news letter” from Panda security. (I think I may have registered for this one at some time in the distant past) Anyway, their headline was “84% of emails received are SPAM*” (* Data from the Internauts Association .)

At first I just deleted it and went on deleting the spam that my 2 spam systems missed. Then it kind of sank in and I started rummaging though my trash looking for it. 84% of email is spam! that means that for every 100 emails you get, roughly 16 of them are legit, and the rest are trying to get you to buy questionable tablets or increase your organ size or buy some stock so that the email writers can sell their shares when you’ve made them worth something. (called a pump and dump).

Currently (in the past 24 hours) I’ve been getting inundated by spam, 3 times more than normal so I have no trouble believing that at all. In fact in my case currently, I’d put my spam ratio at around 92%. It’s a shame that something as useful as email has been relegated to an annoyance because of greedy knob heads.

Here is the funny bit though, I just received a spam email titled “Not all spam is spam. Some spam is kosher.” by some group by the name of “Tikkun” and they state that some email like political and non profit emails should be allowed as they are kind of public service stuff. The problem with his argument, is that email is worldwide.
Tikkun (in this instance) are a US political group. (or at least associated with one.) I’m in Australia. I don’t want my inbox filled with every political group in the world trying to convince me of their point of view. Heck, most of the time I don’t want my inbox filled with Australian politicos trying to sway my opinion to their cause. If I don’t think I know enough about something, I’ll research it. I don’t need help.
Likewise with the non profits, I don’t want the deaf foundation of America (just a made up example) trying to solicit money from me here in Australia. We have a deaf society of our own thank you. What it comes down to, is that email will be replaced with something else. (probably something that costs money) if it becomes useless as is slowly happening now. Money would solve the problem because if it cost money to send spam, it would no longer be viable. As it stands now, these people will kill the medium for everyone and something precious and undreamed of 50 years ago will have been lost.

In short, I’ll not be convinced that any unsolicited email isn’t spam until it can be limited to the region for which it is relevant, And probably not even then. Am I alone in that?