Not that long ago Microsoft released the Vista operating system to what appeared to be an eager world. That appearance was apparently a mistake. Vista has been said to be rather slow unless it’s running on high end equipment and lots of hardware is still lacking drivers for Vista. DRM or Digital Rights Management software is probably partially responsible for the lower performance of systems running Vista. (compared to the same software running XP). Encoding and decoding encryption needs lots of CPU and ram power, and Vista is loaded with it, all designed to make sure you can’t do anything Microsoft and possibly any of their affiliates don’t want you to do. That’s right, it’s all designed to police what you can and can’t do with your computer. And best of all (from their perspective) YOU ARE PAYING THEM TO REDUCE YOUR CHOICES AND FREEDOM!. For example, the Hardware changes required to adhere to required Vista DRM standards ends up being added to the price of the hardware and Microsoft gets paid hand over fist for it every time someone buys a PC with Vista on it. Recently I downloaded the latest drivers for a piece of hardware I own, and these new drivers were designed with DRM and content protection in mind. After I loaded the new drivers, I found my hardware had slowed down so much that I couldn’t use it anymore. That’s because DRM is not free in any sense, it uses CPU cycles and it uses memory. Strangely enough, it doesn’t seem to surprise anyone, that the latest hardware running the latest Windows seems to do heaps of stuff slower than the previous versions running on slower hardware. Where is all that extra CPU and memory going?

There is allot more to this issue, but it’s been really well covered by people much more knowledgeable then myself. Here are some pages you really should read if you are curious or angry about paying a company to take away your choices. Let’s start with this informative article entitled “Who owns your computer“. Then follow it up with a “Cost Analysis of Windows Vista content protection” which will open your mind and possibly freak you out a little as it did me. After reading it, you can watch an add for Microsoft Vista and marvel at how Microsoft’s high priced advertising execs can spin it all to sound like stuff you really want to pay for. As long as they focus on the eye candy and gloss over everything else, most people don’t know enough to look any deeper.

After much recent studying of search engine trends and our rankings and hits from them here, I realized something. Although you don’t have to be a techie to know what HTML is, or to Google for error messages you get while installing scripts you found on the net, you are apparently allot more techie than the vast number of people who are searching for words like webpage or web design or Myspace or just blog. So after looking at our site carefully, I realized something, we overshot the biggest audience with most of our content. We’ve completely left the total newbies behind. Although we cover terms like HTML really well, absolute newbies have no idea what HTML is so why would they search for it? We don’t have anything for people who want to know, but don’t know what to search for to find out. (and so they look for things like “How do I create a website” and stuff of that nature.

To that end, I’ve started writing a beginner tutorial that aims to explain in the simplest way possible what the Web is and how it works, and filled it up with beginner search terms in the hope that it will result in more traffic from MSN/Yahoo/AOL/Altavista not to mention Google. Admittedly, this is partly an experiment in Search Engine Optimization on my part, but I believe that when I’m done it will be a very useful resource for absolute beginners to at least gain a basic understanding of how it all works.

Sigh….. I waited for someone to use that heading so I wouldn’t have to reveal myself as a closet Star Trek fan. Stories have been bandied about by all about a new report by scientists at the National Research Council which basically says that life elsewhere may have little in common with ours. My first thought was that they are right, aliens are probably very different from us, for example they are probably intelligent.

Then I put aside my cynicism and decided to wait and see how long it would be before someone used the lyrics from “Star Trekkin“. Sad to say, it turned out that it only occurred to me (or nobody else would admit to it).

All I can say in regards to the recent report, is that I’ll be terribly disappointed if, after watching years of sci-fi, it turns out that the only life out there is intelligent slime.

Shockingly, a Motorola phone has killed a 22 yo man in China after it exploded (or at least the battery did) in his pocket and pushed broken ribs into his heart. The man was welding at the time and the temperature is the most likely cause of the explosion. China is apparently rife with counterfeit cheap batteries and this may have played a part in the incident. Read more here.

I’ve been busy writing the new version of the Statistical hit counter, and in doing so, I write helper scripts to show me what new browser strings, screen resolutions etc are out there now that perhaps should be counted. In doing so, I’ve noticed an odd new trend and I believe I have an explanation for it. Of the past 6781 unique hits we’ve had, 2470 have apparently had JavaScript turned off, the beta counter that has been runing behind htmlfixit has been tracking JS and non JS (including bots) for about 2 years now.

Think about that, of 6781 hits, 2470 of them didn’t have JavaScript (not counting known search engine spiders either). I didn’t believe the number could be that high, but I’ve manually reviewed the stats data myself and it’s legit. If my maths is correct, that’s over 35% of all traffic having no JavaScript. Everyone else I’ve read about has claimed figures of more than 80% having JS turned on, so why are my figures so off?

After some thought, I realized or at least theorized as to what the cause is and the answer is something I imagine other CMS or forum systems (systems in which strangers can post or comment) could testify to. Automated Comment spamming programs, I wrote a quick program to print out the details of those browsers that did not support JavaScript, (or had it turned off) and group them by popularity. The program is here, and the list is many and varied, which is to be expected because spammer submit bots can pretend to be anything at all, so copying popular browser strings would be common sense to them. I don’t believe all of these are faked, this is a techie oriented site and many of use are security conscious enough to turn off JavaScript, I just don’t believe anywhere near 35% of us are.

We have some systems in place here in our CMS, a blacklist based on IP or domain and a system that works in much the same way that SpamAssassin does (except HTTP not email) by grading comments on known criteria and blocking comments below a certain threshold. These systems stop allot of bad hits (in fact we’ve had not at all get though in ages, wish they’d stop trying), but most of the site is not in the CMS and therefore gets bad hits. (like the forum which had 80 odd spiders in it at once last week.) I’m now looking at the possibility of using mod_security (which we already have) in conjunction with a blacklist of known spammer IP’s to block them before they ever get a full connection to apache. My concern with this is the risk of blocking normal nice non-scummy people as well, so it has to be done carefully.

Interestingly enough, a bit of Googling shows me that I am not alone in this thinking, and even better I’ve found a mod_security rules filter that looks like it will really make a difference just in the pattern checks alone. I am going to try this now and I’ll let you know how it goes. (Assuming we haven’t accidentally blocked you from accessing our server.) If all goes well, and I’ll know how successful the measure is by how many comment blocks are listed in our spam log. I’ll let you know how it goes and what exactly I did.

Update: We are now running the latest mod_security, with the latest gotroot rules and comment spam blacklists. In the past few days we’ve seen a 50% drop in comment spamming attempts. I’d imagine that blocking these hordes of comment spamming bots will show a significant drop in bandwidth usage also. I was required to make a couple of exclusion rules so that all our various programs work as planned without being blocked by mod_security, but other than that it has been a straight forward and easy (and worthwhile) experience. The more people that use this fantastic tool, the more the blacklists will be effective and up to date. If you have control over your own server, you could do allot worse than to install mod security.

Analyzing the traffic details reported by my Advanced Stats counter, I’ve noticed some interesting trends regarding the various search engines. HTMLfixIT is now getting roughly 98.5% of our search engine traffic from Google. Now we have worked hard to create a site that would do well with search engines, and we have loads of links from other peoples sites to our own, so it isn’t surprising that we do well with Google. What is surprising, is that we do far better from Google than Yahoo, Microsoft and the others despite the fact that their search engine algorithms are growing closer and that now days we usually rank the same or similar for many of our search terms in all engines. Lets take this brief example search string:
‘column count doesn’
Which is the start of a MySQL error that we explain in one of our tutes. The full string is:
Invalid Query: Column count doesn’t match value count at row 1 MySQL
but we don’t need the full string as the partial works rather well to demonstrate my point. and the addition of the ‘ in doesn’t seems to confuse things with some of them. (though it doesn’t change the conclusions I’m drawing here as I’ve tried it with many tech search terms that bring us traffic from Google.)

That string ranks the following in the top 5 search engines.
Google Number 3
Yahoo Number 1
Microsoft Number 1
Altavista Number 1
AOL Number 10

Now for the above term, Google has sent us 213 unique visitors in the past 5 days, Yahoo has sent us 2 visitors for that term and the others have sent us none at all. It sounds like an odd string to use for this example, but I’ve tried it in what is possibly the most searched error message on the Internet as well. “premature end of script headers” and it the same trend is found.

Now I have a friend who has a great many domain names and the vast majority of them are not tech based. They are mostly spiritual in nature, (meditation, psychic readings etc). This friends sites get far far more traffic from the other engines then does htmlfixit, and in fact their ratio of traffic from Google to the other engines is much better also.

So what can we conclude from this?
Well if htmlfixit can rank higher on MSN and Yahoo (for example) than on Google for a popular tech search term, and still get a hundred times more traffic for that search term from Google. Then the logical conclusion is that Google is the search engine preferred by tech oriented people.
That makes sense really, for example; Firefox and Safari both use Google as their homepage, and it is the tech aware users who are installing these browsers (on Windows anyway) on their machines. The people using Microsoft search are likely doing so because it’s the default search engine setup on Windows and they don’t know any better. Yahoo seems to be more about entertainment and eye candy, so they would likely draw the entertainment seeking, less serious searching, younger market.

More proof of the non techie users of Microsoft search is that the most technical search term anyone had used to find us was: “anti virus for windows 98se” which is somewhat lower on the “tech” ranking than CGI or MySQL error rankings. Yahoo is definitely heaps better than Microsoft at this stage, but they are still light years behind Google for tech terms.

As a test, I’m going to see if I can prove this conclusion by using a couple of pre-existing sub-domains and create non tech hotspots for certain keywords. Then after a year or so has gone by, I will analyze the traffic and see how much better the non Google engines do for non techie search terms.

I don’t think I’ve said anything revolutionary in the above, if I’d been asked a week ago what search engine techies preferred, I’d have said Google anyway, but I wasn’t expecting the ratio to be that high and growing. A year ago, about 94% of our search engine traffic was Google, now it’s 98.5%. Since we are talking about many thousands of referrals it’s quite a significant increase.

Recently I had to create a website for a customer that contained quite a long video clip. After much searching around and some experimentation with wmv and divx I decided to use Flash. After all, if it’s good enough for YouTube…… Flash can achieve very good compression rates while maintaining “watchability” and was supported by a larger percentage of the population than any other format, which was exactly what I needed. The video I had to work with was 1.8 gigabytes, so the compression I needed was going to have to be pretty extreme.

Next step was to find a free (preferably) tool to do the encoding, which I figured would mean I’d lose the very latest codecs, (and right I was) but I was hopeful I’d still achieve the results I needed. (and I did.)

After much searching I came across Super which is a video processing/encoding tool that offers flash encoding as one of its benefits. This is a well designed, functional piece of software and I got what I needed from it without once needing to read any documentation. However the design of the Super pages at the eRightsoft site is deplorable beyond measure and I can’t believe it was designed by the same people that created the tool. I could not download the tool from their website no matter how hard I tried, the page just kept looping. I had no proxy, no software firewall , nothing was blocking my referrer and I tried 3 browsers (IE, Firefox and Safari), then I gave up and downloaded it from Afterdawn.

I was thinking about writing a tutorial detailing the steps I took making my videos, but it really was too easy and not really worthy of one. The only tip I could really give is to right click on a blank part of the main window to see all the config settings.
The site I worked on was finished on time (2 days) and I had a perfectly watchable 10MB flash video embedded in the page as well as offering Xvid, DivX and WMV higher quality versions available for download. Perfect!