June 26th, 2008
It turned out that the upgrade did not solve the problem as it was header.php in my theme that was modified to include a <u> element with display:none that included a long list of bad links. (see excerpt at http://www.pastie.org/223173 ).
I assume it happened via sql injection that I hope had been resolved in WordPress 2.5.1
I assume it got in via sql injection that the new version has solved.
June 25th, 2008
This blog has been hacked in a very peculiar way. See the summaries for the various blog entries on Google:
Google’s cache and the pages themselves seem “clean” - no mention of various drugs and such and no “bad” links. So I am unsure what is the benefit to the hackers. In any case I used this as an opportunity to upgrade to WP 2.5.1.
Hopefully the problem will go away when Google & Yahoo reindex.
January 30th, 2008
I mentioned this in an earlier post about how the site’s traffic from google has wild fluctuations without any significant change to the site.
Overall the traffic to the site has been growing as a result of additional content, but I have periods that the traffic dries to practically zero only to rebound again. Its not that the pages are not indexed - they are, but for some reason during the “dry spells” they are buried deep in the search results. Take a look at this report from google analytics:
As you can see every so often the traffic drops significantly - usually for a few days, but in December this lasted for about two weeks. In these periods the site “practically” vanishes from search results (though as I mentioned the pages are all indexed and can be found if searching for them specifically).
No particular keyword contributes more than 1% of total traffic to the site, but I do track a couple of keywords which warn me of this phenomena : “song list” is usually ranked in the #2 or #3 spot , but during the Google “mood swing” it is not in the 1st 1000 results.
If you visit Google’s webmaster central forums. Many complain about a drop in their ranking for a particular keyword from top page to “buried”. The response from the participants (mostly conjectures, since most of Google’s methods are not public) is that something is “wrong” with the site: bad linking, non valid markup, bad practices etc….
This does not seem to apply to song-list.net, since (a) I do not change anything substantial on the site apart from adding content and (b) If I have done something “bad” in Google’s eyes then the ranking/traffic would not rebound back.
Even if Google continuously tweaks the search algorithm continuously why would it cause such enormous fluctuations. I can understand that the ranking for a particular keyword could fluctuate considerably , but not all keyword would fluctuate.
Anyone care to postulate what is happening ?
August 19th, 2007
Maybe sometime I’ll clean it up and re-release it as a rails generator, but we’ll see if I get to it - I am drowning with lots of things to do.
May 31st, 2007
Finally I’ve upgraded to Rails 1.2.2 with a new database (now around 3100 artists) and added DVD and Book boxes to the song list page.
May 23rd, 2007
Almost immediately after my last post, Google traffic plummeted by about 80% - was anyone reading the blog ??? ;-), so the majority of traffic is now coming from Yahoo even though it has indexed much less of the site. The Google bot still visits much more aggressively than Yahoo slurper. I am very curious as to the cause of the drop in traffic - most of the hits before and after are from pages in the supplemental index and I cannot imagine that the distribution of queries has changed. The different behavior is just about 4 weeks old - I am just working on other things, waiting to see what happens.
April 27th, 2007
My traffic currently is split about evenly between Google and Yahoo (both bringing about 40%). The way they bring traffic is totally different. The Yahoo traffic is directed whereas Google’s traffic is much more random. Let me elaborate.
I track both the artist pages and the total site pages (they are mostly individual track pages) that are indexed by both engines. Here is today’s report:
- Google: 2,130 artist pages out of 89,900 total
- Yahoo: 457 artist pages out of 2,852 total
As you can see Google has indexed 30 times more pages than Yahoo ! It is easy to see that also from the logs - Google’s Bot is extremely aggressive - it spiders the site all day - last week it hit 20,000 pages in one day. Yahoo Slurps at a more casual rate - a few hundred to a thousand pages - dependent on its mood ;-).
One would think that Google would dominate the traffic, but this is not the case. Here are a few keyword hits from both engines:
list of 50 cent songs
celine dion list of songs
list of ray charles song
list of eric clapton songs
list of jay z songs
lil jon songs
“soft by boney james”
le notti di cabiria rota listen
“song of delilah”
Songlist Farewell Angelina
cunla song wikipedia
lil boosie distant lover
The Yahoo traffic comes primarily from searchers that are looking for song lists and the site ranks well for these searches as I would hope. Google’s traffic is coming from particular word combinations that match certain pages very well even if they are hidden in the supplemental index. I hope that the main artist pages will get indexed in the main index and ranked reasonably so I will get the same kind of hits from Google that I am getting from Yahoo. I am sure many people are searching for ” [artist name] song list”, but the site does not pop up in the first few pages.
March 11th, 2007
It’s official - Song List is part of web 2.0. I’ve been slashdotted here. The torrent was hard to withstand… a total of 10 hits! Nonetheless - a milestone.
March 7th, 2007
So.. I added a level of indirection. Now when the users click on an online music store icon - it comes back to the Song List controller. For example:
The MP3 action then redirects to the appropriate music library (in this case itunes). I keep all the neccesary information in the url so the controller doesn’t need to do a database call. I’ve seen some shopping sites do the same only the real link is transmitted via a key - requiring a DB call, and taking much more time.
Now I can see in the logs which music libraries get clicked more often (answer: so far it is Rhapsody).
March 4th, 2007
I just released a new version of the database. There are now over 2,000 artists listed. This is of course just a drop in the sea of available music.
I’ve started a new job, so I have a limited amount of time to enhance the site and collect more data. At the moment I am trying to get the current content indexed a bit better by google and yahoo. This has been challenging - more in a future post.