Posted: Mon Feb 19, 2007 11:49 am Post subject: Bots Browsing Modules Sub-Folders
Ever since I updated to 9.1.1 the search bots seem to be browsing the sub-folders of the modules folder.
I saw some of them searching the "Your Account" pages and then even member profile pages. They seem to like looking at everything in there including the calendar, etc They never did this in 9.0.* unless I never noticed or it was not clearly identified on the whoiswhere block.
My robots.txt is default except for the bot-trap directory I have added: