Miscellaneous ⇒ Search Engines :: Archives ⇒ New Bot? :: Archived ⇒ Community Forums ⇒ CPG Dragonfly™ CMS
Forum IndexSearch Engines

Archived ⇒ New Bot?


I've got someone crawling my photo gallery... not sure if it's a good bot/bad bot? Maybe something called "SE-PICSEARCH" or "Picsearch AB"(see below). Any ideas?

Here's the IP addresses it's using:
217.212.224.143
217.212.224.146
217.212.224.159

ws.arin.net/cgi-bin/wh...12.224.143
Search results for: 217.212.224.143


OrgName: RIPE Network Coordination Centre
OrgID: RIPE
Address: P.O. Box 10096
City: Amsterdam
StateProv:
PostalCode: 1001EB
Country: NL

ReferralServer: whois.ripe.net:43

NetRange: 217.0.0.0 - 217.255.255.255
CIDR: 217.0.0.0/8
NetName: 217-RIPE
NetHandle: NET-217-0-0-0-1
Parent:
NetType: Allocated to RIPE NCC
NameServer: NS-PRI.RIPE.NET
NameServer: NS3.NIC.FR
NameServer: SUNIC.SUNET.SE
NameServer: NS-EXT.ISC.ORG
NameServer: SEC1.APNIC.NET
NameServer: SEC3.APNIC.NET
NameServer: TINNIE.ARIN.NET
Comment: These addresses have been further assigned to users in
Comment: the RIPE NCC region. Contact information can be found in
Comment: the RIPE database at www.ripe.net/whois
RegDate: 2000-06-05
Updated: 2005-07-27

# ARIN WHOIS database, last updated 2005-12-21 19:10
# Enter ? for additional hints on searching ARIN's WHOIS database.


www.ripe.net/whois?for...rch=Search


inetnum: 217.212.224.128 - 217.212.224.255
netname: SE-PICSEARCH
descr: Picsearch AB
org: ORG-PA85-RIPE
country: SE
admin-c: NA530-RIPE
tech-c: NA530-RIPE
status: ASSIGNED PA
mnt-by: TELIANET-LIR
source: RIPE # Filtered
organisation: ORG-PA85-RIPE
org-name: Picsearch AB
org-type: NON-REGISTRY
descr: Search engine
address: Hammarby Fabriksvag 23
address: SE-120 33 Stockholm
address: Sverige
phone: +4686412230
e-mail: noc @ picsearch.com
admin-c: NA530-RIPE
tech-c: NA530-RIPE
mnt-by: TELIANET-LIR
mnt-ref: TELIANET-LIR
source: RIPE # Filtered
person: Nils Andersson
address: Picsearch AB
address: Liljeholmsvägen 30B
address: SE 117 61 Stockholm
address: Sweden
e-mail: nils.andersson @ picsearch.com
phone: +46 8 6412230
mnt-by: NA530-MNT
nic-hdl: NA530-RIPE
source: RIPE # Filtered
% Information related to '217.212.224.0/19AS1299'
route: 217.212.224.0/19
descr: TELIANET-BLK
remarks: Abuse issues should be reported
remarks: to abuse @ teliacarrier.com
origin: AS1299
mnt-by: TELIANET-RR
source: RIPE # Filtered


I did find this on something called "Picsearch AB" to be used for lycos: info.lycos.com/pressro..._press.asp

www.picsearch.com/

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1


Looks good to me, maybe it should be added to dragonfly as a recognizable bot. Looks like it identifies itself as "psbot"

www.picsearch.com/menu...item=Psbot

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1


Coppermine is off limits acording to the default robots.txt file. Right?

If so, and if you have a stock robots.txt file that bot doesn't follow the rules of robots.txt.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Unix / 2.0.46 (Red Hat) / 0.9.7a / 4.1.9-standard / 4.3.2 / 9.0.6.1


How to Avoid Being Indexed
If you do not wish Picsearch to index your images please add 'psbot' to your robots.txt file according to the The Robot Exclusion Standard (example provided below). If you do not have access to the robots.txt file Psbot also obeys the robots meta tag and the psbot meta tag. (examples provided below).

* robots.txt example:
Include the following in your robots.txt file:

User-agent: psbot
Disallow: /

* Robots META tag example:
Include the following in the head section of your web-page:

<meta name="robots" content="noindex,nofollow">

* Psbot META tag example:
Include the following in the head section of your web-page:

<meta name="psbot" content="noindex,nofollow">

Please enter your server specs in your user profile! 😢


Never said I didn't want to allow it... maybe I do? I'm asking if you guys know what this bot is and if maybe the bot should be recognized by DF? It looks like it's indexing coppermine (and other various pages) to me.

I know how to prevent bots, that's not the point of this thread.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1


If it's a legit lycos bot then it sounds like they are launching some new image search like google images. The problem that google ran into was it was jacking copyrighted images too. The holders of those images sued google. So lycos better not be bypassing robots.txt to get at images or they'll find themselves sued too.

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux/1.3.33/4.4/4.3.11


djdevon3 wrote
If it's a legit lycos bot then it sounds like they are launching some new image search like google images. The problem that google ran into was it was jacking copyrighted images too. The holders of those images sued google. So lycos better not be bypassing robots.txt to get at images or they'll find themselves sued too.


doesn't look like they are bypassing robots.txt. However, I would say that CPGNuke should recognize the bot. Right now, it only recognizes it as a guest and not a bot in the who is where menu. Can that be fixed?

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1


picsearch (psbot) is far from being a new bot - many people disallow it because it indexes all your pictures so others can "borrow" them. It can also be very server intensive.

If you want to recognize it as a bot you will have to mod your version of includes/classes/cpg_member.php around line 188 plus.

The next version of Dragonfly has it listed as a known agent in the new security system and no longer uses cpg_member.php

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Ubuntu/Apache 2.2.22/MySQL 5.6.34/PHP 7.1.22/DragonFly 10.0.48.9418


PicSearch has recently turned up in my Who's Online Block (I'm running CVS) - It's either suddenly turned up and seems to be indexing a lot more sites, or it's been there all the time as a Guest, and is now classified as a bot Smile

Norbie

www.norbiesworld.co.uk

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Linux / Apache Custom Version / 4.0.26-standard (client: 5.0.15) / 4.4.4 / 9.1.1


Phoenix wrote
If you want to recognize it as a bot you will have to mod your version of includes/classes/cpg_member.php around line 188 plus.


Thank you!

Phoenix wrote
The next version of Dragonfly has it listed as a known agent in the new security system and no longer uses cpg_member.php


... and thank you... this is what I was getting at. Smile

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1


yw - I have a fair acquaintance with psbot as it's a frequent visitor.
Picsearch : December
Visits: 41386
Visits / Day: 1335.0
[8514] 217.212.224.142
[9379] 217.212.224.143
[6032] 217.212.224.144
[5141] 217.212.224.145
[6982] 217.212.224.146
[5338] 217.212.224.159

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Ubuntu/Apache 2.2.22/MySQL 5.6.34/PHP 7.1.22/DragonFly 10.0.48.9418


Phoenix wrote
yw - I have a fair acquaintance with psbot as it's a frequent visitor.
Picsearch : December
Visits: 41386
Visits / Day: 1335.0
[8514] 217.212.224.142
[9379] 217.212.224.143
[6032] 217.212.224.144
[5141] 217.212.224.145
[6982] 217.212.224.146
[5338] 217.212.224.159



haha, yea... me too. Never around, then all of a sudden it's been here since mid-december almost 24/7. I don't mind it, but I'm wondering how long it'll take for it to finish crawling.

Where did you pull that info from?

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1


Alright, I edited it so that PSBot will show as a bot, however it still shows the IP addresses as individual visitors. How can I fix that?

:edit: nevermind, I think it fixed itself?

Server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS):
Debian Linux/apache 1.3/4.0.23_Debian-1-log/4.3.10-2/9.0.6.1

All times are UTC