Miscellaneous ⇒ Server Chat ⇒ Site using too many resources ⇒ Community Forums ⇒ CPG Dragonfly™ CMS

Site using too many resources Reply to topic

Go to page Previous  1, 2, 3, 4  Next
Author Message
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
They've taken my site down again so looks like flood protection is not working! Will need to get it back up again to try some of the other things...


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
Phoenix wrote
Using phpmyadmin, in table cms_security_agents:
- find the Baidu bot entry
- change the agent_ban field to -1

Yandex is also another useless Russian bot that consumes resources.
In the same table, find the YANDEX bot:
- change the agent_ban field to -1

Now wait for their session to finish or remove their entries from your session table.


What should the Baidu entry look like? I deleted it and put in a new one with the agent full name as baidu* - do wildcards work? It seems from IPtracker that nothing identifying itself as Baidu is getting through but there are still masses of hits from baiduspiders...

I also banned the range 220.181.108.1 - 220.181.108.255 but IPTracker still shows lots of hits from that range.

Am I barking down the wrong tree with these bots? Should I be looking elsewhere, like at the calendar module for example as nanocairdo suggests?


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8840
Location: Netizen
Post
Baidu user agent is baiduspider - heaven knows what position it is in your db table now - my instructions were sufficient to block it.

Your calendar block looks as though it has been modified because the usual one doesn't have anywhere near that many queries.

You would be better off restricting it to members and using the rss feed in a block for public view of your events until you sort out your site.

Games is also a drain on your resources and should be limited to members.

_________________
DonationsPro for DragonflyCMS & SMF

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
I've really got to keep the calendar visible to non-members if I can. I've never used the rss feed for a block - is it straightforward?

I disable the games module too as it's rarely used...


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8840
Location: Netizen
Post
Laughing it's not very visible when your site is always down.

You have a constant problem so you need to start managing your site conservatively or you won't have one.

We have no idea what else you have on your site/server so we can only advise on what is visible. In that regard, I find it hard to believe you are pulling excess resources on what we can see.

So, you're going to have to sacrifice some niceties until you sort out your real problem, whatever that may be.

Just replace your fancy customized calendar block with the file rss/nucal.php.

You don't have to disable games, just make them available to members only.

_________________
DonationsPro for DragonflyCMS & SMF

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
Site should be there at the moment - no idea what the hosts are playing at!

I've set the scrolling calendar block to registered users only - it has reduced the number of db queries on the front page from 107 to 63 so a fair amount. Would this be sufficient for the problems I've had?

Incidentally with that IP range I've banned it is still showing up in IPTracker...


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8840
Location: Netizen
Post
Your host is correctly protecting the other server users from resource hogs, which is what your site appears to be to them. You're lucky - in times gone by Site5 would simply have put your site down until you could prove to them that it could be safely returned online and that might have been days/weeks.

I'm not getting involved in your ip banning - I just don't have the time.

If you ban through security, just bear in mind that it won't be immediate (refer my earlier post).

_________________
DonationsPro for DragonflyCMS & SMF

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Photo Gallery
NanoCaiordo
Developer
Developer

Offline Offline
Joined: Jun 29, 2004
Posts: 4200
Location: Italy
Post
We need to know if the cpu stats provided are the top 3 processes or the average. We also need to know the time length used to capture those figures.

  1. Go to Dragonfly CMS Admin -> Database.
  2. Do not select any tables and Check.
  3. If any broken table: select the table and Repair one table each time.
  4. Do not select any tables and Optimize.

Do not repair multiple tables at once.
Do not repair an entire database at once.
Repair one table at the time or you risk big.

_________________
.:: I met php the 03 December 2003 :: Unforgettable day! ::.

Linux 64bit / Apache 2.2 / PHP 5.4.9 / MySQL 5.5.22 / v9, v10
Linux 32bit / Apache 2.2 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10
Windows 64bit / IIS 7.5 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10




NanoCaiordo's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
mixed
View user's profile Visit poster's website Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
NanoCaiordo wrote
We need to know if the cpu stats provided are the top 3 processes or the average. We also need to know the time length used to capture those figures.

  1. Go to Dragonfly CMS Admin -> Database.
  2. Do not select any tables and Check.
  3. If any broken table: select the table and Repair one table each time.
  4. Do not select any tables and Optimize.

Do not repair multiple tables at once.
Do not repair an entire database at once.
Repair one table at the time or you risk big.


Thanks. The only table error came in

chantill_dragonfly.chantill_pronews_sections

warning

Found row where the auto_increment column has the value 0

I didn't repair this. I've then optimized and most tables were up to date but a fair few came back saying 'OK' so presume they've been optimized.

I'm waiting for a reply from site5 to see if the changes I've made so far have made any difference.

Thanks for taking the time - I don't wish to lose this site!


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
Apparently "CPU usage is calculated as a rolling 24 hour average"


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
NanoCaiordo
Developer
Developer

Offline Offline
Joined: Jun 29, 2004
Posts: 4200
Location: Italy
Post
Then if its a 24h average your site should always be slow. Im pretty sure those are Top processes for the last 24 hours.

Anyways table need to be repaired. Select the broken table and repair it.

_________________
.:: I met php the 03 December 2003 :: Unforgettable day! ::.

Linux 64bit / Apache 2.2 / PHP 5.4.9 / MySQL 5.5.22 / v9, v10
Linux 32bit / Apache 2.2 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10
Windows 64bit / IIS 7.5 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10




NanoCaiordo's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
mixed
View user's profile Visit poster's website Photo Gallery
layingback
Forum Admin
Forum Admin

Offline Offline
Joined: Apr 19, 2004
Posts: 1303
Location: Andalucia
Post
Nano, Rightly or wrongly Pro_News is designed to have 1 entry in cms_pronews_sections with key value of 0. I (now) know that key value of 0 with auto_index is deprecated, but it is never used as such. Table is initially loaded with records at 0 and 1, so the 0 key never gets to be auto-incremented.

I can't see how an error of a deprecated key value can cause excessive CPU cycles.

_________________
Pro_News CM™ - Content Management for Dragonfly CMS™ - Now Released as Version 4

layingback's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Linux / 1.3.42 - 2.2.12 / 5.1.72 / 5.4.21 / 9.4
View user's profile Visit poster's website Photo Gallery
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8840
Location: Netizen
Post
He's not saying that at all - it's sound webmaster practice to ensure your db tables are regularly checked, repaired and optimized.

What we are saying is sort out the basics, and operate conservatively until the problem(s) can be found.

_________________
DonationsPro for DragonflyCMS & SMF

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 650
Location: Paris, France
Post
So if I repair that table what will it do to the index?


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
View user's profile Visit poster's website
NanoCaiordo
Developer
Developer

Offline Offline
Joined: Jun 29, 2004
Posts: 4200
Location: Italy
Post
What cause CPU spikes are damaged, corrupted and/or fragmented tables.

We are only trying to guess, and if sql was the cause to also fix, the issue.
A table come out with a warning and I decided to take actions.

@rosbif If the table was designed like this it can stay that way.

@layingback Remember that the module is subject to be broken, I did make sure to be understood when I asked to check and then repair only broken tables, one table per run, because i know the risks involved.
This table, also, cannot be repaired. Hope it doesn't get damaged or repairing the table may delete the 0 index ... re-add the index later? If mysql lets you to.

@robif Let us know how it goes. What puzzle me is the usual "nothing changed but from 2 days my site is overloading the server"

Is you site sending out mails? Do you have any module with "tell a friend feature", have you or anyone else modified anything?

_________________
.:: I met php the 03 December 2003 :: Unforgettable day! ::.

Linux 64bit / Apache 2.2 / PHP 5.4.9 / MySQL 5.5.22 / v9, v10
Linux 32bit / Apache 2.2 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10
Windows 64bit / IIS 7.5 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10




NanoCaiordo's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
mixed
View user's profile Visit poster's website Photo Gallery
Display posts from previous:
All times are UTC
Go to page Previous  1, 2, 3, 4  Next
Post new topic Reply to topic Forum IndexServer Chat
Page 2 of 4



Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum


User Info

Welcome Anonymous



(Register)
Community

Support for DragonflyCMS in a other languages:

Deutsch
Español