Home Private Messages Search
CPG Dragonfly™ CMS
Toggle Content
 
Forums ⇒ Miscellaneous ⇒ Server Chat ⇒ Site using too many resources


Site using too many resources
Talk about good (DedicatedNOW) and bad (AdventureHost) hosts, or any other question about servers and hosting, but no advertising or pricing in any form.
Go to page Previous  1, 2, 3, 4  Next
Post new topic    Reply to topic    Printer Friendly Page     Forum Index ⇒  Server Chat

View previous topic :: View next topic  
Author Message
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Thu Nov 03, 2011 9:47 am
Post subject: Re: Site using too many resources

They've taken my site down again so looks like flood protection is not working! Will need to get it back up again to try some of the other things...


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Thu Nov 03, 2011 9:57 am
Post subject: Re: Site using too many resources

Phoenix wrote:
Using phpmyadmin, in table cms_security_agents:
- find the Baidu bot entry
- change the agent_ban field to -1

Yandex is also another useless Russian bot that consumes resources.
In the same table, find the YANDEX bot:
- change the agent_ban field to -1

Now wait for their session to finish or remove their entries from your session table.

What should the Baidu entry look like? I deleted it and put in a new one with the agent full name as baidu* - do wildcards work? It seems from IPtracker that nothing identifying itself as Baidu is getting through but there are still masses of hits from baiduspiders...

I also banned the range 220.181.108.1 - 220.181.108.255 but IPTracker still shows lots of hits from that range.

Am I barking down the wrong tree with these bots? Should I be looking elsewhere, like at the calendar module for example as nanocairdo suggests?


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8837
Location: Netizen
PostPosted: Thu Nov 03, 2011 10:24 am
Post subject: Re: Site using too many resources

Baidu user agent is baiduspider - heaven knows what position it is in your db table now - my instructions were sufficient to block it.

Your calendar block looks as though it has been modified because the usual one doesn't have anywhere near that many queries.

You would be better off restricting it to members and using the rss feed in a block for public view of your events until you sort out your site.

Games is also a drain on your resources and should be limited to members.

_________________
DonationsPro for DragonflyCMS, SMF, MyBB, vBulletin

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Thu Nov 03, 2011 10:57 am
Post subject: Re: Site using too many resources

I've really got to keep the calendar visible to non-members if I can. I've never used the rss feed for a block - is it straightforward?

I disable the games module too as it's rarely used...


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8837
Location: Netizen
PostPosted: Thu Nov 03, 2011 12:15 pm
Post subject: Re: Site using too many resources

Laughing it's not very visible when your site is always down.

You have a constant problem so you need to start managing your site conservatively or you won't have one.

We have no idea what else you have on your site/server so we can only advise on what is visible. In that regard, I find it hard to believe you are pulling excess resources on what we can see.

So, you're going to have to sacrifice some niceties until you sort out your real problem, whatever that may be.

Just replace your fancy customized calendar block with the file rss/nucal.php.

You don't have to disable games, just make them available to members only.

_________________
DonationsPro for DragonflyCMS, SMF, MyBB, vBulletin

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Thu Nov 03, 2011 12:54 pm
Post subject: Re: Site using too many resources

Site should be there at the moment - no idea what the hosts are playing at!

I've set the scrolling calendar block to registered users only - it has reduced the number of db queries on the front page from 107 to 63 so a fair amount. Would this be sufficient for the problems I've had?

Incidentally with that IP range I've banned it is still showing up in IPTracker...


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8837
Location: Netizen
PostPosted: Thu Nov 03, 2011 1:07 pm
Post subject: Re: Site using too many resources

Your host is correctly protecting the other server users from resource hogs, which is what your site appears to be to them. You're lucky - in times gone by Site5 would simply have put your site down until you could prove to them that it could be safely returned online and that might have been days/weeks.

I'm not getting involved in your ip banning - I just don't have the time.

If you ban through security, just bear in mind that it won't be immediate (refer my earlier post).

_________________
DonationsPro for DragonflyCMS, SMF, MyBB, vBulletin

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website Photo Gallery
NanoCaiordo
Developer
Developer

Offline Offline
Joined: Jun 29, 2004
Posts: 4173
Location: Italy
PostPosted: Thu Nov 03, 2011 1:10 pm
Post subject: Re: Site using too many resources

We need to know if the cpu stats provided are the top 3 processes or the average. We also need to know the time length used to capture those figures.

  1. Go to Dragonfly CMS Admin -> Database.
  2. Do not select any tables and Check.
  3. If any broken table: select the table and Repair one table each time.
  4. Do not select any tables and Optimize.

Do not repair multiple tables at once.
Do not repair an entire database at once.
Repair one table at the time or you risk big.

_________________
.:: I met php the 03 December 2003 :: Unforgettable day! ::.

Linux 64bit / Apache 2.2 / PHP 5.4.9 / MySQL 5.5.22 / v9, v10
Linux 32bit / Apache 2.2 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10
Windows 64bit / IIS 7.5 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10




NanoCaiordo's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
mixed
Back to top
View user's profile Visit poster's website Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Thu Nov 03, 2011 2:22 pm
Post subject: Re: Site using too many resources

NanoCaiordo wrote:
We need to know if the cpu stats provided are the top 3 processes or the average. We also need to know the time length used to capture those figures.

  1. Go to Dragonfly CMS Admin -> Database.
  2. Do not select any tables and Check.
  3. If any broken table: select the table and Repair one table each time.
  4. Do not select any tables and Optimize.

Do not repair multiple tables at once.
Do not repair an entire database at once.
Repair one table at the time or you risk big.

Thanks. The only table error came in

chantill_dragonfly.chantill_pronews_sections

warning

Found row where the auto_increment column has the value 0

I didn't repair this. I've then optimized and most tables were up to date but a fair few came back saying 'OK' so presume they've been optimized.

I'm waiting for a reply from site5 to see if the changes I've made so far have made any difference.

Thanks for taking the time - I don't wish to lose this site!


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Thu Nov 03, 2011 3:06 pm
Post subject: Re: Site using too many resources

Apparently "CPU usage is calculated as a rolling 24 hour average"


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
NanoCaiordo
Developer
Developer

Offline Offline
Joined: Jun 29, 2004
Posts: 4173
Location: Italy
PostPosted: Thu Nov 03, 2011 9:49 pm
Post subject: Re: Site using too many resources

Then if its a 24h average your site should always be slow. Im pretty sure those are Top processes for the last 24 hours.

Anyways table need to be repaired. Select the broken table and repair it.

_________________
.:: I met php the 03 December 2003 :: Unforgettable day! ::.

Linux 64bit / Apache 2.2 / PHP 5.4.9 / MySQL 5.5.22 / v9, v10
Linux 32bit / Apache 2.2 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10
Windows 64bit / IIS 7.5 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10




NanoCaiordo's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
mixed
Back to top
View user's profile Visit poster's website Photo Gallery
layingback
Forum Admin
Forum Admin

Offline Offline
Joined: Apr 19, 2004
Posts: 1272

PostPosted: Fri Nov 04, 2011 7:21 am
Post subject: Re: Site using too many resources

Nano, Rightly or wrongly Pro_News is designed to have 1 entry in cms_pronews_sections with key value of 0. I (now) know that key value of 0 with auto_index is deprecated, but it is never used as such. Table is initially loaded with records at 0 and 1, so the 0 key never gets to be auto-incremented.

I can't see how an error of a deprecated key value can cause excessive CPU cycles.

_________________
Pro_News CM™ - Content Management for Dragonfly CMS™ - Now Released as Version 4

layingback's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Linux / 1.3.42 - 2.2.12 / 5.1.72 / 5.4.21 / 9.4
Back to top
View user's profile Visit poster's website Photo Gallery
Phoenix
• Many Posts •
• Many Posts •

Offline Offline
Joined: Apr 19, 2004
Posts: 8837
Location: Netizen
PostPosted: Fri Nov 04, 2011 7:44 am
Post subject: Re: Site using too many resources

He's not saying that at all - it's sound webmaster practice to ensure your db tables are regularly checked, repaired and optimized.

What we are saying is sort out the basics, and operate conservatively until the problem(s) can be found.

_________________
DonationsPro for DragonflyCMS, SMF, MyBB, vBulletin

Phoenix's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website Photo Gallery
rosbif
500+ Posts Club
500+ Posts Club

Offline Offline
Joined: Jan 13, 2005
Posts: 647
Location: Paris, France
PostPosted: Fri Nov 04, 2011 8:21 am
Post subject: Re: Site using too many resources

So if I repair that table what will it do to the index?


rosbif's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
Back to top
View user's profile Visit poster's website
NanoCaiordo
Developer
Developer

Offline Offline
Joined: Jun 29, 2004
Posts: 4173
Location: Italy
PostPosted: Fri Nov 04, 2011 12:49 pm
Post subject: Re: Site using too many resources

What cause CPU spikes are damaged, corrupted and/or fragmented tables.

We are only trying to guess, and if sql was the cause to also fix, the issue.
A table come out with a warning and I decided to take actions.

@rosbif If the table was designed like this it can stay that way.

@layingback Remember that the module is subject to be broken, I did make sure to be understood when I asked to check and then repair only broken tables, one table per run, because i know the risks involved.
This table, also, cannot be repaired. Hope it doesn't get damaged or repairing the table may delete the 0 index ... re-add the index later? If mysql lets you to.

@robif Let us know how it goes. What puzzle me is the usual "nothing changed but from 2 days my site is overloading the server"

Is you site sending out mails? Do you have any module with "tell a friend feature", have you or anyone else modified anything?

_________________
.:: I met php the 03 December 2003 :: Unforgettable day! ::.

Linux 64bit / Apache 2.2 / PHP 5.4.9 / MySQL 5.5.22 / v9, v10
Linux 32bit / Apache 2.2 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10
Windows 64bit / IIS 7.5 / PHP 5.3.19 / MySQL 5.5.22 / v9, v10




NanoCaiordo's server specs (Server OS / Apache / MySQL / PHP / DragonflyCMS)
mixed
Back to top
View user's profile Visit poster's website Photo Gallery
Display posts from previous:   
Post new topic    Reply to topic    Printer Friendly Page    Forum Index ⇒  Server Chat
Page 2 of 4
All times are GMT
Go to page Previous  1, 2, 3, 4  Next



Jump to:  


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum


 
   Toggle Content User Info

Welcome Anonymous

Nickname
Password
(Register)

   Toggle Content Mercurial Commits
· Added: Security sessions management Bugfix: infinite loop in $leochanged check
· Bugfix: Poodle\Mail\SMTP failed sending Bugfix: Security system undefined data B
· Added: Poodle\HTTP\URLInfo Bugfix: Poodle\HTTP\Request headers parsing and body
· Bugfix: \Poodle::getKernel()->IDENTITY can be undefined in debugger Change: spli
· Changed another L10N->get() to L10N->dbget() for CFG->global->slogan and CFG->gl
· Changed L10N->get() to L10N->dbget() for CFG->global->slogan and CFG->global->si
· Improved L10N admin
· Update version due to L10N changes
· Bugfix: L10N date parsing Added: L10N dbget() check v9 constants
· Merge

read more...

   Toggle Content Community

Support for DragonflyCMS in a other languages:

Deutsch
Español

You are seeing squares or questionmarks on this page?

All content of this website is copyrighted by the Creative Commons NC-SA
The logos and trademarks used on this site are the property of their respective owners
We are not responsible for comments posted by our users, as they are the property of the poster.
Our server runs on a P3 1.2GHz with 512MB RAM with no accelerators
Support GoPHP5.org
Interactive software released under GNU GPL, Code Credits, Privacy Policy