Cheshire Cat Computing

When my script rus it consumes 100% of a dual 3.0 gighz machine, I have removed the 95th percetile out of the .conf file and little change. Right now it consumes 100% for 28 seconds straight and also eats over 300megs of RAM and creates 70 perl proccesses. I have 50 config files and 1200 monitored ports, is this normal usage for these numbers? I want to add another 1000 ports or so to monitor, but dont want to imapact this server more...

This seems a bit excessive. We have 375 config files containing 7838 targets running on a twin CPU 3GHz xeon with Linux, and nothing like this usage.

To improve performance (see the routers.cgi:Advanced forum) we've done a couple of things.

1) Scheduled the MRTG data collection to run staggered over the 5-min window. We dont just kick off all data collection at the same time, this resulted in a heavy load at that point.

2) Install speedycgi for Apache. You could also use mod_perl. When one of these is installed and enabled for routers.cgi, then you can switch on caching in the routers2.conf file which will speed up the frontend considerably (as it doesn't need to re-read the config files to build the menus each time).

3) Have oodles of memory (actually, 2Gb in our case) and nice fast disks (superfast Adaptec controller with fast SCSI disks) MRTg and RRD are very IO intensive (naturally) and this helps a lot. Previously, we has some crappy EIDE disks and only 512Mb memory, and performance sucked.

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11

Thanks for the reply Steve, so are you saying go to a 5-minute monitoring cycle, right now I thought we have a 1 minute cycle, I am new to all of this so I will have to look and see, I will give your recommedations a try.

It also sounds as if your CPU is being eaten by the MRTG data collection, rather than routers.cgi (which only runs when you are requesting graphs). The 95th percentile calculation will have minimal effect on your performance at this level, and again only when you are requesting graphs.

Switch to a 5-min interval, spread your data collection over the 5 minutes, and get very very fast disks...

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11

Can you provide some instructions on how to spread them out over that 5min period.

Thanks,
Steve GOod

I wrote a custom script to do it evently, but your scripting skills may not be at that level? Alternatively, split your .cfg files into five groups, and make a master .cfg for each group that includes them. Then use your scheduler to schedule the five collection tasks at one-minute intervals.

_________________
Steve Shipway
UNIX Systems, ITSS, University of Auckland, NZ
Woe unto them that rise up early in the morning... -- Isaiah 5:11