I run a handful of sites on a dedicated server, getting around 30,000 total unique visitors a day. I've had that number higher without any problems.
However, whenever my site that's running WSN Links (Game Links) gets more traffic (even just a little extra), my server load goes up big time, and often my Apache resets due to the high load. Game Links is only getting about 2,000-2,500 visitors a day, which already causes the load to be higher than it should be. Has WSN Links been tested on a site with this much traffic yet?
This site is the only one I run on the server that uses php or mysql, so it doesn't even have to share those type of resources with another site. The crashes and high load didn't start until I started using WSN Links and began receiving decent traffic.
My server specs: P4 - 2.0Ghz - 80GBHD - 1GB RAM
I've already had techs check over my server and do everything possible to optimize it.
It depends on how you use it. You haven't specified your page generation time, so I can't comment.
This site gets maybe 5000-7000 page views a day (no idea about unique visitors, but surely that'd have no importance here). I couldn't tell you what it does to the server, though, since it's a shared host with presumably lots of other people's sites on it and I've no idea the specs of the server.
The site gets about 20,000 page views a day. Apache is resetting about once a day, and I see the load at 3-9 at various times. The site definitely loads slower than it should - sometimes there is a 5 second delay before the pages start loading. My other sites on the server don't have this problem at all, so it appears to be a wsn links specific problem.
Where do I find the page generation info? All I see when I log into the admin section that might be relevant is:
MySQL Stats: Uptime: 79349 Threads: 1 Questions: 1261908 Slow queries: 0 Opens: 283 Flush tables: 1 Open tables: 64 Queries per second avg: 15.903
I'm talking about the debug mode option. Anyhow, if your pages are taking longer than a second or two to generate you should look into what settings you're using to cause that. Of course posting your queries might give an indication.
Looks a bit slow, but the query numbers are about the same as mine. Though I can't really comment since you haven't shown the queries, it sounds like you need to either limit peak loads -- perhaps you're not controlling spiders -- or use a faster server.
Load time so far: 0.15 seconds. Performing query: SELECT id,name,parent,validated,description,time,parentnames,parentids,numlinks,hide,lastlinktime,custom,lastedit,moderators,headerinfo,related,numsub,type,permissions,subscribers,subscriber,mixtypes,isalbum,orderlinks,totalcomments,ulsearchmax,ulsearchquery,ulsearchadult,alias,catcols,linkcols,dmozpath,sponsorlinks,views,customwrap FROM wsnlinks_categories WHERE validated=1 AND hide=0 ;
Number of rows is 9 for Resource id #41
Load time so far: 0.15 seconds. Performing query: LOCK TABLES wsnlinks_email WRITE, wsnlinks_links WRITE, wsnlinks_categories WRITE, wsnlinks_members WRITE;
Load time so far: 0.15 seconds. Performing query: SELECT id,`to`,subject,message,headers,replacement,mime FROM wsnlinks_email WHERE timesent=0 ORDER BY id ASC LIMIT 0,20;
Number of rows is 0 for Resource id #42
Load time so far: 0.16 seconds. Performing query: UNLOCK TABLES;
Load time so far: 0.16 seconds. Performing query: SELECT id,name,parent,validated,description,time,parentnames,parentids,numlinks,hide,lastlinktime,custom,lastedit,moderators,headerinfo,related,numsub,type,permissions,subscribers,subscriber,mixtypes,isalbum,orderlinks,totalcomments,ulsearchmax,ulsearchquery,ulsearchadult,alias,catcols,linkcols,dmozpath,sponsorlinks,views,customwrap FROM wsnlinks_categories WHERE parent=0 AND validated=1 AND hide=0 AND isalbum=0 ORDER BY name ASC LIMIT 0,75;
Number of rows is 9 for Resource id #45
Load time so far: 0.17 seconds. Performing query: SELECT id FROM wsnlinks_categories WHERE parent=0 AND validated=1 AND hide=0 ;
Number of rows is 9 for Resource id #46
Load time so far: 0.18 seconds. Performing query: UPDATE wsnlinks_sessions SET starttime='0' WHERE time < 30;
Load time so far: 0.18 seconds. Performing query: UPDATE wsnlinks_sessions SET time='0' WHERE time < 30;
Load time so far: 0.18 seconds. Performing query: SELECT starttime FROM wsnlinks_sessions WHERE memberid=1 ;
Number of rows is 1 for Resource id #48
Load time so far: 0.18 seconds. Performing query: UPDATE wsnlinks_sessions SET time='1115685328', location='http://www.gamelinks.com/index.php', catid='', threadid='', areaname='Main' WHERE memberid=1;
Load time so far: 0.19 seconds. Performing query: SELECT id FROM wsnlinks_sessions WHERE memberid>0 ;
Number of rows is 1 for Resource id #49
Load time so far: 0.19 seconds. Performing query: SELECT id FROM wsnlinks_sessions WHERE memberid=0 ;
Number of rows is 108 for Resource id #50
Load time so far: 0.3 seconds. Performing query: SELECT COUNT(*) FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ;
Load time so far: 0.32 seconds. Performing query: SELECT id,title,url,description,rating,votes,validated,catid,sumofvotes,email,time,hits,numcomments,hide,ownerid,voterips,voterids,lastedit,type,filename,filetitle,notify,suspect,downloads,pendingedit,funds,suspended,alias,expire,ip,inalbum,typeorder,recipurl,hitsin,recipwith,hitsinips,hitsoutips,lastcomment,related,inhidden,viewers,threadviewers,hitsintemp,hitsouttemp,origtype,GAMEFORMAT,GAMEICON,importance,parentids,timesdead,timesemailed,threadclosed,threadposters,GAMEKEYWORDS,price,longdescription,xwidth,yheight FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ORDER BY time DESC LIMIT 0,12;
Number of rows is 12 for Resource id #52
Load time so far: 0.38 seconds. Performing query: SELECT COUNT(*) FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ;
Load time so far: 0.4 seconds. Performing query: SELECT id,title,url,description,rating,votes,validated,catid,sumofvotes,email,time,hits,numcomments,hide,ownerid,voterips,voterids,lastedit,type,filename,filetitle,notify,suspect,downloads,pendingedit,funds,suspended,alias,expire,ip,inalbum,typeorder,recipurl,hitsin,recipwith,hitsinips,hitsoutips,lastcomment,related,inhidden,viewers,threadviewers,hitsintemp,hitsouttemp,origtype,GAMEFORMAT,GAMEICON,importance,parentids,timesdead,timesemailed,threadclosed,threadposters,GAMEKEYWORDS,price,longdescription,xwidth,yheight FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ORDER BY time DESC LIMIT 4,12;
Number of rows is 12 for Resource id #54
Load time so far: 0.56 seconds. Performing query: SELECT COUNT(*) FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ;
Load time so far: 0.59 seconds. Performing query: SELECT id,title,url,description,rating,votes,validated,catid,sumofvotes,email,time,hits,numcomments,hide,ownerid,voterips,voterids,lastedit,type,filename,filetitle,notify,suspect,downloads,pendingedit,funds,suspended,alias,expire,ip,inalbum,typeorder,recipurl,hitsin,recipwith,hitsinips,hitsoutips,lastcomment,related,inhidden,viewers,threadviewers,hitsintemp,hitsouttemp,origtype,GAMEFORMAT,GAMEICON,importance,parentids,timesdead,timesemailed,threadclosed,threadposters,GAMEKEYWORDS,price,longdescription,xwidth,yheight FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ORDER BY time DESC LIMIT 8,12;
Number of rows is 12 for Resource id #56
Load time so far: 0.65 seconds. Performing query: SELECT COUNT(*) FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND (votes>9) AND alias=0 AND inalbum=0 ;
Load time so far: 1.01 seconds. Performing query: SELECT id,title,url,description,rating,votes,validated,catid,sumofvotes,email,time,hits,numcomments,hide,ownerid,voterips,voterids,lastedit,type,filename,filetitle,notify,suspect,downloads,pendingedit,funds,suspended,alias,expire,ip,inalbum,typeorder,recipurl,hitsin,recipwith,hitsinips,hitsoutips,lastcomment,related,inhidden,viewers,threadviewers,hitsintemp,hitsouttemp,origtype,GAMEFORMAT,GAMEICON,importance,parentids,timesdead,timesemailed,threadclosed,threadposters,GAMEKEYWORDS,price,longdescription,xwidth,yheight FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND (votes>9) AND alias=0 AND inalbum=0 ORDER BY rating DESC LIMIT 0,30;
Number of rows is 30 for Resource id #58
Load time so far: 1.17 seconds. Performing query: SELECT COUNT(*) FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ;
Load time so far: 1.19 seconds. Performing query: SELECT id,title,url,description,rating,votes,validated,catid,sumofvotes,email,time,hits,numcomments,hide,ownerid,voterips,voterids,lastedit,type,filename,filetitle,notify,suspect,downloads,pendingedit,funds,suspended,alias,expire,ip,inalbum,typeorder,recipurl,hitsin,recipwith,hitsinips,hitsoutips,lastcomment,related,inhidden,viewers,threadviewers,hitsintemp,hitsouttemp,origtype,GAMEFORMAT,GAMEICON,importance,parentids,timesdead,timesemailed,threadclosed,threadposters,GAMEKEYWORDS,price,longdescription,xwidth,yheight FROM wsnlinks_links WHERE validated=1 AND hide=0 AND inhidden=0 AND alias=0 AND inalbum=0 ORDER BY hits DESC LIMIT 0,30;
I definitely need to check for votes>9, otherwise links with a perfect vote will beat out a links with 100+ votes that are near perfect.
Is there some other way I can reduce load with that toplist that you cant think of? I wouldn't mind that portion being a static file (and including it via php), but my past experiences of trying to get the static file generation to work was unsuccessful.
I doubt the votes>9 has much of anything to do with it. I just said that to identify the toplist, as opposed to any other toplists you may have. It's just toplists in general that can tend be slow when dealing with large database tables. Reducing the number of items on it might speed it some, but most of the time appears to be spent in calculating the number of possible results. (Perhaps extra conditions that limit the possible results would help, or perhaps not.)
It occours to me that your debug above is really the fault of multipage toplists. There needs to be a mechanism to skip the query that's required for multipage ones when the template doesn't call for it to be multipage.
In templatefunctions.php, change $totalpossible[$loopthis] = $db->selectcount($table[$loopthis], $condition);
Might have helped a small bit, but I notice that when I validate a link, the load on the machine skyrockets for several minutes, every single time, causing just this site to lag. Other sites on the server aren't affected as much - I can still access them fine, but the site running wsnlinks becomes unusable for a while.
Just validated a link and here's what my TOP looked like:
Synozeer wrote: when I validate a link, the load on the machine skyrockets for several minutes
You're sure you're not auto-generating HTML pages? That's the only thing I can see causing a load when validating a link. Otherwise validating doesn't trigger any activities except the confirmation email (which surely wouldn't cause much of a load).
0/5
1
2
3
4
5
This thread is closed, so you cannot post a reply.
Comments on WSN Links very resource intensive?
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
I run a handful of sites on a dedicated server, getting around 30,000 total unique visitors a day. I've had that number higher without any problems.
However, whenever my site that's running WSN Links (Game Links) gets more traffic (even just a little extra), my server load goes up big time, and often my Apache resets due to the high load. Game Links is only getting about 2,000-2,500 visitors a day, which already causes the load to be higher than it should be. Has WSN Links been tested on a site with this much traffic yet?
This site is the only one I run on the server that uses php or mysql, so it doesn't even have to share those type of resources with another site. The crashes and high load didn't start until I started using WSN Links and began receiving decent traffic.
My server specs: P4 - 2.0Ghz - 80GBHD - 1GB RAM
I've already had techs check over my server and do everything possible to optimize it.
Any help would be appreciated.
Thanks,
Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
It depends on how you use it. You haven't specified your page generation time, so I can't comment.
This site gets maybe 5000-7000 page views a day (no idea about unique visitors, but surely that'd have no importance here). I couldn't tell you what it does to the server, though, since it's a shared host with presumably lots of other people's sites on it and I've no idea the specs of the server.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
The site gets about 20,000 page views a day. Apache is resetting about once a day, and I see the load at 3-9 at various times. The site definitely loads slower than it should - sometimes there is a 5 second delay before the pages start loading. My other sites on the server don't have this problem at all, so it appears to be a wsn links specific problem.
Where do I find the page generation info? All I see when I log into the admin section that might be relevant is:
MySQL Stats: Uptime: 79349 Threads: 1 Questions: 1261908 Slow queries: 0 Opens: 283 Flush tables: 1 Open tables: 64 Queries per second avg: 15.903
-Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
I'm talking about the debug mode option. Anyhow, if your pages are taking longer than a second or two to generate you should look into what settings you're using to cause that. Of course posting your queries might give an indication.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Gotcha.
Turned on debug, here's a few results:
30 total queries
This page was created in 1.6085078716278 seconds
31 total queries
This page was created in 1.0952470302582 seconds
31 total queries
This page was created in 1.1056339740753 seconds
46 total queries
This page was created in 1.9353528022766 seconds
It's 1:15am though, so traffic is low now. I'll revist this tomorrow during the day and see what it's like during heavier traffic times.
Adam
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Here's a few more. One took over 5 seconds, and it had less queries:
23 total queries
This page was created in 5.9070420265198 seconds
30 total queries
This page was created in 2.1424849033356 seconds
30 total queries
This page was created in 1.3928120136261 seconds
30 total queries
This page was created in 1.4357650279999 seconds
Any ideas?
Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
Looks a bit slow, but the query numbers are about the same as mine. Though I can't really comment since you haven't shown the queries, it sounds like you need to either limit peak loads -- perhaps you're not controlling spiders -- or use a faster server.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Here's one of my queries:
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
You have a toplist where votes>9 that's taking almost half of your total execution time there (0.65 to 1.17).
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
I definitely need to check for votes>9, otherwise links with a perfect vote will beat out a links with 100+ votes that are near perfect.
Is there some other way I can reduce load with that toplist that you cant think of? I wouldn't mind that portion being a static file (and including it via php), but my past experiences of trying to get the static file generation to work was unsuccessful.
Adam
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
I doubt the votes>9 has much of anything to do with it. I just said that to identify the toplist, as opposed to any other toplists you may have. It's just toplists in general that can tend be slow when dealing with large database tables. Reducing the number of items on it might speed it some, but most of the time appears to be spent in calculating the number of possible results. (Perhaps extra conditions that limit the possible results would help, or perhaps not.)
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
It occours to me that your debug above is really the fault of multipage toplists. There needs to be a mechanism to skip the query that's required for multipage ones when the template doesn't call for it to be multipage.
In templatefunctions.php, change
$totalpossible[$loopthis] = $db->selectcount($table[$loopthis], $condition);
to
if (strstr($totalpage, '{TOPLIST'. $loopthis .'MULTIPAGE}') || strstr($totalpage, '{TOPLIST'. $loopthis .'CURRENT')) $totalpossible[$loopthis] = $db->selectcount($table[$loopthis], $condition);
Should help.
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Thanks Paul. I'm going to give this a try.
-Adam
Forum Regular
Usergroup: Customer
Joined: Jun 02, 2004
Total Topics: 32
Total Comments: 142
Might have helped a small bit, but I notice that when I validate a link, the load on the machine skyrockets for several minutes, every single time, causing just this site to lag. Other sites on the server aren't affected as much - I can still access them fine, but the site running wsnlinks becomes unusable for a while.
Just validated a link and here's what my TOP looked like:
20:17:03 up 16 days, 21:16, 1 user, load average: 13.32, 7.83, 4.49
230 processes: 225 sleeping, 5 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 3.7% 16.5% 13.1% 1.7% 2.1% 62.6% 0.0%
Mem: 1017320k av, 721172k used, 296148k free, 0k shrd, 46488k buff
530600k actv, 87224k in_d, 10788k in_c
Swap: 2048276k av, 117680k used, 1930596k free 362568k cached
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7867
when I validate a link, the load on the machine skyrockets for several minutes
You're sure you're not auto-generating HTML pages? That's the only thing I can see causing a load when validating a link. Otherwise validating doesn't trigger any activities except the confirmation email (which surely wouldn't cause much of a load).