Resource usage

News items for the DSLR Users website. Also covers upcoming but important events. Locked at present time, although recognised members may post messages in response to news items posted.

Moderator: Moderators

Forum rules
Please ensure that you have a meaningful location included in your profile. Please refer to the FAQ for details of what "meaningful" is.

Resource usage

Postby gstark on Tue Dec 20, 2005 10:48 am

Those of you who are hammerring the server waiting for Birddog's specials: DON'T!

You are also hammerring our resource usage, and that causes me real grief. I've just banned about 7 IPs for excessive hits on the server. No doubt the affected users will email me in due course, although it's interesting to note that some of the IPs that I banned last time this happenned still haven't been in touch.

As an observation, I'm home most of the day, and we have two users hitting the site from here. Our total usage yesterday was around 550 hits on the site. Bear in mind that we're not just two userrs, but we're both admins!

Noting that number, I am astounded to see that some users are hitting the site in excess of 2000 times during the same period. Those users need to seriously consider getting a life.

Get out and take some bloody photos!

Be warned that anyone hitting the site more than 1500 times in a day is now likely to be be banned, because you are causing me too many headaches. Please permit others fair usage and enjoyment of this site.

The bottom line is that I may have to - reluctantly - move the services back to NoDaddy, because of the popularity of the site, and the fact that DH won't permit us to use the resources fully.
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby Alpha_7 on Tue Dec 20, 2005 10:58 am

Any in the 202.92.87.x or 220.233.162.x range ? I have no idea how much my usage is.. but I know I don't have much of a life outside of the forums these days :P
User avatar
Alpha_7
Senior Member
 
Posts: 7259
Joined: Sun Aug 14, 2005 6:19 pm
Location: Mortdale - Sydney - Nikon D700, x-D200, Leica, G9

Postby the foto fanatic on Tue Dec 20, 2005 11:03 am

1500 hits in a (say) 10 hour day is still 150 hits per hour, or 2.5 hits per minute.
Even allowing for a 15 hour stint at the computer, we are talking 1.66 hpm.

I don't think anyone (perhaps with the exception of Birddog himself) could possibly post so frequently, so in my view, this limit is somewhat generous still.
Or am I missing something?

Edit: If you are "refreshing" just to catch a Birddog special, one refresh every 20 minutes guarantees you at least 10 minutes in his 30 minute window - this would only equate to 45 hits over 15 hours. :)
Last edited by the foto fanatic on Tue Dec 20, 2005 11:08 am, edited 1 time in total.
TFF (Trevor)
My History Blog: Your Brisbane: Past & Present
My Photo Blog: The Foto Fanatic
Nikon stuff!
User avatar
the foto fanatic
Moderator
 
Posts: 4212
Joined: Tue Aug 24, 2004 7:53 pm
Location: Teneriffe, Brisbane

Postby owen on Tue Dec 20, 2005 11:03 am

I don't know how many hits I am doing either, although I do know it is more than my bosses would like :?
http://www.ausphotos.com - My Gallery

http://www.doesgodexist.com - a very interesting site.
User avatar
owen
Senior Member
 
Posts: 1699
Joined: Thu Jan 06, 2005 3:21 pm
Location: Nowra, NSW

Postby Mj on Tue Dec 20, 2005 11:04 am

gstark wrote:Be warned that anyone hitting the site more than 1500 times in a day is now likely to be be banned


Just to clarify... you really meant to say 15 times didn't you :?: :?: :?:
Cause 1500 times a day requires some serious traetment :shock: :shock: :shock:
User avatar
Mj
Senior Member
 
Posts: 1048
Joined: Fri Aug 20, 2004 3:37 pm
Location: Breakfast Point, Sydney {Australia}

Postby losfp on Tue Dec 20, 2005 11:06 am

I hope I'm not being too much of a drain... I tend to have one or two favourite sites in the background all day while I work, and check for new posts every 10-15 minutes.

But 1500 hits means more than 2 hits every minute for TWELVE HOURS. That's some hitting :)

I also run a Sydney Swans messageboard in "my other life", and we got some pretty intense resource drain during this year's finals series, for obvious reasons. During the Barry Hall tribunal hearings in the week before the grand final, we exceeded the entire monthly bandwidth (12GB) within TWO DAYS, what with people constantly hammering the server to check the latest news (the infamous BarryGate incident of Sept '05).

I guess Birdy's specials are a bit like that :) Only 2 more days of speculation! From memory, he said there would only be specials until the 21st.
User avatar
losfp
Senior Member
 
Posts: 1572
Joined: Thu Nov 17, 2005 12:45 pm
Location: Quakers Hill, Sydney

Postby stubbsy on Tue Dec 20, 2005 11:07 am

Craig. Like you I visit here frequently :D but no way would it be 2,000 hits in a day. Let's say you're on 16 hours in a day (unlikely, but stay with me) then that's roughly 1 hit (page visit) EVERY 30 seconds of that 16 hours. If you're on 8 hours throughout a day then it's every 15 seconds etc.

No matter how you look at it 2000 is a hell of a lot of hits in a day.

Edit - looks like I was too slow with my maths since a few of you already said the same thing :shock:
Peter
Disclaimer: I know nothing about anything.
*** smugmug galleries: http://www.stubbsy.smugmug.com ***
User avatar
stubbsy
Moderator
 
Posts: 10748
Joined: Wed Dec 08, 2004 7:44 pm
Location: Newcastle NSW - D700

Postby radar on Tue Dec 20, 2005 11:33 am

Gary,

why not post the IP addresses that are hammering the site? The "offenders" can then see that they really need a life. It can be self-regulating. If they continue to hammer the site, then go and ban them and see if they suffer from withdrawl ;-)

It would also be interesting to see what phpBB make "one" hit. Is it the actual full page or various sections count as a page hit, eg News, topic headlines, etc?

cheers,

radar
User avatar
radar
Senior Member
 
Posts: 2823
Joined: Mon Jun 13, 2005 11:18 am
Location: Lake Macquarie (Newcastle) - D700, D7000

Postby phillipb on Tue Dec 20, 2005 11:34 am

This is probably a silly suggestion, but is it possible to have log of people and their average clicks? similar to their total post counts.
__________
Phillip


**Nikon D7000**
User avatar
phillipb
Senior Member
 
Posts: 2599
Joined: Sat Aug 07, 2004 10:56 am
Location: Milperra (Sydney) **Nikon D7000**

Postby gstark on Tue Dec 20, 2005 11:37 am

Alpha_7 wrote:Any in the 202.92.87.x or 220.233.162.x range ? I have no idea how much my usage is.. but I know I don't have much of a life outside of the forums these days :P


Craig,

The fact that you posted that message means that you're not (yet :) ) affected.

The ban I've put in place uses the .htaccess file, and totally bars them from accessing dslrusers.net.
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby Alpha_7 on Tue Dec 20, 2005 11:42 am

Glad I'm ok then.. I use the view posts since last on (ALOT) as in I basically use it to catch up on every post I've missed.. and then start digging deeper.. I wasn't sure if viewing it like that creates more load or generates more hits.
User avatar
Alpha_7
Senior Member
 
Posts: 7259
Joined: Sun Aug 14, 2005 6:19 pm
Location: Mortdale - Sydney - Nikon D700, x-D200, Leica, G9

Postby gstark on Tue Dec 20, 2005 11:44 am

cricketfan wrote:1500 hits in a (say) 10 hour day is still 150 hits per hour, or 2.5 hits per minute.
Even allowing for a 15 hour stint at the computer, we are talking 1.66 hpm.

I don't think anyone (perhaps with the exception of Birddog himself) could possibly post so frequently, so in my view, this limit is somewhat generous still.
Or am I missing something?

Edit: If you are "refreshing" just to catch a Birddog special, one refresh every 20 minutes guarantees you at least 10 minutes in his 30 minute window - this would only equate to 45 hits over 15 hours. :)


Remember that a "hit" isn't the same as a page serve. One page serve results in a significant number of hits, as the components of a page (icons, images, etc) get farmed out, so the numbers I've quoted, while accurate, aren't the best indicator of a person's activities.

However, any comparison between any other IP and mine is, because of the simple fact that we have two users here, both of whom are admins, both of whom might be here for several hours throughout the day, yet our numbers are significantly below those of some others ...

Go figure. :)
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby MHD on Tue Dec 20, 2005 11:47 am

DoS?
New page
http://www.potofgrass.com
Portfolio...
http://images.potofgrass.com
Comments and money always welcome
User avatar
MHD
Moderator
 
Posts: 5829
Joined: Sat Sep 11, 2004 8:51 pm
Location: Chicago Burbs

Postby Nnnnsic on Tue Dec 20, 2005 11:55 am

From who, Scott? Maxwell? :lol: :lol:
Producer & Editor @ GadgetGuy.com.au
Contributor for fine magazines such as PC Authority and Popular Science.
User avatar
Nnnnsic
I'm a jazz singer... so I know what I'm doing
 
Posts: 7770
Joined: Sun Aug 08, 2004 12:29 am
Location: Cubicle No. 42... somewhere in Bondi, NSW

Postby Alpha_7 on Tue Dec 20, 2005 11:57 am

dpreview ?
:shock:
User avatar
Alpha_7
Senior Member
 
Posts: 7259
Joined: Sun Aug 14, 2005 6:19 pm
Location: Mortdale - Sydney - Nikon D700, x-D200, Leica, G9

Postby gstark on Tue Dec 20, 2005 11:58 am

radar wrote:Gary,

why not post the IP addresses that are hammering the site? The "offenders" can then see that they really need a life. It can be self-regulating. If they continue to hammer the site, then go and ban them and see if they suffer from withdrawl ;-)

It would also be interesting to see what phpBB make "one" hit. Is it the actual full page or various sections count as a page hit, eg News, topic headlines, etc?

cheers,

radar


See my comments above regarding what a "hit" actually is.

I'm loathe to publish the IP addresses that I've banned; I'm not sure what publishing them would achieve - they couldn't see that anyway, because they now cannot get here; I'm sure they'll find a way to email me in due course. :)


Of interest though, might be the following extract from our daily stats ... Note the variation between requests and pages ...

I've also included the CPU minutes used for each of those days. 40 minutes daily is the target DH want us to not exceed (by too much).

While there's no direct relationship between page hits and CPU usage, clearly there must be something there.


Monday ...

Successful requests: 70,009
Average successful requests per day: 71,803
Successful requests for pages: 15,073
Average successful requests for pages per day: 15,458

51 minutes CPU


Sunday ...

Successful requests: 44,082
Average successful requests per day: 42,774
Successful requests for pages: 8,172
Average successful requests for pages per day: 7,928

27 minutes CPU


Saturday ...

Successful requests: 40,686
Average successful requests per day: 42,361
Successful requests for pages: 8,097
Average successful requests for pages per day: 8,429

24 minutes CPU

Friday ...

Successful requests: 65,519
Average successful requests per day: 63,575
Successful requests for pages: 15,066
Average successful requests for pages per day: 14,618

42 minutes CPU
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby gstark on Tue Dec 20, 2005 12:03 pm

MHD wrote:DoS?


Not evident in the logs, Scott.

Last time there was one clear "offender, being a GoogleBot, which accounted for around 50% of the traffic encountered. This time it all looks just like excessive popularity.

We also need to take into account time differentials between the logging (US based timezones) and when we're most active here.

I'm thinking I may need to purge some of the history from the database, thus making query times shorter.
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby bago100 on Tue Dec 20, 2005 6:58 pm

Naughty Googlebot! :evil:

Gary - Birdie; Have you thought about a mass email to announce the next sale? That might save a heap of cpu time - might not either, I wouldn't know but just a thought anyway. :D

Cheers

Graham
User avatar
bago100
Senior Member
 
Posts: 862
Joined: Mon Sep 06, 2004 6:42 pm
Location: Shanghai China until Feb 2010

Postby gstark on Tue Dec 20, 2005 7:14 pm

Graham,

Short answer is no, but with spam being what it is, I'm rather reluctant to (ab)use the resources we have and risk the ire os those members (such as my good self) who absobloodylutely abhor junk email.

Interesting suggestion though; thanx.
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby elffinarts on Tue Dec 20, 2005 8:33 pm

perhaps rather than using the phpbb database to send out emails for those sales (which I've never been in time for LOL) - make an opt-in mailing list instead. One that requires verification to be added to the mailing list and therefore no reason to complain of spam.

I can see from all this that my own site will never have issues with bandwidth since the 5gig/day allowance is miles from what this far larger community uses.

why not switch to the same server? http://www.powweb.com
Mark Greenmantle
http://www.elffinarts.com / mark at elffinarts dot com
D70, 50mm/F1.8, kit lens, 80-200mm/F2.8, 35-70mm/f2.8, two 160w/sec slave strobes, sb600, "taller than me" astronomical tripod "can I have that step ladder please"
User avatar
elffinarts
Member
 
Posts: 495
Joined: Fri Aug 12, 2005 2:17 pm
Location: Albion, Brisbane

Postby pippin88 on Tue Dec 20, 2005 9:23 pm

elffinarts: I don't think it's raw bandwidth that is the problem, but rather CPU and Ram usage?

Gary: how did you get the CPU time figure?
- Nick
Gallery
User avatar
pippin88
Senior Member
 
Posts: 1107
Joined: Thu Jan 06, 2005 6:42 pm
Location: Newcastle / Sydney

Postby thaddeus on Tue Dec 20, 2005 9:35 pm

A friend and I have a dedicated server which hosts a number of websites. We have plenty of bandwidth and CPU available. The reliability of the server is very high, as Leek, rog and embi will attest.

I am happy to host dslrusers at my cost. That way, all the worrying about hits, CPU usage, banning IPs, etc should go away. (DOS aside!)
User avatar
thaddeus
Member
 
Posts: 418
Joined: Fri Jan 28, 2005 10:04 pm
Location: Sydney

Postby oli on Wed Dec 21, 2005 12:37 am

thaddeus wrote:A friend and I have a dedicated server which hosts a number of websites. We have plenty of bandwidth and CPU available. The reliability of the server is very high, as Leek, rog and embi will attest.

I am happy to host dslrusers at my cost. That way, all the worrying about hits, CPU usage, banning IPs, etc should go away. (DOS aside!)


I was about to ask about this. Is this site hosted on a shared server where the host can say it's using "too many resources" and threaten to cut you off?
Photo gallery online <a href="http://photoden.net/oliver">here</a> and some more on deviantArt <a href="http://oliau.deviantart.com">here</a>.
User avatar
oli
Member
 
Posts: 240
Joined: Sun Aug 28, 2005 9:27 am
Location: West Beach, South Australia

Postby gstark on Wed Dec 21, 2005 9:40 am

Nick,

When you get notified of the "problem" they start providing additional information which includes this data.

Russell, could you please email me details of the server?

The issue, as Nick points out, isn't bandwidth, but DH places an arbitrary limit of 40 CPU minutes per day on each site.

Of course, there's no mention of this in their blurbs as and when you sign up.

And they also provide a 97 day money back guarantee .... thinking .....
g.
Gary Stark
Nikon, Canon, Bronica .... stuff
The people who want English to be the official language of the United States are uncomfortable with their leaders being fluent in it - US Pres. Bartlet
User avatar
gstark
Site Admin
 
Posts: 22896
Joined: Thu Aug 05, 2004 11:41 pm
Location: Bondi, NSW

Postby DaveB on Wed Dec 21, 2005 11:56 am

Meanwhile, I'm left wondering if we'll ever get the RSS feed back!
User avatar
DaveB
Senior Member
 
Posts: 1850
Joined: Wed Aug 31, 2005 10:57 pm
Location: Box Hill, Vic


Return to Announcements

Who is online

Users browsing this forum: No registered users and 24 guests