 |
W3Perl speed
|
|
One w3perl weakness come from its speed to deliver stats. Of course, the
package is not able to compute as fast as package written in C but the following data will show you
that w3perl can do its job in a reasonable amount of time ... and as CPU speed are going faster and
faster, time is less a critical value.
Package | Langage | Speed (lines/s) | Time to process an 1 Gb logfile | Time to process an 1 Tb logfile
|
Analog | C | 23 000 | 3 min 40 | 1 h |
Webalizer | C | 10 000 | 8 min 20 | 2 h 20 |
AWstats | Perl | 4 100 | 20 min 20 | 5 h 40 |
W3Perl | Perl | 1 600 | 52 min | 14 h 30 |
An average 10 000 hits/day will produce a 1.2 Gb logfile/year and so stats will need around 1 h to complete.
If W3Perl is slower than AWstats, this is because W3Perl produce more detailed stats.
Don't forget that this is the
time need to compute stats from scratch. Daily run are incremental and take a few minutes to complete
whatever your logfile size.
|
|
|
Lines / sec
|
Stats have been computed on a Pentium III 450 Mhz 128 Mb under Linux.
The number of lines which have been rejected by the user configuration
are not counted, it means in fact that only lines processed are displayed.
One point is one day, so a day took
between 3 and 20 seconds to compute. To give an idea of access, there are
around 10,000 hits a day or logfiles are around 1800 Kb large. Eight months of logfiles have been computed.
|
|
|
CPU timing
|
If you want to send me your timing, feel free to email me your configuration.
A file is produced in your /data/cputime, you could send it to me.
Website |
Intervalle |
Logfile size (compressed) |
Hits / day |
CPU (reverse dns off) |
OS |
Speed (lines / sec) |
W3Perl |
73 days |
3,5 Mb |
5,000 |
3 min 55 |
PIII 450 (Linux) |
1000 |
W3Perl |
421 days (14 months) |
11 Mb |
2,000 |
16 min 20 |
PIII 450 (Linux) |
1000 |
INRP |
30 days |
3,6 Mb |
10,000 |
4 min 59 |
PIII 450 (Linux) |
1000 |
INRP |
244 days (8 months) |
47 Mb |
10,000 |
36 min 00 |
PIII 450 (Linux) |
1000 |
To give an idea, compressed logfiles are 90% smaller than uncompressed ones.
|
|
|
Hints
|
If you want to increase speed :
- Don't use the reverse dns option, querying dns server could take
several seconds for each IP address. Results are stored in a file for
the next calls but it really make things very slow. You could win
a three to five speed increase.
If your logfile have only IP address, the 'countries' stats could not
be computed without dns reverse on.
- Don't select too much pages in the @selection variable.
- Use only one language output.
|
|
|