Log Analysis
Log Analysis
Hi,
Now I'm able to get web access log via TFTP.
Any recommendation of software to use to generate reports based on that log file ?
Thanks,
Miguel
Now I'm able to get web access log via TFTP.
Any recommendation of software to use to generate reports based on that log file ?
Thanks,
Miguel
Re: Log Analysis
Dear Miguel
Logs are generated using a common format which should be recognised by almost all known log file analysers. Unfortunately I can't recommend you a software in particular.
Best Regards,
Logs are generated using a common format which should be recognised by almost all known log file analysers. Unfortunately I can't recommend you a software in particular.
Best Regards,
Re: Log Analysis
Dear David,
I have 2 more questions:
- What is the standard format of the log?
I've tried with:
# 1 - Apache or Lotus Notes/Domino native combined log format (NCSA combined/XLF/ELF log format)
# 2 - IIS or ISA format (IIS W3C log format). See FAQ-COM115 For ISA.
# 3 - Webstar native log format.
# 4 - Apache or Squid native common log format (NCSA common/CLF log format)
but is not any of this.
- Can you configure the log format ?, for example a field delimiter would be very useful
This is how my logs looks like, just in case.
Thanks,
Miguel
192.168.15.62 - [21/Aug/2015:13:41:58 -0400] "POST http://clients1.google.com/ocsp HTTP/1.1" 200 813 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:41:59 -0400] "POST http://clients1.google.com/ocsp HTTP/1.1" 200 813 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:00 -0400] "POST http://clients1.google.com/ocsp HTTP/1.1" 200 813 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:07 -0400] "POST http://ocsp.digicert.com/ HTTP/1.1" 200 855 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:11 -0400] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:11 -0400] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:11 -0400] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TCP_MISS HIER_DIRECT
I have 2 more questions:
- What is the standard format of the log?
I've tried with:
# 1 - Apache or Lotus Notes/Domino native combined log format (NCSA combined/XLF/ELF log format)
# 2 - IIS or ISA format (IIS W3C log format). See FAQ-COM115 For ISA.
# 3 - Webstar native log format.
# 4 - Apache or Squid native common log format (NCSA common/CLF log format)
but is not any of this.
- Can you configure the log format ?, for example a field delimiter would be very useful
This is how my logs looks like, just in case.
Thanks,
Miguel
192.168.15.62 - [21/Aug/2015:13:41:58 -0400] "POST http://clients1.google.com/ocsp HTTP/1.1" 200 813 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:41:59 -0400] "POST http://clients1.google.com/ocsp HTTP/1.1" 200 813 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:00 -0400] "POST http://clients1.google.com/ocsp HTTP/1.1" 200 813 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:07 -0400] "POST http://ocsp.digicert.com/ HTTP/1.1" 200 855 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:11 -0400] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:11 -0400] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TCP_MISS HIER_DIRECT
192.168.15.62 - [21/Aug/2015:13:42:11 -0400] "CONNECT www.google.com:443 HTTP/1.1" 200 0 TCP_MISS HIER_DIRECT
Re: Log Analysis
Hello Miguel
You can find the log format at http://www.cacheguard.net/doc/command/log.html. This format is used by Squid (an open source proxy).
Actually the delimiter is space and the request field which may contain spaces is enclosed with quotation marks.
Best Regards,
You can find the log format at http://www.cacheguard.net/doc/command/log.html. This format is used by Squid (an open source proxy).
Actually the delimiter is space and the request field which may contain spaces is enclosed with quotation marks.
Best Regards,
Re: Log Analysis
Hello,
Thanks, I've tried http://www.webalizer.org that supports squid format but does not recognize the file. Do you know any software that works with the format of CG ?
Now that AD auth. is working my log file looks like:
192.168.110.4 prueba [07/Sep/2015:22:57:27 +0300] "GET http://www.ibm.com/ HTTP/1.1" 302 221 TCP_MISS HIER_DIRECT
192.168.110.4 miguelp [07/Sep/2015:22:58:10 +0300] "GET http://www.eldeber.com.bo/files/article ... 0-420.jpeg HTTP/1.1" 200 5146 TCP_MISS HIER_DIRECT
Cheers,
Miguel
Thanks, I've tried http://www.webalizer.org that supports squid format but does not recognize the file. Do you know any software that works with the format of CG ?
Now that AD auth. is working my log file looks like:
192.168.110.4 prueba [07/Sep/2015:22:57:27 +0300] "GET http://www.ibm.com/ HTTP/1.1" 302 221 TCP_MISS HIER_DIRECT
192.168.110.4 miguelp [07/Sep/2015:22:58:10 +0300] "GET http://www.eldeber.com.bo/files/article ... 0-420.jpeg HTTP/1.1" 200 5146 TCP_MISS HIER_DIRECT
Cheers,
Miguel
Re: Log Analysis
Dear Miguel
Actually CG uses a custom log format which is similar to Apache log format with additional information related to the caching provided by Squid. As David mentioned the CG log format is described at http://www.cacheguard.net/doc/command/log.html.
Webalizer allows you to use a CLF (Custom Log Format). The LogType and ApacheLogFormat statements should allows you to configure the right format to use. Please test the following webalizer configuration:
LogType clf
ApacheLogFormat %h %u %t \"%r\" %>s %b - -
Refer to http://www.stonesteps.ca/projects/webalizer/README.asp for further information.
Best Regards,
Actually CG uses a custom log format which is similar to Apache log format with additional information related to the caching provided by Squid. As David mentioned the CG log format is described at http://www.cacheguard.net/doc/command/log.html.
Webalizer allows you to use a CLF (Custom Log Format). The LogType and ApacheLogFormat statements should allows you to configure the right format to use. Please test the following webalizer configuration:
LogType clf
ApacheLogFormat %h %u %t \"%r\" %>s %b - -
Refer to http://www.stonesteps.ca/projects/webalizer/README.asp for further information.
Best Regards,
Re: Log Analysis
Hello Charles,
This command:
ApacheLogFormat %h %u %t \"%r\" %>s %b - -
Should be set in the Apache server, is not a directive for webalizer.
(see below extract from: ftp://ftp.mrunix.net/pub/webalizer/README)
That´s why I'm asking if I can configure the format of the log of the CG.
I've tested webalizer with:
LogType clf
and
LogType squid
and it tells me:
Skipping bad record for all the records in my log file.
Any ideas ?
Thanks,
Miguel
CLF format logs by default. For Apache, in order to produce the
proper log format, add the following to the httpd.conf file:
LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\""
This instructs the Apache web server to produce a 'combined' log
that includes the referrer and user agent information on the end of
This command:
ApacheLogFormat %h %u %t \"%r\" %>s %b - -
Should be set in the Apache server, is not a directive for webalizer.
(see below extract from: ftp://ftp.mrunix.net/pub/webalizer/README)
That´s why I'm asking if I can configure the format of the log of the CG.
I've tested webalizer with:
LogType clf
and
LogType squid
and it tells me:
Skipping bad record for all the records in my log file.
Any ideas ?
Thanks,
Miguel
CLF format logs by default. For Apache, in order to produce the
proper log format, add the following to the httpd.conf file:
LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\""
This instructs the Apache web server to produce a 'combined' log
that includes the referrer and user agent information on the end of
Re: Log Analysis
Dear Miguel
I think that Charles is talking about "Stone Steps Webalizer" (http://www.stonesteps.ca/) which is a fork of Webalizer. Webalizer development was stopped in 2002. I suggest you to upgrade to Stone Steps Webalizer which allows you to use the ApacheLogFormat statement as described by Charles.
Best Regards,
I think that Charles is talking about "Stone Steps Webalizer" (http://www.stonesteps.ca/) which is a fork of Webalizer. Webalizer development was stopped in 2002. I suggest you to upgrade to Stone Steps Webalizer which allows you to use the ApacheLogFormat statement as described by Charles.
Best Regards,
Re: Log Analysis
Hello David,
Thanks for the answer, but still not working.
I suppose this issue belongs to the Stone Steps Webalizer forum, I've posted there:
https://stonestepswebalizer.codeplex.co ... ons/644734
If you have any ideas, let me know.
Thanks,
Miguel
Thanks for the answer, but still not working.
I suppose this issue belongs to the Stone Steps Webalizer forum, I've posted there:
https://stonestepswebalizer.codeplex.co ... ons/644734
If you have any ideas, let me know.
Thanks,
Miguel
Re: Log Analysis
Hello,
I will really appreciate your help, because the other forum (stone steps), is totally inactive.
Thanks,
Miguel.
I will really appreciate your help, because the other forum (stone steps), is totally inactive.
Thanks,
Miguel.