  ULOGD 2.x - the Netfilter Userspace Logging Daemon
  Harald Welte <laforge@netfilter.org>, Eric Leblond
  <eric@inl.fr>
  Revision 2009/04/18

  This is the documentation for ulogd-2.x, the second generation Netfil-
  ter Userspace logging daemon. ulogd makes use of the Linux >= 2.6.14
  nfnetlink_log and nfnetlink_conntrack subsystems, but also provides
  backwards compatibility for Linux >= 2.4.0 ipt_ULOG.
  ______________________________________________________________________

  Table of Contents


  1. DESIGN
     1.1 CONCEPT
     1.2 DETAILS

  2. INSTALLATION
     2.1 Linux kernel
     2.2 Userspace libraries
     2.3 ulogd
        2.3.1 Recompiling the source

  3. Configuration
     3.1 iptables NFLOG target
        3.1.1 Quick Setup
        3.1.2 ulogd_output_PCAP.so
        3.1.3 ulogd_output_SQLITE3.so
        3.1.4 ulogd_output_SYSLOG.so

  4. QUESTIONS / COMMENTS


  ______________________________________________________________________

  11..  DDEESSIIGGNN

  11..11..  CCOONNCCEEPPTT

  ulogd-2.x wants to provide a flexible, almost universal logging daemon
  for netfilter logging. This encompasses both packet-based logging
  (logging of policy violations) and flow-based logging, e.g. for
  accounting purpose.

  ulogd consists of a small core and a number of plugins. All the real
  power lies in the plugins, and in the user who configures the
  interactions between those plugins.


     IInnppuutt PPlluuggiinnss
        Input plugins acts data source. They get data from somewhere
        outside of ulogd, and convert it into a list of ulogd keys.

     FFiilltteerr PPlluuggiinnss
        Filter plugins interpret and/or filter data that was received
        from the Input Plugin. A good example is parsing a raw packet
        into IPv4 / TCP / ... header information.

     OOuuttppuutt PPlluuggiinnss
        Output plugins describe how and where to put the information
        gained by the Input Plugin and processed by one or more Filter
        Plugins.  The easiest way is to build a line per packet and
        fprint it to a file.  Some people might want to log into a SQL
        database or want an output conforming to the IETF IPFIX
        language.

  By means of the configuration file, the administrator can build any
  number of Plugin Stacks. A plugin stack is a series of plugins,
  starting with an Input plugin, none, one or multiple filter plugins,
  and one output plugin on top.

  11..22..  DDEETTAAIILLSS

  The major clue is providing a framework which is as flexible as
  possible.  Nobody knows what strange network protocols are out there
  :) But at the same time, logging of a packet filter is often very
  performance critical.  Like all ulogd releases since 0.3.x, the
  ulogd-2.x core doesn't do any dynamic allocations at runtime. Yes,
  obviously, at startup time the config file is parsed, and allocations
  are made. But after that, everything is pre-allocated. As an
  additional improvement over ulogd-1.x, there are also no hashtable
  lookups for key resolval. All input/output keys of plugins within
  every stack are resolved at config file parsing time, and directly
  interconnected by pointers.

  22..  IINNSSTTAALLLLAATTIIOONN

  22..11..  LLiinnuuxx kkeerrnneell

  To use the NFCT or NFLOG input plugin, you will need a 2.6.14 or later
  kernel.  For old-style ULOG logging, you need a kernel >= 2.4.18.

  22..22..  UUsseerrssppaaccee lliibbrraarriieess

  If you plan to use NFCT and NFLOG input plugin, you will need to
  compile libnfnetlink, libnetfilter_conntrack and libnetfilter_log
  libraries which can be downloaded from
  <http://ftp.netfilter.org/pub/>.  A simple './configure; make; make
  install' will be enough to have library installed on your system.

  22..33..  uullooggdd

  22..33..11..  RReeccoommppiilliinngg tthhee ssoouurrccee

  Download the ulogd package from  <http://ftp.netfilter.org/pub/ulogd/>
  and untar it.

  If you want to build ulogd with MySQL support, type './configure
  --with-mysql'. You may also have to specify the path of the mysql
  libraries using '--with-mysql=path'. To build ulogd without MySQL
  support, just use './configure'.

  To compile and install the program, call 'make install'.

  33..  CCoonnffiigguurraattiioonn

  33..11..  iippttaabblleess NNFFLLOOGG ttaarrggeett

  33..11..11..  QQuuiicckk SSeettuupp

  Just add rules using the NFLOG target to your firewalling chain. A
  very basic example:


       iptables -A FORWARD -j NFLOG --nflog-group 32 --nflog-prefix foo



  To increase logging performance, try to use the


       --nflog-qthreshold N



  option (where 1 < N <= 50). The number you specify is the amount of
  packets batched together i plugin automagically runs a procedure with
  arguments taken from a the configurable table; It connects to pgsql
  during the startup phase of ulogd and obtains a list of the columns in
  the table. Then it tries to resolve the column names against keys of
  interpreter plugins. This way you can easily build your own procedure
  and select it arguments just by modifying the layout of the table.

  If, for example, your table contains a field called 'ip_saddr', ulogd
  will resolve this against the key 'ip.saddr' and put the ip address as
  32bit unsigned integer into the table.

  The file 'doc/pgsql-ulogd2.sql' contains a schema for both packet and
  flow logging.

  The module defines the following configuration directives:

     ttaabbllee
        Name of the table which ulogd will use to build arguments list.

     pprroocceedduurree
        Stored procedure that will be run with the argument specified in
        the table variable.

     sscchheemmaa
        PGSQL schema to use.

     ddbb Name of the database.

     hhoosstt
        Name of the mysql database host.

     ppoorrtt
        TCP port number of database server.

     uusseerr
        Name of the sql user.

     ppaassss
        Password for sql user.

     rreeccoonnnneecctt
        Number of reconnection attempt before declaring the output
        plugin as dead.

     ccoonnnneecctt__ttiimmeeoouutt
        Database connection timeout.

  33..11..22..  uullooggdd__oouuttppuutt__PPCCAAPP..ssoo

  An output plugin that can be used to generate libpcap-style packet
  logfiles.  This can be useful for later analysing the packet log with
  tools like tcpdump or ethereal.

  The module defines the following configuration directives:

     ffiillee
        The filename where it should log to. The default is:
        /var/log/ulogd.pcap

     ssyynncc
        Set this to 1 if you want to have your pcap logfile written
        synchronously. This may reduce performance, but makes your
        packets appear immediately in the file on disk. The default is 0

  33..11..33..  uullooggdd__oouuttppuutt__SSQQLLIITTEE33..ssoo

  An output plugin for logging into a SQLITE v3 database. This is only
  compiled if you have the sqlite libraries installed, and the configure
  script was able to detect them. (that is: --with-sqlite3 was specified
  for ./configure)

  The plugin automagically inserts the data into the configured table;
  It opens the sqlite db during the startup phase of ulogd and obtains a
  list of the columns in the table. Then it tries to resolve the column
  names against keys of interpreter plugins. This way you can easily
  select which information you want to log - just by the layout of the
  table.

  If, for example, your table contains a field called 'ip_saddr', ulogd
  will resolve this against the key 'ip.saddr' and put the ip address as
  32bit unsigned integer into the table.

  You may want to have a look at the file 'doc/sqlite3.table' as an
  example table including fields to log all keys from ulogd_BASE.so.
  Just delete the fields you are not interested in, and create the
  table. This file contains two tables, one for packet-based logging and
  another for flow-based logging.

  To create the database file with the tables, you have to invoke the
  following command: sqlite3 ulogd.sqlite3db < sqlite3.table

  To check that we are logging stuff into it correctly: sqlite3
  ulogd.sqlite3db "SELECT * from ulog_ct"

  The module defines the following configuration directives:

     ttaabbllee
        Name of the table to which ulogd should log.

     ddbb Name of the database.

     bbuuffffeerr
        Size of the sqlite buffer.

  33..11..44..  uullooggdd__oouuttppuutt__SSYYSSLLOOGG..ssoo

  An output plugin that really logs via syslogd. Lines will look exactly
  like printed with traditional LOG target.

  The module defines the following configuration directives:

     ffaacciilliittyy
        The syslog facility (LOG_DAEMON, LOG_KERN, LOG_LOCAL0 ..
        LOG_LOCAL7, LOG_USER)

     lleevveell
        The syslog level (LOG_EMERG, LOG_ALERT, LOG_CRIT, LOG_ERR,
        LOG_WARNING, LOG_NOTICE, LOG_INFO, LOG_DEBUG)

  44..  QQUUEESSTTIIOONNSS // CCOOMMMMEENNTTSS

  All comments / questions / ... are appreciated.

  Just drop a note to netfilter-devel@vger.kernel.org.


  The preferred method for reporting bugs is the netfilter bugzilla
  system, available at  <http://bugzilla.netfilter.org/>.



