Logging

There is a lot of ways to think about logging. Nearly every software project in the world uses some variation of that. NetBeans indeed is not an exception. Actually NetBeans is an exemplary show case demonstrating how to do and also not do logging. Because this web page is currently occupied by third incarnation of logging tools. The older ones are dead, replaced by newer versions and indeed the hope of current maintainers is that the new version will survive for ages.

Logging in the Common Sense

When people talk about logging, they usually mean the hooks in the code that help to diagnose and record program's behavior. It means either log4j or Apache's Common Logging or JDK's logging. NetBeans used to be special - we had our own ErrorManager However now, after few years of having our own facade, NetBeans 6.0 finally adopted JDK standard - e.g. java.util.logging framework.

For information on this kind of logging read the Logging in NetBeans document.

Logging of UI Gestures

Because the case of common logging is now sufficiently handled by JDK, this project needed another vision. After few months of searching, one has been found: Let's create a framework that will observe, report and analyze actions users do in their own IDEs (or other NetBeans based applications). This framework consist of a log collector that collects reports of log agents running inside the user system and per user request transmits them to our server. The server then does analysis of the data, reports graphs, usage statistics, suggestions, etc.

Visit the analytics server...

Dear users of NetBeans IDE...

...please help us make NetBeans IDE better! Send your UI Gestures to our our analytics server. There are two ways of doing so in version 6.0. Either by using an exception reporter integrated in 6.0 builds, or by getting Interactive UI Gestures Collector from 6.0's update center, which adds an info icon into bottom right corner, allowing explicit transfer of your gestures to the server and opening the server to show you hints, advices, see various statistics graphs, use interactive statistics, etc. This icon is not visible immediatelly after the module is installed. It waits for sufficient amount of gestures is made and then it appears to inform you that now is the right time to submit data to the analytics server.

Visit the analytics server, check our presentation, or read the introduction...

Dear Developers...

...please help us make the UI Gestures Collector better!

Having to maintain a server like this is indeed a never ending project (unless the server gets shutdown, of course) and needs help. We are looking for skilled people that could improve the server side with useful graphs, information, etc. Recently we managed to integrate the server with knowledge base database provided by documentation team. Every support is welcomed.

Modify your modules to be UI Gesture Ready

Another way how to help the UI Gesture Collector project is to instrument modules with UI logging agents (send a LogRecord(Level.INFO) into "org.netbeans.ui.something" logger). This is very similar to common logging, yet indeed with a little bit of specification called structured logging. Please identify the interesting pieces of your code and instrument (+10 lines) them (+6 lines) with structured logging - this is the first step and very simple to do step that can help you understand the behavior of your users.

Learn every detail about structured logging or visit our FAQ...

Create Statistics for your server

If you log something, you also need to process it, of course! For that you need to write a Statistics and process the LogRecords sent to the server and also at least one JSP page that will show the results of your statistics to the user. Unlike the NetBeans APIs, the server is still under development and as such it has no stable API. That is why the best approach is to choose a working example, copy it and modify to suite your needs. As an example of a working statistics try to use the Gestures.java and their associated JSP page gestures.jsp . Just modify the server at your wish and check that your changes are working good enough. Then if you have a commit right, create a new branch and check in your changes into it, otherwise sent patch to jtulach.

Getting and compiling the sources. The logging project is part of NetBeans.org source tree and requires at least JDK 1.5 for compilation. That is why just follow the common instructions to get the sources. All the projects related to logging are buildable by ant from command line, and also ready to be opened in the NetBeans IDE. So we advice you to get some recent version of it (as of April 2007, it is enough to download NetBeans IDE 6.0 Milestone 8) to make your development experience better.

The minimal set steps to compile the server and run its tests are:
hg clone http://hg.netbeans.org/main/misc
ant -f misc/logger/uihandlerserver/build.xml all
During the build, additional libraries needed for compilation and test execution are downloaded and kept in logger/external directory.
Running tests takes long time. If you don't want to run tests, you can use deploy task to build sources and deploy without starting tests.
ant -f misc/logger/uihandlerserver/build.xml deploy

After doing the initial build, the logger/uihandlerserver is ready to be opened in NetBeans IDE 6.0 and operated from the GUI as usual for NetBeans-ready Ant projects.

Testing the IDE and Server

The simplest way and also the way to start the server that we test is to type
ant -f misc/logger/uihandlerserver/build.xml deploy -Danalytics.dir=dir_with_logs
This creates new tomcat_base directory and starts the server with it. The server will be configured to use the provided directory for reading and storing the log files.

If you have a server that can handle your data and an IDE that can produce such data, feel free to test that they can work in a harmony. Start the web application locally - it will usually run on http://localhost:8084/analytics/ address. Put breakpoints into it, modify it with logging messages, etc. Then start the IDE with your logging module with following argument: -J-Dorg.netbeans.modules.uihandler.Submit=http://localhost:8084/analytics/upload.jsp that way your IDE is always going to connect to your local server and you can test the interaction between your modules and your statistics running on your server. You can use Interactive UI Gestures Collector from 6.0's update center to submit data to your test server.

The advice given in previous paragraph is now automated if you choose logger/uihandler as your main project in NetBeans IDE. Its build script is modified to start the Analytics server before the IDE is launched in execution or debug mode. Also the org.netbeans.modules.uihandler.Submit is set up correctly to point to the just started server.

Happy Coding!

Enjoy the UI Gesture Collector logging, let me know if you have any thoughts to share and also do not miss slides from my presentation about this topic.