Simple example of custom file processor

Let's imagine a simple text file transformation like, for example, converting a text to uppercase.

The file:

Hello world
This is my simple
test file.

must become:

HELLO WORLD
THIS IS MY SIMPLE
TEST FILE.

we will see how to do it with the Piper framework.

Writing the java processor class

First, write the simplest code doing this file transformation. Assuming that we have a BufferedReader (called 'in') and a BufferedWriter (called 'out') ready to use and that they will be correctly closed later, it will look like:

        String line;
        while ((line = in.readLine()) != null) {
            out.write(line.toUpperCase());
            out.newLine();
        }

Well, the core of the processing is very simple ... just 5 lines. To turn that code into a useful tool with a nice graphical interface, it's quite simple with the "File Piper" framework.

You have to provide an implementation of the org.filePiper.model.FileProcessor interface. You can directly implement the interface or inherit from base classes. If your file transformation is one-to-one like here (one input file always produce one output file), it's simpler to extend one of the two provided specialized abstract classes:

  • OneToOneByteFileProcessor: implementation customized for processing binary files (using buffered input/output streams).
  • OneToOneTextFileProcessor: implementation customized for processing text files (using buffered readers/writers).

Those two abstract classes will provide default implementation for input/output file management. All you have to do is to implement the few remaining abstract methods. In our example we will choose OneToOneTextFileProcessor because we process text files (So we want to manipulate readers/writers rather than input/output streams).

Here's is the complete implementation:

public class ToUppercaseProcessor extends OneToOneTextFileProcessor {

    public String getProcessorName() {
        return "to upper case";
    }

    public String getProcessorDescription() {
        return "This processor converts the input text to uppercase";
    }

    public String getProposedNameSuffix() {
        return "up";
    }

    public void process(BufferedReader in, BufferedWriter out, FileProcessorEnvironment env) throws Exception {
        String line;
        while (((line = in.readLine()) != null) && env.shouldContinue()) {
            out.write(line.toUpperCase());
            out.newLine();
            linesProcessed(1);
        }
    }

}
        

You just need to implement 4 methods:

  • getProcessorName(): Give the processor name to be displayed in the drop-down list used for processor choice.
  • getProcessorDescription(): Optional method giving a short description of the processor. This description is shown in the generated GUI of the processor.
  • getProposedNameSuffix(): Give a filename suffix used to propose a default filename for the processed file. Here, the proposed name for a input file named "myFile.txt" will be "myFile_up.txt". The tool interface let you choose another output file name if you wish, this suffix is just there to make a default proposal.
  • process(...): It's the file processing itself. The readers/writers are setup by the parent class, so this method can focus on the real processing. There are only two additions to the bare processing: they are there to be able to interact with the encompassing tool.

Interacting with the tool:

To have an attracting GUI, it's important that the tool is still responsive during the file processing. It means that the tool should continuously display its progression and that it reacts correctly to 'abort' requests and errors events. This the purpose of the extra code found in the process(...) method.

  • shouldContinue() is a method of the FileProcessorEnvironment. This class allows interaction with the tool. The shouldContinue() method returns always true unless the user press the 'Abort' button or an exception occurred somewhere. The return value of this method must be checked inside your processing. Otherwise, your processor will ignore the 'Abort' command and behave badly in case of error.
  • linesProcessed(int nbrLinesProcessed): Inform the superclass that some lines are processed. The piper GUI will ask on a regular basis the status of each file processor (to display it in the status bar of the processor). With this information, the superclass will generate a meaningful status message like "375 lines processed from 13 files...". It's important to notify the status as much as possible to avoid that the GUI looks "frozen".

Note: As a aim of this tool is to be totally stream-based (to be able to chain processors and to process very big files), we don't try to calculate what is the total amount of processing to do. So, by design choice, the tool cannot display a progress bar or some percentage of completeness.

You can see that the Piper framework doesn't require a lot of plumbing or glue code. The ExampleProcessor class is not more complex that the bare code doing the job for one file.

Registering the new processor

To register custom file processors, the Piper framework uses java SPI (Service Provider Interface). SPI is a feature of the "JAR File Specification" that is very simple but quite powerful.

To notify the framework that you have a file processor available just use SPI to notify that you provide an implementation of the org.filePiper.model.FileProcessor interface: Create a jar containing your class (name it as you want). Then, just add a file named META_INF/services/org.filePiper.model.FileProcessor with the full name of your class as content.

Finally, put your jar in the "lib" directory of the Piper and that's it. When you launch the Piper, your new processor is available in the processor list.

As adding a custom processor dosn't require to modify any of the existing Piper configuration (you just add a jar in the correct directory), it's very easy to automate that task.

The FilePiper-ExtensionExample directory, provided with the sources, contains all the examples in a Maven2 project. See running the examples for explanations on how to build and deploy the examples.

Next step

We describe here the simplest processor integration. In next example we will describe how to define parameters that will show in an automatically generated GUI and persist the user input in configuration file.