Programming rules and neural networks

If the user wanted to customizing rules or neural networks for either the Color Particle Analyzer or for the letter matcher it demanded some programming until ShapeLogic 1.6. Now you can do most of this by loading a categorizer setup file. In most cases this should be a lot simpler, but occasionally you might sill need to do this. It is not too complicated but does requires knowledge of Java.

Defining rules for categories in ShapeLogic 1.5

This is done by sub classing ColorParticleAnalyzer or ColorParticleAnalyzerIJ. Override the categorizeStreams() method.

Use ColorParticleAnalyzerIJ if you work with ImageJ and ColorParticleAnalyzer if you work without.

Here is the current categorize code. It is only there to serve as an example:

The rule syntax is very straightforward. There are a lot of numeric streams with one lazy calculated number for each particle. E.g. aspect ratio stream. Then you define one rule for each constraint you want to put on a given category.

In order to belong to the category Flat the aspect ratio for a particle just has to greater than 1.1.

In order to belong to the category "Light round" the aspect ratio for a particle just has to between 0.9 and 1.1 and the brightness have to be greater than 150.

Note that a particle that satisfy more categories will not belong to any of them.

  • Default rules and neural network for categories in ShapeLogic 1.5
            @Override
            protected void categorizeStreams() {
                    loadParticleStreams.exampleMakeParticleStream();
            if (_useNeuralNetwork) {
                    defineNeuralNetwork();
            }
            else {
                loadLetterStreams.makeXOrStream(StreamNames.PARTICLES, LoadParticleStreams.EXAMPLE_PARTICLE_ARRAY);
                _categorizer = (ListStream<String>) QueryCalc.getInstance().get(StreamNames.PARTICLES, this);
            }
            }
    
            private void defineNeuralNetwork() {
                    String[] objectHypotheses = new String[] {"Tall", "Flat"};
                    String[] inputStreamName = {StreamNames.ASPECT};
                    double[][] weights = ExampleNeuralNetwork.makeSmallerThanGreaterThanNeuralNetwork(1.); 
                    _neuralNetworkStream = new FFNeuralNetworkStream(
                                    inputStreamName,objectHypotheses, weights,this);
             _categorizer = _neuralNetworkStream.getOutputStream();
            }
            
            
            final static public String[] EXAMPLE_PARTICLE_ARRAY = 
            {"Flat","Tall","Light round", "Dark round"};    
            
            /** This shows what to do to define rules for the color particle analyzer.<br />
             * 
             * This is not useful.<br /> 
             * Light and dark is turned around if inverted LUT is used.<br /> 
             */
            public static void exampleMakeParticleStream() {
                    LoadLetterStreams.rule("Flat", ASPECT_RATIO, ">", 1.1, null);
                    
                    LoadLetterStreams.rule("Tall", ASPECT_RATIO, "<", 0.9, null);
    
                    LoadLetterStreams.rule("Light round", StreamNames.COLOR_GRAY, ">", 150, null);
                    LoadLetterStreams.rule("Light round", ASPECT_RATIO, "<", 1.1, null);
                    LoadLetterStreams.rule("Light round", ASPECT_RATIO, ">", 0.9, null);
    
                    LoadLetterStreams.rule("Dark round", StreamNames.COLOR_GRAY, "<", 120, null);
                    LoadLetterStreams.rule("Dark round", ASPECT_RATIO, "<", 1.1, null);
                    LoadLetterStreams.rule("Dark round", ASPECT_RATIO, ">", 0.9, null);
            }

    Currently you have to override categorizeStreams() to customize neural network or rule set, this will be re factored so there are 2 protected methods that can be overridden:

  • defineNeuralNetwork()
  • defineRules()

How to override Color Particle Analyzer category rules

Here is a very simple example that overrides both the declarative rule set and the neural network:

Overriding rule set example

This example only has 2 categories Flat and Non flat. It is flat if it twice as wide as it is high.

Overriding neural network example

This example only has 2 categories Light and Dark.

Link to neural network and machine learning.

Copy the following text into a file called CustomParticleAnalyzer_.java:

import org.shapelogic.imageprocessing.ColorParticleAnalyzerIJ;
import org.shapelogic.logic.CommonLogicExpressions;
import org.shapelogic.machinelearning.ExampleNeuralNetwork;
import org.shapelogic.machinelearning.FFNeuralNetworkStream;
import org.shapelogic.streamlogic.LoadLetterStreams;
import org.shapelogic.streamlogic.StreamNames;
import org.shapelogic.streams.XOrListStream;

public class CustomParticleAnalyzer_ extends ColorParticleAnalyzerIJ {
        
    @Override
    protected void categorizeStreams() {
        loadParticleStreams.exampleMakeParticleStream();
        if (_useNeuralNetwork) {  
            //----------Override neural network----------
            String[] objectHypotheses = new String[] {"Dark", "Light"};
            String[] inputStreamName = {StreamNames.COLOR_GRAY};
            double[][] weights = ExampleNeuralNetwork.makeSmallerThanGreaterThanNeuralNetwork(128.); 
            _neuralNetworkStream = new FFNeuralNetworkStream(
                inputStreamName,objectHypotheses, weights,this);
             _categorizer = _neuralNetworkStream.getOutputStream();
        }
        else { //
            //----------Override declarative rule set----------
            LoadLetterStreams loadLetterStreams = new LoadLetterStreams(this);
            loadLetterStreams.rule("Flat", CommonLogicExpressions.ASPECT_RATIO, ">", 2, null);
            loadLetterStreams.rule("Non flat", CommonLogicExpressions.ASPECT_RATIO, "<", 2, null);
            loadLetterStreams.makeXOrStream(StreamNames.PARTICLES, new String[] {"Flat", "Non flat"});
            _categorizer = (XOrListStream) getContext().get(StreamNames.PARTICLES);
        }
    }
}

How user can change Letter Matcher to recognize numbers instead

DigitStreamVectorizer is an example showing what is needed to define a match of the numbers.

To run this from ImageJ in the shapelogic menu select

  • Digit Match Stream Vectorizer, to only do the vectorize and number match
  • DigitMatch, to also turn into binary and Skeletonize
public class DigitStreamVectorizer_ extends StreamVectorizer_ {

        @Override
        public void matchSetup(ImageProcessor ip) {
                loadDigitStream();
                
        }
        
        public static void loadDigitStream() {
                LoadPolygonStreams.loadStreamsRequiredForLetterMatch();
                makeDigitStream();
                String[] digits = {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"}; 
        LoadLetterStreams.makeLetterXOrStream(digits);
        }

        public static void makeDigitStream() {

                rule("0", HOLE_COUNT, "==", 1.);
                rule("0", T_JUNCTION_POINT_COUNT, "==", 0.);
                rule("0", END_POINT_COUNT, "==", 0.);
                rule("0", MULTI_LINE_COUNT, "==", 1.);
                rule("0", CURVE_ARCH_COUNT, ">", 0.);
                rule("0", HARD_CORNER_COUNT, "==", 0.);
                rule("0", SOFT_POINT_COUNT, ">", 0.);

                rule("1", HOLE_COUNT, "==", 0.);
                rule("1", T_JUNCTION_LEFT_POINT_COUNT, "==", 0.);
                rule("1", T_JUNCTION_RIGHT_POINT_COUNT, "==", 0.);
                rule("1", END_POINT_BOTTOM_POINT_COUNT, "==", 1.);
                rule("1", HORIZONTAL_LINE_COUNT, "==", 0.);
                rule("1", VERTICAL_LINE_COUNT, "==", 1.);
                rule("1", END_POINT_COUNT, "==", 2.);
                rule("1", MULTI_LINE_COUNT, "==", 0.);
                rule("1", SOFT_POINT_COUNT, "==", 0.);
                rule("1", ASPECT_RATIO, "<", 0.4);
                
        }
}

This file is not defined inside the shapelogic package system.

Users do not need to download the source file for ShapeLogic to use it, they only need shapelogic and other dependent jar files.

Compile from within ImageJ

  • Place the CustomParticleAnalyzer_.java into ImageJ/plugin/ShapeLogic directory.
  • Compile it from within ImageJ, by going to Plugin -> Compile and Run ...

Compile from outside ImageJ

  • Compile CustomParticleAnalyzer_.java someplace else using Eclipse, NetBeans or javac and take the resulting class file:

    CustomParticleAnalyzer_.class

  • Move it into ImageJ/plugin/ShapeLogic directory.