Declarative logic

ShapeLogic is trying to develop a framework for declarative programming that works well for the computer vision and image processing domain. This has been growing along a letter match example application. It has been a trail and error process that so far has produced these approaches:

3 different approaches to declarative programming in ShapeLogic

Here is a chronological listing of ShapeLogic's 3 different approaches declarative logic:

  1. Declarative goal driven logic engine. From ShapeLogic 0.2.
  2. Logic filter language. From ShapeLogic 0.8 The syntax and development of the logic language is better described in Logic language .
  3. Lazy streams. From ShapeLogic 0.9.

Result of different approaches so far

For the letter matching example, the lazy stream approach have been both simpler and more powerful than the goal driven logic engine.

The Artificial Intelligence choice tree is build into the logical structure of goal driven logic engine, so this approach might work well when reasoning under uncertainty.

The logic filter language is used with both lazy streams and goal driven approach.

ShapeLogic is a toolkit, all the 3 approaches are available with unit tests.

For now development is focused on the lazy stream approach.

Lazy streams

The idea is that you have definitions of a data stream, say:

  • Natural numbers,
  • Fibonacci numbres
  • Sequence of polygons in an image
  • Sequence of sequence of polygons in an image sequence

The definition does not cause anything to be calculated, but when the data is needed it will be calculated. So it is a generalization of a lazy calculation.

Idea behind lazy streams

Lazy streams have the following features

  • You can define a lazy stream based on other lazy streams
  • They works as a kind of UNIX pipes or Legos
  • They serves as your query construct, you can directly query them

Lazy streams in letter example

Stream used in letter match example:

  • Polygon stream, this is generated from either and SVG file or and image
  • Cleaned up polygon stream, this can be done in different ways
  • A lot of feature streams:
    • hole count
    • point count
    • number of end point in lower left corner
    • number of t junction in upper right corner
  • Each letter is an And stream build of a feature stream and a predicate, for letter A:
    • hole count == 1
    • point count > 5
    • number of end point in lower left corner == 1
    • number of t junction on the right side == 1
  • The Letter match stream is an XOr stream of the letter stream.

Technical details about lazy streams

The streams in ShapeLogic are different than the streams in Haskell, Oz, Scheme and Scala. These are all based on data structures similar to a LISP list.

Immutable stream in Java List

In ShapeLogic the streams are assumed to be immutable. The data is put in a normal Java List.

The advantage of Java List
  • Faster access to data
  • Works better with Java
Advantages of LISP list / disadvantage of Java List
  • In a LISP list sub lists are also lists
  • This enables elegant functional style recursive programming

Using streams for concurrent programming

Streams can also support parallel or concurrent programming, which is important with the CPU intensive operations in image processing and computer vision. Especially with the advent of cheap multi processor machines.

Example: Find polygons in a stack of images

You define a lazy data stream for this and set a stream property

randomAccess = true

This indicates that individual elements can be calculated independently. The factory creating the stream could create a parallel version of the stream and assign each operation its own thread.

Note that the result would be a stream of polygons for each image.

Ideas behind declarative goal driven logic engine

This is the first approach for declarative programming, currently not the focus of development.

Influences for goal driven logic engine

  • Prolog language
  • AI choice tree
  • Oz language

Important ideas

  • Tree of task and sub tasks
  • Hierarchical contexts that is binding objects to name
  • Object hypothesis called OH in the code. OH = any match result

Hierarchical contexts

E.g. the context makes it possible to have the string "Letter" bound to both "A" and "B" at the same time in difference context. When matching a letter it will have a context for A and one for B, eventually one will be chosen, and the other discarded. The context is set up a hierarchically, like in most programming languages, so that names bound in a higher context can be seen in a lower one. So the string polygon is set at the root context level and can be seen by all the other context.

This relies on Java Expression Language (JEXL)

Tree of tasks

ShapeLogic is driven by a tree of task, the super class is called BaseTask.

A task can have sub tasks, but also have a context that inherits from the context of the parent task. Not all tasks have their own context.

Example of task hierarchy in letter matching

For examples of simple rules and task hierarchy look in the classes

  • LetterTaskFactory
  • LetterTaskTest

At the top level of the tree there is an Exclusive or task, XOrTask. Underneath is an AndTask for each letter. For each AndTask there are several simple property rules, of type ParametricRuleTask

Each AndTask has its own context that values can be set in, but all the SimpleTasks under an AndTask share a context. When a letter is finally chosen then the context from the chosen context is propagated up.

Example of logic rules in letter matching

The vectorizer will put a MultiLinePolygon object into the string "polygon" in a JEXL context.

Then there are rules that go in and look at the object using reflection. Here is an example of a simple NumericRule:

NumericRule("A", "pointCount", "polygon.getPoints().size()", 5.),

Here is an example of all the rules for a letter:

NumericRule[] numericRulesForA = {
    new NumericRule("A", POINT_COUNT, POINT_COUNT_EX, 5.),
    new NumericRule("A", LINE_COUNT, LINE_COUNT_EX, 5.),
    new NumericRule("A", HORIZONTAL_LINE_COUNT, HORIZONTAL_LINE_COUNT_EX, 1.),
    new NumericRule("A", VERTICAL_LINE_COUNT, VERTICAL_LINE_COUNT_EX, 0.),
    new NumericRule("A", END_POINT_COUNT, END_POINT_COUNT_EX, 2.),

where

POINT_COUNT = "pointCount";
POINT_COUNT_EX = "polygon.getPoints().size()"

All these rules are combined into an AndTask. A whole letter match will then be an XOrTask with an AndTask for each letter.

Logic filter language

The Logic filter language allows you to do Boolean combinations of filters with parameters

If you need to do more complex boolean combinations of filter on polygons try the following:

"polygon.filter('PointRightOfFilter(0.3) && PointAboveFilter(0.3)')"
"polygon.filter('PointOfTypeFilter(PointType.T_JUNCTION) && PointLeftOfFilter(0.5)').size()"

These will filter:

  • Points that are in in the right 30% and top 30% of the bounding box for the polygon.
  • Points that are in in the left 50% of the bounding box for the polygon and having the annotation T Junction.

Logical operators

Currently ShapeLogic is using the standard Java, C, C++ notation:

  And use &&
  Or use ||
  Not use !

Influences for the declarative goal driven logic engine

Here is a little more of about the influence for declarative goal driven logic engine.

Prolog

Up till ShapeLogic v 0.9 goal driven tasks with subtasks was a corner stone in the logic for image processing. Starting in ShapeLogic v 1.0 this is getting replace by a lazy stream based approach that is simpler and more powerful.

Prolog history

Prolog was all the rage in the early 1980s, the Japanese 5 generation project was mainly based on Prolog. It turned out that it did not scale well to real world problems, partly because it was only good at handling symbolic information and not doing computations, and Prolog fell out of favor.

Despite this, I still think programming in Prolog is almost as simple as SQL.

ShapeLogic is trying to preserve this simplicity of Prolog, while maintaining ability to do efficient numerical calculations.

ShapeLogic is not using backtracking or unification, 2 of the main components in Prolog, but shares the idea of programming as goals and tasks with sub goals or horn clauses

AI choice tree

Unlike Prolog, ShapeLogic can have multiple values for a name at the same time, and can explore different choices at the same time.

Oz

Oz is an experimental language boasting:

  • Declarative programming
  • Functional programming
  • Object-oriented programming
  • Constraint programming
  • Fast concurrent programming

ShapeLogic merges contexts when faced with different options in a similar way to Oz, though this is not fully implemented yet.

Ideas for future declarative programming constructs

IoC, Inversion of Control

There is an external definition that determine what type of objects should be created, and how they should be instantiated and create and instantiate all dependent object.

Once the system become more complex this will probably be essential.

The system is currently simple so now this would only add complexity, and is not used yet.