acefael

.. on gis software

How ACPs Work

| Comments

The Magik VM is a little older than current offerings. At the time it was built there probably were more urgent matters than third party software integration. But today you cannot live without.

At any time it is good practise to reuse computer software that is already written, and many libraries would be costly or impossible to implement in Magik. Unfortunately the Smallworld Magik VM does not have a FFI or Plugin API. That’s where ACPs come into play.

What it is

ACP stands for Alien-Co-Process, meaning “something that runs along and is strange”. Fundamentally it is a separate Process that the Magik VM talks to. Process here means a distinct running program in your operating system.

Smallworld GIS provides means to control such external processes and it also has a standardised way of talking to these processes. In fact I know two distinct established ways of talking to ACPs but one is deprecated.

The Interface

When the Magik VM starts an ACP it launches a separate process. You must tell it what to run and it really can be anything, but let us assume the ACP is another exe file. The way the Magik VM talks to that other process are the standard file descriptors that each process inherits from its parent: stdin, stdout and stderr. A basic ACP that adds two long integers would read two long integers from stdin and print one back on stdout. Communication is binary, that is, a long integer is 4 bytes of binary data.

All other details of the interface can be read from the Class Documentation of sw:user_acp. There are methods for reading and writing signed and unsigned integral types byte, int, long, short, for 4- and 8-byte floating point numbers, 8- and 16-bit characters as well as for arrays of all of these.

The Interface - Alien Side

The ACP interface from the ACP side is naturally the same as from the Magik side. What a sentence. No, really, on the ACP side you read and write just the same bits and bytes that are communicated from Magik. Just, your ACP may be a C or Perl or Tcl or Java Program.

For C based ACPs Smallworld has something for you. It is in $SMALLWORLD_GIS/data/acp_source/acp-io.h and acp-io.c. Therein are function definitions for reading and writing byte, short, int, long, float and so on. These match the methods in sw:user_acp.

The Code - Magik

An example ACP is provided by Smallworld in $SMALLWORLD_GIS/data/acp_source. It looks a lot like the following lines:

def_slotted_exemplar( :acefael_acp , {}, {:user_acp})
$
_method acefael_acp.add(a,b)
  _local result
  _protect
    _self.lock()
    _self.put_long(a)
    _self.put_long(b)
    _self.flush()
    result << _self.get_long()
  _protection
    _if result _is _unset _then _self.close() _endif
    _self.unlock()
  _endprotect
  >> result
_endmethod
$

Points of interest are:

  • in init the .command slot is initalised with the name of the executable.
  • init is called directly, so the exemplar is usable. One exemplar in memory has state about one child process. So the advice is to call the exemplar directly, except if you are sure you want to run many ACP subprocesses simultaneously.
  • The actual .add method uses .put_... and .get_... methods to talk to the subprocess and return a value. Calls to .lock(), .unlock(), and .close() as well as the _protect block are for thread synchronisation and subprocess handling. The code structure is from the Smallworld Core help and you should keep it unless you know what you are doing. An ACP is a single threaded executable and only one communication is possible on stdin/stdout at the same time. That’s why synchronisation is needed.

The Code - ACP

Now for the C side. C code is more verbose that Magik, so see the example for all the nitty gritty. Of relevance is the main loop in the main function of the program:

int main()
  ...
  while(1) {
    long a = acpGetLong();
    long b = acpGetLong();
    acpPutLong(a+b);
    acpFlush();
  }
}

Here you see the acp-io functions in action. In the loop two numbers are read and the sum written out again. The ACP does this indefinitely, or at least until the input stream is closed from the Magik session. Flushing the stream from both sides is important to avoid a hanging process when your bytes are being kept in a buffer and not actually written out.

The recommendation is to malloc all memory required in the loop, before the loop. That’s for performance reasons. But I never noticed any significant performance drawback malloc'ing each time around.

From the function definitions in acp-io.c you can get a good idea about what these functions actually do, and as I said it is just bits and bytes in the end.

Outlook

An ACP that adds two numbers is not so useful. The thing to do is to integrate software into your ACP, e.g. libtiff for reading elevation data or proj4 for doing grid shift transformations. That’s where the value is with ACPs.

Conclusion

ACPs are a primitive way of IPC that allows to do things with Magik that would not be possible otherwise. The overhead is not small, the communication is not trivial but once running they are typically fast and reliant.

AFIK ACPs are supported on Smallworld 5 - I have seen swxml_acp.exe running under my alchemy session.

Comments