acefael

.. on gis software

:ds_file_iterator

| Comments

The day comes when you want to backup an entire VMDS SOM or take a snapshot copy to test your upgrade. How would you do that? Shut everything down and use Explorer? Backup Manager? Iterate files on the filesystem and ds_transfer stuff out? What about Superfiles?

Most of these things are live and working somewhere. They are all fine in their place. One small thing that has been bugging me for a while is how to correctly copy out Superfiles, because you cannot reliably tell their size limit by looking at the filesystem alone.

The API

So it turns out that the wonderful Smallworld GIS has an API to tell you muchas cosas sobre los achivos.

It boils down to SOC → SOM → ds_file → Superfile components. The Superfile components ‘know’ their size limit.

With that it is possible to loop over known datasets and extract the information from there:

_for ds _over gis_program_manager.cached_datasets()
_loop
    _if ds.responds_to?( :files )
    _then
        _for f _over ds.files.elements()
        _loop
            _local (path_name,regular?,convertable?,readonly?,temporary?) <<
                f.superfile_query_file()

                ...

                _local component_info << _unset

                _if regular? _isnt _true
                _then
                    _local ( file_id, block_id, comp_total, comp_max ) <<
                        f.superfile_query_index()

                    ...

                    component_info << sw:vector.new( comp_total )

                    _for i _over range(1, comp_total)
                    _loop
                        _local ( pname , max_size , used_size ) <<
                            f.superfile_query_component( i-1 )
                        component_info[ i ] << { pname , max_size , used_size }
                    _endloop
                _endif

                # do *something* to the file

            _endif
        _endloop
    _endif
_endloop

The critical pieces are those .superfile_query_*() methods on ds_file. Notheworthy is the check for support of the .files protocol on the SOM. This serves to establish we are looking at a VMDS / SWDP SOM.

How To Use

In above code, where it reads “do *something* to the file”, the things available to make use of are f (a ds_file) and components (a simple_vector, or _unset). ds_file_iterator.magik contains the full implementation of the iterator method ds_file_iterator.each and a tried and tested way to use it.

ds_file_iterator.each takes as argument a _proc that is being called with exactly a ds_file and the components of that superfile, if any.

The sample method ds_file_iterator.backup_file shows how to apply .each. If you set $DATABASE_ROOT and $BACKUP_ROOT accordingly it can copy your files from c:\Smallworld\cambridge_db\ds\ds_gis\rwo.ds to c:\Smallworld\backup\ds\ds_gis\rwo.ds. It uses ds_transfer for that.

The rest is details. It’s in above mentioned file. Important is: you won’t ever have to worry about added Superfile components anymore.

How to Improve

ds_file_iterator.each can obviously only iterate VMDS SOMS. Anything else is simply skipped. It is blissfully unaware of files like replication_metadata.ds.

This could have been an _iterator _method.

The Long End

This has been tested on 4.11.

Be sure to let me know if you like it!

Comments