4.4. INTERLIS Datenaustausch

This represents a guide on how to import/export data between TWW and INTERLIS 2 Transfer Format (XTF).

4.4.1. Allgemeines

The TWW plugin includes an INTERLIS import/export feature. It is currently capable of importing and exporting the following models:

  • SIA405_ABWASSER_2020_1_LV95

  • DSS_2020_1_LV95

  • VSA_KEK_2020_1_LV95

Note that currently, exports are possible in German only. Translated exports are on the roadmap, let us know if you are interested in this feature.

4.4.2. Voraussetzungen

4.4.2.1. Java

Sie benötigen Java auf ihrem System.

Windows : Java for windows download MacOS : Java for MacOS download Linux : use the package manager

4.4.2.2. Python Bibliotheken

Sie benötigen sqlalchemy und geoalchemy2.

To install : before launching QGIS, open OSGeo4W Shell

../../_images/osgeo4wshell.jpg

and type:

>>> pip3 install "sqlalchemy~=2.0.38" "geoalchemy2>=0.14.7"

If the command fails, try:

>>> pip3 install --upgrade pip

Bemerkung

If pip cannot be found, make sure you’ve installed the python3-pip package using the OSGeo4W network installer (if you’ve installed the standalone version of QGIS, it should be included).

If this command still fails, try:

>>> pip3 install --target=c:\somewhere\other\than\the\default --upgrade pip

where c:\somewhere\other\than\the\default is the path to your current pip package

The same –target flag can be added to install sqlalchemy and geoalchemy2 in another location.

4.4.2.3. TWW Version

The export only supports up-to-date TWW data model (2020.1 at the time of writing). Ensure your data model is fully updated before trying to import/export.

4.4.3. Benutzung (GUI)

4.4.3.1. Aktivieren Sie den Admin-Modus

In the plugin>TWW>settings dialog, under the Developer options tab, ensure Admin mode is enabled. Restart QGIS.

../../_images/settings_dialog.png

You should now see new import and export buttons in the TWW toolbar.

../../_images/toolbar.png

4.4.3.2. INTERLIS Export

../../_images/tww_interlis_export_button.png

To export your TWW data, click on the INTERLIS export button.

The following dialog will appear.

../../_images/tww_interlis_export.png

At first, select the export model.

If you have an active selection in the nodes and/or reaches layer, you can choose to restrict the export to that selection. This is especially useful in combination with the upstream/downstream selection tools.

../../_images/tww_interlis_export_selection.png

The export tools is capable of exporting label positions for different scales. You can choose which scales you are interested in exporting by selected/deselecting them.

Bestätigen Sie das Dialogfeld und wählen Sie einen Speicherort für die .xtf-Datei.

Bemerkung

Note that windows file pathes with empty strings in the directory path or filename are not supported at the moment.

Bemerkung

Beachten Sie, dass der Befehl bei grossen/vollständigen Exporten sehr lange dauern kann.

Bemerkung

Note that QGIS standard behaviour for too long labels is to turn them horizontally (orientation = 0). You can adjust the properties of the vw_tww_reach layer - Symbology - Placement - Label overrun from „no overrun“ to e.g. 7 mm (Millimeter)

../../_images/layer_vw_tww_reach_label_overrun.png

Bemerkung

Note that remark fields are truncated to 80 characters on INTERLIS Export, as the INTERLIS definition is like this. If you have remark fields with more text then consider to move this data to documentation with the classes „file“ and „data_media“ so it can be exported to INTERLIS completely where you can add any document, photo or video to a class.

The following validations are run as pre-process: * Validate subclasses before export: check_subclass_counts

Bemerkung

Currently you will only find an ERROR message in the tww procotol tab if this check fails: 2024-07-26T10:29:03 teksi_wastewater.utils:ERROR: number of subclass elements of structure_part NOT CORRECT in schema tww_od: checksum = 1 (positive number means missing entries, negative means too many subclass entries)

Exports include a validation step at the end using ilivalidator, which will inform you whether the export contains INTERLIS validation errors.

If the export was successful a green success message appears. If the export is not successfull you will get either error messages during the export process or an error warning at the end of the process.

You can check the protocol window and the text logs for further details.

There are four different log files:

Heading Export step

Heading Description

Heading Log File

Export schema

Creation of export schema with ili2db and the selected INTERLIS model file

*.ili2pg-schemaimport.log

Data export intermediatary schema

Export of data from tww_od to the new intermediary schema, translation from English to German

*.tww2ili-export.log

Data export to xtf

Data export from intermediary schema to INTERLIS Transfer file (xtf)

*.ili2pg-export-MODELNAME.log

INTERLIS validation

Validation of the created xtf file(s) with ilivalidator

*.ilivalidator-MODELNAME.log

4.4.3.3. INTERLIS Import

../../_images/tww_interlis_import_button.png

To import xtf`files, click on the `INTERLIS import button and navigate to the .xtf file.

Bemerkung

Note that windows file pathes with empty strings in the directory path or filename are not supported at the moment.

Organisations before data !

Bemerkung

Starting with release 2020 all organisations are in a separated dataset and need to be imported first, else a bunch of errors will be thrown on all references like fk_dataowner, fk_owner, fk_provider, etc. Download the VSA Organisation data set from https://www.vsa.ch/models/organisation/vsa_organisationen_2020_1.xtf and start with importing that data set. This data set is updated regularly - so please come back and check (and maybe re-import) at a later stage to have access to all available organisation.

If you have organisations that are not yet in that data set please inform the VSA and hand in an application to be added.

If you have additional own local organisations that are not (yet) in the VSA organisation data set continue with importing those before you start importing your network data. TV inspection data usually comes last, as it references your network data.

After launching the import process your data set will be validated and imported in a intermediatary schema. Then the following dialog will appear.

../../_images/import_dialog.png

Im linken Teil dieses Dialogfelds werden alle Elemente aufgelistet, die aus der .xtf-Datei importiert werden sollen, so dass Sie überprüfen können, was importiert werden soll, und Elemente abwählen können, die Sie eventuell überspringen möchten. Ausserdem wird der Validierungsstatus jedes Objekts angezeigt, der angibt, ob vor dem Import weitere Massnahmen erforderlich (UNGÜLTIG) oder empfohlen (WARNUNG) sind.

The right part of this dialog shows a form specific to the type of element selected in the list, allowing to adapt the import.

Bemerkung

Currently de-selecting and selecting objects might take a long time depending how many data sets are in that respective class.

Bemerkung

Special feature for TV Inspection import: For instance, it allows to attach „examinations“ to their pipes.

Wenn Sie mit den Importoptionen zufrieden sind, bestätigen Sie den Dialog, um die Änderungen in Ihrer Datenbank zu übernehmen.

If the import was successful a green success message appears.

4.4.4. Benutzung (Befehlszeile)

For advanced usage, the import/export tool can also be used as a command line tool.

4.4.4.1. Dependencies

The QGIS python classes must be available in the python interpreter.

  • On windows you can use the OSGeo4W environment

  • On linux you can extend the PYTHONPATH pointing to you QGIS installation, for example: export PYTHONPATH=“$PYTHONPATH:/usr/share/qgis/python“

4.4.4.2. Import

Import example

$ python3 plugin/tww_cmd.py interlis_import --xtf_file plugin/teksi_wastewater/tests/data/minimal-dataset-SIA405-ABWASSER.xtf --pgservice pg_tww

Full usage

$ python3 plugin/tww_cmd.py interlis_export --help

usage: tww_cmd.py interlis_export [-h] --xtf_file XTF_FILE [--selection SELECTION] [--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}] [--logs_next_to_file]
                                               [--label_scale_pipeline_registry_1_1000] [--label_scale_network_plan_1_500] [--label_scale_overviewmap_1_10000] [--label_scale_overviewmap_1_5000]
                                               [--label_scale_overviewmap_1_2000] [--selected_ids SELECTED_IDS] [--pgservice PGSERVICE] [--pghost PGHOST] [--pgport PGPORT] [--pgdatabase PGDATABASE]
                                               [--pguser PGUSER] [--pgpass PGPASS]

options:
  -h, --help            show this help message and exit
  --xtf_file XTF_FILE   XTF outup file
  --selection SELECTION
                        if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}
                        Model to export (default: DSS_2020_1_LV95)
  --logs_next_to_file   Put log files next to XTF output file
  --label_scale_pipeline_registry_1_1000
                        Export labels in scale 1:1'000, can be combined with other scales (Leitungskataster/Cadastre des conduites souterraines)
  --label_scale_network_plan_1_500
                        Export labels in scale 1:500, can be combined with other scales (Werkplan/Plan de reseau)
  --label_scale_overviewmap_1_10000
                        Export labels in scale 1:10'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_5000
                        Export labels in scale 1:5'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_2000
                        Export labels in scale 1:2'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --selected_ids SELECTED_IDS
                        If provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --pgservice PGSERVICE
                        Postgres service name
  --pghost PGHOST       Postgres host
  --pgport PGPORT       Postgres port
  --pgdatabase PGDATABASE
                        Postgres database
  --pguser PGUSER       Postgres user
  --pgpass PGPASS       Postgres password

4.4.4.3. Export

Export example

python3 plugin/tww_cmd.py interlis_export --xtf_file "output.xtf" --pgservice pg_tww

Full usage

$ python3 plugin/tww_cmd.py interlis_export --help

usage: tww_cmd.py interlis_export [-h] --xtf_file XTF_FILE [--selection SELECTION] [--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}] [--logs_next_to_file]
                                               [--label_scale_pipeline_registry_1_1000] [--label_scale_network_plan_1_500] [--label_scale_overviewmap_1_10000] [--label_scale_overviewmap_1_5000]
                                               [--label_scale_overviewmap_1_2000] [--selected_ids SELECTED_IDS] [--pgservice PGSERVICE] [--pghost PGHOST] [--pgport PGPORT] [--pgdatabase PGDATABASE]
                                               [--pguser PGUSER] [--pgpass PGPASS]

options:
  -h, --help            show this help message and exit
  --xtf_file XTF_FILE   XTF outup file
  --selection SELECTION
                        if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}
                        Model to export (default: DSS_2020_1_LV95)
  --logs_next_to_file   Put log files next to XTF output file
  --label_scale_pipeline_registry_1_1000
                        Export labels in scale 1:1'000, can be combined with other scales (Leitungskataster/Cadastre des conduites souterraines)
  --label_scale_network_plan_1_500
                        Export labels in scale 1:500, can be combined with other scales (Werkplan/Plan de reseau)
  --label_scale_overviewmap_1_10000
                        Export labels in scale 1:10'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_5000
                        Export labels in scale 1:5'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_2000
                        Export labels in scale 1:2'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --selected_ids SELECTED_IDS
                        If provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --pgservice PGSERVICE
                        Postgres service name
  --pghost PGHOST       Postgres host
  --pgport PGPORT       Postgres port
  --pgdatabase PGDATABASE
                        Postgres database
  --pguser PGUSER       Postgres user
  --pgpass PGPASS       Postgres password

4.4.5. Qualitätskontrolle

4.4.5.1. Qualitätskontrolle mit VSA Checker (Fachprüfung) (online)

https://vsa.ch/wp-content/uploads/2020/04/Daten-checker-d-f-it.jpg

For details see explanation on the VSA Homepage

Information about access and licensing you can get here