4.4. Échange de données INTERLIS

This represents a guide on how to import/export data between TWW and INTERLIS 2 Transfer Format (XTF).

4.4.1. Général

The TWW plugin includes an INTERLIS import/export feature. It is currently capable of importing and exporting the following models:

  • SIA405_ABWASSER_2020_1_LV95

  • DSS_2020_1_LV95

  • VSA_KEK_2020_1_LV95

Note that currently, exports are possible in German only. Translated exports are on the roadmap, let us know if you are interested in this feature.

4.4.2. Prérequis

4.4.2.1. Java

Vous avez besoin de Java sur votre système.

Windows : Java for windows download MacOS : Java for MacOS download Linux : use the package manager

4.4.2.2. Bibliothèques Python

Vous avez besoin de sqlalchemy et geoalchemy2.

To install : before launching QGIS, open OSGeo4W Shell

../../_images/osgeo4wshell.jpg

and type:

>>> pip3 install "sqlalchemy~=2.0.38" "geoalchemy2>=0.14.7"

If the command fails, try:

>>> pip3 install --upgrade pip

Note

If pip cannot be found, make sure you’ve installed the python3-pip package using the OSGeo4W network installer (if you’ve installed the standalone version of QGIS, it should be included).

If this command still fails, try:

>>> pip3 install --target=c:\somewhere\other\than\the\default --upgrade pip

where c:\somewhere\other\than\the\default is the path to your current pip package

The same –target flag can be added to install sqlalchemy and geoalchemy2 in another location.

4.4.2.3. TWW Version

The export only supports up-to-date TWW data model (2020.1 at the time of writing). Ensure your data model is fully updated before trying to import/export.

4.4.3. Utilisation (GUI)

4.4.3.1. Activer le mode admin

In the plugin>TWW>settings dialog, under the Developer options tab, ensure Admin mode is enabled. Restart QGIS.

../../_images/settings_dialog.png

You should now see new import and export buttons in the TWW toolbar.

../../_images/toolbar.png

4.4.3.2. INTERLIS Export

../../_images/tww_interlis_export_button.png

To export your TWW data, click on the INTERLIS export button.

The following dialog will appear.

../../_images/tww_interlis_export.png

At first, select the export model.

If you have an active selection in the nodes and/or reaches layer, you can choose to restrict the export to that selection. This is especially useful in combination with the upstream/downstream selection tools.

../../_images/tww_interlis_export_selection.png

The export tools is capable of exporting label positions for different scales. You can choose which scales you are interested in exporting by selected/deselecting them.

Ensuite, confirmez le dialogue et choisissez où enregistrer le fichier .xtf.

Note

Note that windows file pathes with empty strings in the directory path or filename are not supported at the moment.

Note

Notez que pour des exportations importantes/complets, l’exécution peut prendre beaucoup de temps.

Note

Note that QGIS standard behaviour for too long labels is to turn them horizontally (orientation = 0). You can adjust the properties of the vw_tww_reach layer - Symbology - Placement - Label overrun from « no overrun » to e.g. 7 mm (Millimeter)

../../_images/layer_vw_tww_reach_label_overrun.png

Note

Note that remark fields are truncated to 80 characters on INTERLIS Export, as the INTERLIS definition is like this. If you have remark fields with more text then consider to move this data to documentation with the classes « file » and « data_media » so it can be exported to INTERLIS completely where you can add any document, photo or video to a class.

The following validations are run as pre-process: * Validate subclasses before export: check_subclass_counts

Note

Currently you will only find an ERROR message in the tww procotol tab if this check fails: 2024-07-26T10:29:03 teksi_wastewater.utils:ERROR: number of subclass elements of structure_part NOT CORRECT in schema tww_od: checksum = 1 (positive number means missing entries, negative means too many subclass entries)

Exports include a validation step at the end using ilivalidator, which will inform you whether the export contains INTERLIS validation errors.

If the export was successful a green success message appears. If the export is not successfull you will get either error messages during the export process or an error warning at the end of the process.

You can check the protocol window and the text logs for further details.

There are four different log files:

Heading Export step

Heading Description

Heading Log File

Export schema

Creation of export schema with ili2db and the selected INTERLIS model file

*.ili2pg-schemaimport.log

Data export intermediatary schema

Export of data from tww_od to the new intermediary schema, translation from English to German

*.tww2ili-export.log

Data export to xtf

Data export from intermediary schema to INTERLIS Transfer file (xtf)

*.ili2pg-export-MODELNAME.log

INTERLIS validation

Validation of the created xtf file(s) with ilivalidator

*.ilivalidator-MODELNAME.log

4.4.3.3. INTERLIS Import

../../_images/tww_interlis_import_button.png

To import xtf`files, click on the `INTERLIS import button and navigate to the .xtf file.

Note

Note that windows file pathes with empty strings in the directory path or filename are not supported at the moment.

Organisations before data !

Note

Starting with release 2020 all organisations are in a separated dataset and need to be imported first, else a bunch of errors will be thrown on all references like fk_dataowner, fk_owner, fk_provider, etc. Download the VSA Organisation data set from https://www.vsa.ch/models/organisation/vsa_organisationen_2020_1.xtf and start with importing that data set. This data set is updated regularly - so please come back and check (and maybe re-import) at a later stage to have access to all available organisation.

If you have organisations that are not yet in that data set please inform the VSA and hand in an application to be added.

If you have additional own local organisations that are not (yet) in the VSA organisation data set continue with importing those before you start importing your network data. TV inspection data usually comes last, as it references your network data.

After launching the import process your data set will be validated and imported in a intermediatary schema. Then the following dialog will appear.

../../_images/import_dialog.png

La partie gauche de ce dialogue liste tous les éléments qui vont être importés du fichier .xtf, permettant de revoir ce qui va être importé et de désélectionner les éléments que vous souhaitez ignorer. Elle montre également le statut de validation de chaque objet, indiquant si une action supplémentaire est nécessaire (INVALID) ou recommandée (WARNING) avant l’importation.

The right part of this dialog shows a form specific to the type of element selected in the list, allowing to adapt the import.

Note

Currently de-selecting and selecting objects might take a long time depending how many data sets are in that respective class.

Note

Special feature for TV Inspection import: For instance, it allows to attach « examinations » to their pipes.

Une fois que vous êtes satisfait des options d’importation, confirmez la boîte de dialogue afin de conserver les changements dans votre base de données.

If the import was successful a green success message appears.

4.4.4. Utilisation (ligne de commande)

For advanced usage, the import/export tool can also be used as a command line tool.

4.4.4.1. Dependencies

The QGIS python classes must be available in the python interpreter.

  • On windows you can use the OSGeo4W environment

  • On linux you can extend the PYTHONPATH pointing to you QGIS installation, for example: export PYTHONPATH= »$PYTHONPATH:/usr/share/qgis/python »

4.4.4.2. Import

Import example

$ python3 plugin/tww_cmd.py interlis_import --xtf_file plugin/teksi_wastewater/tests/data/minimal-dataset-SIA405-ABWASSER.xtf --pgservice pg_tww

Full usage

$ python3 plugin/tww_cmd.py interlis_export --help

usage: tww_cmd.py interlis_export [-h] --xtf_file XTF_FILE [--selection SELECTION] [--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}] [--logs_next_to_file]
                                               [--label_scale_pipeline_registry_1_1000] [--label_scale_network_plan_1_500] [--label_scale_overviewmap_1_10000] [--label_scale_overviewmap_1_5000]
                                               [--label_scale_overviewmap_1_2000] [--selected_ids SELECTED_IDS] [--pgservice PGSERVICE] [--pghost PGHOST] [--pgport PGPORT] [--pgdatabase PGDATABASE]
                                               [--pguser PGUSER] [--pgpass PGPASS]

options:
  -h, --help            show this help message and exit
  --xtf_file XTF_FILE   XTF outup file
  --selection SELECTION
                        if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}
                        Model to export (default: DSS_2020_1_LV95)
  --logs_next_to_file   Put log files next to XTF output file
  --label_scale_pipeline_registry_1_1000
                        Export labels in scale 1:1'000, can be combined with other scales (Leitungskataster/Cadastre des conduites souterraines)
  --label_scale_network_plan_1_500
                        Export labels in scale 1:500, can be combined with other scales (Werkplan/Plan de reseau)
  --label_scale_overviewmap_1_10000
                        Export labels in scale 1:10'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_5000
                        Export labels in scale 1:5'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_2000
                        Export labels in scale 1:2'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --selected_ids SELECTED_IDS
                        If provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --pgservice PGSERVICE
                        Postgres service name
  --pghost PGHOST       Postgres host
  --pgport PGPORT       Postgres port
  --pgdatabase PGDATABASE
                        Postgres database
  --pguser PGUSER       Postgres user
  --pgpass PGPASS       Postgres password

4.4.4.3. Export

Export example

python3 plugin/tww_cmd.py interlis_export --xtf_file "output.xtf" --pgservice pg_tww

Full usage

$ python3 plugin/tww_cmd.py interlis_export --help

usage: tww_cmd.py interlis_export [-h] --xtf_file XTF_FILE [--selection SELECTION] [--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}] [--logs_next_to_file]
                                               [--label_scale_pipeline_registry_1_1000] [--label_scale_network_plan_1_500] [--label_scale_overviewmap_1_10000] [--label_scale_overviewmap_1_5000]
                                               [--label_scale_overviewmap_1_2000] [--selected_ids SELECTED_IDS] [--pgservice PGSERVICE] [--pghost PGHOST] [--pgport PGPORT] [--pgdatabase PGDATABASE]
                                               [--pguser PGUSER] [--pgpass PGPASS]

options:
  -h, --help            show this help message and exit
  --xtf_file XTF_FILE   XTF outup file
  --selection SELECTION
                        if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}
                        Model to export (default: DSS_2020_1_LV95)
  --logs_next_to_file   Put log files next to XTF output file
  --label_scale_pipeline_registry_1_1000
                        Export labels in scale 1:1'000, can be combined with other scales (Leitungskataster/Cadastre des conduites souterraines)
  --label_scale_network_plan_1_500
                        Export labels in scale 1:500, can be combined with other scales (Werkplan/Plan de reseau)
  --label_scale_overviewmap_1_10000
                        Export labels in scale 1:10'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_5000
                        Export labels in scale 1:5'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --label_scale_overviewmap_1_2000
                        Export labels in scale 1:2'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
  --selected_ids SELECTED_IDS
                        If provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
  --pgservice PGSERVICE
                        Postgres service name
  --pghost PGHOST       Postgres host
  --pgport PGPORT       Postgres port
  --pgdatabase PGDATABASE
                        Postgres database
  --pguser PGUSER       Postgres user
  --pgpass PGPASS       Postgres password

4.4.5. Contrôle de la qualité

4.4.5.1. Contrôle de qualité avec le vérificateur VSA en ligne (Fachprüfung mit VSA Checker (online))

https://vsa.ch/wp-content/uploads/2020/04/Daten-checker-d-f-it.jpg

For details see explanation on the VSA Homepage

Information about access and licensing you can get here