Data Provenance, Milestone 1

Data provenance development in midPoint is moving forward. The implementation has reached its first milestone. For those interested in the details there are already few things to have a look at.

We started with analysing the use cases and preliminary list of metadata uses and items. This lead us to solution architecture and we have started designing the meta-data structure.

However, the greatest challenge was the schema modeling. This was quite expected. We are at the end of the road for XML Schema Definition (XSD). It is quite clear we have to make a big step forward. Therefore, we started working on a new schema language: Axiom. We started designing the language and implementing a prototype parser. The code is available in axiom module in feature/axiom branch of midPoint source code. The code is stil very experimental. Our plan is to validate that the language works by trying it on ourselves – we would like to express all midPoint schemas in Axiom evetually. Until we can pass through such validation both the Axiom language and the code is likely to change.

We have also made some progress on the user interface side and we are quite ahead of plan with that. We have extended Prism interfaces to introduce support for value metadata. We will use this to display data provenance meta-data in midPoint user interface. Implementation of user interface support is already in progress and we have first screenshots:

Data provenance M1 screenshot: simple metadata

Data provenance M1 screenshot: transformation metadata

There is still a lot to do. Axiom has to mature, we have to connect all the pieces by using midPoint mappings, improve the user interface and do a lot of smaller things. But we are on the right track and we are moving ahead. This milestone is more important than it may seem. This effort is part of midPrivacy initiative, which is long-term plan to implement data protection mechanisms in midPoint. And this is the first real milestone in midPrivacy implementation. Data protection is one step closer to become a reality.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the NGI_TRUST grant agreement no 825618.

Leave a Reply

Your email address will not be published. Required fields are marked *