Ladies and gentlemen, please welcome midScale. MidScale is a project to significantly improve scalability of midPoint. The goal is to support midPoint deployments with millions or tens of millions of identities.
MidPoint is almost 10 years old. When we started midPoint project, the primary goal was a mid-size enterprise. We designed midPoint components to fit that evironment. Also, that was early 2010s. The world was a different place. We have to prioritize support for many databases. We need to write a generic code that would fit several databases, which means we were not able to fully utilize the power of any of them. We had to make compromises and trade-offs. And, of course, we also did some mistakes in early design.
Over the years midPoint grew, the deployments got bigger and more ambitious. Fortunately, we expected that this could happen. Therefore the repository layer in midPoint is replaceable. The plan for midScale is to replace the generic multi-database implementation with a code that will be written specifically for PostgreSQL. We also plan to redesign the structure of database tables to make them more scalable. This should give us a lot of boost in performance and scalability.
Database-related improvements are at the heart of midScale project. However, there is so much more to do to make the entire system scalable. We will need to review user interface code to improve performance. Prism, the data modeling foundation of midPoint, needs to be improved as well. Task management needs to be made much more “cluster native”. And there is a huge effort that we need to invest in testing and the infrastrucuture to make sure that we have reached our goals.
MidScale project offically started few days ago. MidScale is mid-term project that will span two midPoint releases: 4.3 and 4.4. Some of the improvements will be delivered in midPoint 4.3 release, but all the work will be finished in midPoint 4.4. MidPoint 4.4 is also meant to be a long-term support (LTS) release, therefore the plan seems to fit nicely. We will be publishing all the midScale material on project web pages for anyone to follow the progress of the project.
This is significant project that we have planned for several years. However, we were not able to secure sufficient funding up until now. Fortunately, we have been successful in NGI_TRUST open call, which provided partial funding for the project. We are very grateful to NGI and European Union for this opportunity. Even though most of the cost of this project will be covered by Evolveum, this project would not be possible without European community money. Creating and maintaining open source projects is a hard work, yet it provides value to a broad community. We are glad that European Union recognizes the value of open source software and contributes to its development.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the NGI_TRUST grant agreement no 825618.
Great! Hopefully midPoint will get optimized for users with many (e.g. 5K) assignments as well. And I wonder, if less-optimized deployment with Oracle or MSSQL DB option will stay. Unfortunatelly these DBs are still quite popular in commercial subjects.
The current DB support will remain, at least for near future (midPoint 4.4 LTS). What will be the future after that point is not yet entirely clear. However, we know that we have very little motivation to support MySQL/MariaDB and they will not be supported for long. The future of Oracle support is not clear yet. We think would like to support MS SQL, but the question is whether it is worth the cost. I guess everything depends on the experiences and reception of the PostgreSQL repository. The future will tell.
Not sure about the roles, though. That could heavily depend on role content and structure. It is not currently a priority, but the midScale improvements might help there.