Разработка корпоративного программного обеспечения переживает трудные времена. Растущая сложность программного обеспечения уже сейчас приводит к серьезным сбоям. Значительные расходы необходимы для борьбы с ними. Существующие технологии могут лечить только симптомы, но не причину заболевания. Цель этой статьи — пролить свет на то, сможет ли язык навигации для приложений помочь выяснить причину заболевания и вылечить его.
Сначала попробуем выяснить, с какими проблемами сталкиваются компании и к чему они приводят.
Проблемы с программным обеспечением в финансовой индустрии
Согласно опубликованным BBC News статьям о сложности банковских систем — за последние месяцы банки и другие финансовые учреждения пострадали от различных сбоев программного обеспечения. Проблемы, сформулированные в статье, говорят о том, что такие ошибки в будущем будут более вероятны из-за возрастающей сложности программного обеспечения.
Котировки
· «Только в январе мы видели, как Банковская группа Lloyds использовала Систему более быстрых платежей, предназначенную для ускорения денежных переводов, из-за которой возникли задержки с выплатой заработной платы и оплатой счетов; сбой в мгновенных платежных уведомлениях PayPal приводит к тому, что некоторые пользователи сталкиваются с дублированием платежей; и американский биржевой оператор Bats Global Markets признают, что «системная проблема» означала, что более 400 000 сделок не смогли выполнить гарантию лучшей цены.
· “And these faults pale in comparison with last year’s bungled software upgrade which disrupted millions of customers’ accounts at NatWest, RBS and Ulster Bank; and the computer-controlled trading error at US market-maker Knight Capital that decimated its balance sheet.”
· «Modern computer systems are so complicated you would need to perform more tests than there are stars in the sky to be 100% sure there were no problems in the system,» he explains.
· «Business software is becoming increasingly complex, composed of sub-systems written in different programming languages, on different machines by disparate teams.
· «This means no single person, or even group of people, can ever fully understand the structure under the key business transactions in an enterprise. Testing alone is no longer a viable option to ensure dependable systems.»
And this is only a tip of the iceberg of all the problems existing in many large companies — the problems that are not escalated to the level of press. Problems are so serious that existing technologies are not able to address.
Legacy systems
Lots of companies use legacy systems to automate their operations. They are usually written in old programming languages and contain a huge amount of data. The development of the business systems require often modifications. Flexibility embedded in the system, determines the level of how the system can be modified during the operation. Systems are used to be designed with regards to the existing at that time realities, but in several years law and business climate can change dramatically. Limitations incorporated into the system may not allow further evolving the application and at some stage it may end up when the system is not able to adjust to the business needs but more over the business has to adjust to the system. Instead of investing resources into development and grows, automate business they are spent on the maintenance of outdated systems that of course prevents business development.
Financial Post says — SimCorp StrategyLab Operations Survey Shows Legacy Systems Hinder Investment Growth
The study found that buy-side firms running legacy systems need to spend more in maintenance costs versus state-of-the-art systems that can scale and adjust with growth.
Business processes are built up above Legacy systems
Large companies implement different projects to automate and optimize business processes (document tracking and other internal operations) and use various Business Process Management tools. According to the plan they are supposed to be using the resources of the existing systems for business processes automation.
But the reality looks a bit different – the implementation and maintenance of such systems does not meet the expectations and often very resource-consuming. To maintain and develop such systems requires highly-qualified specialists that should be familiar with the variety (if not all) of technologies used within the company.
Let’s see what articles say about that. In blog of Garnter analytics — Has BPM Lost Its Luster?
and also analytics of Forrester — Tackle The Most Common BPM Challenges open up a number of problems of effective BPM implementation and the return of investments.
It looks like a term BPM is used to master organization budget and carries a ghostly benefits from using it. It seems like BPM tries to cure the symptoms of the disease but not its root cause by trying to build up flexible processes over sluggish systems.
Integration of the systems turns into a lifelong project
Projects integration increases a probability of errors. Any changes in one system can lead to the errors in dependent from it systems. This requires constant retesting of all coupled systems. If the systems are big enough it may take ages to finish up the integration.
Computerworld UK spoke to Martin Fantozzi, senior architect at SITA, who explained how the four year project is ongoing and he believes that they won’t ever be in a position where the integrations are complete.
Applications instead of monolithic are offered to make modular
OSGI technology positions itself as a technology decreasing software complexity by using modular approach. It makes it possible to dynamically couple different modules with each other without system reboot. This brings a number of benefits during system operation, especially when there is a requirement to provide system operation 24/7 and be able to update it without a reboot.
But in June 2011, Rod Johnson, the founder of SpringSource and the creator of Spring Framework, said the OSGI technology is not so easy to use and it is not as productive as expected. He claimed that their clients quite rare requested such functionality and they believe it is possible to achieve the same results without using OSGI technology.
Products adaptation to different customers
As you know every client is unique, even within the same US state, the same EU country the requirements can vary significantly, sometimes even they can mutually exclude each other. It makes it not so easy to create a unified system, a system suiting all customers’ needs. That’s why it is very important to be able to build up unique applications for the particular customer. At the same time to be able to use a common set of modules to build up a final solution. What about the other markets?
Entrance to the new markets
Adaptation of ready systems to the requirements of the other countries can be accomplished by difficulties of cultural differences. It is not enough just to translate interface to the foreign language. It’s quite important to be able to re-model the system, create missing components and work out the solution corresponding to the cultural specifics.
Business processes within a single application — an impossible dream.
Usually applications are developed by people who are not too much familiar with the term of business process management. They don’t understand the importance of those things for business. That’s why systems are usually created as a set of Lego and you can collect it either with the help of instructions or if you don’t have instructions nearby try to guess how and what component should be connected with the rest of components. I believe that possibility to create and manage instructions (i.e. business processes) should be incorporated within the system from the very beginning. If it is not done in the early stages it will have to be implemented with the help of BPM above the existing systems inheriting all their benefits and shortcomings.
I suspect that all those problems will even get worse in the nearest future. They tell us that during creation, maintenance, integration and adaptation of the applications was missed something important, fundamental — a kind of architectural mistake.
In the pursuit of money applications are created only for sale. They look nice at the moment of the purchase and work as expected. But the world is constantly changing. More and more money will be spent on their maintenance and development in future. Probably, the problem is hidden in the fact that applications are usually created based on the static models when the world is dynamic.
A new navigation language could be able to solve these problems
Basic principles. Dynamic models
Behind the navigation language will lay the dynamic application model and will link the fine-grained, well-established components in a complete application. The model will be described on a platform independent language that could be an extension of the UML Statechart notation. Different modules with that model will be connected, written on different languages, they can be reused in future.
Also the navigation language could allow creating several models of the same application for different cases. For instance, for every role in the system a corresponding model will be created, for every type of the application: desktop, mobile or web models can have common parts as well as individual ones meeting the requirements of a particular type of application.
The applications with the new navigation language could be naturally deployed and work in the Cloud and affectively use its processing power.
The systems with the navigation language could be easily integrated with each other, as well as to set hyperlinks in HTML documents.
New applications
Since behind the system based on the navigation language the dynamic model will lay then new applications created based on it will have a number of benefits. The model described on that language will let architectures, analytics, programmers and QA engineers to use the same language to interact with each other.
· Architect may initially develop a prototype of the system and show it to customers at an early stage.
· Business analysts can describe business processes in the system, using the ready-made modules / components.
· Functional analysts can be focused on how the independent system components are supposed to operate.
· Developers will develop these components according to specifications, using TDD, unit testing.
· Architect will link the components into a single system using the new navigation language.
· Testers will create system tests based on the application model.
Support and Development of applications
Support of the systems based on the new navigation language will be a natural process and not be a burden. When changes are made, the testing will only affect adjacent modules but not the entire system. Changes in business processes of the organizations can quickly be reflected in the application due to the built-in support.
Extensible, integrated business processes at the system level
Creation of business processes within the application could optimize the work of employees and easily integrate them into global business processes of the company.
Iterative migration to the system based on the new navigation language
Legacy systems could be migrated to the systems based on the new navigation language iteratively, implementing functions step by step.
Scaling and distributed systems
The systems based on the new navigation language could be easily scaled, could work in the Cloud, and be easy to integrate with other systems based on it. A global business processes can be created on the same principles as within the systems.
Existing languages have exhausted themselves
According to the article in Parallelism and Moore’s Law
Currently to be able to use all that extra processing power that Moore’s law predicts it has become necessary to use parallel computing. For many years, processor manufacturers are constantly increased clock rate and instruction level parallelism so that old single-threaded applications started to run faster on the new processors without any changes in the code.
Now for various reasons processor manufacturers prefer multi-core architectures and that’s why the programs should be re-written in some extent to be able to get all the benefits from the increased CPU performance. However, for fundamental reasons, it is not always possible.
Support for parallel computing in popular languages is too low-level. It can be compared with the writing of contemporary programs on Assembler. To write really distributed applications you have to learn how to work with threads, with the fork / join, with the actors, semaphores, monitors, and other scary words. And this is just only for a parallel processing within a single system. If we have an ambition to develop a program that is supposed to be working within a distributed network then we have to also learn the technologies such as RMI, JMS, Sockets, CORBA, WebServices, BPM, ESB, etc.
Modern programming languages based on the procedural and functional languages are perfect choice for the systems when algorithm of work are known in advance or will be known. Also it will not be changed significantly over the time. These languages were originally designed to process predetermined commands sequentially. Multithreading and asynchronous processing have been added as a supplement.
I believe the solution of the problem of distributed applications lies in the sphere of system modeling that takes into account features of the distributed infrastructure. Also the interpretation of that model on a higher level than the level of proposed by the existing third generation programming languages such as C++, Pascal, Java, Scala, C#. They suggest too fine-grained approach for building distributed applications.
Perhaps the creation of the new navigation language will let us solve those problems. It will initially be based on dynamic event driven application model that allows changing the program flow based on external events.
Similar to HTML, the main markup language for creating web, the new navigation language will specify the structure and behavior of distributed applications. Unlike HTML the language will be interpreted not by the browser but on the server-side connecting servers into a single worldwide distributed system.
Lexaden Story
Company Lexaden has set a quite ambitious goal — to build up a product for generating and optimizing timetables for junior high/high schools and universities of different countries worldwide. Application design was divided into two relatively independent tasks. The first is to create an easily adjustable administrative part of the program. The second is to create and incorporate optimization algorithms into the system.
During the implementation of the first problem we had implemented the system letting us to build up a full-blown application most effectively combined from various specialized modules according to the needs of the particular organization.
To be able to simplify modules’ reuse every module has been made responsible for every particular action. For instance well-known CRUD was split up into Create, Read, Update, Delete modules. The same principle was applied to the other modules that were also specialized on well-defined actions such as Import, Export, Print, Analyze, Mass Edit, Mass Assign, Transform, Render, Optimize and so on.
Splitting up the system into specialized modules it was necessary to provide their interaction. For that purpose the special language Lexaden Web Flow had been created that allows modules communicating between each other by means of events.
Lexaden application is like a city
Let’s imagine the City with all the buildings, streets and avenues. Cars are driving on the streets and avenues, people are walking on the pavements. City is like a big application with its business processes. Rules of Lexaden Web Flow language can be illustrated by the hierarchy .
In the case of Lexaden Web Flow an application – is a roadmap of the City. It describes the location If the houses, roads, apartments, their entrances and so on. City map contains a number of different profiles – mini-maps created for the particular role. For instance role of mayor is possible to compare with profile of system administrator. Role of tourist is possible to illustrate with read-only profile when it allows only going sightseeing but not interfering into the city life.
<application id="application" ...> … <profile id="admin" ... /> <profile id="readonly" ... /> … </application>
Profile – is part of the city map, describing routes city inhabitant can move on to solve some problems. The mayor can travel around the city and influence the city life but the tourist is limited only by going sightseeing.
The same as with the application, the administrator can manage the system but an unregistered user can have only the limited access to different parts of the application. Profiles determine modules or buildings that user can visit. Also determine connections between buildings, in what order they could be visited to solve different problems.
Map of profiles can describe different directions from the particular building depending on the events taking place inside it. For instance during visit of the polyclinic user can be sent to home in case of the illness or to the work if everything is ok.
<profile id="burgess" ...>
…
<module id="policlinic" … >
<on event="sick" to="home"/>
<on event="healthy" to="work"/>
</module>
<module id="home" … >
</module>
<module id="work" … >
</module>
…
</profile>
Or let’s see how it would look like in a CRM application:
<profile id="customer" ...>
<module id="orders" extends="readonly" >
<on event="go_account" to="account"/>
<on event="go_addresses" to="addresses"/>
</module>
<module id="account" extends="crud"/>
<module id="addresses" extends="crud"/>
</profile>
Events on module level are like the cars transporting people between buildings. In case of the application user can move between different parts of it. Module determines the scheme of the building or the house showing the location of the corridors or rooms – in LWF they are represented by controllers. Every room is reserved for the particular actions.
The same is within the application where the particular actions can be executed in different windows or on different pages.
For instance the kitchen is for food preparation, hall is for having rest and discussions and bedroom is for sleeping. Events move the user from room to the room.
<module id="home" initial="wardrobe">
...
<controller id="wardrobe" … >
<on event="hungry" to="kitchen"/>
<on event="tired" to="bedroom"/>
<on event="ok" to="living_room"/>
</controller>
<controller id="kitchen" … >
…
</controller>
<controller id="living_room" … >
…
</controller>
<controller id="bedroom" … >
…
</controller>
...
</module>
Coming home the person first goes to the wardrobe to put off the clothes. After that when he is hungry – “hungry”, may come to the kitchen, when feels tired – “tired” – goes to the bedroom. When feels OK then goes to the “living room” – to watch TV.
Using the similar way let’s see how the module could be defined for Create, Read, Update, Delete operations:
<module id="crud" initial="list" ...>
<controller id="list" extends="controller">
<on event="create" to="create"/>
<on event="read" to="read"/>
<on event="update" to="update"/>
<on event="delete" to="delete"/>
</controller>
<controller id="create" extends="controller">
…
</controller>
<controller id="read" extends="controller">
…
</controller>
<controller id="update" extends="controller">
…
</controller>
<controller id="delete" extends="controller">
…
</controller>
</module>
In the Lexaden Web Flow the controllers are connected to the particular components of the application such as windows and pages. Those components are supposed to be solving the particular highly tailored problems, for instance creation, update or analysis, scheduling of some object(s) of the system. Components are activated to the user depending on the particular events taking place within the system.
I would like to draw your attention that Lexaden Web Flow determines only maps when the knowledge, rooms, cabinets (i.e. windows, panels or pages) are independent from each other. This allows using the same objects to solve different problems by different people within the same application infrastructure. Also allows creation of new components independently on the rest of the system. They are convenient to test separately and include into users profiles later on.
To get more information how it works in details you can read in the article: Building An Advanced Navigation for Enterprise Applications
Conclusion
A lot of problems have piled up in the sphere of enterprise software development. Existing technologies don’t address them effectively; support and systems integrations consume a round sum of money. Probability of serious errors using the existing mix of technologies is increasing every year. Perhaps it is a high time to take a pause and look around. To look at the problem from another angle and to put application development to a higher technological level that will open up new opportunities for business development and automation.