Nsi enterprises. Conducting nsi with practical examples. Unpreparedness of IT systems for integration of reference data and MD

The functioning of automated systems (AS) that are part of the corporate information system of an organization (enterprise, institution, integrated structure, government body) and ensuring the implementation of key business processes is impossible without the use of regulatory reference information (RNI). Accordingly, management efficiency modern organization largely depends on the quality of the reference data.

LM Soft company offers services for creating corporate system management of the reference data (KSU NSI) of the organization, which allows maintaining the quality of the regulatory and reference support of its AS at the required level.

A key element of LM Soft's methodological approach to reference data management is the organization, within the corporate information system, of a single Reference Data Fund - a set of interconnected reference books, classifiers, dictionaries and regulatory documents used by all AS. The management of reference data arrays included in the Fund is carried out centrally by a special service - the corporate reference data center.

The creation of the NSI CMS is a set of works to establish and automate three main groups of processes for managing the NSI Fund:

    processes for maintaining the Reference Data Fund (updating reference data arrays included in the Fund, and information services for reference data consumers – subscribers of the Fund);

    processes of expert and methodological support of reference data arrays;

    processes of organizational support of reference data arrays.

The work includes activities to form the main types of system support:

    organizational (preparation of organizational structure, organizational documents and regulations);

    informational (reconciliation, normalization, classification, coding, development of new master data arrays, etc.);

    methodological (development of necessary techniques);

    software

A generalized diagram of the organization of work is shown in the figure.

expand all sections

Prerequisites for creating a corporate reference data management system

Data processed in the automated system of any organization can be divided into two categories:

    operative information;

    regulatory and reference information.

Operational information – characteristics of specific operations performed by divisions of the organization. For example, on industrial enterprise it is formed in the AC class:

    CAD/CAM (Computer Aided Design - Computer-aided design/Computer Aided Manufacturing - Automated production preparation);

    PLM/PDM (Product Lifecycle Management - Management life cycle products/Product Data Management - Product data management);

    MES (Manufacturing Execution System - management system production processes);

    ERP (Enterprise Resource Planning - Enterprise Resource Planning);

    CRM (Customer Relationship Management), etc.

Thus, operational information supports the management of business processes of design, production, sales, procurement, financial management, etc.

Reference information is conditionally permanent information borrowed from regulatory documents and reference books and used in the operation of the AS. In foreign practice, the term Master Data is used in relation to reference data. In the AS database, regulatory and reference information is represented by arrays of reference data - sets of data generated on the basis of documents.

Operational information is generated on the basis of master data. Consequently, with low quality of reference data (errors in records, duplication, contradictions, incomplete description of objects), the efficiency of enterprise management is significantly reduced.

Typical reasons for low quality reference data are:

    lack of unified classification and coding systems for reference data arrays with information about homogeneous objects;

    lack of standards for describing objects represented by records of master data arrays;

    lack of regulations for maintaining master data arrays.

Poor quality of reference data leads to the following main problems:

    the complexity of integrating the enterprise's automated systems into a single information space;

    inability to promptly obtain consolidated reporting and monitor key indicators;

    the impossibility of fully organizing centralized processes (purchases, after-sales service, etc.);

    mismatch of information about products at various stages of its life cycle.

Emerging problems are solved by creating a corporate reference data management system.

Organization of the functioning of the NSI CSU

KSU NSI provides automated execution of the following processes:

1. Maintaining the NSI Fund arrays:

    updating the NSI Fund arrays;

    information services for subscribers of the NSI Foundation;

2. Expert and methodological support of the NSI Foundation:

    formation and normalization of NSI Fund arrays;

    quality control of reference data arrays;

    monitoring information resources and the needs for reference data of departments and AS of the organization;

    development of normative and methodological documents for the management and support of the NSI Fund.

3. Organizational support of the NSI Foundation:

    maintaining the register of the Fund of reference data (register of arrays of reference data);

    maintaining a register of reference data suppliers;

    maintaining the register of subscribers of the NSI Foundation;

    administration of users of the NSI Foundation.

The functioning scheme of the NSI Fund is given below:


Basic software

When creating a CSU, reference data are used software products class MDM (Master Data Management) - Master Data Management), implementing basic functionality:

    management of the contents of master data arrays (data entry/editing/export/import, support for data normalization processes and monitoring the integrity of related arrays);

    viewing and searching the contents of master data arrays (including displaying complex hierarchies and searching for duplicates);

    data quality control (creation and execution of complex scenarios for identifying and correcting errors various types);

    management of applications for adding/changing master data objects (including the implementation of complex approval processes);

    integration (synchronization) of data (including the creation and execution of data synchronization scenarios for any array of reference data and each target AS - consumer of the reference data);

    administration of users and system resources.

Both the most common foreign industrial software products of the MDM class (master data management) - IBM InfoSphere MDM, SAP NetWeaver MDM, Oracle MDM, as well as domestic specialized ones can be used as base software software package LM Soft MDM.

The main work on the creation of the NSI CSU

The development of the NSI CSU includes the following main works:

    determining a list of the main functional areas of the organization’s activities, within the framework of which the management of reference data is automated, defining the goals and objectives of the Reference Data Fund;

    formation of a register of automated enterprise systems, a register of local arrays of reference data in functional areas;

    formation of a register of the Reference Data Fund (register of centralized arrays of reference data);

    development of classification and coding systems for the NSI Foundation arrays, harmonized with industry, state and international directories and classifiers;

    development of standardized description formats;

    functional modeling processes of collecting and making changes to the datasets of the Reference Data Fund, information services for subscribers of the Reference Data Fund;

    construction of a generalized model for storing reference data and standard schemes for integrating AS in terms of reference data;

    development of hardware and software architecture for MDM solutions;

    setting up a basic MDM system and developing additional software;

    preparation of reference data arrays (classification, coding, reconciliation, normalization);

    development of organizational and regulatory documents for the management of the NSI Fund;

    development of methods for transferring master data arrays to centralized management;

    transfer of master data arrays to centralized management.

Advantages of the proposed approach to creating a CSU for reference data

The main advantages of LM Soft’s approach to the development of reference data management systems include the following:

1) Master data arrays are developed in accordance with generally accepted international practice approach to the description and codification of objects (cataloging).

Each group of homogeneous master data objects must have a standardized description template with a standard base name and a fixed set of values ​​for key characteristics. The composition of characteristics in the template allows you to uniquely identify an object and assign it identification code. Lists of group names, attributes, descriptions of elements and their values ​​are maintained in the format of dictionaries (directories) in accordance with regulations that ensure their timely updating, uniqueness of values ​​and compliance with regulatory requirements, which reduces the cost of maintaining master data arrays and integrating enterprise automated systems.

2) System design is carried out in accordance with the methodology of the GOST 34 complex (Automated systems).

The NSI CSU is divided into functional subsystems (FS) according to the company’s areas of activity (materials and technical support, fixed assets management, contract management, financial management, etc.) and supporting subsystems (software, Information Support, methodological support, organizational support and so on.).

Each FP is responsible for the centralized maintenance of a set of interconnected directories and classifiers for this area of ​​activity. All FPs use common resources provided by supporting subsystems (software, integration mechanisms, organizational structure and so on.).

As a result of the implementation of the system, a single automated centralized Master Data Fund begins to function in the company, supported by single unified types of collateral. This reduces the cost of maintaining it, adding new functional subsystems and developing existing ones.

3) When creating a control system for reference data, any industrial software system of the MDM class can be used. At the same time, the LM Soft company offers its own specialized software package.

The LM Soft MDM complex is implemented on the basis of the version of the 1C: Enterprise platform certified by FSTEC of Russia.

Use of domestic software system MDM solves the problem of import substitution in terms of software, significantly reduces the cost of the implemented master data management system (compared to the use of MDM systems from foreign manufacturers) and facilitates the task of creating secure master data management systems.

Effects of implementing the NSI CSU

The effects of implementing the NSI CCS can be divided into two main groups:

1. Economic (quantitative indicators characterizing the results in the areas of activity of the Group of Companies, in physical or monetary terms):

    reduction of costs for maintaining reference data;

    reducing the cost of supporting existing speakers;

    reducing the time for introducing new systems;

    reducing the cost of AS integration;

    increase in sales of final products and spare parts;

    reducing costs for the purchase of material and technical resources;

    decrease warehouse stocks and illiquid assets;

2. Managerial (qualitative indicators characterizing positive changes in the areas of activity of the Group of Companies, which are difficult to evaluate in physical or monetary terms):

    increasing the reliability of analytical data;

    reduction of time for preparing consolidated reports;

    increasing the number and degree of detail of analytical features;

    increasing the efficiency of design and production processes;

    increasing the efficiency of after-sales service processes;

    increasing the efficiency of managing receivables and payables;

In the context of the transition to a digital economy, companies have finally become convinced that data is an asset that is important to properly store, process, analyze, use for making decisions and making forecasts. The efficiency of these processes is ensured by a single repository into which verified quality data must be loaded. The task of consolidating them from different sources involves collating and synchronizing directories in various IT systems. This is why businesses need regulatory management systems background information(NSI).

According to TAdviser, the volume of the market for master data management systems is about 1.5 billion rubles at the end of 2017. The demand for these solutions is growing by 20-25% per year - in direct proportion to the growth of business digitalization. The acceleration of dynamics is facilitated by the growth of penetration cloud services in the domestic market (at a level of about 20% per year), as well as the launch of initiatives for informatization of the state and society as part of the implementation of the program.

The general path to digitalization dictates the need for a unified knowledge base about customers, products, etc. For digital initiatives to succeed, it is necessary to effectively manage data that first needs to be brought together to form a reliable and correct provision of a “single version of the truth” for all business units. Accordingly, there is a growing demand for data collation and consolidation tools that enable rapid access to information regardless of its source, analysis of patterns and anomalies, and secure distribution of data.

In Focus

Business representatives are becoming more and more demanding of the quality of reference data and its management processes. Questions about the quality of reference data also arise locally, among specialists. As the volume of data organizations accumulate increases dramatically, the performance requirements become higher. information systems. The volumes of reference books are constantly growing. Modern solutions in the field of reference data management are expected to be able to support work with more than 1 billion records.

If 10 years ago, at the end of the 2000s, the tasks of reference data were more often understood as the process of migrating directories as part of the implementation of accounting information systems, then in 2018, businesses approach the tasks of managing reference data more consciously and structurally, with the involvement of functional departments that directly use this information in business processes. Tasks related not only to equipment and materials, but also to contractors and other reference data are clarified.

The current situation requires more high level automation and formalization: everything that can be “hardwired” into a clear automated algorithm must be formalized, because Without strict rules, work with reference data turns into chaos. Also, the involvement of various business departments in NSI projects increases their duration. As a solution, modern means of automating data quality using mechanisms come to the fore,” comments Bair Danilov, head of the reference data department at IBS.

As of 2018, up to 75% of this market comes from consulting and about 25% is occupied by licenses. This situation is due to the fact that in addition to the direct creation of the directory, companies need its integration with other information systems, and for customer directories - with personal data protection systems, Krok notes.

New trends

Among the “hot” global technological trends changing the reference data market, IBS experts note the expansion of the scope of directory management, i.e. management not only of basic data - contractors and materials, but also of a unified chart of accounts, production assets and other necessary reference books for key business processes of the enterprise. Also in focus is the automation of the process of checking reference data, including using machine learning technologies, the development of uniform standards for maintaining counterparties and materials, as well as the creation of digital ecosystems in which manufacturers and buyers can freely exchange transparent information about goods and transactions.

Improvement remains the defining trend. Machine learning technologies allow for higher-quality deduplication in an automated manner. In general, the development noticeably changes previously established approaches to working with reference data - the efficiency of data recognition and correction increases, the ability to use multimedia information is added, make data more visual, etc.

Today, not only economic giants, but also medium-sized companies are showing interest in the quality of reference data. IBS notes an increase in the number of requests for scientific research projects from the pharmaceutical industry, Food Industry, mechanical engineering and Agriculture. This interest is also spurred by import substitution initiatives - it is the introduction of NSI that makes it possible to solve the problem of phased and rational import substitution of foreign solutions.

Top 8 players in the Russian market of reference data management systems

Krok IBS SDI Solution "NCIT "Intertech" TaskData Lanit EAE-Consult Navicon
Revenue from NSI projects 2016RUB 135 millionRUB 99.9 millionRUB 63.3 millionRUB 51.9 million44 million rub.28 million rub.RUB 11.2 million7.5 million rubles.
Revenue dynamics for NSI projects 2016/2015 13% 6% 32% 50% 90% 10% -10% Height
Number of NSI projects 2017 4 7 7 6 completed, 2 in progress 6 5 4
Number of NSI projects 2016 4 in progress, 1 completed 4 7 4 5 3 4
Solutions/platforms usedCroc NSI Suite, Talend Platform for MDM, MDM, Informatica MDM, a number of Oracle systems, as well as the domestic Unidata platformSAP, Ataccama, own development (20%), 1C MDMproprietary development of Semantic MDM. DBMS Microsoft SQL Server, Oracle, PostgreSQL* own development - software platform for managing reference data systems Ontologic (registered in the Register of Russian Software No. 4114 dated December 11, 2017);

2. Mining and metallurgical company - creation of an Automated system for managing regulatory and reference information, development of a materials and equipment classifier, normalization of the materials and equipment directory and the directory of contractors. More than 2000 users. Based on SAP MDM, SAP PI, SAP Portal, SAP BPM.

3. Federal executive authority - Consolidation and cleaning of received information, integration of the solution into the corporate IS. Based on Informatica MDM, Informatica Power Center, Informatica Data Quality, Oracle BPM.

1. United Engine Construction Corporation - “Creation and implementation of a Corporate system for managing normative and reference information on the platform of the Semantic Reference Information Management System”

2. Development of an automated system for managing regulatory and reference information of JSC Kalashnikov Concern on the platform of the Semantic reference data management system.

3. “Development of an automated system “Management of electronic directories of an enterprise” for the needs of PJSC RSC Energia.

1. Design, implementation and commissioning industrial operation corporate management system for scientific research data of the Inter RAO Group;

2. Creation of a Unified System of Regulatory and Reference Information of the State Oil Company of the Azerbaijan Republic (SOCAR);

3. Creation unified system management of regulatory and reference information in the Company CJSC "ABI Product";

4. Implementation of the master data management system (extended reference books) of PJSC MMC Norilsk Nickel;

5. Normalization of the Unified Directory of Materials and Equipment and mapping in the records of the Unified Nomenclature Directory as part of the project to introduce a unified concept for managing corporate master data of PJSC Polyus;

6. Creation of a methodological and regulatory framework for regulatory and reference information regarding the directory of material and technical resources and normalization of the directory of materials and equipment of Irkutsk Oil Company LLC;

7. Creation of a Unified system for managing regulatory and reference information of basic data of the Power Machines group of companies.

1. Industry center for the development and implementation of information systems (OCRS). The functionality of the first stage of ASOUP-3, which includes an Automated complex for maintaining reference data, has been developed.

2. Federal agency forestry (Rosleskhoz). Creating a control subsystem normative and reference information (PNSI).

3. United Instrument-Building Corporation (UPK). A project to build a model of a reference data management system as part of the implementation of the “Network Integrated Settlement and Information Management System” (SIRIUS) project - a centralized procurement management system for the defense industry.

1. Development of an automated system for maintaining reference data in one of the largest banks in Russia (on the Microsoft platform using the NORMA reference data management system, Oracle database).

2. Development of a master data management system for Gazprombank (on the Microsoft platform using the NORMA master data management system, Microsoft DBMS SQL Server).

KSSS turn 8 - translation of directories to the IBM MDM platform, interfaces to the SAP PI bus, quality control of reference data;

Integration of KSSS with 1C DO - integration of the Counterparties directory with 1C systems in DO;

KSSS-NSI RREM - translation and creation of RREM directories on the IBM MDM platform, interfaces to the SAP PI bus. Oracle DBMS is used for the lower storage layer and for the master data mart.

1. Food Union (consolidation of reporting from several branches and production facilities, the ability to accept management decisions based on a constantly updated data set, implementation in the Microsoft Azure cloud environment).

2. Gazprom Gazenergoset (automation of loading aggregated data from the accounting systems of subsidiaries and dependent companies (SDC) into the corporate data warehouse (CDW) in the central office).

3. Specialized depository “Infinitum Specialized Depository” (optimization business processes in terms of maintaining regulatory and reference information, optimizing the architecture by creating a centralized master data repository, eliminating duplicates and double data entry)

The largest projects by number of directories in 2015-2017 1. Project 1. The volume of reference books is more than 30.

2. Project 2. Volume of reference books: about 20.

3. Project 3. The volume of reference books is more than 200.

1. Project of JSC "UEC". The volume of reference books is more than 20.

Integration of KSSS with 1C for SIP - 31 organizations of the LUKOIL Group;

KSSS-NSI RREM - PJSC LUKOIL and 4 NGDO

1. Specialized depository "Infinitum" (about 40,000).

While working on large-scale automation projects and creating new information systems, we were each time faced with the need to implement a subsystem for maintaining directories, classifiers, registers and other similar objects that make up the customer’s reference information (RNI). Over the 15 years of working at LANIT with data management systems, life has given us clients with a wide variety of requirements. And, of course, different situations arose on these projects. I will tell you about several instructive stories that happened to us. In the article you will find examples that will be useful to many who are involved in software development. Well, for those who work directly with the NSI, it will be even more interesting - their own shirt is closer to the body.

Special thanks to the wonderful artist Vasya Lozhkin for the illustrations.

Case one. How to load a wagon and small cart

Creation of a unified counterparty management system for a large production company with many factories throughout the country and abroad.

Objective of the project– create a unified database of counterparties for all divisions. Counterparty management is carried out on the basis of requests, which are assigned priorities from low to urgent. An urgent application must be processed by NSI experts within 2 hours, regardless of the time difference between departments.

Living history
The project was agreed upon with all interested parties (the customer's management convinced us of this) and developed within the given time frame in accordance with the approved requirements.

The presentation of the created counterparty management system went smoothly until one prominent woman - the head of the Siberian branch - stood up and very energetically, using Russian idiomatic expressions, brought to the attention of those gathered that when a railway car came to her for loading finished products, she would not wait 2 hours until someone in Moscow reviews the application to add a buyer.

She is not going to pay for the downtime of the car while the application is being approved, but will enter the buyer’s data into the system as is and ship the goods, and the Moscow comrades can then deal with the information about the buyer as much as they want.

This statement was supported by several more heads of the company's branches, which almost completely destroyed the centralized methodology of conducting single directory counterparties based on orders.

As a result, the project was modified in such a way that all branches had access to the counterparty database and could make changes to it directly, but at the same time automatic search similar records that were displayed to the branch employee, and he made a decision on the need to adjust the data, which was later checked by an expert group.

What we remembered: do not trust the words of managers and responsible persons on the customer’s side that all decisions have been agreed upon, everything is on topic and there are no objections. Identify all project stakeholders and try to find out system requirements and constraints directly from them.

Case two. We use it as we want

Creation centralized system customer management for an insurance company with a large number of branches and agents throughout the country.

Objective of the project– creation of a consolidated client base for use in analytical applications. The database was collected from all branches, the data was verified, supplemented, and duplicate objects were eliminated. The number of clients in one branch ranges from a thousand to several million. At the same time, there is practically no overlap in clients between branches.

Living history

Once a consolidated customer database was created, it had to be periodically compared with branch databases to identify differences, then process them and upload changes to the consolidated database. The growth of the client base between reconciliations amounted to several thousand records.

To perform the reconciliation, a special module was created, the architecture of which was designed based on the fact that it should quickly compare a large number of records and generate a relatively small XML file with changes for download. The XML format was chosen by the customer.

After implementing the system, we received a message from the customer that the reconciliation module works extremely slowly and generates a huge file for loading into the consolidated database, which they cannot open in any way.

What did it turn out to be? The customer carried out the initial loading of data from branches into the consolidated directory. The experts found this work tedious and time-consuming, and they simply took the reconciliation module and fed it with the complete data of the new branch, which had never been loaded into the consolidated directory.

The reconciliation module, which, in accordance with the technical specifications, was supposed to generate information about differences in the number of several thousand records, received two million records as input, and all of them were missing from the consolidated directory.

As a result, after several hours of superhuman effort, the reconciliation module nevertheless generated a file for downloading, which included all the branch data. And, yes, this file was huge.

The reconciliation module was not used by the customer for its intended purpose, but the customer liked the very fact that reconciliation allows for initial data loading, and he was going to continue working in this way, only he asked to significantly speed up the work of the module and do something with the created file so that it could be opened in a text editor.


In response to our objections that the reconciliation module is not intended for initial data loading, the customer happily showed the technical specifications and asked, where is it written here? We use it as we want!
As a result, we had to make changes to the architecture of the reconciliation module in order to process large amounts of data and generate an output file in CSV format, since the customer absolutely did not want to give up such a convenient tool.

What we remembered: Always include a description of the limitations in the specification - what your system should not do. Well, or create solutions that take into account all possible use cases, which is much more expensive.

Case three. Not a baby elephant, but an elephant, and it has to fly too

Creation of a centralized system for maintaining master data for a financial organization.

Objective of the project- creation of a centralized system for maintaining directories and classifiers with distribution of changes to interested systems and databases. Providing access to external systems to directories through the web services of our system.

Typically, customers have an average number of entries per directory from several hundred to several thousand. Our recent record holder is a directory that had 11 million entries. But this customer gave us a surprise. His directory contained over 100 million entries. We downloaded it for more than a day, because... During initial loading, many data checks were performed. This would not have been a big problem, but the customer demanded that the directory download in a few minutes.

As a result, we had to greatly change the way the system works with this reference book. In fact, it is maintained outside the system, and we only provide an interface for its use. We are currently developing new ways for our system to work with very large directories. We hope the customer will like it.

What we remembered: In the modern world, there is more and more data, and its growth rate is constantly increasing. The system must be ready for high loads even where they were not initially expected. We are constantly developing our solution taking into account modern trends data growth and increased requirements for the speed of their processing.

Case four. Difficult trick with files

Creation of a centralized system for maintaining master data in a large bank.

Objective of the project– creation of a centralized system for maintaining directories and classifiers with distribution of changes to interested systems and databases. A special feature of the project is the very complex processes of propagating changes that affect many systems.

Since in the future I will have to mention our own solution to manage the NSI, I will allow myself a small lyrical digression.

Read more about the NORMA system.

The tasks of our customers are largely similar, and we decided to reduce software development costs and reduce project time by creating our own universal platform for maintaining master data and master data (Reference Data Management & Master Data Management). The system has existed for more than 10 years, and all these years we at LANIT have been actively developing it.

NORMA supports centralized and distributed reference data management. All data and meta-information are maintained taking into account the history of changes, and the system allows you to view and change the entire array of reference data for an arbitrary date in the past or future. Processes for coordination and approval of changes can be configured for directories. The system includes a dedicated change distribution server, which allows you to interact with external systems through various interfaces and create fairly complex integration business processes (a sort of mini BizTalk Server). We have data export/import packages that can upload/load directory data into databases and files of various formats. Maintaining conversion tables for external systems is supported.

NORMA includes a graphical query builder and report designer. In addition to working with its own directories, the system allows, through its interface, to view and change directories that are located in databases external to it, as well as use these directories in the query builder and export/import packages.

In response to the occurrence of various events in the system, for example, events of changes to the directory, plug-in software components written in C# can be launched, which can both check data and interact with external systems and, in fact, the NORMA system itself. Almost all system functions are available through web services.

The system can be scaled both vertically by increasing the power of the application server and database, and horizontally by using a multi-node application server, in which each node or group of nodes is responsible for performing a separate function. To store reference data, the system can use Microsoft SQL Server, Oracle or PostgreSQL.


Typically, when creating references and change propagation processes, the customer consults with our analysts about which tool or set of tools provided by the system is best to use for a particular task. This time the customer said that he would create directories and processes independently.

After some time, one of the customer’s specialists contacted us with a complaint that his data was not being loaded into the system. As confirmation, we were sent a data import package, a source file with the records being loaded, and an error message stating that the data being loaded was of the wrong type.

Let's start to figure it out. We twist the package this way and that, try it different variants presentation of the original data, but we cannot repeat the mistake. We contact the customer with questions: maybe the import package has connected software components, maybe some additional restrictions are imposed on the directory, maybe the data is not from this process? We get the answer to everything - there is nothing like that, everything should load easily and worked before.


It turns out that this import package was just the tip of the iceberg. Briefly and greatly simplified, the following happened. The import procedure loaded the correct data from source file to the directory. The original file was deleted. Our system then propagated the changes to multiple databases, one of which compared its own data with our changes and generated a discrepancy file that was returned to our system for download. Moreover, to download this file, the customer used the same import procedure as for the source file. And this particular file, generated by the external system, contained data of the wrong type. Obviously, when analyzing the original file, we could not find any errors, and we were not told anything about the second file and the sprawling process of distributing changes.

What we remembered: Always check the information you receive, even if they tell you that we have a little problem here, and it’s in this very place, I swear to my mother! Analyze the problem in context.

Case five. I'm getting used to the inconsistencies

Creation of a master data management system in a manufacturing company.

Objective of the project– creation of a system for maintaining reference data in management company with many branches, factories and design departments.

This time we did not progress beyond a few presentations. The techies really liked our NORMA system. She covered all their existing problems. Then it was the turn to show the system to management, and here the bummer of the decade happened. High leader looked, listened and said: “We all work here on Apple products, they have a certain style, but your system does not fit into this style. We won’t even consider it.”


What we remembered: Customers are different, and for some you are simply not suitable. The style is different.

Similar stories happen in various projects. What was interesting in your project life? What was an unexpected lesson for you? Share in the comments.

Tags: Add tags

Understanding the need to create a single source of reliable (reference) reference data, the correctness of which will be recognized by all consumers of reference data within the company and organizations of the Transneft (OST) system, the company’s management initiated a project to create an automated reference data system system as one of the priorities within the IT strategy implementation program .

Centralized directories are the foundation on which the entire building of the Circulation Complex is erected. Therefore, the Automated Reference Information Management System (ASU NSI) was one of the first systems implemented as part of the IT strategy implementation program, says the project manager, chief technologist department information technologies PJSC Transneft Sergei Strogov. - The NSI automated control system was developed over two years and was put into permanent operation in January 2015. More than 150 company specialists and contractors took part in creating the system.

The purpose of the ACS NSI is to provide all automated information systems of the company and OST with reference normative and reference information. ACS NSI directories form the basis not only of the information systems of the Circulation Complex, but also of the systems currently operated in the OST.

As a result of the creation of this system, corporate regulations for maintaining basic directories, methods for normalizing directories were developed, the initial filling of directories was completed, and modern software tools for maintaining and updating directories were created.

The functions of maintaining reference data up to date are assigned to the department for maintaining reference data of Transneft Finance LLC. Users of the ACS for reference data from OST and companies, faced with a lack of information in the directory, form a request to add new or change existing records in the ACS for reference information. Experts from the reference data management department check data in requests for compliance with normalization methods and other documents regulating the quality of normative and reference information. As a result, a new entry is created in the directory or the user receives clarifying questions and recommendations for using current entries.

The directory must meet several parameters - the data must be current, complete and consistent, says Head of the Research Data Management Department Dmitry Morozov.- In order to achieve this, within the framework of the ACS of NSI, we solved the problems of developing and implementing an effective methodology for maintaining directories and classifiers. We built a system for keeping data up to date, ensuring its completeness, monitoring the integrity and consistency of data.

As already mentioned, master data changes based on user requests.

Suffice it to say that every day up to five hundred requests are received from all directories, and during the period of formation of REN procurement plans in OST, the number of requests from NSI users (responsible for REN planning, purchasers of material and technical support, those who work with operational accounting documents) doubles , - commented Dmitry Morozov.

The system provides flexible approaches to integration, and this reduces the overall costs of connecting new systems. ACS NSI uses standard methods of integration with the Integrated Automated System for Management of Production Assets (ACMS), the Corporate Information System for Automation of Financial and Economic Activities (CIS FHD), the System electronic document management(SED), Module for Preparation and Storage of Specifications (MPHS), CIS "Galaktika", CIS "Flagman", Financial and Economic Activities Management System "IKAR" and other systems.

The company has defined rules for the names and attributes of the elements of the ACS reference data information directories, says Sergei Strogoff. - Currently, 46 OSTs and more than 15 thousand users are connected to the NSI ACS system. ACS NSI is the main working tool for the department of organizing and conducting tenders, the planning department of REW, transport and supervision of construction control and other structural divisions.

ACS NSI: composition of reference books

Currently there are four main directories in the system: “Material and technical resources” (MTR), “Counterparties”, “Objects”, “Types of work/services”. The main volume of work fell on the first two reference books of the ACS NSI: “Materials and Materials” and “Counterparties”.

The basis of the MTP directory is the basic classifier, which includes more than two thousand lower-level classes. The basis was the classification adopted by the company. It was significantly detailed during the development of a new MTP classifier and normalization techniques for each class.

Block Implementation Approach methodological tasks was typical for this kind of project, notes project manager from the contractor Oleg Enin.- However, during the implementation of technical tasks, unique solutions were proposed and developed Russian market solutions based on maximum automation of developed techniques. First of all, a solution for automatically generating the names of material and transport records according to templates provided by normalization techniques. Users select the values ​​of the main characteristics of the MTP - the name is generated automatically.

This solution allows you to almost completely eliminate errors caused by the so-called human factor, and, as a result, simplify the processes of generating a request for a new record by the user of the ACS reference data and subsequent processing of the request by experts of the reference data management department.

In the case when it is necessary to change normalization methods, for example, when replacing normative document, removal of one or another hardware from production, adding new attributes to a record, a request of a separate type is generated. This request First, they are reviewed by experts from the reference data management department, then by curators from the relevant departments of Transneft PJSC.

Totality methodological support And technical solutions ACS for reference data can significantly reduce the processing time of requests in the reference data management department and provide users of business units with normalized normative and reference information in the shortest possible time.

At the moment, the volume of the MTR directory exceeds 1.2 million items, including about 1 million items of spare parts, - evaluates the work done Sergei Strogoff. - The directory contains not only a range of spare parts, but also keeps track of their prices. This information is available to all subsidiaries and allows for more efficient use of the company's financial resources.

For example, the group of materials and equipment "Automotive parts, tires and batteries" is used by organizations of the Transneft system in the process of forming lots for the purchase of services for maintenance and repair Vehicle, special and road construction equipment.

The Department of Vehicles and Special Equipment of the Department of REN, Transport and Construction Control Supervision of Transneft PJSC annually organizes work to update manufacturers of spare parts, the list of products, names and their costs, - gives an example Chief Specialist Department Alexey Kryuchkov. - In the period 2016-2017, the products of 170 manufacturing enterprises were updated in the ACS NSI directory. Based on the results of updating the nomenclature and cost in the ACS directory, NSI organizations of the Transneft system annually enter into contracts with consignment warehouse operators additional agreements for changes in the cost of spare parts.

The materials and equipment directory is integrated with 33 information systems that are operated in 32 OSTs.

The “Counterparties” directory contains a list of all legal entities, with whom employees of the company and OST work. The source data for the directory was information from the Master Data Management System of OOO Transneft Finance. Using the normalization technique, the rules for filling in the attributes of the counterparty record were determined. The directory now contains more than 90 thousand entries.

ACS NSI solves the storage problem up-to-date information and, most importantly, ensures consistency of its own directories with all-Russian and industry classifiers and directories, such as the KLADR address classifier, the BIK Central Bank of the Russian Federation directory of banks, classifiers of countries of the world, currencies, units of measurement, OKATO, OKVED, - adds Dmitry Morozov.

Thus, it turns out that the Transneft company, the state and industry enterprises speak the same language that is understandable to everyone.

The “Counterparties” directory is integrated with 15 information systems that are operated in 14 OSTs - with some, integration is organized with receiving a confirmation response about the acceptance of records, others, using the retrieval service, independently request information about counterparties from the NSI automated control system on a schedule.

The “Objects” directory is defined in the Perspective Program and is updated quarterly based on adjustments to the Program. It includes:

  • objects of the OST Capital Investment Plan under the Investment Program;
  • objects of the Capital Investment Plan under the Technical Re-equipment and Reconstruction Program;
  • Capital Improvement Program facilities;
  • objects of other capital investments;
  • objects of the Elimination Program.

The main purpose of creating the directory was to implement the functions of replicating reference data into OST information systems. Initially, data was downloaded from MS Excel files sent to the master data management department. Now we are setting up integration with the Corporate Information System for Program Management (CIS UP) as a supplier of reference data for further replication into information systems.

The “Types of Work/Services” directory is planned to be used in the Circulation Complex, in particular in the Automated Procurement Management System (APS).

This is a foundation for the future,” comments Sergei Strogoff. - It reflects the grouping of works and services. It is based on lists of works and services that are determined by the regulations of the Transneft company. In fact, this is a “cast” of our regulations.

ACS NSI: development

ACS NSI is a “living” and continuously improving system. Since it was put into permanent operation, more than 400 changes to the base classifier have been made, technical support specialists have implemented about 750 improvements, and 17 new information systems have been connected.

The company initiated a project to develop the functionality of the ACS for reference data and add new directories. Thus, within the framework of the project, it is planned to implement an automatic comparison of the positions of the “MTR” directory with elements of the register of main types of products (OVP), which contains a list of manufacturing enterprises that have passed inspections by the commission of Transneft PJSC, which will make it possible to more effectively use the automatic lotting functionality in ACS Procurement. A directory of tariff sections is added. The OST directory will be maintained more fully and in detail, including the structural divisions of subsidiaries. Integration with the developed Supplier Remote Service System (SDOP) is planned, which will allow external contractors, namely design institutes, to be connected to the NSI ACS.

It is planned to develop the materials and equipment directory in terms of its centralized application. The task to be solved in the near future is to create a unified database of unused materials and equipment. Data will be sent from all subsidiaries to the ACS Procurement system, which will help in selecting options for their use.

Due to various reasons, during production and economic activities in OST, materials and equipment remain unused, and not all of them can be used in production at the same enterprise, says Aleksey Ukraintsev, Deputy Head of the Department for Planning and Implementation of REW Plans, Transneft PJSC. - Such MTR are identified structural divisions OST are subject to inclusion in the consolidated list of unclaimed materials and equipment, and we are faced with the task of involving them in production. In order to unify this process, the relevant industry regulations were updated in August 2016. The OST control checks regularly carried out by the company are also aimed at ensuring that information about unused materials and equipment “rises” to the company level and is publicly available to all interested users. Currently, in order to involve their own unused materials and equipment, specialists from the logistics departments and OST procurement departments are doing a lot of work. However, given the growing volume of information, it is more expedient to search for options for using materials and equipment within the company using an information system.

Of course, the implementation of such a task is impossible without the effective exchange of information about unused materials and equipment and the use of a centralized directory and a unified encoding of material and equipment nomenclature. Therefore, the integration of OST accounting systems with the ACS of reference data is, among other things, the first-priority stage in creating a mechanism for the objective and quick selection of possible ways to involve unclaimed materials and equipment in OST.

The creation of an automated control system for reference data is one of the successful projects implemented in the company and formed long-term information support for existing and future information systems, - summarizes Deputy Director of the Information Technology Department of Transneft PJSC Dmitry Lebedev.- ACS of reference data is a basic element of the Circulation Complex, the tasks of which are to ensure the use of uniform, correct and up-to-date reference data. The NSI ACS describes and classifies the data necessary to carry out business processes and regulates the company’s activities.

Christina Gutovec

Federal State Educational Institution

Higher Professional Education

National Research Technological University "MISiS"

Department of Automated Control Systems

Coursework for the course

"Systems Theory and Systems Analysis"

Completed: Avdoshina Olga

Group: MA-10-1/I810-4

Teacher: Morozov E.A.

Moscow2014

1.Definition of normative and reference information 3

2. Problems and needs of companies regarding the master data management system. 3

3.Unified reference data management system (EU reference data) 5

4.Creation automated system reference data management 8

4.1. Analysis of master data 8

4.2.Choice of architecture and estimation of the cost of creating an automated control system for reference data 10

4.3.Implementation 15

5. Persons responsible for maintaining reference data 16

6.Efficiency of implementation 18

7. List of used literature 20

  1. Definition of normative and reference information

The operation of each automated system is based on regulatory reference information (RNI). Master data is a semi-permanent part of all corporate information that does not undergo significant changes in the daily activities of the organization. The master data includes: dictionaries, reference books and classifiers, the elements of which (for example, codes, names of materials, services, contractors, units of measurement, etc.) are used in the generation of current documents.

Reference data is used in automated systems when generating operational documents, planning and reporting. Accordingly, the quality of this planned, operational and reporting information directly depends on the quality of the master data. Management errors associated with poor-quality information sometimes cost businesses millions of dollars in losses.

  1. Problems and needs of companies regarding the nsi management system.

Companies, as a rule, use several automated systems that support various business processes, in which the same directories are maintained independently of each other. So completely typical situation causes the following problems:

Additional costs for independent maintenance of the same directories;

Additional costs associated with ensuring information interaction between systems using different directories of the same master data objects;

Great labor intensity and high cost of generating consolidated reporting based on data in which the same reference data objects (goods, services, contractors) have different codes and names;

Low quality of regulatory and reference data.

What does “poor quality” regulatory reference data mean? This is reference data that:

Have problems structuring materials and equipment into groups;

Duplicate or contradictory data from the directory of material and technical resources (goods and services) in 70% of cases lead to a significant increase in the enterprise's inventory and the formation of illiquid assets. For example:

The absence of the necessary parameters in the product description in the directory may lead to the purchase of a product that does not meet the required characteristics. As a result, illiquid assets are formed in warehouses;

The presence of duplicates in the directory will not allow you to correctly perform the automatic consolidation of all ordered materials and equipment of the same name in order to obtain a consolidated application. As a result, the order will be placed with the supplier in different batches and the company will not receive a discount for placing a large-volume order, and therefore the purchase will be completed at a higher price;

The use of different codes and names of materials and equipment by different departments does not allow analyzing the availability of materials and equipment in warehouses and the use of existing stocks, instead of purchasing new materials and equipment, which also leads to financial losses.

The low quality of reference data is a consequence of the lack of specialization in the management of reference data. The tasks of increasing business efficiency, the need to build a modern foundation for the development of the IT landscape of companies, the construction of new corporate ERP systems and the development of existing ones require increased efficiency in managing regulatory and reference data. The introduction of the Unified Reference Data Management System solves this problem.