Content of PetroWiki is intended for personal use only and to supplement, not replace, engineering judgment. SPE disclaims any and all liability for your use of such content. More information
Drilling data management systems
Exploration and production (E&P) project data-management systems are database-management systems designed to support integrated suites of exploration and production applications.
- 1 Overview
- 2 Key features and functions of project data-management systems
- 2.1 Broad application support
- 2.2 Open extensible environment
- 2.3 Technology based on a standard database
- 2.4 Efficient handling of bulk data
- 2.5 Rich suite of project data-management tools
- 2.6 Support industrywide standards
- 2.7 Integrate user experience across applications
- 3 References
- 4 See also
- 5 Noteworthy papers in OnePetro
- 6 External links
Applications share data through a single common data store. Data are administered with a single set of tools for importing, exporting, viewing, editing, and performing database administration. By centralizing all data available for an E&P asset in a single integrated data store, project data-management systems greatly reduce the amount of time spent moving data between applications. Data-management systems serve as readily accessible repositories for the knowledge about an asset. This knowledge comes from a variety of studies and continues to grow over the life of the asset.
Before integrated project data-management systems, each fit-for-purpose E&P application had its own private, proprietary data store. Each individual data store supported different data-exchange formats and procedures. Moving data between applications involved exporting files, reformatting them manually, and importing them into the target application. This process was often so cumbersome that data were not exchanged at all, or they could be exchanged only by manually retyping the data into each application. Project data-management systems allow applications to share data without moving them from application to application. Outputs created by one application are automatically available in all applications connected to the project data-management system.
Key features and functions of project data-management systems
Broad application support
Project data-management systems should support a rich set of E&P applications that solve a broad range of technical problems. Key workflows should be completed entirely within a system, without the use of external tools for data manipulation.
Open extensible environment
Given the diversity of the oil and gas industry, no software vendor can offer a solution to all problems. Instead, project-management systems should provide an open-development environment, allowing niche application vendors to plug in “best of breed” applications.
Technology based on a standard database
Many requirements of an E&P project data-management system are similar to those of database systems in other industries. Systems based on common horizontal-market technologies allow the use of relatively cheap and powerful horizontal-market database tools.
The most mature database-management systems are “relational databases.” Relational databases have been used by many industries to store mission-critical data for more than 25 years. Researchers at IBM performed much of the early research on the relational model in the late 1960s and early 1970s. A relational model views data logically as a series of tables and columns, with a mathematical model for operations on these structures. The physical arrangement of data is hidden; instead, one depends only on a simple, logical view of tabular data. All data are reduced to the simple “flat” tabular form. Relational databases support SQL, which allows users to build queries that filter the rows of a single table. SQL queries can also combine data from one table with data from another on the basis of shared foreign key fields. This allows SQL statements to join data from multiple tables and build powerful ad hoc reports.
Relational database-management systems range from desktop databases to enterprise data-management systems. Robust database systems typically provide:
- Network access to data
- flexible and powerful data security
- Tools for “hot backups,” allowing a system to be backed up without shutting down
- Recovery tools in case of a system crash.
- A rich set of utilities that allow administrators to configure, control, and monitor the system.
Several very powerful tools have been developed for working with relational databases. These include systems that generate flexible ad hoc reports with rich format control and systems to build queries graphically without the use of SQL. Many tools that were originally designed for the horizontal market can be used when working with E&P data. These tools expose the data model of the underlying database. For a project database to be readily accessible by these generic tools, it must use relational technology and have a relatively simple and well-documented data model.
Object-oriented databases are a newer trend in database-management systems. They have the flexibility to store complex, structured data, and to associate software logic with that data. Object databases allow data to be stored in the form needed by today’s object-oriented applications. This can create a performance advantage, but the flexibility offered by object databases makes it difficult to write generic tools or evaluate arbitrary user-defined queries against object databases. Many large database vendors are moving toward a “hybrid database.” Hybrid databases contain most features of relational databases, but they extend the table/column view of a relational database to allow a column to contain rich, complex, user-defined data types. If used carefully, this allows developers the best of both worlds. Most data are modeled relationally for flexibility and ease of query. When performance becomes critical, however, certain columns are modeled with optimized object structures. This is particularly important for E&P data because this flexibility is important for managing certain E&P data types.
Efficient handling of bulk data
Many E&P data types lend themselves readily to representation as relational rows and columns. However, a few critically important data types in the petroleum industry have special performance concerns because of their size. Well logs, seismic surveys, and continuous sensor readings are examples of data types that can produce very large amounts of data. Large data items cannot be stored efficiently using the row/column abstraction of relational databases. These data are stored more efficiently in unstructured “blob” data types, either within the database or within operating-system files outside of the database. An E&P project-management system should blend these specialized data types seamlessly with more traditional relational data in integrated user presentations and displays. The location of data should be irrelevant to the user.
Rich suite of project data-management tools
Although many key data-management functions are provided by generic database-management systems, it is the responsibility of the E&P project-management system software to provide both data-type and domain-specific functionality to manage E&P technical data. Project-management systems provide rich data-management utility applications that allow:
- Flexible importing/exporting of data.
- Data browsing/querying/editing.
- Simple project database administration.
Project data-management tools should isolate end users from the complexity of working directly with the database when performing common data-management tasks.
Data import/export routines support a wide variety of common data-exchange formats. They should provide management for units of measure and unit conversion, if necessary. They also should convert surface locations between different map-projection systems. Other domain-specific functionality is provided when importing particular data types (e.g., the computation of wellbore paths from directional survey information).
Data browsing, querying, and editing tools should fill in the gaps left by horizontal-market query and browse tools, offering industry-specific displays for key data types such as well logs, production plots, and seismic displays. These tools should allow the updating and editing of project data and should enforce standard business rules and data integrity.
Project-administration tools should allow users with relatively little database knowledge to perform the following:
- Project database creation.
- Control of user access to a project database.
- Backup and restoration of a project database.
- Allocation of disk space and other database resources to a project database.
Support industrywide standards
Several industry consortia, including the Petrotechnical Open Software Corp. (POSC) and the Public Petroleum Data Model Assn. (PPDM) offer standard data models for many common E&P data types. As vendors move to support these standards, it should become easier to integrate data between project data-management systems from different vendors.
Integrate user experience across applications
Project data-management systems should provide a framework for applications to work together seamlessly using the same data. This requires more than sharing the same database. Applications should be notified when another application changes data in the shared database, allowing them to refresh their display to reflect changes made in other applications. In addition, a data selection made in one application should be available in another application. Consider selecting a well in map view to view in a utility that provides cross-sectional views of downhole well equipment. It is more efficient for a user to select a well in map view and send it to the utility than for a user to type in the name of the well in each utility.
This integrated functionality typically requires an interprocess communication scheme, allowing different applications in a user session to communicate. In addition, “session management” is needed so that users may select parameters that apply to all applications in a session (e.g., the active project or units of measure for display).
- Codd, E.F. 1983. A relational model of data for large shared data banks. Commun. ACM 26 (1): 64-69. http://dx.doi.org/10.1145/357980.358007.
Noteworthy papers in OnePetro
Wayne D. M. and John E. S. et al. 1978. INTEGRATED DATA MANAGEMENT AND CONTROL SYSTEM FOR OFFSHORE DRILLING RIGS - DESIGN AND OPERATIONAL EXPERIENCE, Offshore Technology Conference, 8-11 May. 3327-MS. http://dx.doi.org/10.4043/3327-MS.
A.R. Morley, 1991. Workstations for the Wellsite: An Integrated Information Management System, SPE Computer Applications Volume 3, Number 2. 20330-PA. http://dx.doi.org/10.2118/20330-PA.
Duhon, Howard. 2015 "Managing Project Complexity." Web Events. Society of Petroleum Engineers, https://webevents.spe.org/products/managing-project-complexity.