Adaptive, Component Based System Architecture for Monitoring Data Storing Distributed Systems Research Group Department of Computer Science AGH-UST Cracow, Poland Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG The need and the problems The need: More and more resources which must be monitored and monitoring data which must be stored. Current solutions for Grid: MDS, EDG NMA, Ganglia, Nagios, MapCenter, SNMP, JIMS The problems: –cannot exchange data and store in one common database (lack of common information model), or does not support monitoring data storing at all, –are not adaptable to: different monitoring environments, various resources types, possibility of dynamic resource attachment and dynamic attributes changes, different monitoring modes. –monitoring data storing processes is not scalable/efficient enough to store massive data.

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Wishes list - functionality Heterogeneity of monitored resources, Dynamic resource attachment, Runtime modification (extension, removal) of attributes, Support for compound attributes and multi-valued attributes, Different monitoring modes (push, pull, tracing), Universal interfaces for data upload and query, Monitoring domain independency, Make use of existing monitoring agents. A new system have to be created

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Can be meet, Let’s use component architecture! Wishes list – other features High availability, Load balancing, Automatic failover, Scalability, Efficiency, Nodes heterogeneity, Dynamic reconfiguration.

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Usher – general architecture Usher – uniform storage for heterogeneous environment monitoring Problems: common data model, agent, universal access (upload and query) interfaces.

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Common object data model META DATA DATA Type, Resource, Attribute: –Simple –Structured –Multi-valued Meta Data Values: - String - Double - Long Data

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Common object data model example

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Data upload conceptions 1. Values are being send together with all meta-information (eg. Ganglia) Big overhead for simple data 2. Separated mode - at the beginning meta information is configured - data are being send with proper attribute identifier only Low overhead, possible detection of meta data inconsistency at the agent side

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Data upload interface Operations sequence: Configure meta data: –getResource() If resource does not exist in the system create it with addResource() –getRootAttributesForResource() If resource exposes attributes from outside the list, add attribute with addRootAttribute() or extend structure attribute with extendAttribute() Upload monitored values –periodically call addMonitoringData()

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Usher agent Agent responsibility: interaction with a resource or an external monitoring agent meta data structure setup in USHER data conversion and periodical storing in USHER system resource’s attributes changes tracing and appropriate meta data structure modifications

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Data query conception

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Data query interface Operations sequence: 1.call getRootDTO() to obtain root object of the meta data tree, 2.mark nodes to be extended, 3.call getSubcomponents(), 4.repeate 2,3 until you will have desired simpleAttributes in leaves – metadata tree is completed, 5.call getData() to obtain values of the requested attributes according to specified queryClauses (time, number or results etc.).

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Component implementation Implementation technology details: –component technology: Java, J2EE, –application server: JBoss, BEA WebLogic (can be any), –data base: Oracle, PostgreSQL (can by any with JDBC support).

Adaptive, Component Based System Architecture for Monitoring Data Storing Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński DSRG Conclusions Proposed architecture is based on: –conception meta data for monitoring data, –universal access interfaces, –component technology for non functional requirements meeting. USHER advantages: –integrates data from different monitoring agents, –is adaptative to different monitoring environments, various resources types, possibility of dynamic resource attachment, dynamic attributes changes and different monitoring modes. Further work –non functional characteristic need to be experimentally verified (lots of work has been made in this area but it is not covered in this presentation), –usher agents have to be build for various monitoring systems (ongoing integration with JIMs and WBEM).

Thank you for your attention Questions please....