Visual market + health score mechanism, volcano engine DataLeap lowers the threshold for resource optimization for enterprises

From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

For more technical exchanges and job opportunities, please follow the WeChat official account of ByteDance Data Platform and reply [1] to enter the official communication group

With the large amount of data maintained by the data warehouse and the R&D technical team, the resource usage is large, the cost is higher, and the optimization pressure is greater. How to proactively discover ineffective or inefficiently used resources and conduct proactive governance periodically and efficiently has become one of the core demands of team governance goals. Under traditional circumstances, resource optimization mostly relies on team experience and human consciousness control.

Currently, DataLeap, the Volcano Engine big data R&D governance suite, provides a combination of resource dashboards and health sub-monitoring. Targeting the health sub-panel, drill down to problem resources for proactive resource optimization, identify problematic asset libraries, and the individuals or teams that have the greatest impact on resource consumption. , proactively carry out expediting, achieve the goal of efficient use of service resources, and effectively reduce enterprise resource costs.

For the role of governance leader , in the cost governance link, the goal is to reduce storage and computing resources based on the overall situation of the assets, thereby reducing management costs. The Volcano Engine DataLeap resource optimization rule base can be flexibly enriched according to the current governance planning needs of the business, and the current overall governance strategy can be selected as needed; for the role of the governance interface person , custom rules can be delineated through the Volcano Engine DataLeap. If the data life cycle validity time is not set , the data table has not been queried for a long time, etc. In order to achieve the purpose of improving the overall health score of assets and saving storage.

At the same time, for teams that have not done governance for a long time or have governance needs, the Volcano Engine DataLeap data governance platform will provide automatic regular reminder capabilities; ultimately, through the governance of the market to check whether the effect of resource optimization is in line with expectations, the optimization rule experience and general rules will be precipitated , which can be directly referenced by other teams.

Key functions of Volcano Engine DataLeap resource cost management:

Volcano Engine DataLeap provides enterprises with unified tools and specifications to help enterprises carry out related resource management based on business status quo.

Storage governance: supports viewing the details of storage assets, and provides public rules and governance suggestions, which can quickly locate the main focus of governance
Computing governance: Can evaluate task resource usage details, queue resource rationality, and computing health points. The governance interface person can directly use the platform's public rules such as continuous task failures, empty output tables, no popularity in the output tables, unreasonable resource applications and other optimization items. At the same time, the platform proposes parameters and recommended optimization suggestions.
Diagnosis evaluation rate: The three diagnosis frequencies provided by the platform can be configured according to actual business needs. Specify the diagnostic frequency and end time of the scheme rules

Diagnose once: The diagnostic rules are executed immediately, and the assets in the governance domain will be scanned immediately and will not be run again in the future.

Daily diagnosis - latest results: The plan will run according to the rules every day before the end date, and the plan page displays the latest results of the daily run. Historical running results can be viewed in the historical version and are not accumulated with historical diagnostic results.

Daily diagnosis - cumulative results: For example, there has been no access in the past 30 days on January 1, and asset table A is scanned. After the scan, Table A was queried once. After the plan is run on time the next day, Table A will also be displayed in the list, achieving the effect of normalized monitoring and management. While optimizing resources, it greatly releases labor costs.

Volcano Engine DataLeap resource cost management advantages:

Low-threshold autonomy: full life cycle cost governance
Low-threshold optimization: calculation of governance benefits, storage calculation validity period management, and popularity monitoring
Self-service analysis and positioning optimization direction: analyze the composition of cost resources and quickly locate the optimization direction
Autonomous review: review of results, independent review of governance results, support down to table and personal review of governance benefits

Paying attention to cost optimization helps to coordinate data assets from a global perspective, which not only improves the collaboration efficiency in the governance process, but also enables the team to be self-driven and self-executed, thus abandoning complex evaluation and execution systems. In addition to the above governance capabilities, Volcano Engine DataLeap can also provide data integration, data development, operation and maintenance, assets and other capabilities to help users improve data research and development efficiency, reduce management costs, and accelerate the digital transformation of enterprises. It has been applied to pan-Internet, Manufacturing, new retail, automobile and other fields.

Click to jump to Big Data R&D Governance Suite DataLeap to learn more

Visual market + health score mechanism, volcano engine DataLeap lowers the threshold for resource optimization for enterprises

Supongo que te gusta