Product introduction
Huawei fusioninsinsight HD is a distributed data processing system, which provides large capacity data storage, query and analysis capabilities, and can meet the following requirements of major enterprises:
● quickly integrate and manage different types of high-capacity data
Provide advanced analysis of information in its native form
● visualize all available data for special analysis
Provide a development environment for building new analytical applications
● workload optimization and scheduling
Technical features
Huawei fusioninsight HD distribution follows the latest technology of the open source community, integrates the latest components quickly, and makes enterprise level enhancement in reliability, security, and management, and continues to keep the technology leading.
The enterprise level enhancement of fusioninsight HD is mainly manifested in the following aspects.
security
● architecture security
Fusioninsight HD is based on open source components to enhance functions and maintain 100% openness without using private architecture and components.
● certification security
The unified authentication system based on user and role follows the RBAC (role-based access control) model to realize the role-based privilege management and batch authorization management for users.
The security protocol Kerberos is supported. Fusioninsight HD uses LDAP as the account management system, and uses Kerberos to authenticate the account information.
Provide single sign on, unify the management and authentication of manager system users and component users.
● audit users who log in to fusioninsight manager.
File system layer encryption
Hive and HBase can encrypt tables and fields, and plaintext storage of user information in the cluster is prohibited.
Flexible encryption: the encryption algorithm is plug-in, which can be expanded or self-developed. Non sensitive data can not be encrypted without affecting performance (encryption has about 5% performance overhead).
Business transparency: the upper layer business only needs to specify sensitive data (hive table level, HBase column family level encryption), and the encryption and decryption process is completely insensitive to the business.
reliable
All management node components implement ha (high availability)
It is the first product in the industry to implement ha of all components to ensure the reliability and consistency of data. Namenode, hive server, hmaster, resources manager and other management nodes
HA was realized in all of them.
Cluster remote disaster recovery
The industry's first big data platform supporting remote disaster recovery of more than 1000 km provides the best practice of reliability so far for log detailed single class storage.
Data backup and recovery
Table level full backup, incremental backup, data recovery (check the integrity of locally stored business data, and self recover when data is damaged or lost).
Easy to use
● unified operation and maintenance management
As the operation and maintenance management system of fusioninsight HD, manager provides unified installation, alarm, monitoring and cluster management.
Easy integration
At present, it supports syslog interface, and the interface message can adapt to the existing system through configuration. The whole cluster adopts unified centralized management,
In the future, the northbound interface can be flexibly expanded according to the requirements.
Easy to develop
It provides automatic secondary development assistant and development samples to help software developers get started quickly.
Application scenarios
Typical application scenarios in the financial field are shown in the following figure:
Scene features
Facing the competitive pressure of Internet finance, financial enterprises need to reconstruct the decision-making and service system based on big data analysis and mining to improve their competitiveness and customer satisfaction. In the era of big data, banks will shift from transaction centered to data centric, in order to meet the challenges of more dimensional, larger and more real-time data and Internet business.
Huawei fusioninsight HD can solve the problems of financial enterprises from different aspects and enhance their competitiveness. For example:
Real time query of historical transaction details
Real time query of user's historical transaction details can improve the query range from 1 year to more than 7 years, and realize millisecond level query of 100 TB historical data table.
Real time credit reference service
The credit card time of users has been reduced from about 3 days to less than 10 minutes.
Forecast of small and micro loan business
The prediction accuracy rate of Top 1000 small and micro loan users is more than 40 times higher than that of the traditional mode.
Precision marketing
● greatly shorten the collection cycle of distributed online banking logs, provide accurate marketing based on user behavior statistics and analysis of online banking logs, and greatly improve the effect of online banking user experience.
● only less than 20% of the original recommended SMS can basically cover all the original effective purchase users and realize precision marketing.
