The client:

- Internet-based television service allowing remote computer viewing of local TV worldwide.

Challenge:

-Client wanted to build distributed cluster system for storage, data processing (e.g. transcoding).
-And analysis (search, media search) for large amount of media data.-initial data size is 200+TB of 1.6M media files.-key requirements are: commodity hardware, fast and
easy integration into existing system, high load capabilities, possibility to grow to petabytes of storage space, interfaces.
-For further integrations and data analysis development.


Client needed distributed tvguide/epg service which is responsible for the epg synchronization, import, update, and delivery of the epg data to the client apps.

Solution:

We proposed to build system based on:
-Sun Java SE-Apache Hadoop as distributed cluster-Apache
ZooKeeper for maintaining and distributed.
synchronization-Zabbix for monitoring-Apache httpd and
Apache benchmark for stress testing.

Result:

Client got:


-Distributed cluster system for media data storage built on commodity hardware in US and European data centers.
-Integration interfaces of NFSv3 and HTTP for fast integration into existing infrastructure.
-Each webHDFS node of the cluster can handle 61000 requests per second.
-Redundancy and write data protection with mirrored Name nodes, implemented tools to replace master name node with slave in case of master failed.

Back to home