Big Data Architecture – Application Platform Service Categories
This table displays the service provision and service categories by the corresponding open-source technology. The technologies included in this Big Data Reference Architecture are:
- Secure OS – The secure operating system. Examples are a multi-level secure Linux or multi-level secure Solaris.
- Router – The multi-level secure router. This provides TCP packet extensions and routing based on security classification markings.
- Apache – The Apache Web Server, which provides HTML and other rendering services.
- HDFS – Hadoop File System is the persistence (storage) structure that allows Hadoop to distribute data and operate on it.
- HADOOP – Scalable Massive Parallel Processing architecture for distributed and scalable computing applications.
- Mahout – A machine learning platform implemented on Hadoop for classification and prediction of discrete and continuous data.
- openNLP – A natural language processing platform on Hadoop for unstructured text analysis (sentence marking, tokenizing, part-of-speech extraction, entity extraction, etc.)
- SOLR – Opensource Apache search platform built on Lucene.
- Lucene – Full-text indexer and search engine. Lucene will accept outputs from Mahout and openNLP models to aid searching results of analysis.
- Hive – Apache Hive is a data warehouse infrastructure that may be used for content storage, retrieval, indexing, and other core DBMS functions.
- HiveSQL – SQL-like Hive DDL/DML, non ANSI-92 compliant used to warehouse massive parallel datasets and operate upon them.
Data Interchange Services: |
Secure OS |
Router |
Apache |
HDFS |
HADOOP |
Mahout |
openNLP |
SOLR |
Lucene |
Hive |
HiveQL |
|
X |
|
|
|
|
|
|
X |
X |
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
X |
X |
|
X |
|
X |
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
X |
X |
X |
X |
|
X |
|
X |
|
|
X |
X |
|
X |
X |
X |
|
X |
|
X |
|
|
|
|
|
|
X |
X |
|
|
|
X |
|
|
X |
X |
X |
|
X |
X |
|
|
|
X |
|
|
X |
X |
X |
|
X |
X |
|
|
|
X |
|
|
X |
X |
X |
|
X |
X |
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
X |
|
|
X |
|
X |
|
X |
|
|
|
|
|
|
|
|
Data Management Services: |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
X |
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
X |
X |
|
X |
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
X |
X |
X |
X |
|
X |
|
X |
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
X |
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
|
X |
X |
X |
|
|
|
|
X |
X |
Graphics and Imaging Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
International Operation Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
Location and Directory Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
X |
X |
|
|
|
|
X |
X |
|
|
|
X |
X |
X |
X |
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
X |
X |
X |
X |
Network Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
X |
|
X |
X |
|
|
X |
X |
X |
|
|
X |
X |
|
X |
X |
|
|
X |
X |
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
Operating System Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
Software Engineering Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
Transaction Processing Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
User Interface Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
|
X |
|
X |
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
Security Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
System and Network Management Services: |
|
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
X |
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|
|
X |
|
|
|
|
|
|
|
|
|
|